Codec Smackdown

Modern digital acquisition, post and distribution wouldn’t be possible without data rate reduction, AKA compression. People like to disparage compression, but I dare say that few folks – including most post production professionals – have actually seen much uncompressed content. In fact, by the time you see a television program or a digitally-projected movie it has passed through at least three, four or more different compression algorithms – i.e. codecs.

Avid Media Composer and Apple Final Cut Pro dominate the editing landscape, so the most popular high-end HD codecs are the Avid DNxHD and Apple ProRes 422 codec families. Each offers several codecs at differing levels of compression, which are often used for broadcast mastering and delivery. Apple and Avid, along with most other NLE manufacturers, also natively support other camera codecs, such as those from Sony (XDCAM-HD, HD422, EX) and Panasonic (DVCPRO HD, AVC-Intra). Even these camera codecs are being used for intermediate post. I frequently use DVCPRO HD for FCP jobs and I recently received an edited segment as a QuickTime movie encoded with the Sony EX codec. It’s not a question of whether compression is good or bad, but rather, which codec gives you the best results.

Click on the above images to see an enlarged view. (Images from Olympus camera, prior to NLE roundtrip. Resized from original.)

I decided to test some of these codecs to see the results. I started with two stills taken with my Olympus C4000Z – a 4MP point-and-shoot digital camera. These images were originally captured in-camera as 2288-pixel-wide JPEGs in the best setting and then – for this test – converted to 1920×1080 TIFFs in Photoshop. My reason for doing this instead of using captured video, was to get the best starting point. Digital video cameras often exhibit sensor noise and the footage may not have been captured under optimum lighting conditions, which can tend to skew the results. The two images I chose are of the Donnington Grove Country Club and Hotel near Newbury, England – taken on a nice, sunny day. They had good dynamic range and the size reduction in Photoshop added the advantages of oversampling – thus, very clean video images.

I tested various codecs in both Avid Media Composer 4.0.5 and Apple Final Cut Pro 7. Step one was to import the images into each NLE. In Avid, the conversion occurs during the import stage, so I set my import levels to RGB (for computer files) and imported the stills numerous times in these codecs: 1:1 MXF (uncompressed), DNxHD145, DNxHD220, DNxHD220x, XDCAM-EX 35Mbps and XDCAM-HD422 50Mbps. In Final Cut Pro, the conversion occurs when files are placed on the timeline and rendered to the codec setting of that timeline. I imported the two stills and placed and rendered them onto timelines using these codecs: Apple 8-bit (uncompressed), ProRes LT, ProRes, ProRes HQ, DVCPRO HD and XDCAM-EX 35Mbps. These files were then exported again as uncompressed TIFFs for comparison in Photoshop. For Avid, this means exporting the files with RGB levels (for computer files) and for FCP, using the QuickTime Conversion – Still Image option (set to TIFF).

Note that in Final Cut Pro you have the option of controlling the import gamma settings of stills and animation files. Depending on the selection (source, 1.8, 2.20, 2.22) you choose, your video in and back out of Final Cut may or may not be identical to the original. In this case, choosing “source” gamma matched the Avid roundtrip, whereas using a gamma setting of 2.2 resulted in a darker image exported from FCP.

Click on the above images to see an enlarged view.

You’ll notice that in addition to various compressed codecs, I also used an uncompressed setting. The reason is that even “uncompressed” is a media codec. Furthermore, to be accurate, compression comparisons need to be done against the uncompressed video image, not the original computer still or graphic. There are always going to be some changes when a computer file is brought into the video domain, so you can’t fairly judge a compressed video file against the original photo. Had I been comparing video captured through a hardware card, then obviously I would only have uncompressed video files as my cleanest reference images.

I lined up the exported TIFFs as Photoshop layers and generated comparisons by setting the layer mode to “difference”. This generates a composite image based on any pixel value that is different between the two layers. These difference images were generated by matching a compressed layer against the corresponding Avid or FCP uncompressed video layer. In other words, I’m trying to show how much data is lost when you use a given compressed codec versus the uncompressed video image. Most compression methods disproportionately affect the image in the shadow areas. When you look at a histogram displaying these difference results, you only see levels in the darkest portion of an 8-bit scale. On a 0-255 range of levels, the histogram will be flat down to about 20 or 30 and then slope up quickly to a spike at close to 0.

This tells you that the largest difference is in the darkest areas. The maximum compression artifacts are visible in this range. The higher quality codecs (least compressed), exhibit a smaller histogram range that is closer to 0. The more highly-compressed codecs have a fatter range. This fact largely explains why – when you color grade highly compressed camera images – compression artifacts become quite visible if you raise black or gamma levels.

The resulting difference images were then adjusted to show artifacts clearly in these posted images. By adjusted, I mean changing the levels range by dropping the input white point from 255 to 40 and the output black point from 0 to 20. This is mainly for illustration and I want to reiterate that the normal composite images DO NOT look as bad as my adjusted images would imply. In fact, if you looked at the uncorrected images on a computer screen without benefit of a histogram display, you might think there was nothing there. I merely stretched the available dynamic range for demonstration purposes.

Of these various codecs, the Apple DVCPRO HD codec shows some extreme difference results. That’s because it’s the only one of these codecs that uses horizontal raster scaling. Not only is the data compressed, but the image is horizontally squeezed. In this roundtrip, the image has gone from 1920-pixels-wide (TIFF) to 1280 (DVCPRO HD) back to 1920 (exported TIFF). The effects of this clearly show in the difference image.

Click on the above images to see an enlarged view.

There are a couple of other things you may notice, such as level differences between the Avid and Apple images and between each of these and the originals. As I said before, there will always be some differences in this sort of conversion. Plus, Apple and Avid do not handle color space, level and gamma mapping in the same way, so a round trip through each application will yield slightly different results. Generally, if 2.2 gamma is selected for imported stills, the Apple FCP image will have a bit more contrast and somewhat darker shadow areas when compared to Avid on a computer screen – even when proper RGB versus Rec. 709 settings are maintained for Avid. This is mainly a result of the various QuickTime and other conversions going on.

If I were to capture video with Avid DX hardware on the Media Composer and AJA, Matrox or Blackmagic hardware on FCP – and compared these images on a video monitor and with scopes – there would likely be no such visible difference. When I used “source” gamma in FCP, then the two matched each other. Likewise, when you review the difference images below, 2.2 gamma in this case resulted in a fault difference composite between the FCP uncompressed and the original photo. The “source” gamma version more closely resembles the Avid result and is the right setting for these images.

The take-away from these tests should be that the most important comparisons are those that are relative, i.e. “within species”. In other words, how does ProRes LT compare to ProRes HQ or how does DNxHD 145 compare to DNxHD 220x? Not, how an Avid export compares with a Final Cut export. A valid inter-NLE comparison, however, is whether Avid’s DNxHD220x shows more or less compression artifacts than Apple’s ProRes HQ.

I think these results are pretty obvious: Higher-data-rate codecs (less compression) like Apple ProRes HQ or Avid DNxHD 220x yield superb results. Lower-date-rate codecs (more compression) like XDCAM-EX yield results that aren’t as good. I hope that arming you with some visible evidence of these comparisons, will help you better decide what post trade-off to use in the future.

(In case you’re wondering, I do highly recommend the Donnington Grove for a relaxing vacation in the English countryside. Cheers!)

Click on these images to see an enlarged view.

©2010 Oliver Peters

MyMusicSource

Production music is a subjective decision. You can never have enough resources to satisfy clients. I routinely use a variety of options, including SmartSound, Adobe’s tracks for Soundbooth, Apple’s tracks for Soundtrack Pro and the whole range of music from Killer Tracks, FirstCom and others.

Now editors have a new option: MyMusicSource, which comes complete with a new plug-in for Apple Final Cut Pro. The plug-in was developed in partnership with and marketed through BorisFX. Right up front, let me disclose that I know the principals, have a little stock in the company and have been involved in some consulting and beta testing. MyMusicSource is the brainchild of Michael Redman – a veteran composer, producer, recording engineer, facility owner and software entrepreneur. In addition to Final Cut Pro, My Music Source is also actively developing other import plug-ins for various NLEs and DAWs.

Getting started

The release of the FCP import plug-in is of interest to Final Cut editors, of course, but anyone can use MyMusicSource with or without this plug-in. It’s a web-based, online resource for production music, so you can access, search, license, purchase and download music tracks using any regular web browser. The beauty of the FCP plug-in is that you can start and end the process from inside the FCP interface, but it isn’t essential. The plug-in itself is a free download, as is establishing an account with MyMusicSource. The company makes its money licensing music for productions.

Here’s a quick overview of how the plug-in and process works. Once you install the MyMusicSource plug-in (download from BorisFX), an option for MyMusicSource is added to FCP’s import menu, alongside XML, Sony XDCAM, EDL, etc. Select this and it launches your default web browser to the start page for FCP users. Log in using your established account and you are off and running. At this point the process is similar to other online music services. You can select and preview music by various search criteria in different genres. As you browse clips, add them to a project cart for later review.

One key difference from other companies is that MyMusicSource is upfront about licensing costs. Their whole approach is to “pre-clear” the music before you can download. At the beginning of your search, you should establish the intended production use for the music, before you add a track to your project cart. As a producer, you may purchase tracks with a Preview License for $.99 per track. This allows you to purchase and download a full-length, full-quality track and temporarily use it within your production (in-house preview use only).

Once a final set of tracks has been decided upon and the correct use established, you may purchase an upgrade to the license for legal use of that music. If you know in advance what the target use will be for the production, such as non-commercial web, you have the option to select that license rate instead. Each cut of music will display a price based on the selected licensing, so you instantly know what it will cost as you browse through the inventory. Non-commercial rates for personal use start at $5.

Project carts may be shared with your clients. If you’ve selected a handful of possible tracks for a client’s review, then share the cart and the client can access and preview these tracks. As with any shopping cart system, finalize your choices and proceed through checkout. Once you’ve paid, move on to the download center, where you find three options: Send to Final Cut Pro, Zip and Download NOW or Zip and Email. The last two options are the same as if you accessed the site without the FCP plug-in. Option one is enabled if you have the FCP plug-in installed.

You may also select between MP3 and 48K AIFF audio file formats. MP3 files are a faster download, but require a render in your FCP timeline. AIFF files will take a bit longer and are larger files, but work fine inside FCP. One option is to download MP3 files (using method 2 or 3) and then drag them into FCP via Digital Heaven’s Loader application. This converts the MP3 files into 48K AIFF. Another option is to convert MP3s using QuickTime Player Pro. These last two approaches work fine, but it means a tad more work and obviously detours away from the roundtrip magic. I normally opt for the AIFF files. One issue I’ve found is that the Send to Final Cut Pro feature has some access issues with FireFox, so use Safari 4 if you encounter these when using this method.

The last step of the roundtrip is back into FCP. A MyMusicSource media folder (containing the downloaded tracks) is placed into the same folder as your active FCP project file. A bin with the tracks is imported into the FCP project and shows up in your FCP browser. If you have more than one project open, you’ll receive a prompt to let the plug-in script know which project to use. Another handy feature of MyMusicSource is that when tracks are downloaded, you will also receive a PDF of the actual licensing information. This is great for the end of the project when you have to turn in music cue sheets and clearance information. It’s all right there from the very start!

OK, so the process is simple and straightforward, but what about the music itself? As I said at the start, music is subjective. The choices are good, but a big difference with the MyMusicSource inventory is an attempt to have a very contemporary sound. The selections are more artist-centric than I tend to see in the competition. There are also more vocal selections. A popular production trend is to use songs instead of just scores. That can get very expensive if you try to license songs that you’ve heard on the radio or on iTunes. In my opinion, MyMusicSource offers a wider selection of good vocal tunes than other libraries, so if your production needs the catchy sound of some indie, alt-rock band, then you’ve got plenty of options to choose from!

©2010 Oliver Peters

Easy Canon 5D post – Round II

RED’s Scarlet appears to be just around the corner and both Sony and Panasonic seem to be responding to the challenge of the upstart photo manufacturers. No matter what acronym you use – DSMC, HD-DSLR, HDSLR – these hybrid HD video / still photo cameras have grabbed everyone’s attention. 2010 may indeed be the year that hybrid digital SLR cameras hit their stride.

The Canon EOS 5D Mark II showed the possibilities in late 2008 when Vincent Laforet released Reverie, but like all of these new camera products, the big question was how to best handle the post. The 5D (so far) only shoots video at a true 30fps – lacking both the filmic 24fps rate – or any of the video-friendly frame rates (29.97, 25 or 23.976). That oversight was corrected in Canon’s EOS 7D and EOS 1D Mark IV models and may soon be corrected by a firmware update to the 5D. Even so, the 5D has remained a preferred option, because of its low light capabilities and full frame sensor. Photographers, videographers and filmmakers love the shallow depth-of-field, so a 24p-capable 5D is certainly on many wish lists.

Click the above image to enlarge

Until the 5D gets a 24fps upgrade [EDIT: coming in March, download will be here] , folks in post will have to contend with the 30fps footage generated by the camera. Last year I wrote an article on how to post a 5D project, which covers a lot of the basics. I’ve since done more 5D projects and formed a number of opinions and workflow tips. I’ve picked up many of these from reading Philip Bloom and Bruce Sharpe (PluralEyes inventor) and at the end of this post, I’ll include a number of useful links.

My first observation on the several 5D projects I’ve posted is that you get the best results from these new cameras when you treat them like film. Use classical production methods – slow pans, steady hand-held work, tripods, dollies and record audio as double-system sound. Secondly, allow time for processing files and syncing sound before you expect to start editing. 35mm film shoots typically require a day or more between the production day and post for lab processing and film transfer. The equivalent is true for HDSLRs. Whether it’s RED or an HDSLR, you have to become the film lab and transfer house. Once you wrap your head around that concept, the workflow steps make a lot more sense.

Click the above image to enlarge

I recently cut another Canon 5D Mark II job with Director/DP Toby Phillips. This was an internet commercial for the wine growers of the Yarra Valley region of Australia. Yarra Valley is to Australia, what Napa Valley is to California. Coincidentally, it’s also the region ravaged by the horrific fires of 2009. In order to keep the production light, Toby’s crew was bare bones and nearly all images were shot under available light – including sodium vapor lighting in warehouse areas. The creative concept was intended to be tongue-in-cheek. Real workers discussed why their job was the most important role in winemaking. The playful interplay between worker comments and winery/vineyard footage round out this :60 commercial.

Production tips

Toby rigged his camera with a modified plate, rails and matte box from his existing film equipment. This includes Arri and Manfrotto parts modified by Element Technica. The 5D records passable sound on its own, but it really isn’t ideal for the best quality. To get around this, a Zoom H4n handheld recorder was used for double-system sound. The Zoom has XLR inputs for external mics, in addition to its built-in XY-pattern stereo mics. A Sennheiser shotgun was plugged into the Zoom, which in turn recorded uncompressed 16-bit/48kHz WAV files. The headphone output of the Zoom was connected to the 5D, so that the camera files always contained reference audio.

There are a number of important tips to note here. First, there’s an impedance mismatch in this connection and the 5D uses an AGC circuit to attenuate audio, so the camera file audio will be clipped. To avoid this, turn down the headphone output level to a very low volume. Second, because the audio is clipped, if you forget to press record on the Zoom, the 5D’s audio is NOT acceptable. Following the traditional approach, a slate with clapstick was used for every sound take. The Zoom records numbered, sequential files, so the crew also wrote the audio file number on the slate for each take. These two steps make it easy to identify the correct audio take and to sync audio and video later in post.

Post workflow / pre-processing

This production configuration isn’t too different than shooting with other tapeless video cameras, but post requires a unique workflow. Key steps include video format conversion, speed adjustment and syncing the sound.

Video conversion – The Canon EOS 5D Mark II records 40Mpbs H.264 QuickTime movies in a 1920x1080p/30fps format. H.264 is not conducive to smooth editing in its native form. 5D files can be up to 4GB in length (about 12 minutes), but there is no clip-spanning provision, as in P2 or XDCAM. Where and when you convert the native H.264 camera files depends on your NLE. With Avid Media Composer, files are converted into Avid’s MXF format upon import. The import will be slow, since it’s also transcoding, but this is a one-step process. Unfortunately it ties up your NLE, so maybe in the future Avid’s MetaFuze or AMA will come to the rescue.

I cut with Apple Final Cut Pro, which does permit direct editing with the H.264 files, but you don’t really want to do that. I typically convert 5D files into Apple ProRes, using a batch setting in Compressor. You can use other codecs, of course, like DVCPRO HD, ProRes HQ, ProRes LT, etc. Philip Bloom likes to convert his files to the EX format using MPEG Streamclip. The reason for EX, according to him, is that the data rate is similar to the 5D files, so storage requirements don’t expand significantly.

The wine commercial had 127 camera files (2 hours 11 minutes of raw footage), which were converted to ProRes in about 4 hours on an 8-core Mac Pro. Storage needs increased from 40GB (H.264) to 142GB (ProRes). The nice part of this step (at least for FCP users) is that the conversion can be left as a batch to churn unattended. One word of caution, though. Compressor has a tendency to choke and crash when you throw tons of files at it, like 100+ camera files. So I usually do these conversions in groups of 20 or so files at a times.

Video speed adjustment - The 5D files are a true 30fps and not the fractional video rate of 29.97fps. Avid will convert these files to the correct rate on import, if audio and video tracks have been separated. According to Michael Phillips of Avid (one of their workflow gurus), “If the MOV file is video-only, then I use the ‘ignoreQtrate true’ console command and get a frame-for-frame import, resulting in a .1% slow down.” This is analogous to what happens when film is transferred to video. In my testing, it was important to first strip off the audio track of the MOV in order for this to work. You can do this using QuickTime Player Pro 7.

Final Cut permits native 30fps editing, but then your files won’t play through standard video gear, like a KONA card. I suppose for an internet spot this wouldn’t matter, however we had other uses, so a speed adjustment would have to happen at some point. I could either convert to 29.97 first and be done with it – or I could cut at 30fps and convert the finished spot. I normally opt to convert the ProRes files to 29.97fps first. To do this I use the Cinema Tools “conform” feature. That’s a nearly instantaneous process, which only alters the file’s metadata. It tells media players to run the file at the fractional frame rate of 29.97fps instead of 30fps.

Audio speed adjustment - Changing the frame rate from 30 to 29.97 means the picture has been slowed by .1% and so audio must also undergo the same pulldown. If you use a location sound recorder capable of a 48048kHz sample rate, then Avid Media Composer will automatically adjust the rate upon import back down to 48kHz and achieve the pulldown. In addition, there are various utilities that can “restamp” the metadata for the sample rate. A good choice is Sound Devices’ free Wave Agent. The Zoom recorder created 48kHz files, but these could be restamped as 47952kHz by such a software utility. In the case of Media Composer, the software sees this on import and slows the file by .1% to achieve the desired 48kHz sample rate. Thus the audio is back in sync.

Final Cut Pro works differently than Media Composer so your results may vary. FCP simply tries to maintain the same duration and thus would force a render in the timeline to convert the sample rate to 48kHz without altering the speed. Instead, I recommend that you render new versions of the audio before importing the files into FCP that have an applied speed change. When I initially tried the restamp approach, I got sync drift. After posting this entry, I tried it again with Wave Agent and the results were dead-on in sync. The only issue is that then you have to render the audio in FCP to get the correct sample rate. I’m not a big fan of how FCP renders audio files and so prefer to correct them prior to import into FCP. I have also had inconsistent results with FCP and how it handles sync with external audio files.

Because of these various concerns, I used Telestream Episode Pro and created an audio-only preset that included a speed change with a .999 value. I used this preset to batch-convert 20 16-bit/48kHz WAV files from the Zoom recorder (1 hour 9 minutes of raw dialogue) into “pulled down” AIF files. This took about two minutes. Whichever approach you take, I urge you to do this only with copies of files. Some of these various utilities use destructive processes, so you don’t want to change your originals.

(Note: For a better understanding of how BWF (broadcast wave files), QuickTime and Final Cut Pro interact, check out this product (BWF2XML) and description by Spherico.)

Syncing the dailies – After these conversion steps, the files are ready to import into FCP. Audio and video files are now in optimized formats that will match FCP’s native media settings. Next, you’ll have to sync the audio and video takes. If the crew used a clapstick, it’s easy to sync in either Avid or Final Cut using the standard group or multiclip routines.

For this wine spot, I used Singular Software’s PluralEyes to automatically sync all sound takes. PluralEyes was one of the highlights of NAB 2009 and is about as close to magic as any software can get. It analyzes audio waveforms to compare and align the reference camera audio against the separate audio files. This is why it’s critical to record even poor-quality reference audio to the camera in order to give PluralEyes something to analyze. Unfortunately for the Avid editor, PluralEyes only works with Final Cut and Sony Vegas Pro. It’s not a plug-in, but works on a timeline labeled “pluraleyes” in an open and saved FCP project.

Here are the steps:

a) Create a blank FCP timeline named “pluraleyes”.

b) Drag & drop all camera clips with dialogue (audio & video) onto the timeline (random order is OK).

c) Drag & drop all separate audio files onto the same timeline onto unused audio tracks (random order is OK).

d) Disable any redundant audio track (speeds up analysis).

e) Save the project, launch PluralEyes, start analysis/sync processing.

After a few minutes of processing, PluralEyes will automatically create a series of new FCP sequences – one for each sync take. The audio will be aligned so that the double-system sound files are now perfectly in sync with the camera audio.

Post workflow / edit / mix / grade

Now that you have sync takes, you can pretty much edit anyway you like. I picked the following tip up from Bloom’s blog. To make editing easier on the wine spot, I took these new sequences and renamed them according to the person who was speaking and which take it was. I export the sequences as QuickTime references movies (not self contained) to a location on my media drives. I then re-imported these reference movies, in effect turning them into master clips with merged 5D video and Zoom audio. These became my source for all sync takes. Any b-roll shots came from the regular ProRes files.

The rest of the edit went normally. I’ve got my MacPro set up with two internal 1TB drives configured as a software RAID-0 for media files (2TB). No issues with cutting ProRes this way. I bounced the audio to Soundtrack Pro for the final mix. No real reason, other than to take advantage of some of the plug-ins to add a touch of “sparkle” to the dialogue.

I used Apple Color for the grade. If you follow my blog, you know that I could have tackled this easily with various plug-ins and stayed inside FCP, however, I do like the Color interface and toolset. This spot was ideally suited to go through a grading pass using Color. As it turned out, this step might have been a bit premature due to client revisions. In hindsight, using plug-ins might have been preferable. I thought the cut was locked, so proceeded with the correction in Color.

The first version of the spot was a faster paced cut (57 shots in :60), so the client requested a second version with a little more breathing room and a few alternate dialogue takes. This necessitated going back into the footage. Those familiar with Color know that it generates new media files when it renders color correction. This is required to “bake in” the color corrections. If you assign handles of a few seconds to each shot, you have some room to trim shots when you are back in Final Cut. This doesn’t help you with other footage.

I decided to step back to the sequence before “sending to” Color and cut a second, more-relaxed version (46 shots in :60). Although this meant starting a new Color project, I was aided by Color’s ability to store grades. I could save the settings for each of the shots in version one and apply these settings to the similar or same shot in version two, within the new Color project. Adjust keyframes, tweak a few settings, render and bingo! – the grade is done. With :02 handles on each shot, version one (57 shots) rendered in about 40 minutes and version two (46 shots) took about 30 minutes. Both as 1920×1080 ProRes (29.97fps) media. Of course, like many commercials this wasn’t the end and a few more changes were made! The final version ended being a combination of these two cuts.

(As an aside, Stu Maschwitz has done a nice post about Color Correcting Canon 7D Footage on his ProLost blog.)

Post-processing / 24fps conversion

This could have been the end of the post for the wine spot, but there’s one more step. A big reason people like these HDSLRs is because they provide a very cost-effective way of getting that elusive “film look”. One part of that look is the 24fps frame rate. Yes – some film is shot at 30fps for spots and TV shows – so technically the 5D’s 30p footage is just fine. But clients really do want that 24fps look.

You can convert these 5D files quite cleanly to 24fps. This is a process I picked up from Bloom and discussed in my previous Canon post.  Here are the steps:

a) Note the exact duration of the 29.97fps timeline.

b) Export a self-contained QuickTime movie of the finished 29.97 sequence.

c) Bring that exported file into Compressor and set up a ProRes-to-ProRes conversion. Use a frame rate of 24fps (it actually is 23.98, but Compressor labels it as 24).

d) Turn Frame Controls on, set Rate Conversion to Best and change Duration from 100% of source to the exact duration of the original 29.97 timeline.

Now let Compressor crunch for a while.  My :60 spot took about 36 minutes to convert from 29.97 to 23.98. For good measure, I also take the finished file into Cinema Tools and conform it to 23.98, just in case it’s 24 and not 23.98. Then I import the file back into FCP. I create a new 23.98 timeline and edit the clip to the file. If everything is done correctly, this media should match without any rendering needed. Then I’ll copy and paste the audio from the 29.97 timeline to the 23.98 timeline. This should be in sync.

A couple of additional pointers. Since I don’t want to have this conversion process get confused with titles and dissolves, I remove all graphics and make dissolves into cuts (with handles) in the 29.97 sequence, prior to export. I actually exported the wine spot timeline as 1:04 instead of :60. When I was back in the 23.98 timeline, I fixed these trims, added back the fades, dissolves and graphics in order to complete the sequence.

The second issue is speed changes. I sped up two shots, which actually passed through Color and this 24p conversion just fine – except for one problem. My 29.97 timeline was actually an interlaced timeline. This doesn’t matter for the camera files, as they are inherently progressive. However, any timeline effects, like speed changes, titles and transitions are processed with interlaced motion. This affected the two sped-up shots in the 24p conversion, resulting in interlace artifacts. The simple fix was to replace these with the normal-speed media and redo the speed change in the 23.98 timeline. No big deal, but something to be mindful of in the future.

Finally, although this conversion is very good, it isn’t perfect. Cuts do stay as clean cuts and slow action converts cleanly looking as if it were shot at 24fps. Fast motion, however, does introduce some artifacts. These mainly show up as blended frames in areas of fast activity or fast camera movement. It’s no big deal really, as it tends to add to the filmic look of the material – a bit like motion blur.

Remember that this is an OPTIONAL and SUBJECTIVE step. I personally think that 30p is a “sweet spot” for LCD and plasma screens. This is especially true for the web and computer displays. In the end, my client decided they liked the 30p image better, because it was crisper.

Click the image to see the video in HD on Vimeo.

Or here for the “Alternate Cut” at 30fps (no 24p conversion).

Additional tools

Since the media files the HDSLR cameras generate are an outgrow of file creation at the consumer product level, there is very little metadata in them that an NLE would care about. No reel numbers, SMPTE timecode, edge numbers, etc. That’s good and bad. Good – in that the folder and file structure is quite simple and very malleable. Bad – in that you can have duplicate file names and there’s no ability to span clips. Think of it like a roll of 35mm negative. That would have about 11 minutes of capacity and new metadata is added when it’s transferred to video.

Since files are sequentially numbered on the memory card, once you start recording to the next card, it’s likely to have repeating file names. This is true both in the camera and on a recorder like the Zoom, simply because there is no reel (i.e. card) ID name or number.  The good news is that you can easily change this without corrupting metadata – as you would with RED or P2 – but, it means you have to manually impose some sort of structure yourself.

R-Name - One utility that can help in R-Name. Unfortunately it may be out of development, but I still use version 3, which works with Snow Leopard. You might be able to find a download still lurking in the depths of the internet – or, if not – a similar utility or an Automator routine. R-Name lets you rename files (as the name implies), but you can also append prefix or suffix character strings to a file name. For example, a set of media files from a 5D may be named MVI_1073.mov through MVI_1200.mov and you’d like to add a prefix for Card 1. Simply create an R-Name batch that adds a prefix such as “C001_” to all these files. Run the batch and voila – your files are now named C001_MVI_1073.mov through C001_MVI_1200.mov. Follow this process for each card and it becomes a nice, fast way of organizing your media.

QtChange – If reel numbers and timecode are important for you to have, then check out VideoToolshed’s QtChange. This is a comprehensive QuickTime utility, which lets you alter several file parameters. Most importantly, you can add or change reel number and timecode values. Although this isn’t essential for you to cut in FCP, certain functions, like dupe detection, won’t work without an assigned reel number. There are several ways to alter this info in QtChange, but one of the ways it can work is to automatically use the date stamp of the file for the reel number and the time stamp as a starting timecode number. Files can be changed in a batch, but be careful as these are destructive changes. Developer Bouke Vahl has been making ongoing changes to the product and recently added Avid Log Exchange functionality.

MetaCheater One deficiency of Avid Media Composer has been the inability to directly read all of the metadata from a QuickTime file. For instance, older versions of Media Composer and Symphony would not read QuickTime timecode. This has been corrected in the most recent versions; these apps now import the timecode, but still no reel number. In addition, the Canon cameras don’t generate timecode or reel numbers so you must add them if you need such information. You could use QtChange to add reel IDs and timecode, which Media Composer would import, but then there’s still the reel ID problem. MetaCheater is a simple way around this. This program extracts QuickTime metadata and creates an Avid Log Exchange file (ALE) with proper reel numbers and timecode values. Import the ALE file into Media Composer and then batch import the corresponding QuickTime movies. In this process, Media Composer uses the timecodes and reel numbers from the ALE instead of default values, with the result that your Avid bins properly reflect the reel and timecode information added to the 5D files. It would be just as if this media had been captured from a videotape source.

Here are a few comparisons of the color grading applied to these shots.

Click each of these images to see an enlarged view.

Original Image

Graded Image

Split Screen

Original Image

Graded Image

Split Screen

Addendum (Feb 2010)

After I initially wrote this article in January, I pulled it down for some tweaks. In the interim, I got busier for a few weeks until I could repost it. In that time, was able to do some more testing with Avid Media Composer 4.0.5 on another Canon 5D spot. I am adding my observations here, since many of my readers are Avid cutters and want to know the best way to handle these files in Media Composer.

Unlike FCP, there’s no simple drag-and-drop method in Avid. If you elect to convert the files using an external encoding application, you still have to bring the files in through Avid’s import routines. This adds a step and effectively doubles the total time it takes to convert and import as compared with FCP. Another frustrating issue is that when you move from the native camera files into Avid, you have to move out of the QuickTime color and gamma architecture and into an MXF structure using Avid codecs.

In the Avid world, video files are treated using the rec. 601/709 colorspace (16-235 on an 8-bit scale) and computer files are assumed to be in RGB space (0-255). When you import or export files to and from Media Composer, you always need to check the proper setting – RGB or 601/709. Unfortunately (or fortunately depending on your POV), this is largely hidden from view in the QuickTime world. Furthermore, Canon really hasn’t provided documentation that I’m aware of regarding the colorspace that these cameras work in and how closely color scaling conforms to either RGB or rec. 709. The long and short of it is that when you move in and out of QuickTime, you are often fighting level and gamma changes to varying degrees.

I tried a number of different import and encoding methods with Media Composer. All of them work, but with various trade-offs. The easiest method is as I outlined earlier in this article – simply import the H.264 camera files into Media Composer. When you do that, select RGB color space. The import time will take approximately 3:1 to 4:1 on a fast machine, depending on the target codec you choose to use, because the media is being transcoded during this import stage. I had the fastest encoding times using the Sony XDCAM-EX codec, which is now natively supported by Media Composer.

A second option is to use Apple Compressor (or another QuickTime encoder) to convert the camera files into QuickTime movies using an Avid DNxHD codec. This is the same approach as converting to Apple ProRes 422. Unfortunately, Avid still imposes a longer import time to get these files from QuickTime MOVs into the MXF media format. Although Compressor offers a choice between RGB and 709 when you select DNxHD, it doesn’t seem to make any difference in the appearance of the files. The files are converted to 709 color space and so should be imported into Avid with the import setting on 709. I hope that this import step will be eliminated at some point in the future, when and if Avid decides to support QuickTime files through its AMA feature.

The fastest, current method was to use Episode Pro again. MXF is now supported in this encoder, so I was able to convert the H.264 files into MXF-wrapped XDCAM-EX files that were ready for Avid. The beauty of is that the work can be done on an external machine in a batch and the import back into Media Composer is very fast. No transcoding is needed, as this just becomes a file copy. The EX codec looked clean and wasn’t too taxing on my Mac Pro. You also have the option of using XDCAM-HD and XDCAM HD 422 (50Mbps) codecs in the MXF file format. The only issue was that one of the media files appeared to be corrupt after encoding and had to be re-encoded. This might be an anomaly, but we ARE dealing with two long-GOP codecs in this process! Another benefit of this route is that no user interaction is required to determine color space settings.

Now to the level issues. In all of this back and forth – once I exported back out to QuickTime (ProRes 422 codec, using RGB setting on export) – no conversion identically matched the original camera files. When I compared versions, direct import of the files (H.264 into Avid) yielded slightly darker results. External conversion to DNxHD and then importing, yielded a slight gamma shift. Conversion/import via the MXF route appeared a bit lighter than the original. None of these were major differences, though. If you are going to color grade the final product anyway, it doesn’t really matter. I finally settled on a 2-step conversion workflow (described in my February 21 post) that yielded good results going from the 5D files into Media Composer and then to FCP.

As far as editing, syncing and grading, that is the same as with any other acquisition media. I used the same preparatory steps as outlined earlier (Cinema Tools conform to 29.97 and a .999 speed adjustment of the audio) – then converted and imported the video files. Inside Media Composer (1080p/29.97 project), everything synced and edited just as I expected.

Also in early February, Canon announced its EOS Movie Plugin-E1 for Final Cut Pro. Click here for the description. It’s supposed to be released in March and if I understand their description correctly, it allows you to import camera clips via FCP’s Log and Transfer module. During the import stage, files are transcoded to ProRes. Unfortunately there is no explanation of how frame rates are handled, so I presume the files are imported and remain at their original frame rate.

My conclusion after all of this is that both FCP and Media Composer are just fine for working with HDSLR projects. FCP seems a bit faster at the front, but in the end, you’re just traveling two different roads to get to the same destination.

I leave you with one last tidbit to ponder. Apple has just introduced Aperture 3, which includes HD video clip support in slideshows. I wonder how apps like Aperture, Lightroom and Photoshop (already supports some video functions) will impact these HDSLR workflows in the future?

(UPDATE: If you got here through links from other blogs, make sure you read the updated Round III post as well.)

Useful Links

5DMk2 blog – 1001 Noisy Cameras

Assisted Editing

Philip Bloom

Canon Explorers of Light

Canon Filmmakers

Cinema5D

DSLR HD

DVinfo

DVXuser

Element Technica

FreshDV

Tyler Ginter

Vincent Laforet

ProLost

Red Rock Micro

Bruce Sharpe

Spherico

Peter Wiggins

Planet5D

Video Toolshed

Zacuto

©2010 Oliver Peters

The Final Cut Studio centric facility

Back in the day, the last large mainstream facility that I helped design, install and manage was geared towards episodic TV series post. We had several dedicated linear bays, DaVinci film transfer, graphics/VFX and audio mixing suites. This was a powerful (but expensive) combination for the time, however, it lacked the versatility to change the dedicated function of any of these rooms into something else. If you needed more edit bays or 2 film transfer suites for a short period of time, there was no way to do that other than simply shuffling clients around or running extra shifts. It kept our operations folks quite busy adjusting the schedule.

Fast forward to this century. Although nonlinear software has changed the facility paradigm, I’m not too sure that many facility managers aren’t still operating their rooms as if they only served one function. One of the biggest benefits of the Final Cut Studio bundle of applications, is that it truly replicates all the parts of most facility service offerings: edit (Final Cut Pro), audio editing/mixing (Soundtrack Pro), color grading (Color), motion graphics (Motion) and compression/web/authoring (Compressor, DVD Studio Pro). Apple has done this better than its competitors, making it easy to stay within the Apple/Final Cut ecosystem. Adobe’s Creative Suite comes closest, but Soundbooth isn’t as strong as Soundtrack Pro and there is no equivalent to Color. Avid’s bundle is strong for editing, but the rest of the apps are designed to augment Media Composer without providing equal strengths in other areas.

Optimizing the boutique

Lets take a look at a typical small-to-medium boutique facility with five installed post production suites. In a scenario where each of these workstations is equipped with Final Cut Studio, any of the rooms can be dedicated to any of the functions of the Studio bundle. If you need 5 edit bays on Monday, then 5 mixing suites on Tuesday and finally 5 color grading suites on Wednesday, it’s simply a matter of which application you launch at any given time. In short, the scheduling department has no problem reassigning the functions of these rooms in a flexible manner, based on the five core roles of Final Cut Studio – edit, sound, color, motion graphics and compression/authoring.

Of course, I’m ignoring several impediments to making this work out in the real world. First off, I’m assuming you will actually use these applications for the intended tasks. Pro Tools is still the preferred DAW in most markets and After Effects tends to be the “go to” motion graphics tool over Motion. I’m not going to argue those points, except to say that in the right hands, both Soundtrack Pro and Motion are viable applications that can be used to turn out excellent work. There’s no reason to spend extra money and effort adding Pro Tools or After Effects into the equation. The second issue is manpower and training. Most editors are not good at all of these tools. Depending on your market, this can be fixed through freelance support and/or additional operator training. I’m not saying that your current roster permits you to properly staff 5 color grading suites, merely that the software and facility configuration can take advantage of this versatility if you can make it work for you.

That brings us to the main point, which is how to best optimize your facility to achieve such a “musical chairs” approach. The first key in my mind is shared storage. There are lots of ways to solve the storage issue, with a variety of solutions including Facilis Technology (Terrablock) and EditShare on one end and Maxx Digital’s Final Share on the other. In keeping with the Apple theme of this article, the optimum solution (budget-permitting) would be Apple’s Xsan networking software and Active Storage arrays. Xsan requires a professional installation to make sure it is set up right, but from first hand experience, I can attest to the fact that it’s extremely easy for the user, once installed. You can move among rooms and access projects and media without issue. Editing in Suite A on Monday, mixing in Suite B on Tuesday and grading in Suite C on Wednesday – all from the same project – is a breeze with Xsan.

Don’t forget the rest of the room. The reason a mixing or color grading suite tends to be built as a single purpose room is because of accessories like custom control panels, monitoring and acoustic treatment. Many of these items have come down in price and I would offer that these features can be cost-effectively added to the design of all of the suites. For example, I’m a fan of control surfaces – partial to panels from Mackie and Euphonix for mixing. It’s not terribly expensive to add small-footprint panels like Euphonix’s MC Mix and MC Color or the Tangent Devices Wave to all five of the suites in our example. Furthermore, decent audio, video and tech monitoring should be a given, no matter what. If you add it all up, I would suggest that each suite can be equipped with good monitoring, panels and an acoustic kit for approximately $10K – $15K per room. Now that may sound high to some, but the linear facility I described at the outset came with Sony BVM-1910 monitors and a set of Tektronix scopes in every room – far greater expense than $10K/room!

I do recognize that may be a big number to many, so lets see how to offset the cost. First off, when I recommend a Final Cut workstation to folks, I usually suggest a KONA3 or KONA LHi capture card. Our hypothetical boutique don’t necessarily need a high-powered card like this in every room. For example, if most of the work through the facility is file-based, using tapeless acquisition – and delivery rarely involves tape – then each of these stations only need be equipped with a card that provides good, but low cost output. This can be easily handled by one of the entry level Blackmagic Design cards or one of the Matrox MXO2 products. In other words, save on the workstation capture card and spend it on the panels, monitors, etc.  Another item that fits neatly into this scenario is Telestream’s Pipeline. This is a high-quality I/O device, like a KONA or Blackmagic card, but unlike these, is accessed over a LAN. As such, you can share a Pipeline among these five suites for the occasional ingest or output to/from videotape decks.

How to augment the bundle

What’s missing in this picture? First off you need to evaluate your use of plug-ins and fonts. In order to be able to easily move projects among workstations and applications, you need to make sure that all have the same plug-ins and fonts installed. Since we have five bays in our example, it’s probably unlikely that most owners are going to buy five sets of several expensive plug-in packages. I recommend that you limit this to one comprehensive package and buy it for all stations. Then augment those with a selection of many of the free Final Cut filters available on the web.

The next item would be productivity tools, like Microsoft Office. Editors frequently have to access scripts, schedules and other client materials at their workstation. These tend to be in Word or Excel formats. The home edition of Office isn’t that expensive and does the trick, but alternatives include Apple’s iWork, TextEdit (included) or NeoOffice (free). Remember, you’ll want this on each workstation.

The last big issue is Photoshop. I find that almost every project I touch makes use of Adobe Photoshop. Unfortunately it’s become pretty pricey, so I can see where a facility would be reticent to install a seat on every workstation. There are a number of alternatives in various price ranges to consider. This depends on whether you need actual graphic design, compositing,  photo manipulation or other features. Don’t forget that a lot of this editorial graphics prep can be done in Motion. Other options include GraphicConverter, Photoshop Elements, Pixelmator and Corel Painter. If you only need photo tools, then of course there’s iPhoto, Aperture or Lightroom. In addition, iWork’s applications also include some built-in image editing tools.

Variations on a theme

Maybe you want to set up one dedicated design station, because you have a talented staff artist and don’t want to tie up an edit bay fulltime. Or maybe you need to log a lot of footage and again view this as a task that can be done outside of the suite. This is the scenario where an iMac might be the ideal station. Systems for either of these situations could connect to the media pool using Ethernet instead of fibre channel. Our graphics station might have a full installation of Adobe’s Creative Suite so that the artist can use After Effects, Illustrator, In Design and more.

In the example of the logging station, the Final Cut ecosystem comes to our aid. First off, the Telestream Pipeline includes separate capture utilities, so an iMac could be used to capture media to the Xsan media pool, without the need to install a seat of Final Cut Studio on that machine. Secondly Digital Heaven’s MovieLogger was specifically designed for this need. A story editor can use MovieLogger to open and review QuickTime files without Final Cut. Markers and description may be added and exported as XML documents. Once imported into Final Cut, the media and master clips, complete with marker information, are available to the editor. Additional useful tools to study include those from Video Toolshed, XMiL and Assisted Editing.

The last variation to discuss is the “hero” room. Granted that’s the opposite of how I started this article, but many facilities still structure their pipeline like a funnel. One central high-end “finishing” suite as the gateway for the completed project, including color correction and visual effects. The reasons may be cost, personnel strengths or a client whizz-bang factor. It’s a scenario that makes Autodesk’s Smoke on the Mac attractive to Final Cut Studio facilities. A lot of pixels have been spilled discussing the relative merits of the product and its price. Although $15K for Smoke (plus a $2K/yr subscription) is high in typical Apple terms, remember that you wouldn’t spend as much on the other 4 or 5 rooms. Since a “hero” room wouldn’t follow the same “musical chairs” approach I described earlier, you probably wouldn’t buy panels, capture cards and high-quality monitors in each of the other bays. These rooms would be centered around rough-cutting, not finishing, therefore no need to invest $10K-$15K/room for accessories. That leaves you money to invest in Smoke.

As a former facility manager, I can appreciate that Final Cut Studio is the first software product mix that optimizes what can be done in the boutique post operation. It’s this level of versatility that continues to make Final Cut attractive and a hard act to beat.

©2010 Oliver Peters

Mixing formats in the edit

The sheer mix and volume of formats to deal with today can be mind-boggling. Videotape player/recorders – formerly a common denominator – are a vanishing breed. Post facilities still own and use VTRs, but operations at the local market level, especially in broadcast, are becoming increasingly tapeless. Clearly, once the current crop of installed VTRs become a maintenance headache or are no longer an important cog in the operation, they won’t be replaced with another shiny new mechanical videotape transport from Sony or Panasonic.

It all starts with the camera, so the driving influence is the move to tapeless acquisition – P2, XDCAM-HD, GFcam, AVC-HD and so on. On the bright side, it means that the integration of another format will cost no more than the purchase of an inexpensive reader, rather than a new VTR to support that format. Unfortunately this will also mean a proliferation of new formats for the editor to deal with.

The term format should be clarified with tapeless media, like P2. First, there is the codec used for the actual audio and video content (essence). That essence is defined by the compression method (like DVCPRO HD or AVC-Intra), frame size (SD or HD), pixel aspect ratio and frame rate. The essence is encapsulated into a file wrapper (MXF), which holds the essence and metadata (information about the essence). Lastly, in the P2 example, the files are written to a physical transport medium (the P2 card itself), using a specific folder and file hierarchy. Maintaining this folder structure is critical in order that an NLE can natively recognize the media, once it’s copied from the card to a hard drive.

Nonlinear editing systems have been built around a specific media structure. Avid Media Composer uses OMF and MXF. Apple Final Cut Pro is based on QuickTime. In theory, each can ingest a wide range of tapeless file formats, but the truth is that they only work well with a much narrower range of optimized media. For instance, DVCPRO HD is handled well by most NLEs, but H.264 is not. You can toss a mix of formats onto a common timeline, but the system is internally operating with specific settings (codec, frame size and frame rate) for that timeline.

These settings are established when you first create a new project or a new sequence, depending on the application. Any media on the timeline that deviates from these settings must either be scaled and decompressed on-the-fly by the real-time effects engine of the application – or must be rendered – in order to see full-quality playback.  Most systems are optimized for NTSC, PAL, 720p and 1080i frame sizes. Even Final Cut Pro – touted as resolution independent – works best at these sizes and effectively tops out at 2K film sizes. All the desktop NLEs freely allow you to mix SD and HD content on a timeline, but the rub has been a mix of differing frame rates. FCP could do it, but Media Composer wouldn’t. That barrier disappeared with Avid’s introduction of the Mix & Match feature in the Media Composer 4.0 software. Now, if you edit a native 23.98p clip into a 29.97i timeline, all of the leading editing applications will add a pulldown cadence to the 23.98p clip for proper 29.97i playback.

When editing a project that has a mix of SD and HD sources and formats, it is best to select a timeline or project setting that matches the predominant format. For instance, if 75% of your media was shot using a Panasonic VariCam at 720p/59.94, then you’d want to use a matching timeline preset, so that the 720p footage wouldn’t require any rendering,  except for effects. In this example, if the other 25% was NTSC legacy footage from Betacam-SP, you’d need to have a system equipped with a capture card capable of ingesting analog footage. The Beta-SP footage could be upconverted to HD during the capture using the hardware conversion power of a capture card. Alternately,  it could be captured as standard definition video, edited onto the timeline and then scaled to fill the HD frame. Betacam-SP clips captured as standard definition video would ultimately be rendered to match the 720p/59.94 settings of the timeline.

Until recently, Avid systems transcoded incoming media into an Avid codec wrapped as an MXF file. This creates media files that are optimized for the best performance. Final Cut would let you drag and drop any QuickTime file into the FCP browser without a transcode, but non-QuickTime files had to be converted or rewrapped as QuickTime MOV files. These frontrunners were upstaged by applications like Grass Valley EDIUS and Sony Vegas Pro, which have been able to accept a much wider range of media types in their original form. The trend now is to handle native camera codecs without any conversion. Apple added the Log and Transfer module to Final Cut and Avid added its Avid Media Access (AMA). Both are plug-in architectures designed for native camera media and form a foundation for the use of these files inside each NLE.

Final Cut’s Log and Transfer is recommended for importing P2, RED, XDCAM and other media, but it still doesn’t provide direct editing support. Even natively-supported codecs, like REDCODE and AVC-Intra must first be wrapped as QuickTime files. When clips are ingested via Log and transfer, the files are copied to a target media drive and in the process rewrapped as QuickTime MOV file containers. It’s Apple’s position that this intermediate transcode step is a safer way to handle camera media without the potential of unrecoverable file corruption that can occur if you work directly with the original media.

If you want true native support – meaning the ability to mount the hard drive or card containing your raw media and start editing at full resolution – then the Avid Media Composer family, Grass Valley EDIUS and Adobe Premiere Pro provide the broadcaster with the strongest desktop solutions. All three recognize the file structure of certain camera formats (like P2), natively read the camera codec and let you use the media as an edit source without the need to transcode or copy the file first. These APIs are evolving and are dependent on proper media drivers written by the camera manufacturers. Not all applications handle every format equally well, so select a system that’s appropriate for you. For example, P2 using the DVCPRO HD or AVC-Intra codec is becoming widely supported, but Panasonic’s AVCCAM has less support. Sony hit snags with XDCAM-EX support for Final Cut Pro when Apple upgraded the Mac OS to 10.6 (“Snow Leopard”). Fortunately these issues are short-lived. In the future it will be easier than ever to mix taped and tapeless camera media of nearly any format with little negative impact.

Written for NewBay Media and TV Technology magazine

©2009 Oliver Peters