Post Production Mastering Tips

The last step in commercial music production is mastering. Typically this involves making a recording sound as good as it possibly can through the application of equalization and multiband compression. In the case of LPs and CDs (remember those?), this also includes setting up the flow from one tune to the next and balancing out levels so the entire product has a consistent sound. Video post has a similar phase, which has historically been in the hands of the finishing or online editor.

That sounds so sweet

The most direct comparison between the last video finishing steps and commercial music mastering is how filters are applied in order to properly compress the audio track and to bring video levels within legal broadcast specs. When I edit projects in Apple Final Cut Pro 7 and do my own mixes, I frequently use Soundtrack Pro as the place to polish the audio. My STP mixing strategy employs tracks that route into one or more subgroup buses and then a master output bus. Four to eight tracks of content in FCP might become twenty tracks in STP. Voice-over, sync-sound, SFX and music elements get spread over more tracks and routed to appropriate subgroups. These subgroups then flow into the master bus. This gives me the flexibility to apply specific filters to a track and have fine control over the audio.

I’ll usually apply a compressor across the master bus to tame any peaks and beef up the mix. My settings involve a low compression ratio and a hard limit at -10dB. The objective is to keep the mix levels reasonable so as to preserve dynamic range. I don’t want to slam the meters and drive the signal hard into compression. Even when I do the complete mix in Final Cut, I will still use Soundtrack Pro simply to compress the composite mix, because I prefer its filters. When you set the reference tone to -20dB, then these levels will match the nominal levels for most digital VTRs. If you are laying off to an analog format, such as Betacam-SP, set your reference tone to -12dB and match the input on the deck to 0VU.

Getting ready for broadcast

The video equivalent is the broadcast safe limiting filter. Most NLEs have one, including Avid Media Composer and both old and new versions of Final Cut. This should normally be the last filter in the chain of effects. It’s often best to apply it to a self-contained file in FCP 7, a higher track in Media Composer or a compound clip in FCP X. Broadcast specs will vary with the network or station receiving your files or tapes, so check first. It’s worth noting that many popular effects, like glow dissolves, violate these parameters. You want the maximum luminance levels (white peaks) to be limited to 100 IRE and chrominance to not exceed 110, 115 or 120, depending on the specs of the broadcaster to whom you are delivering. In short, the chroma should stay within the outer ring of a vectorscope. I usually turn off any RGB limiting to avoid artifacts.

It’s often a good idea to reduce the overall video levels by about five percent prior to the application of a broadcast safe filter, simply so you don’t clip too harshly. That’s the same principle as I’ve applied to the audio mix. For example, I will often first apply a color correction filter to slightly lower the luminance level and reduce chroma. In addition, I’ll frequently use a desaturate highlights or lows filter. As you raise midrange or highlight levels and crush shadows during color correction, the chroma is also driven higher and/or lower accordingly. Red, blues and yellows are most susceptible, so it’s a good idea to tone down chroma saturation above 90 IRE and below 20 IRE. Most of these filters let you feather the transition range and the percentage of desaturation, so play with the settings to get the most subtle result. This keeps the overall image vibrant, but still legal.

Let me interject at this point that what you pay for when using a music mastering specialist are the “ears” (and brain) of the engineer and their premium monitoring environment. This should be equally true of a video finishing environment. Without proper audio and video monitoring, it’s impossible to tell whether the adjustments being made are correct. Accurate speakers, calibrated broadcast video monitors and video scopes are essential tools. Having said that though, software scopes and modern computer displays aren’t completely inaccurate. For example, the software scopes in FCP X and Apple’s ColorSync technology are quite good. Tools like Blackmagic Design Ultrascope, HP Dreamcolor or Apple Cinema Displays do provide accurate monitoring in lower-cost situations. I’ve compared the FCP X Viewer on an iMac to the output displayed on a broadcast monitor fed by an AJA IoXT. I find that both match surprisingly well. Ultimately it gets down to trusting an editor who knows how to get the best out of any given system.

Navigating the formats

Editors work in a multi-standard world. I frequently cut HD spots that run as downconverted SD content for broadcast, as well as at a higher HD resolution for the internet. The best production and post “lingua franca” format today is 1080p/23.976. This format fits a sweet spot for the internet, Blu-ray, DVD and modern LCD and plasma displays. It’s also readily available in just about every camera at any price range. Even if your product is only intended to be displayed as standard definition today, it’s a good idea to future-proof it by working in HD.

If you shoot, edit and master at 1080p/23.976, then you can easily convert to NTSC, 720p/59.94 or 1080i/29.97 for broadcast. The last step for many of my projects is to create deliverables from my master file. Usually this involves creating three separate broadcast files in SD and two HD formats using either ProRes or uncompressed codecs. I will also generate an internet version (without bars, tone, countdown or slate) that’s a high-quality H.264 file in the 720p/23.976 format. Either .mov or .mp4 is fine.

Adobe After Effects is my tool of choice for these broadcast conversions, because it does high-quality scaling and adds proper cadences. I follow these steps.

A) Export a self-contained 1080p/23.976 ProResHQ file from FCP 7 or X.

B) Place that into a 720×486, 29.97fps After Effects D1 composition and scale the source clip to size. Generally this will be letterboxed inside of the 4×3 frame.

C) Render an uncompressed QuickTime file, which is lower-field ordered with added 2:3 pulldown.

D) Re-import that into FCP 7 or X using a matching sequence setting, add the mixed track and format it with bars, tone, countdown and slate.

E) Export a final self-contained broadcast master file.

F) Repeat the process for each additional broadcast format.

Getting back there

Archiving is “The $64,000 Question” for today’s digital media shops. File-based mastering and archiving introduces dilemmas that didn’t exist with videotape. I recommend always exporting a final mixed master file along with a split-track, textless submaster. QuickTime files support multi-channel audio configurations, so building such a file with separate stereo stems for dialogue, sound effects and music is very easy in just about any NLE. Self-contained QuickTime movies with discrete audio channels can be exported from both FCP 7 and FCP X (using Roles).

Even if your NLE can’t export multi-channel master files, export the individual submixed elements as .wav or .aif audio files for future use. In addition to the audio track configuration, remove any titles and logos. By having these two files (master and submaster), it’s very simple to make most of the future revisions you might encounter without ever having to restore the original editorial project. Naturally, one question is which codec to use for access in the future. The preferred codec families these days are Avid DNxHD, Apple ProRes, uncompressed, OP1a MXF (XDCAM) or IMX. FCP editors will tend towards ProRes and Avid editors towards DNxHD, but uncompressed is very viable with the low cost of storage. For feature films, another option to consider would be image sequences, like a string of uncompressed TIFF or DPX files.

Whichever format you standardize on, make multiple copies. LTO data tape is considered the best storage medium, but for small files, like edited TV commercial masters, DVD-ROM, Blu-ray and XDCAM media are likely the most robust. This is especially true in the case of water damage.

The typical strategy for most small users who don’t want to invest in LTO drives is a three-pronged solution.

A) Store all camera footage, elements and masters on a RAID array for near-term editing access.

B) Back-up the same items onto at least two copies of raw SATA or SSD hard drives for longer storage.

C) Burn DVD-ROM or BD-ROM copies of edited master files, submasters, project files and elements (music, VO, graphics, etc.).

A properly polished production with audio and video levels that conform to standards is an essential aspect of delivering a professional product. Developing effective mastering and archiving procedures will protect the investment your clients have made in a production. Even better, a reliable archive routine will bring you repeat business, because it’s easy to return to the project in the future.

Originally written for DV magazine/Creative Planet/NewBay Media, LLC

©2012 Oliver Peters

Why 4K

Ever since the launch of RED Digital Cinema, 4K imagery has become an industry buzzword. The concept stems from 35mm film post, where the digital scan of a film frame at 4K is considered full resolution and a 2K scan to be half resolution. In the proper used of the term, 4K only refers to frame dimensions, although it is frequently and incorrectly used as an expression of visual resolution or perceived sharpness. There is no single 4K size, since it varies with how it is used and the related aspect ratio. For example, full aperture film 4K is 4096 x 3112 pixels, while academy aperture 4K is 3656 x 2664. The RED One and EPIC use several different frame sizes. Most displays use the Quad HD standard of 3840 x 2160 (a multiple of 1920 x 1080) while the Digital Cinema Projection standard is 4096 x 2160 for 4K and 2048 x 1080 for 2K. The DCP standard is a “container” specification, which means the 2.40:1 or 1.85:1 film aspects are fit within these dimensions and the difference padded with black pixels.

Thanks to the latest interest in stereo 3D films, 4K-capable projection systems have been installed in many theaters. The same system that can display two full bandwidth 2K signals can also be used to project a single 4K image. Even YouTube offers some 4K content, so larger-than-HD production, post and distribution has quickly gone from the lab to reality. For now though, most distribution is still predominantly 1920 x 1080 HD or a slightly larger 2K film size.

Large sensors

The 4K discussion starts at sensor size. Camera manufacturers have adopted larger sensors to emulate the look of film for characteristics such as resolution, optics and dynamic range. Although different sensors may be of a similar physical dimension, they don’t all use the same number of pixels. A RED EPIC and a Canon 7D use similarly sized sensors, but the resulting pixels are quite different. Three measurements come into play: the actual dimensions, the maximum area of light-receiving pixels (photosites) and the actual output size of recorded frames. One manufacturer might use fewer, but larger photosites, while another might use more pixels of a smaller size that are more densely packed. There is a very loose correlation between actual pixel size, resolution and sensitivity. Larger pixels yield more stops and smaller pixels give you more resolution, but that’s not an absolute. RED has shown with EPIC that it is possible to have both.

The biggest visual attraction to large-sensor cameras appears to be the optical characteristics they offer – namely a shallower depth of field (DoF).  Depth of field is a function of aperture and focal length. Larger sensors don’t inherently create shallow depth of field and out-of-focus backgrounds. Because larger sensors require a different selection of lenses for equivalent focal lengths compared with standard 2/3-inch video cameras, a shallower depth of field is easier to achieve and thus makes these cameras the preferred creative tool. Even if you work with a camera today that doesn’t provide a 4K output, you are still gaining the benefits of this engineering. If your target format is HD, you will get similar results – as it relates to these optical characteristics – regardless of whether you use a RED, an ARRI ALEXA or an HDSLR.

Camera choices

Quite a few large-sensor cameras have entered the market in the past few years. Typically these use a so-called Super 35MM-sized sensor. This means it’s of a dimension comparable to a frame of 3-perf 35MM motion picture film. Some examples are the RED One, RED EPIC, ARRI ALEXA, Sony F65, Sony F35, Sony F3 and Canon 7D among others. That list has just grown to include the brand new Canon EOS C300 and the RED SCARLET-X. Plus, there are other variations, such as the Canon EOS 5D Mark II and EOS 1D X (even bigger sensors) and the Panasonic AF100 (Micro Four Thirds format). Most of these deliver an output of 1920 x 1080, regardless of the sensor. RED, of course, sports up to 5K frame sizes and the ALEXA can also generate a 2880 x 1620 output, when ARRIRAW is used.

This year was the first time that the industry at large has started to take 4K seriously, with new 4K cameras and post solutions. Sony introduced the F65, which incorporates a 20-megapixel 8K sensor. Like other CMOS sensors, the F65 uses a Bayer light filtering pattern, but unlike the other cameras, Sony has deployed more green photosites – one for each pixel in the 4K image. Today, this 8K sensor can yield 4K, 2K and HD images. The F65 will be Sony’s successor to the F35 and become a sought-after tool for TV series and feature film work, challenging RED and ARRI.

November 3rd became a day for competing press events when Canon and RED Digital Cinema both launched their newest offerings. Canon introduced the Cinema EOS line of cameras designed for professional, cinematic work. The first products seem to be straight out of the lineage that stems from Canon’s original XL1 or maybe even the Scoopic 16MM film camera. The launch was complete with a short Bladerunner-esque demo film produced by Stargate Studios along with a new film shot by Vincent Laforet (the photographer who launch the 5D revolution with his short film Reverie)  called Möbius.

The Canon EOS C300 and EOS C300 PL use an 8.3MP CMOS Super 35MM-sized sensor (3840 x 2160 pixels). For now, these only record at 1920 x 1080 (or 1280 x 720 overcranked) using the Canon XF codec. So, while the sensor is a 4K sensor, the resulting images are standard HD. The difference between this and the way Canon’s HDSLRs record is a more advanced downsampling technology, which delivers the full pixel information from the sensor to the recorded frame without line-skipping and excessive aliasing.

RED launched SCARLET-X to a fan base that has been chomping at the bit for years waiting for some version of this product. It’s far from the original concept of SCARLET as a high-end “soccer mom” camera (fixed lens, 2/3” sensor, 3K resolution with a $3,000 price tag). In fact, SCARLET-X is, for all intents and purposes, an “EPIC Lite”. It has a higher price than the original SCARLET concept, but also vastly superior specs and capabilities. Unlike the Canon release, it delivers 4K recorded motion images (plus 5K stills) and features some of the developing EPIC features, like HDRx (high dynamic range imagery).

If you think that 4K is only a high-end game, take a look at JVC. This year JVC has toured a number of prototype 4K cameras based on a proprietary new LSI chip technology that can record a single 3840 x 2160 image or two 1920 x 1080 streams for the left and right eye views of a stereo 3D recording. The GY-HMZ1U is derivative of this technology and uses dual 3.32MP CMOS sensors for stereo 3D and 2D recordings.

Post at 4K

Naturally the “heavy iron” systems from Quantel and Autodesk have been capable of post at 4K sizes for some time; however, 4K is now within the grasp of most desktop editors. Grass Valley EDIUS, Adobe Premiere Pro and Apple Final Cut Pro X all support editing with 4K media and 4K timelines. Premiere Pro even includes native camera raw support for RED’s .r3d format at up to EPIC’s 5K frames. Avid just released its 6.0 version (Media Composer 6, Symphony 6 and NewsCutter 10), which includes native support for RED One and EPIC raw media. For now, edited sequences are still limited to 1920 x 1080 as a maximum size. For as little as $299 for FCP X and RED’s free REDCINE-X (or REDCINE-X PRO) media management and transcoding tool, you, too, can be editing with relative ease on DCP-compliant 4K timelines.

Software is easy, but what about hardware? Both AJA and Blackmagic Design have announced 4K solutions using the KONA 3G or Decklink 4K cards. Each uses four HD-SDI connections to feed four quadrants of a 4K display or projector at up to 4096 x 2160 sizes. At NAB, AJA previewed for the press its upcoming 5K technology, code-named “Riker”. This is a multi-format I/O system in development for SD up to 5K sizes, complete with a high-quality, built-in hardware scaler. According to AJA, it will be capable of handling high-frame-rate 2K stereo 3D images at up to 60Hz per eye and 4K stereo 3D at up to 24/30Hz per eye.

Even if you don’t own such a display, 27″ and 30″ computer monitors, such as an Apple Cinema Display, feature native display resolutions of up to 2560 x 1600 pixels. Sony and Christie both manufacture a number of 4K projection and display solutions. In keeping with its plans to round out a complete 4K ecosystem, RED continues in the development of REDRAY PRO, a 4K player designed specifically for RED media.

Written for DV magazine (NewBay Media, LLC)

©2011 Oliver Peters

Levels – Avid vs. FCP

One of the frequent misconceptions between Avid and Final Cut editors involves video levels. Many argue that FCP does not work within the proper video level standards, which is incorrect. This belief stems from the fact that FCP is based on QuickTime and permits a mixture of consumer and professional codecs. QuickTime Player often changes a file’s appearance as compared with FCP, when it is used to play the file directly. QuickTime Player is trying to optimize the file to look its best on your computer monitor; however, it isn’t actually changing the file itself. Furthermore, two identical clips will appear to be different within each NLE’s interface. Avid clips look flatter and more washed out inside Media Composer. FCP clips will be optimized for the computer display and appear to have more contrast and a different gamma value. This is explained well by Janusz Baranek in this Avid Community thread.

Contrary to popular opinion, both NLEs work within the digital video standards for levels and color space – aka Rec. 601 (SD) and Rec. 709 (HD). Digital video levels are generally expressed using an 8-bit/256-step scale. The nominal black point is mapped to 16 and the white point to 235, which permits level excursions without clipping: 0-16 for shadow detail and 235-255 for highlight recovery. This standard was derived from both camera design and legacy analog NTSC transmission. On most waveform monitors digital 0, analog 7.5 IRE and 16 on this scale are all the same level. Digital 100 (700 millivolts on some scopes), analog 100 IRE and 235 on the scale are also equal. Higher and lower levels will be displayed on a waveform as video above 100/100IRE/235 and below 0/7.5IRE/16.

I want to be clear that this post is not a right/wrong, good/bad approach. It’s simply an exploration in how each editing application treats video levels. This is in an effort to help you see where adjustments can be made if you are encountering problems.

Avid Media Composer/NewsCutter/Symphony

Video captured through Avid’s i/o hardware is mapped to this 16-235 range. Video imported from the computer, like stills and animation can have either a full range of 0-255 (so called “full swing”) or a digital video range of 16-235 (so called “studio swing”) values. Prior to AMA (Avid Media Access), Avid editors would determine these import values in the Media Composer settings, by selecting to import files with RGB values or 601/709 values. You can “cheat” the system by importing digital camera files with an expanded range (spreading the levels to “full swing” of 0-255). Doing so may appear to offer greater color grading latitude, but it introduces two issues. First, all clips have to be color corrected to adjust the levels for proper output values (legal for broadcast). Second, some filters, like the BCC effects, clip rendered files at 16 and 235, thus defeating the original purpose.

It has now become a lot more complex in the file-based world. The files you import are no longer just stills and animation, but also camera and master files from a variety of sources, including other NLEs, like FCP – or HDLSRs, like the Canon 5D. Thanks to AMA in Media Composer 5, this is now automatically taken care of. AMA will properly import files at the right levels based on the format. A digital graphic, like a 0-255 color bar test pattern, is imported at the full range without rescaling the color values from 0-255 to 16-235. A digital video movie from a Canon 5D will be imported with values fitting into the 16-235 range.

Because of the nature of how Avid handles media on the timeline, it is possible to have a full range (0-255) clip on the same timeline next to a studio range clip (16-235) and levels will be correctly scaled and preserved for each. Avid uses absolute values on its internal waveform (accessed in the color correction mode), so you are always able to see where level excursions occur above 235 (digital 100) and below 16 (digital 0).

I would offer one caveat about AMA importing.  Apparently some users have posted threads at the Avid Community Forums indicating some inconsistencies in behavior. In my case, everything is working as expected on multiple systems and from various Canon HDSLR cameras, but others haven’t been so lucky. As they say, “Your mileage may vary.”

Apple Final Cut Pro

If you capture video into FCP using one of the hardware options and a professional codec (uncompressed, ProRes, IMX, DV/DV50/DV100), then the media files will have levels mapped to 601/709 values (16-235). From here, the waters get muddy, because the way in which those levels are handled in the timeline is based on your processing settings. This affects all imported files, as well, including graphics, animation and media files from cameras and other NLEs.

Confusion is compounded by FCP’s internal waveform monitor, which always represents video with a relative 0-100 percent scale. These display numbers do not represent actual video levels in any absolute sense. When you process in YUV, then the full display area of the waveform from top to bottom equals a range of 0-255. The “legal” digital video standard of 16-235 is represented by the area within the 0-100% markings of the scope. However, when you process in RGB, then the portion within the 0-100% marks represents the full 0-255 range. Yes – in an effort to make it simple, Apple has made it very confusing!

When you set the sequence processing to YUV, with “white” as “white”, then all timeline video is mapped to a “studio swing” range of 16-235. On the scope 0% = 16 and 100% = 235. If you import a “full swing” color bar pattern (0-255), the values will be rescaled by the sequence processing setting to fall into the 16-235 range.

When you set the sequence processing to YUV, with “white” as “superwhite”, you’ve extended the upper end of the range, so that the 16-235 scale now becomes 16-255. The 0-255 color bar pattern is now effectively rescaled to 16-255; however, so is any video as well. Digital video that used to peak at 100% will now peak at 120%.

The YUV processing issues are also affected by the 8-bit, versus “high-precision” rendering options. When you elect to process all video as 8-bit, excursions above 100% and below 0% caused by color correction will be clipped. If you change to “high-precision YUV”, then these excursions are preserved, because they fall within the 0-16 and 235-255 regions. Unfortunately, certain effects and transition filters will still clip at 0% and 100% after rendering.

One way to fully protect “full swing” 0-255 levels is to work in RGB processing. A 0-255 color bar pattern will be correctly displayed, but unfortunately all video is spread to the full range, as well. This would mean that all clips would have to be color corrected to adjust for proper video levels. The only way that I’ve found for FCP to display both a 0-255 and a 16-235 clip on the same timeline and maintain correct levels is to apply a color correction filter to adjust the levels on one of these clips.

For general purposes, the best way to work with FCP is to use the ProRes family of codecs and set your sequence settings for YUV processing, white=white and high-precision rendering. This offers the most practical way of working. The only caveat to this is that any “full swing” file will be rescaled so that levels fall into the 0%-100% (16-235) “studio swing” range. If you need to preserve the full range, then FCP’s color correction filters will allow you to expand the range. The levels may appear to clip as you make the adjustment, but the rendered result will be full range.

Real world examples

I’ve done some quick examples to show how these level issues manifest themselves in actual practice. It’s important to understand that for the most part, the same clip would appear the same in either Media Composer of Final Cut as viewed on a broadcast monitor through output hardware. It will also look the same (more or less) when displayed to a computer screen using each app’s full screen preview function.

The following screen grabs from my tests include a 0-255 color bar chart (TIFF original) and a frame from an H.264 Canon 5D clip. The movie frame represents a good spread from shadow to highlights. I imported the files into both Avid Media Composer 5 (via AMA) and Apple Final Cut Pro 7. The FCP clips were rendered and exported and then brought into MC5 for comparison. The reason to do this last step was so that I could check these on a reasonably trustworthy internal scope, which displayed an 8-bit range in absolute values. It is not meant to be a direct comparison of how the video looks in the UI.

Click any image to enlarge.

Imported into Final Cut. ProResHQ with YUV processing. White as white. Note that the peak white of both images is 100%.

Imported into Final Cut. ProResHQ with YUV processing. White as SuperWhite. Note that peak white of both images exceeds 100%.

Imported into Final Cut. ProRes4444 with RGB processing. Note the boundary limits at 0% and 100%.

Imported into Media Composer 5 using AMA. Note that the color bars are a 0-255 range, while the Canon clip is 16-235.

FCP7 YUV export, imported into MC5 via AMA. Note that the color bar pattern has been rescaled to 16-235 and is no longer full range.

FCP7 YUV export with SuperWhite values, imported into MC5 via AMA. Note that the color bar pattern has been rescaled to 16-255 and is no longer full range. It has a higher top-end, but black values are incorrect. This also alters the scaling values of the levels for the Canon clip. Color correction filters would have to be applied in FCP for a “sort of correct” level match between the bars and the Canon clip.

FCP7 RGB export, imported into MC5 via AMA. Note that the color bar pattern exhibits the full 0-255 range. The Canon clip has also been rescaled to 0-255. Color correction filters would have to be applied in FCP to the Canon clip to bring it back into the correct relative range.

I have revisited the YUV settings in FCP7. This is a ProResHQ sequence rendered with high-precision processing. I have applied a color corrector to the color bars, expanded the range and rendered. Note the regions above 100% and below 0%.

FCP7 YUV export (color correction filter applied to the color bars), imported into MC5 via AMA. Note that the color bar pattern spreads from 0-255, while the Canon clip is still within the standard 16-235 range.

©2010 Oliver Peters

Codec Smackdown

Modern digital acquisition, post and distribution wouldn’t be possible without data rate reduction, AKA compression. People like to disparage compression, but I dare say that few folks – including most post production professionals – have actually seen much uncompressed content. In fact, by the time you see a television program or a digitally-projected movie it has passed through at least three, four or more different compression algorithms – i.e. codecs.

Avid Media Composer and Apple Final Cut Pro dominate the editing landscape, so the most popular high-end HD codecs are the Avid DNxHD and Apple ProRes 422 codec families. Each offers several codecs at differing levels of compression, which are often used for broadcast mastering and delivery. Apple and Avid, along with most other NLE manufacturers, also natively support other camera codecs, such as those from Sony (XDCAM-HD, HD422, EX) and Panasonic (DVCPRO HD, AVC-Intra). Even these camera codecs are being used for intermediate post. I frequently use DVCPRO HD for FCP jobs and I recently received an edited segment as a QuickTime movie encoded with the Sony EX codec. It’s not a question of whether compression is good or bad, but rather, which codec gives you the best results.

Click on the above images to see an enlarged view. (Images from Olympus camera, prior to NLE roundtrip. Resized from original.)

I decided to test some of these codecs to see the results. I started with two stills taken with my Olympus C4000Z – a 4MP point-and-shoot digital camera. These images were originally captured in-camera as 2288-pixel-wide JPEGs in the best setting and then – for this test – converted to 1920×1080 TIFFs in Photoshop. My reason for doing this instead of using captured video, was to get the best starting point. Digital video cameras often exhibit sensor noise and the footage may not have been captured under optimum lighting conditions, which can tend to skew the results. The two images I chose are of the Donnington Grove Country Club and Hotel near Newbury, England – taken on a nice, sunny day. They had good dynamic range and the size reduction in Photoshop added the advantages of oversampling – thus, very clean video images.

I tested various codecs in both Avid Media Composer 4.0.5 and Apple Final Cut Pro 7. Step one was to import the images into each NLE. In Avid, the conversion occurs during the import stage, so I set my import levels to RGB (for computer files) and imported the stills numerous times in these codecs: 1:1 MXF (uncompressed), DNxHD145, DNxHD220, DNxHD220x, XDCAM-EX 35Mbps and XDCAM-HD422 50Mbps. In Final Cut Pro, the conversion occurs when files are placed on the timeline and rendered to the codec setting of that timeline. I imported the two stills and placed and rendered them onto timelines using these codecs: Apple 8-bit (uncompressed), ProRes LT, ProRes, ProRes HQ, DVCPRO HD and XDCAM-EX 35Mbps. These files were then exported again as uncompressed TIFFs for comparison in Photoshop. For Avid, this means exporting the files with RGB levels (for computer files) and for FCP, using the QuickTime Conversion – Still Image option (set to TIFF).

Note that in Final Cut Pro you have the option of controlling the import gamma settings of stills and animation files. Depending on the selection (source, 1.8, 2.20, 2.22) you choose, your video in and back out of Final Cut may or may not be identical to the original. In this case, choosing “source” gamma matched the Avid roundtrip, whereas using a gamma setting of 2.2 resulted in a darker image exported from FCP.

Click on the above images to see an enlarged view.

You’ll notice that in addition to various compressed codecs, I also used an uncompressed setting. The reason is that even “uncompressed” is a media codec. Furthermore, to be accurate, compression comparisons need to be done against the uncompressed video image, not the original computer still or graphic. There are always going to be some changes when a computer file is brought into the video domain, so you can’t fairly judge a compressed video file against the original photo. Had I been comparing video captured through a hardware card, then obviously I would only have uncompressed video files as my cleanest reference images.

I lined up the exported TIFFs as Photoshop layers and generated comparisons by setting the layer mode to “difference”. This generates a composite image based on any pixel value that is different between the two layers. These difference images were generated by matching a compressed layer against the corresponding Avid or FCP uncompressed video layer. In other words, I’m trying to show how much data is lost when you use a given compressed codec versus the uncompressed video image. Most compression methods disproportionately affect the image in the shadow areas. When you look at a histogram displaying these difference results, you only see levels in the darkest portion of an 8-bit scale. On a 0-255 range of levels, the histogram will be flat down to about 20 or 30 and then slope up quickly to a spike at close to 0.

This tells you that the largest difference is in the darkest areas. The maximum compression artifacts are visible in this range. The higher quality codecs (least compressed), exhibit a smaller histogram range that is closer to 0. The more highly-compressed codecs have a fatter range. This fact largely explains why – when you color grade highly compressed camera images – compression artifacts become quite visible if you raise black or gamma levels.

The resulting difference images were then adjusted to show artifacts clearly in these posted images. By adjusted, I mean changing the levels range by dropping the input white point from 255 to 40 and the output black point from 0 to 20. This is mainly for illustration and I want to reiterate that the normal composite images DO NOT look as bad as my adjusted images would imply. In fact, if you looked at the uncorrected images on a computer screen without benefit of a histogram display, you might think there was nothing there. I merely stretched the available dynamic range for demonstration purposes.

Of these various codecs, the Apple DVCPRO HD codec shows some extreme difference results. That’s because it’s the only one of these codecs that uses horizontal raster scaling. Not only is the data compressed, but the image is horizontally squeezed. In this roundtrip, the image has gone from 1920-pixels-wide (TIFF) to 1280 (DVCPRO HD) back to 1920 (exported TIFF). The effects of this clearly show in the difference image.

Click on the above images to see an enlarged view.

There are a couple of other things you may notice, such as level differences between the Avid and Apple images and between each of these and the originals. As I said before, there will always be some differences in this sort of conversion. Plus, Apple and Avid do not handle color space, level and gamma mapping in the same way, so a round trip through each application will yield slightly different results. Generally, if 2.2 gamma is selected for imported stills, the Apple FCP image will have a bit more contrast and somewhat darker shadow areas when compared to Avid on a computer screen – even when proper RGB versus Rec. 709 settings are maintained for Avid. This is mainly a result of the various QuickTime and other conversions going on.

If I were to capture video with Avid DX hardware on the Media Composer and AJA, Matrox or Blackmagic hardware on FCP – and compared these images on a video monitor and with scopes – there would likely be no such visible difference. When I used “source” gamma in FCP, then the two matched each other. Likewise, when you review the difference images below, 2.2 gamma in this case resulted in a fault difference composite between the FCP uncompressed and the original photo. The “source” gamma version more closely resembles the Avid result and is the right setting for these images.

The take-away from these tests should be that the most important comparisons are those that are relative, i.e. “within species”. In other words, how does ProRes LT compare to ProRes HQ or how does DNxHD 145 compare to DNxHD 220x? Not, how an Avid export compares with a Final Cut export. A valid inter-NLE comparison, however, is whether Avid’s DNxHD220x shows more or less compression artifacts than Apple’s ProRes HQ.

I think these results are pretty obvious: Higher-data-rate codecs (less compression) like Apple ProRes HQ or Avid DNxHD 220x yield superb results. Lower-date-rate codecs (more compression) like XDCAM-EX yield results that aren’t as good. I hope that arming you with some visible evidence of these comparisons, will help you better decide what post trade-off to use in the future.

(In case you’re wondering, I do highly recommend the Donnington Grove for a relaxing vacation in the English countryside. Cheers!)

Click on these images to see an enlarged view.

©2010 Oliver Peters

Mixing formats in the edit

The sheer mix and volume of formats to deal with today can be mind-boggling. Videotape player/recorders – formerly a common denominator – are a vanishing breed. Post facilities still own and use VTRs, but operations at the local market level, especially in broadcast, are becoming increasingly tapeless. Clearly, once the current crop of installed VTRs become a maintenance headache or are no longer an important cog in the operation, they won’t be replaced with another shiny new mechanical videotape transport from Sony or Panasonic.

It all starts with the camera, so the driving influence is the move to tapeless acquisition – P2, XDCAM-HD, GFcam, AVC-HD and so on. On the bright side, it means that the integration of another format will cost no more than the purchase of an inexpensive reader, rather than a new VTR to support that format. Unfortunately this will also mean a proliferation of new formats for the editor to deal with.

The term format should be clarified with tapeless media, like P2. First, there is the codec used for the actual audio and video content (essence). That essence is defined by the compression method (like DVCPRO HD or AVC-Intra), frame size (SD or HD), pixel aspect ratio and frame rate. The essence is encapsulated into a file wrapper (MXF), which holds the essence and metadata (information about the essence). Lastly, in the P2 example, the files are written to a physical transport medium (the P2 card itself), using a specific folder and file hierarchy. Maintaining this folder structure is critical in order that an NLE can natively recognize the media, once it’s copied from the card to a hard drive.

Nonlinear editing systems have been built around a specific media structure. Avid Media Composer uses OMF and MXF. Apple Final Cut Pro is based on QuickTime. In theory, each can ingest a wide range of tapeless file formats, but the truth is that they only work well with a much narrower range of optimized media. For instance, DVCPRO HD is handled well by most NLEs, but H.264 is not. You can toss a mix of formats onto a common timeline, but the system is internally operating with specific settings (codec, frame size and frame rate) for that timeline.

These settings are established when you first create a new project or a new sequence, depending on the application. Any media on the timeline that deviates from these settings must either be scaled and decompressed on-the-fly by the real-time effects engine of the application – or must be rendered – in order to see full-quality playback.  Most systems are optimized for NTSC, PAL, 720p and 1080i frame sizes. Even Final Cut Pro – touted as resolution independent – works best at these sizes and effectively tops out at 2K film sizes. All the desktop NLEs freely allow you to mix SD and HD content on a timeline, but the rub has been a mix of differing frame rates. FCP could do it, but Media Composer wouldn’t. That barrier disappeared with Avid’s introduction of the Mix & Match feature in the Media Composer 4.0 software. Now, if you edit a native 23.98p clip into a 29.97i timeline, all of the leading editing applications will add a pulldown cadence to the 23.98p clip for proper 29.97i playback.

When editing a project that has a mix of SD and HD sources and formats, it is best to select a timeline or project setting that matches the predominant format. For instance, if 75% of your media was shot using a Panasonic VariCam at 720p/59.94, then you’d want to use a matching timeline preset, so that the 720p footage wouldn’t require any rendering,  except for effects. In this example, if the other 25% was NTSC legacy footage from Betacam-SP, you’d need to have a system equipped with a capture card capable of ingesting analog footage. The Beta-SP footage could be upconverted to HD during the capture using the hardware conversion power of a capture card. Alternately,  it could be captured as standard definition video, edited onto the timeline and then scaled to fill the HD frame. Betacam-SP clips captured as standard definition video would ultimately be rendered to match the 720p/59.94 settings of the timeline.

Until recently, Avid systems transcoded incoming media into an Avid codec wrapped as an MXF file. This creates media files that are optimized for the best performance. Final Cut would let you drag and drop any QuickTime file into the FCP browser without a transcode, but non-QuickTime files had to be converted or rewrapped as QuickTime MOV files. These frontrunners were upstaged by applications like Grass Valley EDIUS and Sony Vegas Pro, which have been able to accept a much wider range of media types in their original form. The trend now is to handle native camera codecs without any conversion. Apple added the Log and Transfer module to Final Cut and Avid added its Avid Media Access (AMA). Both are plug-in architectures designed for native camera media and form a foundation for the use of these files inside each NLE.

Final Cut’s Log and Transfer is recommended for importing P2, RED, XDCAM and other media, but it still doesn’t provide direct editing support. Even natively-supported codecs, like REDCODE and AVC-Intra must first be wrapped as QuickTime files. When clips are ingested via Log and transfer, the files are copied to a target media drive and in the process rewrapped as QuickTime MOV file containers. It’s Apple’s position that this intermediate transcode step is a safer way to handle camera media without the potential of unrecoverable file corruption that can occur if you work directly with the original media.

If you want true native support – meaning the ability to mount the hard drive or card containing your raw media and start editing at full resolution – then the Avid Media Composer family, Grass Valley EDIUS and Adobe Premiere Pro provide the broadcaster with the strongest desktop solutions. All three recognize the file structure of certain camera formats (like P2), natively read the camera codec and let you use the media as an edit source without the need to transcode or copy the file first. These APIs are evolving and are dependent on proper media drivers written by the camera manufacturers. Not all applications handle every format equally well, so select a system that’s appropriate for you. For example, P2 using the DVCPRO HD or AVC-Intra codec is becoming widely supported, but Panasonic’s AVCCAM has less support. Sony hit snags with XDCAM-EX support for Final Cut Pro when Apple upgraded the Mac OS to 10.6 (“Snow Leopard”). Fortunately these issues are short-lived. In the future it will be easier than ever to mix taped and tapeless camera media of nearly any format with little negative impact.

Written for NewBay Media and TV Technology magazine

©2009 Oliver Peters