Posting RED

blg_redalert_grade.jpg

In two short years Oakley founder Jim Jannard has piloted RED Digital Camera from an unknown upstart to an industry force. Unofficially buyers have reserved over 3,500 cameras, of which several hundred are now out in the wild. If everything is on track, I estimate that by year’s end RED Digital Camera will have shipped around 3,000 of its innovative RED One cameras. Many in the industry scoffed at first – especially those with a vested interest in competing high-end cameras like the Sony F900 and F23, the Arri D-20 or the Grass Valley Viper; but RED One is proving itself out in the field, providing VFX shots on Wanted and Jumper, as well as 100% of the acquisition on Steven Soderbergh’s Guerilla, The Argentine and The Informant. In addition, the directing team that brought you Crank is also shooting Game as an all-RED feature film production. Not to mention many lower-profile productions ranging from commercials and music videos to small indie film projects.

 

Camera Raw

 

In creating RED One, Jannard and company decided to apply the concepts of digital still photographic technology to a moving image camera. One example of this is in the development of RED’s Mysterium sensor – a single CMOS 4520 x 2540 pixel chip, which covers the equivalent image area of a 3-perf Super 35mm film frame. RED One captures images as camera raw data using a Bayer-pattern filter. This pick-up pattern was first developed by Dr. Bryce E. Bayer of Eastman Kodak and is used in many digital still cameras. A Bayer filter pattern uses small color filters that are arranged as 50% green, 25% red and 25% blue in a repeating GRGB pattern. From this larger sensor area, RED records 4K, 3K and 2K slices with either a 2:1 or 16:9 aspect ratio.

 

Early camera builds did not yet have the full feature set turned on and only recorded 4K 2:1 images (4096 x 2048 pixels) at 24fps. Subsequent software builds have enabled more features and those early cameras are being retrofitted by RED Digital Camera. (March is the end of the official beta period.) The Bayer-pattern image is then stored to flash cards or hard drives with the application of a visually-lossless, variable bit rate wavelet code, named Redcode. To give you an idea of the amount of data reduction made possible by Redcode, a 665-frame ( about :28 ) Redcode 4K sample file I used is 738MB. That’s compared to 32GB if this same clip were an uncompressed 4K media file.

 

A lot of forum posters have debated whether a Bayer-pattern sensor results in true 4K resolution, in contrast to a camera with three separate, smaller chips. I’m not a director of photography, so I won’t dwell on this, except to say that the images look very impressive. They are certainly higher resolution than HD or film scans at 2K, so that’s good enough for me. These 4K images look great when downsampled as 2K, HD and SD thanks to the benefits of oversampling.

 

After The Shoot

 

The workflow surrounding the post of RED’s files may seem confusing to many who are accustomed to run-and-gun video productions. In reality, it’s a lot like film post or other file-based formats, such as P2. Think of post with RED files like bringing camera raw still photos into Apple Aperture or Adobe Lightroom. All image manipulation at the camera, except the lens opening, frame rate, motion blur and focus, are saved only as metadata. In Aperture, when you alter white balance and other colorimetry values, you are only changing the metadata until you actually export the altered photo to a standard image format. The original raw file remains unaltered, like a film negative. Likewise in RED’s tools, all metadata dialed in by the camera operator or DP may be altered until a final format is exported, “baking” those settings into the new file. Since you have a high degree of control after the fact, workflows for RED mimic many of the methods familiar to folks who shoot film, including dailies and DI.

 

There is no single workflow that is best for everyone. It varies depending on your computing platform, NLE of choice and deliverables. The RED user community and post facilities getting ready to service RED customers are quickly testing and developing best practices that will change this from a huge science project to a viable business model. Think of this approach as the digital lab of the future. Some of the folks gearing up include Offhollywood Studios (New York), Cineworks (Miami), Plaster City Post (Los Angeles) and Alphadogs (Burbank).

 

The Toolset

 

There are currently three applications that read and manipulate native Redcode (.R3D) files: RedAlert, RedCine and Assimilate SCRATCH. RedAlert and RedCine are free software tools from RED Digital Camera that can be used to open, grade, scale, crop and export Redcode files into various editable formats, like QuickTime media and image sequences. RedCine works on both Mac Intel and Windows computers, but RedAlert is only available for Intel-based Macs. If your goal is to establish a RED-oriented digital lab or service bureau, then SCRATCH – a Windows-only DI application – becomes the best solution for efficiently generating dailies and/or final output at up to 4K resolution. Assimilate paired with hardware vendor MaxVision to offer the SCRATCH RED PrePost solution. This is a self-contained unit, incorporating SCRATCH software modules specifically optimized for Red workflow and built into a small, ruggedized configuration called The Cube.

 

RedAlert is especially useful in generating QuickTime reference movies in a near-instantaneous process. An original .R3D file opened in RedAlert displays an image based on the metadata assigned during shooting, including ISO rating, exposure, white balance and more. If you accept these values, one command updates the reference movies. Change the settings and overwrite the QuickTimes movies to save a different look. Redcode is designed to generate a 2K, 1K and .5K reference movie from the same 4K camera file. (The camera itself can also generate a 4K reference movie, but that currently cannot be used in post.)

 

The file size of the reference movies is only about 8KB, as compared with the original .R3D camera file that may be hundreds of Megabytes or more in size. If you start at 4096 x 2048, then the three reference movies are 2048 x 1024, 1024 x 512 and 512 x 256, respectively. As long as the original .R3D file and reference movies stay together and you install the Redcode QuickTime component on your Mac Pro or MacBook Pro computer, any QuickTime-compliant application can play movies. This includes Apple Final Cut Pro, QuickTime Player, Adobe After Effects and even Adobe Premiere Pro CS3. Some RedAlert adjustments, such as curves, are not yet programmed into the Redcode QuickTime component and won’t show up in the QuickTime reference file.

 

The Nitty Gritty

 

You start to realize how much latitude there is in the RED One image during color grading. There is almost as much control as working with film negative; however, there’s more going on than just grading. Since you are starting with a Bayer-pattern image, the most important thing is to properly demosaic the pattern and restore the proper RGB color information. There are two ways that algorithms compute this: less accurate, but faster and more accurate, but with longer rendering. The former is used for the QuickTime reference files, which is why they are generated instantly and can scale and demosaic the image on-the-fly. According to Graeme Nattress, part of RED Digital Camera’s codec development team, “Half of the green pixels are omitted and the nearest set of RGB pixels are assumed to be co-sited (but they’re not) to produce an RGB image with half the resolution of the original. This is very good and quick and as it’s a downsample without a downsampling filter applied, very sharp. But on the downside, there can be fringes on colors and some aliases. It’s a good compromise for quick playback.”

 

Processes that involve rendering during the export from either RedAlert, RedCine or SCRATCH use the full resolution and downsample filtering, resulting in the best possible quality. For example, you can directly export a rendered QuickTime file with the Apple ProRes codec from RedAlert or you can export a reference file from RedAlert and then use Apple Compressor to render that into a ProRes file. Theoretically the image quality will be better exporting directly from RedAlert than it would be going through the extra step of the reference movie. In actual practice, during my tests, I saw almost no difference when viewed side-by-side on an HD screen. Granted, this was only a visual assessment – I didn’t do any sort of difference matting. Nevertheless, I would be more than comfortable with any of these faster methods if the deliverable is for television and not theater screens. For more critical situations, the best results were when I rendered 4K TIFF image sequences. I then “stitched” these together in QuickTime Player Pro and converted and scaled them to uncompressed 2K and/or HD files.

 

Part of what you are buying in a camera with traditional video technology, like a Sony F900 or a Panasonic VariCam, is circuitry to sharpen and artificially enhance the image. RED Digital Camera engineered a high-resolution camera designed to be free of artifacts, like ringing and aliasing. The Mysterium sensor employs optical low pass filtering (OLPF) much like a Canon EOS or Nikon D3 digital still camera. Some folks might incorrectly interpret its images as soft, so RedAlert and RedCine include tools and settings to adjust noise, detail and OLPF compensation, if you feel sharpening is needed.

 

RedCine is the other key component of the RED post production toolset. This application was co-developed with Assimilate and looks a lot like SCRATCH “lite”. It is a free desktop DI tool available for Mac Intel and PC, so if you are a Windows-based editor, you’ll spend a lot of your time with this application. It offers similar image control as RedAlert and can export to various file types, with the exception of QuickTime reference movies. RedCine is not intended as a free NLE, but as an application to pull and assemble selected takes and perform basic, primary color grading. You can string together shots, make head and tail trims, color grade in context and then export the result as a single file or a series of files.

 

Rendering

 

Since every export option other than QuickTime reference files requires rendering, the impact in time is definitely a consideration when you’ve shot a lot of footage. I ran various render tests on different computers ranging from my laptop to a quad-core Windows machine. The bulk of my testing was on a one-year-old Mac Pro with two 3Ghz dual-core Xeons, 8GB RAM and an ATI Radeon X1900XT display card. This is a typical machine for many Final Cut Pro, After Effects and Avid Media Composer users. Sharpening settings didn’t present much of an impact in rendering time with RedAlert. My 665-frame ( :28 ) test clip rendered as a TIFF or DPX sequence at the 4K size in about 17 minutes and under 10 minutes for a 2K image sequence. RedCine is more dependent on GPU and processes differently than RedAlert. 4K TIFF sequences without sharpening rendered in 16 minutes and 20 minutes with sharpening. Rendering a 1920 x 1080 ProRes file took between 13 and 20 minutes, depending on settings.

 

When I rendered an image sequence, stitched it together and exported it as a QuickTime media file, the extra rendering added about two minutes for this second export. Finally, using a Compressor droplet to turn a 2048 x 1024 QuickTime reference file into a 1920 x 1080 ProRes took only three minutes to render. I think this yielded excellent image quality on most clips, so for general TV deliverables where turnaround time is important, it might be the best rendering path to follow. The good news is that the toolset is by no means done. RED just released the RedLine utility that allows command line batch processing in the OS X Terminal. In addition, many of these rendering processes are conducive to the installation of cheap render farms for faster throughput.

 

Editing RED with Final Cut Pro

 

RED Digital Cinema recommends Apple Final Cut Pro 6.0.2 if you intend to cut a project today. With the Redcode codec installed, you can import the QuickTime reference files and immediately start editing, although Apple isn’t actually promoting this workflow. Both the 1K and .5K reference movies played smoothly enough on the four-core Mac Pro and even a MacBook Pro, but the 2K files dropped frames.  To get real-time performance, allow the sequence settings to be changed to match the clip when the first edit is made, work in the “unlimited” RT mode and set the playback rate to “full”. Adjust the video quality as needed, but “dynamic” worked fine for my tests.

 

This means that you can do creative editing using a laptop, even though you’re linked to 4K media; however, if you intend to do a lot of RED editing, then an octo-core Mac Pro is in your future. There you should see relatively smooth playback with 2K reference movies and, of course, computing performance is only getting better. It is definitely possible on a fast machine to place the bigger proxies (2048 x 1024) on a ProRes HD timeline, render and master to tape – all without leaving Final Cut Pro.

 

RED is promoting the native workflow because it offers the advantage of never locking in your color grade. You can always go back to the camera raw files and change the look, but this is also where some of the holes exist in the current workflow. The only full-featured, finishing application that can currently read and edit native .R3D files is SCRATCH. If you edit on Final Cut for film delivery, then the next step is to use XML or an EDL to get your timeline data to a SCRATCH system. SCRATCH will link to the camera raw files and conform the clips to the sequence. An alternative is to use RedCine, which uses XML, although a different flavor than Final Cut. The RED user community approaches this camera much like other open source projects, such as Linux. In that spirit, one user created RedTrip to translate FCP XML to RedCine XML. This gives you the ability to pull the trimmed shots with handles into RedCine, grade, scale/crop and render new media. These new files can be relinked in FCP for output to tape.

 

Bear in mind this is still a work-in-progress and there’s a lot more to come. Apple showed a technology demo at IBC using the FCP Log and Transfer module to ingest and quickly transcode RED files to ProRes or other codecs. This should be available once RED and Apple have each completed their development. The main incentive here is to is maintain frame-accurate metadata for reel and timecode information, which is potentially lost going through an application such as Compressor.

 

Other Editing Solutions

 

Editors using other NLEs, such as Avid, Autodesk or Quantel systems, must take a different approach, since they can’t work natively with the QuickTime reference movies or .R3D files. Avid editors might prefer to work in the DNxHD codec, which offers several resolutions from draft to mastering quality. Avid products don’t yet read embedded timecode from QuickTime files during the import, so if you require source timecode, you’ll want another small user-created application called MetaCheater. This reads the metadata from the QuickTime files and creates an Avid Log Exchange (ALE) file. This shot log can be imported into an Avid bin and the batch imported DNxHD files will retain the camera-generated reel and timecode data from the ALE information. An EDL exported from the edited sequence matches back to the reel and timecode information of the .R3D files and can be used for a DI finish with SCRATCH.

 

Editors finishing on an Avid NLE often “decompose” a sequence before recapturing higher resolution media. This step creates new, shorter master clips representing only the portions used in the final sequence, so you don’t recapture any unnecessary media. Unfortunately, imported files cannot be properly decomposed and re-imported as a shorter file in current versions of the Avid products. In this case, selected RED clips would have to be re-imported at their original duration so that they show up at the right point in the sequence. Nevertheless, Avid editors can use RedAlert and/or RedCine to create draft resolution files at DNxHD 36 for creative cutting and later generate color-corrected clips for finishing at a higher quality setting, like DNxHD 175. Avid is developing its own XML workflow involving the FilmScribe application, which should eliminate many of these bottlenecks.

 

One approach that I favor, but is probably an anathema to the diehard proponents of the native workflow, is to lay off your .R3D clips to tape. With SCRATCH, RedAlert or RedCine, it is possible to do primary grading, being careful not to crush shadows or clip highlights. Export these clips as uncompressed 1920 x 1080 QuickTime files (or directly from SCRATCH in real-time) and output them as dailies to HDCAM-SR in 4:4:4 log color space. These tapes now become your digital submasters from which everything else is converted, conformed, graded and ultimately output to every deliverable format, including film. This lacks some of the metadata interchange available with file-based workflows, but it’s been used quite successfully for various major film releases shot on 35mm negative, so there’s no reason it won’t also work with RED files. That is until true 4K display technology comes along – another item on Jannard’s To Do List.

 

Native Apple Final Cut Pro and Assimilate SCRATCH workflows will be on display in the RED Digital Cinema booth at NAB. Yet even now, all of RED’s software, including the Redcode codec, is still in beta development. RedAlert and RedCine were updated a few times with major improvements during my research. According to Ted Schilowitz, RED’s Leader of the Rebellion, “It’s a constantly moving target. We continue to re-work, re-engineer and re-think things on a regular basis to improve the camera, the workflow and the overall user experience.” RED Digital Camera plans to announce sometime after NAB how they plan to open up their software so that other companies can have access to the files. Judging by the interest generated thus far in RED One’s short life, there’s bound to be a healthy ecosystem developed for years to come around this innovative product.

 

Written by Oliver Peters for Videography magazine (NewBay Media, LLC)