Tracking is the key to believable visual effects and one of the leaders in this technology is Imagineer Systems. Mocha, a 2D standalone tracker, is one of their better known products. If you purchased one of Adobe’s Creative Suite 4 bundles that included After Effects, then you already own mocha for After Effects, whether you know it or not. This year Imagineer released mocha for Final Cut Pro, bringing the same tracking power to FCP editors.
Many software packages already include tracking technology. Avid Media Composer, Apple Motion and Adobe After Effects all include built-in trackers. So, why buy another? All of these trackers are “point” trackers. You isolate one or more obvious targets on an image and position a tracker over it. Usually this is an area of a few pixels with a high contrast difference, like a clear logo or sign in the frame that moves with the object you are tracking. As the object moves through the frame, the tracker hopefully stays “locked” onto this target area while the software does an analysis pass of the video clip. If the tracked point moves off screen or out of focus, most point trackers will have trouble following and new tracker targets have to be picked where the first track leaves off. Often tracks have to be manually adjusted.
The information generated by tracking results in keyframe data that can be applied to stabilize shots, corner pin objects or to moving masks for filters and other effects. More accurate tracking is achieved by adding more point trackers. Corner pinning – used to replace one logo with another – generally requires four trackers. Mocha differs from these other tracking systems because it is a planar tracker. Instead of tracking isolated points, you draw a spline shape around an object and mocha will analyze all the pixels within that shape. This results in a more accurate track, even when part of the tracked area moves off screen, goes out of focus or when a foreground object briefly cuts across part of the tracked area.
Mocha for After Effects and for Final Cut are not true plug-ins, but are separate applications. The difference between them is the export module. To work in either version, simply import the clip to be tracked. At this point you are in mocha, which is basically the same as the full-blown, standalone version. Once you have completed the track, made adjustments and are satisfied with the results, you are ready to export the data. This last stage is where the plug-in versions differ. Both generate either basic motion information (translate, scale, rotate) or distort (corner pinning) values. The After Effects version generates text files that can be copied-and-pasted into AE as keyframes. The Final Cut version exports XML files that can be imported into FCP.
The data can also be inverted during the export. For example, if you are using the tracking data to stabilize an image, you’ll want to invert the data, so that the image is stable and stationary, but the frame around it appears to be moving. If you intend to use this tracking data in Apple Motion, then you first have to import the XML into FCP and “Send to a Motion project” from FCP.
When you import the XML file, the clip is imported along with motion tab data applied to it. Depending on which data you exported, this will either consist of scaling/position/rotation keyframes or distort keyframes for each corner. The keyframe data can be copied-and-pasted (paste attributes) onto a logo or mask. To place a new logo into a shot, cut the clip onto V1. Highlight the clip and copy. Cut the logo onto V2 and paste attributes (which came from the V1 clip). Now remove attributes from the clip on V1. If you used corner pinning, you can still adjust scale/position/rotation of the V2 logo for a better fit – or – if you applied basic motion, then you can still adjust the corner positions (distort).
I recently used both versions of mocha (AE and FCP) on a commercial for several shot repairs. The clips were of a large stadium video screen and above the screen was an LED sign with the words “Kansas City”. Unfortunately on the day of the shoot, the panel containing the middle “S” was not completely working. The production couldn’t be held up, so the decision was made to fix it in post. When I first saw the shots, I thought the fix was going to be a piece of cake inside FCP. Simply duplicate the clip so the same clip is on V1 and V2. Offset and crop the V2 clip so that the second “S” overlapped and hid the first “S” and all would be fine. Both clips would be moving in sync and the two letters would match perfectly. So much for theory! The shots were Steadicam shots with a right-to-left movement throughout the shot. These were also low angle shots resulting in enough optical difference between the positions of the two letters to make a simple fix impossible.
The next approach was tracking. Let me point out that mocha is a 2.5D planar tracker. In the real world it does a good job with objects that stay on the same plane relative to the lens, including with perspective changes. You won’t be immune from problems created by arcing or trucking 3D camera moves. All of the nice demos and tutorials are often done using moving subjects that are within static camera shots. Rarely are both moving.
Another consideration is film and 3:2 pulldown. These spots were shot on 35mm film and transferred to Digital Betacam. As with most NTSC footage, I had to contend with the whole-frame/split-field-frame cadence of film transfers. Although mocha can track across these split-field frames, the resulting data doesn’t necessarily composite well back in Motion or Final Cut. My solution was to first remove the pulldown in After Effects – one of the best tools for that. Then simply render out a 24fps progressive-frame file.
( Note: Most video apps express the video-friendly version of 24p as 23.98. That’s a rounded-up value. After Effects uses the 3-digit version of 23.976. Most apps make no distinction and use the same math, but in the case of AE, there is a difference between 23.976 and 23.98. So, use 23.976 in AE and you’ll still be OK as 23.98 back in FCP. )
Step one done. My clips were now progressive frame media at 24fps (23.976). Now for the fun. I pulled a single frame into Photoshop, fixed the “S” and cut out the sign to form the new foreground element. I would track this onto the clip to replace the original sign. In order to get the most rock solid lock, I ended up using a whole slew of tracking solutions, including Motion, After Effects and both versions of mocha. I exported both motion data and corner pinning data to try it either way.
In all cases, I found the track from mocha to be more precise than After Effects or Motion, but that didn’t always translate to the best compositing results. Corner pinning data sometimes results in information that is too precise and the object appears to jitter in a composite, because of the minute changes in each corner at each of the many keyframes. On the other hand, motion data results in an objects that appear to float too much and don’t look as locked as you’d like. As I said, mocha provided a great track, but this doesn’t mean that the keyframe values are precisely interpreted in the host compositor.
The last shot gave me the most fits, even though it looked the easiest. The sign was large in the frame and tracking points stayed in frame. Any of the trackers should have done well, but they didn’t. As the camera moved, someone in the foreground crowd was clapping and his hand intersected one of the corners for a few frames. I couldn’t get a good track with any of the point trackers. In this situation mocha shined. The analysis ignored the hand, since the larger spline area covered the entire shape of the sign. Instead of corner pinning, I used basic motion data and composited the shot in FCP.
( Note: After I was done tracking these shots, I stumbled upon this quick tutorial by Mathias Mohl, which combines his After Effects MochaImport script and Red Giant Software’s Warp to better deal with such perspective distortion issues. )
Once I had fixed all of the shots and had new 24p media, I brought these files back into After Effects. There I rendered new 29.97 clips with new 3:2 pulldown, so that the clips could be cleanly cut back into the spots.
Although no single solution provides the silver bullet to fix some of these issues, Imagineer Systems’ mocha goes much farther than the built-in solutions. If tracking is something you need to do often, then mocha for FCP is a pretty cost-effective answer.
© 2009 Oliver Peters
digitalfilms 