SpeedScriber

Script-based video editing started with Ediflex. But, it really came into its own when Avid created script integration as a way to cut dialogue-driven stories, like feature films, in Media Composer. The key ingredient is a written script or a transcription of the spoken audio. This is easy with a feature that’s been acted according to defined script lines, but much harder with something freeform, like a documentary or news interview. In those projects, you first need a person or service to transcribe the audio into a written document – or simply cut without it and hunt around when you look for that one specific sentence.

Modern technology has come to the rescue in the form of artificial intelligence, which has enabled a number of transcription services to offer very fast turnaround times from audio upload to a transcribed, speech-to-text document. Several video developers have tapped into these resources to create new transcription services/applications, which can be tied into several of the popular NLE applications.

Transcription for the three “A” companies

One of these new products is SpeedScriber, a transcription application for macOS and its companion service developed by Digital Heaven, which was founded by veteran UK editor and plug-in developer Martin Baker. To start using SpeedScriber, install the free SpeedScriber application, which is available from the Apple Mac App Store. The next steps depend on whether you just want to create transcribed documents, captioning files, or script integration for Avid Media Composer, Adobe Premiere Pro CC, or Apple Final Cut Pro X.

If you just want a document, or plan to use Media Composer or FCPX, then no other tools are required. For Premiere Pro CC workflows, you’ll want to download an panel installer for macOS or Windows from the SpeedScriber website. This integrates as a standard Premiere Pro panel and permits you to import transcription files directly into Premiere Pro. The SpeedScriber application enables roundtripping to/from Final Cut using FCPXML.

First, let’s talk about the transcription itself. It should generally be clip-based and not from edited timelines, unless you just want to document a completed project or for captioning. When you launch SpeedScriber for the first time, you’ll need to create an account. This will include 15 minutes of free transcription time. The file length determines the time used. Billing for the service is based on time and is tiered, ranging from $.50/minute (30/60/120 minutes) down to $.37/minute (6,000 minutes). Minutes are pre-purchased and don’t expire.

Once your account is ready, drag-and-drop or point the application to the file to import. Disable any unwanted audio channels, so that the transcription is based on the best audio channel within the file. Even if all channels are equal, disable all but one of them. Set up the number of speakers and language format, such as British, Australian, or American English. According to Baker, support for five European languages will be added in version 1.1. The service will automatically determine when speakers change, such as between an interviewer and the subject. It’s hard for the system to determine this with great accuracy, so don’t expect these speaker changes to be perfect.

The transcription experience

Accuracy of the transcription can be extremely good, but it depends on the audio quality that you’ve supplied. A clean interview track – well mic’ed and in a quiet room – can be dead-on with only a few corrections needed. Slower speakers who enunciate well result in greater accuracy. On the other hand, having several speakers in a noisy environment, or a very fast speaker with a heavy accent, will require a lot of correction – enough so that manual transcription might be better in those cases.

Once SpeedScriber has completed its automatic transcription, you can play the file to proof it and make any corrections to the text that are required. It’s easy to type corrections to the transcription within the SpeedScriber text editing window. When done, you can export the text in a number of different formats. I ran a test clip of a clear-spoken woman with well-recorded audio. She had a slight southern drawl, but the result from SpeedScriber was excellent. It also did a good job of ignoring speech idiosyncrasies, such a frequent “ums”. This eight minute test clip only required about a dozen text corrections throughout.

If the objective is script integration into an NLE, then the process varies depending on brand. Typically such integration is clip-based, although multi-cam clips are supported. However, it’s tougher when you try to connect the transcription to a timeline. For example, I like to do cutdowns of interviews first, before transcribing, and that’s not really how ScreedScriber works best. In version 1.1, FCPX compound clips will be supported, so segments can be cut before transcription.

A clear set of tutorial videos are available in the support section of  the SpeedScriber website.

Integration with NLEs

Media Composer is easy, because it already has a Script Integration feature. Import the text file that was exported from SpeedScriber as a new script into Media Composer and link the video clip to it. If you purchased Avid’s ScriptSync, then you can automatically line up the clip to sentences within the script. This happens automatically thanks to ScriptSync’s speech analysis function. But if you didn’t purchase this add-on, simply add sync points manually.

With Premiere Pro, select the clip, open the SpeedScriber panel and from it, import the corresponding transcription. The text appears in the Speech Analysis section of that clip’s metadata display. It will actually be embedded into the media file so that the clip can be moved between projects complete with that clip’s transcription. You can view and use this text display to mark in/out by words for accurate script-based selections. When you import the script and link it to a multi-cam clip, synced clip, or sequence, text will show up as markers and can be viewed in the markers panel. Premiere Pro is the only integration that can easily update existing speech metadata or markers. So you can start editing with the raw transcript and then update it later when corrections have been made. However, when I tested transcriptions on an edited sequence instead of a clip, it locked up Premiere Pro, requiring a Force Quit. Fortunately, when I re-opened the recovered project, the markers were there as expected.

The most straight forward approach seems to be its use with Final Cut Pro X. According to Baker, “This is the first Digital Heaven product with broad appeal by supporting Avid and Premiere Pro. But FCPX has ended up having the deepest integration due to the ability to drag-and-drop the Library, which was introduced in 10.3. So with roundtripping, SpeedScriber rebuilds the clip’s timeline without any need to export. Another advantage of the roundtripping is that SpeedScriber can read the audio channel status from the dropped XML, which is important for getting the best accuracy.”

There’s a roundtrip procedure with FCPX, but even without it, simply export an FCPXML from SpeedScriber. Import that into your Final Cut Pro X Library. The clip will then show a number of keyword entries corresponding to line breaks. For each keyword entry, the browser notes field will display the associated text, making it easy to find any dialogue. Plus, these entries are already marked as selections. When clips are edited into the sequence (an FCPX Project), the timeline index enables these notes to be displayed under the Tags section.

SpeedScriber shows tremendous potential to accelerate the efficiency of many spoken-word projects, like documentaries. Half the battle is trying to figure out the story that you want to tell, so having the text right in front of you makes this job easier. Applying modern technology to this challenge is refreshing and the constantly improving accuracy of these systems makes it an easy consideration. SpeedScriber is one of those tools that not only gets you home earlier, but will give you the assurance that you can easily find that clip you are looking for in the proverbial haystack of clips.

©2017 Oliver Peters

Advertisements

A Light Footprint

When I started video editing, the norm was an edit suite with three large quadraplex (2”) videotape recorders, video switcher, audio mixer, B&W graphics camera(s) for titles, and a computer-assisted, timecode-based edit controller. This was generally considered  an “online edit suite”, but in many markets, this was both “offline” (creative cutting) and “online” (finishing). Not too long thereafter, digital effects (ADO, NEC, Quantel) and character generators (Chyron, Aston, 3M) joined the repertoire. 2” quad eventually gave way to 1” VTRs and those, in turn, were replaced by digital – D1, D2, and finally Digital Betacam. A few facilities with money and clientele migrated to HD versions of these million dollar rooms.

Towards the midpoint in the lifespan for this way of working, nonlinear editing took hold. After a few different contenders had their day in the sun, the world largely settled in with Avid and/or Media 100 rooms. While a lower cost commitment than the large online bays of the day, these nonlinear edit bays (NLE) still required custom-configured Macs, a fair amount of external storage, along with proprietary hardware and monitoring to see a high-quality video image. Though crude at first, NLEs eventually proved capable of handling all the video needs, including HD-quality projects and even higher resolutions today.

The trend towards smaller

As technology advanced, computers because faster and more powerful, storage capacities increased, and software that required custom hardware evolved to work in a software-only mode. Today, it’s possible to operate with a fraction of the cost, equipment, and hassle of just a few years ago, let along a room from the mid-70s. As a result, when designing or installing a new room, it’s important to question the assumptions about what makes a good edit bay configuration.

For example, today I frequently work in rooms running newer iMacs, 2013 Mac Pros, and even MacBook Pro laptops. These are all perfectly capable of running Apple Final Cut Pro X, Adobe Premiere Pro, Avid Media Composer, and other applications, without the need for additional hardware. In my interview with Thomas Grove Carter, he mentioned often working off of his laptop with a connected external drive for media. And that’s at Trim, a high-end London commercial editing boutique.

In my own home edit room, I recently set aside my older Mac Pro tower in favor of working entirely with my 2015 MacBook Pro. No more need to keep two machines synced up and the MBP is zippier in all respects. With the exception of some heavy-duty rendering (infrequent), I don’t miss using the tower. I run the laptop with an external Dell display and have configured my editing application workspaces around a single screen. The laptop is closed and parked in a BookArc stand tucked behind the Dell. But I also bought a Rain stand for those times when I need the MBP open and functioning as a second display.

Reduce your editing footprint

I find more and more editors working in similar configurations. For example, one of my clients is a production company with seven networked (NAS storage) workstations. Most of these are iMacs with few other connected peripherals. The main room has a 2013 “trash can” Mac Pro and a bit more gear, since this is the “hero” room for clients. If you are looking to downsize your editing environment, here are some pointers.

While you can work strictly from a laptop, I prefer to build it up for a better experience. Essential for me is a Thunderbolt dock. Check out OWC or CalDigit for two of the best options. This lets you connect the computer to the dock and then everything else connects to that dock. One Thunderbolt cable to the laptop, plus power for the computer, leaving you with a clean installation with an easy-to-move computer. From the dock, I’m running a Presonus Audiobox USB audio interface (to a Mackie mixer and speakers), a TimeMachine drive, a G-Tech media drive, and the Dell display. If I were to buy something different today, I would use the Mackie Onyx Blackjack interface instead of the Presonus/Mackie mixer combo. The Blackjack is an all-in-one solution.

Expand your peripherals as needed

At the production company’s hero room, we have the extra need to drive some video monitors for color correction and client viewing. That room is similarly configured as above, except with a Mac Pro and connection to a QNAP shared storage solution. The latter connects over 10Gb/s Ethernet via a Sonnet Thunderbolt/Ethernet adapter.

When we initially installed the room, video to the displays was handled by a Blackmagic Design UltraStudio device. However, we had a lot of playback performance issues with the UltraStudio, especially when using FCPX. After some experimenting, we realized that both Premiere Pro and FCPX can send a fullscreen, [generally] color-accurate signal to the wall-mounted flat panel using only HDMI and no other video i/o hardware. We ended up connecting the HDMI from the dock to the display and that’s the standard working routine when we are cutting in either Premiere Pro or Final Cut.

The rub for us is DaVinci Resolve. You must use some type of Blackmagic Design hardware product in order to get fullscreen video to a display when in Resolve. Therefore, the Ultrastudio’s HDMI port connects to the second HDMI input of the large client display and SDI feeds a separate TV Logic broadcast monitor. This is for more accurate color rendition while grading. With Media Composer, there were no performance issues, but the audio and video signal wants to go through the same device. So, if we edit Avid, then the signal chain goes through the UltraStudio, as well.

All of this means that in today’s world, you can work as lightly as you like. Laptop-only – no problem. iMac with some peripherals – no problem. A fancy, client-oriented room – still less hassle and cost than just a few short years ago. Load it up with extra control surfaces or stay light with a keyboard, mouse, or tablet. It all works today – pretty much as advertised. Gone are the days when you absolutely need to drop a small fortune to edit high-quality video. You just have to know what you are doing and understand the trade-offs as they arise.

©2017 Oliver Peters

Adobe’s Late-2017 Creative Cloud Updates

According to Cisco, 82% of internet traffic will be video by 2021. Adobe believes over 50% of that will be produced video and not just simple user content. This means producers will be expected to produce more – working faster and smarter. In the newest Creative Cloud update, Adobe has focused on just such workflow improvements. These were previewed at IBC and will be released later this year.

Adobe Premiere Pro CC

With this release, Adobe has finally enabled the ability to have more than one project file open at the same time. You can move clips and sequences between open projects. In addition, projects can be locked by the user, making Premiere Pro the first NLE to enable multiple open projects and locking within a single application. In addition, Adobe has expanded project types to include both Team Projects (your project is in the cloud) and shared projects (your project is local). The latter is ideal for SAN/NAS environments and adds Avid-style collaboration.

Editors will enjoy specific timeline enhancements, like “close all gaps” and up to 16 label colors. The Essential Graphics panel gets some love with font filtering and a visual font preview window. Graphics templates will now include a minimum duration, so that these clips can be extended on the timeline, while leaving the fade-in and fade-out constant.

Adobe is doubling down on VR using its acquired Skybox technology. New are 19 immersive effects and transitions specific to VR projects. These are needed to properly seam wraparound edges when effects are added to VR clips. They are all GPU-only effects; however, as some VR clips can be 5K wide and larger, performance can be challenging. Nevertheless, Adobe reports decent performance with 6K VR clips at half-resolution on laptops like the HP z820 or the 2017 15” MacBook Pro. There is also an immersive playback viewer designed for HMDs (head mount displays). It will display the image along with the Premiere Pro timeline window.

Premiere Pro’s non-VR editing updates, including shared projects, are explained well by the reTooled blog (video here).

Adobe Audition

Audition is the place to finalize your Premiere Pro mix, so a new auto-ducking mix tool has been added. This is based on Sensei, Adobe’s umbrella name for its artificial intelligence technologies. To use auto-ducking, the editor simply has to adjust sensitivity, amount of reduction, and fades, and then let Audition do the rest. Under AI, it will detect pauses in the dialogue and adjust music volume accordingly.

Other Audition enhancements include a timeline timecode overlay for the video viewer, the ability to simultaneously adjust dual-sided fades on clips, and new record and punch-in preferences for ADR work (“looping”).

After Effects

Here’s another example of this focus on time-savings. After Effects gains a new start-up window to set-up the first composition. It also gains a keyboard command editor, and in this release, will add the same font previewing tools as Premiere Pro. The biggest new feature is an expansion of the expression controls. These will be tied to data files for the quick updating of template graphics. If you create a graphic – such as a map of the US with certain information displayed by colors for each state – and it’s based on a template tied to data, then changing the supporting data information will automatically update the graphic. Other enhancements include GPU acceleration for third-party plug-ins that use the Mercury Playback Engine.

Character Animator

This live-capture, cartoon animation tool finally comes out of beta. A new feature will be the adjustment of the responsiveness of the animation tracking. This will permit live animation to look more hand-drawn. Actions can now be triggered by MIDI control panels. Triggers are editable in the timeline with a waveform for better matching of lip-sync.

There’s plenty of good user news, too, including the the release of 6 Below, an ultra-wide film designed for the Barco three-screen format. It was edited by Vashi Nedomansky using Premiere Pro. Other Premiere Pro news includes the dramatic feature film, Only The Brave, edited by Bill Fox, and Coup 53, a documentary in post being cut by Walter Murch. Both of these noted editors have been using Premiere Pro.

For more in-depth info, check out these links for a solid overview of Adobe’s soon-to-come Creative Cloud application updates:

ProVideo Coalition – Scott Simmons

Premiere Bro blog

Adobe’s own Digital Video & Audio blog

Originally written for Digital Video magazine / Creative Planet Network

©2017 Oliver Peters

Customize Premiere Pro with Workspaces

Most days I find myself in front of Adobe Premiere Pro CC, both by choice and by the jobs I’m booked on. Yes, I know, for some it’s got bugs and flaws, but for me it’s generally well-behaved. Given the choices out there, Premiere Pro feels the most natural to me for an efficient editing workflow.

Part of what makes Premiere Pro work for me is the ability to customize and fine-tune the user interface layout for the way I like to work or the tasks at hand. This is made possible by Adobe’s use of panels for the various tools and windows within the interface. These panels can float or be docked, stacked, or tabbed in a wonderfully large range of configuration possibilities. The Adobe CC applications come with a set of preset workspaces, but these can be customized and augmented as needed. I won’t belabor this post with an in-depth explanation of workspaces, because there are three very good explanations over at PremiereBro. (Click these links for Post 1, Post 2, and Post 3). My discussion with Simon Ubsdell made me think the topic would make a good blog post here, too.

It all starts with displays

I started my NLE journey with Avid and in the early days, two screens (preferably of a matching size) were essential. Bins on the left with viewers and timeline on the right. However, in the intervening years, screen resolution has greatly increased and developers have made their UIs work on dual and single-screen configurations. Often today, two screens can actually be too much. For example, if you have two side-by-side 27” (or larger) displays, the distance from the far left to the far right is pretty large. This makes your view of the record window quite a bit off-center. To counter-balance this issue, in a number of set-ups, I’ve taken to working with two different sized displays: a centered 27”, plus a smaller 20” display to the left. Sometimes I’ll have a broadcast display to the right. The left and right displays are at an angle, which means that my main working palette – the viewers and timeline – are dead-center on the display in front of me.

I also work with a laptop from time to time, as well as do some jobs in Final Cut Pro X. Generally a laptop is going to be the only available display and FCPX is well-optimized for single-screen operation. As a result, I’ve started to play around with working entirely on a single display – only occasionally using the landscape of the secondary display on my left when really needed. The more I work this way, the more I find that I can work almost entirely on one screen, if that screen offers a decent resolution.

So in order to optimize my workflow, I’ve created a number of custom Premiere Pro workspaces to serve my needs. (Click any of these images to see the enlarged view.)

Edit layout 1

This is the classic two-screen layout. Bins on the left and dual-viewer/timeline on the right. I use this when I have a lot of footage and need to tab a number of bins or expand a bin to see plenty of list details or thumbnails.

Edit layout 2

This layout collapses the classic layout onto a single screen, with the project panel, viewers and timeline.

Edit layout 3

This layout is the one I use most often, because most of what I need is neatly grouped as a tab or a stack on the left and right sides of a single viewer window. Note that there are actually source and record viewers, but they are stacked behind each other. So if I load a clip or match frame from the timeline, the source viewer becomes foremost for me to work with. Do an edit or go back to the timeline and the viewer switches back to the record side.

By tabbing panels on the left side, I can select the panel needed at the time. There is a logical order to what is on the left or right side. For instance, scopes are left and Lumetri Color controls on the right – thus, both can be open. Or I can drag an effect from the right pane’s Effects palette onto the Effects Control panel on the left.

Edit layout 4

This is the most minimalist of my workspaces. Just the viewers and timeline. Anything else can be opened as a floating window for temporary access. The point of this workspace is 100% focus on the timeline, with everything else hidden.

Edit layout 5

This workspace is designed for the “pancake timeline” style of editing. For example, build a “selects” timeline and then pull from that down to your main editing timeline.

Edit layout 6

This is another dual-display layout optimized for color correction. Lumetri Color and Effects Control panel flanking the viewer, with the Lumetri Scopes fullscreen on the lefthand monitor.

There are certainly plenty of other ways you can configure a workspace to suit your style. Some Premiere Pro editors like to use the secondary screen to display the timeline panel fullscreen. Or maybe use it to spread out their audio track mixer. Hence the beauty of Adobe’s design – you can make it as minimal or complex as you like. There is no right or wrong approach – simply whatever works to improve your editing efficiency.

Note: Footage shown within these UI screen grabs is courtesy of Imagine Dragons and Adobe from the Make the Cut Contest.

©2017 Oliver Peters

LumaFusion – an iOS NLE

As Apple’s iOS platform becomes more powerful, applications for it begin to rival the power and complexity of desktop software. LumaFusion is a recently introduced nonlinear video editing product from Luma Touch. Its founders created the Avid/Pinnacle/Corel iOS NLE, but LumaFusion takes a fresh approach. Luma Touch currently offers three iOS products: LumaClip (a single-clip editor), LumaFX (video effects for clips), and LumaFusion (a full-fledged NLE that integrates the features of the other two products). All three apps run under what Luma Touch dubs their Spry Engine, a framework for iOS video applications.

LumaFusion works on both the iPhone and iPad; however, the iPad version comes closest to a professional desktop experience. Ideally you’ll want one of the iPad Pros, but it runs perfectly fine on an iPad Air 2 with the A8X chip, which is what I used. I’ve tried other iOS NLEs, including Adobe Clip, iMovie, and TouchEdit, which have their pros and cons. For instance, iMovie doesn’t deal with fractional video frame rates and TouchEdit tries to mimic a flatbed film editor. This brings me to LumaFusion, which has been designed as a modern, professional-grade NLE for the iOS platform.

UPDATE: Watch this video for a rundown of the new features in version 1.4, released in September 2017.

The iOS ecosystem

Like other iOS apps, that tie into the ecosystem, media can be imported from iTunes, Photos, and other third-party applications, like FiLMiC Pro. As a “pro” app, it understands various whole and fractional frame rates and sizes up to 3840 x 2160 (UHD 4K), depending on your device. However, for me, the interest is not in cutting things that I’ve shot with my iPad, but rather fitting it into an offline/online editing workflow. This means import and export are critical.

If you own an iPad Pro, then you can get an SD card reader as an accessory. With the card reader, only native DSLR movie clips will be imported into the Photos app, but not other file formats. Typically, you are going to transfer media using cloud syncing tools, like Dropbox, Box, OneDrive, etc. LumaFusion also includes a number of royalty-free music cuts, which can be accessed through its integrated media browser.

To use it as a rough-cut tool, simply create H.264 proxies on your desktop system and sync those to the iPad using Dropbox (or another cloud service). I created a test project of about 60 clips (720p, 6Mbps, 29.97fps) that only consumed 116MB of storage space. Therefore, even a free 2GB Dropbox account would be fine. Within LumaFusion, import the files from Dropbox and start editing.

LumaTouch will soon start beta testing LumaConnect – a macOS companion application designed to facilitate offline/online editing roundtrips. It will feature automatic iOS proxy creation and the ability to relink high-res media – as well as any iOS-captured content – back on your desktop computer. LumaConnect will also allow the rendering of projects as Apple ProRes files.

User interface and editing workflow

Overall, the interface design and editing model more closely approximates Apple Final Cut Pro X than any other NLE. The app’s design is built around a media pool with various editing projects (sequences). This is a similar approach to FCPX 10.0, which had separate Events (bins) and Projects (sequences), but no combined Libraries. It’s almost like FCPX “Lite” for iOS.

There are three main windows: media browser, timeline, and a single, combo viewer. It uses fly-out panels for tools and mode changes to access clip editing and effects modules. These modules are, in fact, LumaClip and LumaFX integrated into LumaFusion. The timeline is “magnetic”, much like FCPX. Clip construction on the timeline also follows the layout of primary and connected clips, rather than discrete target tracks. A total of three integrated audio/video clips can be stacked vertically, along with another three audio-only clips, for a total of six audio “tracks”. Audio can be adjusted through a fly-out track mixer. LumaFusion includes four clip editing tools: speed and reverse, frame fit, color effects, and audio editing. In addition, there’s a multi-layered title tool, along with a number of customizable title templates to choose from. Clip-based volume and video effect adjustments can be keyframed.

Effects are pretty sophisticated and would often be GPU-accelerated on a desktop system. These include color correction, blurs, transforms, transitions, and more. You can stack a number of these onto a single clip without any impact on playback. The effects priority can be rearranged and the interface also provides an indication of how many resources you are tying up on the iPad.

The editing experience

Serious video editing on an iPad isn’t for everyone, but the more I worked with it, the more I enjoyed the experience. If you have an iPad-compatible keyboard, it follows some generic commands, including JKL playback and I and O for mark-in and mark-out. There are also a few FCPX keystrokes, like W for insert/overwrite (depending on which edit mode is selected). Unfortunately J (reverse playback) only works in the clip viewer, but not in the timeline. I’d love to see a more extensive keyboard command set. Naturally, being an iOS app, everything can be accessed via touch, which is best (though not essential) if you have the Apple Pencil for the iPad Pro.

There are a few standard editing functions that I missed. For example, there’s no “rolling-edit” trim function. If you want to move a cut point – equally trimming the left and right sides – you have to do it in the overwrite edit mode and trim the incoming or outgoing side of one of the clips. But, if you trim it back, a gap is left. J-cuts and L-cuts require that you detach the audio from the clip, as there is no way to expand an a/v clip in the timeline.

It is definitely possible to finish and export a polished piece from LumaFusion. You can also export an audio-only mix. This enables you to embellish your audio track outside of LumaFusion and then reimport and marry it to the picture for the final version. Because you can layer vertical tracks, cutting a two-camera interview piece on your iPad is pretty easy. Rough-cutting a first pass or pulling edited selects on an iPad becomes completely viable with LumaFusion.

Sharing your edit

Once you’ve edited your piece, it’s easy to share (export) your final sequence as a single audio/video file, audio-only file, project (currently only compatible with LumaFusion), or trimmed media. Be aware that there’s a disconnect between the frame rate terminology for settings versus exports. For example, with project settings, you can pick 24 or 30, which are actually 23.98 or 29.97; however, on export, you must pick between 24 and 23.98 or 30 and 29.97. Nevertheless, exports up to UHD frame sizes are fine, including downscaled sizes, if needed. So, you can import and cut in UHD and export a 1080 file. A flattened H.264 movie file of your sequence – wrapped in either an .mp4 or QuickTime .mov container – may be exported at up to 50Mbps (1080p) or 100Mbps (UHD).

If your intension is to use LumaFusion for “offline” editing, then for now, your only option is to embed “burn-in” timecode into the media that you send to the iPad. Then manually write down edit points based on the visible timecode at the cuts. The upcoming LumaConnect macOS application will make it possible to send projects to both Final Cut Pro X and Premiere Pro via XML. According to Luma Touch, they will also be adding XML export from LumaFusion as an in-app purchase, most likely before the release of LumaConnect.

Using an iPad or iPad Pro as your only computer isn’t for everyone, but LumaFusion is definitely a tool that brings iOS editing closer to the desktop experience. To get you started, the company has posted over 30 short tutorials on their YouTube channel. Sure, there are compromises, but not as many as you might think for simple projects. Even if an iPad is only a supplemental tool, then like so many other iOS apps, LumaFusion is another way to add efficiency in the modern, mobile world.

Originally written for RedShark News.

©2017 Oliver Peters

Baby Driver

You don’t have to be a rabid fan of Edgar Wright’s work to know of his films. His comedy trilogy (Shaun of the Dead, Hot Fuzz, The World’s End) and cult classics like Scott Pilgrim vs. the World loom large in pop culture. His films have earned a life beyond most films’ brief release period and earned Wright a loyal following. The latest film from Wright is Baby Driver, a musically-fueled action film written and directed by Wright, which just made a big splash at SXSW. It stars Ansel Elgort, Kevin Spacey, Jon Hamm, Jamie Foxx, and Eiza Gonzalez.

At NAB, Avid brought in a number of featured speakers for its main stage presentations, as well as its Avid Connect event. One of these speakers was Paul Machliss (Scott Pilgrim vs. the World, The World’s End, Baby Driver), who spoke to packed audiences about the art of editing these films. I had a chance to go in-depth with Machliss about the complex process of working on Baby Driver.

From Smoke to baptism by fire

We started our conversation with a bit of the backstory of the connection between Wright and Machliss. He says, “I started editing as an online editor and progressed from tape-based systems to being one of the early London-based Smoke editors. My boss at the time passed along a project that he thought would be perfect for Smoke. That was onlining the sitcom Spaced, directed by Edgar Wright. Edgar and I got on well. Concurrent to that, I had started learning Avid. I started doing offline editing jobs for other directors and had a ball. A chance came along to do a David Beckham documentary, so I took the plunge from being a full-time online editor to taking my chances in the freelance world. On the tail end of the documentary, I got a call from Edgar, offering me the gig to be the offline editor for the second season of Spaced, because Chris Dickens (Hot Fuzz, Berberian Sound Studio, Slumdog Millionaire) wasn’t available to complete the edit. And that was really jumping into the deep end. It was fantastic to be able to work with Edgar at that level.”

Machliss continues, “Chris came back to work with Edgar on Shaun of the Dead and Hot Fuzz, so over the following years I honed my skills working on a number of British comedies and dramas. After Slumdog Millionaire came out, which Chris cut and for which he won a number of awards, including an Oscar, Chris suddenly found himself very busy, so the rest of us working with Edgar all moved up one in the queue, so to speak. The opportunity to edit Scott Pilgrim came up, so we all threw ourselves into the world of feature films, which was definitely a baptism by fire. We were very lucky to be able to work on a project of that nature during a time where the industry was in a bit of a slump due to the recession. And it’s fantastic that people still remember it and talk about it seven years on. Which brings us to Baby Driver. It’s great when a studio is willing to invest in a film that isn’t a franchise, a sequel, or a reboot.”

Music drives the film

In Baby Driver, Ansel Elgort plays “Baby”, a young kid who is the getaway driver for a gang. At a young age, he was in a car accident which leaves him with tinnitus, so it takes listening to music 24/7 to drown out the tinnitus. Machliss explains, “His whole life becomes regimented to whatever music he is listening to – different music for different moods or occasions. Somehow everything falls magically into sync with whatever he is listening to – when he’s driving, swerving to avoid a car, making a turn – it all seems to happen on the beat. Music drives every single scene. Edgar deliberately chose commercial top-20 tracks from the 1960s up to today. Each song Baby listens to also slyly comments on whatever is happening at the time in the story. Everything is seemingly choreographed to musical rhythms. You’re not looking at a musical, but everything is musically driven.”

Naturally, building a film to popular music brings up a whole host of production issues. Machliss tells how this film had been in the planning for years, “Edgar had chosen these tracks years ago. I believe it was in 2011 that Edgar and I tried to sequence the tracks and intersperse them with sound effects. A couple of months later, he did a table read in LA and sent me the sound files. In the Avid, I combined the sound files, songs, and some sound effects to create effectively a 100-minute radio play, which was, in fact, the film in audio form. The big thing is that we had to clear every song before we could start filming. Eventually we cleared 30-odd songs for the film. In addition, Edgar worked with his stunt team and editor Evan Schiff in LA to create storyboards and animatics for all of the action scenes.”

Editor on the front lines

Unlike most films, a significant amount of the editing took place on-set with Machliss working from a portable set-up. He says, “Based on our experiences with Scott Pilgrim and World’s End, Edgar decided it would be best to have me on-set during most of the Atlanta shoot for Baby Driver. Even though a cutting room was available, I was in there maybe ten percent of the time. The rest of the time I was on set. I had a trolley with a laptop, monitor, an Avid Mojo, and some hard drives and I would connect myself via ethernet to the video assist’s hard drive. Effectively I was crew in the front lines with everyone else. Making sure the edit worked was as important as getting a good take in the can. If I assured Edgar that a take would work, then he knew it wasn’t going to come back and cause problems for us six months later. We wanted things to work naturally in camera without a lot of fiddling in post. We didn’t want to have to fall back on frame-cutting and vari-speeding if we didn’t have to. There was a lot of prep work in making sure actions correctly coincided with certain lyrics without the action seeming mechanical.”

The nature of the production added to the complexity of the production audio configuration, too. Machliss explains, “Sound-wise, it was very complicated. We had playback going to earwigs in the actors’ ears, Edgar wanted to hear music plus the dialogue in his cans, and then I needed to get a split feed of the audio, since I already had the clean music on my timeline. We shot this mostly on 35mm film. Some days were A-camera only, but usually two cameras running. It was a combination of Panavision, Arricams, and occasionally Arri Alexas. Sometimes there were some stunt shots, which required nine or ten cameras running. Since the action all happened against playback of a track, this allowed me to use Avid’s multicam tools to quickly group shots together. Avid’s AMA tools have really come of age, so I was able to work without needing to ingest anything. I could treat the video assist’s hard drive as my source media, as long as I had the ethernet connection to it. If we were between set-ups, I could get Avid to background-transcode the media, so I’d have my own copy.”

Did all of this on-set editing speed up the rest of the post process? He continues, “All of the on-set editing helped a great deal, because we went into the real post-production phase knowing that all the sequences basically worked. During that time, as I’d fill up a LaCie Rugged drive, I would send that back to the suites. My assistant, Jerry Ramsbottom, would then patiently overcut my edits from the video assist with the actual scanned telecine footage as it came in. We shot from mid-February until mid-May and then returned to England. Jonathan Amos came on board a few weeks into the director’s cut edit and worked on the film with Edgar and myself up until the director’s cut picture lock. He did a pass on some of the action scenes while Edgar and myself concentrated on dialogue and the overall shape of the film. He stayed on board up until the final picture lock and made an incredible contribution to the action and the tension of the film. By the end of the year we’d locked and then we finished the final mix mid-February of this year. But the great thing was to be able to come into the edit and have those sequences ready to go.”

Editing from set is something many editors try to avoid. They feel they can be more objective that way. Machliss sees it a bit differently, “Some editors don’t like being on set, but I like the openness of it – taking it all in. Because when you are in the edit, you can recall the events of the day a particular scene was shot – ‘I can remember when Kevin Spacey did this thing on the third take, which could be useful’. It’s not vital to work like this, but it does preclude to a kind of short-hand, which is something Edgar and I have developed over these years anyway. The beauty of it is that Edgar and I will take the time to try every option. You can never hit on the perfect cut the first time. Often you’ll get feedback from screenings, such as ‘we’d like to see more emotion between these characters’. You know what’s available and sometimes four extra shots can make all the difference in how a scene reads without having to re-imagine anything. We did drop some scenes from the final version of the film. Of course, you go ‘that’s a shame’, but at least these scenes were given a chance. However, there are always bits where upon the 200th viewing you can decide, ‘well, that’s completely redundant’ – and it’s easy to drop. You always skate as close to the edge of making a film shorter without doing any damage to it.”

The challenge of sound

During sound post, Baby Driver also presented some unique challenges. Machliss says, “For the sound mix – and even for the shoot – we had to make sure we were working with the final masters of the song recordings to make sure the pitch and duration remained constant throughout. Typically these came in as mono or stereo WAVs. Because music is such an important element to the film, the concept of perceived direction becomes important. Is the music emanating from Baby’s earbuds? What happens to it when the camera moves or he turns his head? We had to work out a language for the perception of sound. This was Edgar’s first film mixed in Dolby ATMOS and we were the second film in Goldcrest London’s new Atmos-certified dubbing theater. Then we did a reduction to 7.1 and 5.1. Initially we were thinking this film would have no score other than the songs. Invariably you need something to get from A to B. We called on the services of Steven Price (Gravity, Fury, Suicide Squad), who provided us with some original cues and some musical textures. He did a very clever thing where he would match the end pitch or notes of a commercial song and then by the time he came to the end of his cue, it would match to the incoming note or key of the next song. And you never notice the change.”

Working with Avid in a new way

To wrap up the conversation, we talked a bit about using Avid Media Composer on his work. Machliss has used numerous other systems, but Media Composer still fits the bill for his work today. He says, “For me, the speed of working with AMA in Avid in the latest software was a real benefit. I could actually keep up with the speed of the shoot. You don’t want to be the one holding up a crew of 70. I also made good use of background transcoding. On a different project (Fleabag), I was able to work with native 2K Alexa ProRes camera files at full resolution. It was fantastic to be able to use Frameflex and apply LUTs – doing the cutting, but then bringing back my old skills as an online editor to paint out booms and fix things up. Once we locked, I could remove the LUTs and export DPX files, which went straight to the grading facility. That was exciting to work in a new way.”

Baby Driver opened at the start of July in the US and is a fun ride. You can certainly enjoy a film like this without knowing the nitty gritty of the production that goes into it. However, after you’ve read this article, you just might need to see it at least twice – once to just enjoy and once again to study the “invisible art” that’s gone into bringing it to screen.

(For more with Paul Machliss, check out these interviews at Studio Daily, ProVideoCoalition, and FrameIO.)

Originally written for Digital Video magazine / Creative Planet Network

©2017 Oliver Peters

Bricklayers and Sculptors

One of the livelier hangouts on the internet for editors to kick around their thoughts is the Creative COW’s Apple Final Cut Pro X Debates forum. Part forum, part bar room brawl, it started as a place to discuss the relative merits (or not) of Apple’s FCP X. As such, the COW’s bosses allow a bit more latitude than in other forums. However, often threads derail into really thoughtful discussions about editing concepts.

Recently one of its frequent contributors, Simon Ubsdell, posted a thread called Bricklayers and Sculptors. In his words, “There are two different types of editors: Those who lay one shot after another like a bricklayer builds a wall. And those who discover the shape of their film by sculpting the raw material like a sculptor works with clay. These processes are not the same. There is no continuum that links these two approaches. They are diametrically opposed.”

Simon Ubsdell is the creative director, partner, and editor/mixer for London-based trailer shop Tokyo Productions. Ubsdell is also an experienced plug-in developer, having developed and/or co-developed the TKY, Tokyo, and Hawaiki effects plug-ins. But beyond that, Simon is one of the folks with whom I often have e-mail discussions regarding the state of editing today. We were both early adopters of FCP X who have since shifted almost completely to Adobe Premiere Pro. In keeping with the theme of his forum post, I asked him to share his ideas about how to organize an edit.

With Simon’s permission, the following are his thoughts on how best to organize editing projects in a way that keeps you immersed in the material and results in editing with greater assurance that you’ve make the best possible edit decisions.

________________________________________________

Simon Ubsdell – Bricklayers and Sculptors in practical terms

To avoid getting too general about this, let me describe a job I did this week. The producer came to us with a documentary that’s still shooting and only roughly “edited” into a very loose assembly – it’s the stories of five different women that will eventually be interweaved, but that hasn’t happened yet. As I say, extremely rough and unformed.

I grabbed all the source material and put it on a timeline. That showed me at a glance that there was about four hours of it in total. I put in markers to show where each woman’s material started and ended, which allowed me to see how much material I had for each of them. If I ever needed to go back to “everything”, it would make searching easier. (Not an essential step by any means.)

I duplicated that sequence five times to make sequences of all the material for each woman. Then I made duplicates of those duplicates and began removing everything I didn’t want. (At this point I am only looking for dialogue and “key sound”, not pictures which I will pick up in a separate set of passes.)

Working subtractively

From this point on I am working almost exclusively subtractively. A lot of people approach string-outs by adding clips from the browser – but here all my clips are already on the timeline and I am taking away anything I don’t want. This is for me the key part of the process because each edit is not a rough approximation, but a very precise “topping and tailing” of what I want to use. If you’re “editing in the Browser” (or in Bins), you’re simply not going to be making the kind of frame accurate edits that I am making every single time with this method.

The point to grasp here is that instead of “making bricks” for use later on, I am already editing in the strictest sense – making cuts that will stand up later on. I don’t have to select and then trim – I am doing both operations at the same time. I have my editing hat on, not an organizing hat. I am focused on a timeline that is going to form the basis of the final edit. I am already thinking editorially (in the sense of creative timeline-based editing) and not wasting any time merely thinking organizationally.

I should mention here that this is an iterative process – not just one pass through the material, but several. At certain points I will keep duplicates as I start to work on shorter versions. I won’t generally keep that many duplicates – usually just an intermediate “long version”, which has lost all the material I definitely don’t want. And by “definitely don’t want” I’m not talking about heads and tails that everybody throws away where the camera is being turned on or off or the crew are in shot – I am already making deep, fine-grained editorial and editing decisions that will be of immense value later on. I’m going straight to the edit point that I know I’ll want for my finished show. It’s not a provisional edit point – it’s a genuine editorial choice. From this point of view, the process of rejecting slates and tails is entirely irrelevant and pointless – a whole process that I sidestep entirely. I am cutting from one bit that I want to keep directly to the next bit I want to keep and I am doing so with fine-tuned precision. And because I am working subtractively I am actually incorporating several edit decisions in one – in other words, with one delete step I am both removing the tail from the outgoing clip and setting the start of the next clip.

Feeling the pacing and flow

Another key element here is that I can see how one clip flows into another – even if I am not going to be using those two clips side-by-side. I can already get a feel for the pacing. I can also start to see what might go where, so as part of this phase, I am moving things around as options start suggesting themselves. Because I am working in the timeline with actual edited material, those options present themselves very naturally – I’m getting offered creative choices for free. I can’t stress too strongly how relevant this part is. If I were simply sorting through material in a Browser/Bin, this process would not be happening or at least not happening in anything like the same way. The ability to reorder clips as the thought occurs to me and for this to be an actual editorial decision on a timeline is an incredibly useful thing and again a great timesaver. I don’t have to think about editorial decisions twice.

And another major benefit that is simply not available to Browser/Bin-based methods, is that I am constructing editorial chunks as I go. I’m taking this section from Clip A and putting it side-by-side with this other section from Clip A, which may come from earlier in the actual source, and perhaps adding a section from Clip B to the end and something from Clip C to the front. I am forming editorial units as I work through the material. And these are units that I can later use wholesale.

Another interesting spin-off is that I can very quickly spot “duplicate material”, by which I mean instances where the same information or sentiment is conveyed in more or less the same terms at different places in the source material. Because I am reviewing all of this on the timeline and because I am doing so iteratively, I can very quickly form an opinion as to which of the “duplicates” I want to use in my final edit.

Working towards the delivery target

Let’s step back and look at a further benefit of this method. Whatever your final film is, it will have the length that it needs to be – unless you’re Andy Warhol. You’re delivering a documentary for broadcast or theatrical distribution, or a short form promo or a trailer or TV spot. In each case you have a rough idea of what final length you need to arrive at. In my case, I knew that the piece needed to be around three minutes long. And that, of course, throws up a very obvious piece of arithmetic that it helps me to know. I had five stories to fit into those three minutes, which meant that the absolute maximum of dialogue that I would need would be just over 30 seconds from each story!  The best way of getting to those 30 seconds is obviously subtractively.

I know I need to get my timeline of each story down to something approaching this length. Because I’m not simply topping and tailing clips in the Browser, but actually sculpting them on the timeline (and forming them into editorial units, as described above), I can keep a very close eye on how this is coming along for each story strand. I have a continuous read-out of how well I am getting on with reducing the material down to the target length. By contrast, if I approach my final edit with 30 minutes of loosely selected source material to juggle, I’m going to spend a lot more time on editorial decisions that I could have successfully made earlier.

So the final stage of the process in this case was simply to combine and rearrange the pre-edited timelines into a final timeline – a process that is now incredibly fast and a lot of fun. I’ve narrowed the range of choices right down to the necessary minimum. A great deal of the editing has literally already been done, because I’ve been editing from the very first moment that I laid all the material on the original timeline containing all the source material for the project.

As you can see, the process has been essentially entirely subtractive throughout – a gradual whittling down of the four hours to something closer to three minutes. This is not to say there won’t be additive parts to the overall edit. Of course, I added music, SFX, and graphics, but from the perspective of the process as a whole, this is addition at the most trivial level.

Learning to tell the story in pictures

There is another layer of addition that I have left out and that’s what happens with the pictures. So far I’ve only mentioned what is happening with what is sometimes called the “radio edit”. In my case, I will perform the exact same (sometimes iterative) process of subtracting the shots I want to keep from the entirety of the source material – again, this is obviously happening on a timeline or timelines. The real delight of this method is to review all the “pictures” without reference to the sound, because in doing so you can get a real insight into how the story can be told pictorially. I will often review the pictures having very, very roughly laid up some of the music tracks that I have planned on using. It’s amazing how this lets you gauge both whether your music suits the material and conversely whether the pictures are the right ones for the way you are planning to tell the story.

This brings to me a key point I would make about how I personally work with this method and that’s that I plunge in and experiment even at the early stages of the project. For me, the key thing is to start to get a feel for how it’s all going to come together. This loose experimentation is a great way of approaching that. At some point in the experimentation something clicks and you can see the whole shape or at the very least get a feeling for what it’s all going to look like. The sooner that click happens, the better you can work, because now you are not simply randomly sorting material, you are working towards a picture you have in your head. For me, that’s the biggest benefit of working in the timeline from the very beginning. You’re getting immersed in the shape of the material rather than just its content and the immersion is what sparks the ideas. I’m not invoking some magical thinking here – I’m just talking about a method that’s proven itself time and time again to be the best and fastest way to unlock the doors of the edit.

Another benefit is that although one would expect this method to make it harder to collaborate, in fact the reverse is the case if each editor is conversant with the technique. You’re handing over vastly more useful creative edit information with this process than you could by any other means. What you’re effectively doing is “showing your workings” and not just handing over some versions. It means that the editor taking over from you can easily backtrack through your work and find new stuff and see the ideas that you didn’t end up including in the version(s) that you handed over. It’s an incredibly fast way for the new editor to get up to speed with the project without having to start from scratch by acquainting him or herself with where the useful material can be found.

Even on a more conventional level, I personally would far rather receive string-outs of selects than all the most carefully organized Browser/Bin info you care to throw at me. Obviously if I’m cutting a feature, I want to be able to find 323T14 instantly, but beyond that most basic level, I have no interest in digging through bins or keyword collections or whatever else you might be using, as that’s just going to slow me down.

Freeing yourself of the Browser/Bins

Another observation about this method is how it relates to the NLE interface. When I’m working with my string-outs, which is essentially 90% of the time, I am not ever looking at the Browser/Bins. Accordingly, in Premiere Pro or Final Cut Pro X, I can fully close down the Project/Browser windows/panes and avail myself of the extra screen real estate that gives me, which is not inconsiderable. The consequence of that is to make the timeline experience even more immersive and that’s exactly what I want. I want to be immersed in the details of what I’m doing in the timeline and I have no interest in any other distractions. Conversely, having to keep going back to Bins/Browser means shifting the focus of attention away from my work and breaking the all-important “flow” factor. I just don’t want any distractions from the fundamentally crucial process of moving from one clip to another in a timeline context. As soon as I am dragged away from that, there’s is a discontinuity in what I am doing.

The edit comes to shape organically

I find that there comes a point, if you work this way, when the subsequence you are working on organically starts to take on the shape of the finished edit and it’s something that happens without you having to consciously make it happen. It’s the method doing the work for you. This means that I never find myself starting a fresh sequence and adding to it from the subsequences and I think that has huge advantages. It reinforces my point that you are editing from the very first moment when you lay all your source material onto one timeline. That process leads without pause or interruption to the final edit through the gradual iterative subtraction.

I talked about how the iterative sifting process lets you see “duplicates”, that’s to say instances where the same idea is repeated in an alternative form – and that it helps you make the choice between the different options. Another aspect of this is that it helps you to identify what is strong and what is not so strong. If I were cutting corporates or skate videos this might be different, but for what I do, I need to be able to isolate the key “moments” in my material and find ways to promote those and make them work as powerfully as possible.

In a completely literal sense, when you’re cutting promos and trailers, you want to create an emotional, visceral connection to the material in the audience. You want to make them laugh or cry, you want to make them hold their breath in anticipation, or gasp in astonishment. You need to know how to craft the moments that will elicit the response you are looking for. I find that this method really helps me identify where those moments are going to come from and how to structure everything around them so as to build them as strongly as possible. The iterative sifting method means you can be very sure of what to go for and in what context it’s going to work the best. In other words, I keep coming back to the realization that this method is doing a lot of the creative work for you in a way that simply won’t happen with the alternatives. Even setting aside the manifest efficiency, it would be worth it for this alone.

There’s a huge amount more that I could say about this process, but I’ll leave it there for now. I’m not saying this method works equally well for all types of projects. It’s perhaps less suited to scripted drama, for instance, but even there it can work effectively with certain modifications. Like every method, every editor wants to tweak it to their own taste and inclinations. The one thing I have found to its advantage above all others is that it almost entirely circumvents the problem of “what shot do I lay down next?” Time and again I’ve seen Browser/Bin-focused editors get stuck in exactly this way and it can be a very real block.

– Simon Ubsdell

For an expanded version of this concept, check out Simon’s in-depth article at Creative COW. Click here to link.

For more creative editing tips, click on this link for Film Editor Techniques.

©2017 Oliver Peters