A Video Tinkerer Solves a Perplexing Audio Problem

The short history of video is full of working videographers who solve major problems in their craft and go on to build companies to share their inventions with others in the industry. Ross Lowell of Lowel Lighting and Anton Wilson of Anton-Bauer are two well-known examples working image-makers turned inventors. Now, add Bruce Sharpe to the list.

In 2000, Bruce Sharpe and his wife shot videos of live events like stage presentations and dances. To get higher production quality, they bought a pair of camcorders. To achieve better audio, they often took the sound feed from the venue’s audio board or placed a small portable audio recorder closer to the performers.

Bruce Sharp had a nine-year quest to sync audio with video

As Sharpe graduated to more professional video jobs, he found the manual syncing of outboard sound with his video to be a time consuming, laborious job. He needed to find a way to automate the sync process. So he began shopping for software that would do the job. To his surprise, none was available.

Thus began Sharpe’s nine-year quest to create his own. “Because of my technical background, which was analyzing satellite imagery and visual data, I figured it was a solvable problem,” Sharpe said. “So I started tinkering with it.”

Sharpe knew that sync could be achieved by viewing and matching audio waveforms. A manual method was to match the peaks and valleys of the audio from the camera and the separate audio track with the eye. Then he would replace the poor audio with the good. But that was way too difficult. A computer could do it much better and faster, Sharpe surmised.

However, the problem was far trickier than Sharpe at first thought. He realized that even though the sound was the same, the waveforms often looked different. The waveform from the camera — which was further away than the better sound source — often didn’t look the same to the eye.

So the tinkering began. He set a goal to create a software application that would automatically sync the higher-quality sound with the picture. The user would not have to tweak it or be bothered with prep in the field. It would just happen quickly and reliably.

From the original concept to finished product took over eight years. In 2009, Sharpe started a company called Singular Software and at that year’s NAB, he introduced two products: PluralEyes and DualEyes. PluralEyes worked with multiple sound sources and video, while DualEyes worked with one sound source and the video.

Sharpe’s timing couldn’t have been more perfect. The previous September, Canon had introduced the EOS 5D Mark II, the first Canon DSLR camera to have high-quality video recording capabilities.

The Canon 5D MkII was a revolution for the cost-conscious videographer

In the design, Canon paid far more attention to the video than audio, and this left a wide window of opportunity for double system sound software like Sharpe had created. “Canon’s introduction blew the whole DSLR thing wide open,” Sharpe said. “They created the need to do separate audio. It broadened the market immediately.”

In July, 2012, Sharpe’s Singular Software merged with Red Giant, a maker of tools for digital video makers based in Beaverton, Oregon. Today, Sharpe is Business Unit Manager of Red Giant’s Shooter Suite. PluralEyes 3 is one of the products in the suite. “It’s all the tools you need as a shooter to get everything ready for post-production,” Sharpe said.

Today, PluralEyes is the market leader for syncing software. A number of smaller competitors have come and gone. One of the newcomers, Pixelynx Labs’ DreamSync, is taking on PluralEyes with a simpler, lower-cost product.

Jon Acosta, designer of DreamSync, said the goal of his company was to create the easiest to use alternative to PluralEyes. He targeted his potential customers as users of basic editing applications like iMovie, Windows Movie Maker and Sony Vegas, which do not have built-in sound sync capability.

DreamSync is ideal for television news, or anyone seeking a very simple way to do double system sound with their video,” Acosta said.

The major competition to PluralEyes comes from Adobe and Apple. Double system audio sync software is now deemed so important that both companies have added basic audio sync capability to their main professional editing products.

In Apple’s Final Cut Pro, the feature is called automatic audio sync. “Final Cut Pro compares waveforms from the camera audio and second audio source to sync sound automatically — down to the sample level,” Apple wrote about its editing feature.

In Adobe Premiere Pro, the audio sync feature is called Merge Clips. “This will streamline the process by which users can sync audio and video which have been recorded separately (a process sometimes called double-system recording),” wrote Adobe. “You’ll be able to select a video clip and synchronize it with up to 16 channels of audio by using the ‘Merge Clips’ command.”

Apple and Adobe’s sync technology, said Sharpe, are very basic. “It works on a few clips at a time,” he noted. “PluralEyes covers a much wider variety of projects and formats. You get a lot of visual feedback. You can see how things are progressing. You can do quality control afterwards. It's much more convenient to use than the very basic functions in those editing applications.”

As to the competition, Sharpe is not worried. “When people hear the idea of sound sync software, some say 'I can do that.' So they try. So far none have been very successful in the marketplace. That's because this is actually a very hard problem to solve.”

PluralEyes is now in beta for version 3.5, which adds “Drift Correction,” a new feature that corrects audio in long clips that can drift out of sync over time. It also adds functionality for more media formats and offers additional playback capabilities.

“We also continue to improve the basic algorithm,” Sharpe said. “It’s now faster and more accurate than in earlier versions.”

For the future, Sharpe said PluralEyes will get greater integration into the Red Giant Shooters Suite. The suite features not only PluralEyes, but BulletProof, an on-set footage prep and delivery solution that takes the user from backup through export. There’s also LUT Buddy for sharing color decisions made on set, Instant HD for up-rezzing and Frames for deinterlacing to integrate new footage with older video formats sharing the edit.

For Sharpe, he picked the right problem to solve at the right moment and has created an essential application for the age of DSLR videography.

You might also like...

Designing IP Broadcast Systems - The Book

Designing IP Broadcast Systems is another massive body of research driven work - with over 27,000 words in 18 articles, in a free 84 page eBook. It provides extensive insight into the technology and engineering methodology required to create practical IP based broadcast…

Demands On Production With HDR & WCG

The adoption of HDR requires adjustments in workflow that place different requirements on both people and technology, especially when multiple formats are required simultaneously.

NDI For Broadcast: Part 3 – Bridging The Gap

This third and for now, final part of our mini-series exploring NDI and its place in broadcast infrastructure moves on to a trio of tools released with NDI 5.0 which are all aimed at facilitating remote and collaborative workflows; NDI Audio,…

Designing An LED Wall Display For Virtual Production - Part 2

We conclude our discussion of how the LED wall is far more than just a backdrop for the actors on a virtual production stage - it must be calibrated to work in harmony with camera, tracking and lighting systems in…

Microphones: Part 2 - Design Principles

Successful microphones have been built working on a number of different principles. Those ideas will be looked at here.