Virtual production stages are invariably built from a collection of pre-existing technologies, and the way those things interoperate is becoming a concern for more and more people.

Gary Feather is someone whose interest in virtual production and direct-view LED displays arises from his decades in the display innovation space and current work as chief operating officer at 6P Color, an organization dedicated to the development of an improved approach to color imaging. In a recent presentation at the Hollywood Professional Association's tech retreat, recognizing the flux in which the field finds itself, Feather began simply by outlining questions potential users should reasonably ask.

"The one resounding message when we started was that we're barely beginning in this space," Feather begins. "Not that what we have is bad, but things are dramatically changing which will make it better depending on what you're trying to do. Some people have volumes with three curved walls, some people have the floors illuminated with images, the ceilings, or they have images and lights set up to work together. You can do anything you want, but it is all about storytelling. The genesis is the same as why you do visual effects of any kind … so you can tell the story better."

Driving Down Costs

One factor driving takeup is that the cost of the most visible component – the display panels – has contracted significantly. "For one cabinet with a 2.5mm pixel pitch in 2013 prices were about $14,000. Now it's $1,400. That's fascinating. It takes 64 of those cabinets to make an HD display." While tighter pixel pitches are possible, Feather reckons that most panels purchased for virtual production are in that range. "Right now, by my calculation, it's about $68million a year in panels alone for virtual production. And they're mostly in the 2.5mm range because you pay by the pixel but you need a lot of area."

Regardless of resolution, anyone who has ever needed to shoot a video image from any kind of display will be familiar with issues of flicker and color balance. Direct-view LED video walls may actually be a tricker target than common TFT-LCD displays, if only because LCDs are often (not always) too slow to produce huge amounts of flicker (though they may). LEDs are far faster; fast enough to create flicker and banding if they're not driven correctly.

"In the virtual production sense," Feather points out, "Hollywood has been doing this for a hundred years. Whether it's a backlit sheet or LED walls doesn't make much difference on some level. But you have to realize an LED is a non persistent display - on some level it is flickering. The human eye is a perfect integrator, it smooths out the flicker, but cameras don't. They have a shutter and it has a frame rate and a shutter angle."

Flicker Challenges

Ensuring those factors don't interfere to create flicker, Feather says, is a responsibility split between the video wall controller, which he describes as "the brain of the operation," and the panels themselves, which may be arranged in a variety of ways. "One-over-32 is a normal multiplexing for LED displays. I have one driver chip running 32 rows, and only one of them is ever lit at once. I can use less drivers that way. If it's one-over-sixteen I need twice as many physical drivers, but each row is lit for twice the time. In Hollywood, if you're 1/12 or 1/8 you're OK. Any higher than that and the camera may see it."

Direct-view LED displays achieve their brightness control using pulse-width modulation, or switching the light source on and off quickly to make it appear dimmer. The technique is usually avoided for on-camera applications because of the risk of flicker; avoiding flicker while retaining precision brightness control requires switching quickly. Traditionally, LED displays have used refresh rates confusingly similar to common video display resolutions, often 3840Hz - not to be confused with the horizontal pixel size of a UHD display. Feather, though, recommends more, especially for high-speed photography. "Refresh needs to be 7680Hz, not 3840Hz, half of that, which is normal for some suppliers.

With flicker and refresh rate issues in hand, the other concern of direct-view LED displays, is moiré patterning. Interference between the grid structure of the LED display and the grid structure of the camera's pixels is caused in part by the inevitably imperfect optical low-pass filtering on many cameras, and in part by the fact that only a very small proportion of many displays actually consists of light-emitting material – it has a low fill factor.

LED Moiré

"The size of the LED point is less than one per cent of the area of a pixel," Feather confirms. "If I blur that, making the dot size much bigger I can reduce moiré." The issue arises particularly when the display may appear in the background of a shot on a long lens with a deep depth of field - as in television. "We've put displays on TV for world cup in Brazil, and we had to put a diffuser in front of it. The camera team there didn't want to deal with it - 'just let me zoom in and out!'."

While external diffusers can work, building that diffusion into the display, or using some other approach to maximizing fill factor, is perhaps better. "One approach is to use a diffuse material when you encapsulate the LED device in its package," Feather says. "The other is to do a four-in-one package, four sets of three RGB elements in there, and I encapsulate that entire package. Then a large area of the display is diffusing, anti-reflective material."

Those and other innovations in the fundamental structure of display panels are available as products now, and some new ideas promise not only reduced costs and finer resolution, but also higher output. Ordinarily, the microscopic gold wiring supplying current obscures some of the emitting face of the LED chip - its die - but reversing the orientation gets the light-scattering wiring out of the way.

"Now people are using flip chip architecture," Feather confirms, "and you get thirty per cent more light from the same size LED. People are also going to chip-on-board – COB – solutions that I'm hoping to test this month, so it's an innovation over the four-in-one a package. Instead of or four pixels, it's four thousand dies on the surface of a single board." Doing that allows the devices to be packaged in a solid block of epoxy: "now you don't have any physical elements to damage. It's a smooth, hard surface, it's durable, I can wash it, it's basically an IP65 surface."

Putting It All Together

At the same time, Feather reinforces that the display itself is just one component of a virtual production environment which must include display panels, a display controller, invariably camera tracking technology and a workstation to put it all together. "A person can spend a little less than $1M, or anything up to maybe seven or eight million, depending on the requirement," he says. "It's not just a display, even the trackers can cost half a million. You have Mo-sys or Optitrack, which work in different ways. You have to address the whole system."

One key crucial advantage of virtual production – beyond the ability to handle elements such as smoke and reflective objects that create challenges on green screen – is the interactive lighting created by the huge output of an LED display. Here, Feather's background in color imaging systems with more than three primary colors intersects with the lighting industry and its interest in ever more capability and precision in low-energy lights." ARRI and Quasar have lighting systems that are RGB for illumination and can integrate with virtual production environments. ARRI Orbiter added amber, lime-green, cyan and white emitters."

With whole new branches of the technology hovering on the brink of the mainstream, and even the incumbent equipment changing quickly, it's clear that configuring virtual production facility is a process. "The prime reason anyone is doing any of this is efficiency and return on investment," Feather concludes." The sheer breadth of possibility, though, is clearly a huge draw to creatives: "a virtual production's art department has to challenge itself to do things differently. I think it's an important tool, it's a case of perfecting the tool. We're not done in this industry. We're only just realizing what's necessary."