All 12 articles in this series are now available in our free 54 page eBook ‘Virtual Production For Broadcast’ – download it HERE.

All articles are also available individually:

LED wall based virtual production relies heavily on a fairly new display technology. Most of the LED video wall panels used in current practice are related to designs intended for advertising, boardrooms and demo suites which can tolerate the cost of such a high performance display. Fantastic as they are, though, those installations are intended to look good to the eye. They weren’t designed to look good on camera or behave as sources of light which are suitable for cinematography. It wasn’t until very recently, in early 2023, that we began to see video wall panels specifically designed to address those concerns, and a few others.

Compare LED Lighting

Accurate, believable interactive lighting which reacts correctly to the environment is one way in which LED wall based virtual production separates itself from green screen, and a large part of how it produces such convincing results. The problem is that most video wall panels use the same configuration of LED emitters as the most basic lights which are intended for live events and nightclubs where color quality matters less than impressive visuals.

LED lighting of that type is at best suitable for bold color effects in film and TV work, and essentially all video wall panels have very similar color problems. Sometimes this situation can be controlled with careful configuration, although no amount of adjustment in production or post can fill in the gaps in the spectrum of light created by a simple emitter array.  The panel does not emit (for instance) turquoise light; a turquoise object may simply look blue.

One solution coming to market in 2023 is video-compatible panels which include a white emitter alongside the red, green and blue. Whenever a single pixel is required to output a color which isn’t entirely saturated, the white emitter will be used to create as large a proportion of the total output as possible, with the red, green and blue emitter used to trim and balance the desired color. It’s not a perfect solution, much as no modern LED production light is a perfect solution, but it’s likely to improve things significantly.

Video Color Standards

Despite the colorimetry issues with LED video wall panels, some cinematographers choose to rely on them as much as possible, even using individual panels as lights to add extra light where it’s required. The motivation for doing that is that while the light is not ideal, it will at least match the wall. Matching production lighting may be more difficult. A video imaging standard such as Rec. 709 defines (among other things) which shade of red, green and blue is used to describe the image. Video walls tend to implement those standards. Lights often don’t.

This complicates techniques such as pixel mapping, where data from the video image is used to control lighting. Send the same numeric values to the video wall and to production lights, and unless those lights implement the same color standard as the wall, the color will not match. Solutions to this problem must come from the people who manufacture production lighting. Only the manufacturer has the requisite information about the exact configuration of emitters and the behavior of the light’s firmware.

Gamuts

Finally, virtual production walls might soon develop the ability to show a wider range of colors. In principle, many of them already can, although broad adoption of the long-established Rec. 709 standard designed for television means that many video walls operate at less than their full potential. Generally this isn’t hugely noticeable as many subjects just don’t include many colors outside the Rec. 709 range, but there are some exceptions. Tropical water should look deep turquoise, but often just looks blueish.

This is an issue of compatibility more than capability. Many video wall panels, the receivers which drive them and the processors which send data to those receivers can implement reasonable coverage of much more capable standards. Software such as Unreal Engine, through its implementation of OpenColorIO, can produce compatible image data. Switching those capabilities on requires the entire signal chain to handle wider color gamut data. Wide color gamut is perhaps overlooked because the shortfalls are often not very obvious without a side-by-side comparison.

Speed

LED video walls control their individual emitters using pulse width modulation, flashing the light on and off very rapidly so it appears to be illuminated at less than full brightness.  PWM can create complicated problems when the flicker of the light interacts with the shutter timing of the camera, especially at high frame rates or when the camera has a rolling shutter (as almost all do, even at the high end). As such, the rate at which a panel performs PWM is a key metric as to its suitability for various productions.

The speed of that pulse width modulation has other implications, too. Faster PWM rates provide more levels of dimming.  If the light can only be turned on or off ten times per camera frame, it can only appear at ten discrete levels of brightness. High-end, modern cinema cameras are often capable of at least 12 or 14 bits of dynamic range, meaning (in the most straightforward sense) 4096 or 16384 levels of brightness. Many video walls used for virtual production are fast enough to achieve 10-bit, or 1024 levels, at common frame rates. That’s already less than most cameras, even at normal frame rates. At higher frame rates, things become more difficult.

Since the human eye is not fast enough to perceive most of these problems, there is little incentive for LED panel manufacturers (or at leas the manufacturers of the underlying LED driver chips) to build faster devices.  As with the white emitter added to panels specifically intended for film and television work, the demand for speed applies almost exclusively to virtual production, which might make solutions rare and expensive.

High Frame Rate

High speed shooting for slow motion is another concern, since the faster the camera, the faster the video wall must be. For live broadcast work, it’s possible for several cameras to observe the same video wall if the cameras are configured with appropriate shutter settings and genlocked such that they shoot their frames sequentially, rather than simultaneously. The tiny timing error will not be noticeable to the audience, and it allows the video wall to switch images in time for every camera to view an image individually perspective-corrected for that camera’s position and orientation.

Current setups can handle perhaps four cameras; many studios prefer more, and faster displays might make that possible, at least to the point where very short exposures on each camera might start to create odd-looking blur artifacts. Similar techniques are used to display camera-tracking data on a virtual production wall such that the taking camera does not detect it, but a witness camera does.

Resolution

Video wall panels for virtual production are often chosen to maximize resolution. That’s partly because finer-pitched LED emitters require less defocusing of the background and create a lower risk of moire patterning. The problem being addressed is fill factor, the proportion of the display which emits light as opposed to simply being black. While LED panels have traditionally had rather poor fill factor, that’s also been part of the advantage: the panel is largely a black object. As such, it’s less obvious when normal production lighting illuminates the wall than it would have been with a white projection screen.

So, panels with denser LEDs are a double-edged sword, though that hasn’t halted the drive for resolution. The finest resolution available at the time of writing was under one millimeter, approaching 0.6mm. The resolution of LED wall panels and the OLED and LCD technologies used for TVs and monitors might soon begin to coincide, perhaps in a technology sometimes called microLED, which seems likely to provide far more resolution than any virtual production stage could reasonably demand.

The push for pixel count sometimes leads to the video wall having a very high overall resolution in the tens of thousands of pixels, which presents a high processing load to the rendering servers. It might be more information than is realistically required for good results. The question of how much resolution is useful or necessary depends on the specifics, but even in current practice, a wall might be driven with an upscaled version of an image which has fewer pixels than the wall. Doing so lightens the workload while still using a wall with closely-spaced emitters for moire avoidance.

Obsolescence?

A key is the effect that future improvements to LED video wall panels might have on the popularity of the existing installed base of facilities. The wall panels are often the largest capital expenditure required to set up a virtual production stage. Whether these potential technologies might become mainstream, and the effect that might have on previous-generation facilities, is hard to predict. At best it will create choice: existing facilities might become more affordable, while more recent installations might become more capable, a status quo which is hard to dislike.