All 12 articles in this series are now available in our free 54 page eBook ‘Virtual Production For Broadcast’ – download it HERE.

All articles are also available individually:

If there’s a single enabling technology for virtual production, it’s the LED video wall. While they aren’t the only type of hardware we might use to create in-camera effects with video images, their performance is hard to beat.

No other type of large-scale display is as bright, and an LED display is not reliant on projecting light onto a screen, making it less subject to the foggy blacks which can be caused by extraneous lighting falling on a white screen surface. That combination of bright highlights and deep shadows minimises the problems common to historical projection techniques, and allows the display to cast meaningful amounts of light on the live-action foreground.

The design of LED walls used in virtual production is broadly similar to those used in advertising or large-scale video display at live events, although important details often vary. Also, the capability of an LED wall is not determined only by the panels. The video signal sent to the wall often uses a common standard such as SDI or HDMI, which is interpreted by a processor before being sent to a plug-in receiver in the panels. It’s possible, with care, to rent general-purpose wall components and create temporary setups specific to particular scenes or locations, so long as the capabilities of the resulting video wall are appropriate to the project.

Resolution

With cinema cameras now capable of very high resolution, it’s instinctive to assume that the display used for virtual production must have similar resolution. Often that isn’t quite true, since the LED wall will often be slightly out of focus. Even so, depending on the situation, a certain minimum resolution will be necessary to avoid the individual emitters becoming visible. As a result, the resolution of larger, often permanently-installed virtual production stages can be very high - perhaps tens of thousands of pixels across the curve of a cove-shaped LED wall - placing a large workload on rendering servers.

The resolution of an LED wall will often be given as a pixel pitch, the distance between emitters in millimetres. Each emitter often includes red, green and blue elements, although alternative layouts exist. Compared to technologies such as LCD and OLED, as used in TVs and monitors, LED panels tend to have poor fill factor, with significant non-emitting black areas between the emitters. That exacerbates moire patterning, and LED panels may be chosen for high resolution not because of the demands for an ever sharper image, but to minimise the gaps between pixels. Even so, making much of the wall surface black helps reject extraneous lighting and maintain contrast; a compromise is inevitable.

Given a long enough lens and a deep enough depth of field, the pixels of any video wall might become visible, so there is no universal solution. Estimating the resolution requirements for a virtual production display depends on the resolution of the camera, the camera’s distance from the wall, the focal length of the lens, the aperture, and where the focus is set, as well as the layout of the live action scene and its blocking and staging. As such there’s no single ideal setup for every situation, although pixel pitches around one millimetre are available.

Refresh Rate

The term refresh rate applies slightly differently to LED walls than it does to conventional desktop and video monitors. Ordinarily, refresh rate and frame rate have been synonymous terms. Computer monitors, for instance, often show 60 or 72 individual images per second, while a video monitor for on-set use might show 24, with each pixel illuminated at the appropriate brightness to display a complete image. An LED wall operates differently, switching each individual emitter on and off rapidly to create the variable brightness required to show video, a technique called pulse width modulation.

For LED walls, “refresh rate” refers to the rate at which the pulse width modulation takes place. Typical refresh rates for LED walls are 3840 or 7680Hz (numbers which are only coincidentally related to the horizontal resolution of broadcast video resolutions). The refresh rate may vary so that the number of pulses per second divides evenly into the video frame rate, so each frame is displayed using exactly the same number of pulses. Refresh rate needs to be high enough to avoid flicker or visible segmentation, especially when the camera pans or tilts quickly.

Some special techniques may require higher video frame rates, and therefore higher refresh rates. One example is tracking systems which display markers in between the taking camera’s exposures. Another is multi-camera broadcast applications which can handle several taking cameras simultaneously on a single set, displaying the appropriate background image for each camera in sequence.

Bit Depth

As with any other digital video application, the precision with which an LED wall can display brightness is limited by the number of bits of data used. A ten-bit system can display 1024 different levels of brightness between the darkest and brightest states of a pixel; an eight-bit system can display 256. Bit depth and refresh rate may interact based on the ability of the electronics in each LED panel to switch the individual LEDs on and off very quickly.

One complexity is that not all those brightness levels may be available to display video. For instance, a director of photography might request that the LED brightness be reduced one stop to accommodate other scene lighting. That’s a reduction equal to one-half the total brightness. A ten-bit system (given certain assumptions) might originally have represented full brightness as a signal level of 1024; now it might only be at 512. Reduce another stop, and there are now only 256 levels between black and white, which might create banding (strictly, quantisation noise) depending on the type of content being shown, the camera system, and other specifics. Making colour and brightness changes may also reduce the available precision. Because of this, some systems use 12 or more bits.

Multiplex

The electronics in video wall panels are not usually capable of lighting every emitter at once. A 1920 by 1080 pixel HD display, for instance, would involve over two million pixels, each with red, green and blue emitters. Even if a single control chip could control 16 pixels, that display would require nearly 130,000 chips. Instead, multiplexing is used, where sets of emitters are lit in sequence, rapidly enough to seem constantly illuminated. Often, there are eight or sixteen sets of emitters in each group, so that only one-eighth or one-sixteenth of the emitters are actually lit at any time.

Lower multiplex numbers may mean higher brightness, since more of the lights are lit at any time, although many LED walls have more than enough brightness for many applications in any case. More crucially, lower multiplex numbers, like higher refresh rates, can reduce the incidence of interference patterns that limit certain combinations of frame rate, shutter angle and timing.

Processors And Receivers

The panels of LED emitters which make up the visible wall are often conceptually simple, dedicated to switching their arrays of LED emitters on and off very quickly. Each of them usually has a plug-in receiver card, several of which communicate with a compatible processor, generally made by the same company, installed near the rendering servers. A simple processor may not do much more than break the incoming image up into sections for each panel. More advanced models might perhaps perform scaling or translation, make colour corrections according to creative or technical requirements, and send the results section by section to the receiver in each panel.

The processor and receiver generally communicate using high-performance computer network hardware. Often, optical rather than copper connections are used for increased bandwidth per cable, reducing the number of cables required, though there’s often a significant amount of cabling between the display and the server room. In principle, conventional computer networking equipment can be used, although certain processor manufacturers may require particular capabilities, or even specific firmware, for best performance, particularly concerning delay and latency.

Colorimetry And Calibration

When LED walls are used for advertising, the only requirement is that the result looks attractive to the eye and that corporate colours are reasonably well matched. Usually, those displays are configured to work in the same way as a conventional video monitor, perhaps with some modifications to appear correct in variable weather conditions and times of day in an outdoor configuration. Virtual production displays may need to fulfil other requirements, particularly with regard to colour matching, compatibility with other lighting, the colour presets selected in the taking camera, and any monitoring LUTs.

Colorimetry is a large subject which applies to all camera and display systems used for film and TV work. Two key factors include the colourspace of the system, which is determined by exactly which shades of red, green and blue are used to create colours, and brightness encoding, which controls how much light is emitted by the display for a given signal level. An in-depth discussion of this is more than we have room for, but for common camera setups, most virtual production facilities will quickly be able to configure the system to produce colour that’s correct, or close to correct, with minor manual adjustments sometimes required to make the virtual world’s colour and brightness look realistic compared to foreground action. The colour and brightness behaviour of virtual production displays, as well as their calibration and adjustment, is a rapidly evolving field.