Colorimetry

LED video walls built for virtual production must satisfy several different demands for colorimetry. That depends on the panels and the types of emitter they use, as well as the ability of the processor, receiver and any associated equipment.

Colorimetry: Calibration For Uniformity & Standardization

First, any display made out of many sub-elements, such as an LED video wall, will require calibration. Small manufacturing variations, as well as differential ageing, will cause different panels to have slightly different performance. Without calibration, that non-uniformity might make individual panels visible as rectangular outlines in the image. Many manufacturers now automate calibration using camera-based devices, storing the relevant data in each panel’s receiver and making reconfiguration easier.

Calibration will also ensure the video wall and processors work together to implement standard video color and brightness behavior. Standards such as the ITU’s Recommendation BT.709, BT.2020, the DCI’s P3, or others, might be used. The choice depends on the cameras in use, the setup of the rendering servers, and creative considerations. Software such as Unreal Engine implements OpenColorIO, making it possible to render images using any of the common color standards. For virtual production, it is normal to overlook some aspects of brightness standardization, scaling the absolute light output of the video wall for appropriate exposure while observing other aspects of a video image standard.

Colorimetry: Suitability For Camera

One complicating factor is that video displays of all types often look very different in person than they do on camera. Even with the latest panel designs, it is often necessary to adjust the color behavior of the video wall for proper on-camera appearance, regardless of the appearance of the video wall to the human eye. The specifics depend on the camera in use, ambient lighting, content to be shown on the video wall, and particularly any lookup tables, filtering, or other factors that might alter the color behavior of the camera or display. Precision monitoring by experienced people using calibrated displays, ideally in isolation from the video wall, is often important.

Film cameras can be used with virtual production, although the time between making an adjustment and verifying the result is necessarily extended by the need to process and transfer the material. In these situations, a digital camera, configured to approximate the behavior of the film camera, is sometimes used as a preview camera during test and setup.

Colorimetry: Video Wall As Lighting Device

Light emitted by the video wall will fall on the foreground scene. That illumination is part of the attraction of virtual production, integrating live-action and virtual parts of the frame in a way that few other techniques can. However, some video wall panels have very poor color quality when considered as light sources. Being based only on discrete red, green and blue emitters, panels designed for advertising or live events work are not capable of accurately creating desaturated colors such as the powder blue of a misty sky, which might contribute a lot to an exterior scene.

Instead, to do that, they might emit deep blue light, and mix a large amount of green and a small amount of red light into it. As a video display, to the human eye, the display should look reasonable. To a camera, as discussed, the display might look reasonable. As a way of illuminating foreground objects, it is far less reasonable. It contains, for instance, some red, green and blue, but none of the teal light it might seem to contain. A teal-colored object might not reflect blue or green, and as a result appear muddy and unsaturated. Crucially, a lack of deep red, as well as a gap in the yellow-orange colors between red and green, can result in poor rendering of human skin.

Panels built to be hybrid display and lighting devices for virtual production sometimes include other emitters, particularly white emitters. Those will be used as part of the panel’s output whenever the desired color is anything but fully saturated, filling in a more complete spectrum and greatly reducing lighting colorimetry problems.

Because of the need to adjust the panel’s behavior in several stages, for calibration, camera suitability and overall brightness, panels selected for virtual production will generally have high refresh rates to provide a wide range of fine adjustment.

Advanced Techniques: Image-Based Lighting

The usefulness of the various techniques of virtual production depends on the demands of individual productions. Most facilities, though, will use many of the technologies discussed here.

Some of them will also involve more advanced techniques, which might affect the design of the video wall. One example is image-based lighting, where information from the video displayed on the wall is used to control lighting. This extends the capability of the video wall itself to cast light on the foreground scene, allowing camera and lighting teams to use familiar instruments which will then react to changes in the virtual environment.

There are several ways to do this. Some lighting control desks have video inputs which can supply data to control lighting devices. Software such as Assimilate’s LiveFX can reformat live video data into DMX lighting control commands. LiveFX also allows for significant video processing to make the video data more suitable for lighting control by reducing noise and averaging areas of the image. With some lights now offering control of many small subpixels, these techniques may involve large numbers of DMX channels. Bulk DMX data protocols such as sACN are commonly used.

Unlike video displays, there are no widely-adopted standards for colorimetry in lighting. Lights often mix red, green, blue and other emitters, but there is almost no agreement for which shades of red, green and blue are in use, or how their brightness reacts to DMX commands. The interaction between an LED video wall and lighting instruments is likely to require some manual intervention while the lighting market moves toward standardisation.

Advanced Techniques: High-Speed Imaging

High-performance panels may be fast enough to display several different images during the time a camera would normally take to expose one frame. Among other things, that enables multi-camera virtual production. Live broadcast configurations have been shown with up to four cameras, genlocked to take frames sequentially with one-half the usual shutter timing. The video wall displays material during the period each camera is taking a frame, so that all four cameras see separate images. The image switches so rapidly that in person, the wall appears to be simultaneously displaying a mix of all four images.

That rapid-switching ability can also be used to display tracking markers during the shutter period, when the camera is not exposing a frame. That can ease considerations around the location and visibility of markers or tracking cameras, especially where overhead video panels are in use. Alternatively, where the taking camera is run at twice the intended frame rate, the video wall image can alternate between the virtual production image and a plain chroma-key color, so that either can be used in post-production.

All of these techniques require fast panels with high refresh rates, so that each individual frame still has enough time for fine brightness control.

A Young Discipline

Much as virtual production becomes a mainstream technique, it is still less than ten years into the mainstream. Nonetheless, commissioning a new facility is already easier than it was not long ago. Greater standardization and integration of that stack of technologies seems likely to continue the trend. Even so, LED video wall technology will always be a key determinant of what is possible, and will always be at the heart of the process.