Virtual sets have also proved to be a valuable tool to help presenters tell complex stories in a way that is more understandable to the average viewer. This increases program loyalty and ratings by leveraging computer-generated graphical elements both in front of and behind the presenter. It also allows the on-air reporter or host to walk through 3D space as the camera moves with them and intuitively interact with virtual charts or other types of infographics. Numerous industry surveys reveal that viewers have grown to love this new sophisticated-looking capability.

Raising The Bar On Creativity

The new generation of virtual sets can be configured in a variety of ways, depending on usage, need for flexibility and budget. And no technology is more budget-friendly than a virtual environment, which requires a graphics artist and a 3D computer graphics software package but can be customized for different shows and repurposed in a matter of seconds. In addition, less crew is required — in most cases there’s only one operator per program production — and broadcasters can start small, perhaps with a single camera, and then grow the virtual space and add more physical and virtual cameras as required.

With virtual sets, physical cameras can move in 3D space, while the image of a virtual camera is rendered in real time from the same perspective. Therefore the virtual scene can adapt at any time to the camera settings. This is what differentiates a virtual set from the traditional technique of chromakeying. And a virtual studio does not require any postproduction work because it is rendered in real time. In addition, virtual sets can be combined with physical sets, adding an extra layer of depth to the physical studio space.

Augmented Reality: The New Wave In Virtual Sets

While the addition of a virtual background can place studio presenters in literally any location in the world, augmented reality (AR) graphics are rendered in front of the on-air talent and help liven up the set in ways physical furniture cannot. When used in combination with each other, as many advanced broadcasters are now doing, the on-air effect can be captivating.

AR systems are used to place virtual 3D graphic elements in the physical world. These elements are arranged three dimensionally and attached to a live video signal, then output as a combination of live video and graphics. The technology takes the idea of virtual graphics from virtual sets but places them in the front of the presenter or in the foreground of a studio. A big benefit is that these graphical elements can then be used in tandem with a physical set, adding an extra layer of depth to the studio, or extending the studio space, without the need for a chromakey or green screen wall. The technology can even be used for outdoor studio environments, such as during live sporting events.

Virtual Camera Tracking

Virtual sets rely on camera movement data that must be carefully tracked with a real-time rendering and composting system that creates these highly realistic environments. Camera tracking is critical in order to convey an accurate on-air impression of a presenter walking around and interacting with a virtual world. The tracking data from the camera and the rendering of the graphics are composited and output as a final image. Keyed graphics can be included on top if necessary.

Currently, there are three types of camera tracking systems used with virtual sets: Mechanical, Optical and Image based.

Mechanical Camera Tracking Systems consist of specialized hardware, typically a camera tracking head and a lens encoder that can be installed on cameras and on pedestals, in the studio or in the field. These mechanical tracking systems work in tandem with a compositing server for each camera (which is controlled through the video production switcher in the control room or OB van). Mechanical tracking systems offer the best accuracy and can cope with any camera movement as long as the camera is put on a tripod, a crane or rails. 

Another type of motion tracking system uses Optical Tracking technology to capture the camera data and the location of the talent within the virtual set. This technology comes from the motion capture capabilities originally developed for the film and gaming industry. A typical mo-cap system uses a series of specialized illumination units to capture reflective, moving markers with a set of cameras. The technology tends to be more expensive than mechanical tracking technology, but it also provides a lot of flexibility to add extra cameras and tracked objects to the set. In addition, it allows for greater freedom of movement for the on-air talent within the virtual set. Therefore, multiple cameras and objects are trackable simultaneously using the same system.

The third, and most recent entry into the virtual space, is Image-based Tracking. Due to its lower cost, image-based tracking appears to be gaining the most momentum with users of virtual graphics technology. With an image-based tracking system, there are no tracking heads on the camera, and no additional gear is required at the venue or on the camera. The technology allows for tracking a camera using the television camera image (pixels) alone to calculate the movement of the camera. A computer application does the tracking calculations based on the recorded TV footage only. The camera can move around in the studio or around the sports field.

Of the three, the big advantage with Image-based Tracking technology is that it can be used anywhere and on multiple cameras, as long as a TV signal is available. Additional cameras are no longer needed to implement live tied-to-field graphics as virtual video content can now be placed onto any in-camera surface during a transmission. Also, only one system is required with image-based tracking to insert virtual graphics in multiple different cameras, and therefore, it becomes a very cost effective solution.

The Business Case For Virtual Technology

The same virtual set and AR technology that is used in a confined space of a virtual studio can also be used outdoors with equal success. Virtual graphics are increasingly being used on the outdoor playing field, for live sports analysis as well as for commercial sponsorships. The players are keyed above the graphics so that the virtual graphics appear as part of the field. Virtual ads can also replace the physical ads in an outdoor stadium or indoor arena. The technology also allows broadcasters to customize and display different ads for different viewer locations.

With their ability to transport viewers to places they have never been before, virtual studios bring flexibility and cost-effective yet high production value to any type of broadcaster or program producer. Take, for example, what the BBC did during its 2015 election coverage in the UK. The broadcaster virtually re-created the government’s headquarters at 10 Downing Street, complete with individual seats displayed to help illustrate election results projections and how the balance of power might shift. It was a marvel for its artistry and realism as much as for the information it imparted, and the BBC’s high ratings reflected this.

Computer-generated virtual sets and AR graphics have become a powerful tool for program differentiation among the thousands of channels shows in the television universe, creating the illusion of being in a bigger studio than might actually exist while making the scene more attractive on air. The use of both technologies also results in increased program production, something very broadcaster wants to improve upon.

Versatility, cost effectiveness, ease of installation and minimal staff requirements are all key benefits of virtual set and augmented reality systems. That’s why the number of broadcasters using computer-generated virtual sets and AR graphics continues to increase with time. The reality is, this technology is here to stay.

For more on how virtual sets work, and the tools required to create virtual graphics of all types, see the Vizrt Video series: Secrets of Virtual Sets Production.