While the techniques used in these early days to broadcast the games has been different in every country—some sports leagues are choosing a minimalist approach while others have experimented with crowd avatars and artificial audio added to the live feed—fans are watching in record numbers. Perhaps due to a desperate need for live sports, the German Bundesliga—the first league to resume live matches without fans—has attracted 50 percent more viewers in the UK alone. Its domestic viewership has significantly increased as well.

For example, Sportcast, which broadcasts Germany’s Bundesliga matches, has decided that just showing the game with tight camera angles (to avoid the empty seats) was what its fans wanted.

Fake Fans

Tim Achberger, head of innovation and technology management at the Deutsche Fußball Liga (DFL) subsidiary, told a recent online gathering hosted by Sports Video Group that they’ve spoken with many technology providers about ways to enhance the game coverage, especially in the field of audio. As the first league to resume live games, Achberger said they discussed three options: producing audio in the stadium by itself; produce audio with pre-clipped examples of earlier games; or bring the sound of viewers from home to the live mix in order to enhance the original sound.

“In the end, we decided not to do anything because we wanted to ensure that everything worked out OK for the first matchday,” he said. “There were so many regulations, so many different topics we had to work on. We checked interactivity features as well but we decided to just focus on the importance of the real basics.”

The English Premiere league is piping in crowd noise supplied by video game developer EA Sports. Spain’s LaLiga is using the same audio technique but went a step further and developed computer-generated fans to fill the seats as a complement to artificial crowd noise.

Different Strokes For Different Folks

None of the choices are perfect for everyone, but they show a willingness on the part of broadcasters to get on with the business of televising live sports in a way that’s familiar for fans sitting at home. Recognizing this, major networks like Sky in the UK and others, are giving viewers two channels to pick from: one with these new enhancements and the other without.

“There certainly is a demand for live sports, but how do you keep people at home engaged with the game when the stadium’s empty,” said Niall Hendry, Product Portfolio Director at ChyronHego, a company with a long history of producing live graphics for a variety of sports telecasts. “That question has not been solved yet. People want to see new things on live sports broadcasts. It’s really an exciting time, from a technology perspective.”

Although it has not been done yet, ChryonHego’s Virtual Placement system could be used to create a wide variety of animations and AR effects that would enhance a ghost game significantly. And in fact, Hendry said he’s been in discussions with sports leagues for how best to use the currently available seating space to insert game stats, sponsor logos and general information.

“I think fans at home would rather see deeper stats on the screen,” he said. “There’s a lot of statistics that can be generated from sports so we should use that canvas of empty seats to inform viewers at home better with engaging content. The opportunity for engagement is really high and the sports world has not yet capitalized on it. At least not yet. Getting brand exposure in the stands is paramount when sponsors are paying the bills.”

Filling Those Empty Seats

Production company Mediapro worked with Vizrt to develop artificial fans and digitally insert them into the live broadcast feed. For a series of LaLiga games in Spain in June, Mediapro created two feeds: a feed with no fan noise and no ghost game overlay, and a second dirty feed of fake fans and fake audio. They let the viewers decide what they wanted to watch.

“We are trying to create a comfortable pair of slippers,” said Jonathan Roberts, senior vice president of Global Sports and Virtual Advertising at Vizrt. “People are watching sport, they are stuck at home, and want a bit or normality. Some people on Twitter have been critical of the graphics but then others have said but we’re in COVID-19, it’s better than empty seats. Mediapro is trying something new and we applaud that.”

Vizrt’s Arena virtual ad placement system was used on one of the cameras in each stadium. They used computer vision to triangulate where camera one is in 3D space and calibrate the system using anchor points from within the stadium to get an understanding of where the camera is in relation to its environment. And to comply with COVID-19 rules, it can be processed remotely, so there’s no need for equipment or a large crew within the venue. Vizrt made sure everything worked as planned by replicating the solution in the labs first to ensure that when it got to the site it worked as planned.

One Camera Feed

“Most of the ghost games are being completed using the Camera One feed,” Roberts said. “The processing can be done in an upstream or downstream environment. Typically, when it’s done upstream it’s a simpler implementation with the enhancements going straight into the mixer and when it’s downstream, the solution is more complex as you need to use cut detection technology [ACD].”

The company's Automatic Cut Detection (ACD) technology, enables a user to upload the wipes and different camera angles. As you move from Camera One to a wipe or replay, you can take the graphics on or off the program feed as required.

Finding A New Angle

“In football [soccer] you can get away with Camera One and change the angles as necessary. When you’ve got no crowd, you can put the cameras anywhere you want. There are no seat-kills. This allows you to place the cameras in positions to get tight shots that are not taking in so much of the background. Therefore, Camera One can be your wide crowd shot and all of the other cameras in a production can zoom in tighter. This way, they become better storytelling tools. And the when you go wide, you have the virtual crowd.”

Roberts noted that for a typical game, crowd shots make up 68 minutes of the 90 minutes of a match. Most of the other angles appear within the replays. So, there’s quite a lot of flexibility with a single camera.

“When you are trying to do multiple games simultaneously of course it does get harder but once the operators have a few games under their belt everyone becomes calmer and its business as usual,” he said.

Then, of course, Mediapro had to deal with different lighting conditions at different stadiums and dayparts. This is always a challenge for graphics insertion when there is shadow and light, which creates challenges for the chromakeyer. Add the fact that you don’t know what these conditions are until you arrive at the stadium, it keeps you on your toes.

Virtual Placement Software

ChyronHego’s Hendry agrees with Vizrt’s Roberts about using upstream and downstream workflows, depending upon the situation at hand.

“It’s important to note that putting virtual fans in the stands is one thing but how engaging is that,” Hendry said. “We think putting more content (stats, brand logos, etc.) is more helpful to the viewer at home. The most successful broadcasters doing ghost games will be those that give viewers the choice to watch [these enhancements] or not. At the end of the day, nothing is normal right now, so we have to try new things to keep viewers engaged.”

The Viewer Will Choose

There are a lot of issues involved in each league’s choice, and some leagues don’t want to feature anything but the game play. Ultimately the fan will decide what the viewing experience will be, but feedback on social media has been positive to all forms of enhancement.

“[In producing the virtual fan graphics] the goal for everyone was to try and create as much value for that ecosystem as possible, rather than waiting for football to fully recover,” Vizrt’s Roberts said.

As a spectator, he noted that when he was watching the channel with no fans and no audio, he was drawn to the areas of the stadium that were empty. When he saw it with fake fans and audio, he was drawn to the football action on the pitch.

“All we’re trying to say to fans is, it’s not perfect, but the world isn’t perfect at the moment. We’re going to try and make your viewing experience a little bit more comfortable. Like a comfy pair of slippers and a little bit more familiar to you.”

A Crowd Kit

It didn’t take long for technology vendors to venture into the virtual crowd game. UK-based Mo-Sys Engineering is now offering “tracking for any camera (including ultra-long box lenses for sport)” as a pre-configured kit that interfaces directly to most render engines. With it, production companies can add photo-realistic augmented reality–such as crowds filling the stands–into live productions.

Michael Geissler, CEO of Mo-Sys, said the kit can help fill the stands with a live production system that generates a “virtual, but enthusiastic, crowd.”

“After so many weeks, sports fans are desperate for any action,” he said. “But the frustration will turn to disappointment if the atmosphere of the game falls flat because of empty stands.”

The Mo-Sys camera tracking encoders are mounted on a Vinten Vector pan/tilt fluid head. The company said there’s “no impact on the camera’s perfect balance and no backlash when panning and tilting.”

Zoom data is collected either by gear encoders or by a serial data link to digital lenses. The combined tracking data is sent over Ethernet to the workstation hosting augmented reality graphics software.

Geissler said they can deliver “precise tracking, including compensation for lens distortion, even when a 100:1 lens is zoomed fully.” Mo-Sys worked closely with Epic Games to interface to the Unreal Engine.

The complete kit includes a bolt-on encoding kit for Vinten heads and the necessary lens calibration tools.