One such person is virtual production supervisor Andy Jarosz, whose collaboration with director of photography Matt Brown on two music videos made good use of the LED facilities at Chicago’s Resolution Studios and offer a rare behind-the-scenes glimpse at a technique that’s often the prerogative of the least accessible projects.

Jarosz’s history involves a variety of relevant disciplines. “I started as a camera assistant, and worked in rentals” he begins, “Eventually I opened a special effects shop, and we made props for shows all over the country. Robots, monsters, all kinds of mechanical and electrical effects. However, I’ve also been a hobbyist game developer for most of my life—so when Unreal engine started being used for virtual production, just by chance I became one of the few people with extensive experience in both disciplines, and the ability to be a translator between both worlds.”

Brown and Jarosz met on the set of a well-known Chicago export – the series Chicago Med. “I grew up in a small town,” Brown begins, “but was always interested in filmmaking. I got a degree in fine art filmmaking from a small school in Tennessee, then I joined Local 600 and became an AC on Chicago Med which is where I met Andy about five years ago.” Having collaborated on shorts and music videos, Brown and Jarosz shared an interest in the growing world of virtual production, with the work of Grieg Fraser, ACS, ASC a particular favorite of Brown’s. “He's one of my favorite DPs - everything he does is fantastic, and when he started talking about virtual production, Andy and I became very interested.”

Opportunity arrived at the request of the Chicago post-punk band Ganser and a pair of music video concepts showcasing virtual production. “They wanted to release two videos back to back, two songs which are being released as singles,” Brown recalls. “The first video, which had the heaviest virtual production component, has the lead singer Nadia being chased through a snowy field and then falling into a pit and being buried alive by her bandmates!” The second production, as Brown goes on, “played off the first - the lead singer of the first video is gone and the lead singer of the second song wakes up in her place inside this grave, gets up and we reveal that we were on an LED wall set. So, we show everything, with the character in this fake environment.”

As Jarosz notes, the facilities to do this are relatively new to the city. “It took a while for anyone in Chicago to build a standing LED volume. We shot at Resolution Studios, who had a 25ft x 12ft LED wall up.” Resolution’s main stage was in a trial configuration set up to evaluate Planar CLI1.9 LED panels. Jarosz warns that not all of the panels used to build LED displays work in the same way; something suitable for giant-screen presentation at an outdoor music concert might not work well on camera.

Some of the more straightforward applications of LED video walls as a tool in their own right have relied on simply playing back video footage, although Brown’s concept for the piece required a true virtual production. “We made a shotlist,” he says, “and realized that what we needed was a 3D field. There are a lot of very snowy fields in Indiana and Illinois, so we went out and we 3D scanned a real field.”

Jarosz describes the scanning setup in a mere few words, although those words imply a lot of technology. “We had a drone, a DJI Mini 2, which we flew and took about a thousand photos of the landscape. We went out with DSLRs and took some separate, more detailed photos of the ground. Using Reality Capture, we turned those into a 3D mesh that could be brought into Unreal Engine. With that that, combined with some HDRIs of the sky, we were able to replicate the look of the actual location really well. We also shot a little bit on location so we could tie in the location photography and the on set photography.”

Images for the video wall were rendered on a custom-built server with Nvidia A-series GPUs. “We were trying to do this fairly simply,” Jarosz confirms. “We had one render node which was also the control machine. It had an A4000 and an A5000 in it, which was able to power the entire screen. The screen wasn't particularly high res - I think it was 3K, but it was nice to be able to not deal with a whole cluster.”

“It was the first time I'd ever lit an LED wall stage,” Brown continues. “The great advantage is that you're getting a lot of light from the wall itself and you can tie in the environment, but in the first video, the sun was right above us and we were having to recreate that high key light. I kept showing my gaffer shots of what we’d done on location so we could match it.” The sunlit exterior, dominated by white snow and a bright sky, created what Brown describes unequivocally as “a high key daylight scene. We wanted to make sure we had a lot of light.”

While an LED video wall has far, far higher contrast than a conventional back projection onto a white screen, as Jarosz points out, the wall can still reflect ambient light. “You can look at LED test shoots and instantly see how experienced that production company is shooting virtual production, given how the dark parts of the wall look. You can tell if there's artificial light there, lifting that up. You can make the plastic around the LEDs as black as you want but ultimately most of the screen is LEDs. To combat it, periodically I'd black out the screens so you could maybe see a huge slash of light across it.”

The bright snow exterior demanded a lot of ambient light, and Brown describes a setup which involved his team in “a game of inches. We were getting out these huge twelve-by solids and tweaking them just enough so we didn't see the line between the fake snow ground and the LED snow ground, and we didn’t get any of that spill. It was just sitting there tweaking solids and taking the light off the wall.”

Exposure, as Jarosz goes on, is not necessarily a straightforward decision. “You want the wall to just be capable of clipping, but it's important to realize that an LED wall is not capable of saturating a modern cinema camera. You're only going to be getting eleven or twelve stops of dynamic range out of the screen. You're always going to be working with an information deficit. For a bright sunny day you might want to over-rate the camera a little bit so you're pushing those highlights farther up the response curve, so in 709 you're not getting grey highlights. You would do the opposite if you were doing say a darker scene.”

To put the finishing touches on the combination of video wall and reality, Brown chose Atlas anamorphics on the Alexa Mini. “The use of anamorphics was a very deliberate choice. I read a lot of interviews with Frasier, and commercial DPs, who'd used walls before. He'd being using LF cameras for walls so he was getting that really nice shallow depth of field you get with an LF sensor. LF was kind of cost prohibitive so I thought of using anamorphics to help with the blend and I think it really worked, it helped blend the location work and the wall extremely well.”

In the end, an LED video wall does not quite match the performance of Star Trek’s holodeck, but the potential for more convincing composites than any chromakey are clear. “For a bright sunny day,” Jarosz accepts, “it's never going to replicate a bright point source, it's never going to become a Fresnel. But it's very good at creating an environment. It was also nice, having shot footage on location, being able to match it. We had a ground truth, knowing how it was supposed to look. If something was a little off we knew it was on our end.”

Brown’s experience on the project promised much for the future. “What’s really great about this is that Andy has engineered so many of the things that are used on larger production, and we've been able to use it on a music video. It was a fantastic experience and it worked really well. I'm excited about the possibilities of this saving smaller productions a lot of money.”

Jarosz’s next project is the setup of an entirely new virtual production studio, Smash Virtual Productions. Brown’s enthusiasm is unabated: “Andy has a lot of very cool ideas for the demos we're going to shoot which will be the first prods at the studio and that’s what I’ll be shooting around May when it'll be open. I’m trying to involve more productions in that are shooting Chicago that are interested in the tech. I’ve been working on tv shows in Chicago for nearly ten years now and there are a lot of shows, coming here year after year, that would benefit.”