One of the problems with both filmless movies and paperless offices is the sheer complexity of interactions between organizations. A sheet of paper is primitive, but almost universally comprehensible in much the same way as a piece of 35mm, 4-perf film footage. Replacing those primitive but universal things with computerization has brought convenience but also complexity. No longer, for instance, can we take any piece of footage to any device and play it back, at least not without a certain amount of setup. Once, the only common choices were flat or scope; now, we must be concerned with resolution, color-space, codec, file format and flat or anamorphic, and even “anamorphic” can mean any of a few things in 2021.

This sort of compatibility concern is a microcosm of the wider issues of coordinating production and post workflows that can easily involve dozens of pieces of software across dozens of companies operating in dozens of countries, doing hugely complex things involving hundreds of people at every stage from location scout to cinema exhibition. The only reason that our current approach is in any sense acceptable is probably because computers make moving and converting media so easy. Original camera negative, in the end, cannot be losslessly backed up; data can. That doesn’t mean there aren’t ways to regularize the often rather ad-hoc way we make movies, though, and that was the subject of a seminar presented by MovieLabs as part of the Hollywood Professional Association’s virtual retreat.

Addressing Issues

MovieLabs is a collaboration between Paramount, Sony Pictures, Universal, Disney and Warner Bros designed to address a range of technological and organizational issues that surround film and TV production. The presentation’s subject was the concept of software-defined workflows, something that MovieLabs’ literature describes as “[supporting] human creative tasks by connecting them to the greatest extent possible through software-mediated collaboration and automation.” That in itself might sound like the title of a truly nightmarish PowerPoint stack, but in practice it might mean some very useful regularization of the increasingly chaotic way that film and TV projects exist across a wide variety of organizations, tools and media resources, particularly given the increasing popularity of off-site storage and computation services – the cloud.

Jim Helman is a technologist at MovieLabs whose background includes standardization of HDR video, web content delivery, security and intellectual property, and who holds a Ph.D. in Applied Physics from Stanford. His experience includes software engineering for Silicon Graphics and consultancy for companies including Comcast, NEC Research Labs, and (latterly-Xerox) PARC, without which organization modern computing might look wildly different. It’s hard to imagine anyone better-qualified to be involved with an initiative designed to regularize the way film and TV production workflows operate.

Going Cloud

The idea, Helman says, is to address both the huge variability of modern workflows and the move cloudwards. He begins, “On one hand you have the complexity and variability of production workflows, but on the other as people move their workflows to the cloud there’s an opportunity, and in some cases a need, to improve the software that supports collaboration.” Here, he mentions something that’s come up in almost every industry but which particularly affects film and TV where data loads may be big enough to make fully-online working tricky even today: “Covid has forced people to do a bit of experimentation, and some of it may have been a bit painful because the tools are not there. Your experience with even a powerful editing bay in the cloud was often not optimal in terms of its latency and responsiveness due to limitations in home internet connections.”

Software-defined workflows are part of MovieLabs’ 2030 Vision initiative. The initiative lists ten high-level principles of which cloud storage is number one, though it quickly becomes clear that while the cloud is likely to be a huge part of post at all levels by 2030, the concept of making workflows more consistent and understandable is relevant either way. Other principles address security, versioning and iteration, and there’s a great push for consistency in terminology.

As Helman puts it, “There's a common ontology and vocabulary definitions. At the lowest level, it's trying to get people to talk about the same things in the same way. If they're exchanging data or communications there's some interoperability around, for instance, what's a shot, what's a slate, what are the attributes of a sequence.” Here, he uses a word – ontology – that might seem more complicated than it is. Strictly, it’s the study of things that exist, which sounds trivial. It’s hard to create a good scheme for productions to describe and define their workflows, though, without first defining all of the pieces.

Codifying Approach

And – at least at this stage – the initiative is about creating the scheme, creating the standard. It’s not, or at least not entirely, about creating software to implement that standard. Persuading the entire production industry to adopt one suite of tools is, fairly instinctively, not a practical thing to do, and so MovieLabs’ idea of software-defined workflow is not an application, it’s a codified approach to doing things. That’ll be a relief to anyone who might have been concerned that the concept might have involved a monolithic, doubtless expensive software suite to which everyone would have to convert, the sort of thing that’s often a barrier to adoption of otherwise well-designed standards.  

Helman confirms that “we're not implementing a platform. That's going to be up to the software vendors, facilities and companies offering production as a service… it's not that someone who's building something to move toward this vision needs to adopt exactly the same [software standards] or exactly the same comms systems, but you want to make it so it's easier to connect those things - to build the shims that will let, say, Shotgun talk to Avid or whatever.” Helman suggests that “there might be some open software components. It might be useful to have reference implementations of some components. The Academy Software Foundation has a number of projects, mostly VFX-focused. They do have a broader mandate than that, though, so there might be open software initiatives [involving] some of the companies which have contributed a lot to the ASWF.”

The sort of code written by a visual effects house might, we’d expect, be about image processing, but the crux of software-defined workflows is much more about organizational tools. For example, he continues, “A lot of workflows are very event driven. Things like messaging systems can propagate events to different components in workflows.” The standardization process might, for instance, involve deciding “what a message that might be part of an approval process looks like and what happens as a result. When something is approved, and the asset doesn't move [because it’s in the cloud], the access controls now need to change because more entities are now authorized to access those assets.”

Considering Security

With that sort of authorization in mind, anyone who’s visited an upscale postproduction house might have noticed signs promoting a dedication to some sort of security protocol. This is sometimes of more importance to nine-figure comic book movies than it is to a fly-on-the-wall documentary, but any architecture intended to formalize workflow design needs to be able to satisfy the sort of big hitters who sponsor MovieLabs’ work. As with so much of the initiative, the approach is necessarily general, to cover the huge variety of situations that any sort of production might bring up. As Helman puts it, “the security architecture stuff we put out is high level blocks. [We describe] how they interact. That makes it easier to plug in an application, or a different workflow component, without having to do significant software development. Ideally things would be more plug and play, but at least there's a place you can put in a translation layer or service that would let you bridge some of the different applications.”

As to the cloud, well, keeping files safe on a server in the basement is one thing; putting those files on an essentially unknowable, remote server farm might make them feel less secure. Given that most cloud services providers take great pains to make their datacenters both hard to find and physically robust against unauthorized access, the main issue is of securing the places where authorized access takes place, not of securing the server itself. “The physical security of an Amazon AWS datacenter is not a current concern,” Helman confirms, “but you still need to secure your endpoints. If you're working on an editing terminal that's using an editing instance in the cloud, you still need to have that location secure to make sure nobody is recording the screen and posting it up on YouTube.”

It would be easy to think a formally-defined workflow might apply only to the most upscale productions, where the technology is most complex and the number of people and processes is largest. Thinking again, though, it’s in the context of cloud services where smaller shows, perhaps preferring to rely less on on-premises equipment, might have most to gain. “If people do offer production-as-a-service platforms,” Helman says, “that'll be used more by smaller productions that don't have infrastructure, and don't have software development teams. You may still have the Marvel [movies] building a lot of their own stuff because they have very specific workflow requirements, and security requirements.”

Software Defined Workflows

At the time of writing, MovieLabs’ concept of a software-defined workflow was reasonably new and necessarily general, a set of ideas – albeit well-described ideas – and a corresponding set of white papers describing the general approach, security, and even a set of flowchart iconography to allow workflows to be diagrammed without ambiguity. These are ideas which might one day create a system that film and TV people find second nature, and which might save a lot of time and effort in mistaken requirements and duplicated effort. As with any grand standardization project, the issue will be uptake. The grander the plan, the trickier that tends to be, and this is certainly a plan which could become very grand indeed.

Also, regardless how important cloud infrastructure might eventually become over any given timescale, Helman acknowledges that physical facilities are likely to continue to exist, and that even in a largely cloud-based ecosystem, compatibility remains an issue. “There are always going to be facilities,” he says, “and they'll be in different clouds, or they'll be on-prem. We're not mandating that there be a single system on the cloud that a studio, or a studio vendor, deploys, and everyone must come to that. It's more the aspirational direction that we think would benefit the industry to adopt, as the various companies that provide these tools reconfigure and their toolchains to go to the cloud. A lot of the major tool providers are all doing, or working on, more cloud native deployments of their toolsets.”

Either way, it’d be nice to think that the current state of digital cinema production – from lens to screen – is about as mature as sound recording was in about 1930s. Hopefully, the slightly chaotic status quo will one day be looked back on with amusement at how disorganized it could sometimes be. Anyone who’s made an unexpected, late-night cross-town dash with a backpack full of hard disks, to catch a dropped ball, will understand the need for a more regular approach. It’s hard to imagine what Dénes Mihály would have thought of our current filmless movies, but it’s equally hard not to hope that something like MovieLabs’ software defined workflows might make them just a little less perplexing.