An unexpected technical crisis is a pop-quiz that nearly every broadcast engineer gets to take at some time or another. The response and its outcome can determine an engineer’s broadcast career future. There are no letter grades. A technical crisis is a public pass-or-fail test likely to occur when least expected and the most people are watching.

Sure, electronic and mechanical equipment can fail anytime, but broadcasting equipment tends to fail at the worst possible moment. I once rented a large portable generator to power a live 4^th of July evening multi-camera philharmonic concert TV production. The generator blew a brush the instant the first live cannon blast in the 1812 Overture exploded. Everything plugged into the generator went dark and the crew didn’t know what to do. Neither did the Engineer-in-Charge (EIC), who happened to be me.

When the microwave link signal vanished, the station switched to the news studio. The crisis control was unprepared news anchors tap-dancing profusely until the station gave up and joined network programming in progress. Meanwhile, other local stations simultaneously aired their successful live coverage of the same event. I didn’t have a Plan B if the generator failed and was nearly fired for it the next day. Suddenly, I gained a new appreciation of alternate plans and live TV crisis control.

Failure magnets

Similar important live TV events, such as local evening newscasts during a ratings month, or national sports championships, are known failure magnets. Broadcast engineering veterans know that surprise is the karma of live TV. The more important the show, the stronger the karma. Trust me, trust nothing else, and be prepared for surprises.

What’s the difference between working a Saturday morning children’s programming shift in Master Control and being the Engineer-in-Charge at a major live TV production? Other than vastly different venues and glory factors, not much. When all is going perfectly well as anticipated, live TV can be fun and easy.

When something fails in either scenario, live TV can turn into an instantaneous crisis without warning. The trick is to keep a crisis from becoming a disaster or a legendary meltdown other stations report on their news. Stay under the radar. Always work towards minimum discrepancies and disruption.

Crisis control

If viewers and sponsors can’t spot anything out of the ordinary, a controlled crisis can remain hidden behind the curtains. Like a magic act, the key is to keep the eyes of the audience focused away from where the magic actually happens. As the Wizard of Oz famously said, “Pay no attention to that man behind the curtain.”

Like magic, TV is often an illusion too. For example, TV viewers may see a well-dressed TV presenter behind a desk wearing a nice sport coat, never knowing that person is also wearing loud shorts and dirty sandals. Similarly, viewers won’t notice a catastrophic production switcher failure if a hot back-up parallel production switcher automatically kicks in.

A flexible director can shield viewers from seeing a problem camera during a multi-camera production, unless the talent points it out on the air. Talent must ignore all but the most grievous most technical problems because like a magic act, live TV success depends on production secrets remaining secret.

Avoiding Plan F

The strategy for live TV crisis control is to identify Plans B, C, D, and E and prepare them ready for immediate, perhaps even automatic deployment, should Plan A fail. Don’t go to Plan F. The F stands for “Finished.” I skipped directly to Plan F when the live philharmonic concert rental generator filed.

Live TV is show business and as such, “The show must go on.” Live TV has also spawned some unique colloquial phrases like “It’s leaving here okay,” and “Don’t make it look like a mistake.” If everyone on the crew abides, and the show gets to the audience without perceptive blemishes, the show will have succeeded technically, even if it wasn’t all Plan A.

Broadcast television and the technology that supports its creation and delivery is supposed be invisible magic, and we’re the magicians. Beyond that, content is king. It’s also the point where all broadcast engineer’s magical powers stop. Technology can’t fix or camouflage bad content, no matter how slick the show appears.

Staff or crew?

Radio stations and sound studios have staffs. Live TV, film and stage productions have crews. An individual radio station staff member can make and be a one-person live show. Very few successful live TV productions are one-person shows. Except for a handful of breaking-news internet livestreams and some how-to YouTube videos, it takes crew teamwork to make compelling live television that viewers want to watch.

Live TV isn’t a committee meeting or a group discussion. The director tells the talent and crew to make certain things happen, and the EIC keeps the technology going that makes the show possible. Technical questions are best asked and answered well before a live TV show begins.

Whether it’s one person managing multiple TV channel playout operations from a master control room during Saturday morning children’s programming, or a World Series TV crew covering a prime-time game, successful live broadcasting needs individuals with specific technical skill sets and a powerful can-do drive. I’ve never met a successful live TV crew member who didn’t have these can-do characteristics.

It’s not Club TV and not everyone is always welcome. 

The EIC

Crews need technical support, and the EIC is the boss of the technology and those who operate it.

During the heat of live production, trait 2 can be the most difficult. Self-control separates great EICs from the rest.

One of the best ways to get the most out of a crew is credit on the credit roll. Behind-the-scenes credit identification could bring in future cash. “Who was that great camera operator?” some future employer might ask. Great EICs make sure everyone is listed in the credits, and that the credits are easily readable. High speed credit rolls insult the crew and viewers.

Torpedo alert

A torpedo is an unplanned problem that can be avoided with a workaround. The middle of a live TV production is the absolute worst time and place to attempt fixing broken equipment. Typically, the risks far outweigh the benefits of mid-show, “roadside” repairs.

The EIC needs a variety of easy workarounds ready to implement, should an operator or piece of equipment become unable to perform. In addition, the EIC must quickly convince the producer and director that the work-around will be virtually invisible, and prove it.

The EIC must also say “No” when asked to do something that might increase the risk of production failure. For example, the day of the show is not the time to upgrade firmware or software, or to fiddle with internal settings. Saying no and sticking with it is probably the most difficult aspect of being the EIC. Something working “good enough” until the show is over is often the safest approach.

The job for most crew members is to tightly focus on every detail of their specific function. Only the Engineer-in-Charge (EIC) can focus on the big picture. The world’s greatest TV gear operators can’t succeed with broken gear. The EIC verifies that all equipment is upgraded and working properly as a system, days before the show. Sometimes, pre-show preparation may call for model-specific operator training and rehearsals. Live TV is not the place for on-the-job training.

N+1

When something fails during the live show, the EIC needs a ready back-up plan. If the budget can afford it, N+1 redundancy is the best Plan B. N is the number of critical technical items such as cameras, mics, switchers and modems the production calls for. The “plus 1” is an unused, on-site, ready-to-go, working spare replacement for each item. If something breaks, wire in the “plus 1” back-up and nobody will notice.

From extra mics and cameras to extra encoders, IP switches to production switchers, having the right spare in the right place at the right time can literally save a live TV show. The audience will never know, unless somebody reveals how the flawless TV magic was really done.