An experienced engineer-in-charge (EIC) of many live, multi-camera field productions I learned from used to say “I like to get everything wired and working two days in advance. That way we can relax the day before the show while confirming, tweaking, rehearsing and bragging about how great it looks.”

When I was young, that technical rehearsal day was a great plan, but seldom did or does it work out that way. Random, production issue surprises typically wait until a day, hours or minutes before a multi-camera live show broadcast begins, or as the show is in progress. Some show-threatening issues defy troubleshooting logic. Live TV Karma explains the extra day and is the focus of this The Broadcast Bridge story.

It Worked Great Until It Didn’t

Rule #1 of Live TV Karma: Mission-critical things break at the worst possible time. ‘Things’ can range from unstable on-site internet and cellphone service to a bad cable or connector.

I recently directed and engineered a live, streaming 6-hour production of a local “Candidate Forum.” It featured a couple of dozen political candidates answering questions and explaining their positions. The audio was simulcast on a local AM and FM station. My job was to provide a multi-camera, streaming TV feed of the event, making it more than a radio-only broadcast. We had three cameras, preproduced graphics, and 9 mics on a lit stage.

We set up for the Wednesday afternoon show on Tuesday morning and by that evening, everything was working perfectly. We left the gear powered up, locked up and confidently went home. I expected to check in early Wednesday morning, relax and brag about the great pictures until after lunch because the show didn’t start until 3 p.m.

Instead of a good morning, I was greeted by an unresponsive production switcher control surface, and the usual computer keyboard switching shortcuts produced strange results. I was dead in the water. Plan B was one camera, no graphics, and not where anyone wanted to go. Nobody had entered the room or touched anything, but somehow a mission-critical problem had developed overnight. Not good, but typical. I didn't tell anyone. It was TV Karma I thought, and I was about to reboot anyway.

Midnight Gremlins

Rule #2 of Live TV Karma: Don’t panic or let viewers know there is a problem.

About a dozen or so reboots later, the switcher was still in trouble and the clock and too much coffee were ramping up the pressure. My radio station friends were nervous. Me too, but I tried not to show it. The problem appeared to be hardware, but what? There are only three hardware components including an undisturbed cable. The production computer works but acts funny but doesn't appear to have rebooted, and the control surface worked last night but doesn’t work or light up at all now. Connecting the USB cable lit a blue pilot light inside the control surface unit, but none of the push buttons lit or did anything.

My radio friends were watching me get nowhere and beginning to panic. They wanted to reload the software, search for updates and new drivers and call tech support, to save a live show with a live studio audience that begins in less than 5 hours. At this point, I knew messing with software and drivers would probably tank the show, but it was a problem we couldn’t tap-dance around. If we couldn’t make it work, we were going to face a roomful of disappointed politicians and live audience members, and a streaming, marketing and branding disaster.

At this point, “we” became me. Sound familiar?

It was too late do anything to the production computer or expect any outside support to help us in time. All I knew is that it all was working together fine when I left the night before. What could have happened? I later discovered what, but to this day do not understand how, which is precisely what Live TV Karma is all about.

Nailed it

“For Want of a Nail” is a proverb dating to the 14^th century, made more famous by one of the US Founding Fathers, Benjamin Franklin. His version said: “For the want of a nail the shoe was lost, for the want of a shoe the horse was lost, for the want of a horse the rider was lost, for the want of a rider the battle was lost, for the want of a battle the kingdom was lost, and all for the want of a horseshoe-nail.”

In the case of the “Candidate Forum” live stream show, it was beginning to appear that an odd Type-A to Type B USB cable was our “horseshoe-nail.” I was out of other ideas and time. But, how could a working USB cable go bad overnight, untouched? It can’t, I thought and I was wrong.

We bring boxes of gear, electrical cords and an ever-expanding cable and adaptor collection to every production. It happened that a similar but used USB cable was in the cable collection. Was it good? Where did it come from? Would it fix the problem, or would a second bad cable mislead me?

Lucky for everyone, the replacement cable worked. The problem was fixed, the bad cable was tossed in the trash, the production streamed as advertised, and few were the wiser. Nearly half the population in the market watched some part of the stream. Only the producers and I knew how close it came to being a disaster. A replacement for the failed USB Type-A to Type B cable that almost killed the show sells for less than US $6 on the internet.

The Rush

Rule #3 of Live TV Karma: Part of the fun is the adrenaline rush. You may have your own list of similar live, multi-camera shoots you’ve sweat through. If you can’t handle an occasional adrenaline rush, live multi-camera TV production is probably not for you.

Every live, remote, multi-camera production I ever worked always had a last-minute technical issue or two on-site, at the studio or with the link in between. Few problems are ever seen by viewers. The crew may not even know of problems. TV Engineers are like a magician’s stagehands; they make the magic work and don’t talk about it. Some broadcast engineers live for the rush. Its fun being a hero, particularly when it’s a secret.

Karma Control

Rule #4 of Live TV Karma: Broadcast engineers can control some, but not all, Live TV Karma. That control begins with being prepared with spare cameras, mics, monitors, power sources on-site. In this case, an unusual spare USB cable on-hand saved the day. This approach is sometimes called N+1, with N = the number of items you think you need, plus one more because you will likely need it at some point.

Control continues with meticulous set-up, testing, monitoring, and more testing end-to-end to confirm that every device and person is talking and hearing as planned. Control concludes, one way or the other, with well-thought out alternate plans for every potential failure you and the crew can think of. There are many more Live TV Karma rules than these four, and they manifest at the worst possible times. Count on it.

My Big Bang

I once was the EIC at a big local summertime local TV event that included fireworks and the city philharmonic performing Tchaikovsky’s 1812 Overture live, complete with actual cannon fire. All the local TV stations were there, broadcasting their coverage of the event live on their channels, hosted by their news anchors on-site. It was huge.

I rented a large generator to power the lights, cameras, and the station production truck because no on-site power was available. The venue was a large city park. Setup went well, we were on the air on time, and looked great until the exact moment of the first 1812 Overture cannon blast, when the generator output instantly died as did everything electrical including the microwave link to the station. The only working electronics were battery-powered walkie talkies with screaming from the newsroom and Master Control.

Our competitors brought their own generators, and nobody had any spare power, cables or time. The show was immediately over for my station, as was the adrenaline rush and apparently my EIC future. My channel joined network programming in progress and the news anchors slid back to the studio.

Returning the failed rental generator the next day, I learned that a worn brush was the point of failure. Apparently, the shock wave from the cannon blast finished off what would have failed a few minutes later anyway. With that knowledge, I begged and pleaded with the station GM and was lucky to keep my job.

The GM said it wasn’t my fault, but we both knew it was. I should have demanded to inspect and replace such likely failure points before I rented it. Prime-time live TV is no place to take chances and it took a big embarrassment and a bigger GM for me to learn that lesson and survive.