Once upon a time the introduction of a new product or piece of gear did not require the total replacement of the previous version. It was common to have different generation cameras, switchers and even multiple types and formats of recorders. While I am not ready to say those were the good ol’ days, they did feel more comfortable than what often happens today. Today when a device goes from working to dead you may be faced with no hope of resuscitation or even an equivalent or compatible replacement.

I was working recently on a project where the systems were barely four years old.  One of the video processing cards failed in a rack-mounted cage. It should have been no big deal, replace the card and move on. Instead it turned into a nightmare.

When contacted, the vendor said that not only was the card obsolete, they had none in stock. With a bit of persuasion, the company did find a similar card lying around somewhere but the company no longer had the ability to test it to be sure it even worked.

No problem says the vendor, we have a new replacement product. It’s the next generation version and it is even more feature rich. Yes, that may be true, but in our case, the card form factor had changed and required a whole new frame and different connections. The client has six of these cards and now they all have to be replaced along with a new frame and controller. The issue did not end there. The device cards are in the mission critical transmission chain and the new version cards will change the interface with the automation software.

This started me thinking. What other facility devices and components might also be near end of life and could result in similar issues? 

After a review, the first thing we did was order what I am calling critical care spares. These are spares for items that are or about to become obsolete and without an upgrade path. In this instance, the review discovered an additional three manufacturers that have changed technology and form factor for equipment that the facility uses daily. That may be the easy step. Don’t forget such purchases may be outside regular maintenance or procurement budgets. Such a dilemma represents a new type of Pandora’s box for engineering departments everywhere.

Plan For These Events

One of my missions at NAB 2018 was to meet with each manufacturer and ask which of their products might be reaching end of life and is the next generation backwards compatible. Will there be conflicts between the two versions of firmware? I also asked about whether the management and monitoring software currently in use would recognize the new products. If not, that represents another tier of investigation and solution development.

Once you realize the potential for a crisis, the next challenge is determining when and how to fully change out an older technology. This usually requires a multistep solution.

First there’s the system and operational aspects of change.

• What services or air operations does this disrupt?
• What other systems are interconnected and how will they be impacted?
• How much needs to be taken off line to install the changes?
• How can they be bypassed or taken offline?
• Once installed, how is the new product configured and put into service?
• Does monitoring and management change?
• Do operators need new training?

Then there’s the physical aspect of change.

• Where is the new product being installed?
• What does it connect to?
• Are the pathways accessible?
• Can it be patched around or rerouted in order to bypass the device?
• Is it physically accessible? What about rack congestion?
• Is the new product the same form factor as the replacement?
• Are the physical connections the same, what about interfaces?

What if it’s software?

• Does it run on the same computer platform and OS?
• What about interfacing with other systems?
• What configuration changes may be required?
• Do the old configurations migrate?
• What about drivers and middleware?
• Are there any API’s involved?
• Do IP assignments or mappings change? If so, don’t forget to document the changes.
• How will the cutover to the new software be accomplished?
• Does the solution work the same way? Is the user interface the same?
• What about training?

Manufacturers are only too ready to promote their new products and services. But it would really be helpful if we also got notices announcing what products are being discontinued and if any of them may be incompatible with previous versions. Even with the top of the line SLAs we had in place and a steady stream of interaction with company support, somehow we were not informed when the product discussed here hit the end of its lifecycle.

Plan for Changes

Imagine what this does to planning and budgeting. It’s one thing to replace a card as a maintenance item. It could be a much larger step to have to buy a new cage, multiple new cards and connectors. You get the idea. Such costs could be well above and beyond what was initially included in the maintenance budget for a few shelf spares.

In the article, When Technology Sunsets, I wrote that version management requires a strategy. All updates are not created equal. Some updates fix bugs or issues. Other updates may add features or, based on the existing version, it may not be necessary to update. Updates to the operating system needs to be managed as well.

All these factors mean it’s time to make equipment lifecycle management a part of regularly scheduled maintenance. Regular upkeep needs to include checking with vendors on product changes, roadmaps and product lifecycles. Only then can you determine if there is the need to warehouse critical spares before they become unavailable. Such planning can protect you from being forced into making critical, and unplanned, changes based on a vendor’s business decisions, and not a device’s lifecycle.

Then there is the possibility that the technology vendor you selected is acquired and some of their products are not on the new company roadmap. Or, what will you do if the company just choses a different business direction and stops making the product?

Finally, I’ve discovered that as companies consolidate/merge their support system changes. Read that as equipment spares and people may go away. Institutional memory or legacy product familiarity may no longer exist.

NAB and other trade shows are great places to see new stuff, get some answers to technical questions and build a base of information to make good decisions. But it is also important to identify what products are obsolete and no longer available before the you-know-what hits the fan and you have no backup.

Editor’s Note: Gary Olson has a book on IP technology, “Planning and Designing the IP Broadcast Facility – A New Puzzle to Solve”, which is available at bookstores and online.