IP Security For Broadcasters: Part 3 - IPsec Explained

One of the great advantages of the internet is that it relies on open standards that promote routing of IP packets between multiple networks. But this provides many challenges when considering security. The good news is that we have solutions through IPsec (IP Security) to alleviate these concerns.


This series originally published in 2022. Security remains a critical issue for broadcasters so we are re-publishing it for those who missed it first time around.

All 12 articles in this series are available in our free 82 page eBook ‘IP Security For Broadcasters’ – download it HERE.


Articles in this series:


In their basic form, IP packets are sent in-the-clear across networks and the internet, that is, without any form of encryption. Anybody who has access to the network can not only read the packets, but also intercept them, change them, and resend them. The internet is effectively an untrusted network as we have no idea who has access to it and who is monitoring our data. For some applications this isn’t really an issue, but for others, it is. For example, a broadcaster will care deeply if unknown people can see, record, or tamper with their high value content when it is transferred over the internet.

Furthermore, the ability to intercept a packet and change its contents is particularly concerning as it lays the field open to viruses being embedded in the data packets, which in turn can be used to infiltrate IP connected devices. And we haven’t even touched on content being substituted for an altogether different message (channel hijacking).

Although broadcast networks are generally very secure as they are protected behind firewalls with network address translation functions, the real challenge becomes apparent when a broadcaster wants to stream media to other facilities across an unsecured network such as the internet.

Remote Security

Outside broadcasts require a network connection between the remote event and the broadcast facility. Using the internet is by far the most cost-effective solution as custom video and audio circuits or leased lines are often prohibitively expensive. However, connecting the outside broadcast to the studio through the internet is going to put the media streams at unacceptable risk.

IPsec is a collection of layer-3 protocols dating back to the 1990s that provide two fundamental functions: data integrity and confidentiality. Integrity guarantees the data packets haven’t been tampered with, and confidentiality stops unauthorized people from viewing and recording the data payload, and hence the streamed media. Although the data packets can be seen by anybody, the data in the payload can only be decoded by somebody possessing the necessary keys.

Virtual Private Networks (VPN) are one of the methods that use the IPsec protocols. Not only are they a tried and tested method of providing security, but their use is ubiquitous throughout the internet and has proved their worth.

Encrypting Packets

IPsec VPNs differ from SSL (secure socket layer) protocols as they work at layer-3 of the ISO seven-layer model, as opposed to layer-4 for SSL. In other words, the IP packets themselves are encrypted. Protocols such as SSL encrypt the data that forms the payloads of the TCP layer and hence the IP packets. Although this works well in server-client web page type exchanges, SSL proves ineffective for streaming high-value media as the IP headers are not encrypted leading to the possibility of man-in-the-middle attacks. This is dealt with using HTTPS (Hyper Text Transfer Protocol Secure) but brings a level of overhead and complexity that any broadcaster who is streaming low-latency media will not welcome.

Fundamentally, the IPsec VPN will stop anybody viewing the encrypted media or changing it, assuming the security of the keys is maintained.

Figure 1 – two private networks consisting of the cameras and microphones of the outside broadcast are connected over an IPsec VPN. Although the latency of the VPN is relatively low, it encapsulates each IP packet and encrypts both the header and payload to guarantee integrity and confidentiality of the streamed media. Even if somebody accesses the data in the internet, they will not be able to tamper with the data or view the streamed media.

Figure 1 – two private networks consisting of the cameras and microphones of the outside broadcast are connected over an IPsec VPN. Although the latency of the VPN is relatively low, it encapsulates each IP packet and encrypts both the header and payload to guarantee integrity and confidentiality of the streamed media. Even if somebody accesses the data in the internet, they will not be able to tamper with the data or view the streamed media.

Establishing a VPN is a two-stage process. The first part creates a virtual pathway over the internet between two trusted partners, such as IP routers, allowing private-public keys to be exchanged. These keys are then used during phase 2 to encrypt the IP packets so that any snoopers will only see seemingly random data.

To establish the first stage, the IKE (Internet Key Exchange) phase-1 protocol takes place by initiating a virtual connection. This relies on two trusted devices establishing virtual channels between each other using pre-shared keys. For example, if two Cisco routers are used, one at the OB and the second at the broadcast facility, they will have pre-shared keys that are known to both devices prior to phase-1 connection. A similar system occurs when connecting a laptop computer over a VPN to the office, the software running on the laptop will have a pre-shared key with the corresponding software running on the company’s on-prem VPN server.

During the IKE phase-1 sequence, the policies define the type of information that is shared between the two devices. This often includes the exchange of a second set of public-private keys that are unique to the broadcaster. These are in turn used to create the IKE phase-2 to provide full encryption.

IKE phase 2 forms the main operation of the VPN session by creating a second virtual channel and encrypts all IP packets including the header and the payload. To enable routers and other devices in the network to work with the encrypted IP packets, phase-2 operation adds an IP header to the encrypted packets, and this IP header is in-the-clear.

Changing IP Addresses

It might seem a bit strange that we’ve gone to all the trouble of encrypting the entire IP packet, including the header, and then added the same header to it without encryption. However, this is not always the case as the IP header addresses can change. In figure 1 there are two routers, one at the OB and one at the studio facility. Each of these will have its own IP address for the connecting link to the internet, and it is these IP addresses that are used in the additional IP header. This adds a further level of protection as anybody snooping within the internet will only see the router IP addresses and not the studio or OB cameras and microphones.

Even if the added IP addresses are the same as those in the encrypted IP packet, only devices that have the necessary keys will decode the packet, compare the header to the appended header and, crucially, authenticate the validity. Unauthorized users, however, are unable to change the encrypted header, and so matches between the encrypted and public headers are highly unlikely.

Also, if the header added in phase 2 has been tampered with then it’s unlikely to reach the correct router and will be discarded by other internet routers as they will not have the public-private key that was provided in phase 1 to decode the original IP packet. And if anybody sniffs the packet then they will see the additional header with a payload full of seemingly random data, that is, the encrypted media stream. Only trusted devices that were established during phase 1 will be able to de-cypher the media.

Compliant Routers

This whole process relies on the VPN-enabled routers at both the OB and broadcast facility being correctly configured. Phase 1 is initiated as soon as a media stream with the relevant IP address is sent to the broadcast facility as the OB router will detect the routing and instigate the VPN phase-1 and phase-2 sequences. Phase 2 stays active for as long as the media stream is being sent to the broadcaster and will only close after a predefined time of inactivity, or an initiated VPN close sequence, thus keeping latency through the overhead of VPN setup times relatively low.

The fundamental method of operation when working with the internet is that we assume anybody can see our data and anybody can alter it. Both have potentially serious issues for broadcasters, but they can be easily alleviated by using security protocols such as IPsec VPN as well as two-factor authentication to avoid issues with compromised passwords. This means media streams are free from being viewed by unauthorized people, and we can be sure that the media that was sent at one end is the media that was received at the other.

And just in case you wondered how IPsec-encrypted packets manage to get through firewalls: IPsec traffic usually uses UDP (User Datagram Protocol) to set up dedicated connections, rather than TCP (Transmission Control Protocol).

Part of a series supported by

You might also like...

IP Security For Broadcasters: Part 4 - MACsec Explained

IPsec and VPN provide much improved security over untrusted networks such as the internet. However, security may need to improve within a local area network, and to achieve this we have MACsec in our arsenal of security solutions.

IP Security For Broadcasters: Part 2 - The Problem To Be Solved

By assuming that IP must be made secure, we run the risk of missing a more fundamental question that is often overlooked: why is IP so insecure?

IP Security For Broadcasters: Part 1 - Psychology Of Security

As engineers and technologists, it’s easy to become bogged down in the technical solutions that maintain high levels of computer security, but the first port of call in designing any secure system should be to consider the user and t…

Designing IP Broadcast Systems - The Book

Designing IP Broadcast Systems is another massive body of research driven work - with over 27,000 words in 18 articles, in a free 84 page eBook. It provides extensive insight into the technology and engineering methodology required to create practical IP based broadcast…

The Big Guide To OTT - The Book

The Big Guide To OTT ‘The Book’ provides deep insights into the technology that is enabling a new media industry. The Book is a huge collection of technical reference content. It contains 31 articles (216 pages… 64,000 words!) that exhaustively explore the technology and…