Bahrain’s Ministry of Information recently went live with an IP-based multi-channel playout system, setting a precedent as the first government entity in the GCC to implement the technology. In an exclusive interview with Vijaya Cherian, Eng Abdulla Ahmed Al Balooshi, Assistant Undersecretary of Technical Affairs, who spearheaded the initiative, and systems integrator First Gulf Company share more details.
Bahrain’s Ministry of Information (MI) has been on an ambitious upgrade path over the last couple of years, replacing old infrastructure with cutting-edge technologies to forge a futureproof course. Its newest project, the migration to IP-based playout and SMPTE ST 2110, will enable Bahrain TV’s five channels to streamline their workflows and provide system flexibility to work in UHD and HD using the same infrastructure. The $4.5m project was designed and executed by Saudi systems integrator First Gulf Company (FGC), in conjunction with key broadcast vendors.
Al Balooshi
“Bahrain is always on the lookout for new technologies that will help it remain futureproof,” says Al Balooshi. “This is the future, as all the new media now require an IP signal. In Bahrain TV, we use the same content but adapt them for our different platforms. So if your base is IP, this integration into new platforms is relatively easy. If you have a normal SDI or any older technology, you need to convert your signal, and that brings multiple challenges as you convert signals from platform to platform or from technology to technology and could potentially lead to loss of quality, especially on the graphics side.
“But IP also brings multiple challenges with it. Vendors often claim that they have a complete solution, but our journey has not been without its challenges. The technology is still relatively new even for the systems integrator. When integrating a lot of different systems and products from different manufacturers and also having to work with some of your existing technology, it calls for thinking out of the box, collaborating more closely with all the partner vendors and coming up with solutions. I think that’s what makes this project rewarding.”
The main mandate for FGC was to create a UHD/HD multichannel playout facility that is futureproof, scalable, resilient, efficient in streamlining the playout workflow and flexible in choosing the transmission resolution format of the TV channels.
“To achieve the specified multicast flow redundancy, FGC adopted an IP network based on ST2022-7 with redundant streams of IP/RTP packets that allow for the creation of a single reconstructed output stream through hitless protection switching,” explains Joe Chbat, Executive Vice President, FGC.
The system has been designed for six playout channels, with each channel having its own chain consisting of Marina automation, main and backup playout and preview ports, Viz multichannel/ticker/engine, Masterpiece MCS, and video and audio monitoring.
The broadcast IP playout system comprises several key components for seamless operation. At its core is the Pebble Beach Marina automation system, boasting fully redundant device controllers, housekeepers and SQL servers to ensure uninterrupted performance. Supporting this infrastructure is the Harmonic Spectrum-X playout redundant server system, offering 10+10 playback ports and eight ingest ports compliant with ST-2110 and ST-2022-7 standards. Storage needs are met by the Harmonic Mediagrid-4000, providing 112TB of usable nearline storage.
Network management is facilitated by the Grass Valley GV Orbit dynamic IP orchestrator, alongside the Arista 7280R3 switch boasting 96x100GbE QSFP ports. The system also incorporates Vizrt multichannel and ticker systems to efficiently manage content across six channels. Rounding out the setup are Grass Valley Masterpiece MCS, SDI<>Gateways and IP multiviewers MV- 821-IP for comprehensive monitoring and control capabilities. Together, these components form a robust broadcast solution tailored to meet the demands of a modern media environment.
As part of its installation process, Bahrain TV migrated its Pebble Neptune system to the latest iteration of Pebble’s enterprise-level automation, enhancing its capabilities for robust multichannel delivery. The ST-2110 solution revolves around Pebble’s enterprise automation, which efficiently oversees both Pebble Integrated playout channels and third-party servers, mixers and routing devices within a unified system. This allows simultaneous playout of six channels, with control over six main and six backup server ports, six mixers and six CGs.
The ingest functionality can handle six simultaneous feeds per server, enabling parallel ingest to both main and backup servers. Furthermore, the solution offers playlist preparation tools, a preview list and Junction preview capabilities. It also facilitates automated file transfers between video servers, nearline storage and the LTO archive, streamlining media movement. The Pebble systems were seamlessly integrated with top-tier components in Bahrain TV’s workflow, ensuring a smooth transition without disrupting viewer experience during the component swapping process.
One top-tier component worth mentioning is Vizrt Multichannel, which facilitates playlist updates and triggers on-air graphics, thereby simplifying playlist playout across multiple channels through a single client interface. It is the central control hub for managing standard graphics such as logos and now-next-later information across all six channels through a single interface.
API integration between Multichannel and Marina automation enables seamless communication. Page names are incorporated into the schedule as secondary events, along with the necessary data. Multichannel retrieves the playlist from the automation system. Secondary events are grouped within primary events within Multichannel.
Furthermore, through MSE (Multichannel Scene Editor), Multichannel triggers graphics on the Viz Engine based on trigger information received from the automation system. While the automation system has complete control over Multichannel, operators at the Ministry also have the option to manually trigger graphics on the Viz Engine via Multichannel.
The core network infrastructure has been deployed with a redundant ST2110 blue and red IP network architecture specifically designed to accommodate the capacity requirements of the UHD/ HD playout system’s future expansion. This infrastructure is fully compliant with ST2022-7 path redundancy architecture, ensuring resilience and reliability. The backbone of this network infrastructure features Arista layer 3 non-blocking switches, providing a substantial bandwidth of 9.6Tbps. These switches are integrated with the GV Orbit control and orchestration system through API integration, streamlining network management and operations.
Arista 96-port 100G switches serve as core switches, capable of adapting to future spine and leaf topology configurations. Endpoints are directly connected to redundant core switches, forming the red and blue networks. Data aggregation Arista PTP-aware leaf switches are employed for management, control and grandmaster clocks, further enhancing redundancy and reliability.
Network synchronisation is ensured by two GPS-locked PTP grandmasters from Telestream, using the best master clock algorithm. Arista switches configured as boundary clocks ensure precise synchronisation with end points. Network orchestration is managed by GV Orbit through API integration with Arista switches and IS-04/IS-05 protocols, facilitating seamless data flow orchestration to third-party endpoints.
To support media-related operations, a separate media network is provided, featuring HP Aruba 5400 switches dedicated to playout servers, Media Grid storage, graphics and other production devices. This segregation optimises network performance and resource allocation.
Redundancy and failover management in the IP-based playout system has two levels: component and path redundancy. Component redundancy mirrors traditional SDI systems, duplicating critical components to ensure system resilience in the event of device failure. The system also leverages hitless protection switching technology (ST2022-7) for path redundancy. This approach involves sending two multicast streams from one endpoint to two separate networks (red and blue), mitigating signal disruption in case of network congestion issues.
Integration of the IP-based playout system with existing broadcast equipment and workflows involves IP gateways that facilitate the conversion of SDI signals to IP and vice versa, enabling seamless interoperability. A total of 128 inputs and outputs connect the ministry’s existing main SDI routing matrix and IP gateways. Once signals are converted to multicast IP streams, they become accessible as sources for any channels, supporting live or ingest applications. Furthermore, all six playout channels’ programme outputs are available in SDI format for distribution and transmission, ensuring compatibility with existing workflows and equipment.
One of the big challenges in terms of compatibility and interoperability was the communication protocol between devices. The ministry’s 2110 IP playout system boasts advanced monitoring capabilities, ensuring superior QoS levels and precise synchronisation of IP/RTP data streams. “We had to make sure that the implementation of communication protocol subsets was consistent across both devices, even if they share the same primary communication protocol, to ensure proper communication functionality,” says Orlando Castro, Chief Technology Officer, FGC.
Chbat explains: “FGC and our technology partners were particularly focused on the design and deployment of the system synchronisation compliant with SMPTE 2059, which is considered the heartbeat of any ST2110 installation. Redundant Precision Time Protocol (PTP) grandmaster clocks locked to GPS signals that are configured to work on best master clock algorithm (BMCA) were implemented in the design of the media network architecture. To avoid system timing issues, we also deployed a proper PTP design to ensure precise synchronisation of the various IP/RTP data streams in an ST 2110 system. As a result, we have achieved error-free 2110 streams with almost no latency.”
Balooshi adds: “Currently, with six fully redundant playout channels, less than 30% of the switching capacity of the core switch is being utilised. The playout system is scalable by simply adding SFPs to the unused 100G ports of the core switches for future expansion.”
For future upgrades, pathways and strategies are being developed for the advancement of the IP-based playout technology. To fully realise the ST2110 deployment, it is imperative that the security layer of the standard be finalised by SMPTE and subsequently applied as an upgrade to the recently deployed ST2110 infrastructure. At present, SMPTE has not concluded the security aspect of the ST2110 standard. In the interim, network isolation measures are being implemented to fortify the infrastructure’s security.
“With the new solution, workflow efficiency at the ministry has improved. All receiving endpoints have access to a central router and can be managed, monitored and controlled from a single orchestration and management system,” remarks Orlando, adding that the deployment of new technology inevitably presents challenges that were not anticipated during the design phase.
“At the physical layer, establishing a reliable fibre cabling network for ST2110 infrastructure presented challenges, particularly in handling high-bandwidth streams. The FGC integration team executed precise fibre termination to accommodate multi-lane 100G QSFP ports, requiring specialised skills in fibre cable management and testing. Transitioning to the orchestration layer, the system orchestrator had to abstract the network accurately and ensure effective communication with third-party playout controllers. Thorough testing of communication protocols helped mitigate compatibility issues, with FGC’s expertise in studio infrastructure and IP deployment facilitating successful navigation of deployment challenges.”
Balooshi, who has overseen the ministry’s broadcast projects for years, says it has a ten-year plan. He concludes: “We just finished our radio setup a few months ago. We have a brand-new news studio with all 4K equipment. Now that the IP playout system is done, we have a few more upgrades in the pipeline. Upgrade of the RF part of the earth station is one of our pending tasks. Bahrain is a high-profile venue for many of the world’s premium sports events, so we have two new OBs planned. In the meantime, we have an archive project that has been awarded to a systems integrator. We are in the process of finalising the design for that one. This will eventually be integrated with our IP-based playout. More importantly, a lot of government media entities have been waiting to see how our IP project pans out to start their own. I believe we have set a benchmark for broadcasting excellence in the region.”
