Driving innovations in streaming

Research firms offer varying estimates of the growth of the global content delivery network (CDN) market, but all predictions point to a spike in revenues at a CAGR of 14-17% between 2020 and 2025 – for example, Markets and Markets estimates growth from $14.4bn in 2020 to $27.9bn in 2025, a CAGR of 14.1%.

The rising need for efficient solutions to enable live and uninterrupted content delivery, the demand for enhanced QoE (quality of experience) and QoS (quality of service), a proliferation of video and rich media over websites, the clamour for enhanced video content and latency-free online gaming experiences, and increasing internet penetration and adoption of mobile devices – these have all created huge opportunities for growth and disruption in the CDN space.

In this context, BroadcastPro Middle East teamed up with CDN specialist Limelight Networks to host a roundtable with industry experts to unearth some of the trends, challenges and recommended approaches for CDN adoption in the Middle East, Africa and Turkey markets.

Charlie Kraus is Senior Manager of Product Marketing at Limelight Networks, overseeing the marketing of online video delivery solutions. He moderated the discussion, with Umut Keten, Senior Architect at Turk Telekom; Faraz Arshad, CTO of StarzPlay; Zareen Mirza, Director of Strategy at Renmore Partners; Emre Usyal, EVP of Technology with Digiturk; and Selver Irican, Media Operations Specialist at TRT participating.

With representatives from Turkey, Africa and the UAE, Kraus opened the discussion by asking what kind of growth the participants were seeing and what they expected going forward.

Umut Keten commented that as a telco, Turk Telekom was bracing for “exponential growth”.

“We are one of the bigger CDN players. We have around 500GB of capacity that will triple to 1.5TB this year. That’s exponential growth, but also for our internal CDNs, we expect to go from 60GB to 400GB, and in some locations, from 80GB to 300GB. This is just to show how quickly we are growing, with requests from customers coming at a really fast pace.”

CDN demand was already quite high, he explained, but it accelerated with the pandemic and education moving online. “Everybody started to work from home for online education. We started seeing explosive growth. For our video services, we try to re-route them and keep them clean at different paths, making sure they have the proper outcome.”

Zareen Mirza, who works extensively in the African market, commented that Africa has been a bit behind the curve, especially with the impact of the pandemic on funding. “Africa doesn’t have proper 4G coverage yet and the global pandemic has contributed to the economic crisis, so the region is on a slightly different trajectory compared to the rest of the world.”

As a way of introduction, Emre Usyal explained that Digiturk, owned by beIN Media Group, has more than 115 linear channels, seven in-house produced sports channels and a huge library of VOD assets with more than 25,000 hours of content.

“Sometimes people see a special moment on social media and want to watch the last five minutes of the match. So that micro transaction suddenly becomes important” – Charlie Kraus, Senior Manager of Product Marketing, Limelight Networks

“Since the pandemic, we have monitored 50% more traffic on our CDNs. We are carrying valuable content, so concurrency is an issue, especially when it is high. So buffering, latency, etc have been significant challenges,” he pointed out.

With traffic increasing on more platforms and the pandemic still keeping many people at home, Kraus asked if the multi-CDN approach was becoming the norm, especially for sports.

“One Premier League match was only available on a streaming service. The provider used five CDNs to ensure it went well. It wasn’t just capacity, but not all CDNs are equally strong in all regions. All their POPs may not have massive capacity. Is that a common trend in this part of the world as well?”

Keten pointed out that as a telco hosting CDNs, Turk Telekom works with multiple partners.

“Turkey especially is a stretched country, so you need to have multiple CDNs. We have around 16 different CDNs we work with currently. When we provide CDNs for news agencies in Turkey, we have CDNs spread out at four different locations in Turkey so the broadcaster can upload and download their videos easily. As a telco, we have to plan our CDN fully and ensure it is fail-safe. We invest in those links, so we don’t clog up the network.”

Everyone agreed that a multi-CDN strategy is the way forward, leading to the question of how this affects the overall cost of a platform. Faraz Arshad, CTO of StarzPlay, stepped in to comment that when one is building a business case, “the multi-CDN approach is important”.

“Are you going to be customer-focused and engagement-focused? Then you need to take the multi-CDN approach. If you are new in the market, the experience and engagement that people need is what you must be able to provide, to avoid churn. StarzPlay started with the multi-CDN approach, but mostly public CDN. However, we experienced massive throttling so we decided to also develop a private CDN and we complemented our public CDN partners with this solution,” he pointed out, adding that the streaming service placed POPs and edge servers closer to its prime audience.

“We have around 500GB of capacity that will triple to 1.5TB this year … For our internal CDNs, we expect to go from 60GB to 400GB, and in some locations, from 80GB to 300G” – Umut Keten, Senior Architect, Turk Telekom

Arshad said he expects everyone in the pipeline to step up their efforts to make service seamless. “It is everyone’s responsibility and not just that of the service provider. I expect upgrades from the public CDNs as well as the manufacturers. The whole system needs to be standardised if everyone is to benefit.”

Keten, however, added that not all clients require a multi-CDN approach. The Netherlands — it was in the news even — they had for example, only one CDN.

“There was a big problem with that one CDN player being down. So the banking system, news websites … everything was down for a pretty long time, and they did not have multiple CDNs to rely on. So at that level, a multi-CDN strategy is useful. But if you have a small streaming service on VOD, even if you need to buffer, people can live with it. It depends on how much you are willing to pay for a service, and you must be able to find that balance if you are to survive.”

Usyal added that the multi-CDN approach is the only way to ensure a good quality of experience for the viewer.

Selver Irican also favoured the multi-CDN approach, explaining that each CDN is strong in specific territories. “Not all CDNs perform well in all territories. A CDN may be great in Europe but not have a good service here. Some may offer a great experience to subscribers in some regions. So a multi-CDN is appropriate.”

With sports events in particular demanding exceptionally low latency, Kraus looked at a few technologies, such as “CMAF encoding to deliver low-latency HLS and DASH where you can have latencies in the 5-to 7-second range, which is typical of broadcast”. He asked what everybody was seeing in terms of the traction for five-second latency.

Usyal commented that while bottlenecks are inevitable, Digiturk is continuously optimising its infrastructure to reduce latency.

Arshad said StarzPlay has turned to various elements to optimise the service on several levels. “We are working with Zixi and the SRT contribution protocol, for instance, to optimise our content. Then we have been optimising the chunk size of the streaming platform internally. On the player side, we do some optimisations to ensure less delay. On CMAF, we have our roadmap and are looking for clients to support us on some more protocols, because not all the browsers on the player side work. They have to adapt more quickly.”

“Since the pandemic, we have monitored 50% more traffic on our CDNs. We are carrying valuable content, so concurrency is an issue, especially when it is high. So buffering, latency, etc have been significant challenges” – Emre Usyal, EVP of Technology, Digiturk

Keten pointed out that with most sports broadcasting and production done by satellite transmission, an automatic six-to-seven-second latency is part of the package. “We already use up a lot of buffer on that. If we really want to drop latency, every kind of live sports and contribution should be done over fibre.”

The discussion then moved to piracy. Kraus asked if the problem is as straightforward as illegal viewing, or if consumers choose pirated content because of the lack of flexibility in packages, unfriendly pricing and other challenges.

“Piracy is not always driven by free. Sometimes it is driven by the exorbitant cost that limits choice. The rights owner does not provide a stream that you can watch on a smart TV, so people watch it on a pirate channel. Some people say there is a lack of flexibility in the packages and the pricing. Could this also be responsible for piracy?”

“It is not necessarily because people don’t want to pay,” said Mirza. “Stats show that people who pursue piracy go to the cinema twice as much. People who pirate content spend four times as much on pay-per-view content. So is it because they can’t consume the content they want to in the way they want to and when they want it?

“The other trend that is coming into play and which feeds into piracy is subscription fatigue. While there has been an unprecedented spike in OTT during the pandemic, most people have three or four subscriptions, which probably cost more than their triple-play bundle now. No one wants to be forced into subscribing to a service for a month if they want to watch only one match.

“I have been involved with Nollywood. So for every movie that is paid for, it is then consumed nine times illegally. That is a shocking stat” – Zareen Mirza, Director of Strategy, Renmore Partners

“In the UK, there was a core discussion around wanting to watch premium content the way they want to watch it. But often you can’t watch it like this, because most rights didn’t allow you to split matches. However, now in the UK, on BT or Now TV, those options are available and a lot of people prefer to pay for a one-off rather than a subscription package. And this offers a watch when you want, how you want, etc, because people want to consume content on their own terms.”

Mirza explained that Renmore Partners works extensively in Africa. “I have been involved with Nollywood. So for every movie that is paid for, it is then consumed nine times illegally. That is a shocking stat. Given that it is the third largest movie house after Hollywood and Bollywood, you wonder how to preserve its revenues. I am involved in this project and there is CAS and DRM to protect it, but it is mostly pirated after it is decrypted.”

She added that a lot of studios now prefer a direct distribution model, hoping they can protect their content better than by going down the long distribution chain. Distribution houses that implement forensic watermarking stand a better chance of acquiring content from studios directly “because they can identify where that leakage is happening and feed it back to the studios”.

Mirza explained that the other approach is to build user engagement and create a better experience that a user can’t possibly get from pirated content. “I guess we have to use a combination of factors and work with the ISPs, the authorities, the telcos, etc … to deter pirates.”

Usyal said Digiturk also adopts multiple precautions, apart from watermarking. “We collect all the CDN logs from all the CDN vendors we work with. We analyse this and if we find piracy there, we cut the source of the video by connecting with the CDN providers.”

“I expect upgrades from the public CDNs as well as the manufacturers. The whole system needs to be standardised if everyone is to benefit” – Faraz Arshad, CTO, StarzPlay

Arshad said piracy mitigation requires a combination of tech expertise, forensic watermarking, and dynamic collaboration with the regulatory authorities and local bodies to block out the apps and websites contributing to it.

Kraus commented that Netflix has tried to make it easier for subscribers by offering different bitrates for lower quality streams or higher quality streams, though it hasn’t pursued people who share credentials.

Keten said new business models need to emerge that make ad hoc buying in the heat of the moment possible, with flexible payment methods. Kraus seconded this, adding that one network is currently experimenting with micro transactions: “Sometimes people see a special moment on social media and want to watch the last five minutes of the match. So that micro transaction suddenly becomes important.”

Another aspect that was discussed extensively was esports and how latency is critical to a successful experience. With esports creating a new ecosystem of opportunities, everyone agreed that jitter and buffer above 20 milliseconds cannot be accepted.

“Not all CDNs perform well in all territories. A CDN may be great in Europe but not have a good service here. Some may offer a great experience to subscribers in some regions. So a multi-CDN is appropriate” – Selver Irican, Media Operations Specialist, TRT

“These games are so fast that the servers are often right there on the stage,” Kraus pointed out.

Keten said Turk Telekom is trying to create a special product just for gamers. “We are trying to have locations at peering points, so if there is a tournament, gamers can make use of those peering points.”

The discussion finally turned to geopolitics. Some regions are more open to adopting in a cost-effective manner, but there is a lot more to be done with the telcos. Making broadband more economical will not cannibalise their revenue, but rather create more opportunities; however, this requires a more progressive mindset.

Kraus summed up the discussion by showing a slide with the 18 POPs that Limelight currently has in the Middle East and Turkey, and how it is investing in four additional POPs in Egypt, Oman, the UAE and Saudi Arabia by the end of this year.