A MarketsandMarkets report predicts the global virtual production market will reach $5.1bn by 2027, with a 14.5% CAGR from $3.1bn in 2023. Recognising the growing regional and regional interest, BroadcastPro ME brought together experts to discuss trends and features that make virtual production relevant to the industry.
Computer-generated imagery (CGI), real-time rendering engines, augmented reality, motion capture, performance capture, and visualisation and lighting techniques, to name a few, have revolutionised content creation in films and TV. Collectively known as virtual production techniques, these groundbreaking technologies have opened up unlimited and exciting possibilities for creators. Virtual production’s potential is also evident from a MarketsandMarkets research report, which estimates the value of the global virtual production market will reach $5.1bn by 2027, from $3.1bn in 2023, a CAGR of 14.5%.
Virtual production’s global and regional boom made it a pertinent topic at the ASBU BroadcastPro Summit in November. Olaf Sperwer from Business Development Virtual Production | XR Stage at ROE Visual Europe and Fadi Radi, Chief Creative Officer at Blinx, spoke about the advantages and potential of virtual production in a discussion moderated by Daniel Kafer, Associate and Supertrends Expert, Supertrends Institute.
The panel started with Kafer commenting on how virtual production had sensationalised the broadcast industry, but there was still some ambiguity around the technology. Prompted by him, Sperwer explained how it works.
“Virtual production is based on the game engine idea that you have a virtual camera connected with a virtual background. To transfer this to the real world, you need a canvas connected to a tracking system and the cameras to create a ‘full illusion’. This means everything is moving, everything is immersed.
“Unlike in the early movies, you have a stationary object in the front and a moving projection or green screen behind. We have made the physical green screen superfluous. We can also create chromagreen with the LED volume if necessary. All you need for virtual production is an LED wall, processing, a studio, a tracking system and the right cameras.”
ROE Visual is a major manufacturer of LED products, especially for virtual production. The company shot to fame after its LED wall was used to film Disney´s The Mandalorian, which went on to become a blockbuster. The series’ success accelerated ROE Visual’s expansion into the film and TV sector, and today the company boasts a robust presence in more than 125 studios worldwide.
One of the first advantages of virtual production came into focus during the Covid-19 pandemic. With restrictions on shooting, production companies started using LED backdrops for scenes they couldn’t shoot outdoors. “This was a huge boost for the whole XR industry,” stated Sperwer.
For new media entrant Blinx, virtual production was thought to be the ideal way to attract its target audience, the region’s Gen Z and millennial demographic. The company’s digital storytelling hub is designed to cater to their entertainment needs.
“We saw a great opportunity to use technology like virtual production and XR to attract our target audience. XR gives us a lot of flexibility and creativity at a high speed and for a very low cost,” commented Blinx’s Radi. “There is substantial investment in the beginning, but in the long run the production costs come down. Imagine changing five studios in a day. And if you have five shows that have to be produced every week, you can only do this with the support of super technology. With virtual production, you can have a set for every occasion, and you can change the set every week and play around with all the elements. You can combine AR elements with XR to give you a more interactive experience.”
Virtual production eliminates green screens
Radi also pointed to the weaknesses of the green screen, which virtual production eliminates. “A green screen can give you the same concept but lacks many features. Producers face a lot of issues when it comes to chroma keying with green spills. Using a green screen for a dark scene used to be a nightmare! It didn’t work because the surroundings must be dark while you needed to cast light on the talent. This made postproduction very tedious, and there was no guarantee it would work,” he explained.
“With a green screen, the talent is also rendered blind. They don’t know what elements are present or if they are moving or stationary. They don’t know how these elements are interacting with each other. By comparison, LED screens and XR solutions help achieve interactivity.”
Cost efficiency and flexibility
“Virtual production also reduces costs and workflow enormously. Our new LED XR [extended reality] technology helps every kind of production, whether it’s news, film or TV. It helps reduce the workflow on the content creation side and gives a full immersive experience,” said Radi.
Another benefit of virtual production is the elimination of introduction training, said Sperwer: “If a news show has new hosts and they don’t have the time to learn the script, the autocues and markers can be hidden in the LED screens and walls without the audience seeing them. So anchors and talents can just walk seamlessly from one set to another, and there is no need for rehearsal and training. This will help TV producers to reduce time and costs in the studio.”
Another decisive advantage virtual production offers is its ability to centralise the production process and even produce shows remotely. “At Blinx, the set-up we have now is completely cloud-based. Content creators can access all the tech we have in the office and use it to film a video in their homes or anywhere else. It’s a seamless remote production offering,” said Radi
Enhancing content through interactivity
According to Radi, the ability to interact with content matters greatly for new-generation creators, and virtual production gives them this ability. “They like doing things in a way we don’t understand. It’s not just seeing the Titanic in the background. They want the ship in the living room, where they can interact with it on an AR level. Virtual production creates this interactivity for them,” he explained.
“While the industry is focused on investing in content, no one is offering interactive content. And this is what sets Blinx apart from other studios. We can create so much interactivity between the storyteller and the audience. Blinx is investing in creating constant interactive content and user experiences that people haven’t seen before. Blinx goes beyond what TV and OTT are doing at the moment. It enables a more interactive experience not just for the storyteller but also for the audience, allowing them to engage more with the story and share it in many forms across digital platforms.”
Kafer agreed that with content creators seeking greater flexibility and interactivity, the old model of building studios had expired. He also likened grabbing and retaining audience attention to a science, one where creators constantly must seek new methods to keep audiences tuned in. He asked the panellists if there were any set-ups that guaranteed audience retention.
“There are many strategies that can be used to engage the audience, but the first impression is always the most important,” said Radi. “Given that creators only have three seconds to capture the audience’s attention and retain them, the story’s first visual frame and the experience it offers within those three seconds are critical to its success. In those three seconds we try to give them the most visually appealing and interactive content so that they stay with us, engage more and share more – for instance, allowing them to play with the elements installed within the XR set.”
This has led to the introduction of elements such as LED ceilings, floors and furniture in virtual production, said Radi. “These elements enhance the visual effects of any story. It’s what gives viewers their famous football player running in front of them or puts them in a spaceship in space.”
Gaming engines are game-changers
Beyond XR and LED fixtures, gaming engines such as Epic’s Unreal Engine have elevated the capabilities of virtual production and become the kernel of all media engines so far.
One of the main features of a gaming engine is the real-time rendering capacity: changes that were previously made in post-production can now be made immediately on set. This has helped speed up the creative and production workflow.
Additionally, gaming engines combined with the right processing and LED volumes allow filmmakers and TV producers to quickly craft fantastical worlds, implementing immersive backdrops, graphics and 3D animations with unrestricted creativity and choice.
“If we count the sold licences of Unreal Engine 5.2, we have 7m developers working to create fabulous immersive content, and these 7m produce games daily, which means the number of assets in the marketplace is vast. Unreal Engine 5.2 now supports generative AI, where you can dream the scene you want and tell AI to produce it. If I need to be on Mars, I can just buy a template of Mars, get it into our studio, get the talent there and start shooting,” remarked Radi.
Sperwer cited the example of the Netflix production 1899, which used a ship model bought for $15 on Unreal Engine. The production designer then used this model and virtual sets to frame the production design and create each set as a piece of this overall 3D model.
“This approach reduced ten production days on the series, saving approximately up to eight shooting days in an expensive film studio,” he said. “Netflix learned what they can do with virtual production creatively and cost-wise, and is worldwide a bar raiser for virtual production. One of the most successful Netflix producers, the Spanish production company Vancouver Media (Money Heist), has been using our LED screens since 2022. The first scenes with the virtual technology can be seen this year at the end of December in the series Berlin.”
Awareness and customisation are key
Despite virtual production’s benefits, Kafer highlighted the dilemma some producers may face in its adoption: “They could have invested heavily in a physical studio or may just not have the budget to move to virtual production.”
According to Sperwer, the only way to deal with this is by raising awareness and offering customisation.
“One client may be focused on reducing costs and another may want to extend the team creativity, so our approaches to both will be very different. We have to be customer-centric and offer what they need. We understand that we need to guide the customer’s journey and help in the decision process with a knowledge base.
“Let’s say somebody in Dubai wants to buy an LED-based studio. We, in the role of the manufacturer, would introduce the customer to a systems integrator to implement all the elements needed to run a virtual set, such as the LED volume, processing, cameras, tracking systems and content engines. This is what we think Is the cost of market leadership.”
Talking about industry acceptance of virtual production, Sperwer highlighted that adoption has often been slow. “It has been rapid in films but not so in broadcast. For instance, in Europe the organisation is very ‘engineer-minded’ with a greater focus on safety. We understand that they don’t want to experiment too much.”
However, he added that broadcasters such as Sky News London and Fox NFL have broken barriers by adopting LED real-time technologies into their sport and news formats. “We have some heroes and some proof-of-concept stories. We are receiving significant orders from news and sports broadcasters.”
The future of virtual technology
Given the dynamic nature of technology, Kafer asked about the forms it has the potential to transition into, and if it would move beyond studios into homes. Both panellists admitted that there is no way to predict the future of virtual production, given the breakneck speed at which it is advancing and innovations are breaking through.
“I don’t know what’s coming up in the future,” said Sperwer. “I assume AI will drive our market. When this light source becomes more and more intelligent and fluent through the merging of game engine technology, camera tracking and immersive tech, it will reach a point where we can support our client with even better storytelling and production tools. Right now, most of us already have LED screens in our homes, and in time, if this can become a wall that ‘speaks’ to us and creates a greater sense of immersion, it may become a hit with viewers who want extremely vivid visuals. Currently, our whole effort is concentrated on making our LED panels smarter, more intelligent and affordable to a broader audience. Innovation is our focus. By next year, we will showcase pixel-less panels at ISE in Barcelona.”
Radi said that elements such as avatars, motion capture and VR wearables presented exciting possibilities for future virtual productions. “Imagine combining these with the storytellers and XR sets. They will create so much potential for an interactive mixed reality and augmented reality that will surround us on several levels, right from the information we get, to the videos that we not only watch but also interact with and become a part of.”
Virtual production’s greatest challenge will be education and training, concluded Sperwer. “We have to train people to understand this tool – to know what kind of storytelling they can achieve with it and how to do it. This is the only bottleneck around it.”
