Virtual production is rapidly gaining traction, with three new studio launches announced within the span of a month in the UAE. For the next few months, BroadcastPro ME is partnering with The Other End, a Canadian virtual production company, on a series of articles on the basics of using this technology accurately – doing away with myths, getting the facts straight, and tips and tricks on creating ace productions.
What’s the first thing that pops into your head when you say ‘virtual’? Is it a 1960s announcer discussing “the future of computational technology”? Maybe the first-ever recorded demonstration of Ivan Sutherland’s VR prototype, which looked like a cross between Joel Schumacher’s Riddler and two Dyson vacuum cleaners? (Yes, it’s real.) Or perhaps the nausea-inducing Virtual Boy pioneered by Nintendo in the early ‘80s? The first Oculus Rift? Or maybe, just maybe, none of the above?
Not to worry. The excitement surrounding virtual this and virtual that appeared for mere brief seconds in media culture, and then just as quickly faded into the ether. It’s seen quite a number of spotlights in fifty-plus years, each iteration setting its sights on the commendable yet somehow insane objective of allowing a user to leave one reality and enter an entirely new but indistinguishably real one.
Virtual meetings to simulate a boardroom filled with your colleagues, or virtual real estate bought with virtual money on a virtual platform … virtually. Suddenly that insane idea doesn’t seem so insane, huh?
Do you know what else shares that goal? Filmmaking. The desire to transport audiences to incredible new realities where they feel entranced and involved in the imagery woven right before their eyes is intrinsic to filmmaking, if not all visual storytelling mediums.
So, taking all this into account, it would make sense that these two industries and concepts would one day form something greater than the sum of their parts, right?
Welcome to virtual production.
Defining virtual production is not easy. In its current state, it’s more of an umbrella terminology than a single means of production; and to properly relay critical components of its successful use, and the power that comes with such leverage, takes some hands-on experience and proper knowledge. Having worked extensively with this new production solution for over two years now, on both the research and the production side, I’d like to offer a unique take on the exciting new world of virtual production – a demystification of the ever-popular branch involving intergalactic bounty hunters, an adorable green puppet and LED monoliths that will change how we think about studio production and visual effects pipelines.
This facet of virtual production is a means of content production with heavy integration of real-time rendering, in-camera visual effects and digital 3D environment projection, all through the use of exponentially scalable LED walls not too dissimilar to those found on the sides of city buildings. It allows digital assets and environments to be captured directly through the lens instead of being applied during post-production.
This all results in something we’ve come to call final pixels, a fancy way of saying ‘in camera’. This is just a brief insight into its definition and application; in practice, it is an enigmatic and complex subject, a multifaceted engine that with proper understanding can be infinitely valuable in today’s media landscape, both for communicating with a team and for intimately understanding how to leverage its true power.
Understanding the many aspects that make up virtual production is not only the best way of lifting the veil, but the best way to define it properly. As I said, it’s an umbrella term covering a few different workflows. What they all share is a dependence on real-time software that can process incredibly powerful visual effects and digital environments without the downtime of most 3D software. No one likes waiting for stuff to render!
This kind of software has existed for decades, but has only recently become powerful enough to process photo-real digital assets with little to no wait time, principally designed for video game development. There are many game engines, but only a few have truly shifted their focus to this new industry use case. The most prominent are Unreal Engine and Unity, both incredibly versatile, powerful and popular. These engines were primarily designed for the construction of games that run beautifully detailed and designed real-time graphical assets, but have now found a secondary use in the world of visual effects and film production, allowing the creation of photo-real 3D set extensions that look just as tangible as their physical counterparts.
Alongside them are other companies like NVIDIA and AMD, who have invested heavily in developing and streamlining this technology for use in virtual production. Their graphics cards are designed to bear the intense loads of raytracing while simultaneously processing photo-real assets up to 8K resolution. Threadripper processors, for example, accelerate real-time production without workflow or communication hiccups. Both companies share an intense focus on advancing this new way of telling a story and capturing a moment.
At this stage, you may be thinking: This is nice and all, but it just sounds like a fancier version of rear projection. We’ve had that for the past century.
You’d be right to think that, because it’s true to some degree. However, the greatest strength of virtual production lies far beyond means of projection or near-instant environmental rendering. Its power comes from the most straightforward principle of gaming: interactivity. The ability to freely and openly change the most crucial elements of the projected world – lighting, asset placement, location – and to craft the perfect frame at any moment, without concern for external factors, was a previously inaccessible and borderline unimaginable luxury.
Yet here we are, staring off into the horizon of incredible possibilities, a phalanx of panels and diodes.
At its core, virtual production is still production. The traditional means of expertly crafting a final image or properly pacing a scene or a beat are still fundamentally unchanged. Just as a light meter helps you properly set up a scene, and a wireless follow focus lets you comfortably and precisely nail a slow and tedious dolly shot, virtual production also enhances the production environment. Its power resides in the freedoms and solutions it offers teams and storytellers who have already mastered the core principles of production.
Through the years of forging forward into the unknown landscape of simulation and virtual reality, the goal has always been to match the reality we live and breathe. And, perhaps, to actually improve our lived reality.
Matthew Collu is Studio Coordinator at The Other End, Canada. He is a VFX artist and cinematographer with experience working in virtual production pipelines and helping production teams leverage the power of Unreal Engine and real-time software applications.
