In the second part of this series, Matthew Collu takes us through some terms used in media that have entirely different connotations in a virtual production environment.
With each new leap in technology and technique, a new lexicon of terminologies usually arrives alongside it. Some terms are similar to existing phrases, others are entirely new and contextualised only by the advent of the new practice itself.
Then there are those that share an existing name but adopt a different meaning. In an age of seemingly daily technological breakthroughs, navigating a developing and unfamiliar landscape can be confusing, not to mention wildly intimidating. There is no better example than virtual production, one of the most significant visual effects breakthroughs in recent memory.
Having much experience on the production and research side, I’d like to lend that expertise to define virtual production terminologies and act as a sherpa through this foggy vista of visual effects. However, describing every new term would take this from an article to a dictionary. Instead, my focus is on three essential aspects: similar, new and deceptively the same.
Assets
Let’s start with the similar, shall we? Assets are defined as anything added that wasn’t there in the original shot or frame, known as a plate. Say we’re shooting a blockbuster with a mighty warrior battling it out with a ferocious, fire-breathing dragon atop a massive stone bridge. The only physical components of this shot are the actor and the ground on which they’re standing. In post-production, the assets are the dragon, and possibly a set extension or matte painting to further the look of the setting. All these are needed to complete the final image, but none were there in the original.
Assets range from 3D visual effects and models to 2D backgrounds and extensions, all added after principal photography. Virtual production’s foundation is the same, but the execution and context differ slightly.
With virtual production, those incredible environmental aspects usually added in post-production can now be captured and controlled during principal photography, by using real-time game engines. Assets in this context are components that make up a larger digital environment as a whole – itself an asset. The towering structures of a castle or distant trees in the surrounding landscape are all individual virtual assets that cement our story in the world. Rather than being elements designed, developed and added in post-production, they’re added in pre-production for on-set, real-time capture.
Volume
The concept of volume in terms of virtual production is still somewhat unfamiliar and often confusing, especially to those with no motion capture experience. It doesn’t refer to how loud a sound is, nor the knob used to get a party started – it more closely resembles the metric of measurement. A volume in motion capture is usually a cubic area surrounded by a motion capture system/solution. Inside this volume, the position of objects in the real world can be tracked and translated into a virtual position, for complete control over CG components.
Imagine a large aquarium tank. Instead of filling it with water, fill it with infrared light. That light fills the space, even though we may not see it, making whatever is in that space move through a pool of photons, bouncing around.
Now, what’s an aquarium without fish? The fish in this instance are tracked objects within a motion capture environment. They can be props, people or cameras; all are bound by the same perimeters. The LED wall commonly associated with this wonder of workflows is placed within the confines of a volume, along with a camera to be tracked; the resulting one-to-one, real-to-virtual translation results in a perfectly shifting window into a digital world.
Tracking
Without assets, your virtual environment is an empty void – no trees to fill an eerie forest or neon-lit alleyways to spook a protagonist. Without volume, there is no dedicated container to supply a lake of light to house your LED wall and any other subjects of choice; but without tracking, there is no way to control your components.
On a traditional film set, tracking means following a subject moving through a shot or scene. In the visual effects world, it means tracking a motion path, mask or pixel to lock an asset perfectly within a moving frame. Tracking in a virtual production environment is an exciting combination – tracking a subject that is the camera itself, to match the perspectives of both physical and virtual, to root the physical talent in the mirage that is the digital landscape. A tracking solution is needed to do this, and to turn a space into a full-blown volume.
Without going into the different systems and methodologies, tracking solutions facilitate the ability to move freely around a space, but also track and translate all the nuance and subtlety of that movement into a digital environment. Effectively, tracking in this context is the glue that holds the whole operation together, for a seamless blend of real-world performance by both cast and crew and real-time virtual sets.
These definitions are just three of the terms and definitions associated with this new and powerful means of production. I hope they act as stepping stones to the new world of virtual production.
Matthew Collu is Studio Coordinator at The Other End, Canada. He is a visual affects artist and cinematographer with experience working in virtual production pipelines and helping production teams leverage the power of Unreal Engine and real-time software applications.
