Lighting is to camera what water is to fish: Without one, it’s hard for the other to do its thing, says Matthew Collu.
Since the very beginning of capturing an image – moving or not – lighting has had one of the biggest, most important roles in the actualisation of breathtaking visuals. For fine arts, it was a way of expressing tonality and evoking a particular feeling in a piece, and as new forms of artistic visual expression arose, these principles were carried over.
Aesthetic and tonal accuracy are part and parcel of crafting a consistent and coherent visual tone, and lighting is one of the largest components of it all staying held together. If you’re watching something meant to be sinister and unsettling, lighting can be the difference between unsettling and just unseen. With the visual effects demands of the modern era, lighting (and expert control of it) is held to an even higher standard, with green screen and chroma workflows to match a digital environment added much later.
Much like anything in the creative world, sometimes it’s accurate and sometimes not so much. With the advent of virtual production and LED set extensions, however, accuracy and control are elevated to a completely different level, without the need to estimate an eventual final image.
For a while now, lighting for visual effects and creating aesthetic consistency while shooting in a soon-to-be-added environment has largely been a lovely waltz of visual alchemy – one part lighting for something that’s not entirely there, one part generating and implementing the missing piece. Many incredibly talented creatives have hit their stride and nailed their end of the formula, but in other instances they’ve had to go back and try again.
This is both costly and meticulous, not to mention time-consuming (I can hear the cries of every upset producer echoing out as we speak). Through no fault of either creative side, artificing something without a complete formula is mighty daunting, and many argue that it would be much easier if that fantastical horizon or sci-fi city backdrop was just simply … real.
With real-time rendering software solutions, LED walls and in-camera visual effects comes a heightened level of interaction. A totally real-time virtual environment brings many added benefits, and one of the most notable is realistic lighting and control over almost any detail. In order to best explain this, along with its core features and functionality, I’ll break down how it changes, creates and enhances workflows.
With the previous examples of the possible pitfalls of lighting an imaginary setting, it’s hopefully easy to see what the workflow stands to gain with a completely integrated and interactive real-time set extension. Bearing in mind the other changes that virtual production brings to creative teams (for the better), the change in how artists and industry can use, control and artistically explore lighting – while always ensuring its accuracy to the world it takes place in – requires some fundamental understanding of how an LED wall works.
In essence, it’s simple. Think of an LED wall or stage as a giant computer monitor. As images appear and colours flash, that monitor kicks out a certain amount of light. Apply that same principle to that of an LED stage, and all the colours and values of the world you display on that screen are cast onto your subject of choice. When that subject is surrounded by this billboard of innovation, you get 100% environmental lighting accuracy on anything placed within the volume.
Explaining why this changes the workflow seems redundant. But how it changes things – that’s worth swan diving into. Traditionally and conventionally, without a reactive, real-time virtual environment blasting light out into your scene, doing it all manually can be extremely temperamental and, as said, possibly inaccurate as it takes new forms downstream. There can also be missed nuances that in some cases would never have had time allocated for refinement on shoot day, or would never have been thought of at the time of principal photography.
By removing the need to properly match the ambient lighting of a scene and relying on the innate interaction coming from the LED wall, resources and energy can be allocated to refining the style and tone of the scene as a whole. This is not only more efficient but also stylistically freeing. Another change with real-time virtual environments is the ability to add things in seconds that could take hours on a traditional set or on location. If you need an additional kicker coming from the right of the frame, or a gel from just above, those elements can be added and displayed on the wall without having to assign labour or resources.
This isn’t to say that traditional fixtures can’t be used to aid in the perfect dial-in of your final look. ARRI, among other vendors, has direct control and functionality features that integrate directly into real-time software apps. Virtual production isn’t about replacing or taking tools away from filmmakers, but rather offering additional solutions to common problems in workflows and enhancing existing tools.
Going back to our primary example of shooting and lighting for a non-existent CG environment, the green/blue/chroma pipeline of modern visual effects also benefits greatly. Instead of projecting the camera’s perspective in the virtual environment behind the subject, a flat colour of your choice can instead very easily be swapped out for use, all while having the accuracy of the environmental lighting still cast on your subjects by the surrounding digital vista. Pair that with the colour levels staying consistent across all values for efficient keying, and you have an enhanced traditional VFX workflow that still allows artists to be stylistically free with how and where they lob those foot-candles.
Part of the allure when discussing virtual production and real-time filmmaking is the in-camera element or the ‘right there’ ideology. Having interactive environmental lighting that is true to whatever environment is projected is not only a wonderful example; most incredibly, it further equips creatives with complete control over worlds and environmental lighting previously only accessible to nature itself.
Lighting is as important to the integrity of a production as paint is to a canvas. Just as painters have been able to completely craft and control their palette in the moment, virtual production now gives filmmakers real-time control over theirs.
Matthew Collu is Studio Coordinator at The Other End, Canada. He is a visual effects artist and cinematographer with experience working in virtual production pipelines and helping production teams leverage the power of Unreal Engine and real-time software applications.
