If you’re a big movie buff (like most of our team), you’ve likely heard the rumblings that virtual production is changing everything. In fact, many have gone so far as to declare that virtual production is the future of filmmaking.
Well, the rumours are true - and we’re pretty excited about it.
While virtual production as a filmmaking practice dates back decades (think: The Lord of The Rings Trilogy), advancements in technology have quickly made it the new industry standard. Some of your favourite shows and movies (Westworld, The Mandalorian, John Wick, even The Lion King) have demonstrated how virtual production can completely transform how television and movies are made.
But if you’re still asking, “what IS virtual production, exactly?” Don’t worry, it’s a fair question - and one that will net you many different definitions, depending on who you’re asking, and what type of virtual production you’re looking for.
Here at CoPilot, we work specifically in extended-reality (XR) virtual production and we sum it up like this:
“Extended-Reality Virtual Production is filming live-action on a real set, surrounded by LED panels that display an interactive background, to achieve in-camera visual effects (ICVFX).”
Let’s break that down a little bit further, into what we call the “Three Pillars of XR Virtual Production”
![Three Pillars of Virtual Production: Tech, Virtual Environment, Physical Assets [White text on black background]](https://assets-global.website-files.com/65c5921398f30a871236dfa8/65c5971a352345eeb2af5c10_61b8c8610bbc09a987feaaa2_yi1wVgg70EmK1M-0buHeCDi_PSfEzY3jG6NQUGQz8Fp9W2HZnHvLWvaqNR3wnygnJQ5DpzLGpijD1G5l2tJcKKty8tLrNUfFAzh7sVs2xQEDstwzO3bPs1AoUno5YWwDQYQrbxE8.png)
The Tech
When it comes to the tech behind Virtual Production - the LED Soundstage is the star of the show, powered by some mega-powerful computers and motion tracking cameras from behind the scenes.
Think of the volume as the “environment” you are shooting in. Physically, it’s made up of an array of LED panels and houses worlds that have been built in video game engines like Unreal Engine.
What’s important to know is that these environments are completely customizable. Compared to the alternative - a more traditional green or blue screen set up - this radically changes the composition of production crews, and what is required during pre-production (location scouting) as well as on set during filming. We aren’t saying that virtual production will completely take over green screen, but a large portion of movies or tv shows that use visual effects could benefit from the use of virtual production. It speeds up workflows, saves time and money, and lets you finish a production with usable footage, without any immediate VFX.
From a filmmaking perspective, this technology also allows for more realistic camera perspective than green screen, because the background is able to shift as the camera moves, in real time, just like you’d have when filming out on location (known as parallax.) This motion tracking technology is truly what creates the final in-camera shot. It is a huge advantage over greenscreen, where traditionally the backgrounds/environments need to be edited, created and placed into the scene in post-production.
The Virtual Environment
The virtual environments that are displayed on the LED Volume are custom built by developers and digital artists prior to the start of production. These environments can include everything from a simple interior scene, to a rugged mountainscape, to a galaxy far, far away.
Again, the biggest advantage of these environments over traditional production is that they can be adjusted throughout production in real time. This includes adjusting the lighting or colouring, the time of day, and even moving entire elements (like the mountain range highlighted in the photo below) to better suit the filming environment.
Consider the implications of this on a traditional movie set. A tremendous amount of time, energy, and money has been spent hurrying to capture the perfect light or to secure a shooting location in the perfect weather conditions. With virtual production, these types of adjustments can be made in a matter of seconds without having to pause the flow of the shooting day (imagine shooting in golden hour for 12 hours straight!)
When it comes to the actors themselves, virtual production provides a much more immersive experience. They no longer have to guess or imagine what is happening around them in the scene, as they would with a green screen (insert image example below). With virtual production software, they can place themselves in the environment more realistically, and ultimately deliver a more compelling performance.
The Physical Assets
Finally, the physical assets within a virtual production are the “real stuff” or better known as “practical objects” - the elements that are physically present on set, and that bring the whole production together visually. This process is known as “set extension.”
These elements serve a few different purposes. First, they give the actors something to interact with (interior furniture, rocks to sit on or climb over, etc.) in the foreground of the scene.
However, they also serve an important purpose with respect to the LED volume. These elements are intended to help “hide the seam”, referring to the real seam between the floor and the LED volume. Physical set decoration “hides the seam” by providing depth, acceptable focal planes, and additional points of interest between the camera and the LED wall to see the virtual scene to the viewer. It’s important to be able to blend these two scenes so the viewer can’t tell the difference between what is real and what is virtual.
These assets, combined with elements like props and costumes all serve to reflect light and add texture to the environment in a way that makes it more difficult to tell that the environment is not real.
When you combine these three pillars: the technology, the virtual environment, and the physical assets, you arrive back at the core of what XR virtual production is all about: real actors, on a virtual set, with real props, blending the real world and the virtual world into one mixed reality.
We know it’s a lot to cover - and we’re just scratching the surface! Here at CoPilot, we’re staying on the pulse of virtual production: keeping up to date with what’s new and changing in virtual production and delivering that information here, to you.
As we learn more about the rapidly evolving world of virtual production, we’ll be updating our workflows and sharing more videos and more blog posts. So if that sounds exciting to you, keep checking back here, or subscribe to our YouTube channel to be the first notified when we have new content available.
