Virtual production (VP) is a new technology revolutionizing the film, media, and broadcast industries. The emerging solution space uses immersive technology to create realistic environments for actors to perform in, saving time and money on production. VP solutions can replace or enhance traditional filmmaking techniques, such as backdrops and shooting within moving vehicles, by providing a safe, repeatable set that shows immersive visualizations in person.
Several VP solutions are available, including high-quality LED walls – also known as volumes -MR Studios, volumetric capture, and AR-enhanced broadcasting. Each solution has advantages and disadvantages, so choosing the right one for the specific project is essential to ensure a positive ROI and audience retention.
VP is a rapidly growing field and will likely grow in popularity in the coming years. Film production teams and professionals can use this powerful tool to create stunning visuals.
Sony Showcases VP Tools at Pinewood
At Sony’s VP showcase event at the famous Pinewood studio in Buckinghamshire, England, XR Today spoke to Sony’s Content Acquisition Solutions Specialist Daniel Listh, who discussed how VP tools and integrated software could save production companies money and improve productivity, collaboration, and efficacy.
Virtual production, in camera, VFX projects require an immense amount of planning. The studios are expensive. You may have an actor for only a certain amount of hours a day. So you need to be very productive when you when you’re there [on set]. So planning is important.
There are some things that you don’t know when you’re planning unless you’ve [already] turned on the camera, such as artefacts that are generated – very important to avoid. That’s why the Virtual Production Toolset [Sony’s virtual set planning system] with the virtual Venice camera is a great tool to have access to.
So, our engineers developed this because we understood there is a pain point. The physics don’t change, but now the tool sets make it possible to plan for something that’s for everybody. So it’s great to have some tool sets to avoid these mistakes so you can plan for something before you see it happening.
Establishing XR-Powered Production Tools
While VP tools have experienced a slow emergence over the past few years, Sony more recently doubled down on its commitment to the VP space. The firm is establishing various resources, institutions, and solutions to provide the foundation of VP and its potential ROI benefits.
Recently, because of Sony’s engagement in virtual production and wanting to understand the market, and seeing the potential business that we can be a part of, but also how we can be a solutions provider. It requires that we understand the market. We need to understand the users and the tools that are there using and maybe how the tools can be improved or fill the gaps.
Because it’s not just one product and turn it on, and then you have virtual productions, there are a lot of different products that need to go in harmony together. So we’ve had various specialists from the display department, from the camera department, from the programming department, software engineers and hardware engineers. How can we work together to find a unique spot in virtual production?
What Jobs Can Come From Virtual Production Tools?
Another interesting factor of the emerging VP landscape is the presence of new jobs and onboarding/training processes for pre-existing roles. Not only does VP provide a space for new roles, but equally, VP tools allow professionals from other sectors, such as gaming or VFX, to transition towards on-site film production work.
Although, an influx of new tools and jobs means that extensive training is required from university to professional levels. This doesn’t just cover using VP tools but also covers on-site etiquette for industry professionals transitioning into on-site film production sets.
There are definitely a lot of schools that want to start teaching virtual production, and they have 30 students, and they have one, maybe two cameras.
They’re looking into investing in a virtual studio as well, but they only have one of them, so they can’t have every single student project at the same time, so they need to be more effective.
So giving a free-of-charge virtual production toolset to students, they’ll have the opportunity to plan for the production perfectly and then hopefully give them a successful experience when it’s their time to go inside the studio.
So it’s been very beneficial for the universities who are teaching students how to work and plan for a virtual production by having a virtual camera with the same parameters as the [real-life] camera.
Planning Properly for a Shot
Moreover, with Sony’s upcoming Virtual Production Toolset, on-set professionals can leverage integrated Unreal Engine tools to plan set layouts, set-ups, and VFX ahead of time. So that when a production team arrive, they can avoid any pain points which interrupt a working day.
So you don’t find out when you’re there, and you’ve built the set, and you’ve turned on everything, and you’ve made your virtual environment – turned on the camera, and then you find out: “Oh, there’s all these artefacts going on. Oh, I didn’t expect that.” And that’s something you can now expect to avoid with the Virtual Production Toolset.
With everything that you’ve done in terms of lighting and the camera settings, you can save that as a file from Unreal Engine, and that file can go directly into a Sony camera. So you can save every single scene. With a single camera profile for each scene. So you know that what you have planned will then also be efficiently outputted from the camera itself.
Digital Media Production Centre in Pinewood and LA
Moreover, to assist with introducing VP technology to the industry at large, Sony is supporting institutions and learning events to spread the potential abilities of VP tools further.
It is to give the opportunity to train people who are about to start new tools or invest in new products and understand this plan that they have in their mind. Does that fit with what’s actually possible in real life?
If they have a big crew, or there are many tutors, for example, for the university, we’ve had universities coming by, and they had the full crew coming to get a full day of learning.
What is it that Sony has to offer? How can we teach them what they don’t know? So they can go back to the university and have confidence and teach that to newer students, so they’ll go out to the market and be much more aware of the latest technology.
We’ve also had students coming here in crews. We also offer a lot of training, and you can either be a student, or you can be a tutor, but you could also be a photographer who’s been in the business for 20/30/40 years. There’s a new technology, there are new tools, and everybody has an interest in learning those new tools and here’s the place to learn it.
Sony’s Place in VP
The company’s VP solutions use Sony’s Crystal LED display technology, and Venice camera capture technology to create a real-time 3D (RT3D) backdrop solution. Sony’s Venice camera capture technology also allows production teams to capture high-quality images that can be used to create realistic virtual environments.
This solution combines hardware and software to import and display an immersive environment behind a subject. The volume product and Venice camera system work together to create a dynamic and reactive studio backdrop.
Moreover, the umbrella solutions leverage Crystal LED display technology, known for its high brightness, wide colour gamut, and high contrast ratio. This makes it ideal for creating immersive environments that look realistic and believable.
Recently, Sony opened a VP studio in Seine Saint Denis, north of Paris, France. The company established the studio alongside project partners Plateau Virtuel and Studios de France. This move allows XR small businesses to use a studio that is fully equipped with Sony’s VP solutions.
The site hosts a 90m² Sony Crystal LED B-series screen that is fully optimized and ready to deliver unrivalled image quality for producers and cinematographers.
Sony customers also leverage the widely adopted Unreal Engine, a powerful real-time 3D engine that can create immersive environments. Sony’s Virtual Production Toolsets professionals import and display real-time 3D environments behind a subject using Unreal Engine. This allows filmmakers to see the environment they are shooting in before they start filming.