The Virtual Production boom, with Asa Bailey from On-Set Facilities – Part I

Part 1 of our interview of Asa Bailey, Director of Virtual Production and Founder of On-set Facilities, a company dedicated to virtual production and computing solutions. We discussed the virtual production boom, the birth of On-set Facilities as a virtual production company, why the industry must look past LED walls and the impact VP has on the whole production chain.

Read Part 2 here: The Future of Virtual Production: Hopes and Challenges, with Asa Bailey – Part II

If you’re interested in virtual production for filmmakers, be sure to read our two-part interview about the first steps of two DPs into virtual production territory.

[powerkit_toc title=”Table of Contents” depth=”2″ min_count=”4″ min_characters=”1000″]

First steps into virtual production and the creation of On-set Facilities

Alvaro Lamarche Toloza, Atrytone Co-founder:

Welcome, Asa! Thank you very much for joining us today. So, you’ve been in the middle of the virtual production boom since the very beginning. How did this adventure start and how did On-set Facilities come to be?

Asa Bailey, On-Set Facilities Founder:

When it all started I was a traditional VFX director and cinematographer, though I’d always been working in technology. My previous background was in software engineering and UX design, I was a digital creative and developer. So when I got into the film industry, I took with me probably 10 years worth of digital experience onto the film sets. And naturally I started to look for new ways of doing things, to make it faster, more efficient.

So I was directing at the time and I started to try to bring in more technology. This was way before it was conceived as virtual production. For me, the frustration as a director or as a cinematographer was to have to wait for results. It was an anxiety that I wanted to remove. And so we started to pre-visualize on set first, then we started to perfect those methods. We started to shoot it all in camera though that was nowhere near good enough for high end cinema or high end series. But in the early days, we were very lucky to be sponsored by a number of big technology companies that utilized our methods to create content for their social channels, for their advertising. And it gave us a big opportunity to work in these new ways.

Again, this was all before it was considered as virtual production. Then the boom came, as you wish to call it. And really that started from SIGGRAPH and the Epic Games project with the motorbike. When that came out, I was there myself with a number of other leaders in the industry at that time. We were on set for that production, watching it roll out. There was a brilliant showcase.

The opportunities of the virtual production boom: LEDs and computing

Alvaro:

Virtual production has grown exponentially for the past year or so; how has this growth manifested itself for you and On-set Facilities?

Asa:

So that was the first time we’d really seen LED being utilized. And that’s when virtual production became known in the real world as “LED virtual production”. Whereas for us, that’s just one very thin line of virtual production whereby we are involved in real-time production of game assets to engineering, to anywhere where real-time interaction happens between people. So the LED stage has now become a predominant big shiny thing that you see when you think about virtual production. That made us bring LED into our already existing virtual production studio to test it and to really see what it can do. The boom actually brought us towards what the market was now actually demanding on a bigger scale, which was the LED side of things.

And that’s when virtual production became known in the real world as “LED virtual production”, whereas for us, that’s just one very thin line of virtual production.

Asa Bailey

But the one thing that we figured out very early on way back in 2016, was that we didn’t have a solid computing platform to operate off. We could all buy configured machines from the internet and have them delivered, but there was nothing there that was low latency enough for the systems that we were building. When we chase latency through a system, the one thing we need to not have to chase is in the computing. We have to track it down in the network, track it down in the video chain. We have to track it down in the way that graphics are transported.

The one thing we don’t want to have to do on set is track down latency within the actual computing platform, and that’s why we built our machines. We then found there was a market for our computers. They were originally built for me to start with on set. They were my computers built for my specification for real-time production, but then the rest of the world started saying “actually, we could do with some of these things!”. So the boom has taken us from a research and development laboratory in the mountains of Snowdonia to becoming a manufacturer of the computing platform, an integrator of the virtual production systems. Our company also now consults, and we have five virtual production studios around the world that we are lead consultants for. All of that has come from this recent growth.

Is virtual production really just about LED walls?

Alvaro:

And how do you feel about the growth of the virtual production space? When you are in virtual production, it may feel as if everyone’s using it, but if you talk to people outside of virtual production, it seems less clear or doesn’t feel as widespread. Do you think that it’s becoming, or it has become a standard, or does it still have a way to go before reaching that stage? 

Asa:

First of all, the growth is very exciting. Thanks to the growth of the virtual production industry, we find ourselves in the center of something that we didn’t expect. We did what we were doing because of need, because of a creative drive to do these things. And to end up in the center of all of this is fantastic. 

I think with the way we communicate today, it’s very easy for it to seem overwhelming, because anybody can put out noise. And so, yes, if you were coming into this space today, I think it’s going to be quite overwhelming and also somewhat confusing because everybody talks about virtual production as LED stages with backgrounds on them. And quite frankly, in the film industry, they’ve been doing rear projection and painted sets and virtual sets essentially for 50, 60, god knows how many years. So to a lot of people in the film industry, if virtual production is sold as LED, they look at it and think, well, it looks good, the lighting potential has its benefits, but is it really all that?

Well, you’re missing a lot of it, you’re missing the point. So once you get beyond that, it gets really interesting. And then you can start to think about how virtual production can help your mandate, your vision. And that depends whether you’re an animator, whether you’re a filmmaker, whether you’re an engineer. It depends whether you are any form of creative that wants to create in real-time. Cause that’s what virtual production is about. We are creating between the real and the virtual space in real-time. And that interaction should be an unbreakable experience for the user.

Alvaro:

So what you are saying is that when virtual production is equated to LED stages, it’s missing out this real-time aspect and what is really happening, taking all these different roles and putting them together in this environment where things can happen in real-time, on set. It lacks a holistic understanding, an approach to what virtual production is as a whole.

Asa:

Exactly. What’s missing is that holistic understanding. It’s a very good way of putting it. What is real-time virtual production at its essence? It sits as the gateway between the real and the physical world, the meta world. Virtual production is like film production. You are doing virtual production, you are making things in virtual space, whether it’s AR or VR, doesn’t matter. You’re still generating synthetic elements and maybe combining them with real elements. Now that could be a performance. So you might not see anyone real, but they’re driven by real people. In the example of real-time animation, you’ve got people in motion capture suits, facial animation, rigs, and what they are creatively doing is then translated into the virtual world, and then being filmed or recorded in the virtual world. And so that is virtual production that doesn’t have any LEDs.

If we are, say, engineers working on a new headset design, and we are in different locations, I might want to see you working on that design. I want to see it spinning it round, bringing out the microphone. And I want to see that, I want to interact with it too. That thing has to exist in the virtual world so that we can interact with it in real-time. And so there are technologies that are coming that enable this real-time collaboration, the real-time visualization on the walls. And this is where virtual production is a holistic concept.

I come from the film industry and the interactive industry. And so I looked at it through my lens as a DVP, as now I’m known as a director of virtual production – not too keen on the name, to be honest – but I’m really a cinematographer that uses a lot of tech. The key is the understanding of it and the benefits of it to multiple layers of the industry. But you need to start to understand the underlying philosophy of these things, because it affects all areas. It affects finance, I mean the cost of doing things, it affects the workloads.

Working with a Virtual Production crew

Alvaro:

Let’s talk a little more about the VP crew. Usually, when you’re shooting on location, you’re not responsible for how the sun works, it’s just there, and the DP can work with that. In this case, working with a virtual set, you have people who have to be responsible for how that sun works and looks. It’s a very technical and perhaps underestimated job to ensure that the director and the DP can work properly with a clean baseline. Can you tell us more of how you work as a director of virtual production leading a team of VP engineers?

Asa:

We just get the set ready for them to move into. And then when they come, we help them. That’s the DVP’s role. And to also orchestrate through pre-production to bring them along on the journey as well. So I take DPs into virtual scouting sessions, people who have never been there before, and I familiarize them with it. And they won’t need the technical know-how to sometimes communicate with the crew that are very high hands on with the system. The creative team will say something that needs translating and communicating back to the VP crew, that’s my job. So I go back to my crew and I say, right, guys, we need to do this, and this is what he really means. Because it’s a different language. It’s a different department in that way.

I’m not a virtual production supervisor – I do that role now and then, but a virtual production supervisor in my view is supervising my systems and making sure that we are taking the camera tracking data correctly, and that it’s being saved correctly. And that at the end of the shoot, we can hand off our data that we’ve taken and collected from the shoot. And we can hand it off to the customer, to the client, maybe the post house, maybe editors that might be going straight out the door. So we have our VP crew, and we’ve had to develop that years ago and everybody now in the industry is going, “so how does the VP crew work?” and we know how we do it.

Alvaro:

This is something that doesn’t get enough publicity, when you see LED stages being presented it’s sometimes as if you basically stream a video on a big LED wall and that’s that. That’s an option, but it’s not what the workflow is about. You’re looking at the tracking, the camera, the frustum, the lights, everything working properly in unison… who’s in charge of that?

Asa:

Let’s take one step back from that. Everybody’s talking about on set because everybody’s looking at the big shiny windows into the virtual world, right? That’s where humans are going, “oh! Amazing!”. Virtual production goes way before that. Whose responsibility is it to LIDAR scan a prop to make it so that it fits in with a real prop you’re going to use? It’s a good question. We see it as it’s the virtual art department’s responsibility. They may not know how to do it, or may not have the systems to do it, or may not have the guidance. That’s where the virtual production team comes in. We will train that department to do it themselves if necessary. But the technology that they will be using is virtual production technology, because they are creating virtual assets. Just as a camera is a film production piece of kit, just like a prosthetic put onto the face of a monster is still part of film production.

The virtual production department  doesn’t just take over the whole of the film production universe. That’s not what we are here for. We’re here to dematerialize film production and take it into the virtual and physical worlds. So who is responsible for scanning that pillar that we would now want? The virtual art department, or in fact the art department. They don’t know how to do it, who do they turn to? To the VP group.

Alvaro:

At the end of the day, it’s a, it’s more of a central piece that ensures that there’s a certain level of cohesion and understanding, and that everything works together. If I understand correctly, your view of this virtual production crew is more to make sure that all the pieces of the puzzle are fitting together correctly.

Asa:

Exactly. This is why you have a director at the top of it, and you have the director in charge of all of us to deliver his vision. And for him to be able to concentrate on his vision and on how he wants the actors to work or how he wants the art to look or whatever it may be, the technology needs to be brought into this invisibly.

For [the director] to be able to concentrate on his vision and on how he wants the actors to work or how he wants the art to look or whatever it may be, the technology needs to be brought into this invisibly.

Asa Bailey

Do we need virtual production in each department? Yes. We are at the stage now where we are asking the question of who owns the virtual production supervisor, because usually we are finding that they come from the VFX house. We don’t own it, we are the virtual production department, but we don’t own the virtual production supervisor. He is, or she is responsible for supervising the vision. Because we are taking their vision and putting it on our platform, we’re taking the VFX company’s work and we are putting it on our technologies on set, and that needs supervising on their behalf. And that’s where the virtual production supervisor comes in from the VFX company. So do we have VFX, do we have VP artists in the art department? I’m sure we should. We might not call them VP artists, but they’re going to use virtual production technology.

Collaborating with competing companies and solutions

Alvaro:

Speaking of which, what has evolved in virtual production technology between the beginning of OSF in 2016 and today? Are there any breakthroughs in particular that you can mention?

Asa:

Well… We’re system integrators. Part of that is not just integrating the technology together. It’s about integrating the companies together as well. So I find a lot of my work is involved in bringing companies together that wouldn’t normally do so. They see each other as competition. They may also be in different markets, but part of the integration process that we enjoy is when we create unique solutions by bringing together, for instance, Ncam and Mo-Sys, who would otherwise not get in a room together. Now we can bring the technology together and create solutions out of both of them. 

That’s when real magic starts to happen. That’s an important part, and so far, we’ve now got official strategic partnerships with Unity, Arri, Nvidia, RED as well. We work with Brompton too. The key point that I want to get across is that it’s not just about making one machine talk to another machine. It’s also about integrating the companies, to start to think in these ways, and we’ve been very successful in doing that.

Alvaro:

Working with so many solutions and companies from Unreal to Unity, from Arri to RED… What are your key takeaways? How easy is it to assemble all the different pieces of the puzzle?

Asa:

Because we own the computing platform, we don’t have to worry about it. It’s a continuous process, but essentially we know that it works. So when we’re trying to integrate two types of technology together, it’s really a question of code of protocols, software development, things that may be missing that we need to put in place. We have a full time research and development laboratory, and I don’t know another company that has one like what we have. It’s not under any business pressure. It’s under a pure exploratory mandate. And that means that our engineers can really solve some of these problems, and that’s why we have so many partners: because we can take their technologies. We then integrate with them and we create new products utilizing them.

If I’m not on set shooting, I’m in the lab and I’m working with them. I’ve got a lab in Los Angeles. I have a lab in the UK. We also then provide all of the computing to Arri studios. So we’re building a network of connected studios. And again, we co-develop with our partners to help them make new products as well. It’s not just about us.

---

Read Part 2 here: The Future of Virtual Production: Hopes and Challenges, with Asa Bailey – Part II

---

Asa Bailey
Director of Virtual Production

Asa Bailey is the founder of On-Set Facilities and has consistently been recognized for his innovative use of on-set technology as well as his unique approach to storytelling using virtual production systems. His work ranges from conducting virtual scouting sessions to building LED stages and developing camera tracking solutions for demanding shots.

Through On-Set Facilities, Bailey is routinely hired by producers worldwide to design, install, and support robust virtual production environments both on-set and cloud based. In this capacity Bailey has worked with DreamWorks, UNBC Universal, Universal Pictures, Unity, and the ARRI Systems Groups among many others. 

Today, Asa Bailey continues to direct his own projects and participates in various industry events, having delivered workshops for NVIDIA, the Directors Guild of America and MIPS. He sits on the board of Advisers of both the DAVE VFX School at Universal Studios and the Studio UK Government Working Group.