dock10 & University of York partner on AI in virtual studios

dock10 and the University of York have partnered for a joint research project on how AI technology could help to simulate realistic light interactions in virtual studios. 

AI researchers from the university and virtual production experts from dock10 will work on the project, which has been awarded an embedded R&D grant from XR Network+. Led by XR Stories at the University of York, the XR Network+ project provides funding and support to researchers working in virtual production technologies, facilitating collaboration between academia and industry.

Currently, it is difficult to acheive realistic lighting interactions, such as an actor casting a shadow or light reflections, in a multi-camera green screen studio - these aspects have to be added in post-production. These are possible in an LED volume, but to date these have been used in single camera film or drama productions - although Mo-Sys recently launched a multi-camera virtual production tool that may allow this to change.

The research project aims to lay the foundations for the creation of AI-powered compositing tools that ultimately will be applicable in real-time, making them suitable for live or ‘as live’ productions such as sports programmes, game shows and music performances. 

Dr. Florian Block, R&D lead, AI & immersive at dock10 studios and a reader in digital creativity at the University of York, said: “Lighting is one of the core elements of creative expression in any TV show, but the lighting options for virtual studio productions are currently extremely constrained. That’s what makes this foundational research project so exciting. We are aiming to create a proof of concept that will ultimately lead to the development of a bespoke AI tool for delivering realistic lighting interactions in virtual studio productions in real-time.”

Richard Wormwell, dock10 head of production innovation, added: “dock10 is the UK leader in virtual studios production and the University of York is the UK leader for research into digital creativity. We are combining our industry and academic expertise to try to crack one of the great challenges for virtual studio productions. To do so, we are exploring innovative, AI driven solutions that have the potential to radically improve virtual production workflows. We are hugely excited to see where this research project might lead.”