Developments in artificial intelligence have transformed a number of sectors, including the creative industries.
The post-production process is certainly an example of a more healthy symbiosis with artificial intelligence. Rather than replacing jobs completely, the AI enhances the work, allowing team members to focus on the creative aspects while the AI carries out tasks that may have been too time-consuming or expensive to do manually.
How AI can Enhance the Post-production Process
Rotoscoping is one example where AI has transformed the process completely. This is where a cut-out was manually drawn over a shape frame by frame, used for various cinematic effects including the lightsabers in Star Wars. Originally this would be achieved by painstakingly going through each frame and drawing the cut out around the shape of the object being enhanced with digital effects.
AI tools can learn what an object looks like, correctly identify it in each frame, cut this out perfectly and automatically apply the digital effect, saving hours of time.
The End of the Green Screen
The advent of artificial intelligence in the post-production process has meant that the traditional green screen effect can be achieved without actually having a green screen itself. The technique of having actors perform in front of a green screen has been made completely redundant by the technology.
AI-powered software can pluck the actors from any setting and place them on the background of your choice. The complexity of the original scene is not important as the AI is not looking at the background but learning to recognise any object or character and training itself to follow the shape of clothing or understand how human hair or a scraggly beard will stick out and obscure parts of the background. A good example of this would be the Roto Brush 2 masking tool for Adobe After Effects.
Using a tool such as Roto Brush 2 the video editor can extract the character from their own back garden and easily place them in Narnia, deep space or wherever they need to be. They can accurately lift the character from the original setting as if they were a solid object, including all the sticking out bits of hair and other details that make the transition much more convincing.
Take our Lives but Not our 5-door Saloons
Of course, this machine learning approach for tagging and extracting items is not restricted to characters on screen and it can be used to intelligently remove any unwanted elements in the frame.
For example, in Braveheart one of the crew’s cars could be seen parked in the background, which did slightly spoil the illusion of a mediaeval battle. It was only on screen for a few brief moments but of course, some eagle-eyed film buffs have noticed the mistake.
With artificial intelligence in the post-production process, the unwanted vehicle could easily be spotted by the software and removed from each frame, filling in the background to match perfectly with the rest of the scene.
Anything unwanted can be identified and removed automatically, from errors in colouring, shadows, camera shaking or even the weather itself.
Gone with the Wind
Anybody who films outdoors has been plagued by audible wind noise ruining the recording. This is an issue affecting everyone with a microphone from online streamers to national news reporters and film producers. Up until now, there has been nothing that can be done about it but tools such as WindRemover AI will analyse the footage and take care of any additional wind noise, removing it from the recording whilst leaving the actors’ voices intact.
Creating Content With Artificial Intelligence
AI tools in the post-production industry can not only remove mistakes and make corrections but also create entirely new material that is seamlessly integrated into the existing content. The software can automatically select the best shots to use, or pull footage from multiple camera angles and match the colour and lighting across all frames whilst creating an entirely new sequence.
New characters, animations and visual effects can be created with AI-enhanced tools, replacing the need for traditional special effects. With Deepfake technology realistic likenesses of people can be recreated and convincingly deliver any lines of speech in the person’s own voice, in some cases with only seconds of audio to analyse.
Of course, this has sparked a debate on how much of your own likeness is owned by the person themselves and what rights they have over its use.
From a technical perspective, one of the key issues with digital doubles and AI tools in film production is the processing power and storage space needed. For example, the storage needed for the facial capture alone can be up to 80 terabytes of data. It is also a time-consuming process with the actor needing to go through their entire repertoire of facial expressions while the data is captured on an array of cameras.
This is one area where machine learning algorithms could cut down on the physical time and processing needed to acquire all this data in a usable form.
What is Workstation as a Service (Waas) and How can it Help Creative Industries?
Workstation as a Service (WaaS) is a remote desktop that gives creative businesses access to high-end processing power and data storage, without needing to outlay the cost of any infrastructure or expensive equipment—with everything accessible from their own devices in any location.
For creative industries, VFX studios and post-production houses, this can have a dramatic impact—giving them almost unlimited freedom of creativity. Whereas they would normally be bound by the technical capabilities of their hardware, this is no longer an issue with Workstation as a Service.
At Lyon, we offer state-of-the-art data centres, cloud solutions, remote desktops and workstations as a service for creative businesses—allowing them to enhance their creativity without limitation, whilst offering full IT governance and 24/7 support. This lets businesses focus on the creative aspects of their work without needing to worry about the technical capability of their hardware.
For more information on how a managed service provider can benefit your creative business, get in touch with us today at 02071128504 or email hello@lyon.tech.
