OpenAI has recently released a new video showcasing some of the capabilities of the Sora AI video generator, which has been met with a mixture of intrigue, excitement and, in some cases, fear of the future.
How exactly does it work though and will Sora be the next big thing or just another in a long list of AI tools that are useful but not necessarily revolutionary or standing out from the crowd in particular?
Or will Sora change the shape of the filmmaking industry forever?
Sora—What is it?
OpenAI’s latest offering Sora is yet to be released but has already got those in creative industries salivating in anticipation at the possibilities this tool will throw up.
Sora is a video-generating tool where users can create perfectly accurate and photorealistic video footage of anything they can imagine, using simple command prompts in the same way as ChatGPT.
The videos produced have a gritty realism to them and impressively recreate anything from a human face to the Apollo space shuttle with every last grain of detail, making them completely indistinguishable from a real film shot with a camera.
Sora can even be used to extend existing videos based on your description, adding new elements or effects to it or extending storylines. For this reason, it could be a vital component of every film editor’s toolbox.
Imagination the Only Limit
Sora can create any type of composition, scene, object or character with exceptional realism but perhaps the most impressive aspect of Sora is how it handles surreal or fictional concepts and designs.
Some of the example videos featured show some fantastical concepts that have been brought to life by the software.
Flying pigs flap their wings looking like a perfectly natural creation, weird hybrids of a cat mixed with an eel and a rabbit with an armadillo back, or a bird crossed with a fox hop about and somehow move with the characteristics of both animals that have been morphed together.
To say it is impressive is an understatement and many comments on the videos express an overall fear for the safety of the human race in general—that’s how good it looks.
Imagine being able to dream up any concept of a strange alien world/character in your head and have it pop into existence as if you are watching a documentary about the planet you just envisioned, with every detail rendered with stunning accuracy.
From a filmmaking perspective, the composition of the scenes created is truly exceptional in terms of artistic talent and how the viewer is drawn in and taken on a journey, with locations and backgrounds morphing and blending into one another seamlessly.
The dynamic effects are actually more realistic than current filmmaking can allow, so if the video perspective takes you flying through a tunnel you will feel like you are about to fall out of your seat.
Videos produced by Sora work almost like an optical illusion, tricking your brain into believing what is happening before you is real.
A good example of the artistic depth of Sora-produced video can be seen in the drone morphing into a butterfly clip.
A drone is seen flying through the ruins of a colosseum, halfway through it flips into a butterfly. By this stage, it is slowly dawning on the viewer that as the angle of the scene shifted, the whole scene has seamlessly transformed from an aerial drone shot to an underwater scene with sunken castles and coral reefs.
Locations can almost fade from one to the next, right before your eyes without any indication of when the change happened. At one stage you are zooming through the corridors of a library, then transported into outer space or the deep valley of some imaginary world, without realising how you got there.
With Sora the viewer just has to strap in for the journey and to say the visual effect is impressive is not really doing it justice. One viewer commented that he might have “pee’d himself a little bit”.
How Does Sora Work?
One of the main features of the makeup of OpenAI Sora is an advanced natural language processing engine.
This allows Sora to interpret the subtle nuance of the text provided to it and extract the essential details and contextual information. By understanding the intricate nature of the language being fed into the software, Sora is able to provide bespoke video creation that is fluid and coherent.
The image synthesis capabilities of Sora mean it can generate characters, backgrounds and effects that match up with the narrative provided in the text. Instead of simply animating the characters, it produces a realistic visual representation of them in every single frame of the generated video.
Dynamic scene composition is a big element in the functionality of Sora that sets it apart from others that follow a more linear timeline.
Sora creates a smoother and more harmonious flow between shots as each scene or location transitions into the next whilst maintaining visual consistency throughout. The sense of perspective and visual effect of the material is engaging and immersive as viewers are drawn into the environment that is created.
Open AI Sora makes use of adaptive learning meaning that there is a continual improvement of its abilities to create videos. The more prompts and text that are fed into it covering different areas, the more it learns to understand different genres and styles of film, refining its processes and knowledge of different techniques.
Another key feature of how Sora is so impressive and realistic-looking is the real-time feedback loop that it uses when processing each frame, constantly going back and making adjustments based on the evolution of the narrative and how each frame matches the one before and after it, fluidly and accurately transitioning from one section to the next.
Sora Impact on Filmmaking and Editing Industry
One of the big questions that film editors will be asking about OpenAI Sora is “Will this put me out of a job?”
Whilst there may be less demand for some entry-level video editing work and labour-intensive tasks—with basic cuts, transitions, and effects being automated—for the most part, Sora should enhance and complement the creativity of the artists working on the project.
Job roles may shift more towards finessing and refining the direction given to the AI tools but the human element of artistic concept is still needed and with tools like Sora new avenues of creativity can be explored that would otherwise be impossible.
Challenges and Ethical Considerations of Using Sora for Commercial Filmmaking and Editing
There are a number of challenges and ethical issues thrown up with the widespread use of AI video generation tools such as Sora in the filmmaking industry.
Intellectual property is certainly at risk from a copyright perspective. Machine learning algorithms collect their information from everywhere, all sources of images and data that are available on the internet and they are not particularly fussy about which is copyright protected or not.
For example, the AI system does not really care who the overall owner of say the Batman concept is and if it is allowed to reproduce or use this character in any kind of film or video being generated.
If you ask the AI to generate a film about Batman vs The Avengers in Space, it will trawl through its databanks and produce realistic versions of those characters with little regard for who owns the copyright on them.
Furthermore, once copyrighted material has been “learned” by the algorithms, there is no real way to extract that part and say you can’t use that, give it back.
Once the data has been gathered it becomes part of the DNA of the algorithm itself and you can’t select any one part of it or delete sections without shutting the whole thing down.
Aside from the copyright and intellectual property issues, there is an ethical problem with allowing the public to dream up videos of anything they wish and the moral considerations could be far-reaching in terms of the potential for misuse.
Harmful or offensive content could be generated without strict controls so along with new freedom of creativity will come new issues and legal concerns surrounding the correct use of AI video generation software such as Sora.
Contact Lyon
At Lyon Tech we provide technological solutions catered specifically for those in the filmmaking and editing industries.
Our range of cutting-edge services allows filmmakers and production houses to reach new levels of creativity that could previously only be imagined.
With cloud-based data storage and computational power provided by remote high-performance CPU platforms and virtual GPUs, film making businesses can utilise the most state-of-the-art AI video and image generation software, have unlimited storage and processing power and make use of any data-intensive or high memory usage tool they need to operate - all accessible from their own devices in any location.
If you would like to know more about how Lyon Tech solutions can enhance your workflows and complement your creativity, get in touch with our industry experts today.
We can help you tailor a bespoke package of solutions that fits the needs of your production house.
