How deepfakes will change creative jobs forever, after this day
Today we only need just one image to create deepfakes.
We have reached a milestone, an earthquake in the creative field, today we can make DeepFakes with only just one image, yes, only one… some persons will say this is witchcraft but the truth is that after this, every workflow of the creative industry is going to change.
From illustration, to animation, to e-commerce, to photography, to teaching… we are going to see a major shift in the creative industry and newborn techniques that will be used in conjunction with those being used today.
The algorithm that have changed it all
First Order Motion Model for Image Animation by Aliaksandr Siarohin ,Stéphane Lathuilière, Sergey Tulyakov, Elisa Ricci and Nicu Sebe is the paper that has set this milestone, my personal congratulations.
What makes the difference for this algorithm is that it doesn’t need any additional information to work properly. With other algorithms you have to use facial landmarks and estimate the poses for your model, rely on good previous data.
This model generates the data by itself, and it’s awesome! look at the image below, they are moving the arms, this it’s crazy. And it works with multiples objects too, not only with faces and human bodies which are the common elements that take action in the deepfakes field.
Not needing additional info about the image lets us to take a more general approach liberating us from gathering data and permits more people to participate in this new technology field.
Let’s see some examples.
New animation and production workflows arriving
Look at the image below, one input video, 8 different models = 8 different animated styles, it’s self explanatory.
This is going to change the rules, being able to test some animations with different styles is going to change the workflow off all major illustration companies if this method is not being used already.
This could be the rotoscoping technique death announcement, just think how much time Don Bluth and Gary Goldman would have saved to produce Anastasia film (1997) if these kind of tools evolve to the point to handle a project of that magnitude.You could shift all your workforce to pay attention only to the style or just to spend more time on special sequences.
If this approach starts to be applied, it could mean a major shift and new era for the artist, focusing only at styles rather than the production? I don’t know but it seems this could be the next possible step in the industry.
Also there are already deep neural style transition techniques (you can see online examples at Deepart.io), that permits you to transfer a style to a form without affecting the form. I can see perfectly big companies such as Disney using this combining it with this new image animation techniques to increase by an exponential factor the possibilities of their next productions decisions.
An example: increasing product decisions
Imaging you are a high level creative employee responsible for the next film production and your team is on the early stages of the project deciding how the final style should look.
A simple approach could be picking all your illustrators and ordering them to produce different sketches and styles, now all your team of 12 illustrator is working on this until you decide the better style for your product.
Imagine now that you have been able, in some way, to gather trough the years the style from your illustrators, as we know every artist has an unique and identifiable style. So now instead of choosing 12 illustrators you just choose one illustrator to create a sketch of the main character and you use it with a deep neural style transition tool.
Now you can use the other rest of illustrators to work in other tasks. But you also want to see how the style affects the dynamics of your character and for that you can combine this with neural Image Animation tools as First Order Motion Model for Image Animation to go even further.
E-commerce and modelling industry
The first example that comes to my mind is the use of this kind of algorithm to cut cost at some processes in the retail industry involving photography, and this might not be a bad thing because it may help some smaller companies reach the same features of some major companies products.
For example, the fashion retailer ASOS has a feature in its marketplace that it’s showing a video of the actual model wearing the product that you are going to buy. A nice way to see how the clothes are really playing in the third dimension, but it cost data, time and logistic to have this kind of feature.
Small companies can not compete with them, but if they manage to hire one model for videos and just take one picture for different models or even the same model wearing different products to sell and they use this algorithm… they could offer the same feature.
I don’t see the reason why this could not be achievable in the short term by a small company, even thought the representation it’s not real at all but it mimics reality so perfectly that it could be used in a real environment.
Low fidelity video prototypes for all?
But it can not only work with human figures it works with objects too, let’s see what they did with this robot arm. Here they used the same general object composition from the video at the photos but they placed the robotic arm at different positions.
As you can see the output results render the robotic arm moving in different positions from its original position. This makes me wonder if we are seeing the born of a new technique to record fast low prototype videos, especially those involving dynamic object composition techniques commonly used in the creative field, as in the marketing creative companies.
And it could even be used in dynamic sculptures, dance, performance …
Museums
Being able to animate pictures it’s always funny, and it’s already being done, but decreasing the requirements is what we need to start seeing deepfakes to take action in our educational system. I can see teachers using this to play historic characters while teaching their students or museums using it to reach a more generic population.
I can’t not wait to the day we will be able to reproduce historic moments merging actual photos of the events with image animation techniques and virtual reality to offer the total immersion experience.
This would mean a better approach to historic facts and also more funny classes.
My personal fear
We are approaching to the age of computing power, this is, to stand out in the next decades we will have to compute vast amounts of data and we will have to invest in pure GPU, RAM, processors… The companies who have more power will have more possibilities to test every option and more chances to success in the market.
But Papers like this one reducing the set of previous constraints help us to democratize technology. Also I fear if the creative mates doesn’t take a step in the code world they could be losing their path in the long term.
