New Face Swapping AI Creates Amazing DeepFakes

New Face Swapping AI Creates Amazing DeepFakes


Dear Fellow Scholars, this is Two Minute Papers
with Károly Zsolnai-Fehér. Recently, we have experienced an abundance
of papers on facial reenactment in machine learning research. We talked about a technique by the name Face2Face
back in 2016, approximately 300 videos ago. It was able to take a video of us and transfer
our gestures to a target subject. This was kind of possible at the time with
specialized depth cameras, until Face2Face appeared and took the world by storm as it
was able to perform what you see here with a regular consumer camera. However, it only transferred gestures, so
of course, scientists were quite excited about the possibility of transferring more than
just that. But, that would require solving so many more
problems – for instance, if we wish to turn the head of the target subject, we may need
to visualize regions that we haven’t seen in these videos, which also requires an intuitive
understanding of hair, the human face and more. This is quite challenging. So, can this be really done? Well, have a look at this amazing new paper! You see here the left image, this is the source
person, the video on the right is the target video, and our task is to transfer not just
the gestures, but the pose, gestures and appearance of the face on the left to the video on the
right. And, this new method works like magic. Look! It not only works like magic, but pulls it
off on a surprisingly large variety of cases, many of which I haven’t expected at all. Now, hold on to your papers, because this
technique was not trained on these subjects, which means that this is the first time it
is seeing these people. It has been trained on plenty of people, but
not these people. Now, before we look at this example, you are
probably saying, well, the occlusions from the microphone will surely throw the algorithm
off, right? Well, let’s have a look. Nope, no issues at all. Absolutely amazing, love it! So how does this wizardry work exactly? Well, it requires careful coordination between
no less than four neural networks, where each of which specializes for a different task. The first two is a reenactment generator that
produces a first estimation of the reenacted face, and a segmentation generator network
that creates this colorful image that shows which region in the image corresponds to which
facial landmark. These two are then handed over to the third
network, the inpainting generator, which fills the rest of the image, and since we have overlapping
information, in comes the fourth, blending generator to the rescue to combine all this
information into our final image. The paper contains a detailed description
of each of these networks, so make sure to have a look! And if you do, you will also find that there
are plenty of comparisons against previous works, of course, Face2Face is one of them,
which was already amazing, and you can see how far we’ve come in only three years. Now, when we try to evaluate such a research
work, we are curious as to how much these individual puzzle pieces, in this case, the
generator networks contribute to the final results. Are really all of them needed? What if we remove some of them? Well, this is a good paper, so we can find
the answer in Table 2, where all of these components are tested in isolation. The downward and upward arrows show which
measure is subject to minimization and maximization, and if we look at this column, it is quite
clear that all of them indeed improve the situation, and contribute to the final results. And remember, all this from just one image
of the source person. Insanity. Thanks for watching and for your generous
support, and I’ll see you next time!

100 thoughts on “New Face Swapping AI Creates Amazing DeepFakes

  1. At 00:50 you talk about the innovation of transferring head movements (other than just facial look and gestures). However, all I see is examples of subject A's face being applied to subject B (while keeping B's gestures and B's head movements). Ok, it's a great thing that using this technique no visible artifacts appear when B moves his head, but where in the video are A's head movements shown to be transferred to B either in real time or after processing? Just trying to understand… Or did miss something?

  2. I can't imagine the single positive use of this research not even a single maybe due to my limited knowledge . If any one knows please tell me . Thank you

  3. Again, requesting papers on detecting ML generated content, or generating falsified data to destroy accuracy of ML over time. That is where the value is at right now.

  4. Researchers are working on projects like these to make people aware of the power of Deep learning and AI, so some one evil cannot use it for their own good.

  5. @Two Minute Papers
    Thanks once agian.
    Question – If I support you on Patreon, would you commit to say a word about moral implications in every video?
    Like – not just get people excited about the tech but also get people thinking about the consequences to society

  6. The fact that 1) it can do this from one picture and 2) can do this without being trained on the subjects is INSANE. This is basically completely incredible.

  7. Face swapping huh. That's just weird…
    I mean, it can't really be used as an imitation of the target subject, given what it does to faces (very obvious with the first example), but still.

  8. Can someone explain what is the purpose of this research ? It’s cool but for what? Making fake videos? Isn’t it more scary than useful?

  9. Xistin Trudeau was funny, but Alfina King is just nightmare fuel.
    Also, I noticed the trend of connecting more and more neural networks together, each with its specialized part of the problem. I wonder whether universal AI will require an insane number of neural networks or whether it is possible to apply neural networks to more general scenarios.

  10. Seems likely that one promising way forward in AI research is the use of multiple AI networks like this. One can imagine as tech and research grows we might have tens or hundreds of networks eventually each specializing on a small bit of a larger task. Our own brains after all work in similar ways with each region have some autonomy but working together.

  11. 3:59 Wait, does this show significant improvement? SSIM is supposed to go up, right? It doesn't. The landmarks scores have huge uncertainties, and so are basically useless. The only significant improvement is seen for the euler metric.

  12. I'm a bit stumped here. Why would you want to add the facial features (not just expressions) of one person on to the other's face?

  13. no mentioning of the issues, e.g. the blending in of mustages or getting a hard nose -> unsubbed, because you are too optimistic

  14. Last year fearmongers were told to calm down, AI only outperformed humans in specific tasks…now narrow AIs start to work together.
    At this rate the world is going to be freaking awesome in no time 🤤

  15. It's funny how deep state is whining about deep fakes a lot these days. It's like if we ever caught any video evidence of them doing bad things, they'll just say "That's not me, that's deep fake".
    Absolute sickening

  16. The future of censorship is bright 😃 When play store has the apps for voice and face then think of the bad Netflix movies I can make

  17. Another pandora's box has been opened! I for one am pretty excited for the chaos this one will cause and changes they will allow.

  18. Recently you've been focusing so much on ML, maybe you can try something from a different topic soon? Like VR or something, I know a lot of universities are doing VR research.

  19. These are coming along great and always improving.

    I would suggest however that more attention needs to be paid to the mouth area and perhaps the eyes as this is where the illusion gets broken and how it is easy to spot a deep fake from the real thing.

  20. The deep state hope to hell we buy this crap. They need an excuse for their face being on videos showing them at Epstein’s house..

  21. The world of news and social media will go out the window. People will not be able to believe anything. What is true and what isn't ? One good thing that can come out of this is the mutual destruction of politicians. They will destroy each other with relative ease.

  22. "It's just not possible to have an IQ over 200". You should look up the great mathematician Terence Tao and the amazing physicist Christopher Hirata, both which were child prodigies and scored IQs over 220. I may be wrong but maybe there were also few other people that scored over 200 throughout history.

  23. I have been waiting for this combination of AI's. I was also thinking that since AI has to mask a face rules should be preset for AI to create a mask subset of a face.

  24. Curious what possible benefit this tech could have in society, seriously. The facial recognition portion, yeah that has some sue (but also negative use in the ability of increased mass-surveillance)

Leave a Reply

Your email address will not be published. Required fields are marked *