This AI Captures Your Hair Geometry…From Just One Photo! 👩‍🦱

This AI Captures Your Hair Geometry…From Just One Photo! 👩‍🦱


Dear Fellow Scholars, this is Two Minute Papers
with Károly Zsolnai-Fehér. In this series, we talk about research on
all kinds of physics simulations, including fluids, collision physics, and we have even
ventured into hair simulations. If you look here at this beautiful footage,
you may be surprised to know how many moving parts a researcher has to get right to get
something like this. For instance, some of these simulations have
to go down to the level of computing the physics between individual hair strands. If it is done well, like what you see here
from our earlier episode, these simulations will properly show us how things should move,
but that’s not all – there is also an abundance of research works out there on how they should
look. And even then, we’re not done, because before
that, we have to take a step back and somehow create these digital 3D models that show us
the geometry of these flamboyant hairstyles. Approximately 300 episodes ago, we talked
about a technique that took a photograph as an input, and created a digital 3D model that
we can use in our simulations and rendering systems. It had a really cool idea where it initially
predicted a coarse result, and then, this result was matched with the hairstyles found
in public data repositories, and the closest match was presented to us. Clearly, this often meant that we got something
that was similar to the photograph, but often not exactly the hairstyle we were seeking. And now, hold on to your papers because this
work introduces a learning-based framework that can create a full reconstruction by itself,
without external help, and now, squeeze that paper, because it works not only only for
images, but for videos too! Woohoo! It works for shorter hairstyles, long hair,
and even takes into consideration motion and external forces as well. The heart of the architecture behind this
technique is this pair of neural networks, where the one above creates the predicted
hair geometry for each frame, while the other tries to look backwards in the data and try
to predict the appropriate motions that should be present. Interestingly, it only needs two consecutive
frames to make these predictions, and adding more information does not seem to improve
its results. That is very little data. Quite remarkable. Also, note that there are a lot of moving
parts here in the full paper, for instance, this motion is first predicted in 2D, and
is then extrapolated to 3D afterwards. Let’s have a look at this comparison – indeed,
it seems to produce smoother and more appealing results than this older technique. But if we look here, this other method seems
even better, so what about that? Well, this method had access to multiple views
of the model, which is significantly more information than what this new technique has
that only needs a simple monocular 2D video from our phone, or from the internet. The fact that they are even comparable is
absolutely amazing. If you have a look at the paper, you will
see that it even contains a hair growing component in this architecture. And as you see, the progress in computer graphics
research is also absolutely amazing. And we are even being paid
for doing this. Unreal. Thanks for watching and for your generous
support, and I’ll see you next time!

78 thoughts on “This AI Captures Your Hair Geometry…From Just One Photo! 👩‍🦱

  1. Each new video from you shows an incredible advance in AI. I even think that maybe in like, two to four years, you will upload on a daily basis research on things that now seems like pure science fiction

  2. Everytime you make a video on a subject I can't help but think of the application of these things in video games, naughty stuff or even propaganda.
    In the future, everything will be made possible, and more!

  3. “And now squeeze that paper…” — the Hold On To Your Papers icon in the corner of the screen should change to Squeeze That Paper icon after those words.

  4. They should make something that captures body and hand poses from just one photograph! Surely that is easier than capturing hair geometry?!?

  5. Hey Safay , can you predict in how many episodes you will present a paper that takes a complete Hollywood movie and extractor on full 3D for us to watch on any angle we want,,, 😜

  6. GTA 6 will add a "draping a wig over NPCs" minigame to its "running a spherical ball over objects" minigame. The entire game is based around soothing a horde of rabid bunnies. I can't wait!

  7. So we have deep fake, and with this man, we can now even replace you Karoly with read 3d Donald Trump model with perfect hair, his voice and gestures in the corner explaining other innovations in fluid simulations or neural networks.

  8. 1:05 A perfectly feminine woman on the left was turned into feminist.

    Watch out for those hair colors guys. The brighter they are, they more mentally sick those women are.

  9. This would work great for digital doubles in movies and TV. When you have to digitally replace an actor, like in a stunt for example.

  10. "Rotate us 75 degrees around the verticle, please." There is a shape change in this girl's hair, we can hypothesize Chris!
    It's actually happened – https://www.youtube.com/watch?v=3EwZQddc3kY

  11. IMO this AI isn't perfect yet. It doesn't seem to be able to handle curled/permed hairs, model elasticity and friction between strands convincingly, and obviously its low sampling size that makes everyone looks like they're partially balding.

  12. I`m a vfx artist in Brazil, and I allways recommend this channel for everyone in my studio, thanks for the great content man ! keep it up

  13. Wow. With this, you could technically recreate a 3D human model from a single pictures.
    If there would be a working solution, I would love to use it.

  14. Hi, I am not a scholar at all, just a young lad interested in those fields..
    What can be the point of recreating the movement through video ?
    Would it be better to create the hair and then applying some physic engine instead of following it frame by frame ?

  15. Elképesztő, hogy milyen hozzáértéssel és kristálytiszta angol tudással, érthetően magyarázod el a témát. Megy a feliratkozás. 🙂

Leave a Reply

Your email address will not be published. Required fields are marked *