OpenAI’s Robot Hand Won’t Stop Rotating The Rubik’s Cube 👋

OpenAI’s Robot Hand Won’t Stop Rotating The Rubik’s Cube 👋


Dear Fellow Scholars, this is Two Minute Papers
with Károly Zsolnai-Fehér. Today, we’re going to talk about OpenAI’s
robot hand that dexterously manipulates and solves a Rubik’s cube. Here you can marvel at this majestic result. Now, why did I use the term dexterously manipulate
a Rubik’s cube? In this project, there are two problems to
solve. One, finding out what kind of rotation we
need to get closer to a solved cube, and adjusting the finger positions to be able to execute
these prescribed rotations. And this paper is about the latter, which
means the rotation sequences are given by a previously existing algorithm, and OpenAI’s
method manipulates the hand to be able to follow this algorithm. To rephrase it, the robot hand doesn’t really
know how to solve the cube and is told what to do, and the contribution lies in the robot
figuring out how to execute these rotations. If you take only one thing from this video,
let it be this thought. Now, to perform all this, we have to first
solve a problem in a computer simulation where we can learn and iterate quickly, and then,
transfer everything the agent learned there to the real world, and hope that it obtained
general knowledge that indeed can be applied there. This task is one of my favorites. However, no simulation is as detailed as the
real world, and as every experienced student knows very well, things that are written in
the textbook might not always work exactly the same in practice. So the problem formulation naturally emerges
– our job is to prepare this AI in this simulation so it becomes good enough to perform well
in the real world. Well, good news, first, let’s think about
the fact that in a simulation, we can train much faster as we are not bound by the physical
limits of the robot hand – in a simulation, we are bound by our processing power, which
is much, much more vast and is growing every day. So, this means that the simulated environments
can be as grueling as we can make them be, what’s even more, we can do something that
OpenAI refers to as Automatic Domain Randomization. This is one of the key contributions of this
paper. The domain randomization part means that it
creates a large number of random environments, each of which are a little different, and
the AI is meant to learn how to account for these differences and hopefully, as a result,
obtain general knowledge about our world. The automatic part is responsible for detecting
how much randomization the neural network can shoulder, and hence, the difficulty of
these random environments is increased over time. So, how good are the results? Well, spectacular. In fact, hold on to your papers, because it
can not only dexterously manipulate and solve the cube, but we can even hamstring the hand
in many different ways and it will still be able to do well. And I am telling you, scientists at OpenAI
got very creative in tormenting this little hand. They added a rubber glove, tied multiple fingers
together, threw a blanket on it, and pushed it around with a plush giraffe and a pen. It still worked. This is a testament to the usefulness of the
mentioned automatic domain randomization technique. What’s more, if you have a look at the paper,
you will even see how well it was able to recover from a randomly breaking joint. What a time to be alive! As always, some limitations apply. The hand is only able to solve the cube about
60% of the time for simpler cases, and the success rate drops to 20% for the most difficult
ones. If it gets stuck, it typically does in the
first few rotations. But so far, we have been able to do this 0%
of the time, and given that the first steps towards cracking the problem are almost always
the hardest, I have no doubt that two more papers down the line, this will become significantly
more reliable. But you know what, we are talking about OpenAI,
make it one paper. This episode has been supported by Weights
& Biases. Weights & Biases provides tools to track your
experiments in your deep learning projects. It can save you a ton of time and money in
these projects and is being used by OpenAI, Toyota Research, Stanford and Berkeley. Here you see a write-up of theirs where they
explain how to visualize the gradients running through your models, and illustrate it through
the example of predicting protein structure. They also have a live example that you can
try! Make sure to visit them through wandb.com/papers
or just click the link in the video description and you can get a free demo today. Our thanks to Weights & Biases for helping
us make better videos for you. Thanks for watching and for your generous
support, and I’ll see you next time!

100 thoughts on “OpenAI’s Robot Hand Won’t Stop Rotating The Rubik’s Cube 👋

  1. Hey i have a question. Isn't machine learning and Ai just a trial and error machine ? Very sophisticated but in the end just a trial and error automatism, right?

  2. why not give it 2 hands instead of one? train 1 hand later when it is capable enough to perform this with 2 hands. 1 hand is difficult even for us humans.

  3. I don't like the way the hand bobbles up and down. They should add some extra smootheness constraints. Also that little finger is way too dextrous for a human!

  4. Will the hand ever be able to move the cube like human one handed solvers do? In what ways is this hand significantly different to a human hand (how it moves, texture, etc.)?

  5. Who else laughs when he says "hold on to your papers"?
    That just cracks me up.
    Fantastic work. Looking forward to the next two papers

  6. Have there been any studies about making accurate inverse kinematics? They are difficult to make and I wonder if anyone's made one with neural networks

  7. lol. just going through ur videos and randomly found out ur working in the TU in vienna. hello fellow vienece xD great videos. i wish my lifepath would have driven me in the same directions. this is so amazingly interesting what AI evolved in .. Keep up the amazing content. Super videos ^^ bin gespannt was die zukunft bringt ^^

  8. Imagine if they made a prosthetic hand with that… then you could solve the Rubik’s Cube one handed, blinded and singing an Eminem song while balancing a watermelon on your head

  9. Shocking realisation.. we are an AI trained in an simulation to get to spaceflight or other things, but instead it creates a loop that all iterations start using AI in a simulation to do it.. 😀

  10. Are you saying the AI robotic hand solve it without visual feedback, so if it miss a move or don't know the start position it is lost?

  11. Why give robots the shape of a human? Maybe when it had like 7 fingers on its hand it could be more capable? Has anyone tried to let AI generate/evolve its own limbs and then build it in real life to see how it manages to move around?

  12. I had the video playing upside-down for the first 3 minutes. How hard would it be for this AI to solve the Rubik's cube holding it upside down?

  13. They should have modeled a rubber palm so the learned strategy could inform professional rubix cube one handed competitions on better solving trees.

  14. This may also open a new way of viewing prostheses. While full brain-computer communication would be the optimal solution, this can potentially be better than stuff we have today. Instead of directly controlling the hand motors individually, you send a signal for it to achieve a specific task, kind like of how we don't think about the movements we have to do to pick up a pen and just think "pick up the pen".

  15. 1:43 Being only limited by available computing power makes learning an almost timeless effort. This is comparable to the Dragon Ball Z time capsule where they could train fighting forever without any "real" time passing.

  16. In case anyone wondered, sub-10s one handed 3×3 speed cubing is a thing… still damned impressive for this hand. https://youtu.be/3FDmmL5vKHg

  17. Elon: you need to make me a dextrous hand a hand so dextrous it can perform various rigorous hand motions even while wearing several rubber gloves …..but Elon why would anyone ever need such a …. JUST DO THIS FOR ME :O)

  18. I don’t even understand anymore whether what I’m watching is rendered or not 🤣🤣🤣 not talking about AI or not AI

  19. They keep pushing that hand idea… yet to me, it seems it's wrong in principle: The way they use the hand is so suboptimal…
    – Can the wrist even rotate? I move my hand above and next to stuff too…
    – It could also be useful to have a table…

  20. Success rate actually drops to 0% for the most difficult case (as the table you include clearly shows). But p good vid otherwise.

  21. I am assuming it is flawed if it has not been trained with being able to see the end of a layer in mind. This would cause different and potentially better transitions between hand poses, otherwise its pretty!

Leave a Reply

Your email address will not be published. Required fields are marked *