transcribe

OpenAI Robotics: Making robots is hard | Wojciech Zaremba and Lex Fridman

Lex Clips · 9m · transcribed Jun 2026
More from Lex Clips Business
𝕏 Share ▶ YouTube 📥 PDF 🤖 .md

Transcript

0:02 so you led the openai robotics team can you give an overview of of the cool things you're able to accomplish what are you most proud of so when we started robotics we knew that actually reinforcement learning works and it is possible to solve very complicated problems like for instance alphago is an evidence that it is possible to to build superhuman and goal players dota 2 is a an evidence that is possible to build superhuman agents playing dota so i asked myself a

0:35 question you know what about robots out there could we train machines to solve arbitrary tasks in the physical world our approach was i guess let's pick a complicated problem that if we would solve it that means that we made some uh significant progress in the domain and then we went after the problem so um we noticed that actually the robots out there they are kind of at the moment optimized per task so you can have a

1:02 robot that it's like if you have a robot opening a battle it's very likely that the end factor is a battle opener and uh and in some sense that's a hack to be able to solve a task which makes any task easier and um ask myself so what would be a robot that can actually solve many tasks yeah and we conclude that that like a human hands have such a quality that indeed they are you know you have

1:31 five kind of tiny arms attached individually they can manipulate pretty broad spectrum of objects so we went after a single hand like a trying to solve rubik's cube single-handed we picked this task because we thought that there is no way to hard-code it and it's also we picked the robot on which it would be hard to hard-code and we went after the solution such that it could generalize to other problems and just to clarify it's one robotic hand solving the rubik's

2:02 cube the hard part isn't the solution to the rubik's cube is the manipulation of the uh of like having it not fall out of the hand having it use the uh five baby arms to uh what is it like rotate different parts of the rubik's cube to achieve the solution correct yeah so what uh what was the hardest part about that what was the approach taken there what are you most proud of obviously we have like a strong belief in reinforcement

2:31 learning and you know one path it is to do reinforcement learning the real world other path is to uh the simulation in some sense the tricky part about the real world is at the moment our models they require a lot of data there is essentially no data and i did we decided to go through the path of the simulation and in simulation you can have infinite amount of data the tricky part is the fidelity of the

2:59 simulation and also can you in simulation represent everything that you represent otherwise in the real world and you know it turned out that that you know because there is lack of fidelity it is possible to that what we what we arrived at is training a model that doesn't solve one simulation but it actually solves the entire range of simulations which uh vary uh in terms of like uh what's the exactly the friction of that cube or the

3:29 weight or so and the single ai that can solve all of them ends up working well with the reality how do you generate the different simulations so you know there's plenty of parameters out there we just pick them randomly and and in simulation model just goes for thousands of years and keeps on solving rubik's cube in each of them and the thing is the neural network that we used it has a memory and as it presses for instance the side

3:59 of the of the cube it can sense oh that's actually this side was uh difficult to press i should press it stronger and throughout this process kind of learns even how to how to solve this particular instance of the rubik's cube back event mass it's kind of like a you know sometimes when you go to a gym and after after bench press you try to lift the glass and you kind of forgot uh and and your

4:32 hand goes like yeah right away because kind of you got this to maybe different weight yeah and it takes a second to adjust yeah and this kind of of a memory that model gained through the process of interacting with the cube in the simulation i appreciate you speaking to the audience with the bench press all the bros in the audience probably working out right now there's probably somebody listening to this actually doing bench press um so maybe

4:58 uh put the bar down and pick up the water bottle and you'll know exactly what uh what check is talking about okay so what uh what was the hardest part of getting the whole thing to work so the hardest part is at the moment when it comes to a physical world when it comes to robots they require maintenance it's hard to replicate a million times it's uh it's also it's hard to replay things exactly

5:29 i remember this situation that one guy at our company he had like a model that performs way better than other models in solving rubik's cube and you know we kind of didn't know what's going on why it's that and it turned out that you know he was running it from his laptop that had better cpu or or better or maybe local gpu as well and because of that there was less of a

6:00 latency and the model was the same and that actually made solving rubik's cube more reliable so in some sense there might be some saddlebacks like that when it comes to running things in the real world even hinting on that you could imagine that the initial models you would like to have models which are insanely huge neural networks and you would like to give them even more time for thinking and when you have these real-time systems then

6:29 you might be constrained actually by the amount of latency and uh ultimately i would like to build the system that it is worth for you to wait five minutes because it gives you the answer that you are willing to wait for five minutes so latency is a very unpleasant constraint under which to operate correct and also there is actually one more thing which is tricky about robots there is actually no not much data so the data that i'm

6:57 speaking about would be a data of first person experience from the robot and like a gigabytes of data like that if we would have gigabytes of data like that of robot solving party's problems it would be very easy to make a progress on robotics and you can see that in case of text or code there is a lot of data like a first person perspective data on the writing code yeah so you had this you mentioned this really interesting

7:25 idea that if you were to build like a successful robotics company so open as mission is much bigger than robotics this is one of the one of the things you worked on but if it was a robotics company they you wouldn't so quickly dismiss supervised learning i correct that you would build a robot that uh was perhaps one like um an empty shell like dumb and they would operate under tele operation so you would invest

7:56 that's just one way to do it invest in human super like direct human control of the robots as it's learning and over time add more and more automation that's correct so let's say that's how i would build a robotics company today if i would be building a robotics company which is you know spent 10 million dollars or so recording human trajectories controlling a robot after you find a thing that the robot should be doing that there's a market fit for like you

8:26 can make a lot of money with that product correct correct yeah so i would record data and then i would essentially train supervised learning model on it that might be the path today long term i think that actually what is needed is to train powerful models over video so you have seen maybe a models that can generate images like dali and people are looking into models generating videos they're like parties algorithmic questions even how to do it

8:57 and it's unclear if there is enough compute for this purpose but i i suspect that the models that which would have a level of understanding of video same as gpt has a level of understanding of text could be used uh to train robots to solve tasks they would have a lot of common sense

9:38 you

Summary

The discussion focuses on the advancements made by the OpenAI robotics team, particularly in using reinforcement learning to enable robots to perform complex tasks, such as solving a Rubik's Cube with a single robotic hand. The team emphasizes the challenges of real-world applications, including data scarcity and latency issues, while exploring the potential of simulation for training robots.

- OpenAI's robotics team aims to train robots to solve arbitrary physical tasks using reinforcement learning.
- They focused on creating a single robotic hand capable of solving a Rubik's Cube, emphasizing the manipulation skills required.
- The approach involved extensive simulation to generate diverse training scenarios, allowing the model to generalize solutions across various conditions.
- Key challenges include the need for high-fidelity simulations and the difficulty of replicating real-world conditions.
- The team highlighted the importance of latency in real-time systems and the impact of hardware on model performance.
- They discussed the potential of using supervised learning and human teleoperation to accelerate robot training in practical applications.
- Future aspirations include developing models that can understand video, similar to how GPT understands text, to enhance robotic capabilities.
© transcribe · For agents Built with care and craft by Gokul Rajaram