Nvidia Wants to Make Humanoid AI Robots Safer Around Humans

0:00 Nvidia wants to make humanoid robots safer around human beings. Allow them to be in closer proximity to human beings. Deepu. Tesla. Why? Well, Ed, first of all, thank you for having me. As you can imagine, like, physically, I in robotics is quite simply the largest opportunity in front of humanity. We've been trying to solve this problem for over 50 years with automation. However, in the world, you know, we need increasingly more intelligent robots. And as robots become more intelligent and work with other robots and humans beside each other. Safety, of course, is extremely important because in the physical world, anything that can go wrong is going to be much, you know, the damage can be extremely large.

0:43 So so what we do at Nvidia is we don't build robots ourselves, or humanoid robots for that matter. We work with every company, and we provide the core technology that will help companies build robots that are intelligent, safe and reliable. Deep in my experience of interacting with humanoids in an industrial setting or in an office setting is the default right now to this generation of hardware is for them to kind of stop short, keep a safe distance right as they perceive the world around them.

1:14 They play it a bit safe. What is it that you are trying to accelerate? What is is the kind of literal proximity that you want the humanoid robot to be able to get to the human on? Yeah. So I think there's five things that we actually need for robotics to really proliferate. The first thing is they need to be intelligent and capable. And second, we need them to be reliable. Number three, they need to be safe.

1:39 Remember, for of course they need to be economical. And lastly, I think, you know they need to be not creepy right. So right now if you look if you look at where all the best in the world, from research labs to startups to, you know, universities and so on, so far they're all working on trying to make these robots general purpose intelligent, right? We haven't reached that point yet where they become useful in the physical world.

2:02 If you think about what happened in the last 3 or 4 years in the digital world with ChatGPT and now Anthropic and Gemini and everybody else, the accuracy has gone higher and higher. However, there's almost always a human in the loop. Like, for example, if you're using cloud to summarize your email or summarize a document, you probably, you know are going to see the final result, I'm going to tweak it. But when it comes to physical robots in the real world, there's not going to be a human in the loop. So the accuracy requirements are 99 point. How many nines is at nine 910 nines?

2:33 15 nines depending on the application. So it means we need the general purpose intelligence to become accurate enough. That's kind of where we are right now. And along the way, as we are improving the accuracy, safety has to be designing all the way from, you know, from the chip level to this hardware system level to the operating system level to the app level. And ultimately it needs to be explainable. So that third party, you know, systems are third parties can actually validate that you are actually, you know, certify that the system is being built in a safe manner. Could we think about this from a sort of localized compute perspective inference, you know, at the edge within the brain, so to speak, of the humanoid. A humanoid is walking toward me, and it has a very heavy object, and it senses I'm there.

3:22 Like, what has been the breakthrough on, on on the compute side that Nvidia has done for what goes into the humanoid robot. Yeah. So we've been building a computer for, you know, robotics, actually, the three computer problem. The computer you're talking about is the third computer, which goes inside the, you know, inside the robot in the brain. But we also need a computer for training the brain and the computer for testing, which happens to be in simulation of Omniverse platform. So now coming to the third computer question, what needs to happen? We started working on this more than a decade ago because this you know, Nvidia Jetson.

3:55 And number one, of course, it needs to have high amount of compute capability because models are going to become more larger in order to make them more intelligent to hit the accuracy mark. Of course, it needs to be real time. It needs to be safety. It needs to be designed from the ground up. right? Energy efficiency also matters. And of course, it also needs to be general purpose programmable. And that's kind of what we see with our Nvidia Jetson computer, with over 2.5 million developers that, you know, are active on the platform, more than 10,000 companies that are building all sorts of robots, some humanoids, but all sorts of other embodiments as well.

4:29 And so that's kind of what, you know, what's needed for robotics, general purpose robotics to happen right now in the field of humanoid robotics, what is the biggest barrier to progress software or hardware? Um, I think there's been tremendous amount of mechatronics miracles that have been done, except for the hands that are being worked on. But I think still, the number one problem remains the brain has to be reasonably, you know, general purpose and accurate. And we haven't hit that. You know, what happened with ChatGPT in November 2022 where you had that that moment, that moment for robotics is right around the corner. But we haven't reached that yet.

5:06 Let's bring this back to safety. To end. You say that the halo system that you guys talked about earlier this week brings, um, awareness to humanoid robots. Define what you mean by awareness. But also what is the state that you want that humanoid to get to from where it is currently today? Yeah. So I think the safety is a multi-layered, uh, you know, ecosystem. You've got to start first making the SoC or the chip needs to be functionally safe because Nvidia we've been working on autonomous vehicles for over a decade.

5:40 We invested over 20,000 engineering years into making, you know, all sorts of, you know, safety techniques inside the SoC. That's the first layer. And we now we are basically taking all of that investment and adding it to general purpose robotics. And then at the system layer, the hardware layer, you have safety microcontrollers that can, you know, for redundancy reasons, you can check against, you know, if the main processor is off for whatever reason, you can do that. And then the operating system above that, you know, whether it's Linux, ah, safety operating system or QNX or a combination of all of these. They're all enable you to operate them in a safe and explainable manner. And then on top of that, you have these algorithms, these AI models, if you will.

6:23 You use a combination of traditional computer vision models to use a combination of now the latest large language models and vision language models. And you can use reasoning with those models which reasoning you can use it for explaining what's why the model is doing what it is doing. You know, you can kind of see all of that. And then lastly, on top of that, you bring in all these certification bodies which are independent third party that can, you know, essentially look at the whole stack on a case by case basis, use case by use case, deployment, deployment.

6:55 They can actually go through the full analysis as to always, you know, how safe is this deployment going to be. And you can actually do all of this evaluation in simulation first millions of times over before you bring the robot into the real world. So I think that's kind of like the breakthrough is, you know, leveraging all of the full stack, but being able to test it in simulation first.