Spencer Huang Discusses NVIDIA's Physical AI Stack for Robotics

0:00 you know, the um I think one of the the  first use cases that I would love to see a humanoid robot is is really take care of uh  you know, grandma and grandpa. A lot of times um as as our our folks get older, there's a couple  of things that are pretty common among all of it. One of them is general accessibility. It becomes  really hard to go through life when um when you're kind of limited in and out of physical capability  or mental capability. The struggle was not making the car work. We already had a working car. The  struggle was trying to get the computer into the car and making sure that it was able to, you  know, adapt to whatever task that we were trying to give it. Robotics and especially humanoid  robotics is struggling from both at the same time. It's when we started being able to create  experts. Experts are what becomes valuable to us.

0:45 And so even in robotics, a generalist baseline is  just the start. And so we're all kind of fighting with a with a data challenge just trying to  get there. And so once we get to a baseline, we can start working on some really really complex  and and valuable problems. The other thing is over time we'll be able to start giving these robots  personalities and the benefit of a personality is it also solves for things like loneliness. I'm at Nvidia GTC in Washington DC and I'm joined by Spencer Huang. Nice to you. How are you doing? Thanks for having me.

1:15 So you're a product lead at NVIDIA for robotics.  Yes. I wanted to start with what was your favorite part of Jensen's keynote. you know the I think my favorite parts, and this is because  I'm always biased towards the robotics, is the physical AI section. I think ⁓ it's  not that it's a favorite because it was this keynote. I think it's a favorite because if you  watched the last couple of keynotes, let's go back a couple of years, you see the progression of  the narrative of where physical AI is going. And it started with autonomous vehicles. And then we  announced Project GR00T and you start seeing ⁓ the dream, if you will, of what we want to  accomplish for humanoid robotics. And as we start coming in today, you start seeing  things really becoming ⁓ into fruition, right? All the work that our teams are doing  are finally starting to materialize into these fundamental technologies that are going to unblock  us in the future. And the other thing is...

2:18 There's a lot of amazing technology that he  showcased, not only for the software side, but the infrastructure side. One of the  biggest challenges that we're going to run into for robotics, and we're already running  into it, is how do you develop the infrastructure? that is required for physical AI because it's  actually inherently different than the type of infrastructure that you might need for LLM.  And the reason for that is large language models are running across homogenous hardware.  They don't need to be necessarily running across heterogeneous compute at the same time.  Whereas for robotics, for autonomous vehicles, we have this concept of hardware in the loop.  And so at some point, you need to be able to test and evaluate the policies that we train. and then be able to put them onto the hardware and you want to test that before you go into reality.  And so we do need to have a somewhat different architecture for the infrastructure that allows  us to do this type of simulation and testing. And so all of the infrastructure fundamentals that he  is revealing today in terms of how we're building out these data centers, we get to leverage those  and use it as a launch pad for how we're going to start building out the infrastructure for  tomorrow's physical AI use cases as we start getting further and further along on the scaling. And I love that he mentioned humanoid robots a lot. ⁓ maybe in last one year, there has been this  explosion of humanoid robot demos on sim to real using reinforcement learning. Robots are dancing,  like walking confidently, doing athletic moves.

3:50 So what has been the unlock in last maybe one  or two years that was not possible before that? Sure. It's been a few things. The first  is just like AI, when we think of AI, we've known the algorithms that we wanted to run.  We just didn't have the compute for it. so the AI winner was more of a hardware issue than it was  a software issue because the math was there, but the hardware wasn't. In a similar sense, when  we think of how we're actually starting to see really impressive achievements in these trained  policies across a condensed period of time. A lot of it comes down to the hardware is  now available and so the software is now learning to be optimized for it. The other  thing that that gives us is the faster that you can start running these RL training, these  RL workloads on top of the hardware, it allows us to start thinking of how are we optimizing  or developing new algorithms that are going to solve for these things. And so two big things  really came out. One of them is flow matching.

4:54 And so if you're unfamiliar with flow matching,  the idea is that you start from a state, and I need to be able to do a task. And so instead  of saying, I need to pick up my arm and actuate my joint this way and angle it this way, you're  trying to create a path. And so you create this flow that gets you from some noisy state where you  start to some end state. And so that allows you to be dynamic in the sense that it's not always going  to be in some static environment. So that's one. ⁓ And it's a high level description. Sorry if I'm a No, no. Terrorizing it. On the other side, there's also deep mimic, which is something that  we've seen very heavily across ⁓ Unitry and Boston Dynamics. And this one is extremely important.  It's the same concept as warm starting when you're training a model. You take a human demonstration  and then you have the algorithm trying to find the optimal reward function that is going to  allow for this agent to be performing the same type of action. And you're trying to adjust for  those deltas and get them as close as possible.

5:53 And so what you end up seeing is you could take  a human demonstration and then train a robot to break dance or to run or to do parkour like  you've seen with ⁓ Skild AI lately. And that type of research is really starting to push  the boundaries on what we know in terms of how do we warm start these? Because if you start  with RL basic, it's a huge search space. You define a reward function. You say, OK, trial and  error. I'll give you a cookie the closer you get.

6:22 The problem with that is it takes forever. And  so just like LLMs where you want to bootstrap it, you can warm start RL. That's kind of  bootstrapping RL. How do you take known demonstrations and then allow the RL agent  and define a reward function that's able to start taking in these demonstrations and then  to be able to perform these types of tasks? And so those are two of the techniques that  have really come out in the last couple of, ⁓ I time flies. I want to save months.  Maybe it was a better part of a year.

6:50 but these are things that are coming out at  exponentially greater speeds. Because once you get an inkling of success, then the further  research really starts to come out quicker because now people have access to the hardware and they're  more accessible. And so they can start doing more more experiments. And so that's definitely what  I believe is probably one of the key factors. And we seeing these techniques  being used for manipulation as well. Do you see reinforcement learning combined with  simulation getting adopted for manipulation? yeah.

7:24 Yeah, So one of the paradigms and relative, one  of the paradoxes of robot learning is that we have two kind of trains of thought. You've got behavior  cloning or imitation learning where I show you exactly what I want it to do and it's gonna  learn to perform that task, but it'll be brittle, right? It'll be good enough in an environment  where there might be slight perturbation, slight. dynamic elements to it, but it won't be something  that's relatively generalized. On the other hand, you have these reinforce learn policies where  the search space is so large and the reward functions are so hard to define. When you  hear that someone trained a policy in 15 minutes but it took them six months to deploy, a  lot of it's tuning. And so the problems that...

8:10 you run into for RL is that the search base  is so large, but the limitations you run into for imitation learning are that they become  brittle and really specialized and overly focused to a single set of tasks. And so when you  combine them together, especially in simulation, you can start seeing quite a bit of improvement.  And now it's not the only way to train these, but we do see that, for instance, we have a  policy that we trained at NVIDIA called Compass.

8:33 And it originally started from a model called  X-Mobility. And the pipeline for that was we will take a ton of demonstrations of  navigating an AMR ⁓ or quadruped. And so it's a cross-embodied mobility model. ⁓ These  different embodiments navigating through spaces, gathering all the data, training it up so that  way they could do it with imitation learning. And then we fine-tuned it with RL so that  way the policy itself became more robust. And so now let's apply this to manipulation.  One of the things that we often think about is We want to get to an end-to-end model. We want  to have fully generalized autonomy for these types of manipulation tasks. it's very difficult  the more dexterous the task becomes. And so we like to split it up. You have gross manipulation  where I pick something up and I put it down. And these are the tasks that we see today. But the  tasks that are highly valuable, the ones that are really going to move the needle in the future,  are to be things, well, how do I take a flex cable for a microelectronics assembly and start  attending these? How am going to take really small things and start assembling them. ⁓ I  have multi-step contact-rich assembly where I have a bunch of gears I need to put in  a certain order and make sure that they're all working like these are things that you can't  do with general pick and place. But they are things that we can use either traditional ⁓ tamp  methods, So tamp being: task and motion planning.

9:52 Or you could use these learn methods. So  how do we combine all of them together? And so simulation actually gives us the benefit  of learning how we can combine them together, how can we start training these policies. And to  some extent, even if we don't use RL explicitly, all of these other methods become data that we  can then use as, again, for these other learning methods. And so it all kind of comes together.  so we're fully believe in RL and simulation for dexterous manipulation. I see.

10:18 ⁓ So you are a big proponent of open  source. You have opened Isaac Lab, open sourced it. You also have a lot of models  that are open source. So how do you decide which part of the ecosystem should be open source  and which part of it should be proprietary? Sure, that's a good question. ⁓ think, so I can  answer very specifically for robotics. think that the strategy might be different across other  parts of the company, of course. But in general, we want open source as much as possible because  we want people to be able to build off of these technologies. Not everyone is going to have  the capability or the resources to pre-train a model. And so by providing pre-trained models  and platforms that they're able to post-train on, obviously this is great for the... the ecosystem because it means that it's not only held within the hands of a few.  So you can democratize this. For robotics, we treat it very much like I look at like the  Linux ecosystem. It's very much similar players.

11:17 And one of the things that's really beneficial  about the open-source ecosystem is people have a lot of different applications that need to be  built and they have a lot of different operating domains that they want to deploy into. And so unlike an autonomous vehicle where it's typically on a street and we  know that the operating domain that even though it changes between regions and you  know, there's a lot of variation there, of course, but it's not as much as we have  completely different worlds that we're going to be working with inside of robotics. You have  industrial manufacturing, you have warehousing, you have sub-aquatic, we're going to have aerial,  you have ⁓ space, right? There's just tons of different domains that we couldn't possibly  cover by ourselves. And so by open sourcing, all of this. It allows the community to  both contribute to the platforms that we can uniquely add value to. And so we choose the,  we like to choose at Nvidia the problems that are extremely hard that we can uniquely add value to.  And we don't try and compete with the ecosystem on things that they're doing because we want people  to succeed. And so if we can help them succeed, we're extremely, extremely happy about that.  And so we want to build these base platforms, open source them as much as we can, allow  people to contribute, allow people to let us know what can be improved and then allow  other people to help each other. And so that's the only way that we could actually solve  for general robotics autonomy, just general robotics in general. Anything that moves is likely  going to be a robot. And so how am I going to, as an individual, be able to contribute to all  these domains and then even as a company or even a single country, how are we able to cover all  these? And so it definitely is something that we believe truly is an open source problem. You're breaking records with GR00T models on hugging face and you also have opened  the Cosmos model world models. ⁓ How do you get people to break records on Cosmos?  What kind of data they can contribute with to accelerate the development of Cosmos? Yeah, I think for both Cosmos and GR00T, there's a mix of data, but it's not necessarily  been contributed to the pre-training. It's really about post-training. And like I said, if  we can provide a really great base model, and we provide the means by which you can go  and post-train it yourself, and then we give you as many references and as many examples as  we can, and tons of data sets, right? So we're trying to constantly the world's largest. physical AI data set. And the reason for that is because if we use it in our work, then we know  that it's high quality. And it also means it gives people a reference for how do you capture data?  How do you consider, how do you quantify quality of data? And so all of this together allows  people to take ⁓ a model like Cosmos or like GR00T N1.5 and be able to post-train it for their  tasks. Now, Cosmos is more commercially viable because the license is less restrictive. And it  really comes down to the fact that it is, you pre-trained on commercially viable data. On the  other hand, GR00T is, it's really fascinating and it's really exciting that the ecosystem is  adopting it so quickly. And they're adopting it even though we're a company that doesn't build  robots and we didn't even have a data collection farm. And so we used open source data. And what  we realized through open source data, some of it's really good, some of it's medium quality, some  of it's not such great quality. But either way, the majority of it was research license only. ⁓ This leads me back to why open source. Well, if we can open source in a way that becomes  commercially viable, then we're able to start pre-training a model that people can then  deploy in commercial and enterprise and in real professional applications. And from there, they  can start collecting more and more data and then post-train these for their tasks. And it can  create a data flywheel within the ecosystem.

15:00 So that way, we can have a good baseline.  And one way to think of it is if chat GPT 1.0 was so, it was actually, it was very good. No  one is gonna deny it. It was an extremely good 1.0. I not many 1.0s come out that strong, but  it came out that strong because the corpus of data existed. The VLAs that we're putting out  and that most of the community is trying ⁓ to, they're all doing amazing work. It's not to  speak ill of any of the work. This is hard and amazing work that our teams are doing.  However, everyone knows that we need more data. And the more data that we can have in the  ecosystem, the better that baseline becomes.

15:36 And then we can start training for more useful  tasks. And so the adoption rates that we've seen are really, it's an indicator that the ecosystem  and the partner network and the professional enterprise and commercial customers, they all want  to be using these engine models because they see the value in it and so they're all kicking the  tires. But when they kick the tires, they also realize here's some fundamental limitations. And so to me, it's a wonderful way of trying to get the community to rally together to start  collecting data and start contributing it back so that way we could have a shared baseline  because it doesn't actually benefit one person or the other. A baseline like ChatGPT 1.0 didn't  make any one company more useful than the other if they brought it into their company. It's  when we start being able to create experts.

16:21 Experts are what becomes valuable to us. And so  even in robotics, a generalist baseline is just the start. And so we're all kind of fighting ⁓ with the data challenge, just trying to get there. And so once we get to a baseline,  we can start working on some really, really complex and invaluable problems. Okay. And so, Nvidia has quite a comprehensive stack for AI for robotics. And ⁓ there  are a lot of companies on the application layer. Where do you see those companies  creating most unique value so that they can become more competitive in this space? And just so I understand the question, at the application layer there's an ecosystem  and a competitive marketplace. And so how are they leveraging these technologies  to become, to add value or...

16:49 I would say what kind of, so on application layer  also there are pieces of the puzzle like data collection, manufacturer robots, operator robots,  and there are various types of applications. So where should the robotics company focus more  on so they can create more differentiation and be more competitive leveraging your platform. Sure, I see, First, let's bucket the ⁓ robotics customers. think broadly, let's do rough  justice. Broadly, you have companies that are building the hardware, companies that are  building the brains, and companies that are building both. And depending on which one they  are, they have their own set of challenges, their own set of opportunities. I think  the ones that are building the hardware.

17:38 the ones that are really trying hard to get the  mechatronics as converged as possible, as mature as possible. Because remember that when we started  with autonomous vehicles, we already knew how to build a car. The struggle was not making the car  work. We already had a working car. The struggle was trying to get the computer into the car and  making sure that it was able to adapt to whatever task that we were trying to give it. Robotics, and  especially humanoid robotics, is struggling from both at the same time. We have hardware. where we are not quite sure what is the appropriate or really the most effective ⁓  morphology or even mechatronic configuration.

18:17 Even the sensor configuration can change. ⁓ And  it really depends on which environment you're going to deploy them into. And so they have to  focus very, very heavily on being able to narrow their scope so that way they're not trying  to create an overly generalist robot. Some of them might be an overly generalist robot. they're not quite overly generalist. so, for instance, ⁓ figure, right? They're able to  start thinking of both industrial and consumer, but the way they're thinking of it is actually  the types of tasks between these two are similar enough that the morphology is ⁓ relatively  convergent. So they can build some really, really great hardware. The ones that are gonna be  going after really hard industrial problems are gonna have different morphologies. And so because  of that, they have different struggles. And then, the brains, the Skild's of the world. They're  generating a lot of ⁓ skills within a ⁓ single model that can deploy across multiple embodiments.  And their challenge is how do they find the beachhead within an industry where they can  start applying this to a very specific set of tasks across a known set of robot embodiments that  are acceptable within that task, that environment.

19:30 And then of course you have the ones that  are doing a full stack. And so that's also, you know, figures doing that as well. And so  they'd be a good example where not only are they thinking of what the brain should be, but  they're also what the hardware should be and how to pair it. And we see this across many of these  companies where they're building up the pipeline and the frameworks for which they can develop the  brain as well as all the mechatronics bit and how they can test it. But it's always coming down to  how do we find like the right use cases in the right order? And when I say the right order, You have to rethink of it as we're training, let's think of it as synthetic humans in some  regard. And this is kind of how we think of it.

20:11 A baby is going to learn at X speed and then if  we go up through high school and through college, we have some knowledge of skills and then we go  into the marketplace and we still have to post train ourselves. You hire someone out of college  and they're smart, but they don't know anything about how to do it in the business because each  business is unique. And so it's definitely a... compound, ⁓ it's a skills tree. How do you start  with the atomic skills? And then how do you start building a method by which you can compound  those skills to do longer, more complex tasks, longer horizon tasks, and then be able to evaluate  how well those skills are being performed. so identifying the right market and the right ⁓  application space that allows you that type of growth allows you to build out your skills tree. That's very cool. I want to wrap it up with a question. What would be the one reason  you would want a humanoid robot in your home?

21:02 You know, I think one of the first use cases  that I would love to see a humanoid robot is really take care of, you know, grandma and  grandpa. A lot of times, as our folks get older, there's a couple of things that are pretty  common among all of it. One of them is general accessibility. It becomes really hard to go  through life when you're kind of limited in physical capability or mental capability. having an assistant that is there to make sure that you're cared for. so having  perceptive models that are able to understand emotional states and pain states, physical states,  the ability doesn't have to carry 100 pounds, but enough to carry ⁓ medicines and food and  blankets, things to keep them comfortable. The other thing is ⁓ over time, we'll be able to  start giving these robots personalities. And the benefit of a personality is it also solves  for things like loneliness. And so I would say, That's that's one of it in the house, but you  could also say that that extends very much into the hospitals where nurses you know I've  I've been using this example a bunch this trip ⁓ Nurses are extremely overworked. They have too  many patients too many tasks, and they don't have enough bandwidth And so how can we use robots  to? Alleviate some of those tasks that could be sued you know just you can give it off to  robots not that it's gonna be replacing a human It's that it's augmenting so that way the human can go and do better work on the patients they do have so that way they can focus and give better  quality care. And so the outcome is much more positive. And I believe that these are two of the  most valuable initial use cases because it's not superhuman dexterity. It's things that you can  imagine happening in just a few short years.

22:50 That's awesome. Thank you so much for chatting.  Thank you. This was fun. Thanks, Spencer.