Transcript
0:00 Yesterday on the show, we talked about the traction Google's TPU has been getting in the market. Today we are double clicking on one important element of that story. Google's TPU software deserves a lot of credit for the new found traction. My colleague Ryan Barwick, who covers Google, wrote about that. I want to bring him on to talk us through his reporting. Ryan, welcome to the show. It's great to have you here. >> Hey man, what's up? How's it going?
0:25 Excited to be here. >> I'm doing good. Okay, so let's talk we have talked on the show extensively about CUDA and we just heard from Chris Miller about software being such a moat for NVIDIA. There's another software out there called PyTorch and you know, I guess we haven't talked about it so much. So what is PyTorch? Is it a rival software? >> It's not exactly a rival software. It was developed by Meta in I believe 2016 2017 and it is essentially this like framework to put these chips into action.
1:03 CUDA is specific to NVIDIA, obviously, but PyTorch is used to I guess kind of speak to CUDA to get things rolling there and it has been widely adopted throughout the industry. >> And so so help me understand this. So you you could conceivably be using if you're a chip customer, both CUDA and PyTorch because PyTorch has to connect to CUDA. Is that the idea? >> I I believe so. I'm still relatively new to the beat, but as I understand it and as I've been kind of focusing on these tools that kind of get these chips integrated into obviously these data centers and and with the companies that are using them. Yes, that's how I understand it.
1:45 >> Okay. So now that that's PyTorch. Now your story centered on a on a different software called Torch TPU. So is this a new version of the software? Is it related? Is it is it different? What is it? >> So >> is a software layer. That is the terminology that Google and I went back and forth on using. Essentially, Google has these chips, these TPU chips. And it was traditionally built with JAX, which is the kind of Google framework it wants its customers to use. The challenge with JAX is I actually have source describe it as alien technology to me. So it's very complex to use.
2:31 And there wasn't necessarily the it was very clunky, so there wasn't really a lot of adoption. So converting your entire workflow to TPUs took a lot of work. Google has obviously been well aware of that and they roll out Torch TPU. They rolled this out in late October. And the general idea is that it will take PyTorch, which is what the industry was using, and convert that into TPUs. So the the idea is to make it as efficient as possible to use Google's chips.
3:05 And already you're starting to see the traction. I think in in Google's third quarter earnings report, Thomas Kurian, who's leading Google Cloud, he specifically says we think this software update is actually going to really open up the market to TPUs. >> And and Torch TPU is open source, right? >> Yes. Yeah. >> And and and is that is that a big reason why it's gotten so much traction? >> Well, I think the idea is they want to make it as easy as possible to use these chips. Instead of having to go back and forth with Google folks, people can just do it themselves.
3:41 And of course, this is even more kind of within the context of AI getting really good at writing software and kind of converting these workflows. The fact that it's open source is just trying to get as many people on the boat as possible. >> Right. So we had a chip investor Ameet Poddar on the show a couple weeks ago. He was talking about how vibe coding has really helped Google chip customers really better integrate into that ecosystem effectively. Does that resonate with what you were hearing?
4:16 >> Yeah, absolutely. I mean that is that is certainly I think this was reported late last week but in so an executives have told investors they were able to get I think it was AMD AMD GPUs. This was a migration that would have taken 6 to 12 months. They were able to do it in 3 days. so what you will see is it's going to become so much easier to change chips, to diversify your chip strategy and Google obviously wants a piece of that and this is all happening in in the I guess in the light of this Nvidia drama, right? So these these two kind of things are happening parallel. Chips are Nvidia is facing a lot of pressure but also chips are getting a lot easier to use and a lot easier to to diversify and go back and forth between chips.
5:08 >> And so just to again I've got some clarification questions here so bear with me. So so we talked about how PyTorch was sort of not really a competitor to CUDA because you kind of have to use them at the same time. Torch TPU that is a a a a rival to CUDA. >> It's not so much I guess it's a rival in the sense that Google as a chip maker is a rival to Nvidia. >> Yeah.
5:37 >> And Torch TPU is the language that will help people use TPU chips. So I guess in that sense it is a rival but ultimately these are just frameworks to get these chips working efficiently. >> Right, okay. And I mean I guess higher level question here is Nvidia doing anything about this from from what you're hear or you know what's been the reaction in the market? >> Yeah, >> it's funny. It's like so when obviously we're having this TPU story and and we're hearing we're trying to get a sense of how many customers there are for these chips.
6:14 And they're still so Google is expected to make a Morgan Stanley estimates they'll make $81 billion on TPU sales. So obviously this is a very large business, but that adoption hasn't exactly we haven't yet seen the full adoption. I was talking to one cloud software provider yesterday and he said of his 5,000 customers only 16 of them are using TPUs at any real volume right now. So I think it's a little too early to to obviously this is a significant business. we've reported that Oracle and OpenAI are buying TPUs.
6:53 but I think there's still obviously you know a a small part compared to Nvidia's chip business. >> Great. Okay. Well Ryan, I want to thank you for coming on and it's it's your first time on the show. We're going to we're going to have you on more often my friend because Google is getting more and more important in the AI story always. That is Ryan Barwick, our Google reporter here at The Information.