Can NVIDIA Keep Its Lead? — With Anissa Gardizy, Max Cherney, and Lauren Goode

0:00 There have been headlines that of the announced AI data centers that are supposed to come up, something like 50% of them are actually being built. is that the case? And if so, why? Ana, do you want to lead us off? >> Sure. I totally believe that statistic and I think it might actually be higher if you include announcements. because of all the planned data centers that are underway, I think it's highly likely that many will be delayed due to higher costs, how hard it is to get labor. But then the number that I'm keeping close track of is announced projects versus actually projects that are being built. And I think we've seen some, you know, pretty crazy announcements from companies like OpenAI with all of these different 10 gawatt, 6 gigawatt projects. And those are numbers that I'm paying a lot of attention to because I think we need to sort of back into them and say, okay, if you wanted 10 gigawatts by this date, how many do you have today, was that a real commitment? How firm is that commitment?

0:59 But on on projects that are actually getting built, I do think, you know, 50% not really getting done on time is is, you know, what I would expect. >> Wait, what percentage would you say have been announced but not start not included in that 50% number? >> you know, when I think of that number, it's I mean, it's quite high. Like even if you think of OpenAI for example announcing a 10 gigawatt project you know we're not going to see 10 gigawatts in the next couple of years or they're going to sort of spread out that bet and I think that's a really good area for reporters to look at is if you just back into announcements it's a really good way to say hey this project is not not on track.

1:37 >> Yeah it's kind of crazy because if you look at the way that stocks have been traded publicly a lot of the market action that we're seeing is entirely dependent on those announcements coming true. So, Max, let's go to you. what are the consequences going to be if this these announced buildouts don't materialize? >> Well, I think there's a lot of shareholders of these public companies that are going to be pretty frustrated. Microsoft, Amazon, etc. have been making big cackback spats as I think everybody in this room knows. They've been going to the market to get debt now, which is, you know, you know, I think shareholders would be pretty interested. in terms of the consequences though, I mean I'm a chip reporter so that's sort of where I that's how I kind of think about it.

2:21 and it's very likely I would say that we're going to lead we're going to see some overcapacity essentially. So like people are building especially memory companies are building tons of factories right now and typically the chip industry is again I'm sure everybody in this this room knows tends to be cyclical. So, the whiplash from this one might be pretty bad. depending on exactly when it comes to an end, if it does. >> I think what Max is saying is that it's good for his job security because the more news there is, the more he'll have to report on. Yeah.

2:50 >> Oh, you guys are not going to get a break anytime soon. Not at all. >> No, I can't I'm very I'm very curious if the if there is a bust, what that looks like exactly and what precipitates it. I think it'll be a lot of fun to cover. >> Well, one of the big questions >> fun. I also think if I can say too, I think this is a particularly unique time because not only do we have these incredibly highly valued, ambitious frontier labs that are getting involved in these circular deals, but also we're in the middle of a memory shortage. there's this unrelenting demand for compute and also this is a midterm election year. And so I think you're going to see a lot of politicizing of the data centers too as you know sort of lawmakers try to appeal to their base because a lot of people are very unhappy about data centers.

3:36 >> Yeah. I mean they're not pulling well at all. >> No. So you know when you think about where the collapse might happen a popular thing for people to discuss is well maybe Nvidia which has been making such premiums on its hardware it can't sustain it or it gets caught. So Lauren you spent a lot of time with Jensen Wong from Nvidia what do you think about Jensen would enable him to sustain Nvidia's lead or do you think that some of these skeptics have a point?

4:07 yes and yes I think some of the skeptics absolutely have a point and I think once you reach the the sort of is it the zenith is the nadar I always get those two confused that Nvidia has you know people are always sort of looking to to take you down a peg and compete but I think >> Nvidia and Jensen has been incredibly good in Nvidia's history at sort of pivoting the company at exactly the moment that they need to in order to make sure that they've sort of caught the next wave. and we certainly saw that happen with not only like you know the GPU to begin with and parallel processing, but then again with sort of pivoting towards crypto which ultimately meant they were in good place for AI and now we see the company doing that by addressing the inference market a lot more closely too and you know Jensen coming out and making these big proclamations that actually they're the biggest CPU maker in the world which I know Intel and AMD must be thrilled about. So, you know, I think he's very smart and very strategic and that there's a good chance that they do maintain their dominance.

5:11 >> I mean, I think I think the question is like how much of the inference market they're going to get. I mean, it's it's whether it's 80% or 40% or somewhere in between or or 90%. I mean, I think that's what everybody's fighting out at the moment. I don't think there's a question that they're going to have some big chunk of it. Just it's just how much, you know? >> Yeah. Yeah, I'm going to go to An in a moment, but Lauren, I just want you to tell us a little bit of what it was like with Jensen on a cover shoot for Wired.

5:35 >> You're talking about when I brushed his hair. >> That would be it. >> Yeah. Okay. so, >> wait, you actually No, >> I really did. >> Yeah. Yeah. I'll be brushing I'll be lining up later to brush people's hair if anyone would like to. yeah, so I did a cover story on Jensen for Wired a couple years ago and our Wired art department is worldclass by the way and so we had this big photo shoot set up and I asked if I could tag along to the photo shoot because I just wanted 15 more minutes with him to ask him some follow-up questions about Blackwell. And so they let me tag along down to the office in Santa Clara, set up this whole set for Jensen, who had like, you know, exactly two minutes to give us. And as the photographer was looking at Jensen under the bright lights, the photographer said, "Oh, he's got some flyaways and that like gorgeous silver hair that he has." And then he said, "Does anyone have a brush?"

6:25 Silence across the set. No one at NVIDIA apparently had a brush. I was like, "Okay." and as anyone who has long hair knows, you always have a brush. So I said, "Well, I have a brush." So then I went to my backpack and got the brush. And then I said, "Okay, here's the brush." And no one moved. And meanwhile, Jensen is like, "What are we doing here, folks?" And I I said, "Oh, okay." So, I walked up to Jensen and I took out my brush and I got really close to him and I said, "Jensen, we're about to get a lot more close." And he said, "Oh, Christ." And and then I brushed his hair and I have to say, I did bring some evidence of this. I think it looks great, frankly. Oh, wow.

7:02 >> I think it turned out really well. So, yeah. Now, the unfortunate the unfortunate conclusion to the story very quickly because I know we need to move on is that people were joking afterwards like you know the net worth of every individual strand of hair on that brush, right? Like like you could sell this on eBay and which I was obviously not going to do as an ethical journalist. But then like several months later, my back my backpack was in my it was my gym bag. It was in the back of my car and downtown San Francisco. I think you know where this is going. My car got broken into and the brush got stolen. So those thieves have no idea the value of what they got away with.

7:40 >> They are cloning Jensen as we speak. >> Yes. I'll see you in the back afterwards if you want your >> hairbrushed. >> Ana, can you talk to us a little bit about what we've kind of hinted at at this point that there's this, you know, Nvidia, the common p perception on Nvidia is their GPUs were great for training, but you can actually do the inference or the act of using a model on a variety of of different chips. And once people once these labs train their models and they're happy with their models, most of the computing is going to go to these inference chips. and therefore, even if Nvidia has a bet there, they're not going to be able to sustain their dominance. Is is that what do you think about that? Is that a potential flaw in the armor for Nvidia?

8:20 I think it is a flaw and I think there's if anyone's going to sort of attack Nvidia's dominance they're going to do it on inference like you said and there is sort of a massive effort underway right now to make all inference chips under the sun work well and so every single company that buys Nvidia chips and is spending a lot of money on NVIDIA chips is trying really hard to make these other other chips work whether it's in-house chips from Google or Amazon or even in OpenAI's case and potentially Anthropic's case you know do we make our own inference chip. so I'd have a hard time, you know, believing that none of those chips are going to pan out, but you know, they might not tackle the bulk of the inference workload even. But then again, even if they do 10% of your inference, maybe you're saving enough money that you think the effort is worth it and it gives you negotiating leverage with NVIDIA if they know that you have an in-house ship team. You know, Jensen's going to be a little bit worried when negotiating with you when you're playing hard ball with him. So, I think everyone's going to have to have an inference chip answer to Nvidia. But when I talk to data center companies about what they're seeing, you know, some data center companies don't really care what chip goes inside of their data center, but they they try to get hints from the companies that they're working with. on the data center side, they're seeing a lot of NVIDIA and in the instances that I'm seeing where there are non- Nvidia chips, it's because of some sort of financial backs stop. So I do wonder how much how long that will have to keep being the case because that will definitely hinder non- Nvidia chips if they need special financing arrangements to get inside the data center.

9:50 >> Am I wrong in thinking the AI world is sort of separating on two poles? There's the Nvidia OpenAI pole and the anthropic Google Amazon poll, right? So it's almost two separate ecosystems competing with each other. I think OpenAI is investing a lot in non- Nvidia hardware. >> Are they trying to move away from Nvidia? >> Yeah. Yeah, they are. they're they're using chips from other companies. Cerebrris is a good example and they have their own in-house chip. so I think maybe publicly they're they're doing some big announcements with Nvidia and I believe it's 5 gawatts they have to deploy in the next couple of years on Ver Rubin. That's that's a lot and that that in itself might hinder them from doing more. But I think I think Sam Alman wants to diversify from Nvidia.

10:37 >> Why? >> Because you know just like any other company it's so expensive and I don't you know they have their own in-house silicon effort. I don't think they they think that they know their model better than anyone else like their their AI model and so I think they think that they're the best to develop a chip that can run their model efficiently. >> Max, you're back and forth to Taiwan very frequently. when we think about the position of Taiwan and Taiwan semiconductor in this world you know the fact that it's in this like tenuous geopolitical place is something people tend to be like oh okay and then move on right away and I want to hear your perspective on whether the independence of Taiwan and the stability of TSMC is this like hidden black swan event that's just kind of in plain sight that people are not paying attention enough attention to. I mean it it is for some reason the modern world decided to decided to put all of our chip manufacturer or most of it next to Kim Jong-un and the rest of it is in China is in is in Taiwan. I I don't really understand who decided that or why we decided that but that's that's the nature of the beast. chip companies the design companies in the US do not plan for this. The contingency the contingency plan excuse me is something along the lines of like well we're all kind of screwed if China invades. which okay, but like there's no real plan there. so and when I say invades, I don't necessarily mean like a literal invasion. I think what's a lot more likely is some kind of soft power exchange like what happened in Hong Kong over time. I think realistically that's that's a more likely option. Although I'm not saying Xiinping won't just invade. Like he absolutely will and he said he would.

12:20 which could be catastrophic for TSMC and the modern world. again. I I do know that chip companies, as I said, there really isn't much planning and much consideration of it because it's it I think people in the industry just think it's an event that's just so crazy complicated and out of this world that nobody's going to do anything about it. So >> Max, do you want to talk about what happens when you try to visit TSMC?

12:45 >> They send me to the gift shop. >> so >> you're in Taipei and that's where you go. >> That's Yep. That's basically it. They they got a a museum where they have like I don't know a bunch of chips that they've made over the years. One of the cerebras chips is in there. They're very proud of it. The dinner plate size one. and I've never been to the museum, but that's where they wanted to always that's where they always want to take me. I think Wired is one of the few organizations, news outlets that's actually been able to go inside of one of the factories. and they wrote an interesting story about it. but >> yeah, I do not get to go to the TSMT Fabs unfortunately. They're not they're not all that interesting inside. like they're cool if you've never seen one, but they mostly look the same. It's like a bunch of white boxes with robots moving around. so it's not like people that know what they're looking for. It's helpful because you can like count the number of EUV machines in there and figure out roughly how many chips they can make and like do that kind of thing and figure out what kind of tools they're using. But if the tours that most chip companies give are not particularly enlightening in general, >> yeah, >> they are fascinating though.

13:42 >> Yeah, they're cool, but they're just they're not useful. >> Yeah. in in like a meaningful reporting sense or an investor's you know from an investor's perspective or something like that. >> I had gotten very excited about TSMC did a ton of reporting spoke with former employees there and then got the courage up and I called them and I said I'm ready to come to Taiwan and they're like you can come to the gift shop. I was like all right well I guess I'm not coming then.

14:03 >> My understanding is that our freelancer who got in there spent about a year negotiating with the PR team to get in. >> I've been working on it for three. So if she has any tips like I could I could definitely >> call up Virginia Hefernon. >> I will absolutely >> Lauren you've been inside a fab in Arizona. What's it like? >> Yeah. Well like this was the Intel Fab 52 which is their most advanced fab.

14:24 It's a 2 nanometer fab and it is on the level of what TSMC does in terms of 2nometer but of course at a fraction of the scale of what TSMC does. And to Max's point about well what is our redundancy or our resiliency plan in the event of an invasion of Taiwan? It seems like right now the US government is very interested in what Intel can do in its fabs, but it's pretty small. and so yeah, I went to Chandler, Arizona.

14:49 saw Fab 52. you know, got the bunny bunny suit on and like Max said, it's it's interesting to see everything is roboticized. It's like if you had this utopian fever dream of what the future of manufacturing looked like and everything was white and everyone was dressed in all white and everything's like being you know moving around. They call them foops that actually carry the silicon around above head and then they just sort of automatically go down to get like etched and carved and stamped and everything. it look it looks like it feels and looks like science fiction.

15:24 But from a reporting perspective, oftent times the companies will cover up the names of the vendors on the different machines and you may be able to get some insight. for example, there there were ASML UEV machines in these giant bays the size of school buses, but then you might see some spaces where they're like, well, we have two more coming and we're waiting for them. And so you're like, okay, what does that mean? you know, UEV is obviously a very you know, a a very coveted sort of technology and they're al it's also very expensive and you know, so so you get some insights from that, but primarily it's just kind of it's good for context for better understanding how this all works, >> right?

16:04 >> I was not allowed to go to this tour just for what it's worth. >> Max, I think you're doing something wrong. >> Max, you got to work on your emails. >> so >> just kidding. That means you're doing actually a great job. >> Thanks, Lauren. there was an there was an interesting moment in the Darkesh Jensen interview which I guess will come up in every session today. where where basically what Jensen is trying to get across to Duvarkesh is if you do not provide chips to China they're going to be there's going to be a constraint they will develop their own models on their own chips. They could become the global standards and then put export controls on the US. Does anybody here think that that is a legitimate concern or do you think Jensen just wants to sell chips in China? wants to sell its chips in China.

16:45 >> Okay, >> I mean it might be a legitimate concern in some sense, but Nvidia has run out of places to grow. they can't sell to any more countries. They can't sell to any more hyperscalers. They can't sell to any more even they're going after the enterprise market, which is messy and complicated and difficult to actually make a lot of money in. So, China is a big market that has basically no access to Nvidia chips right now. I mean, you know, if you're getting those kinds of gross margins, like, yeah, of course, that's like an obvious just a very obvious place to to be able to sell chips, the company's kind of obsessed with it at the moment. And it's why Jensen goes to Washington all the time.

17:20 That's why he >> insisted on going to China. I mean, it's it seems pretty clear to me. >> Anisa, any thoughts? >> Max spends a little bit more time thinking about that than I do. >> I I do understand, you know, obviously he wants to sell to China. It's a huge market and they're taking a hit because they can't sell to China. So, you can't really answer the question without acknowledging that. but I I do think that, you know, from the national security perspective, Chinese companies are developing AI chips, no matter what happens. so, you know, whether we can or can't, I do think that they'll continue to develop alternatives to Nvidia.

17:54 >> I mean, their manufacturing process is is a lot worse. like the one of the research firms just published an analysis of the sort of one of the new Chinese chips like homemade by Smick and it's like it's good but they're they run up to up to the limit of like they can't buy EUV machines the big ones that Lauren was just talking about. So if you can't buy those things it's just fundamentally difficult to make transistors below a certain size and they can't do it effectively.

18:22 >> Yeah. And to Max's point, Nvidia only has so much more room to grow. I mean they're at the point where they have become a major investor in Coreavee and then Corewave is using that money to buy Nvidia chips and so it's it's you know it's really and there was an earnings call earlier this year maybe late last year where Nvidia said that they had a domestic buyer of some of the chips that they weren't able to sell to China that they not disclose the buyer and so they're at the point where they're basically yeah trying to sell them wherever they can and so you'd have to imagine the motivation there is let they want to sell chips to China >> no those leftover H20s that they were >> the Yeah. That were blocked from sales to China that somebody else wanted them because there's people want silicon right now so badly. Any any kind will do.

19:02 >> I think they sold about was it $600 million worth of those? >> Yeah. >> Yeah. Okay. So, a minute left. Let's do this an actual lightning round. So, yes or no? do you think that are you all bit or less in pled which is basically like the reason why there's all this infrastructure is because the AI labs think the more you know GPUs and chips you string together the better the models will get. Do you think that it will continue to improve as these buildouts you know continue to to explode in size or no?

19:26 >> Wait, ask the question once more. >> Is is it is it is the investment worth it in in these data centers or are they ultimately like not going to have much better models even though they have a lot more chips? >> Are we going down the line here? You start with you. So you go >> you start with me. generally yes. >> Okay. >> Mhm. >> Max not not sure. Sorry. Okay. Okay. >> I think it depends on the amortization what ends up happening there.

19:51 >> Okay. Ana, you announced your news on Twitter, right? >> I did and LinkedIn. >> Why don't you tell everybody where where you're going next? >> Oh, July 6th, I'm starting a new job at the Wall Street Journal. >> All right. Let's hear for Ana. >> Thank you. >> Congratulations. And thank you Lauren and Max for joining us as well. Let's hear for them, the whole panel. Thank you guys. Thank you very