Inside the Trillion-Dollar AI Buildout | Dylan Patel Interview

0:00 If the models don't improve, we're absolutely screwed. And in fact, the US economy will go into a recession. >> It's about the highest stakes like capitalism game of all time. >> Godsend in terms of like how much efficiency and value can be created and it doesn't ever have to get to like digital god level. Now, I do believe >> we're going to get to digital god level >> eventually. Eventually, if I could have an intelligence as smart as a Google senior engineer, that's $2 trillion of software value.

0:22 >> Is that the main like bottleneck to be attacked? >> We're popping the bubble right now cuz the limit of AI is infinite. [Music] I was going to lay out this idea of going through the past, present, and future of compute as like the big big idea for our conversation, but since it just happened, I don't think you've heard you talk about it anywhere. I'd love to start by asking about this whole OpenAI Nvidia thing, which uh sounds exciting, seems vague, not really sure what's going on. and maybe you could explain it to us as you see it and what the strategic implications are of the big announcement.

1:01 >> All right. So, I think it's I think it's very very simple, right? You've got OpenAI paying Oracle lots of money. You've got Oracle paying Nvidia lots of money. You've got Nvidia paying open lots of money meme. >> We've got we've got the infinite money glitch here. Uh no, no, no. I that's not actually what's happening, right? What's really happening is open air has an insatiable demand for compute. um the compute precedes the buildup of business. You have to have the cluster before you can rent it out for inference, right? Or rather run models on it for inference. You have to have the cluster to train the model that's good enough that it unlocks new use cases which then can be adopted and there's an adoption curve there for any new use case. So you have to have all these things like sequenced given this is a game of the richest people in the world or rather the biggest tech giants in the world, right? It's Zuck. It's Google, you know, Larry and Sergey or Sergey is like constantly in the business now again, right? It's all the biggest people in the world. It's Elon, right? There's there's very much a risk of OpenAI being too small to matter, right? You know, which is crazy to say because they've got 800 million users, but like where's the revenue? Where's the compute? They could easily get swamped in terms of having of of how much compute they have. Yeah. if they don't move fast enough and if they don't have the most compute or like among the most compute they will get beaten. The magic of OpenAI was that they just spent way more compute on a single model run on GP3 and four and they had the foresight and the vision and the execution. Yeah. But they made that bet and they were able to secure it. And at the time it was like meh, right? It was a few hundred million, whatever, right? you know, that's a ton of money. But like now it's sort of like, well, Mark Zuckerberg sees how much compute he's going to have to get even though he has this insane cash flow >> that he's like, "Oh, wait. I need to go sign a deal with Apollo for, you know, $30 billion on this data center, right, in Louisiana, this mega data center I'm going to build." It's like, "Wait, why didn't you just fund this with cash flows?

2:54 >> You have so much cash flow." It's like, "Because my plans, that's just the physical data center. Now, what am I going to put in it?" Is like so much money. The amount of capital that people are going to have and are dumping into this is is insane, right? Google was slow to wake up and then you know they were slow to pivot their data center operations. They were slow to do everything and so there while they could have way more compute than anyone by a humongous degree they haven't been able to deploy as fast. So so open is like still been on the curve of and then they have like you know how much they allocate to search and you know generative search is not really necessarily competing with open AI right it's it's the mega models. So if you have this like tremendous vision of what's going to happen with AI, you know that it takes a ton of compute to build them, you know pretty much the amount of compute you could dedicate to these models is limitless and they will get better. Now it's a log log scale, right?

3:38 I.e. you need 10x more compute to get to the next tier of performance. You might think of it as diminishing returns, but what if the next tier of performance is like, you know, you know, a six-year-old versus a 16-year-old. Like child labor is like quite like effective versus a six-year-old you can't get to do much, right? you know, and this is not exactly the way to think of AI, but this is the conundrum that OpenAI is in, right? Um, they have to get more compute than anyone or at least among there. They have to race with the giants. These giants are trillion dollar businesses.

4:06 >> So, how does OpenAI get there? Well, it's it's partnering with Microsoft. Well, that soured some, right? Um, it's partnering with Oracle. Well, Oracle can can do a lot, but Oracle doesn't even have a balance sheet like like Google and Microsoft and and Amazon and, you know, etc., right? Micro. So, so it's it's meta Elon >> sport of kings. Yeah, >> this is like this is very much like, you know, the the Pascalian wager nature of all of this with the the tech giants.

4:31 Oracle can be part of it, but it's OpenAI needs allies, right? They need they need people to effectively spend the the capex ahead of the curve and trust that they'll be able to pay the rental income because that's what it is at the end of the day. OpenAI is committing to fiveyear deals. These five-year deals cost X amount of money. It's 10 to 15 billion dollars per gigawatt of data center capacity that you pay a year. And then that 10 to 15 billion dollars of gig for a gigawatt of data center capacity. You're paying that for five years. Okay, that's 50 to 75 billion of of cash that goes out the door to OpenAI for one gigawatt of capacity. And you talk about what Sam's saying is like, hey, I need I need 10 gigawatts. I need more than 10 gigawatts, right?

5:09 >> Then you end up with this like really challenging aspect of like how do you pay for that? And hey, that's only the rental price. If I were to actually do the capex, it's or if I were to like, you know, because it's frontloaded, right? It it becomes who is the balance sheet for this. >> That's the reason these deals are coming about. And so Oracle is making a massive bet, right, Larry? You know, hey, he's getting good margin off of it, but he's making a massive bet that this capex that he's going to pay for OpenAI will actually be paid cuz you know, he signed a $300 billion deal with OpenAI. It's like, >> where's that going to come from?

5:40 >> Yeah. is like you you your revenue is like 15 billion ARR this month maybe right on a run rate basis it'll get to 20 by the end of the year uh pretty pretty clearly maybe it's like 16 now but how do you pay $300 billion of revenue u now >> if if the bet works out they've just made 100 billion of profit right like just pure cash profit it's crazy >> but if they if it doesn't work out they've got this huge and they they're starting to raise debt right they there was a small deal they signed recently um but they're going to start raising more and more debt so this this game now now Nvidia's kind of got the same conundrum right it's like well Google and Amazon are doing these these deals whether it's uh to two other vendors for TPUs or for tranium whether it's anthropic or others they're trying to court openai uh they're trying to court other companies how do I get into this game right okay fine I can rely on Microsoft somewhat I can rely on Oracle somewhat but at the end of the day GPUs if I want GPUs to be king part of it is just like my chip is the best but part of it is also who's going to pay the capex upfront Google and Amazon will pay the capex up front if it's for TPUs or right they won't pay the capex up front necessarily for that same capacity of GPUs so you've got this like >> challenging aspect and so that's where this this Nvidia and open I deal comes from >> I want to dig into the underlying assumptions driving this on the training and inference side because obviously there's the willingness like Zuckerberg just needs to go down the hall to a CFO to get access to all this capital he doesn't even need to go down the hall he can just he can just make it so >> he's got the voting share >> Sam's got to fly to Norway way and you know Saudi and and and other places and we're we're at that tier of capital.

7:13 >> I think you're making it sound way easier than it is. >> I I don't mean to at all. I'm I'm just saying you know Zuckerberg is >> hold on. If it's this easy let's let's raise 100 bill dude. >> We should do it. We can compete. But I want to make sure I understand your thinking on the underlying two sides of this one which is like your view on the diminishing return curve on just like the the return on this.

7:33 >> I I don't think it's a diminishing return right. I think that's important to recognize. Right. >> Start there. I want to ask about inference too but and and like the growth in token you know inference token demand but >> given it's a log log chart right uh scaling laws are right given there's no model architecture improvements you just throw more compute data model size at it it gets better at this pace >> but you're confident that that will continue I >> I think everything has shown that it will continue and it's continued overd wasn't some like >> well GP5 is not not necessarily that much bigger than 4 right and 4 is smaller than four um what what what what's what's changing is sort of paradigm of how you spend the compute and also like if they made a bigger model, could they even serve it? No.

8:12 Right? They did 4.5 and it was terrible. No one could no one could serve it, right? It was actually like quite a bit smarter. Uh but they couldn't actually serve it at any reasonable cost and speed. Um this is why Anthropic has the same issue, right? Or I wouldn't even call it an issue, but like all of their revenue comes from for Sonnet doesn't come from 4.1 Opus, right? Which is the better model. It's bigger, but it's slow because the hardware is not caught up.

8:35 Yeah. In terms of inference speed for that and so no one wants to use a slow model, right? The user experience sucks. >> Yeah. >> But as far as like if the model gets better at each scale of hardware spend, I would say all the tech giants believe it. I believe it. I think a lot of people in the financial community are like this is freaking scary. Yeah. You know, because the moment it stops, you know, you wherever you were on the rung, right? If we went from $50 billion spend to $500 billion spend, well, that $500 billion spend is never going to have ROI, right? It was one thing if 50 billion didn't have ROI, but now this 500 doesn't have ROI. It's a big problem. So anyways, one could think of it as as diminishing returns because if you when you go from $50 billion of spend to $500 billion of spend, you only move up the let's call that one tier of model capabilities in absence of, you know, major algorithmic improvements, right? Um, and so I'm I'm holding those sort of off to the side for now. But that that iterative like performance improvement in the model is is like I like I mentioned earlier, right? It's like a six-year-old versus a >> 13-year-old maybe, right? The the amount of work you can get a 13-year-old to do is >> I mean, if if you do it, right, we we we we frown upon that now in this civilization. Um, but the amount of work you can get a 13-year-old to do is actually quite valuable relative to a six-year-old. And and the same applies to like a college intern versus someone who graduated and has even one year of work experience because there's a learning curve for kids coming out of college all the time. So there's that learning curve and I think you know while it may be incrementally the same you know an order magnitude more of compute the amount of value like if we just had if we just if you made a company full of high schoolers and you had to refresh them every six months so they didn't learn too much right and become really good it would be really hard to create a valuable company the most you could do is like dig trenches and and like do yard work but then all the time these kids wouldn't even show up right like as as a as as a function of like how valuable of a business could you do if you had unlimited high schoolers >> versus if you got a business that refreshed so they didn't build knowledge versus college students versus you know 25 to 30 year olds right the the the value of that business that you can build even though incrementally it's just 5 years between each of them >> yeah it's drastic >> it's it's a drastic value change >> where do you think we are today like which level are we at do you think >> depends on the domain right like for software developers like I think we're we're we're we're really pretty good right um you know and that's where where we're seeing the most value creation happen right where you see anthropic have gone from like what a billion or less of revenue to seven to eight by you know already in it's the fastest revenue ramp we've ever seen for anything of this >> and it's basically all code related >> right I mean like you know some of it's their own cloud code product some of it's you know cursor some of it's GitHub copilot which also offers anthropic models and has since the beginning of the year um it's you know it's windurf it's it's it's all these different avenues to access the same thing and these companies aren't all doing the same thing there there's tweaks and nuances to how they're doing things differently But it's all code and you know in that sense it's like if I had 30-year-old senior engineer um at Google and if that was like if I had infinite of those all it costed was capex for chips and the operational cost is actually quite low >> then you could build businesses worth insane amounts. You could have a replacement for the $2 trillion of wages that go to all the software developers in the world today. or rather you could augment them and build, you know, twice as much or five times as much or 10 times as much if you could augment them because these things don't like just run on their own, right? It's more of like a force multiplier to the existing person.

11:51 So the value creation potential is is there. It's obvious if you've coded at all in your life. I mean, it even works for VBA. It's not that great for VBA. So I know a lot of people in this audience probably know VBA, but like it's it's not even that terribly bad for making macros, but you know, anyways, like the value creation potential there is is incredibly high. So let's capture it. How do you how do you capture? And so sort of this draws back to the OpenAI Nvidia deal because I think most people in the market don't quite get it right. They're like, "Oh, this is just like round tripping." It is to some extent, right? If OpenAI builds a gigawatt of capac they they they agreed 10 gawatts of capacity, Nvidia will do hundred billion dollars of equity investment into OpenAI in the form of cash, right? And and and Nvidia gets returned capital. The first chunk of the deal in the press release is one gawatt, $10 billion, right? So pretty straight line but 10 g one a g one gigawatt to build as we established earlier is like $50 billion. So Nvidia is paying 10 billion open still has to come up with other 40 somehow. Yeah.

12:47 Right now what they can do is go go to the markets get a loan or get someone else to put a loan. Right. There's these infrastructure funds that are trying to get into this. Um there's all these different you know all these commercial real estate people are trying to get in this. there's some way where they'll be able to figure out other people to front the capital right and then and then come up with like a deal much like like it is Oracle but open has to do more of the work in terms of setting up the cluster the software the networking etc. The nice thing for Nvidia is they sell you of that 50 billion they capture maybe 35 billion of that is capex that goes directly to Nvidia. So year zero openai its partner spends $50 billion on the data center. The timing is not exactly that. They spend $50 billion on the data center. 35 goes it to Nvidia. Nvidia's gross margin is 75%. You know again I'm going to make it simple numbers. Let's say it's 10 and 40 because 10 billion COGS 40 billion revenue $30 billion of gross profit. um if we fix the numbers it's effectively like half their gross profit from that deal is going directly to uh to open in the form of an equity investment the 25% that's COGS is staying uh on in you know Nvidia is paying for that and then they keep the other half of the gross profit on their balance sheet or do buybacks whatever they want to do with it so so Nvidia is not necessarily like they are like roundtpping some of this um but open what effectively is happening is openi gets the opportunity to pay for a big chunk of it in equity y and nvidia's lowering their prices without lowering their prices effectively but and they're getting owners ersship of a company who very likely could just like and and but Nvidia comes out great because they're they're they're getting the capex dollars up front. Yeah.

14:10 >> Right. So all they're really doing is they're saying half of my money that's in this sure it does make its way to me somehow but in reality I still made half of that gross profit and the other half is is is equity in a company that may or may not be worth something. A company that may or may not be able to pay hundreds of billions of dollars of compute deals that they've signed, right? In which case they'd be bankrupt, right? So this is this is like the the mechanics of that deal. It's about the highest stakes like capitalism game of all time. Um, and it's so interesting to think about when it might run out. You mentioned like if we hit that final point and we don't see the return like we're we're kind of toast in a big hole.

14:41 But I'm also curious about the other side of ability to serve and just demand for like today's models by inference. You know, the stat I last saw is token demands doubling every two months or something crazy. Obviously, there's all these reasoning tokens that are really exciting for some of the the longer thinking models. How do you think about the growth of the pool of demand for inference tokens themselves up just even in today's models like even if we just like stop things and fix things and we'll leave that other side of the equation just for a second. What's your model for thinking about that today?

15:10 What most interests you in the in the growth of just broad? >> So so the thing I like to call it is tokconomics and I I stumbled upon the word actually it's like a crypto >> kill off crypto finally one once and for all. So I'm I'm trying to make tokconomics uh SEO direct to you know us talking about tokconomics and then hopefully you talking about tokconomics hopefully like everyone using >> say tokconomics 20 more times >> it's the economics of the tokens right >> how much compute is being spent >> how much is the gross profit what's the value being created by these tokens that's that's the end of the day what's what's relevant here right Nvidia keeps saying AI factory which produces intelligence that intelligence has value let's say you have a gigawatt of capacity what can I serve well I could serve a thousand times times of a model that's really shitty. I could serve, you know, amount, right? I could serve one one times amount that's of a model that's good. And I could serve like 0.1 times of a model that's amazing. Now, multiply that by whatever factor of like how many users, what's the number of tokens outputed, but you know, I could do X number of tokens, X time 100, X times a million tokens, right? Depending on the model quality.

16:12 And so this is sort of where, you know, the whole GPD5 thing comes around, right? is open had had a had a challenging, you know, thing, right? They're like, "Hey, we have a couple gigawatts of capacity effectively, right? By the end of this year, roughly a couple gigawatts of capacity too. Um, more or less a little bit less, but you know, right now, but you know, it's how how do they maximize their serving capacity with this?" Um one one avenue is we continue to serve big models and we make bigger models and the tokens are more expensive but this log scale is really challenging because yes the value is an order magnitude you know value is way more but the cost is way more and then the the real whammy is the user experience is way worse >> right if I serve a massive massive model it's slow >> and users are fickle and you need that you need the response to be way faster than they can be hard to >> calibrate yeah >> yeah so so there's this user experience challenge but really in the end it's like you know for a given model level I think there's a saturation point of how many how much demand of intelligence there is right you can only have such large child army right of of like people digging trenches or like con 2012 whatever it is like this is very cancelable but you know um but you could have a much larger army of you know or business of like the larger level of intelligence right and so when you think about hey what what could I have done with GPG3 GP GPD3 even if we paused there paused the model capabilities right you know obviously the cost to serve a model quality of GPD3 has tanked >> 99% or more >> yeah it's like 2,000 times cheaper now >> it's so much cheaper now >> for and then GPD4 same thing right people were freaking out about Deepseek because it's like five six00 times cheaper GPT OSS came out and that's even cheaper than that right and it's the same again like for roughly the same quality actually I would argue the GPT OS open source model is actually a little bit better than GPD4 OG um because it can do tool calling. And anyways um the cost of these things tanks rapidly with algorithmic improvement, right? Not necessarily model getting bigger. Um and as these algorithms get better, you can but but at X level of intelligence, you can only serve so much demand. And then the the the flip side is you know what that demand it takes time for people to realize how to use it. So when GPD3 launched, no one cared. When GBD3.5 launched, it was like still most people didn't care. Chat GPT launched with GBD3.5 people cared a little bit. uh GPD4 launched on chat GPD then people cared a lot but a model tier of GPT 3.5 or three still can be very useful in a lot of world a lot of the world now it's not useful for like a lot of use cases right like for coding it was terrible right for for copyrightiting it's okay right like there's there's but there's some level of use case and it happens to four but it takes time for that adoption to happen um and so you've kind of got this challenge of like if I pause on a model capability then I end up like taking way too long for adoption and also like how can Can I get people to adopt it if I don't let people use it?

19:05 And so, so open had this tremendous problem with GPD 40, right? 4 and then 4 turbo was smaller than 4 and 4 was smaller than 4 turbo. What open basically did was they made the model as much smaller as possible while keeping roughly the same quality or slightly better, right? So 4 to 4 turbo was like the model was less than half the size and four turbo to 40 like 40's cost is way lower than four. Um and they just kept shrinking the cost. Now five, what could they have done? They could have gone, "Oh, we'll go big step." They actually tried that with 4.5. They they screwed up some things cuz it was really hard to get, you know, 100,000 GPUs to work properly. There's challenges there.

19:40 Also, they hadn't figured out the whole reinforcement learning paradigm at that time. So, the C, so they ran out of, you know, it's the the scaling laws are like it's a chart of quality versus compute, but that compute breaks down into how much bigger do I make the model, how much more data do I put in the model, and if the internet only has so many tokens, >> you're kind of screwed, right? So it took you know there was there was potentially a cliff until reinforcement learning happened you know where you can generate data and train the model to be better without the internet having that data. Um but anyway so so they kind of had this problem of you have x amount of compute you can service your users but hey today um if people want to use my API I rate limit them because I can't actually serve them all.

20:20 >> Yeah. >> Oh if I want to use um you know I have to I have to rate limit the people who have chat GPT free pro and max whatever the whatever the $2 $200. There's like different rate limits. You can only do deep research so much. Um I have multiple Chad GPT accounts because I, you know, use deep research. It's like you you kick off a bunch, you read it, and you're like, "Wow, I learned a ton.

20:38 Move on." Right? So you have this challenge of like you can't actually serve your user base enough. So how are they ever going to move up this adoption curve? >> So then as OpenAI, what's your choice? Do you make go from 40 to 5? Do you make the model way bigger and not be able to serve anyone? And plus, because you can't serve anyone and it's slow to serve, the adoption curve doesn't really get going. Um, or do you make the model the same size, which is what they did for GBD5. It's basically the same size as 40 and and roughly the same cost.

21:06 That's actually a little bit cheaper potentially, and then you just serve way more users. >> Um, and get everyone up the adoption curve more. And then you can instead of putting them on a bigger model, you put them on models that do thinking that can do uh, you know, so if you've used GP5 thinking or GP5 Pro, there's more intelligence there. Um, and so this is the whole conundrum they have and this is where the whole tokconomics thing comes into play. the question you had, I wanted to level set it, right, which is how do you serve these users? The demand is growing so much.

21:33 >> I'm not doubling my hardware every two months, >> right? >> Right. Yes, this capex is crazy, but I'm not doubling my hardware every two months, >> but I'm doubling my tokens every two months. So, so there has to be enough of a cost decrease and and there is, right, with with at a given level of intelligence. >> If you could like in if you could snap your fingers and change change a dial somehow that would most unlock and unleash more development, is it just is it just inference latency? because then we could do bigger models and serve them much faster in a way that consumers would enjoy. Is that the main like bottleneck to be attacked?

22:03 >> Inference is like always it's it's it's a curve again, right? Like all of these things are curves and it's a trade-off, right? Everything in engineering is a trade-off. So So you have inference latency versus cost on any given hardware. >> Um GPUs can do lower latency to a certain extent, but then the cost is way higher or you can do really really high throughput and the cost is way uh lower, right? and and you know the company just kind of yolo they set the dial where they think it makes the most sense and there's other types of hardware which kind of aim for their curve to be at a different spot. Maybe the GPU curve is here uh but latency you know over here you know you're in very diminishing returns and so actually someone made a little curve right here. It's like okay maybe that's a useful point but actually the market cares about this point. So anyways there's there's there's a curve of like who cares about latency. I think if I could just press a magic button.

22:46 >> Yeah that's is it is it capacity? Is it latency? >> What is it? I think that I think that's a that's a tremendous like question. I'd probably still say capacity/cost is more important than latency >> really. >> I think existing levels of latency are fast enough for a lot. Um now now if the if the latency was 10x lower for GBD5 then they could have made a model that was 10x bigger and served it at this qual served it at this speed.

23:09 >> Yeah, that's what I'm wondering about. >> But but then you would have the same capacity issue, right? Um, so I guess like if I was if you could have your cake and eat it, which is all the capacity in the world and the lowest latency in the world. >> Yeah. >> Well, then you would just make the best you'd make the models way better, right? Like I think I think it's the physical realities of like if I'm at OpenAI, what do I choose to do? Um, >> do I invest more in the model that people can use or do I invest more in the fast? Do I invest a lot in the model that most people, you know, won't use because it's expensive first of all >> and even those that can afford it will often go back to the regular one, >> right? Um, I have access to Cloud 4.1 Opus. I still use Sonnet more way more >> just because it's a better experience, >> right? It's dumber. It's it's it's objectively dumber, but it's slow.

23:51 >> Yeah. >> And like I don't I don't know. Like my time's worth something, right? I think Openi wouldn't have been afraid to like make a model way way way bigger in a terrible user experience. >> Yeah. And as a result, we're just going to probably have to wait a little bit longer to see what the bigger models are in practice in a way that to to see what consumers actually do with them because it's just going to be too hard.

24:09 >> It's not necessarily even bigger, right? Like there's this whole concept of um overparameterization i.e. if you just throw more parameters in a neural network and even when humans I'll equate it to humans right when you had a vocab test or you had some test you memorized before you understood and it wasn't until you did multiple repetitions and in different forms that you actually understood the content rather than just memorized. Um it takes it takes cycles. Um and when you when you do an LLM it's the same thing right?

24:38 If you throw some data at it, it will memorize it before it generalizes. It's this concept called groing, right? You grocked a subject, i.e. it's like the aha moment, >> trick of understanding. Yeah. >> Yeah. And the models do the same thing. They memorize it up until then they understand it at some point. And if you make the model bigger and bigger and bigger without the data changing, you just memorize everything. And actually, it starts to get worse again because it never had the opportunity to generalize because the model was so big and there's so many weights and there's so much capacity for information. You know the challenge today is not necessarily make the model bigger. The challenge is how do I generate and create data that is in useful domains so that the model gets better at them. Nowhere on the internet to show you how to fly through a spreadsheet you know using only your uh mouse or not using your mouse using only your keyboard and all these like you know functions and all these things right like that's that's a repetition that's that's bars but there's no data on the internet about this. So, how do you teach a model that? It's not going to learn it from reading the internet over and over and over again, which you and I could never do. And so, it hasn't a level of intelligence that we can't do. We can't read the whole internet, but it can't do basic stuff, which is like play with a spreadsheet.

25:45 >> Um, so, so, so how do you get it to learn these things? And so, that's that's where this whole reinforcement learning paradigm kind of happened, >> which is >> giving it environments, specific environments to learn it and then fold back in. >> Right. Exactly. And that's that's where there's sort of a a challenge in terms of building those environments in terms and so there's like 40 startups now in the bay doing these environments and you know questionable whether or not they'll any of them will make it or what will happen but like there's 40 and then these companies are also making their own environments but these environments can be anything and everything. Give me an example just like of one of the startups or something just to get >> these startups are like like they're they're just making environments for open anthropic and others right so it's like as simple as like here is a fake Amazon >> right because Amazon terms of service ban chat models and all these things but here's a fake Amazon full of items um figure out how to click around and purchase items >> right uh figure out how to compare the two items and pick you know I I've generated a list of deodorants three of them are fake one of them's real one of them is not the one I want here's the prompt figure how to buy it and if and and you know it tries many things and you know vary the prompt and all these things but eventually you know it's bought the right deodorant and you've succeeded and you fold it back in.

26:51 That's a simple thing. Or it could be, hey, clean this data, right? Here's this table. Ton ton of dirty data in there. Oh, there's like colons and stuff. The there's an address in one column. You know, I'm going to, you know, how do how do I separate out the columns? So, the address is like, you know, it's it's street address, city, zip code, and it'll try a bunch of stuff, but like, hey, maybe it can't do that yet. So, really, you just drop it like you teach you give it, you know, iterative like here's here's addresses, here's different formats, and you slowly iteratively teach it. So, there's all this like challenge. So that's one that's another example. Another example is like you're in a game and like >> whether it's a tic-tac-toe or Call of Duty or you know a math puzzle, whatever the game is. And that's what a lot of these environments initially have been is like math puzzles. It's like do this math puzzle. Oh well I can't do this one because it's too hard. Here's an easier one. Oh, okay. I can I can spin on this one. Okay, I'm better enough. Okay, now I can learn this one. Right? and and and it has iteratively stepped through those to where you know basically from this you know Q4 of last year to Q2 of this year these things hill climbed up math puzzles like crazy.

27:53 >> Yeah. >> Um and a lot of that was not hey I just know the math. A lot of that was here's how I use Python to uh write something that does the math for me. Um and now these things are actually quite good at math. But you know so so it's like these the environments can be super varied. Um and it doesn't need to be something that's like clear-cut and dry. It can be here's a medical case, what's wrong with it? And then you have another model say, well, here's here's your instructions on how you would grade the result of a case. What looks like they didn't even try this or didn't even look up this.

28:21 Okay, you did that wrong. And you know, you can you can feed these models into so these environments can be very very complicated. So building those out is is a challenge, right? It was one thing to say, I'm taking all the internet data. I'm going to filter it some. I'm going to throw it to the model, right? There's tons of engineering challenges there for sure. There's a different set of engineering challenges that take time to build out >> in those two like in pure raw internet pre-training world and in this new like environments world like what inning are we in in each of those would you say like how far into the potential benefits have we have we eaten >> this is where like the whole like oh well then you know Dylan what you're saying is you never need to make models bigger again right because you've already run out of data and until you figure out how to generate tons and tons of data that's great but actually we haven't right like you know we've seen another angle where it's mostly just been pre-training scaling, right, is is V3 and Banana Nano, right? These Google image and video models um and Genie and like all these Google uh image and video models and that's that's purely like scaling on on multimodality, right? The models still aren't that great at video and audio and images. They're fine, but they could be a lot better. Um so there's like angles of scaling there, right? Cuz when I said we've run out of internet, we've run out of the text, >> tons of video and image and audio, right? We just it's just so expensive.

29:35 So, you know, like we we didn't get to that. >> So, like maybe late innings on text, mid innings on pre-training. >> I think we're early on text. Yeah, we're quite early. And then the other angle is just because you've used the text doesn't mean you can't learn faster, right? You take a class, you give them all a book, you tell them to read it once, and you test them all. It's like, well, one kid's going to get 100 and one kid's going to get a 40, right? It's just the reality of life. And maybe maybe if you if you said if you read the book out loud to them, the kid who got a 100 might get a 30 and the kid who got a 40 might have got a 60, right? So there's like these different parameters and and when we talk about model architecture, the same thing happens there. So it's not like you stop training new models. It's not like you don't have algorithmic improvements or smarter kids, right? You know, it's not like pre-training is done. Yeah.

30:19 >> In fact, it's it's the base of everything. So you want to keep having gains because any gains on pre-training, right? I.e. the model learns a little faster or the model's a little bit smaller for the same quality >> Yeah. feeds into the next stage which is this whole post-training side um which will subsume the majority of the compute at some point >> and inning wise is are we in the second inning of that like how is >> I think we've like thrown the first ball >> wow >> cuz like you know like think about how we >> so many environments >> I think I think my favorite thing my brother just had a baby this baby will literally stick his hand in his mouth and I'm like you I thought about it and then and then and then it's like wait he's like he's like calibrating the senses on his fingers by sticking his hand in his mouth cuz his tongue is the most sensitive thing. He doesn't know he's doing it, but like that's how he's calibrating. He's like, "Oh, that's me.

31:02 Oh, I can touch and feel, right?" It's like, how does the model learn these sorts of things, right? It's like you just have to try stuff and fail. And we're so so early in like, >> you know, think about how much we see throughout our life and how much of that information we throw away, right? We throw all of this information away. I don't remember anything about like, you know, like, do I remember what I had for lunch yesterday? No. But if it was amazing or bad, I would have remembered that. Oh, I don't like this or I like this. Right? It's sort of like, you know, there's all this information we throw away and these models, these environments. Yeah, we're generating tons of data and throwing most of away and training the model, but it's like infantessimal compared to what humans have done. And so, I think there's so many environments you can put the model in. There's people who even think you don't get to the magical AGI until you embody it, i.e. you put the model in something that can interact in the real world, right, as a in a robot. I think like Elon and XAI like they're they're a bit more along that angle of like they think embodiment is required to get to artificial general intelligence because you need the model to be able to say like pick this up or like oh wow this is like a rotating thingy which you could never get from like just watching a video about it. You you wouldn't get the concepts of it even. Yeah.

32:10 >> Um and so I think we're so early in the reinforcement learning because that's what humans are. We're reinforcement learners >> and and the so what of let's say we fast forwarded we're in the seventh inning of that or something like this. What do you think the way that the average person will most feel that difference in terms of the utility of the model? >> It'll be very different like motus of using it right. It's one thing to like ask for information or ask it to organize information versus it just doing things. Those those 12-year-olds, you need to really direct them how to dig a hole >> cuz a lot of them haven't dug a hole.

32:41 >> But you're talking about order me this vitamin and just like it's just done, >> right? And and and we're actually like not too far away from that. I think if you try and research electric toothbrushes, like this is something cuz you know your electric toothbrush, I lose it. I leave it at a hotel all the time. And I've been obsessive about this. Like in 2021, I I like made a spreadsheet of all the electric toothbrushes cuz based on how many IC's were in each one of them, right? Like this one has a Bluetooth IC. Why? I don't know. This one has a display IC.

33:05 Like it has a color display IC. Like what's going on, right? Like so I made a spreadsheet of all this. And so like I don't know. It's like this weird like little thing that I do. I've been finding like every every you know how I research which toothbrush I want to buy now I bought a oral B IO like series 9 or whatever right like whatever it's like but it's like comparing them like these models now can like actually like >> figure out exactly what you want and more than 10% of Etsy's traffic is straight from GPT >> wow >> Amazon blocks GPT but like otherwise it would be really high people make purchasing decisions through GPTs they just don't make the purchase >> open's head of applications or co of applications was at Shopify and created the shopping agent, right? This is is very clear. This is how they monetize.

33:47 The models are going to purchase for you, right? They're going to do actions for you and the model and then the company that does those actions for you, the model that will be able to take some sort of take rate, >> right? Even if it's like 0.1%, even if it's 1%, it's 2%. It'll be like a credit card transaction. Visa is the most amazing business in the world because of this, right? And and chat could be that, too. If I'm making my decisions on purchasing all sorts of things, I mean, I already almost outsource like what am I going to eat to like the front page recommendation of like Uber Eats sometimes or I already outsource a lot of decisions. It's not too much further till I've like completely outsourced a decision and a purchasing intent. That's what's made and Google such amazing companies is they figured out how to get the thing you want to purchase in front of you as best as possible, right? and and all their work on recommendation systems is figuring out what you like how to keep you on the platform longer whether it's YouTube or Instagram or you know or bite dance right with Tik Tok or it's hey here's the ad of the thing you'll probably click on and buy because that's how I get paid and and everyone likes to claim they don't like pay attention to ads but you do right >> before asking even more holistically kind of your view on where we're going there there's a third category which is the reasoning part of the equation so we've got pre-training we've got RL and environments post- training what about just like raw time spent reasoning thing and where that going as its own independent part of the overall scaling law. The scaling laws again like if you zoom out that's not actually what the like original paper is but in spirit sure scaling laws are more compute better intelligence >> and that could be bigger and bigger model each iterative token is better whatever like word garbage I spew out if I went back and I wrote about everything I talked about in this I could make it way more condensed it could be way more clear um potentially right now the benefit of podcast is a lot of times people more fun this way >> driving it's fun yeah exactly right uh they're walking their dog and they're listening whatever it is but like the interesting like an important like thing here is that by putting in these environments, you're teaching it like humans, right? If I asked you, you know, to go figure something out, right? You might not necessarily know the answer right away, but I know you could probably figure it out in a given amount of time. That's reasoning. You're spending more brain cycles. The magic again of like intelligence of humans of of people is not that they are information retrieval like the best at information retrieval, right? Like like GPTs are amazing at information retrieval. We're really good at because we've been trained in these environments which is our world at figuring out how to do things iteratively. And so reasoning and these oral environments are linked together, right? If if I'm telling a model, hey, do this math puzzle, it's not it's not just spewing out like, oh, the answer's one, oh, the answer's two. Oh, the answer's three.

36:22 Okay, the answer was actually seven. And when it got there, I trained it again. It's like, okay, now it knows next time. Oh, the answer is six, seven, or eight. Now it's like seven. Okay, great. It's not like now it instantly knows the answer. It's actually like, oh, here's this puzzle. Oh, like these numbers. Oh, this line, it's sudoku. These numbers add up to this. >> Um, oh, it has one through nine, but it's missing eight. Okay, it's eight.

36:41 Right? Like it's thinking through it, right? Like you and I would solve a sodoku. Now, eventually when you get good enough at sodoku, you could probably just like spit out an answer. Um you could do it in your sleep but for a long time you can do it without like you know and so sort of like this reasoning time is a way of spending more compute more brain cycles on the task without actually you know scaling the model um and then the model becomes more versatile right because humans have a rate if I just held a match against you and you didn't notice it you'd immediately jerk right because you're the the the rate at which you operate is hundreds of hundreds of hertz right you your body can actually take actions at like hundreds of actions per second if you look at like a fighter pilot's reaction time, right? The the peak of human like reaction time. Now, what reaction can they do is like completely like primal instinctual, right? Very little thought is put into it. If you think about like this alien intelligence that we're trying to make, is it is it immediately going to oneshot the answer always? No. But like, you know, at times it needs to. Yeah. At times it needs to be able to, you know, tell me exactly the answer in like two seconds or half a second or um, you know, whatever action it needs to take immediately. But a lot of times also needs to think through the problem, go and do stuff. That's why you hire students. That's why you hire interns cuz you're like, >> "Yeah, I know this data exists. Here's the format I kind of want it on and go figure it out." And then they they spend a whole summer doing something you could have done in like 3 days, but like great, they learned a [ __ ] ton, right?

38:10 And it's like >> these models need to go through that progression. Um, and so when I think about, you know, reasoning RL, it's it's a lot about how the human psyche and and intelligence works. Uh, and and sort of like I wouldn't say, you know, there there's there's a caution of like trying to make it too much like humans cuz it's not the fundamental substrate is not like humans. The processing is not like humans. Our brain is very different from a, you know, how how these ALUs on a chip works. Like the scaling of these things is very different. The raw speed, the amount of words they can, everything is so different. But at the same time, it's important to like reckon back to what actually makes people, you know, smart.

38:46 >> On the topic of like embodiment, uh, and continuing with the human analogy, how do you think about things like short and long-term memory in a human versus just like raw model capacity or something? Like what role does that analogy of memory, I don't mean literally like like semiconductor memory, but like memory in a model, how do you think about the importance that that will play and where are we in that? The magic of transformers was uh attention right i.e.

39:12 I calculate everything in my context length. I cont I I I calculate the attention to each other, right? Basically, in a vector space like king, queen, there's these vectors. There's like dozens of vectors for each number. And king and queen are actually exactly the same on a ton of stuff, but then it's the opposite on one number because one's a male, one's a female. And then that that will have a lot of other like, you know, ramifications throughout other literary stuff like, you know, what adjectives do you put with a male of, you know, this vectors? it's like regal and you know like powerful and could be ruthless whereas a queen could be like dignitary or whatever like I don't I don't know like whatever stupid stupid analogy but when you think about how that applies to you know humans what what we're terrible at like exact recall I could tell you a sentence and tell you to repeat it >> yeah it's like six numbers the average person can remember or something like that >> right but like you get the gist of the sent if I told you like if I told you a whole paragraph you'd get the gist of it and you could you could repeat the meaning of it to someone, you could translate that meaning. So models very different, right? Fundamentally transformer attention has been, you know, calculating the attention to everything to each other and getting the models to actually be able to recall.

40:18 That's been a training data problem. But like you can get the model to repeat exactly what you want, anything in its context length. It's like a needle in the haststack is the like problem that like you know it's a benchmark that people did for a while because models had to get good at that. But now models are just like amazing, right? Like tell me tell me like blah blah blah and random part of your context. But what they really suck at is having infinite context because you have infinite and it's sp what the real word is sparse right you have sparse you you've taken this entire world and you've encoded it in such a small amount of data that lives in your brain and it's so sparse but you understood how to like grab the fundamental reason and put it down there whereas models they haven't been able to create something sparse yet right what is the long how do you how do you how do you reason over the context of infinity.

41:05 And you know, humans maybe we have like a short-term memory and a long-term memory. I think it's a lot more blurry than that. There's no like clear line. Oh, this was in my short-term memory. Oh, this is in my long-term memory. It's like it is much more blurry, but as we as we go back and back and back, it's more and more sparse, right? If we think about, hey, what do you remember as a kid? The most crazy thing in psychology, I remember when I learned it, I was like, wait, my memory of what I did as a kid with my, you know, dad at this like thing, right? Is fake. It's me remembering it and inventing the picture and me remembering that picture like successively but like the actual memory of what happened is like morphed a little bit >> um over time um because it's a spark like we the way humans collapse information um is super super dense and and but we are able to extract all the relevant information out now models they have there's a there's a ton of research going on in this domain of long context right how do I get longer and longer context without blowing up my model cost. This is a big challenge with reasoning. This is why, you know, we had this HBM bullish uh pitch for a while, right? Is like, you know, you need a lot of memory when you extend the context, right? Simple thesis, right? But the fundamental algorithm needs to change and improve over time iteratively to get to something like this short and long context of memory. That doesn't necessarily mean the model has to work like we do, right? Why can't the model just reason and have a database that it writes stuff in or like a word document that it writes stuff in and then it like takes it out of its context, works somewhere and like Roful calls back.

42:35 It's like, oh yeah, right? Like we don't do that, right? Like you and I refer to our notes, we refer to our calendar, we refer to our text, we refer to anything all the shopping list, right? Like great, I know I need food for dinner, >> I go to the store, I'm like I need a shopping list, right? Like cuz otherwise I'm going to buy like stupid [ __ ] right? is like it's like so so the model doesn't necessarily have to fundamentally work the same way as humans. But there is that challenge of like how do I how do I train the model to operate over the context length of a human? How do I train it to interact with these databases and these word documents that it writes to? Because it's never going to learn that from pre-training has to learn that from an environment. But these environments have to be like architected in a way where the model knows it can write stuff down and refer back. And so one of the first things Openi did was deep research, right? deep research is everything is not in deep research's context, right?

43:23 Deep research is working for like 45 minutes. It's outputting millions and millions of tokens and it's creating this amazing like you know thing that it wrote, right? And it's like pretty good research. Um I would say a lot of like memos that you read from people are like on the on par with like deep research at least like a junior. How did they do that? was they they they enabled it to be able to write something down elsewhere and have this recall and and you know and and and effectively use language to compress information that it looked at, put that off to the side, use language to compress other information off to the side, use language to compress other information off to the side and then looking at all this compressed information and writing something, right? And that's that's sort of what deep research is. So how do models get there? I'm not sure, right?

44:02 Like I think it's a fundamental research challenge. It's why these companies need, you know, millions of GPUs to train on. Yeah. Not for, oh, I'm gonna make a million GPU model, but because I need to try a bajillion different things because I don't know what will work and what's going to be what's going to work for humans is so different from what works with models. There's like any number of parameters or things you could tweak that could end up like changing how it develops, right? Um, and how good is it at, you know, if I do it this way versus that way, right? That's that's the whole point of ML research is is you're constantly trying stuff out and and trying to get better and better. If I add all of this up and you know hold the mirror up, it's it seems like I would put you in the category of like unbelievably bullish on what these things are going to be able to do in 10 years time or something like pick your time frame.

44:46 >> Yeah. >> Am I calibrated the right way? Like amongst everyone you talk to who you respect and think is >> I'm much more emarrassed than a lot of people actually which is the crazy thing. help help me understand that distinction. Like if you're where are you 1 through 10 amongst the people that you respect 10 being the most bullish and then like what is the difference between if you're not a 10. What's the difference between you and the person who's a 10?

45:07 >> I respect you, but I know I'm like way more bullish than you and I respect like Mark Zuckerberg, but I know he might be he's probably maybe I don't know if he's more bullish than me, but I know Sam Alman's definitely way more bullish than me, right? He says he says we have artificial general intelligence in in less than a thousand days, right? say, you know, like or Dario, like I respect him immensely, but he's way more bullish than me. Right. My roommates are like one of them is an anthropic ML researcher and one of them is is another podcaster, Dwarvesh, like they're they're both like way more bullish than I am.

45:34 >> Really? >> Yeah. Yeah. Yeah. But like even they are not as bullish as like some researchers in this field. So it's like but then if I go talk to like uh you know someone someone I respect like I don't know like a famous investor right like uh you know any of these famous investors I don't want to name one because I'm scared you know like but there's all these famous investors right it's like well no they're not they're not more bullish than me and the stuff I'm saying sounds like crazy [ __ ] >> some of it though is timeline um I'm I'm actually even more curious about like the upper limit the extent to which there is >> the upper limit I think I'm among the most bullish you can get because >> that's what I mean >> the upper limit of this is that this will just be smarter than humans. I don't think that will happen like anytime soon. Even if that doesn't happen anytime soon, there's so much valuable stuff that can be done with these models that economically we will skyrocket. There's so much value that can be created in the world just by hey, if the models know how to do uh cobalt to like C and Python migration of like main frames, >> just migrate everything >> migrate everything from mainframes to cloud the world is how much more efficient? Hey, making making all these random applications and like automated reports and like stop using Excel as a database, but instead like you can make a real database and and manipulate stuff in Excel, but like you know there's all sorts of like humongous business efficiencies that could happen or automation that could happen without the model ever being, you know, we could literally just pause it at like a 6 months from now time frame of like how good it is at software development and it would be like godsend in terms of like how much efficiency and value can be created >> for the economy and it doesn't ever have to get to like digital god Now, now I do believe >> we're going to get the digital god >> eventually. Eventually, is that 10 years? Is that 5 years? Is that 100 years? Is that a thousand years? I don't know. Cuz there's there's so many unknown unknowns. Like I mentioned, right? Like these these babies are putting their freaking hand in their mouth to calibrate. And then later they put their foot in their mouth >> and they're like, "Oh, that's my foot.

47:22 Oh, here's the senses on it." And then they can pick up stuff in their hand and they no longer have to put it on their most sensitive part of their body because they know what it is. Or they're like, "Oh, this is a speck on the ground. What is it? It's not food. But now I know what it feels like inside my hands and I've calibrated, right? It's like the models have not gotten there yet, right? Like it has no idea how to do this. Digital god is like well one I like kind of believe in embodiment and like you need non-digital god. You need a physical and you need the capability of like having touch and feel and all that to truly be uh have have an experience like humans and be smarter than us in every way.

47:54 >> But you know that's so far away. >> What do you think about what physical intelligence is doing? attacking the whatever you want to call it large movement model or large robot model or something. >> What are they actually doing today is like holy [ __ ] it's so simple in terms of like to a human. >> Yeah. >> It's like to to models it's like picking this up is freaking hard. Like how much do I squeeze my pinky versus this finger versus this finger versus finger? I don't know. Like but you pick up a glass of water and you tilt it and it's like this is impossible for a model today and it's likely like at the level of dexterity like you know if I if I if it was a wine glass and I was swishing it.

48:25 Think about how simple that is. You don't even think about it, but like you instinctually pick up a wine glass and you swish it and it lets the aroma out and you smell it, but it's like, oh, that little swish is so much tactile feedback and movement and it's like these models can't do that [ __ ] yet. Like nowhere close. So, I mean, I think yes, we have, you know, but it doesn't need to be that good. It doesn't need to be able to swish a wine glass and not break the wine glass and put it back down and tilt it perfectly and not spill it. Doesn't need to be able to do any of that to be tremendously valuable. What it needs to be tremendously valuable is pick this up and put it down here after knowing what it is. Um, so there's so much value that could be created just by being, you know, really good at like >> getting data. Yeah.

48:58 >> Yes. I like, you know, I think the robotics world is huge. I think we're, you know, we're we're like we literally warming up. >> We haven't even left the dug dugout, right? Like we're we're like nowhere close to, you know, the scaling on on robotics. There's a ton of like the data flywheel needs to get going there. >> One of the most interesting things, the subplots of this whole world is the talent wars. And a cool idea is that as these things get better, maybe we begin to automate some of the research function that people formally would have played. Do you see a world where like we're squeezing down the fewer and fewer number of people that really matter that will have all the impact on where we go uh in terms of like net new research and that means that all this crazy spending that's happening at Meta or elsewhere makes a lot of sense that like maybe even those numbers should be higher or something like this.

49:41 >> I think it's like tremendously hilarious that people are like, "Oh my god, this person's getting paid a billion dollars." It is infeasible. It's like how could this person possibly be worth that much? Well, they're running the experiments on chips that cost, you know, hundred billion if every wasted experiment they do, if they just used like a third of the compute and and their ideas and their impact on it wasted the compute. It was an idea that was already done or like you know like there's so much wasted compute. I'll say I call it wasted. It's trying stuff and failing. But like none of us know what to try and what not to try and and these things are so complicated. There's like a group of people just trying different stuff on the existing data. How do you mix it?

50:20 What order do you feed it into the model? Um how do you filter it? Like what's the architecture? There's different people working on long context. There's different people working on every single aspect of the model that like if you just make them a little bit more efficient that they come up with the idea that's 5% more efficient. Well, fantastic. I just saved not only 5% of my compute time, training time, I also save 5% across my entire inference fleet. And then I do it again and again and again and again because we're so far away from like these models being anywhere near as efficient as a human brain and we know it can at least get as efficient as us. Maybe the compute substrate isn't the same, but like whatever, right? Adding more people to the problem doesn't make it faster, right? Because there's so many things you're trying. You run these experiments, you learn something, and then you implement it. You tweak the knobs in these ways in 100 different ways and then you see the trend line and you're like, "Oh, so actually I should tweak it this way. Let's implement that." Right? There's so much like gut feel. there's so much like reading data, understanding it, imple reimplementing it into these things that if you add people, you're going to slow it down. In a sense, a lot of like Meta's problems before they did the super intelligence thing was that they just had too many people that weren't led by leadership that was amazing. Um, and they had like a lot of failed experiments and wasted time doing things that didn't matter.

51:28 Um, there's a there's a tweet from one of my friends uh at OpenAI. Um, he's pretty famous on Twitter. His name is Run. He's like, I get visibly viscerally angry every time I think about how many H100s Meta is wasting. >> It's such a funny tweet because it's like, well, yeah, they're wasting a ton of compute. They were, you know, maybe they still are, but you know, like, and everyone's wasting compute, right? Opening eyes wasting tons of compute cuz, you know, what's the paralo optimal model architecture? Who knows? Another thing I saw Run say recently which was so interesting was uh why don't we just go make even more ridiculous offers around the people that have process knowledge for things that we want here in the US in other countries like why don't we if if we're getting pretty good at the Arizona you know fab that we've built and we we think that we can sort of extract the process knowledge from the people why don't we like go aqua hire like all the best people in Shenzen or all the best people in like other places in the world do you think it starts to escalate to that level like so much is dependent on the process knowledge of a relatively small group of people and the and the war the talent war should actually be it shouldn't be meta and open AI it should be like the US maybe through meta and open AI and like people from all over the world like do you think it starts to get that extreme and should it >> that's almost a function of why like Intel is has fallen off a lot right is like um you have all these geniuses in you know you know nanochemistry and PhDs and all these like random like you know like things whether it be chemistry physics all these incredibly smart people, but there's a whole class of incredibly smart people that never went that way because they're like, "Oh, those guys are making like 200k." Like, why would I do that? I'm gonna go I'm gonna go to Google and make 800k and now I'm going to go to OpenAI and make 10 mill or no I'm going to go to Meta and go make 100 million, right? Like any like smart 18-year-old is going to be like, "Fuck that. I'm doing this, right?" Why do the smartest doctors, and I don't mean to say the smartest doctors in a general sense, but their skews really smart population of doctors that want to be dermatologists and anesthesiologists. It's like, is that the most valuable thing for them to do?

53:22 >> No. But those are the two professions that give you like good working hours and great pay. >> Yeah. >> Not to say that, you know, the general doctor is not smart as them. But if you took the population of general, like family doctors, like just the random doctor, and you took the population of dermatologists, the newest coming out of school, the ones that are being dermatologist and anesthesiologist are way smarter or at least were scored better, were able to get into the field that was coveted. And so, yeah, talent wars like it it is truly like, you know, we've sort of been through this process of like capital has, you know, it's it's always been human human capital and and capital goods sort of those two vying off of each other. And for a long time with mechanization, industrialization, we had the human capital decreasing as the industrial capital increased. Um and sort of that got to a point where especially in the 70s it really started to tank as the ability to globalize and and all these things started to really hit the US and and that's why we have all a lot of the like population level dynamics and income inequality that we have today that like is very bad for the psyche of the US and and the stab stability of it. But then you have now you now have like we're in such a age of like well actually like manufacturing things is pretty commodity like most of the value doesn't come from the manufacturing of it. comes from the creation of the idea. One thing Jensen told me which I thought was like amazing, right? He's like, you know, Dylan, he's like, the reason America is rich, like people have it all wrong. The reason we're rich is because we've exported all the labor, but we've kept all the value. And that's what Nvidia does, right? They've exported the labor of making their chips and Apple, right, everyone. It's it's done in Asia.

54:52 >> Um, and those those companies make money, >> not as much money as Nvidia and Apple, >> right? All the gross profits are going to them. Um, and then they're either reinvesting it or buying back stock or whatever. However they allocate the capital is like you know different concern. If like as you said the process knowledge is so valuable why aren't we why aren't we doing this? That's that's a that's a that's a great idea. >> Run's idea not mine.

55:10 >> Yeah. No I mean I think I think the challenge is how to choose people is really difficult. So for some roles someone who can talk the talk they're great right? Like people just automatically assume they're great because they can talk the talk. But you know how many people suck at talking and are really freaking good >> at doing. Yeah. >> Yeah. But then you don't know. You don't know, right? Because it's like, well, but then there's people who talk about being able to do better than the person who's doing and like and then like you know these tests are never as good, right? Like so work trials. It's like how do you select?

55:44 >> Um and this was a big challenge for Meta. Um so some of the criticisms are like they didn't get all of the best people. They actually got a lot of like bad people. It's like the cope from like OpenAI and Enthropic and you know these kinds of companies are like no no no they didn't get our best people. That's what Sam said, right? He's like they didn't get our best people. It's okay. Meanwhile, he did have to do counter offers internally, right? So it's like you know um so so as far as the process knowledge I think I think the ML researchers are an extreme of how much value one can do. But my favorite analogy that I came up with recently is that ML research is the exact same as semiconductor manufacturing. You know, there's a ton of jobs in in in semiconductor manufacturing that don't exist in ML research, but it is a ton of tune a thousand different knobs, right?

56:28 >> Oh, you put the wafer in this tool. You're going to change the pressure of the chamber when you're doing the deposition. Oh, you're going to change the mix of the chemicals flowing in. Which chemicals you're putting in, right? Like what what speed you do it at. Do you do it for 30 minutes? Do you do it for 31 minutes? Do you do it for you know, obviously it splices way down. There's so many knobs on every single tool >> and you have a thousand >> input and process knobs, >> right? Process knobs on each tool plus it's like the sequence of them all and and so like you frankly cannot test everything, right? It's impossible. It's it's too large of a search space just like just like designing a chip is too large of a search space. You have 100 trillion transistors. How you going to possibly try every single thing?

57:03 Impossible, right? You just have to have enough intuition like pick that point, pick that point, pick that point, see the data. Oh, okay. I think the answer is here, right? And then just yolo, right? Um and and you know obviously once you you think the answer is here you test here and you're like okay here but like a different person might have seen these three and then said okay the answer is actually here not here and like the data is like fuzzy it's like somewhere in the center but like you know it's like this this whole like idea of like ML research is you spend a lot of time on compute training doing what effectively were useless things besides teaching yourself what's the right thing to do and what's the wrong thing to do and and semiconductor manufacturing is the same way and actually all process Process manufacturing is the same way.

57:43 If you're iterating super fast and you're trying to get better and better and better or you're optimizing a process on a chemistry or whatever it is, you you try, you fail, you learn, you do. In semiconductor manufacturing, maybe it's just running tens of thousands of wafers. And so your R&D cost of you know an of an Intel and and or or your your cost of your you know main fab that is running the R&D is very very high and it's producing zero economic value besides that it's teaching you how to do the next node which then you can deploy at volume and that is like like what actually makes the money.

58:13 >> I want to go back to where all the way to where we started and and ask about what I'll call like the like the wellspring or the fountain of power in this whole ecosystem. So I want to understand how you think about who has the power and how to keep or generate power as a business. I mean um because it seemed like talent maybe talent is like the very beginning of the chain and he who has the talent like on a long enough timeline has the power or something like that but also there's structural stuff like just the scale the industrial scale of some of these things which just takes forever to build or whatever. How do you think about um even smaller zoomed in examples like okay cursor is unbelievably popular the revenue is insane um so much of it goes back to anthropic like who is the power in that relationships how does that dynamic change over time it just seems like the power dynamics are so um so fascinating in this world and I'm curious where you think it like kind of comes from in the first place like where where it exists today and where it will go in the future >> when we think about like the power structures right like you mentioned a really interesting one does anthropic hold all the cards in this cursor relationship. Cursor has like I don't know like nearly a billion dollars of revenue now on a on a on if you do current month times 12. That's a ton.

59:20 But like again like their margins are what they are and they're sending most of it back to Anthropic. You know some people say their margins may be negative. I think they're slightly positive but regardless they're sending most of it back to anthropic. >> The gross profit dollars are at Enthropic right now. >> And but then Enthropic is taking anthropic is taking all the gross profit dollars and putting them into compute y >> for training. Yep. >> So then all those gross profit dollars are going to like >> Jensen laughing hilariously.

59:42 >> Well, maybe Jensen or maybe like Amazon who's then sending it to like uh or or or or Google who's sending it to Broadcom, right? Like the gross profit dollars are going to the hardware layer um from all of this for sure. But like does anthropic have all the power? Like you know like the common view is yes from a lot of people but then it's like well >> but anthropic only makes the model that's generated the code. There's a lot more in this system, right? cursor gets all of the data. Um they get all of the users. They get how do they interact with this. Enthropic doesn't get that.

60:13 They get a prompt. They send a response. Prompt response. Now, now they have cloud code which is like taking share. Um and it's very different than cursor. But like you know anyways like they get prompt response and then like cursor is like oh well I'm training embedding models on your code database and I have there's actually multiple models that I've made. I've made the embedding model. I've made the autocomplete model. I've made, you know, oh, I can switch the enthropic model to open the eye model whenever I want to. I'm only using the entropic model because it's the best one. Oh, and because I have all this data, maybe I can train a model not for everything better than you, but for the segment better than you. And so it's like the power dynamics are, you know, >> it's weird. It's it's it's their frenemies, right? Everyone's a friend, right? Same as with Open Eye and Microsoft, the most crazy power dynamic that's going on in the world. um where they signed aou that said they had an understanding of like what the deal would actually be for them converting to for-profit. Like what is going on here?

61:04 Like this this sounds like the most non-announcement announcement ever. The power dynamics of this all it's it's the most fascinating soap opera ever, right? Like uh there were one of my friends was telling me about like K-pop Demon Hunters. I don't know if you've heard of this. >> I haven't done a nine-year-old daughter, so it's all it's all I hear about. >> You've seen it a lot. I I had just heard about it and they're like, "Oh, let's watch it." I'm like, "What? Whatever."

61:26 And but like there's drama. There's like But like this this real world power drama is way cooler than this. Like >> at least for you and I. >> Which parts of the drama interest you personally the most? Like what where do you think the stakes are the highest in in the various like subplots? >> The Microsoft opening I1 is absurdly interesting because at one point right like 2023 it was like Microsoft's going to own the world. Yeah.

61:47 >> Right. 2024 a lot of it too. And then like H2 2024 Microsoft backed down a lot. Right. They pulled back because because uh Amy Hood and and whoever else at Microsoft, Mipundar, whoever were like, "Maybe we don't need to be on the hook for a $300 billion. We're not going to build out $300 billion worth of compute for Open AI." Like that's they can't pay for it. Yeah. Right. It was like like was at least had to go through their head when they cut back. And so they paused a bunch of data centers, right? And they said, "Oh, you know, we don't need to be the exclusive comput provider. You can go to Oracle. It's fine." Right? Like and they like relinquish this power, right? Now Oracle has that deal. But then like OpenAI sends like 20% of their revenue to Microsoft or API revenue or something like this. And then you know they have Microsoft has this like 49% capped profit structure on OpenAI and then there's like this whole like IP sharing like this deal like you you it's like really hard to understand the mechanics of the OpenAI Microsoft deal even. Um so you have this like whole power dynamic and they're trying to renegotiate this like OpenAI doesn't want and the whole deal is like oh one we have AGI you no longer have API rights or IP rights and it's like [ __ ] does that mean? Right?

62:52 Like if you ask someone 20 years ago and you put them in front of Chad GPT, you know, >> AGI >> like this is [ __ ] AGI. Like it knows everything and it could have a conversation. I can't tell it's not a human. Actually, I can tell it's way smarter than a human. Yeah. >> But now it's like ah whatever. I can't do XYZ. So the thing the bar always moves no matter what the level of intelligence is. And for me it's not going to it's going to be like when the thing puts its hand in its mouth and it's like yeah, this is me cuz I'm a human. Right? Like you know that's sort of like the sentience the consciousness of it all. Right? That's one power dynamic that's like crazy what's going on there. U another power dynamic is the one around Nvidia and the hyperscalers, right? Nvidia is the king. All of the gross profit is going to them today, right? Pretty much all of it. Sure, TSMC makes some, sure, SKH makes some, but they have to invest a ton in capex.

63:37 Sure, Broadcom makes a bunch and you know, Broadcom makes a ton of gross profit off of these companies, but like Nvidia makes by by far the most grow gross profit in the industry and it's not even close. And so going back to like the analogy of like well they're king and they want to continue to be king and they want to make sure GPUs continue to be most used, but also like they can't buy anything. Like they can't buy any companies. They weren't even allowed to buy ARM when they were like a nobody, right? I don't I don't mean nobody, but they weren't like they were like pretty much a nobody on the grand scheme of things and they weren't allowed to buy ARM, you know, in like 2020 or whatever or 2021, whatever the time frame was. They totally could not buy any major companies. Um, you know, they'll buy smart startups like you know, I bought a startup that I was like a seed investor in and like an adviser in like all these things like recently, but like they they can't buy a real company. So, what do they do with all this cash flow?

64:24 >> And like, sorry, but you're a loser if you just do buybacks. Like that just that's admitting that's admitting you can't get higher returns. Yep. On your capital. >> On your capital, which is fine. Like, you know, Meta, Apple, Google. They were mature companies for a while. Guess what? They're gonna those companies aren't going to do buybacks ever [ __ ] again, right? Or not like ever again, but for a while >> cuz like they have way more they think there's better ROI for their capital now.

64:47 >> And Nvidia, like you know, if you look at Jensen, he's like he's like he's always like flirted with buybacks, but like mostly he's been like reinvesting in the business, >> but you can't reinvest that much into the business. >> So like how do you >> He's doing demand guarantees. He's doing like all this crazy stuff now. >> Yeah. Right. Right. He's using his balance sheet to >> win. >> Yeah. Try and win more. Right. Um which which is an interesting uh dynamic. I don't know if there's ever been anything like this in terms of the non non anti-competitive nature of this, right?

65:13 Like where you backs stop clusters so that you know like Cor recently got a deal with Nvidia where it was like they backs stopped a cluster right now core would have never built this cluster because it's for like short-term demand and renting GPUs on short-term is like a terrible business model, right? You want to do long-term contracts. you've and you want to do long-term contracts to people with balance sheets. That's the golden like goose, but that doesn't exist so much. So, you do long-term contracts of people who don't have a balance sheet like OpenAI. And if you can't do that, then you'll do, you know, short contracts with people who don't have a you who do have a balance sheet, right? Like there's this whole matrix of like who you rent GPUs to. But from Nvidia's interest, it's like, you know what I really love is when venture capitalists fund a company and then 70% of their round is spent on compute. They [ __ ] love that, right? And that's what's happening with all these companies like these and and like it's like whether it's physical intelligence who you know they're spending a lot on like robot arms and [ __ ] too but they're also spending a lot of compute or it's like you know any other startup that's raising cursor whoever right and even if it's not directly >> it's indirectly going to GPUs um they love they love when people spend their entire round on a GPUs >> would be really good is if it wasn't like a two-year deal or three-year deal for that compute if it was oh yeah yeah you can spend 70% of your round on one training run.

66:26 >> You know, leave a company with these ideas, gather the data, do the training run, and then you have a product and like you try and you show how good the model is, then you try and raise again. That's what would be really great for Nvidia, but no one wants to build a cluster who's predicated on that as the business model. That's crazy. So, they have to back stop a cluster to do that. Or, hey, you know, OpenAI might go to, you know, their own chip. They might go to um some ASIC from another company, right? They they they might even buy TPUs. There's a there, you know, they might even like go to Amazon, right?

66:54 Like they don't really care. They're not beholdened to Microsoft anymore, >> trying to serve a product to a customer. Yeah. >> And they want to build the digital god and they want to serve a product, right? Make revenue, right? So they don't have to go to Nvidia. Nvidia is the best option. >> But you know, it' be really really helpful is if I could, you know, going back to the earlier part in this this in this discussion is if the first year I get the compute up front and I don't have to pay for the compute for the first year, >> right? like I was mentioning, you know, the $10 billion for the So, it's like it'd be really good if I could do that because then >> I can for a full year I can do training.

67:22 I can I can subsidize inference. I can do all these things that build up a user base and then I can and then I can actually pay for it. I have a year of a gigawatt to figure out a business model, right? Whether that is serving free tokens and then implementing uh this purchasing right of of you know purchasing stuff for the free user or it's and and so a lot of that is like almost no fee initially purchasing and then and then like slowly rising the fee over time right or it's hey you know I have to serve this model at worse gross margins or negative gross margins initially but then eventually I can serve it at positive gross margins because the models keep getting cheaper or it's I train the next generation model that's so much better than everyone else and then I'll win all the business for that level of intelligence because I'm the only one with a 18-year-old. You guys all have 14-year-olds, right? Like, you know, who are working for you. So, it's like, you know, this is this is a they can do whatever they want with this allocation.

68:13 It's not an allocation of capital, per se. Allocation of compute. They get to decide what they'd allocate that compute to. And Nvidia's helping them by effectively frontloading it if they can find a capital, you know, and and and that company's like, "Oh, yeah, yeah, Nvidia's backing this, too. Oh, you know, there's all these other things." It's much more reasonable for someone to say, "Oh, yeah. I'll back I I'll pay the capex because I know the first year is already going to be paid because you've got that investment from Nvidia. What about the next four years?"

68:34 >> If you ask a bunch of investors who are like students of economic cycles through history, like Carlo Perez type stuff, they'll say that the concern is that every shortage is followed by a glut and we always overbuild on long lead time big capex projects and you've got multi- gigawatt, you know, power being installed. You've got all this crazy stuff in semiconductors and like at some point like it just gets overbuilt. All the stuff we talked about earlier feels like we're not really close to that.

68:59 Like there's so much freaking demand. >> If the models don't improve, yes, we will overbuild, right? Like it's pretty simple. It's like yes, there will be like supply chain things where switches from one supplier to another and like that's a lot of the stuff that we like nitty-gritty stuff we focus on. At the end of the day, if the models don't improve, we're absolutely screwed. In fact, the US like you know in another year if if this lasts another year and then it happens like the US economy will go into recession like straight up because of this and probably Taiwan as well and probably Korea as well, right?

69:25 Because there's so much buildup and revenue flowing through to us for this, right? But you know when you look at these other things like the bubbles of the past, some of them were just silly nonsense, right? Like tulips, silly nonsense, right? Crypto complete Ponzi scheme, right? But then there's other stuff that's like no, this was real, right? like the the UK like spent like some absurd percentage of their GDP on railroads for like a decade. >> 6% or something crazy.

69:49 >> Yeah, we're nowhere close to GD 6% of our GDP. Like holy [ __ ] Um but like that was like okay there's tangible but it's like oh well we over did overbuild because like how many goods are there to transport? >> But like also you must like reduce you must build these railroads to reduce the cost of transport so much because you have no clue when the demand stops and you've overbuilt and because there's 10 people trying to do it at once you're obviously going to overbuild at some point. Um same thing with fiber. A lot of the argument against this is like, well, no, but this time it's the strongest balance sheets in the world.

70:16 It's the world's most profitable companies. They can all pull the plug at any point. >> Yeah. >> Microsoft pulled the plug at one point before they're like, oh [ __ ] no, no, plug it back in. Right. They recently plugged it back in. They're like, "Oh, wait. We're starting. We're restarting this. We're going out into the market. We're signing deals with uh Nebius for GPUs." Like, I don't remember how big the deal was. It's like 10 plus billion.

70:33 >> Yeah. It's like 19 billion for Nebius. It's like, well, if they had just not pulled the plug on their data centers, they wouldn't have had to do that. they wouldn't have to pay those gross profit dollars to Nebius, right? But, you know, Nebius made the bet that the demand is there and they were right. Um, and so, you know, when you think about this, it's like, >> what is the level of demand where this stops, right? If if scaling laws continue, right? How I mean, of course, there's a adoption curve, there's a pace, there's realities with capital, there's realities with supply chains, things take time. But if you like boil it down to it, it's like your demand for 30-year-old senior engineers at Google who know how to make and program anything is effectively like I don't I want to say infinite, but it's $2 trillion of value.

71:12 >> Yeah. >> Right. If I could have an intelligence as smart as a Google senior engineer, that's $2 trillion of software value, right? Because that's how much I pay the world pays to software engineers today. >> Um and you just go down the list of every other use case, right? If you have just a simple, you know, phys physical intelligence robot that can do this, that can recognize headphone versus water and pick up or versus phone, right? And pick up the right thing and manipulate it properly and put it in the right spot and sort it, that's worth how much to the distribution uh supply chain, right? Like I I don't know, but a lot, right? So, it's like there's you we don't need to get digital god for there to be immense value. But the interesting thing here is that, you know, it's like human capital, capital goods. All of these other revolutions have been human have been capital goods that reduce the amount of human capital you need.

71:53 >> Whereas this is just creating human capital, >> right? In a sense. In a sense, right? If I like sort of get everyone bowled up, right? And we're we're on this podcast, right? You know, there's like this I don't know if you've like uh heard the curse, right? It's like if you talk about the stock on in on this podcast, it goes down, right? I've heard word of it. >> We're popping the bubble right now cuz the limit of AI is infinite. For the record, we went and did we went and did the math one time because I was sick of hearing about this [ __ ] curse and it's just market performance. It's not >> Oh, really?

72:21 >> So, so, so I last time I was it wasn't your it wasn't this podcast, it was your other podcast. I talked about applied materials and the stock was up like 70% the six months after. I was like, >> there you go. Yeah, I broke the curse. >> I was like, hell yeah. >> What do you think about all the companies in the middle? We've talked a lot about Nvidia and then like people at the end serving applications. What about these companies like together and base 10 and fireworks and you mentioned Nebius like all these interesting middle middle layer players are there amazing businesses to be built there do you think are they temporary patchwork to make the system work and serve you know serve demand like what do you think about that this middle layer >> the cloud business model right like uh let's let's say neocloud business model so there's you sort of you mentioned inference providers and neoclouds um the neocloud business model is absolutely amazing or terrible depending on how you do it right it's terrible if you like sign short-term contracts and you just hope and pray you have short-term contracts forever and actually initially your short-term profits have amazing cash flows, right? Because you're selling it at like you you bought a GPU and you put it in a data center and the power and all that. The cost per hour over a six-year period for Blackwell is $2. Let's just call it for simplicity sake, it's $2. It's not exactly that, but if I sold it for 6 months, I could get like north of I could get like 350 or $4. Like, holy [ __ ] that margin's insane. But what happens two years from now, three years from now when I'm still selling six-month contracts or one-mon contracts and the next generation of Nvidia chip is out or and it's 10x faster for 3x the cost, right? Okay. So now, you know, naturally the price of this should tank. The other way to do it is is hey, I actually like have a long-term contract of well, I'm selling to opening I'm selling to micro the other end of the spectrum is what Nebius just signed. I'm signing [ __ ] $19 billion to Microsoft. They will stay no matter what. The market literally believes Microsoft will pay its obligations before the US government because it's like literally a cheaper bond rate, right? Which is like insane to me, but whatever. This $19 billion has like a huge gross profit cuz the price per hour and and it's not exactly $3 and it's not $2, but like the margins here are like really good. Like Nebius is going to make like at least $6 billion of gross profit off of this. Um, and then obviously they have their operational cost, but like like $6 billion of gross profit off of this deal is like insane.

74:28 So like I would do that all day and Cororeweave did until Microsoft stopped going to Cororeweave, right? Um and but like Cororee's turned around and they found other customers and all these things, right? Selling to Google and selling to OpenAI. Oh, but now Oh, OpenAI is definitely not like a real, you know, you can't rely on their balance sheet. Okay, I still have amazing margins when I sell to OpenAI, but they don't have a balance sheet. So how can I be sure that they're actually going to pay the thing that they've signed up to? Right? So, so you know, I know in theory this contract is worth a ton of money and in Cory's books today are that that contra all the contracts they've signed are mostly Microsoft, mostly money in the bank, right? But the open egg contracts like what if they can't afford to pay for this? Okay, now there's like there's a bigger risk and there's a longer and longer tale of like these businesses. So like yeah, you absolutely can make a ton of money. Um, you know, there have been more recent deals with crypto miners, Google, and Fluid Stack because Google's really short on data center capacity. Um, people want to use more TPUs. um they can't they can't serve them all themselves. So, they're going to sell TPU systems to uh providers. Um you know, Google Yeah, they're backstopping the deals with uh Terowolf is one of the companies. Um I can't remember the other one, but there's two companies they've signed deals with where they're backstopping the data center >> plus like selling the the TPUs physically to another company and then they're being deployed and then they're getting rented. Um and Google still makes all the money. like you know there's there's those companies you know yeah that's great as well but then there's a long tale of like is the enterprise demand there is the who's taking the risk right and it's like open taking the risk because they're betting their entire company could go bankrupt if it doesn't come Oracle's taking a risk because they're they're signing up for you know $300 billion of contract okay $200 billion of hardware spend across data centers and and chips and of that like they're going to have to go get debt right um so so they're on the hook and they they'll probably be able to pay for it if it happens but they'll just be like their their EV will like tank if opening I can't pay for all the hardware that they bought right and and luckily for them it phases in over time and whatever right but like and then you go to the inference providers and it's like yeah there is there's a business to be made here too right like I'm serving models maybe Roblox comes to me and they want to put an LLM in their in their game right because of XYZ reasons okay Roblox is a good customer or like hey you know another company like uh Shopify wants to put an LLM for customer service and you know yes they could do it themselves but actually there's inference is a hard thing especially as you get to larger and larger models and more complicated models and all the things or you know there's all these different use cases um where people want to serve models and maybe it's maybe it's just open source models and maybe it's fine-tuning of those open source models which those companies can help you do or you can do and they can serve for you and they have scalable reliable capacity like it's like there's businesses to be made here but there's also like yolo I'm selling tokens to like random people who are trying to build SAS apps in NSF and maybe they run out of runway right and and and like okay that funding doesn't directly go to Nvidia, but you go through some steps and it's going to Nvidia after some value chain. Um, and and Nvidia's holding no risk. Everyone in the middle's got a lot of risk.

77:25 >> I'd love to hear your thoughts on like going back to the other side of the equation, the the app side, you know, stuff we're going to use these models to do at the significance of this switch from like deterministic code to a much different thing. And it seems like what we're doing is the thing we always do, you know, Apple used to call this like the schemorphic era where you just basically use the new technology to do the old thing you used to do. So, we're making engineers better. You know, that would be like an an obvious current example, but we seems like we haven't yet gotten into the world where we're going to start using this this technology to do things that we couldn't do before with deterministic code. I'm curious how you think about like that side of like pushing the envelope.

78:02 >> Why is that? Like I feel like that's exactly what we do with it, right? Is like the cost to develop things is so high that you can't do it, right? like or the cost to like you know have someone go buy stuff for you. It's like, okay, great. You might have an executive assistant and you can tell them to like, you know, do this, but like the vast majority of people don't. And now GPT is on the cusp of doing that, right? Go buy go do this, go buy this for me, and they'll find the best thing and they'll buy it, right? And you just trust them enough, right? Um it takes time to trust them, but like it's like these things are proliferating across, you know, the massive, you know, tech tech is the most deflationary thing in the world ever, right? Uh in terms of quality of life, it's it's so it gets cheaper way faster than the revenues go up, >> right? But the revenues still go up.

78:45 That's sort of like the fundamental basis of semiconductors, of tech, everything, right? Are we doing things that we can't do before with tech, with AI? Sure. I mean, like the COVID vaccine was like created with AI. Like it was like AI drug discovery. Like you can there's like there's like entire briefs about like how it was done with AI. And guess what? If like another pandemic happened, I bet it'd be even faster to discover the vaccine if there's a vaccine for it or whatever, right? Like there's all these protein folding things. There's all these like optimization things. There's AI for material science and AI for like, you know, all these other like aspects of society. There's optimization. Maybe it's not in your face, right? It's like, oh my god, AI just made this drug, right? It's like, no, I mean, AI worked with the researchers who made the COVID vaccine and so we didn't have to all like, you know, be stuck inside forever or whatever, right?

79:31 >> Point being, we're it's already happening. And the whole like use the new thing to make the old thing faster. It's like sure, but like if I go back three years, how many people would it have taken to deploy a image recognition model that looks at every data center in the world and and looks at like what's the pace it's of constructions in and what equipment they have and like >> assuming this is something you do.

79:55 >> This is something we do, right? It's like it's like how many people would that have taken? I don't think it would have been possible. my business model like this is the second highest revenue product for us would not have been possible >> if it weren't for AI like vibe coding like you know being able to dig through permits and regulatory filings being able to run image recognition on satellite photos like this would not be possible this business is not possible without AI and like am I using it directly like oh yeah sure I'm scraping through the regulatory filings and permits through with LLMs and then manually reviewing it with people and like you know and or like doing it doing that the same with like the the images, satellite images. Yes, there's a lot of stuff that you know the image recognition model does. We just also look at them a lot >> and then it's like compiling them and selling a spreadsheet that you get like bi-weekly reports on like all the data centers or what's changed or like hey actually this Amazon data center the fans are starting to spin so actually there's revenue going on from this Amazon data center so we can forecast Amazon's revenue right it's like oh okay like this is like relevant right like you know but like I don't think this would have been possible just a few years ago um at least like off the proof right now and especially like you know there's demand for it because everyone wants to track this and it's so important but it's like it begets each other and I think like at least in my daily life it's like I don't think I could have taken that step from where I was in a business which was still a research provider but like that is a monumental jump and like being able to do it with three people out of the gate versus like 50 or 100 like I don't know how many people it would have taken but I don't think it's possible and it's like mainframe migration is something people have always wanted to do Amazon leaving Oracle took [ __ ] 20 years right and they wanted to do it 20 years ago and they had their highest revenue products after EC2 were like the next four were database products at AWS and yet they still freaking used Oracle's database because it's hard.

81:39 >> Now there's like mainframe migration can be way faster or like migration from one tech stack to another can be way faster. You can make your business more efficient. You add more automation. Yes, the tech exists. Go to all the businesses around the world and it's like they aren't using the leading edge of what they could. They aren't using, you know, what a 2020 company could have done without AI, >> right? No one is doing that. And if they did, they'd be so much more efficient, right? But like all of these things just take too long to build. They're too expensive to build. You have your existing processes. How do you hand them over? How do you switch them over? How do you teach people to do this? AIS can help you with all of this, right? So it's sort of like you can take the pessimistic view of like, oh, we're just doing the same things, but it's like the value here is humongous.

82:19 >> If it's tokens on one end, we haven't talked much about like watts at the very beginning and power. What are your thoughts on like what is going on here and how like humanity is responding to this crazy new demand for just raw power? The first approximation is that like we're being a bunch of pansies and it's not that much power yet, >> right? Like AI data centers are like 3 4% of the US economy. Power, not economy. Um or just data centers period of that like two is regular data centers and two is like AI data centers, you know, that's nothing, dude. Like that's literally nothing. Uh it's just we haven't built power in like 40 years, right? Or like we've transitioned from coal to natural gas more and more over 40 years. It's like so mostly we just don't know how to and there's these regulations and like there's not enough labor and like the supply chains for like Evanova and their dual combine cycle gas reactors are not there yet and same for like Mitsubishi and you know you you know oh like this random e UV curing process for transformer coils is like you know is like there's only this much capacity and it takes two years to build them. It's like it's just like it's a supply chain thing. It's a like lack of labor thing. It's not that it's like actually that much yet, but like at the end of the day, it's like, okay, wait, wait, you're telling me open is making a data center with 2 gigawatts and that's like the entirety of the power consumption of like Philadelphia.

83:31 >> Like that is real. Yeah, >> that's insane. That's insane, right? But like we used to get like excited about finding like a couple hundred megawatts new data center. >> Now it's like if it's not a gigawatt like I I remember like the the the guy who leads that team, he he was like, "Oh, it's just 500 megawws, whatever." I was like I like immediately opine I was like I I I also agreed immediately then afterwards I was like wait a second dude that's like a lot of power that's like how much wait 500 megawatts is $25 billion of capex like come on like once you put in the GPUs and everything right it's like that's a ton of money but like snore because there's so many of it happening right we're learning how to build power again right um we're we're getting the the supply chains to do it again we're reshaping the grid uh there's all these challenges with these AI data centers that with regards the demand response and making grids unstable, right? Um, you know, you could, you know, AI workloads because they change so much so fast, especially training, you can just blow, you could just you can just cause like brownouts or blackouts. Um, especially if the grid doesn't have enough inertia or if you're not putting enough like things to dampen it in between the workload and and the grid. And even if it's not destroying it, uh, the grid runs at like 59 hertz or whatever, right? If you you skew it up and down too much, um you these transient power responses, your refrigerator will break down sooner the motors in it and and you might not even know it because the data center's nearby. So like there's like this there's all these things. There's so many like third order effects here with like AI data centers. But like the funnest one is just that like we're building power, right? And it's like whether it's, you know, gas, which is a lot of it, whether it's through efficient dual combine cycle reactors or it's like, you know, random generators that are like not nearly as efficient, single cycle or even worse. Um, diesel generators. There's there's a company that's putting a bunch of truck engines in parallel like like diesel truck engines because the capacity the industrial capacity for diesel truck engines is huge.

85:22 >> Like and no one's tapped it yet. So why don't we just put a ton of them in parallel and create this power generation thing right here, right? and and and then you're generating power with a bunch of diesel truck engines in parallel and then you're able to power a data center, right? Like, okay, great. Because I can't get turbines, right? And it's like or like, you know, there's there's all these like crazy things people are doing. Uh Elon buying some power equipment from Poland and shipping it to America because he needed that power equipment, but like whatever, couldn't get it here because the supply chains were weird. I'll just get it over there. Any lacks capacity in the supply chain is being eaten up immediately and then everyone's like, "Okay, let's invest." So G is like, I'm going to double my turbine production. It's like, holy crap. Okay, that's awesome. And Mitsubishi is doing the same thing. And you know, you move, you go down the list, it's like, my my transformer supply chain is expanding like crazy.

86:05 And like, you know, they're fully sold out, so I'm going to go to the Korean guys. And that's fully sold out, so I'm going to figure out how to get the Chinese stuff in, even though it's not exactly like, you know, what people want to do, right? It's like there's all these like weird things like electrician wages have like doubled for mobile electricians that can work on data center stuff or rather contract. Like if you're down to move to West Texas, it's like it's like it's like 2015 again and like being a fracking guy, right? You don't need to be super duper skilled.

86:32 You can go to West Texas and make a shitload load of money off of fracking. But there's not enough of those people. That's why, right? Like if there were enough electricians in West Texas, if there were enough electricians in America, we could build these data centers faster. So there's like all these like little supply chain quirks and weirdy weirdities. Everyone's supply chain is different because the way Google makes their data centers is different from the way Vantage makes their data center which is different from the way that Edge Connects makes their data center which is different from the way QTS makes their data centers which is different from the way Amazon makes their data centers. So their supply chains are not exactly the same. Um and so you get all these weirdnesses in all these different supply chains. No one really knows it because everyone who tracked the supply chain or like knew it. Like you go talk to like power people, it's like it's like on one end of the spectrum is like Daario and then you take a few steps and it's like it's like ML researchers, the average ML researcher, then it's like me and then it's like you in terms of how bullish we are on AI and the guy at the power utility is like over here, right?

87:24 Um like only there's like a few more people. There's like the standard New York stock, you know, New York investor, semi-investor, then there's the New York like, you know, um not semiism investor. And then there's like, you know, the Sequoia guy who thinks that A has been a bubble since 2023. And then there's this this utility guy, right? Um you know, this utility guy is like, "I'm not building power. Power doesn't go up, you know, whatever." And then you have like the regulations around it. Um you know, it's like, "How can I build a data center in this density?" Because um okay, well then great. Like I'll build the data center in this density. I'll have all this backup generators. Great.

87:56 Now all of a sudden the grid's like um yeah. So what we're going to do is we're going to So this has happened in Texas or it's happening in PJM um which is the main you know the sort of northeast kind of areaish. um these two grids are putting these like rules where hey we're gonna actually say hey big loads we can tell you 24 hours or 72 hours beforehand we're going to cut off half your power so that we can like which is fine right because like >> we need to because we need for something else >> yeah like people people need to have their homes powered we're not [ __ ] like Taiwan where if we're in a drought we limit people's power us water usage and not the fat right which is which is a real story right I think there was there was it was like 2022 2021ish >> there was like multiple like cities where they were like okay yeah we're going to limit the showers you can take to three a day or three a week, which is fine because they're East Asian and they have they don't have the smelly gene like you know you know like if you did this in in India like it'd be it'd be cooked. I mean it's already cooked but like you know like you know but anyways like you know they'll limit the water to these people before they'll limit the water to TSMC because and it makes sense the economic value of TSMC is way above the economic value of people showering three times a week. But like the US grid is not going to work that way. We're not that authoritarian or you know people have more say. Okay. So, anyways, like it's like in Texas and in PGM, you can cut half the power if you give them a notice. And if you do that, then you need to turn on the generators that are there on the site. You know, it's often diesel generators, maybe it's gas, um maybe it's like hydrogen stuff. There's all sorts of weird stuff people try to do just to ramp up power uh for that period of time. But then all of a sudden, oh crap, the density of my generators means that I fail the air permit if I run the generators for more than eight hours a month. So now what do I do? Right? It's like there's all these like weird regulations. Even if it's like Texas, it's it's really fun that uh >> we get to watch it. you get to see watch it and then watch the supply chain and try and like you know at least from my perspective provide the data so people can trade on it or provide the data so people can adjust their supply chains you know industry-wise right people who go to your audience they can trade on it or they can like see and invest and make money and like allocate capital more efficiently right >> if if I were to line up all the stages of this between the US and China so you know power semis models applications etc where do you think the most interesting differences are like what are the what are the story lines between us and China at those various layers of like the AI stack that are the most interesting to you?

90:10 >> When you look at China, it's like they're a very formidable competitor. Um I think if we didn't have the AI boom, the US probably would be behind China and no longer the world hegeimon by the end of the decade if not sooner. And a world where the US is not the hegeimon is >> is a bad one for Americans at least. Um, you know, I I I, you know, I'm sort of like a, you know, >> sure, >> [ __ ] bald eagle like carrying like American. I'm like, it's bad for the world. Without AI, like we're definitely just going to lose, right? Our supply chains are slower. They cost too much.

90:43 Uh, we're we're sliding. Our debt is like unsustainable. Like, you know, our economy is not growing fast enough to maintain the level of debt. Like our we're over consuming relative to what we produce. um the financialization there's like this all this like darth and like human like in like the US in terms of like social instability partially because of income inequality but also largely because of um the visualness the visual like nature of of income inequality and the tendency of people to flaunt their wealth more because of social media and how that hacks people's brains and then also like because the algorithm serves people different content we're drifting further and further apart in culture right monoculture of everyone watching watching the same movies in the ' 50s and 40s and 30s versus like now like you and I are pretty similar and our feeds are completely different. So think about someone who's not in this world like in in our like you know similar worlds like their feed is like insanely different. I think the US would literally fall apart if we don't do something like and and by do something I mean like >> AI has to dramatically accelerate GDP growth. Once you start talking about dividing the pie you're screwed, right?

91:47 Um it has to be growing the pie and you know this this whole thing, right? Um, so I like I'm I'm like, you know, the US really really needs AI. China's view is like I think it's like a little bit different, right? They don't necessarily need AI to win. They've always played this long game. They did it with steel. Um, they've done it with like, you know, rare earth minerals. They've done it with solar panels. They've done it for, you know, producing phones. They've done it for PCBs. They've done it for so many freaking industries. Incrementally, they're just going to continue to do that. And then they're going to win because they they work harder and they're on average smarter. If we don't have super powerful AI systems, we'll run out of like easily accessible like nickel and cobalt and oil and natural gas and we won't be able to make solar panels efficient and fast enough and everything will start to get more expensive and the pies will reduce and we'll also tear each other apart in that way like uh sort of the so I have like a very pessimistic view that if we don't accelerate we die. If that's your world view then like we really need to win AI.

92:40 Um, and China's worldview is sort of like, you know, they want to be the world hedgeimon. Like, I mean, who doesn't want to be the world hedgeimon? But there's only two countries in the world that can legitimately do it and legitimately are trying, right? The US and China. The way like the Chinese AI ecosystem thinks about this is, well, we don't necessarily need to have the biggest compute cluster. When OpenAI is trying to make a 2 G data center full of GB200s and GB300s, like all these different chips, and those chips are way faster than the chips that will sell China/ the chips China can make themselves. and China is deploying less of them. You know, the girth of compute is huge. You know, we're kind of doing what China's done historically, which is dumping tons of capital into something and the market becomes >> interesting.

93:19 >> And the beneficiary is like, oh, if OpenAI, you know, they have 800 million users today when they have three, four, five billion users across the world, which is possible, right, of of chat, GPT, and whatever applications they come up with, then they're on our system and then then they can start to make money, right? It's sort of like YouTube lost money forever, but now it's the platform for watching videos across the world, right? and Chad GPT will be the same thing. So sort of that like there there's that like aggregation theory.

93:42 China doesn't necessarily think of it the same way. Um but they are still incredibly pilledled on like well we want to be able to make everything ourselves, right? So make all of the chips ourselves. We're not necessar we don't actually care that much about making all the chips ourselves. Sure Trump's doing the tariffs, sure we had the chips act, but those were drops in the bucket compared to how much money China's releasing into the semiconductor ecosystem and have has been for the last 10 years. they've dumped, you know, at least like $4500 billion into this ecosystem through SOE's through um who, you know, through certain tax policies, through certain like land grants, through provincial governments, through uh the big funds, uh which is like government venture funds almost. So, they've dumped so much more capital into semiconductors than we have in an unprofitable way because they want to build that ecosystem. Um and and over time, you know, it's like, well, if you take any country in isolation, China is the one that has everything at the highest level on average, right? Sure, they're like 30 years behind on jet engines, but they don't actually need to go for or 20 years or 10 years, whatever it is, but they don't need to go outside of China for any of the materials besides like raw materials. Whereas like the US needs like titanium from here and like, you know, blah blah blah from there, right? And the same applies to their semiconductor ecosystem, right?

94:54 Sure, the US and Taiwan and Korea are way ahead, but then they also have the accumulated capital base of all of the existing equipment and all of the existing fabs, but then they they don't have um you know the cap like they they need to import from all these different places because it's a global supply chain. And so China is like much more concerned today about being insular than being the best in doing this like sort of aggregation theory. But because they're so talented and they have an insular supply chain, um yes, they purchase some stuff from the foreign world. They rent stuff. They have Bite Dance who's I think the third largest user of GPUs in the world. Um after OpenAI and um you know probably Meta, although bite dance may be bigger than Meta, but third largest user of GPUs in the world or second maybe even Bite Dance is um they they have they have all the other major Chinese tech companies.

95:45 They have all of these amazing graduates. They don't have a talent pool. companies don't poach from each other, right? Deepseek engineers make a lot more than other engineers, but they're not making $10 million even though they may be worth it, right? Um there's this like real big perception difference. And China could build way faster than us. If they wanted to build, you know, a 2 gawatt or 5 gawatt data center, they could probably smuggle a lot of chips. It's not like a pure derog a derivative of them wanting to smuggle shitloads of chips because hey if they wanted to build a 10 gawatt data center I bet they could build it in like a few years. Um whereas the US is not going to build a single 10 gawatt data center for >> for a while right like the total capacity of an open AI will be like 10 gawatts in a few years right um optimistically you know they don't have the best chips speed rating up trying to get better and better and faster and faster. Um they don't have the best memory. Um they're trying to get better and faster there. they do have the most power. They can build stuff way faster, right? We're impressed at how fast Elon does stuff. Elon's slow compared to China. Um, and I think he knows that.

96:48 Um, which is why he's maybe like the one who's like actually using the Chinese ecosystem more in terms of like, you know, the the battery facilities making in China and, you know, all these things, right? He's he probably recognizes it too. Um, you so there's there's like these these major differences in like viewpoint and approach because China wants an insular supply chain. They want to have supply chain security. We talk about wanting that, but we don't actually put the money behind it. We're, you know, where's the, you know, where's the the Russian roulette of like where's the American uh or the slot machine of where the American capital is being allocated. Um >> it's building the biggest data centers.

97:21 It's training the best models. Whereas in China, the capital's being allocated into >> um growing the EV supply chain, growing the semiconductor supply chain, catching up in all these areas. And like the US, sure, we want to catch up, but actually we're just gonna give like terrible. >> Maybe Jensen was right that like what you want to own is the end customer thing. And >> yeah, export export to production and and import doing the same thing they've done forever, which is like prepare at the base level and be behind at the customer side and the value the value happens close to the customer. But then like you get to the point of like okay well what happens in like three four years even if the US AI is amazing we have no you know like the doomsday scenario of like you know China decides to blockade Taiwan or even invade it or create some political instability.

98:06 People talk about like Cambridge analytics and like Russian trolls whatever like China could do a billion times that into Taiwan especially with AI with how good AI is now and and somehow subvert it or coup or blockade or whatever and we no longer have Taiwan US economy kind of free falls, right? Because we can't make refrigerators without Taiwanese chips. We can't make, you know, cars. We can't make AI data centers. We can't grow any of the cloud.

98:34 We can't That means we can't deploy any more SAS applications. Like, what the hell can we do? >> Back to going to acquire all the talent, get them over here, do that. >> Right. I I think like that's sort of like the the like catch 22 of this all is like if you push China too hard, they totally will. Like they're going to start swinging. They have the talent, they could go crazy, they could, you know, if if we no longer have Taiwan, actually, China could build a way bigger cluster than us. And if comput is all that matters, right? Like they could do all of these things and they own the means of production for everything, right? So sort of like it's like, you know, there's this like challenging aspect of like geopolitical risk is like, you know, that's why people don't want to invest in TSMC, but it's like almost like you can't invest into Amazon or Apple or Google or like Microsoft if you have geopolitical risk. if you believe Taiwan has risk and so it's like yolo invest in TSMC. I know a lot of people's PMs are like oh you can't invest in TSMC because geopolitical risk and it's like no dude you can't invest in [ __ ] Apple.

99:28 >> Who is your favorite AI bear? Like someone that is is far distant from you on just their perspective on the direction of this whole thing that you nonetheless like and respect. >> There's some of the like AI researcher like gods like Yan Lun and like these kind of people who are AI bears. I I respect them. I like what they like their ideas. I think they're completely wrong, but you >> and their argument is what? Like if you had to sum it, you know, >> the ways we're doing this won't work, right?

99:54 >> LLM's on scale or >> right, but and it's like, okay, yeah, yeah, auto reggress like it's like, okay, auto reggressive pre-training on the internet doesn't work to get you to AGI. So, he's completely right on that, but then like he'll turn around and be like, well, no, no, no, but like RL systems and all these things are not the right way either, right? Like, you know, it's sort of like, you know, it's like the no butts. Um, I think there's also like there's some investors that I know who like think this is [ __ ] but they're just making tons of money on it anyways.

100:18 >> I think it's [ __ ] in what sense? Like >> it's an overspend and like, you know, this is a dumb way to do it. And like, yes, I bought Oracle before earnings because, you know, you know, we see we see these deals happening, whether it's through our data or some other means. They saw these deals happening um with OpenAI, but we don't think OpenAI can pay for it, but we know the market's perception will be this and therefore the stock will go up and so we'll own it, right? Um, and so there's like I guess I would respect them to some sense, but it's it's it's it's um I think more and more the level of evidence that's there that this stuff is going to get super powerful, it's hard to not, right? Again, like this AI bubble is going to pop because this podcast, man, >> I assure you it's just a market return.

101:00 It's a coin toss. >> What startups interest you the most? >> So, one of the startups is um that I've I've you know, it's the most recent investment I've made. It's called periodic labs. Um it's mostly open AI people. It's a Google guy and you know the couple material scientists. Um the area of AI that we've all been talking about is like large scale web training RL all text all digital god right. You know I you know we want to make digital god.

101:25 >> Yeah. >> But you know what would drive a shitload of value for the economy besides you know automating programming of everything is like if we just like came up with like a battery chemistry that was like 25% more efficient. Mhm. >> Like holy [ __ ] >> You know, like the main main cap against like us all having like face glasses, you know, things like that is like batteries are not good enough, right? And the power dissipation, but the battery is like terrible. So, you have to do make all these compromises. But if I could have the processing power of like, you know, a laptop on my face, we'd be way further ahead. And like and then and then if we all had like these like super powerful machines attached to our face, we could do inference on things and recognize and interact with the AI at much higher speed and velocity and and that would like dramatically improve our productivity, right? Like things like this are like so like gated by hard tech moving faster. And so what Periodic is trying to do is they're taking this RL paradigm, but they're trying to do it with like real world, right? test chemistry for something, right? A here's a here's a chemistry um here's an optimization, here's something that the model spit out, but then you also want to test it in the real world and then feed that feedback back into the model. And so you do this like chain of of circles, right? But instead of purely being, you know, digital, right? Which is which is why like RL is like really hard because you need to generate a bunch of responses, test, and then train the model. So the flywheel is so freaking fast. Yeah. Right. the flywheel in the physical world is so slow, right? Oh, if I you mean I need to make a chemistry, I need to try this. I need to test the the thing. Um I need to input it back in and and you know, I need to keep calibrating and keep doing this. It's so much more expensive. It's so much harder to do.

103:01 But actually, there's a ton of low hanging fruit there, I bet. >> What about in the hardware world? Like just in the pure hardware space attacking some other interesting bottleneck >> when we talk about like where we are in tech, right? Like it's like um semiconductor manufacturing is super space age, right? It's like the most complicated tools we make in the world. Um, including like tools that cost like half a billion dollars, right? Like, and and they're super super like amazing feats of engineering. Then the software behind them all is like really [ __ ] right? So, it's like, you know, you could you can accelerate all that. Um, but really it's like um in the hardware world, the biggest challenge is that like I'm not really a big bull on the accelerator companies. I've never been.

103:38 Companies make competing with Nvidia. >> Yeah, I got it. um competing with Nvidia, with TPUs, with Tranium, you know, with with AMD. Not a big bull on those kinds of companies >> because it's too hard. >> It's just too many things to do. It's too capo intensive. It's too there's not enough of a revolutionary leap. There's too many predicated things. Um you know, I'm I I wish it could happen, right? It'd be fun. Um maybe it does happen, but it would take hell of a badass thing. But I think there's a lot of individual parts of the supply chain which are not spaceaged, right? Um, Nvidia space age, yes, it's the biggest value value owner today, but their supply chain has so much old [ __ ] right? And whether it's their supply chain or the hyperscale supply chain, transformers have not changed in like 50, 100 years, right? Like >> there's a guy building a company in that space.

104:19 >> Solid state transformers, right? Like things like this. Yeah. So, there's all sorts of interesting things there. There's so many interesting companies in that space because there's so much innovation to be done and there wasn't that much of a need to do innovation before. Another area is like networking between chips because as we extend context length the memory requirements become bigger and bigger and yes new memory technologies would be awesome but DM is an industry has so much invested capital goods so much so existing factories it's really hard to attack. Um but you know networking is less so and there's more breakthroughs that can be done in networking that okay maybe you don't have better memory technologies but you've tied the chips closer together so you can use each other's memory on the problem. There's so much more that you can do in terms of the optics space bridging the gap between electrical connectivity and optical connectivity because look you know Nvidia created Blackwell they had a ton of manufacturing problems and challenges with it um for their supply chain um balance sheets went up for you know various companies in the supply chain were building servers and stuff because they're trying to figure it out AI server AI data center deployments were slowed because of these challenges there's reliability challenges because these things are connecting to each other at you know at absurd bandwidths right every chip in the rack and connect to every other chip in the rack at 1.8 terabytes a second, right? That's that's if you think about how much data that is, right? Like like the amount of bandwidth is so high for connecting these chips together. Like I like you can't fathom what a terabyte a second is. You can't fathom what a gigabyte a second is. You know, it's like okay, a gigabyte a second is like a video, right? Like or like less than a video, right? Or like a megabyte a second, but actually that's a million bits of information. What's what's a kilobyte a sec? a bite a sec. Okay, you can understand what a bite a second is because that's eight bits.

106:03 >> Okay, I'm transmitting eight bits to you back and forth every second. That's pretty fast. >> That's what it used to exist. And it's like where we are now, there's still tons of innovation left to be done there. Like I think part of the reason Intel is behind is also that uh data sharing internally was terrible >> and just within the fab the lithography team doesn't want to share their data with um the etch team and they can't use and and that data can't leave the fab and go to an AWS data center to run you know uh you know correlations and all these other things. So you don't learn from the experiments you do fast enough.

106:35 >> Right now TSMC is not perfect here either. They won't send their data to a cloud either. But like you know this experimentation experiment analyze the data figure out the new experiments cycle is slow and how you break that is actually like you know partially it's it's changing these companies culture which I think lian is trying to do. Um but also it's a lot of it is like building better simulators simulating the world more accurately. So world models generally are like hey I'm going to simulate the world. I'm going to walk around in, you know, the common one I think is G3 that Google made, right?

107:06 Where you can walk around and at at Cape State and you walk around the world and you can like see cars driving and like they're talking about like but it's like actually what a world model could also be is just like simulating molecules or simulating um but not through classical methods, right? It's not computational fluid dynamics. It's it's the model experiencing this enough and then running and training a model on physics and then feeding that back through and doing it through a AI method instead of and so world models can be doing any sorts of things, right? You can put make a world model to train robots how to pick up cups, right? Or you can make a world model that is simulating some chemistry in a chemical reaction, right?

107:42 Or um a fire, right? Like you could do all sorts of different things. So there's a lot of world model companies out there. Some of them are really interesting, especially when they're targeting the physics and reality of the world. Most of the cool innovation is just happening at big companies or already existing companies, right? That's just the nature of it all, right? Like actually TSMC is doing the most cool innovation and Nvidia is doing the most cool innovation and like you know Amphanol is doing cool innovation. It's like all these companies are doing cool innovation. C >> could we do like a uh a quick speed round where like I say some company and you just give me like you know a sentence or two on like your impression of them just like just like how how you feel about them in this moment. Yeah, >> start with Open AI.

108:18 >> Oh, yeah. Super awesome. >> That's it. >> I mean, we've talked about them all day. >> Anthropic. >> I'm actually more optimistic on anthropic than I'm Open AI. >> Why? their revenue is accelerating way faster because what they're focused on is more relevant to that two trillion dollar software market versus OpenAI is split between yeah they're going to do that but they're also going to do these other things but they're also going to do like target AI for you know science and they're going to also target AI for um you know the consumer app and doing the like take rate thing which all of these businesses could be amazing and open maybe executes on all of them but Anthropic is definitely executing on the software side better >> yeah AMD >> I love them but they're pretty mid >> why do you love them if when you grow up like like building computers and like liking computers and like AMD's innovating and they're always like fostered this underdog mentality against Intel and against Nvidia, evil Intel and evil Nvidia, you know, and like AMD is like, you know, the nice company that's like the underdog and like they're always they've always got the oh, they're going to take share from them thesis. It's like it's like it's hard not to love them, you know, like and I know so many people there and I like the I like all these major hardware companies, right? There's not one that I don't like as in terms of the people, but like AMD's got a soft spot because like I think it was my first multibagger as well.

109:29 >> Like my first multibagger. I can't own stocks anymore because compliance. Sorry for the rant, but I [ __ ] love AMD, you know? I also love Nvidia, >> but mid >> but mid >> XAI. >> They're in a real danger of not being able to raise capital. U Elon's the best C. Of course, everyone's going to give Elon capital, but like the scale of capital required uh for him to keep up. He can get the next bet. He can get to Colossus 2, right? uh this mega data center that he's building, largest data center in the world when he builds it, uh 300,000, you know, black wells, 500,000 black wells, right? Like it's going to be really great, but if he doesn't figure out a business model besides like Pornbot, >> um which is what Annie is, >> which also I think he's monetizing the wrong way. Like I think he could monetize it so much better. how >> you've captured the zeitgeist with a cute anime girl that talks to you in a cute voice and like will rz you up and and you've got like these users who actually fall for it and it's like not realistic enough yet, but it will slowly get more realistic. And you're not like you're selling like outfits for the same price. You should make it a random like, "Hey, you have a chance to buy the outfit that is actually her being nude."

110:35 Or like, "Hey, you have the chance to buy the outfit of her like looking like this one anime girl from this one anime." Or like, "Hey, you have the chance to buy this outfit that's her in a nun suit." I mean, obviously like, you know, people at XAI hate this and like a lot of them and and many of them have some of them have left. But I think like he has to figure out like some business model beyond just this, although I think this could be a big business, right?

110:54 like he he should partner with Only Fans and make like make manifestations of the Only Fans creator with that are Annie and then he subsumes the Only Fans platform into X >> the everything app >> and be like XXX like you know it's like you could you could just like Trojan horse like Only Fans away cuz like the discovery mechanism for only fans is like Instagram and Twitter as far as I understand >> and like you own one of them >> and you could partner with the biggest Only Fans creators to like you know get them over right like you know and then they they don't have to respond to all the losers like they can also just like train a model that like acts and looks like them and talks to them. Anyways, like there's all these different monetization methods and I don't think that's what he should only focus on, right? To be clear, XAI can get to the next stage of compute. They won't have more compute than opening. I won't have more compute than any individual company at Google, Meta, etc., But they will have the biggest individual data center >> and what he does with that and he and they'll be they'll have a very focused team and what they do with that they have to do something like really big otherwise they will fall behind in the race and and Elon will not let that happen like he doesn't want that happen but he can't he can he can subsidize and fund this round but like he can't go to a 3 gawatt data center >> unless he gets capital which he can't do unless he gets revenue and fundraising.

112:05 >> Oracle >> Oracle is going to make so much [ __ ] money if you believe if you believe OpenAI is successful. But if you think Open AI is going to be successful enough to pay $300 billion dollars to them, how many users do they have and what's that IP worth? Like maybe and also like you know there's reasons you shouldn't own OpenAI, like the Microsoft stuff and like the risks around Enthropic and all these things, but like you know >> in most worlds where open where Oracle gets paid $300 billion by OpenAI, OpenAI is like a$10 trillion or $5 trillion company or something crazy. We'll end with the the OGs, the the old last generation best two business models.

112:40 First being Meta. >> I think Meta's got the cards to potentially like own it all. I don't know if you've seen these new glasses that they came out with the screen. >> Yeah. >> As we go through the history of computing, you have, you know, initially it was like ter it was like punch cards programming. Then it was like DOSS terminals, right? And then it was like then it was like, oh, you have gooies and mouses and keyboards. Then you had touch. And the next paradigm, a human computer interface is we don't actually have to touch it at all. We tell the AI what we want and the AI will translate that into reality, right? Whether it's, hey, send an email to this person, send a text to this person. That's basic stuff that you can already do that with Siri or whatever, right? But like, oh, go buy this. We're so close to all of these things. The input method into a computer changing entirely. And the only company in the world who has the full stack from good hardware that is a you know what what Meta just showed with their glasses with the screen plus the good models um plus the capacity to serve them plus the uh knowledge and knowhow around recommendation systems to know what content to put in front of the user. Um because it is ch it's not just generating the content. It's not just interpreting the user's word and taking actions. It's also putting the right content in front of the user. It's all four of these that you need to put in front of the user >> plus the capital.

113:56 >> Plus the capital. Um and I think I think Meta is so close to being the only company that can do that. >> U there's a lot of risks there too, right? So I like Meta a lot. >> Google to finish it off. >> I it was pretty bearish Google like two years ago, but I'm I'm like super bullish Google. >> Why would change? >> They're waking up on every front front. You know, they're taking the TPUs, they're selling them externally. They're taking the um their models and they're actually like competitive on them and they're training much better and better and better. Um they're being aggressive on infrastructure investments. Um there's still a lot of dysfunction throughout the company, you know, but you know, they do have the hardware business that they can pivot into this.

114:31 They won't be as head as Meta is. Um they won't be as good as Apple is, but like they they they do have Android. Um they do have YouTube. They do have like all these IPs. They have search that can come together when we turn to that next interface of consumer. But also they can also dominate the professional sense too potentially. Whereas Meta I don't think can dominate that professional sense um only the consumer sense and I think Google's well positioned to go capture both markets >> or a meaningful share of both.

114:58 >> I feel like we've covered like an incredible amount of ground. Is there anything that we haven't talked about that you feel is like really critical to what happens in the future that we didn't cover? I think the question, you know, sort of of everyone that I constantly get asked is like, okay, Dylan, you know, you're lucky your obsession is that you loved hardware and you like followed it and you followed the supply chain and and you built this business on it, but like you really like you don't follow the software side nearly as much and all the value is going to get created there, right? When is that going to when is that flip coin going to flip over? Um, but I think the thing that most people don't realize, software is not the same as it was 5 10 years ago. you've had dramatic changes in software and the business model is going to change as well. Right? If we go back like 5 years, three years, whatever when SAS was the darling November 21, I remember SAS started tanking and um at the time it was like it was like mostly like they were over earning and all these other things doesn't matter. The interesting thing about the business model is that it was it is such a good business model when your R&D is sort of this it stays flat, right? And you grow a little bit but really R&D doesn't flex that much. Your cogs are super low. Um you're you know the flip side is in a SAS business your customer acquisition cost is quite high. Yeah. And so when you look at what like what like certain companies have done when they've acquired a business is they've just crushed the customer cost acquisition cost or crust sa they made the business amazing whether it's like Broadcom with VMware and stuff. It's not really customer acquisition. They just had a bunch of wasted SGA but like this SGA this customer acquisition that was most of your cost. R&D was small but not like crazy. And then once you hit critical mass, you just you just cash cash money money. But software changes a lot when the cost to build that software that you have tanks like crazy. You look at non US markets and the prevalence of SAS, it's very different. I will bring up China as an example and a counterpoint. China doesn't have that much of a SAS business. Actually, their cloud business is pretty small too, right? relatively to the US not despite them like importing tons of CPUs and storage historically right there most people just did stuff on prem and design their own software because the cost of developing software in China was so much less than America >> that the SAS business model didn't work as well people could just build rather than rent it out and buy and that creates inefficiency in the market I'm sure those weren't the best of breed solutions always anyways that's what the software development cost may be like you know it was like software developers in 2015 in China were getting paid maybe fifth of the US and they were maybe twice as good or something like that. So 10x lower cost of software. I'm I'm making up numbers, right? Um you know they had 10x lower cost of software and so SAS never happened. Cloud never happened and and at least as big of a way as it did in the US and around the world for all the companies that use that sort of that have that same economic reality and that's despite the outsourcing right to India and and and Eastern Europe and South South America, etc. Um you you you changed all of this with AI software development, right? um and AI SAS products generally, right?

117:57 Not just AI software development. So there's two sort of coins here. So AI software development tanks the cost of building a competing software stack. Do you now move to a world where X can just build I can just build instead of buying renting? Two is if you are a SAS business and your customer acquisition cost remains the same and most businesses in AI and in SAS are going to remain having a high customer acquisition cost. Sales is hard. Uh breaking into a competency is hard. But now you add this AI part of it, you've now added a humongous cogs, right? Your cost of goods sold in any AI software is really hard and really big. And this is partially why I think Google also has an advantage. They have the lowest cost of goods sold for any token of any company because they have their own vertical stack on TPUs. Anyways, coming back to this because you have this high customer acquisition cost and you have this high uh cogs and then the cost of anyone developing it themselves or competitors in the market means you're going to have a very fragmented SAS market or they're just going to build it themselves and therefore you never hit the escape velocity where your customer acquisition cost and your R&D get and get amortized and because you have such a high COGS your amortization point means your gross your net profitability is actually much worse and so I think like the era of like software only businesses is really really tough in the age of AI now already scaled businesses can do great right I think YouTube is going to have its glory days and I'm sure it'll it'll always be amazing but with with with the cost of generation of content falling and falling creating content he who controls the platform is going to win and win and win and win but like >> you know there's like the functionality you build within Salesforce is actually going to be like way less like what you can build on your own like or like you know there's there's or like whatever it is. I'm not saying it's a take on Salesforce itself specifically, but I think many software businesses will have a reckoning with the fact that their COGS is going to sore, their customer acquisition cost isn't going to fall and they have a lot more competitors and so then they don't hit that escape velocity. And I think that's the >> the thing that uh maybe software um it's something I've I've like sort of like thought about. There's a couple people in my company Doug Douglas Olaflin and he's the one whose idea this actually is.

120:07 >> This has been incredibly fun. I I love love learning from you and listening to you and reading what you put out. I think I think you're just one of the most um energetic and awesome thinkers in this whole space right now. So, thank you for all the work you've done. When I do these, I ask the same traditional closing question. What's the kindest thing that anyone's ever done for you? >> Done for me? H I mean, it have to be my brother. Everything he's done in my life. Um I've been a [ __ ] my whole life and I still am a [ __ ] Um, and so like every time he like pulls me back on path, he corrects me. He loves me unconditionally. I think my brother is probably the most he's done the kindest things for me, right? And I've been an [ __ ] like so much of my life, right?

120:46 Like unconsiderate and like everything, right? He's just always been there for me and always been >> Why were you an [ __ ] >> Why? >> Yeah. If you're aware of it, it makes it into >> No. No. It's terrible. Yeah. And maybe this is like the MMO of like who I am and maybe that's why I'm like a good thinker, but like I I like vibe really hard and I'm in the moment really hard and I digest tons of information, but I'm very like bad at like um like how would I say like task orientation, remembering to do specific things. Like I'm very bad at those things. And thankfully I've like been able to surround myself in my life whether it's through birth or not. um with people who help me with the things I'm bad at >> because I'm very bad at a lot of things like I think like you know as far as like you know radar plot of like how good I'm at things and so when I don't like >> call people or like think be considered of what they're thinking because I'm just vibing and I'm doing whatever you know I'm like like kind of like focused in on like this path and like that path ends up hurting someone else right whether it's like hey I didn't I didn't call someone or I didn't like think about their feelings when I did an action or when I said something that makes be an [ __ ] right? And yes, I should be more conscious of this and I try to be, but it's like it's just one of the things I'm going to wrestle with in my life forever. And a lot of times I don't even realize I'm being a freaking idiot >> until my brother's like, "You're a freaking idiot."

122:04 >> God for your brother. >> And so like I I you know, that's the kindest thing anyone's ever uh done for me is like my brother through my whole life. >> I love it. I love it. Wonderful place to close. Thanks so much for your time. >> Thank you so much. Yeah. [Music]