Ep8. AI Models, Data Scaling, Enterprise & Personal AI | BG2 with Bill Gurley & Brad Gerstner

0:00 Elon is building a much much bigger cluster to train a much much bigger model as is open AI as is Zuckerberg. I mean What Sam just said that bigger models aren't the Well, I mean he may be doing the same game that everybody else is doing and trying to throw everybody off the scent.

0:30 Great to have you guys. What a week. It's been nuts. There's so much to talk about and we have our good buddy Sunny. How you doing Sunny? Sundeep Madra in the house. Sunny somebody Bill and I go to often when we're talking through all things AI. Currently at Grok working on the inference cloud. So you're deep in thinking about all these things all these models AI and we're going to talk a lot about models today with the release of of Lama 3. So it's good to have you Sunny.

0:57 Good to be here. Thanks guys. Bill, why are you in town? A board meetings couple board meetings. Good to have you. I always like doing this in person. Yeah, I know you do. Okay, so let's so let's roll. favorite episodes when you do them live. Okay. There we go. There we go. So models models models models. If AI is the next big thing then this felt like another really important week. I mean we got models being dropped by meta with Lama 3. That was the one that was really you know the category 5 earthquake.

1:27 Microsoft Snowflake everybody seems to be out with a new model but let's start with Zuck. Huge Lama 3 unveiling three distinct models and 8 billion and 70 billion and a 405 billion parameter model which is still training and still learning they're telling us which is pretty fascinating. But what seems to have you know shocked the market is that meta could pack so much intelligence into such a small model. And so both models quickly shot up the rankings this week. We have some you know, a screenshot here of that. Of course, the four 405 is still training and there there have been some hints out of the a recent podcast with Zuck and Dwarkesh about it may in fact kind of come in at the top of the polls. We'll see. It's probably going to train for another couple months. But I'd love to hear from both of you guys. What were your big takeaways from the launch of Llama 3?

2:20 And maybe start with you, Sunny. Walk us through kind of just the what and the how of model 3 and why it really kind of shook things up. Yeah, I would say you know, the biggest impact of Llama 3 is its capabilities and at the size and what you know, Zuck shared in that interview was that they basically took the model and kept training it past the Chinchilla point. And so really by doing that, which is generally considered like sort of the point of diminishing returns, they were able to pack much more information and much more capability into this model with the same data set. So so so just for for for everybody listening, so the Chinchilla point if I understand it correctly, right? That it's the byproduct of this paper out of Google which basically talked about the relationship between the optimal amount of data to use for an for a certain amount of compute. Yep.

3:12 But in the case of Meta, when they were training Llama 3, they were basically continued with these forward passes of the data. So they were curating the data, refining the data, pushing it back into the model and I think several people who are working on pre-training at Meta said they were even surprised that it was still learning when they took it offline on that data. Yeah, and they only took it offline to reallocate the resources to you know, 405 and other efforts. And I think he said Llama 4.

3:42 And Llama 4. Right. So the rate of innovation certainly not slowing down there. So a 15 trillion parameter model 15 trillion tokens used to yeah. 15 trillion tokens used to train you know the model. I know at Grok you guys are deploying Llama 3. I think you deployed it the same day that it came out. so, how important is this? How important a development is it in the world of models? Well, really, you know, Zuck came out and threw down for the entire world folks that are building models.

4:16 And it's it's really disruptive because when you look at the rankings, you have a model that's much smaller, so much easier to run on all different types of hardware. and much faster. And so, those two things are like catnip for developers. And for us, we saw within the first 48 hours it become the most power most popular model that we run on Grok. And so, really Replacing what? Replacing Mistral 8x7 for us. Interesting. Which was, you know, generally considered the best open-source model at that point. And what the capabilities have happened beyond us sort of running it, the developers that use it, the use cases we've seen it in are incredible.

4:56 And people are doing a direct replacement with with OpenAI across the board. They come to us, so they come to, you know, all the different providers, and they replace out OpenAI, and they don't really see any performance impact or any reasoning impact or which is quite amazing. What what's what is being optimized in the switch? Price performance, right? You get probably on from a GPT-4 you're more than 10 times cheaper, right? And you're 10? Yeah, 10 times cheaper.

5:27 and well, let me just tell you, GPT-4 is $10 per million tokens input and $30 per million token output. And Llama 3 70B is 60 cents for million tokens input and 70 cents per million tokens output. this seems to be playing right into your thesis around kind of just these models generally. Commoditization. Yeah, you you've been you've been skeptical about the amount of of dollars it's taking to train some of these venture-backed models and and the business models that would come out the other side. Now we have a business in Meta, right, that just announced they're going to spend 40 billion dollars this year on CapEx, that just trained a model that is 10x less expensive than, you know, the most performant model on the market. I mean, what does this mean for everybody else? Well, there's a couple couple things that I put into the into the mix as I analyze this and answer your question.

6:30 You know, first, he made Meta AI free and he didn't he didn't proclaim that this was temporary or that he might pull it up later and so, you know, that combined with I think Perplexity claiming they're going to have an ad portion and and OpenAI hinting at that, at least for the time being, I think the $20 concept is gone. And that was a big part of OpenAI's revenue, apparently, as we or we believe rumored to be.

6:59 50% of their revenue is I think from consumer. and so, that's gone. As long as Meta's free, I don't think anyone pays the $20. And and and I will say that is as things sit today. If some crazy feature comes along, you know, we've talked about personal memory, maybe that comes back, but for now, it feels dead. And then and then I thought the podcast that you mentioned was just incredibly like disclosive, transparent, thoughtful, like like You're talking about Zuckerberg on Dwarkesh.

7:35 Yes. Yes, I thought it was incredible. And and it and it's funny cuz it came out at the exact same time that Sam and Brad were on Harry Stebbings in 20 BC and and Dario was on Ezra Klein and I would encourage people to listen to all three of them. Right. But but Dario and and Sam talked in these high-level platitudes about how this stuff's going to cure cancer and we're all going to not have to work anymore. And and Zuck was down in the weeds in the meat He's in the arena. Yeah, being super transparent and I was just like, holy maybe this guy's in charge now.

8:10 Right. Right. You know, and I mean, I saw a lot of people on on Twitter saying this was checkmate on all of these closed models that have gotten started and venture backed over the course of last year. And and certainly you know, I'm not cheering for that. I'm not sure that I'm I would go so far as to declare it, but I think if you're in the business of producing a closed model, right? We've talked a lot on on this pod. There are one of two ways that you can build a business. You either have to sell it to a consumer for 20 bucks a month or advertising. You have to get a billion consumers to use your product or you have to sell it to an enterprise.

8:43 Somebody has to pay you. And now you have a disruptor coming along and saying it's open, it's cheap or free, and I'm not going to charge for it. And that's hard to compete with. and I use them all cuz I'm curious and I I love playing I like I'll I'll do the same query on four of them. But right now, it's hard to believe that any of them, including the Google one, like are going to have escape velocity cuz the differentiation. I'm not seeing I maybe you guys are. I'm not seeing an element of differentiation on the consumer-facing tool that's radically different that's going to cause 80% market share.

9:21 happening on the infrastructure side. And there's another thing that, you know, Zuck said in that, which was I think a big throw down, which is we're going to spend a hundred billion dollars on this. Yeah. But if if the community, cuz he's made it open, makes it 10% better, right? In in some parameter, yeah. that's $10 billion savings for us. Right. If you just look at what they did this week, they pushed AI search across their entire family of applications. They have 3 billion people using those every day.

9:51 He said on his earnings call this week, they already have tens of millions of people running AI searches on their applications. was even honest in the podcast where he's like, "I don't know if this is where people want to do these searches." I just earned so much credibility with me when someone kind of, you know, comes clean that way. one of the other interesting developments, again, just getting back if if I had one big take takeaway, it was the this this large versus small, okay? And so, I think we we have Before you go there, I got a question for you. Yeah. Could the CAPEX thing be a throwdown? Like, could it be a a signal to the rest of the community, this is where we're going to be? Like, just to Well, I mean, listen, it's it's Could it be a a move to tell everyone else, if you want to stay in this game, you got to play it this way?

10:42 It's also two birds with one stone, right? I think there is that, but there's the second one, which is, you know, also talked about in another set of tweets this week, by putting more more effort and resources towards the training, they reduce the inference costs. Yeah, right, for sure. And and for them, you can imagine everything that you're talking about, making it free everywhere, putting it inside all the products, millions of people using it, that's a huge impact for them.

11:06 I would remind people, you know, something we talked about several episodes ago, but there was a there was a podcast I listened to where Amazon was talking about their Alexa product, and they said, you know, inference was way more of the cost than than the training. Like, night night and day. And so, there's a real-world application that's been alive, and and if that's Do you believe that's true for almost all development projects? Oh, yeah. Yeah.

11:37 Right. And so is is is weird to you know, to your question, is capital a signal? Of course it is. Yeah. I mean in 2020 we talked about capital as as as being the kingmaker, like who would who who would win the game? I think there are four important ingredients to compete in this market. Number one, you have to have capital and and the leaders are spending 40 billion a year. There aren't many sovereigns on the planet that can afford to spend 40 billion dollars a year. Second, what you need we just the big you know, innovation this week is that data is scaling in the way that compute is scaling. So you need a lot of data. They have a massive amount of data. Third, you need compute.

12:16 That's not just about capital. You have to have people who know how to build and stand up infrastructure. You have to have relationships with the entire supply chain, right? And fourth, you have to have distribution, right? And so they're touching 3 billion consumers. They have a business model. I think he said 50% of the content on Instagram was AI-generated in the quarter, right? So it not AI-generated, it was AI-suggested. Suggested. Right? So it was no longer about your friends looking at something.

12:45 It was something that you know, they did. So that gives them a huge advantage. But I do think there's you know, when I get back to this bifurcation and why I think this is important. We have these smaller models that are going to be specialized and have specialized use cases. you know, Microsoft is out with Phi and I want you to talk a little bit about that, Sunny. But we don't see any slowing down on the push to bigger models as well. So we we really see both of these things happening simultaneously. An an analog I was discussing with our team was if you think about different use cases. You might build a small rocket to get a satellite into space.

13:25 And you might build a big rocket to try to get to Mars. Okay? Now, those are both rockets, but they have radically different use cases and radically different cost structures. And I think that the cost structures that are going to be associated with frontier level models, there are going to be very few companies on the planet that are going to be able to build those models because I think the latest discussions, whether it's you know, Stargate out of Microsoft, 100,000 GPU cluster, Elon's talking about 100,000 GPU cluster, Marcus talking about that. I just don't know many companies that are going to be able to compete with that. Yeah, I'll take maybe just a slightly tangential view to that, which is if you think about you know, Meta's history in open source, open compute project, PyTorch, React JS, these are just infrastructure components for them.

14:15 Right. Right? And they put the investment in so that they can drive improvements in the supply chain, they can drive the ecosystem to make it better. And I think they've really taken that approach with this technology and said, "Hey, this is a infrastructure level component that, you know, we want the ecosystem to make better." And everyone else is in the business of models, whether you're a hyperscaler or whether, you know, you're one of these model companies. And I think that's a distinctly different approach for them that puts them at an advantage.

14:44 Well, by the way, I I think this is worth drilling in on. So, in unless one of you correct me, they are not in the cloud hosting business and remain not in the cloud You're talking about Meta. Meta. Correct. And they're so so the people that they're up against have businesses they're running based on these things. They're developing this thing, spending some number of billions Yes. and putting it out as open source. I think it's a little different than open compute part Okay. where where I don't think they felt the differentiation of their architecture had any impact on the strategic execution of their company. Almost the opposite, like it's a commodity, so let's exploit it like a commodity. Here, this feels more kind of like a badass throwdown.

15:36 Where where there's a very intentional element of burning, you know, the strategic ground out there for everyone. You know, it similar I think to what maybe Google did with Android when it came out. Like just protect protect all around me by making it very hard to have differentiated products built on AI that you might come after me with. one of the I think it's fair I think I think it's fair, but I I I think I just want to make a couple points that he that that Mark Zuckerberg talked about on the on the on the DWARKISH podcast. One was he said that they do in fact have revenue sharing relationships with the hyperscalers such that when they use their models, they ought to get compensated something for that. Now, he didn't go he I think he said it wasn't a very big number, but relative to 165 million in revenue, nothing is a big Any color on that?

16:31 You know, nothing is a big number. You know, the hyperscalers have definitely been squeezing all the model makers, right? And they're they have a really interesting position because you know, especially the ones that are creating their own cuz they have to create a marketplace and they have to ensure that they're operating sort of in a free market capacity, but it's difficult, right? When you're when you have your own models cuz there's obviously a lot of interest to I can I can definitely confirm that the data clouds are paying a revenue share to the open models. I don't know what the revenue share is, but there will be some compensation. And listen, that compensation can change over time.

17:08 So, that's one bit of it. The second thing is super important for all of us to to to listen to this again. Zuck said, "We believe in open source, but there may come a time where we have a discovery in our largest model, perhaps, that is fundamental and economic to our business, where we will elect to no longer open source said model." So, you can see a world where they will always open source the 7B, or you know, he said he wants to build a 1B, or a 500 million parameter model, or the 70B, but you can also see a world where their most sophisticated model is not open source, because he says, "Listen, I want to build the best personal AI in the world. It's central to what our business is about. We You we want to have the advantage associated with that." So, I think the strategy for me, it feels like the reason that earth shook this week is that this felt like the most significant development and disruptive element in the model marketplace. I think it's going to be very difficult for new entrants to be venture-backed, because to to to, you know, OpenAI will continue to get funding, because they have this incredible team. They have 100 million people using the product and paying them for the product. But, I think for all the other closed models, they're going to have trouble getting follow-on financing, and I think any new models that come along, you would have to have something so different, such as orthogonal angle of attack, in order to get funding. So, I think to your point, by throwing down on the CapEx that you're going to spend, you are clearing the market of potential competitors.

18:48 Right? It's a very quickly depreciating asset. Boy, I mean, these That's That's this just so like unbelievable is the steepness of the price curve on a slightly older model. Like, and if people are maximizing ROI on an inference basis, they're going to use They're going to take advantage of that like crazy. I mean, it's going to took Llama 2 out of Grok Cloud. It's not even available. We just took it out and replaced it with Llama 3 and all the developers went to Llama 3. But it's already Llama 3 is already Yeah. one 20th one whatever. Yeah. So, there's no reason. Well, I mean, it seems to me where the value is again coming back to maybe maybe maybe we'll switch to you know, this is the right transition to talk about enterprise AI. Yeah. Because the value is not in the model.

19:45 Right? Just like the value is not in storage. Right? You could say storage is a part of the AWS cloud, but that There's not a lot of value in that thing unto itself. The value is in the enterprise relationship. The value is in right the the the the the number of services that you're offering to your customers. So, Microsoft and Google are out tonight. Both clouds accelerated their growth on the back of AI. 64% of Fortune 500 customers are now are now Azure OpenAI customers, which I thought was pretty extraordinary.

20:18 Big numbers. GitHub Copilot growing 35% quarter over quarter. and the number of use cases seem absolutely wild. And what's even crazier is Satya said on the call, the revenue growth would be even higher, but they're GPU constrained. Yes, you heard me say it, Bill. They're GPU constrained. We'll come back to that. So, I look at this in you know, GCP is accelerating. Azure's accelerating. My assumption is you We heard it out of ServiceNow. Their demand you know, is accelerating. So, clearly enterprises are finding value in use you know, in this. So, Sunny, talk to us a little bit about what you're seeing. I know you have 100,000 developers in the long tail now using or I think a lot of big enterprises as well using using the Grok Cloud, but what are these enterprise use cases and are you surprised when you see these hyperscalers racking up these numbers?

21:14 I'm not surprised and let me level up the question for a quick second and to like where is that spend coming from? And right now and this is you know even verified by this report that Andreessen Horowitz put out a couple weeks ago around enterprise AI and what they really showed is like the distribution of use is coming from IT to the business units to support and it's not in these innovation arms because when usually you see these technologies when they're there you understand the budgets are limited. So that's awesome. They also just as a relative point they showed folks are tripling their AI spend this year.

21:48 Right? And so that that kind of lines up to what we're seeing there. Right and we'll show these slides. show these charts there and you know I think the most interesting thing and I'll get into the use cases that 82% of the respondents said they are or either already on open source or will move to open source. So that's the interesting fact that's happening there. The use cases really and you know let's maybe we got a little bit of alpha from Michael Dell a couple weeks ago when you know he really talked to us about this use case for enterprise rag. Yeah.

22:21 Right? Where there's all this data and I want to be able to reason over that data with a model. Right? And so you know his interest obviously with what he's selling alongside you know his partners, but I think in the cloud you're seeing that heavily happen right now. Customer support is number two. I know you guys just financed a company in this space so congratulations on that deal which is really interesting. And then I think content moderation and content generation. I think we don't really talk about it enough, but if you think about a business this is happening all over the place all the time and we see a ton of use cases still there where whether it's a daily report or whether it's you know something you send out to your customers and all of that coming out of those enterprise systems and being sent out. I mean Bill, do you remember, I mean 18 months ago, pre-chat GPT, I imagine less than 5% of enterprises in this, you know, were building AI production use cases.

23:16 Today, I don't know an enterprise that's not at least running a test use case. What you said, what was the percentage? There wasn't 100. No, no, but then that was 64% using Azure. Azure. But but I think it's probably close to 100%. I can't imagine a company in the S&P 500 that's not at least testing AI, right? You would really have to be asleep. Do you remember any other technologies that went from zero to ubiquity this fast? I mean maybe the internet itself. People said, "Oh my god, I got to get on the internet." Right. I think mobile.

23:50 Although this one But but I don't think that's a secret. We've talked about how the incumbents moved very quickly here and I think you can give open AI a lot of credit because they were out Absolutely. selling the mission and out talking to the customer base and and doing everything they could to promote It's also easy to use. Like when you talked about some of these other technologies, like going to cloud was like a real effort.

24:14 Right. Right? Using AI had to migrate your entire database. Exactly. You had to do a lot of real work. This is an API call. Right. Right. Like and again, credit to open AI, they're the ones that led everyone down that path. And everyone else now is open AI compatible or has a similar looking API. It's very easy to use. And part of the reason, you know, one of the things, you know, you mentioned Michael Dell, he tweeted the other day this Barclays survey that I thought was really fascinating. So this is among enterprise CIOs, moving back to hybrid and on-prem. The number was that 83% of respondents said that they were going to repatriate at least some of their workloads, right, back to on-prem, and that was up from 49% or 43% in 2020. Right? And so, I think it's an interesting case that they're you're moving back. My sense is it's because they don't trust certain data in the cloud, right? So, they want they don't want to run maybe code generation, you know, tools in the cloud. And the other one is just data gravity.

25:20 Maybe they have on-prem databases, and they don't want the cost and the headache associated with moving that to the cloud. Do you see this in other parts of of your world, Sunny? Yeah, you know, definitely I'll add a third one, which is I think the there's still a lack of trust, and this gets expanded every time, you know, we had that interview with the OpenAI CTO, where they asked her, "Hey, have you trained this on, you know, data?" And she didn't answer the question well. And so, I think and I've heard this, you know, in conversations with hyperscalers, where customers will not trust and hyperscaler will legally sign that will they will not train, they still will not trust them. Right. they just they believe that all these stories around the data make these models better, that everyone is just wants a way to get access to that data to make the models better. So, I think the combination of those three factors is 100% what we see. And so, what what happens, you know, with us, which is just, you know, basically, maybe a pattern, people come and try something in the cloud, make sure that it works, and then immediately want to get on the phone with you and say, "Hey, can I can I get this on-prem?"

26:20 Interesting. Interesting. sequestered. Or sequestered, yeah. Like I use on-prem as like some I think I thought there were two things in the the these podcasts that we keep referencing that relate to the enterprise decision-making. one, you know, Zuck said something that kind of makes sense to me. Just said like, you know, cramming data in the context window feels a little hacky. Or he I don't know what his exact words were. And so, I think there's still this this future in front of us to data gets deeper integrated in the model and and the trust issues there and we don't quite know how that's all going to come together. it's still TBD.

27:02 Yes. You guys probably haven't tried it because it's it's not, you know, feasible if you're not a developer, but using like a million you know, contact like a contact window of like a million tokens, it's like really hard. Yes. You can't use it up, you're saying. Well, you can use it up, but the amount of gathering and work you have to do to get a million to you know, think about it. It's like several books and you know, and so it you know, people talk about it like it's this wonderful thing, but it's not it's not you know, overly usable. And then the other one, I thought the most interesting thing out of the the the Sam podcast was he he talked about whether or not developers were kind of going wholesale on top of Open AI or whether they were just using it in a lightweight way and then doing a bunch of stuff externally. and he implied that most people are doing the latter, but then he said if you do that, we're going to steamroll you and you need to bet on us being successful. And which would mean dumping your data and and trusting Open AI more fully. I I don't know, what was your interpretation of what he was trying to say?

28:10 exactly that plus the following. If you if you take his, you know, take him for what he was saying, which is the models are going to get better. Well, what room does it leave for anything else? Because if you shouldn't be taking a model and wrapping it with your own, you know, your own code or your own technology or framework, and then you're going to assume the model gets better. Well, why do you need what why do you need whatever I'm building if the model can just do everything?

28:32 I actually I actually thought Sam and Brad were really articulate on this point, whether you believe them or not. And I think it was consistent with the tweet that Aaron Levie sent out yesterday, which is people are not thinking ambitiously enough as to where these things are going. And you know, today we're really in the land of answers, right? We're running some rag over some, you know, HR data that we have in our company and building a little chatbot so it can answer questions more efficiently than my HR group can can answer questions. But they're saying it really needs to think about agentic thinking. Like, what is that multi-step reasoning that can be done in the business? And you know, I know whatever How big is your HR group?

29:15 Well, here it's pretty easy to do. Here it's pretty easy to do. But you know, so my sense is that I'm kind of in this Aaron Levie camp that when you look out two or three years, I mean, listen, every week we're blown away by, you know, how these models are progressing. It's hard for me to think in three years at the rate of progress and the amount of investment that's going into this that we're not going to be a a lot further down the path in terms of this in terms of this reasoning. And when we get there, I think people are going to want that to be more proprietary because I think the advantages that are going to accrue to the enterprise are are are even more.

29:54 Let me throw one other thing in here. You know, I was sitting with my team this week and we're trying to figure out who are the winners and losers, not of the providers of the arms, but the buyers of the arms, okay? So, if every Fortune 500 company is buying AI, one of the things that Bill often reminds me is, "Fine, it will give a little improvement to an airline that starts using AI, but airlines are a competitive industry and they're just going to compete away all the profits.

30:22 And so, that's a defensive move, right? You don't actually improve the business model because all the earnings get competed away. So, what you want to find is a market leader, somebody who has 70 or 80% of a given market who gets to hang on, right, to all of the Or compounder lead. Right, or compounder lead. And so, you know, I there's a company coming coming public in a few weeks called Lineage, which is in the cold storage business. So, they basically are an integral part of the food supply chain.

30:52 You know, any refrigerated storage of, you know, a food, and I think they have a huge percentage of the market. And I think they have 50 data analysts and scientists now in San Francisco because if they can turn the screw a quarter of an inch on spoilage, a quarter of an inch on energy consumption to keep this food cold. And so, it's all to their bottom line, and by the way, it doesn't get competed away, right? And so, they're looking at leveraging, you know, I happen to know cuz because they were a Snowflake customer, and they were using some Snowflake AI, you know, to improve these use cases. And so, I think they're going to be a whole host of businesses, Bill, industrial businesses that capture some of these profits and get to hold on to them. I'm being interested in Sunny's reaction to to your question and and maybe to my answer, which is I when when I am meet a company and see them using AI in a way that feels like ultra compelling from us improvement of their own strategic business position, it's almost always a more traditional AI model that's running a very particular optimization problem. It's not a LLM application.

32:05 and this stuff's all happening simultaneously, you know. I think that I think that I think that's true. I don't think it particularly matters because what generative AI has done, what the ChatGPT moment has done, is it's caused every enterprise to get off their ass to get all their data organized because that's a condition required to benefit from any of this stuff. But then, I think what they do figure out along the way is some basic, you know, machine learning around time series or forecasting or things that have been around for quite a while, Bill, is where they get the most bang for the buck. Maybe not from the generative AI, but they might get there because they got into the pool because they were motivated by generative AI.

32:47 I certainly think it's an accelerant based on everything we're seeing. Yeah, you know, I'll disagree with you. I think, you know, what what this technology really enables is you know, we get spoiled in Silicon Valley cuz we can get the best engineers to build like the most difficult things. But I think for the average business to do most problems, whether it was, you know, pre-generative AI was very, very difficult. Now you can basically take a generative AI model and have it do one of the most advanced things in the world. And you know, we've shared an example in our chat, right? Where you can take a picture of a plate of food and tell it to return to you what's in that food and how many calories might it be and what's the portion size, right?

33:30 That's done sort of again with one prompt. And so now you've given that ability to every business, every small business, right? So like this business you're talking about, they can do a lot of improvements without having to have 50 people in San Francisco. So I think that's where the improvements are really going to come. Although I could push back on you and use your own statement about Open AI and if they achieve everything like Well, they achieve that thing may be commoditized just because Well, someone has to take it and apply it to that business, right? And it may just be the one, you know, the one tech person in that business. It's AI. I was on a walk last weekend with a great economist over at Stanford and we were talking about whether or not about the amount of productivity improvement that would be unleashed into the economy because of AI.

34:17 And what was interesting is, you know, productivity's actually been under assault in this country because we've limited immigration, which was a huge source of productivity, because deglobalization is actually hurting productivity, because we're not moving the productions of goods and services to the lowest cost places. You know, anything that's causing friction. So, he's like, all the goodness to come out of AI, we need it just to replace the headwinds that we have to productivity in other places. But, I digress. Let's move on to the Well, can I add one thing to that, which is I think it just building on the point that you said like Aaron Levie was talking about we're not thinking about it big enough.

34:58 And, you know, where I where I you know, someone shared this on Twitter and I I can't find the original author, but if we can at some point we'll share it, which is in the Industrial Revolution, you saw, you know, car making go from something bespoke, one car per day to a factory making a thousand, same for clothing, same for farms. And, you know, we've looked at technology as this huge accelerant, but we really haven't had the Industrial Revolution for technology. It's still pretty bespoke, you know, one developer writing code.

35:26 And, and now you have this idea where, you know, go back to a place where you spend a lot a lot of time travel search. Yes. Right? You could have one agent do a thousand an agent and a thousand, you know, instances of it do a thousand searches for you and find what you're looking for. We haven't seen that in technology yet. And, I think that's that's the era we're really about to go into, which ties back to, you know, the point that you said you're having on your walk around efficiency for for for society.

35:52 Yeah, I mean, I think I think about it in the context of what we've called business intelligence, right? We've been investors in companies like Tableau, you know, obviously Snowflake, etc. over the years. And, you know, it's not really business intelligence, right? Issuing me a report that tells me how many black t-shirts I sold yesterday, right? Is nice, but it's not all that informative. What you would like is an agent to scour all of your data, compare it to all the data of other companies and say, here's something that is anomalous or we can predict something or suggest something based upon patterns we're seeing in other businesses. That's all, you know, we've been talking about that for a decade, right? I actually think we're getting a lot closer to that moment where now we're going to be able to have these resources because what do these things do really well, Bill? They devour data, they spot patterns, and they predict. Right?

36:47 you said times a thousand. Shouldn't be a single agent. Could be a thousand of them doing it on your data. We we beat up what it's going to do in the enterprise, but you know, one of the areas that I'm even more excited about as these models get smaller is what it's going to mean for personal search and personal AI. so, when we think about that, you know, Google reported tonight they had billions of what they call their SGE searches. So, these are you know, their AI searches. they talked about dramatically driving down the cost of inference of those searches that you can probably tell us a little about. You know, Meta has rolled out search across all of their apps.

37:27 there's a search bar on Facebook, on IG, on WhatsApp. and you can search any topic, right? You can go there and say, "Hey, show me the recipe for fried chicken, or show me how to, you know, play a guitar, or show me where I should stay at a hotel, you know, when I'm visiting Milan." And Zuck did say in his announcement, kind of as a shot across the bow at at at Claude and at Chat GPT that they had the most cap the the most capable free personal assistant, right? You know, that that you could get out there.

38:02 You know, we had Apple announce open ELM, which were these models from 270 million parameters to three three billion parameters. You know, it seems like the next step that everybody's looking at is really these smaller models that can get us to, you know, a personal assistant on device, whether it's on phone, whether it's on glasses, etc. so, when you looked at the announcements this week, you can go to either of you, it felt to me like the disruption caused by Llama 3 was almost more impactful to what we're going to see along the lines of consumer AI and search than it was in the enterprise. Any thoughts about that?

38:43 Yeah, I think it, you know, it ties back to a point we touched on earlier, right? The as we make smaller models more capable, and we make even smaller and smaller models that can maybe reference those larger models, we're onto a place where it becomes more affordable. Right. What we don't really think about, you know, you know, if you think about the larger models as even so crazy as like like a year ago, right? Right. All the way back a year ago.

39:09 a year ago, you're using a, you know, $30,000-plus unit of compute to run this thing with, you know, hundreds of gigabytes of memory. Now, whether you look at the Apple stuff or Phi that came out of Microsoft, you can run that on your phone. People are already running it on their phone. I saw a demo of some folks running it in Apple Vision Pro. Right. No specialized hardware. Right. And the key is, you know, if we're going to run it on the phone, we got to compress all of that intelligence into a smaller and smaller model that's less power consumptive, right?

39:43 If you put one of these larger models on, it burns up the battery, burns up the phone, too much heat, you know, generated by that. Bill, you referenced a quote, you know, Zuck from Dwarkesh where he said, "I don't think in the future we're going to be primarily shoving all these things in the context window to ask more complicated questions. There will be a different stores of memory or different custom models that are more personalized to people." One of the things that I was most intrigued by in that interview was his focus on the personalization to people. He went so far as to say understanding the content around emotions is a different modality unto itself. which got me thinking, you know, not only are they producing smaller models, but they probably have the largest store of human emotions, what reactions to one another, emoticons to one another, you know, biggest certainly social graph on the planet, which seems to put them in a really good position when it comes to this personal assistant that we all talk about. I know you you your view is we're not going to get anywhere close until we get memory, and we haven't solved memory. Well, I mean, he hinted at it, but everyone hints at it. It comes up a lot. And And there's a it and I mean, I push it to Sunny, but it's it's unclear whether you can accomplish what people hope to be achieved in a personal assistant with rag, with fine-tuning, or if you really need a model to be actually, you know, trained on my data. And that latter part, no one knows how to do a fact cost-effectively. Yeah. And so, I I don't know I don't know what pieces have to fall in place for us to get to that place. Yeah. And And it there's no secret. Like, everyone seems to be aware that that's the end goal, but I don't I think there are a few breadcrumbs that were I don't know if you you you know, I'll suggest some of them, maybe you can. So, a few breadcrumbs that were dropped out there both by Apple and by by Zuckerberg in this regard. I mean, I'll just kick it off by saying what he said in that podcast is like, "Listen, in the first instance, what we do is we build software around the model that kind of hacks this stuff together, and we see kind of what works." And so, yes, in the first instance, it may in fact be you have a really small model, you do some rag on it, maybe in certain instances, it communicates with a more sophisticated model, but in that it it you know, in that rag can be a lot of personal information. I think Apple has said the same thing. But then what he importantly said is, if that works, then on the next go around, we figure out how to build that into the model itself.

42:23 Yeah. I think, building on that, if you look at the breadcrumbs from, you know, all the major folks, and I think there was like a a Wired article that came out where Sam said, you know, the next model necessarily won't be bigger. He did say that. I thought that was interesting. Yeah. And I And the reason is, and you know, you had a thing Brad last year at the barn where you had, Brad Lightcap speak, right? And the general message that keeps coming out of the Open AI contingent is that customization and memory. And so, my And I don't have anything beyond this, but I would say my guess is that's what they focused on with GPT-5.

43:03 And by the way, that's an important point. Like, I think in GP GPT-5, it's not going to be the final state, but I think you're going to see the beginnings of memory and the beginnings of actions, right? And this is, you know, months away. And you you and I have a a bet on this, I know. Well, well, yeah. but that could be a major another, you know, tremor in this year. one interpretation of the statement that the models aren't going to get bigger, it one, it could be a mea culpa to the Zach thing, like, okay, like, I don't want to play this game anymore. But two, it could mean that the the LM training has kind of just run its course. And you got to go do this next thing. But the next thing is not a necessarily an exponential leap. It may It may be like a early alpha or beta, and it may be a little more stumbly as you bring think I I see little evidence that that the scaling has run its course. I mean, like, the smartest people on the planet who are putting their own money, real money up against this, Elon is building a much, much bigger cluster to train a much, much bigger model. As is OpenAI, as is Zuckerberg. I mean What Sam just said, that bigger models aren't the problem.

44:17 Well, I mean, he may be doing the same game that everybody else is doing, Bill, and trying to throw everybody off the scent of building a bigger model. Why is he trying to build his own chips, nuclear power plant, and everything else if he's not going to build big models? I mean So, you don't take him at his word. Well, I I'm just saying that I think the world, as I said earlier, is bifurcating into two like a world of specialized models. We are going to have very large frontier models. There will be a point at which you hit the you hit diminishing returns.

44:47 Yann LeCun has said we're going to need a different architecture to get to AGI. He's speculated that it's probably two or three generations more of scaling before we get to that point where it no longer makes economic sense to continue to scale it. But, we're going to I mean, that's a lot of if if it continues apace, that's a lot of developments over the course of the next two, three generations before we hit the upper limits of that. And by the way, I think we're already seeing some creative things like the data scaling that we saw, you know, past the Chinchilla point. Those are really creative innovations to get around or to augment kind of the compute problem.

45:26 so, to me, I come back to this and I you know, it makes me really excited again about where we are in this state of consumer search, you know, and personal assistants. Google's probably innovating better than they ever have because they're pushed out of their monopoly position by everybody else now. it sounds like, you know, they're seeing some great results come out of that. you know, I thought it was really interesting, you know, when you see you know, Dolly and and and David Woodland, who's the product lead on on on Meta glasses, talk about what they announced this week. Now it has meta AI with vision. It's you know, now available to everybody. You know, not only can you use these things to call and to message using WhatsApp, but it has all these integrations and these overlays. So, I mean, we haven't seen this kind of shake up in the world of search and in the world of consumer products in a while. And now is you know, there was this all this noise this week about Humane.

46:29 Yep. Right, the startup up in San Francisco and you know, it got panned in a consumer review. You know, and one of the biggest challenges with that product because I used the product as well, right? Is the models weren't small enough. It doesn't have it can't run the inference on device. So, it has to go out to the cloud to do it. And the second you have to go out to the cloud, it ruins the experience because now you have latency. We're a year away, probably max, from that thing being able to have a billion parameter or 500 million parameter model that basically has all the capability you need it to have.

47:03 Totally agree. And we're also compressing the amount of time that it takes to go out to the cloud cuz we'll get those models to start running faster. So, that we're going to see a a convergence there on two fronts, the local and then the ability for that model to reach out in the cloud and get a faster response out of the cloud. That's what I think, you know, is being underestimated. You know, just swinging back around to model size, right? The smaller models run faster, just naturally.

47:32 Mhm. And so, that gets you to faster responses. And we know the internet's been on a huge push for lower latency across whether it's loading web pages or search results or whatever it is. And so, I think we're starting to see a push in that direction. We all got kind of comfortable with the pace of chat GPT, but if you kind of go away for a second and try one of these smaller models somewhere else and go back to chat GPT, you'll really have that like we all had that moment for a bit between dial-up and high-speed internet where, you know, we maybe had dial-up at home still and high-speed at work. That's the feeling that you get when you switch between those two things. And one of the debates I know, Sunny, you've been having, you know, with our team, and I and I'm firmly, you know, in your camp on is, you know, this idea that dating back 20 years, when it comes to consumer products, even, you know, speed improvements that that are are barely perceptible Yeah. at Google have pretty important implications for their revenue, right? And so, I think what we're seeing with these smaller models and all of these other developments, and you guys are helping, you know, certainly to lead the way at Grok, you're just seeing massive improvements in token per second. And I think, you know, when you start having agents talk to agents, you take who humans out of the loop, right? Now, computers can talk really fast to one another, but we have to have low-cost, fast inference, you know, that's able to support that.

48:55 We do, and and we think of the use cases that we all like, and I I think we all love Perplexity, but you think about, you know, what happens behind the scenes when you type like a small request, it shoots off something into a couple different places, including a search results, pictures, and all those kind of things, and all that has to be processed by the LLM like really quickly. Did you play with the Meta AI picture generator?

49:17 I did. Where you just add another little word or add to do that's What do you think of it? That speed is insane. Yeah. And compared to like a year ago when you were doing it when you'd wait, you know, 15 seconds to get one. the next image. Well, this this gets back to Sunny's point as well. When you're doing that, just think if the cost of inference was really high, there's no way he could roll that out to 3 billion people, right? Because all of a sudden people would start playing with it, and his op ex on, you know, on the business would blow up in his face. Part of the reason he's, I think, pushing toward these smaller models, opening these models, you know, and he said on in the in the podcast interview, they helped me lower the cost of inference.

49:56 you know, we eat the cost of training, so you know, we can we can lower the cost of inference. Well, maybe to you know, just to wrap, we can I wanted to hit on a few topics, Sunny, that we've covered over the course of last few weeks. of course, Bill and I did have been doing a couple deep dives on full self-driving, at Tesla, as well as, you know, their ride-share project that's now moved front and center because of the breakthroughs they've had on FSD.

50:24 and on the Tesla earnings call this week, they Bill answered some of our questions. So, a couple of the questions Bill and I had is is this going to occur within the Tesla app? Well, you can see here, you know, this beautiful depiction, you know, of a ride-share within the Tesla app. There are a lot of Tesla app holders. we had a question as whether or not it was going to be owned and operated or whether it was going to leverage the millions and millions of cars that are out there in the fleet.

50:49 And Elon and I thought elegantly put this, you know, we're going to be both Uber and Airbnb. You know, we're going to, you know, own some of the fleet. We're also going to let, those people who buy cars from us, you know, put their cars into the fleet. My own hunch is that it will also be distributed both 1P and 3P, although he didn't go so far as saying that. and what I mean by that is not only distributed in the Tesla app. My hunch is that as this scales, it'll make sense to do a partnership with with Uber. And frankly, I wouldn't be surprised to see some of the people who operate on the Uber platform become fleet operators of Teslas for Tesla.

51:27 and so I think there's a really interesting opportunity for an integration there. but I thought that was pretty consistent. We we we weren't too far off, in in in in terms of our estimation there. And there and there's I mean, this is kind of I they're obviously have already made it clear they're going to be talking about this for a long time, but this is the kind of first draft, if you will. I think there's a lot to see as this stuff rolls out. I, you know, Waymo's had to apply for these licenses to get these cars on the street. Yes. We don't have Tesla's the one they haven't even applied for those things, but we don't have them driverless on the road yet. Right.

52:07 Which which would be a a step that would need to Right. take place before this was rolled out. But it it it intersects with that really big purchase of you know, H100s that they talked about as well. I you know, we now have a lot of companies that have reported and there's not one of them yet that has not raised their CapEx guidance to buy more to buy more GPUs Well, I mean Elon himself is going to like his in the past four weeks the incremental purchases they've signaled are in in not just Tesla, but there's data on the internet today that x.ai has raised 6 billion.

52:50 Presumably most of that's going into infrastructure as well. And it you know, you and I have a couple bets going, but you know, when it comes to whether or not GPUs are under supplied or over supplied, you you know, what I've stipulated is every supply shortage does ultimately result in a glut. but people have been calling for this glut now for you know, 12 months anyway. and they're calling for it again this year. We're not going to see it again this year. There's, you know, and So you bought the dip? We We own planning.

53:22 And you know, and you and you just see it you know, in fact you know, Meta was down 15 or I think it ended up down 10 or 11%. and one of the major reasons it was down is Zuckerberg said I'm going to put the, you know, the accelerator to the floor. He increased it the midpoint of his CapEx guide by three or four billion dollars. You know, which I said, you know, I had a lot of people inbound to me and say, "Hey, you know, they're no longer being efficient or they're no longer being fit." Which you know, to which I responded by saying, "In two years, that company has gone from 22 billion in net income to 55 billion in net income. They've reduced their head count from 85,000 people to 69,000 people. What they are demonstrating is what you can do when you're efficient. You can redeploy all of that incremental profitability into investing not in some 10-year project that we don't know what the payback is, but directly into GPUs and AI where you can see the payback in a pretty short period of time, leveraging it in their core business." And so, you know, it was you know, while we're on it, you know, Well, he went and bought gym equipment.

54:30 I I did see that, too. I did see that, too. Bill, you and I talked about IPOs. you know, Dan Primack you know, came out with this article that was pretty controversial, I think, among VCs. I saw a lot of people responding to it. it was entitled VCs, You're Blowing It. And there was one line in there that that caught my attention, you know, where he said, "VCs let startups stay private too long, often well past their hyper growth phase that justified sky-high valuations."

55:02 You and I debated this last week. How much revenue do you have to have go to go public? I think you and I are both in the camp that if you have 100 million of trailing revenue, you're growing well, you have great unit economics, you can certainly go public if you price it right. I'm you know, I've said it on Twitter, I say it in boardrooms. I think being in the public markets is a great place for companies to be. I think it you know, it puts them in the big league. It makes them you know, there's plenty of room to innovate there. However, what I would say to Dan is, you know, when we sit on the board, we can advise, but ultimately we're not the decision-maker, right?

55:43 It's got to be a collaboration with the founder of the company, and ultimately I think the company should go public when it's the right time for the company to go public. And for some companies that is at that earlier phase. I think a lot more could go public at that earlier phase, but I also think there certain situations, you know, take SpaceX for example, where I think it's behooved them to stay private longer, right? And they've had plenty of access to the private markets to raise capital. So, I didn't know if you had any reactions to the Dan situation.

56:12 Well, I mean, I think one I think in a lot of ways I agree with what you're saying, and I I disagree with maybe with the way it was positioned, but keep in mind Dan's one of the very few analysts and writers that focuses on LPs. Like, most of these writers focus on VCs or the founders themselves or whatever, and he's he's constantly talking to LPs, and I think there is a a very real situation, especially where we came out of ZIRP, where there's a vast amount of paper marks that are sitting on these LP books that are aging out, that are that are exposed to dilution, you know, on annual basis, and I suspect they're very nervous, and I suspect they're talking to him, and that that's where he's building this thesis.

57:02 And I think that's probably right. I also think that a number of people, you know, that invest in late stage, and people that we know have have built a business model where they kind of like companies staying private longer. They are the ones that everyone talks about Amazon went public at this price, and then the public captured it. They they kind of view their game as capturing that growth instead of the public markets. And the third thing I would just say is our business has gotten nothing but more competitive from the minute I entered it to today and I think that's going to keep happening. And that competition forces people to be very founder-friendly to say what they want them to hear, to support secondaries which we've talked about. that I when you support massive secondaries, you're taking the number one pressure oh out of the system that used to lead founders to to want to go public cuz their employees are Like I need liquidity, I need liquidity. So you you you you you do a release valve and you take that away and and I do think there will be a lot of situation and then the they actually one last thing to mention just because of where we came from evaluation perspective, we we know a lot of people are sitting there afraid that they can't meet their last mark. For sure. And so then you're kind of in never-never land and how do you get out of this and what are the odds you're going to grow back to that? And there's kind of a lack of So anyway, the those dimensions I think he's hitting at it right. I agree with you and this is where I think he got it wrong is that like no single VC's going to stand up and make a company go public. Like that's not going to happen.

58:41 Right. I mean I do see the market evolving. I listen, you got to get you're in you you you you raise 10-year funds. You need to get liquidity. You're in the business to provide returns and liquidity to your partners. if I look at the private equity business, right? They evolved in such a way where they didn't have to take the companies public. They would just sell to another private equity company. And it may very well be in the VC landscape Bill and I see this more and more.

59:06 you know, I know a big company right now raising it over 10 billion and I know a lot of early-stage VCs who are selling into that round. Right? And so that would Well, I thought there was one in a didn't Rippling say 600 million was going to people that are actually Early early So that that's not the one I was referencing, but Rippling may in fact be one of those. And so, the market may be responding to some of these imperatives, you know, by LPs to get liquidity.

59:35 but, what I would say to Dan is certainly with the with the venture capitalist sitting around this table, you have two people who think that the public markets are a great place for companies to innovate, companies to thrive, companies to raise capital, companies to recruit, build build brands, et cetera. And and and we know we're in the business of liquidity. Speaking as an entrepreneur, though, Can I add one thing to that, which is I think we're going to see like a bifurcation in builds on our conversation from earlier. There's going to be companies that are cheaper and cheaper to build and run.

60:05 And you know, that has one impact, but there's companies that are going to get a lot more capital to build and run. And so, that may force folks into the public markets, sort of like the way it was in the late '90s with a lot of those businesses cuz they needed a lot more capital. What do you guys think about that? I I There was a There was a interesting article that Tim O'Reilly wrote about a month ago. I could put the link in there, but he he implied that He also attacked the VCs and said they were going about it all wrong, but he he he implied that that AI was now in its Uber phase, where the you know, we're I think talking about Uber and Lyft and DoorDash all raising billions and it's spilling out on the floor. And I personally think part of it is simply a recognition by the investment community writ large that network effects exist and increasing return exists.

60:57 Yes. And so, when they think this is the next big thing and they see OpenAI take a lead, their gut response is, "Well, if I had invested early in Amazon or Google or whatever, I'd get paid almost no matter what the price." And so, it's the institutionalization of a belief in network effects that's leading to the the money pouring in. And then and then it's a competitive dynamic. Like, once your company raises, you know, 200 million, a billion, if you're in that market, you raise it, too. And it does create chaos. Like, I do think it creates chaos. well, maybe we'll just we'll end with just kind of the volatile week it's been in markets.

61:39 you know, we talked you know, you you you asked me what I thought was going to happen this week. You told me you'd give me a scorecard, but you know, the reason it's so volatile this week is not just because we've had some mixed earnings reports or at least mixed reactions. but, I think the economic backdrop is, you know, is unsettling. GDP came in a lot weaker than expected this morning. At the same time, the PCE for Q1 came in a little bit higher. Now, we got the monthly PCE report coming out tomorrow. We'll see where that shakes out. But, this idea that we could have a slowing economy at the same time that we have inflation continuing to go up, right? This is this very fearful place called stagflation that nobody wants to be in. Now, the market is now pushed out the rate cut forecast to December of this year, right? So, this higher for longer is now in place. Remember when we started the year, we thought we're going to have six rate cuts, very accommodating, Goldilocks environment.

62:38 What's surprising to me, to be perfectly honest, is how well technology stocks have performed, outside probably software, but how well they performed notwithstanding this fact. And the only reason they've been able to do that is the reacceleration caused by AI. So, if you look at what happened tonight, Google beat, they issued a dividend for the first time, they announced a buyback, right? So, they're really their their margins are expanding, so they're finding efficiencies in that business.

63:06 They're listening to the markets. but, you know, I think impressively, you know, what they're doing in that core business, the things they announced around SGE, you got to give that management team a lot of credit. we talked about Meta doing? the stock was up I think 10 or 15% after hours. So, I think it's at an all-time high and and and and doing incredibly well. Meta, you know, missed is down 10% but still up 30% on the year. So, I imagine those those two companies are about in the same area, you know, year-to-date. We talked about why they why they got hammered is because, you know, they're going to invest even more aggressively in AI, which is a long-term investor I'm pretty thrilled about. And then if you look at Azure or Microsoft's quarter it was pretty blockbuster.

63:52 and we'll show this chart by Jamin, but Azure AI contributed 7% of growth this quarter. So, it's now translates into about a 4 billion run rate business, you know, that didn't even get broken out until five quarters ago. you know, so again I I think if you look at technology generally, it's performing it's performing really well despite this kind of volatile economic backdrop. and we'll see what where where PC rolls in on Friday. We'll see where the rest of technology comes in. My hunch is that the largest companies in technology, back to your network effects and your scale advantages I'm not sure that smaller tech technology companies are seeing the benefits that the largest technology companies are.

64:39 Certainly what it looks like. you know, so we'll see that as it reports. But I think the largest data platforms and hyperscalers continue to benefit. Boys, this has been fun. Yep. Great. Thanks to it again. Thanks for being on, Sunny. Until next time. Good night.