Ep5. Stock Comp, AI Cold War, Valuations for LLM | BG2 with Bill Gurley, Brad Gerstner, & Bob Mylod

0:00 We even have a court criticizing Elon for taking a an options package where he made no money unless he saved the company from bankruptcy. Meanwhile, the CEO of his archrival who created no shareholder value over a period of 5 years is making tens of millions of dollars a year in RSUs.

0:32 Hey man, great to see you. Good to see you. Good to good to have you back. Glad to be here. Man, I raced up here from Nvidia GTC. How was that? I mean, it's a you know, I I texted Lincoln yesterday when I was down there for the keynote and he said, you know, I said, "How'd your day go at school?" And he said, "Good." He's like, "Hey Hey Pops, what are you up to?" And I said, I you know, I'm just watching the future. And and he said he asked a great question. He said, "Well, why is this any different than all the other computers in the world?"

1:00 Right? And I said, you know, like we've talked about before, we're going from this era where computers were highly efficient calculators. And I said, you know, we're not far off from it be computers being able to reason and think and being smarter than every human who's ever lived on the planet. So, it's I mean, it's an exciting time, disruptive time. We're going to talk about that more later in the show. But we thought for episode 5 we we we'd mix it up a little bit. I mean, one of the things you and I talked about when we started the pod was not only do we want to talk about the topics of the day, but we also wanted to do some deep dives on topics that you and I have talked about over the years. You know, I think about the topics that you've covered on Above the Crowd, whether it's network effects or whether it's a rake too far. And so, you know, one of those things we're going to talk about today. After that, we're going to get into some of these Google Apple rumors. We're going to talk a little bit more about the TikTok fallout, etc.

1:59 But one of the we've been we've been pretty passionate and talked a lot, argued a lot over the years about stock-based compensation. Yep. And it's so important to Silicon Valley because it's all about incentives and alignment. it's such a special part of what makes this place great, but it also has been a source of a lot of consternation from Warren Buffett down the line. so we we knew we wanted to talk about this, but in order to talk about it, we had to invite one of our old friends.

2:26 Correct. who may be you know the best expert on the topic, the legendary CFO of Priceline, Bob Mylod, who served on tons of VC boards. and he's he's truly got gets this from both the startup perspective and the public company perspective. So Bob, thanks for joining us. Gentlemen, thanks for having me. It's great to be here. Yeah, if if any of my CEOs or CFOs are of the mind that they want to get this right, there's no one on the planet I'd rather, you know, introduce them to and have them talk to them, Bob. Bob, I think you joined Priceline in 1999.

3:05 you know, we must have met around 2000 or 2001. You were the CFO of the company from when I believe it was a a billion-dollar company all the way to it being worth you know, upwards of a hundred billion. Certainly on on the board for that entire period of time. You're still on the board of Booking and you know, it's become extremely valuable in the public market. There was a little bit of a there was a big roller coaster along that way, but that's generally the math and I wasn't CFO the entire time. But yes, that's the general math of it.

3:39 Bill, why don't you frame for us why is this topic so important and importantly a little bit about how we got here with stock-based compensation. Yep. And and I think everyone in Silicon Valley knows that equity participation is part of the model. Always has been, right? And I think in general, when it was started, venture capitalists felt strongly that that it created alignment, right? You wanted managers and shareholders owning the same equity and having the same interest. And if you create one of these monster outsize, you know, total wins, everyone's a winner. And so it just made sense, right? And I I think it's been Since certainly since I got here, but way before I got to Silicon Valley, I think it's been a part of of what people do.

4:27 In the late '90s, you know, we went through the dot-com boom and bust. And prior to that time, I think the primary vehicle that was used to incent executives and employees were stock options, employee stock options. And they would be priced at the current price of the company. And you know, while companies private, you had this, you know, evolving argument of how you value the the options prior to being public. They're typically a discount to the preferred. Like preferred things could could make that argument stand.

5:03 anyway, coming out of 1999, you know, you had a bunch of companies boom bust. You had a bunch of companies go out of business that that people said were never a business. And you had a lot of skepticism coming in from, you know, people like writers for the Wall Street Journal, but even in ISS, Institutional Shareholder Services. You might want to tell people quickly what ISS Yeah, I mean, you know, ISS is is basically, you know, a body that helps advise owners, shareholders of public companies on on how to vote on a variety of topics. Very Including compensation.

5:41 including compensation. Quite controversial. But they have they have a voice. But they have influence. matters in this world. And people also were upset about repricing, backdating, you know, there were options that were backdated, and then regranting. So, the stocks collapse, and then they just issue new options, and people didn't like that. And people began to argue that options created too much risk. And so, there was this idea, you know, that maybe we needed a different instrument. Along that same time, a very large investor in a very prominent internet company, was worried about dilution. He's worried about the dilution from options. And for those that want to just use a rough framework, you know, when a a startup would get started early years, you might have as much as say 10% dilution from options. You might hire three executives during that year, and it's a big grant.

6:35 But then, as you move towards being public, people would typically, and these are very gross rules of thumb, steer towards, you know, a 3% type dilution rate. This particular investor in this very large public internet company felt that if they moved to zero-based RSUs from options, and this gets into a technical detail around Black-Scholes, that, you know, there's an argument from Black-Scholes that a single RSU is worth about two and a half or three options.

7:05 So, if you could switch from these options to an RSU, this investor thought dilution might drop from, say, 4% to 1.2% or something like that. And there would be, quote, less dilution. And so, that change happened at this one company, but then these zero-based RSUs became prevalent everywhere. Okay, so let's let let's just pause there for a second, because I think there's a potential we've lost half of the audience because it's a very technical and detailed topic. So, if I just step back for a second, I want to invite you into the conversation, Bob. The reason, obviously, we care in the case of this large internet company they're granting options. They want employees to be incentivized and aligned with them. They may not have a lot of cash in order to pay cash, but you said something critical. As they get toward the public market, right? There is these things are obviously expensive to the potential shareholders. So, folks who are going to buy it, Fidelity, Wellington, others in the public market, they care and are sensitive to that cost. Bob, you know, Bill's describing the world that existed about the time, right, that you guys were going through this, you know, at Priceline. So, tell us a little bit about, you know, kind of your experience in those early years with options and then, you know, kind of this transition, that we struggle with our issues.

8:33 Yeah, I think Bill does a very good job of describing what the world looked like in venture capital and private equity for that matter. That's what I did for 10 years before I joined Priceline. It was a very simple, straightforward model of investor makes investment at a certain price per share and sets aside an option pool to be granted to management at the same price. There wasn't some There was no exercise or, of you know, contorting oneself into figuring out how to make the strike price lower. It was a simply as investors reap rewards from the outcome of the of the company growing, employees share in those rewards.

9:19 so, by the way, I I became kind of battlefield promoted during the the the dot-com bust. So, I I became CFO of Priceline in the fall of 2000. And this was, you know, Brad, as you mentioned, I joined the company just after it was founded. less than a year or almost exactly on the one year found one year anniversary of the founding, the company went public. And a few months after that, we had a market value of about $35 billion. We were a tiny little company, but this was the absolute heyday Right. of the of the of the dot-com bubble. And then, you know, the the dot-com bubble burst and essentially our stock went to zero. If you go back and look at the stock price run of Priceline, you know, we were all the way down to $5 a share, which is a post reverse split adjusted value. I became CFO at that point, and I wasn't too I wasn't too much worried about stock stock options.

10:20 Right. our stock had just plummeted so much that all of the stock options that were sitting in the hands of whether it was myself or every single employee, they were under underwater. So, the only possible way for us to compensate people at that point was in cash. Which really sowed the seeds for, you know, some of the ultimate successes. When you start to think of your business in the form of you're paying people based upon the cash that you have in your bank, and you have no more ability to raise money, it's very very clarifying about what you can afford to pay people. and that's that was the model that we operated for a couple of years operating Priceline. And then, what I will define as one of the great unintended consequences of an accounting change occurred right around the 2002 to 2004 time frame. And there were two things. First, there was the adoption of Sarbanes-Oxley, where people like me and Jeff Boyd, our the CEO, we had to, you know, sign our names on the financial statements under, you know, penalty of criminal prosecution.

11:29 And there was the adoption of FAS 123, which required that historically any granting of stock options to employees did not come with a charge. The theory historically was it's hard to value these stock options because they really only have value if a stock goes up. And therefore we won't try and do that even though there is such a thing as a Black-Scholes model and the things do have value. Historically a stock option granted resulted in no charge.

12:01 Right. the SEC and the FASB were trying to get to a place where companies had to recognize that granting of a stock option did did represent the transfer of economic value from the employer to the employee and that needed to be accounted for. So FAS 123 came along and every public company had to expense the granting of a stock option. Well, as you can imagine for somebody like me with a hugely volatile being the CFO of a company with a hugely volatile stock and you know, I'm a University of Chicago grad and you know, I studied the Black-Scholes model and had a very clear understanding that there there is a lot of inputs into the calculation of a Black-Scholes calculation that you know, there's some subjectivity to it. And that made CFOs like me very uncomfortable as to signing your name to financial statements with you know, these stock option charges in it. Not to mention the fact obviously that when we're sitting down talking to employees about their compensation and you say to them, "Hey, here is your stock option and by the way, you know, Fischer Black and Myron Scholes says it's worth you know, they say it's worth $100,000 and you're looking at it and saying, "Well, the stock is trading for $20 a share and you're telling me my strike price is $20 per share, so in my mind it's worthless. Right.

13:35 And in fact in fact it is, Bob, unless the stock goes up in value, but the reason it has some intrinsic value according to Black-Scholes is they look at the volatility and other measures related to the stock, and it's effectively a probability weighted, you know, estimation of what this thing is worth today, right? And so Exactly. from from an employee Go ahead, Bill. I I mean I believe, and Bob could correct me cuz I didn't go to University of Chicago, but I believe Black-Scholes was backed into by looking at how you can create it one synthetically with the short and a call option and these kind of things. And and Explain that to an employee. But I understand, but but you said it didn't have value.

14:16 At-the-money call option has value, and they trade on the public markets today. That's my only point. Absolutely. And and and they do have value. and again, which is why for, you know, the dawn of the venture capital era, stock options were granted and they did have value and they created huge value, but again, only in the event that value was created for shareholders. Right. as the 2000s moved along, there was more and more companies, and by the way, this includes Priceline, we moved away from stock option grants to RSU grants.

14:54 And let's let's just explain again to folks what what some of these things are. So, stock option, I get a security, it says the strike price is $20 that you referenced. Our stock today is trading at $20. According to Black-Scholes, that would have some intrinsic value. Today, you would have to recognize an expense that over time, but for me as the employee, I'm just looking at this and saying this is only going to be valuable to me if this stock goes up over time. And in fact, if I quit the company 3 years later and the and it's still at 20 bucks a share, then I don't walk away with anything.

15:29 The move to RSUs, restricted stock units, means that I'm going to own a security and zero basis RSUs, I'm going to own This is just like get somebody handing you you know, a share of stock. And when I leave 3 years later, whatever I vested in that share of stock, I'm I can cash that out and I can walk away with that cash value. So, it it it it it works very differently from an incentive and alignment perspective than options were working. Sorry to interrupt you, Bob, weren't we? Yeah, no, that's exactly right. So, if you give you know, one share of an R of a restricted stock unit at a value of 20, then the recipient has $20 of value.

16:13 Right. now, if the stock happens to go down by 50%, the employee still has $10 of value. Whereas in the in the case of the stock option, if it goes down 1% yet alone 50, it has zero value. And here's where we get into questions of of alignment with shareholders because you could theoretically have a management team that leads a company, you know, over a year to a 20% decline and pick up 80% of the value of their equity award. Right. Right.

16:47 Whereas shareholders are down 20. You as a large public owner of shares, you'd be down 20 and management gets 80% of their equity value. That's right. So, Bob, why did you go from options to RSUs? Well, again, I'll I'll just say that before the adoption of FAS 123, if you were to grant an RSU, the full cost of that RSU would have to be expensed in the P&L in the income statement of a public company. So, treated like cash. Same as cash.

17:16 was treated as and by the way there were some companies that did it but there was it was very clearly understood by all people that that was essentially no different than cash. It would be no different than you know, issuing a share of stock to the public for $20 and then turning around and taking that $20 and handing it to the employee. so that that was always expensed. It was just when the stock option also got expensed that issuers looked at that and said, "Okay, well, if I now have to expense stock options and I always had to expense RSUs I might as well just do RSUs.

17:58 and it was at this time also that companies got into the practice of adding up all the charges related to this stock-based compensation and pro formaing the effects of those out of the income statement so that they could effectively compare current results to what the results would have looked like before FAS 123 was adopted. Okay, so so hold hold hold on a second here. So you shifted from stock options which as a shareholder I like because nobody makes money unless I make money to RSUs where people made money even if I lost right? And then because that was all now in the P&L that you had to expense that like cash which Warren Buffett and many others had been pushing for since '97-'98 but in order to get around that everybody started adjusting what they you know, their their earnings EBITDA. They started adjusting the stock-based compensation expense out of EBITDA. Okay? So as though the expense no longer counted.

19:11 did you start that? I I don't know that I started it, but we did So, we did do the pro forming, but the reason that we did the pro forming wasn't because we didn't view that every single dollar of RSUs that we were handing out had real, very significant value. It was more that we were going through this transition period where some of the compensation we had handed out historically was were stock options.

19:41 And now we were moving to a world of of RSUs. So, there was a multi-year period where some of the stock-based compensation was option-based. Some of the stock-based compensation was RSU-based. So, that would have created even more confusion around, you know, what how to how to think about core earnings. and so we did pro forma them. And, you know, obviously almost all companies pro forma them.

20:12 I think the big difference is is that we as as a company we always viewed $1 of of an RSU as fully being worth $1. In fact, if you really are bullish on your company, which we were, we viewed that a dollar of RSUs was actually worth more than a dollar because we certainly had a view, hopefully, that if we did our jobs that that the stock was going to go up and so that the it's it was our most valuable form of currency that we used to pay employees. and we certainly thought about that all along the way. We never It never occurred to us that that it shouldn't count or that it didn't count. and we always looked at it in terms of the dollars associated with, you know, employee compensation, period. We never looked at it in terms of what dilution what what what are these share grants represent as a percentage of the the total outstanding shares or you know quote unquote dilution.

21:15 I assure you as somebody who sat across the table paying, you know, paying people their compensation, I wouldn't sit across the table at year end and say congratulations we've had a good year, your salary is X and your bonus is Y and we're giving you a stock grant of percentage points of dilution. Percentage points of dilution do not show up in a W-2. percentage points of dilution are not what you pay taxes on.

21:49 People people on the other side of the table are looking at dollars. So we always looked at dollars and I think where some other companies have strayed is they've forgotten that those dollars are dollars and for a tech company often times stock based compensation happens to be the single largest expense component of of a tech company. And yet somehow inexplicably even though it's the largest single expense component of that company it is not considered to be even an expense in some companies and in some boardrooms. And the result of that is that you've seen runaway stock based compensation on an almost uninterrupted basis for the better part of 10 to 15 years. As this happened, exactly what Bob talked about and as it became way more prevalent we were entering that ZIRP period.

22:53 Yes. And so there was spend at all costs. There there wasn't a lot of attention Correct. to profitability or even kind of paying attention on the expense side because that window happened. Coming out of the ZIRP window, people like yourselves are finally raising their hand. I'm not blaming this on you and saying, "Hey, wait a minute. You know, this isn't right." Like not only is there a non-alignment problem, but these numbers are massive. And this this drumbeat has just started to work.

23:23 Right. And you know, like if we boil it way down, right? Because I think we all have benefited from the magic of stock options, right? As an incentive and alignment for all the value creation that's occurred in Silicon Valley over the course of the last 20 years. I think what we're trying to point to, you know, and I want to drive toward is kind of what are those best practices if you're a founder, if you're on the comp committee on a board, if you're an investor.

23:50 And to me, when you don't when you when you pro forma something out, right? When you stop treating it like cash, right? It leads to a misallocation of resources. Because you're effectively saying, "This doesn't exist." And when you you know, what what happens when the cost of something disappears? You get a lot more of it. So, let me give you a a very specific example that I that I heard the other day. Take for example the recent competition for AI engineers in places like OpenAI and Google, okay? Because venture-backed private companies have to compete in an ecosystem against public companies.

24:30 So, there are rumors that lead engineers are offered millions of dollars a year, okay? But instead of cash, for example, Google's offering 500,000 in cash plus a million and a half dollars in RSUs vesting monthly, right? To these engineers. So, every single month those RSUs are immediately available for sale. And then as a shareholder, you have to ask the question, how much of this expense hits the book at the year end of the year?

25:03 Right? If you're adjusting out the cost of those RSUs, then you're pretending that the cost of that engineer is only $500,000. Right? And, you know, that to me is what leads to the spiral in inflation, Bob. You saw this in the early 2000s, and you said, you know, like, when you had to treat it as cash, you got a lot more sober about what everybody was paid. Now, the difference in that period of time is the whole world deflated. Right?

25:34 Like, everybody kind of came down together. But, today, we have public companies that are pro forming this out. They're still spending a lot on these engineers. So, for me, the problem with this is when we it leads to excess, in the public markets, which then misallocates resources away from startups, and I think leads to less innovation. It it it it's we're we're now whatever, we're 20 years since the adoption of FAS 123. And and I especially thought when ZIRP disappeared, that people were going to wake up and say, you know, we're actually not counting profits correctly here.

26:16 Yeah. these tech companies are far less profitable than we actually think they are. I and I've been on I've been involved in so many companies now as a venture investor that have gone public. I've sat on multiple compensation committees of both private and public companies. And, Brad, exactly to your point, I can't tell you the number of conversations I've had with the leadership of a company who, you know, I can think of a specific conversation where CEO called me and said, I'm trying to hire this really talented software engineer.

26:50 but that person wants $400,000 annual salary, which really screws up our compensation here, because that will have that person making $100,000 more per year than anybody else at our company. And what do you think? Like, should we stretch for this person? And, you know, I said, well, what's what's the what's the RSU grant? And the answer was $10 million. Yep. Wow. And I'm sitting here like, wait, I just got a phone call from somebody worrying about the $100,000 salary. And of course, I of course that, you know, the CEO should worry about the $100,000 salary, cuz if he's playing the game where that is the number that counts and the other one doesn't, then they're going to focus on the 100,000. Of course, I'm looking at it going, no, like, you really want to focus on the 10 million.

27:43 Absolutely. And you know, it seems to me like just just back to why I care so much, you know, obviously, when I think about companies getting fit, right? The companies, you know, dealing with the post-COVID hangover, you know, where we had excess hiring, excess pay, you know, they they got out of shape. Like, let's make it very concrete. If a company has 20 to 20 bucks a share in earnings, but has 10% dilution, right? Every year, then my claim on that earnings as a shareholder is going down. Absolutely.

28:21 Right? If they produce $20 in earnings with no dilution, right? They're not issuing stock at all, they're paying it all in cash, then I have a claim on that $20 in earnings. And I think the same thing in venture-backed companies. I think what some people find as shocking is they get to the end of the journey, Bob, they start off owning, you know, you you're a Series A investor, a Series B investor in a company, you think you own a certain percentage of the company.

28:50 But, if you're not paying attention to that 10 million share grant, right? By the end of the time, like, your ownership claim, by the way, founders should care because their ownership claim as founder is going down. But, unfortunately, they've had to compete in this arms race where the public companies are all pretending this isn't an expense. Yeah. Yeah. Yeah. and and look, I think competition's real. Like, I don't want to be seen as this investor type that's saying, "Oh, you know, the owner should keep everything." Like, like, you're you're out there competing for employees and talent every day, and the price has gone up, and you are competing with large companies. Listen to these numbers. You know, big companies have an advantage from my point of view. Microsoft's SBC is 2.8 billion a quarter. So, that's over 11 billion a year. And this morning, we woke up and read about them taking some employees from a hot AI company in the valley. And when you're handing out 11 billion a year in RSUs, you know, you've got a lot to play with.

29:52 Now, that's 11.2 billion happens to only be 10% of free cash flow and 35 basis points of dilution. It's 1/3 of 1% because they're worth what? 3 trillion? Yes. You can't be a big company unless you sort of in my opinion, unless you establish much better hygiene when you're the smaller company or even before you're a public company. And I want to be very clear, like, Booking Holdings, you know, where I'm I'm chair of the board of Booking Holdings, and I'm on the comp committee, and I want to be clear, like, I view that our our employees, I I know that our employees have been very very generously rewarded over, you know, a multi-decade decade period.

30:39 Partly because we live in a competitive marketplace, and we have to make competitive stock grants. But the biggest reason is because our stock has appreciated over that period so rapidly. Has gone from $10 a share to $3,400 a share. Like that that is the appreciation that has driven returns for not only the the employees but driven returns for all the shareholders of booking.com.

31:10 Correct. And at least from our perspective, our culture has been hey, the way to the way to the way to get rich the way the way to make a lot of money is for to create huge shareholder value. And the senior management and the board of directors along the way is going to try to put our thumb in the on the scale in the form of smartly using our capital to further reward shareholders. So Bill, again, like you're exactly right. If you look at the the top certainly the NASDAQ 100 companies but absolutely tech companies below that market cap, what you will generally see is you know, a year-over-year increase in share count.

31:53 Literally the expansion of the number of slices of pie because of these stock granting practices. You'll see some companies that do share buybacks and they'll literally state I'm doing the share buyback to offset dilution associated with employee you know, stock-based compensation. And thereby doing that, taking a huge cash expense in the form of stock buybacks and putting it below the line so that when people look at operating cash flow or free cash flow it looks like one number but yet below the line there's all this money being spent just to buy back the stock that employees selling when their stock, you know, vests.

32:41 We've taken the the view that we want to be very very fair, but again, back to the thumb on the scale, we take our excess cash flow or we've tried to take our excess cash flow and buy our stock back. So, if you were to look at our share count, say 10 years ago, I think we probably had 50 52 million shares outstanding, and today we have 35 million shares outstanding. Meaning Incredible. Meaning that you know, during the last 10 years, if if the value of our company, the if the enterprise value has doubled, the share price has tripled. And the the biggest beneficiaries of that are the people who have gotten the grants that we gave in the prior periods. So, to me, creating that virtuous cycle where you really are counting that stock as an expense, that's the way to do it. And a long, long time ago, we adopted the practice of not pro forma-ing out the cost of stock-based compensation because it's a very, very real cost. Yep. And we do reference, you know, cash flow, and we do reference free cash flow, but in our most recent proxy statement, we actually pointed out that the thing that we really look at is is how much of our free cash flow is being spent on stock-based compensation. And you know, maybe that's a segue, Brad, to you know, how you know, what are the solutions to get this under control, but like we have always at Booking, and certainly with all the companies that I'm involved with, I try to get everybody to focus on the fact that a dollar of stock-based compensation is actually not only should it not be ignored, it's the most single valuable expense.

34:26 You had a chart, and in the chart you compared Booking Holdings, and we'll show this, the percentage of operating cash flow that various tech companies spent on stock-based comp. And so again, I've been on comp committees on public companies. So if I'm on the comp committee of a public company board, generally what I do at the start of every year when I'm negotiating the options or the RSU packages for the employees, we bring in a comp consultant, right? And and this comp consultant makes some recommendations to us based upon the peer groups that look like this particular company.

35:08 Not surprisingly, because everybody has adopted this practice, you know, it's a little bit in out. And so if, you know, if everybody's, you know, following a dumb comp practice, that is your peer group, you know, compare. But tell us a little bit about this chart. And, you know, if I'm we're on a public company comp board, and then I'm going to Bill, I'll ask you the same thing as it relates to venture. But if I'm on that public company comp board, what is my takeaway in terms of what the gold standard is?

35:39 What is this the way that I should be measuring it, or should I be paying attention to Compensia and these other comp consultants that are coming my way? I don't know that there's a gold standard. To me, the gold standard is to just count it. Again, as I said, like you would never think of not counting expenses that are denominated in euros, if you were if you had a European subsidiary. the just the the dollars count. I guess what I'm saying is is like it shouldn't be excluded to begin with.

36:08 Now, one easy way of looking at the practices of of various companies is to look at, okay, what is what is the dollar value of stock-based compensation expressed as a percentage of OCF or FCF. that is a blunt way of looking at it. And by the way, it will shock you when you look at just how much of a company's free cash flow goes directly back to stock-based compensation. Yeah. Well, I think we've done We we we we've done that analysis, Bob. You've done that analysis.

36:44 you know, I think on a chart of maybe 40 or 50 names, we have maybe 10 names that are under 20% of their free cash flow as as a percentage of stock-based compensation. So, it is like the thing that jumps off the page at you is just for tech companies, which is what we're we're we're here talking about, what we care about, it's a massive, if not the largest single expense for them. And it's the one thing that way too many of them are not paying attention to. I think to to give some credit where credit is due, I do think some of the adjustments, for example, that you've seen Meta make, both in terms of head count and now how they're talking about SBC as a real expense, Uber's talking about SBC as a real expense. Like there has partly due to your leadership, partly due to this move to get fit, people are paying attention and are talking about it. And I think from our perspective, it's not that you should stop giving you know, using stock, but to your point, you should discount the damn thing and treat it as cash, and we should have an honest conversation on an all-in basis what these various companies are earning.

37:56 Yeah. And and and look, let me talk First of all, I do think shareholders are starting to care. Like it's it's it's it's the conversations are happening. I think most of them oddly happen in one-on-one conversations. And and and as was said earlier by Bob, the industry seems to kind of accept the suggested EBITDA thing. It shows up in the sell-side research. It's in the It's in the earnings releases from the companies. There are many companies who only talk about that on the earnings call and never talk about the net income. So, it is but but but I think the shareholders are paying attention. I think they're very unhappy with where things are.

38:36 a couple things Bob hit on. You know, if you're in a boardroom where people are saying, "We should buy back the exact number of shares to offset the SBC." That you were I'm this is going to sound like I'm really you know, but you're dealing with financially ignorant people and they just don't understand. And those same people I think are being widely disrespectful to people in your practice. Yeah. Brad of owning shares, you're not that stupid to where you would say, "Oh, if you buy it back then it never happened." Like it's I've had this argument many times.

39:12 Right, you're taking you're taking my earnings that we could either use to, you know, invest in new projects, invest in new countries and you know, or or just hand give back by way of dividends and instead you're buying back the shares which effectively just acknowledges that this in fact was the cash that we said it was in the first place. the RSUs vest, they sell them. So, you're selling and buying at the same time. Right. Wait, wait, what's the point of that?

39:39 Anyway, I also agree with with the comment that was made that I think management teams treat cash. I thought that conversation about the the hiring example you mentioned. But, management gets very concerned about spending a dollar of cash and there in many of these companies they they almost don't care about spending the equity which is a weird bizarre thing cuz they're they're actually the same thing in these cases. Well, it's not weird and bizarre if at the end of the year, remember, a lot of their bonuses are based on adjusted EBITDA.

40:11 Okay? So, again to Bob's point, if we adjust this out then, you know, it gets back to Charlie Munger. Show me the incentives and I'll show you the the behavior. And let me jump off of that. So so having been in many public comp committee, I I Here's what happens. Like it There is no one in that committee that's thinking about things from a shareholder alignment perspective. They're they're all thinking about what are the common practices. So you go hire a comp consultant. And the comp consultant's going to tell you what your peers are doing.

40:45 Right. In fact, you create a peer list. You study what the peers are doing and you build your program based on what the peers are doing. And if if enough people choose the 75th percentile, then you move you know what happens there. You move even even higher. but nowhere does anyone I don't think the comp consultant or anyone stand up and say, "Hey, is this a good program for the shareholders?" And when I've tried to do that in the board meetings, I get pushback. Right. And you know, there's a great Ironic given that, you know, there's a fiduciary responsibility to the shareholders, but I agree with you.

41:18 And and I love this quote. Warren Buffett said, "I'd rather stick a viper down my shirt than hire a comp consultant." And my big takeaway from that is the best and the brightest companies have to have the courage to lead and go in their own direction and create a compensation program that is unique to them and that is financially rigorous and understands these things. We've heard a great example of Booking. I think another one is Netflix.

41:46 Yeah. So Netflix today has a remarkably unique compensation structure. No one does anything like them. But today, if you're an employee, you get to choose either options or cash. And there's a slider and you get to pick where you want to be on that slider. They don't have RSUs. None. Yeah. And in a very recent comp committee consultant conversation, I said, "I'd rather use options." They said, "I wouldn't recommend it. No one else is doing it. Yeah, no one else is doing it.

42:14 No one else is doing it. Unbelievable. And so, anyway, you Well, this gets back to the conversation you and I had last week, right? We We even have a court criticizing Elon for taking a an options package where he made no money unless he saved the company from bankruptcy. Meanwhile, the CEO of his archrival, who created no shareholder value over a period of 5 years, is making tens of millions of dollars a year in RSUs. And by the way, I'll say it again, I said it before, I would offer any CEO I work with the Elon package, and I'm certain none of them would take it.

42:48 Yes. and and two, you know, if Delaware doesn't get this right, they really need to understand that shareholder alignment and executive compensation is a very, very important piece of making these markets work. So. But I'll just I I'll again, Bill, I'll agree with you that it cuz I've been on both sides of it, obviously, when I was on on the management side putting together these compensation plans and presenting them to our compensation committee and to our compensation consultant.

43:19 And of course, as, you know, as a director of many public companies and many, you know, comp committees, the dynamic is kind of almost always the same. A company establishes its peer group. And no board ever wants to think of their management team as being anything other than above the mean. Yeah. And you have 100% of all those companies that are thinking of their management teams as being above the mean. So, you have this constantly upward pressure on on these on the granting practices. It takes a mindset of like, how do we How do we as executives and how do we as leaders of of the company want all of us to make money? And, you know, I I think one of the big back to sort of what are solutions.

44:09 Comp consultants, comp committees, venture capitalists, even investors, they're they're very far too focused on you know, the the term you know, annual dilution percentage. you know, hey, this is a this is a reasonable granting practice cuz it's only 2% or it's only 3% or whatever. But really all all anyone is saying when they say that is they're making a commentary on the valuation of the company. Because if I'm a company making you know, a billion dollars of free cash flow, but I have a I have a $100 billion market cap.

44:50 Right? And and I and I've got a 1% dilution rate. Well, you know, I theoretically look like, you know, I'm managing the company well, it's 1% even though I just wiped out all my free cash flow. Right. You you just have to focus on dollars, ignore you know, dilution percentage. And if you want, yeah, you could look at you know, stock-based compensation expressed as a percentage of free cash flow, but again Brad to your point on that slide, it's a shocking number. The the percentage of free cash flow that goes right back in to stock is a huge number. And when you do that over and over and over again, if it's 25, 30, 25, 35% every single year, which you know, that's kind of the math of it for many many years, the dilution is actually if if you dilute free cash flow by 30 to 35% every single year for 20 years, guess what?

45:50 Your dilution is 35%. Right. It's not a half of it's not 50 basis points, it's not 1%, it's not 2%, it's literally, you know, a third of the company. I think the you know, you make such a good point, Bob. You know, I go back to this Buffett interview he did on CNBC and because this gets pretty esoteric and and you know, you sit in these comp committee meetings, I see the eyes glaze over. There's a social dynamic at play. You sit on the board, you're in the room. You often know these people, right? You want to treat them fairly. you know, but Buffett said this and we'll show we'll show the score. He said, "I think the best way to compensate people is with cash. And if you want to give them a bonus for exceptional performance, give them cash.

46:35 It's simple, it's straightforward, it's clean." He's He said, "I don't believe in using stock options as a form of compensation. I think it's just a way to get around paying out cash and I think it's a way to sort of camouflage what's really going on." Now, I disagree with Buffett on the idea of whether or not we should use options. I actually think there's a tremendous amount of of of of alignment and incentives that are created through that. But if you ask me, Bob, what do you think is the best thing if you're sitting on a comp committee?

47:06 The conversation that I would force is let's have a discussion about cash. Right? What is the value of what we're giving up, right? In order to achieve our objectives. And does it make sense? Because if not, then it it it obscures bad business models. It obscures sloppy performance. Right? To me, it comes back to this question of like getting fit. Like you want to know, am I really fit? You know, if you if you're going to work out and you know, you you you're not measuring your weight or your body fat or anything else, you know, like you're just you're you're kidding yourself. And so, to me, the measure of whether or not a company is really performing is the cash equivalent, you know, you know, focus. So, Bill, on this the the thing I wanted to ask you is, you know, what has been the rule of thumb in Silicon Valley? Like, when if you're a founder, I mean, you used to invest almost exclusively in Series A companies, first institutional money in, and you know, I don't know, the rule of thumb is there would be a 20 or a 30% you know, stock option pool in addition to what the founders owned in the business. That would go down over time, I think, and you know, and as you march to the public market, you would you would have to focus on, you know, again, treating this as cash and getting getting profitable.

48:34 Where are What are the rules of thumb today in Series B, Series C, Series D? And where do you think, where do you think you're seeing some of the sloppiness? Well, look, I don't I don't what the thing I'd start with in answering that question is most of these startups never get to this place that we're talking about, you know, and and it's a lot of the commentary in Silicon Valley is always about the the 10 or 20 or 50 companies at the top, and the vast majority of these companies never get to that place. And so, you know, the general rules of thumb, as I started with, are like 10% headed down to three.

49:11 It is done as a dilution perspective. profitability first, I think, is if you have a company, especially one that burns a lot of cash, and if if new fundraising and capital is expensive, the one thing I think one of the reasons people start with this adjusted EBITDA number is it does represent a point in time at which you no longer have to raise capital from the markets. True. and you could argue over whether that matters or not if you're issuing stock, but it does. It may matter, especially if capital is hard to come by. and then I would just say, you know, some of the same things we've been saying, don't fool yourself. Like, don't don't assume cash is cheaper or more expensive than equity. They're really the same thing. And eventually it's going to matter. Like I do think Wall Street figures it out. I don't think there are any secrets.

50:03 I think alignment is really, really important. I encourage companies to and I I do this when I'm recruiting, too. Build Look at scenarios of the stock being up 20%, 40%, 80%, 100% and look at the payouts that that people are going to make on that. And see if you have alignment. I think if you're only doing zero-basis RSUs, you've got a horrible alignment problem. And don't be mad you like when you think you're EBITDA profitable and your stock doesn't have the same multiple as as the other company.

50:35 and and I guarantee if you look under the hood, they've got a lower SBC expense and they're not, you know, pissing away their free cash flow. Right. Right. So, I don't think there are any secrets. I think competition is a real issue. One thing I wanted to go to with Bob, you know, I I I saw this really interesting presentation from Gary Steele who turned around Splunk in a a similar way to what happened at Meta. Yeah. and Gary, you know, crushed it at Proofpoint and a huge value creator. And he said, you know, in a very short window he went he he he said he took stock participation from 87% to 15. And I saw so many CEOs in the audience like their face the the just went into shock.

51:20 And it has become almost ritual in Silicon Valley to assume you have to give everybody stock, including the chef and like, you know, it it literally is part of oh, we're cool like everybody else. And so I think that's being re-examined. Like and then the other thing he said was they moved a lot of employees out of Silicon Valley. Right. And that's an interesting like thing. And I hear I'm hearing that from more and more CEOs. They say Silicon Valley is a wonderful place to start a company.

51:49 It's a horrible place to scale a company. Yeah. My goal isn't to sort of come on this podcast and say, "Hey, everybody's getting overpaid." Far from it. It's more just how I think people, you know, we we we need to have a better model by which people are get paid. And at least at at Priceline Booking, we always had the mantra of you know, a dollar has to be worth a dollar, whether it's in cash or comp.

52:19 And if if we can't convince somebody that a dollar worth of stock is worth a dollar, then you actually shouldn't grant it. You should just give a dollar of cash cuz they'll actually value it, you know, greater. Now, I think over the over time because we've been so careful about this and because we have been able to create shareholder value through many ways, mostly through the through the incredible work of of the employee base, but also, as I said, through smart capital management that has allowed us to reduce our share count. That's an incredible benefit that employees have gotten over the past many years. Like, when the share count is getting reduced, theoretically, the stock price is supposed to go up. And that benefits not only shareholders, but the the RSU recipient. Hey, Bob, do you have any sense how many employee millionaires that Booking Priceline has made over the last 20 years?

53:19 no, I don't. But it's it it it is it is certainly it is certainly a Extraordinary number. Yeah. Extraordinary number. And you know, maybe this is a a good place, you know, to wrap. I think your leadership on this issue, right? With this fundamental thing. This is not about undermining founders and employees' opportunity to you know, make an extraordinary return, make millions of dollars, you know, in these startups. In fact, it's about protecting that. Yeah. And it starts in the public markets. Because if the public markets all pretend that this doesn't exist, then they're going to pay excessive amounts which deprives startups and the venture capital community of being able to hire those employees and compete. It's you know, it's the example you just gave us the the the just now of Microsoft. And so, I think that accountability, treating stock as cash as public shareholders. And the good news is, like I said, there's a lot of evidence that this is starting to you know, you know, be a real matter of conversation on Twitter, on CNBC, in annual letters, you know, folks like Bob leading the way.

54:34 And then, I think we need to you know, begin telling the stories again in Silicon Valley. The reason the Michael Dells of the world own as much of their companies at the end of the day as they do is because from day one, they were concerned about giving shares away where they didn't need to give shares away. And so, that discipline cascades down. I think all companies are better when their DNA from the get-go, right, is just honest, right? They treat all of these things as expenses. It forces you to build a better business model, right? We were lucky to be partnered with Bob investors from the early days. And they were forced to really reboot and think about the business model that they were going to focus on. And if they kept pretending, right, that this didn't matter, I think it would have been a very different outcome for that business. I think you would have produced far fewer millionaires than the number that you've produced. and you know, again, in Silicon Valley today, I do see this also cascading down into the companies that we're investing in. Certainly the companies that were on the comp committees of the boardrooms that were in. This is a matter of conversation. you know, I would say during the heyday of ZIRP, I was seeing, you know, some really crazy stuff in terms of annual dilution.

55:53 And now it's a tough conversation at each of those boards because I think a lot of venture capital firms and founders woke up and said, "Wow, I just gave away a lot of the business. I thought that was okay because the valuations had gone from 500 million to 5 billion, but now that I know I'm really only worth a billion dollars, you know, that the consequence of that dilution was way more dramatic than I thought." And I I would just close by by, you know, once again going back to this notion of independent leadership. The only companies I see kind of getting out of this box are ones where you have people that are thinking from a first principles standpoint like like Bob did at Booking, like, you know, in the Netflix example I gave, you know, they're they're only diluting 1.5% a year, which is pretty left rail kind of thing if you look at them on one of these charts we're going to post. and so, you know, it it it takes someone that really understands this stuff at a fundamental level to be willing to step outside the box cuz once again the comp consultants, you know, maybe even ISS are going to tell you to do what everybody else is doing. And and I don't think that's going to get you to the finish line. Right.

57:04 Thanks for being with us, Bob. Well, Brad, I thought maybe we'd hit two or three more topics today. And and and wrapping it up, I just thank you to once again to Bob. And I I wanted everyone to know that we're going to put a bunch of resources in the show notes. We've taken snapshots of 50, 100 different companies, looked at SBC as a percentage of free cash flow, which which Brad and his team believe is the most important, but also revenue and market cap. I think those are all useful ways to kind of get a sense of how companies are doing on this issue. And then lastly, my good friend Mike Mobusson, who many people in the internet finance world know about, he he put out a really great piece on stock-based comp and and you know, reading it put a lot of thoughts in my head as as we went through this. So, I'll put a pointer out there to that as well.

57:56 Okay, so there's some a lot of new stuff happening, Brad. since we talked last. one of them that's kind of interesting and I think if I think I saw a photo on the internet of Sundar and Tim Cook at a restaurant that may have kicked this off, but there's a rumor that Apple's talking to Google about leveraging Gemini, I believe. And tell tell me what you think of this. Yeah, I mean I I think the news broke yesterday. both their stocks were up a bunch on it. This idea that I think the implication of the rumor or the the the the tweet was that Apple was somehow maybe outsourcing or partnering with Google on Gemini.

58:36 And you know, I tweeted in response to this that it you know, I put the probability of them {quote} outsourcing AI to Gemini at next to zero. We talked about this on the pod a few weeks ago. And I think that we you know, I go back to that analysis, which is this. Apple is in this enviable position to be able to build, right, the killer personal assistant. Yep. that has memory, that has persistent, that can do transactional thing for us, whether it's make a phone call, send a text, book a restaurant reservation, book a hotel, etc. So, we suggested that they're going to work on their own native small language model that really doesn't need to be Einstein, it just needs to serve me really well as my personal assistant.

59:27 I think what's likely, you know, being commingled here is I presuppose that Apple is also talking to Google, to Meta, to OpenAI, and others about bringing those models in, right, as part of a generative search application that will sit on the deck as well. So, think of that maybe similar to Perplexity. And then, of course, they have their search deal with Google. So, you know, to me I I don't think anything really changes in my view about Apple. I think they continue to be in this really interesting position. I'm sure they're going to talk about it in June at their developers conference.

60:07 I think it's unlikely that you're going to see some transformative thing get get shipped this fall. I think you may very well see a generative search application that gets searched this fall gets shipped this fall that may in fact have multiple of these answer engines embedded inside it, including Gemini. but, you know, the final thing you know, I would just say on this is you know, I've been over the course of the last couple months pretty openly critical in my critique that Google needs to face up to this innovator's dilemma. Yes. that needs to make some real changes in terms of how they're going to market.

60:47 And, you know, I you and I have those debates all the time. And I I I think there you know, is a group of people you know, on Twitter and otherwise who assume that I must somehow, you know, that must mean that I'm short Google. you know, and we've said many times this show is not investment advice. don't presuppose you know, you know, where we stand on a particular stock. I'm often asking tough questions of stocks that we own. I mean, go back to to Meta. and in fact, we own Google.

61:19 We weren't shorted this week. you you know, when when when this news came out. And so, I think, you know, it's an important reminder, you know, much like Stan Druckenmiller, at this moment in time, when you're in the middle of these phase shifts, mental flexibility is incredibly important. And, you know, we are going to follow the facts, and we may change our opinions on a particular stock. I think, ultimately, for Google, you know, the challenge remains the same. The challenge for Apple remains the same. You should imagine that everybody is talking to everybody Yeah.

61:53 right now, as they try to source, you know, sort out where they're going to, where they're going to compete. But, I don't know. What was your reaction to it, Bill? Well, it's funny, because you I think you can have two two reactions that are exactly opposed to one another. So, you could view it You said both stocks were up. So, so, so that's the interpretation that this is positive for Apple, that they're they're maybe getting their AI game ready to go.

62:19 you could view it as weakness on Apple, that in order to get AI right, they need to go talk to Google, and they can't do it on their own. As we have said before, I think the handset is a remarkable asset in the long-term AI race. And, you know, there One issue that could could come out of this is regulatory. I mean, if you combine the two of them, that's nearly every handset. So, if the same AI code were running on both, I don't think that'd be great.

62:46 I I do believe that that the handset is so valuable here. If I were Apple and wanted to prove to the world that they had their AI game together, and I know this is a bit of a broken record with people, but fix Siri. Like, get the Siri up and running. That is the small language model that I'm talking about, right? That is going to be the new back end for Siri. Like, there's no way they're outsourcing that to Google.

63:09 And, by the way, on that topic, I think voice AI may be more interesting than, say, Sora or the video stuff that OpenAI released. I I think more and more people are realizing because LLMs are language models that people would love to just talk to them. You brought up the point that that Meta may be more focused on the sunglasses with with an earpiece, right? Right. Right. And and you want to be able to talk to that. And so for me the irony of this is I think voice AI may be a new frontier that you may see a bunch of these people race after and getting that to work appropriately so that you feel comfortable I always like to reference the movie Her just talking to this thing and not having any errors.

63:57 I think that's super interesting. So we'll see how this all plays out. let's move on to the next topic. So there's been a lot of conversation in the past few weeks about TikTok and ByteDance and obviously it very quickly moved to the center of almost all discussion because it it it it you know it involved the House and the Senate and it's on the Sunday morning morning talk shows. It's in every newspaper and it's been discussed quite a bit. But but I'd love to hear your thoughts, Brad, on this situation. Yeah, well you know, I've said publicly before, you know, we were early shareholders in ByteDance. I think we've been involved almost a decade, right, in the company. so take everything I share with with with with that grain of salt. first, I think you know, everybody knows ByteDance is a super large company.

64:49 there's a lots of numbers that are rumored out there, but the reporting suggested that over 90% of its revenues are outside the United States and over 100% of its profits are outside the United States. What's that mean? Well, I think I've seen other numbers out there that they're doing somewhere between 8 and 10 billion in TikTok US revenue and that they're losing money on TikTok US. So I think that it's important just to understand where that sits in the context of the entire business.

65:15 Secondly, with respect to myself, you know, I'm a dad and certainly an American first. And, you know, my kids use TikTok, they create on TikTok, etc. But, you know, I I you know, I read all the same things everybody else reads and I think it's impossible for me to prove or disprove. And I think there's legitimate debate as to what's going on. you know, I think there's also lots of legitimate debate about the house bill itself and whether or not that's narrowly tailored enough, whether the language needs to be improved, etc. But, clearly it passed the house, you know, by a wide majority.

65:53 I think there's a real question as to whether or not that will pass the Senate. But, I think that misses the point a little bit. Set all of that aside. What's become clear to me over the course of the past 12 months and certainly over the course of the past 30 days is irrespective of the merits of the matter, right? This is now in the sovereign domain, right? AI over the course of the last 12 months has gone from an interesting technology to literally the leading edge of where we're battling with global sovereigns over trade, national security, national economic interest. So, given that, I think it's almost inconceivable for me now to see how TikTok US has a path forward under kind of the status quo.

66:45 So, where does that leave us? It leaves us on the one hand with, you know, the proposal that I think, you know, President Trump had made toward the end of his term and which I hear bantered about by lots of senators and house members, which is some sort of spin to US shareholders where maybe ByteDance swaps some of its ownership or sells some of its ownership to somebody like a Microsoft or an Oracle and the app can continue running in the United States, okay? I think there are a lot of US regulators that would find that an acceptable outcome.

67:16 Yeah. but I'm increasingly of the mind that the Chinese regulators would not find that an acceptable outcome, okay? And the US regulators won't allow the app to continue as is. So, I think there's a real chance that when we're looking at this 12 18 months from now that it's an absolute no go, that they can't find a middle ground and that the app is shut down. Which of course you know, we saw President Trump tweet like the implications of what that you know, makes meta more valuable or or more powerful as he suggests or perhaps these other US companies. But the thing I guess that interests me the most is what is you know, that's just the first step, Bill.

67:58 Right? Like that's the first the the the first move on the AI chessboard between these two sovereigns if you will. It's hard for me to see a world where TikTok US is shut down or banned where there's not some response by China. And I think we just had this conversation about Apple putting Gemini AI on the Apple phone. I'm certain, you know, Apple China is a monstrous business, is a huge business in China. And I imagine that now that AI is going to be embedded in these phones, that there's probably a little bit different conversation that's being had in China about you know, about there as well. So, I think we're seeing just the early innings, unfortunately.

68:42 Right? I tend to think that global trade is a good thing for human prosperity. I think it's driven a lot of our global GDP and prosperity over the course of last 20 years. So, I don't celebrate that we're entering what appears to be a global AI Cold War on the economic front. and I and I'm certainly not critical of the fact that US regulators have these concerns. but unfortunately, I don't see much of a way out here. I I hope I'm wrong. I hope that, you know, you know, when President Xi was here a few months ago, he said, "I don't want a cold war. I don't want a hot war. I want a partnership." But that's not what it looks like from my vantage point today.

69:22 Yeah. I'll give you a couple of my thoughts. First of all, I thought our friends over at All-In did a really great job of of tearing this apart. And even where they disagreed, I think every single one of them agreed that the data probably should not be going to the CCP. Like, I haven't seen anybody that said, "Oh, I'm completely cool with data going to the CCP." So, if everyone's in agreement on that, and and and I do understand the arguments that maybe the house bill's too flexible or too vague or too broad, and you could tighten it up and achieve the same things. as I've told you, I feel like that the company, and just the company I'll say is either US TikTok or ByteDance, could have done a better job of proving data independence. And I think they could have hired an auditor, you know, KPMG or Pricewaterhouse or whatever, rather than simply saying, "Oh, well, it's in Oracle in Texas."

70:16 Like, to me, there's not much certainty in that comment. I would also say relative to your Apple comment, I do believe Apple has to keep all their data locally over there and prove it to the CCP. So, I think But that may not be enough. No, I mean, but this new context is different than before. But I'm just saying from a reciprocity standpoint, I think they're already doing what is being asked here. and as we all know and we've said, the social networks here aren't allowed to operate there. So, you could have a I think you could have a pro-China mindset and just still hold the line and say fairness is fairness.

70:51 so we will see how it plays out. I'm hopeful that, there's a way to get it together, but but who knows? we will see. anything else on that one? No, I I mean I I I I think that it's once you start going down this slippery slope, right? you know, we have a lot of US companies that have a big footprint in China. and so We're already down the slippery slope. Well, I I mean we haven't banned it yet.

71:22 So, I think if I think if and when this effectively you know, if this were to get banned in the United States again, I would like to think that that would be, you know, where this ends. All I'm suggesting is that Chat GPT only came out 18 months ago. And the wheels And you can't use it in China. And the well, the wheels the wheels of international relations, I think turn slowly. and when we look at it it first started with silicon, right? With restraints on China's ability to get Nvidia chips. and again, I don't disagree with you know, our ability to keep our best stuff to ourselves, but it's hard to look at the facts on the field and say that this is not we're not entering a new phase of global international trade relations, you know, vis-à-vis China. Now, one might say that we're just, you know, creating a more fair and balanced playing field, right? Relative to the restrictions that have been imposed on US companies, but I think it would be naive of us to think that there's not going to be, you know, counter responses out of China. I hope not, but I I take the other side of it simply because we're not even getting to the point of equality here. So, anyway, we'll leave it at that. and then this morning, just this morning, we rarely have news that broke just this morning, a very unusual thing has played out with Microsoft and Inflection. Why don't you tell people what went down. Well, Inflection, you know, is you know, just as a reminder, was one of the early LLMs to be trained on H100s.

73:00 They have a consumer product called Pi, which stands for personal intelligence, founded by Mustafa Suleyman, one of the co-founders of DeepMind. you know, and and a friend. and you know, what they did was, you know, as one of the com And by the way, just to put in it I think there's another way to frame this. There were three LLM companies in Silicon Valley that had raised over a billion dollars. Although some of that was with credits, but there were three. Anthropic, OpenAI, and this one. Correct. So, you know, there's a large startup, but what they were going after was really think the ChatGPT consumer market with Pi. And you know, today we heard we heard news that Microsoft has effectively hired the employees out of Inflection. We don't know all the details Yeah. you know, of of the deal here.

73:56 but you know, we looked at investing Inflection as many others did, and you know, one of the reasons that strategic investors have displaced a lot of venture capital investors in the LLM game, and we talked about this I think on, you know, episode two, is the quantum of capital it takes to compete in the LLM space is very high. And on the other side, understanding the durability of the business model, like in this case, getting consumers to pay you, you know, for Pi is far from clear.

74:30 Particularly in a world where you got to go compete against Apple, you got to compete against Meta and others who are going to build a personal assistant. And so I think it was a very difficult venture bet to underwrite. I think it was a logical bet for folks like Microsoft, you know, to underwrite because you know, they want to be in the consumer answer engine space, the consumer chatbot space, if you will. I think the most interesting thing out of this to me, and if you read Satya's tweet that he sent out this afternoon, is he said he's excited to have them build consumer AI. Yeah. Which you know, he said the co-pilot like the co-pilot that is loved and benefits people from around the world. So, just step back from a second here. He owns 49% of Open AI. Chat GPT is is charging really hard to be you know, kind of my personal co-pilot, if you will.

75:25 and now they have another you know, another team in the hunt. So, you know, I think this is smart by Satya and Microsoft. some might view it as hedging their bets a little bit against Chat GPT. but I think it's great for the consumer. There're going to be a lot of at-scale players that are vying to be your personal assistant, that are vying to answer your questions. We just talked about Apple. We talked about Meta AI.

75:52 Google's certainly going to be in that hunt. Yeah. and now it looks like Microsoft is doubling down on the consumer co-pilot, which will bring even more competition to the table. What was your What was your your takeaway? Well, look, I I mean, I I got to tell you, I was floored a bit when I saw this. mainly because of the peculiarity of it all. So, to have a company that is one of the three let's just call it biggest backed AI companies in Silicon Valley not get bought. Like, if they got bought, that means, "Oh, Microsoft thought this was strategic.

76:28 They Microsoft" You remember when when Facebook bought WhatsApp? It was 10% of market cap. Yes. Like, so 10% of Microsoft's market cap is 300 billion. Right. So, so they could have paid 10 billion. And the last round here was at 4 billion and you'd go, "Oh, great. You know, everyone loves AI. Investors are getting paid. Everything worked out." But, this isn't everything worked out. This is employees all the key employees leaving the company and the blog post suggests the company goes on with a new CEO. And to me, I just had to look this up cuz I don't watch racing, but the yellow flag comes out when when they there's a hazard on the track and everyone needs to slow down. And you look at the series A here, a well-known venture firm, Greylock, 260 or 250 at a billion pre.

77:21 These are these AI deals that have remained like ZIRP-like, right? And this is a warning sign that you can't just pay any price for any deal in the AI sector and get paid. And cuz I don't, you know, this I don't see this now. I don't see the current status as a company that's going to have a venture-like return from here. There's a lot of details, there's a lot of these credit things that are all tied up, there's stuff we don't know.

77:51 But, it's it's unusual and I view it as a yellow flag that we should slow down on the track on the AI track and maybe be a little more careful. Well, one of the things that it brings up too is a conversation you and I were having, you know, just about LLMs generally. Yeah. You know, I I spent the last 2 days down at Nvidia, you know, GTC, their global developers conference. It's just extraordinary. The the ecosystem of AI that they are consuming, right? This is not a chips company, this is not just a systems company or a supercomputer company. They are now expanding on the CUDA stack with a totally new inference software cluster called NIMs.

78:35 And in NIMs, you know, they have embedded, you know, open source models that are, you know, highly and tightly integrated with their overall software stack. And one of the things we've talked about and debated the last 12 months is just the durability. Like, what's the business model for the LLM? Yeah. Is it going to, you know, you have these open source models by well-financed companies like Meta that don't seem intent on charging for them.

79:06 And when you have companies that are valued at 10, 20, 90 billion dollars, Yeah. right, around that, the real question was about the durability of revenue. And, you know, we had a we have a a vibrant debate within Altimeter on this, right? So, the the acronym we all know when it comes to software companies like these LLMs is ARR. So, everybody wants to know what's their revenue. Yeah. ARR stands for annual recurring revenue.

79:37 And recurring revenue, like a monopoly cable subscription, gets a really high multiple. That's why softwares have gotten high multiples. They're very predictable. But when we look at what's going on in AI, I I've banned the use of the term ARR within Altimeter when we talk about AI revenues because, they're certainly not recurring like a and they have some amount of cogs tied to Right. And so, well, one of the things we call them ERR, experimental run rate revenues, right?

80:12 And so, when I look at these, you know, one of the things that in inflection and these other models that it's been very challenging for us to get our arms around backing, even though they have extraordinary founders and are building extraordinary things, is just if you have to pay a high price, then you need to have confidence in the durability of those revenues and profits on a go-forward basis. and I think you have some thoughts on this, Justin.

80:39 there's nothing that I've spent more time on recently just trying to learn and ask questions and but you know, both try and understand how these things work, but also how they're being used. And one thing you constantly hear from developers on LLM, and let's separate this maybe from the consumer game that I guess now and then there were two or three, right? You got Microsoft. I guess you got Google for Plexi and Open AI, $20 a month. That's a whole separate conversation, but on this B2B side, the model that just seems so prevalent with all the companies I talk to, you might test your thesis against one of the the the premier models, Right. but then the minute you go to run it, you then run it on four or five others and see which one and and at different price points and see which one is cost-effective for you. And the promiscuity Mhm. is unbelievably high. Another way of saying that one one of the things that leads to high valuation multiples is switching cost. And today, you know, as it stands, I would say if you were to rate these LLMs on their switching cost with zero maybe being none and 100 being perfect, I'd say they're two. Yes.

81:52 Like it's remarkably low. Mhm. And there's another huge irony some people believe, and I I won't try and make this argument, I'll just say I know some smart people have said it, that as the context window gets bigger, there'll be less training and even less fine-tuning, which means you're just dumping data into the prompt. Mhm. And this is this data is not being dumped in in a structured way, it's just PDFs or CSV files or whatever. There there is no lock-in. Right.

82:20 You just dump it in the next one. So, I think one thing that's imperative on any of these LLM models, if they want to be successful in the long run in terms of creating equity value. They've got to find a way to get to switching costs. Right. And I, you know, I I don't know what that is exactly right now. I think some element of fine-tuning or embedding data or memory like we've talked about, but I don't think they have them today.

82:42 And maybe this is maybe this is a sign of that. I don't know. It'll be interesting to watch. Well, I I I think along those lines, I had dinner this week with one of the heads of data and AI at one of the largest banks in the United States. And they did what I think a lot of other people did after they saw ChatGPT. So, remember, ChatGPT comes out in Q4 of '22. The CEOs of all these banks and every other enterprise went home and that's what what the conversation was around the holiday table. So, the CEOs or the board came back in Q1 and they said, "Hey, what are we doing in AI?"

83:15 And they turned to their tech team and they said, "Let's get something done." So, in Q2 of 2023, you saw all of these new customers sign up the Azure OpenAI API, right? Or do it directly with OpenAI because it was the only game in town. But quickly, all of the other data platforms, whether you were whether you were AWS, whether you're Google, Snowflake, Databricks, etc., they all spun up their own AI capabilities. Now, they all embed free models, whether it's Llama, Mistral, etc. Even Microsoft now has competitive open models. And you know, my hunch is that OpenAI will also be found on some of these data platforms, not just on Microsoft.

84:01 and so it seems to me the switching cost is not with the LLM. I agree with you on this. The switching cost is really about where your data resides, right? We've heard this talked about data gravity. So, this particular bank, which had spun up the OpenAI API in Q2, has been playing with it on a big rag over the course of the last year. But I asked them what their plans were going forward. They said, "Oh, we're moving We're going to run all of our AI in AWS, right? On open models on our production data." Right? So, it seems to me that the AI is finding its way back to the data. And the only way, I think, for Azure anybody else to keep this experimental revenue when people run to use these models is you have to deliver something sufficiently differentiated that it gets somebody to pull all of their data to you. And that's a hard road to hoe.

84:57 and and I'm reminded I don't know if I've used this metaphor before, but there was a time before digital music where you just could There was so much music in the world you couldn't imagine it. And I remember one day back in the Napster days where someone tapped me on the shoulder and you said, "You see this?" And he was holding a hard drive and he said, "This is all of the music ever recorded." And like he just had it. And to a certain extent, you know, the LLM it may be, and I've heard this argument from some aca- academics, it may be that most of the data, think Wikipedia, everything we know is already in there.

85:31 Yes. And like that it it's gotten in, you know, small very quickly. And what The reason I bring that up relative to what you said is is it may be that the open-source LLMs are good enough. Yeah. And that putting them proximate to your data Mhm. is a smarter play than trying to put your data proximate to the leading proprietary LLM. Yeah. And if that plays out, that'll be super interesting. And And one thing's for sure, the open-source LLM horse is out of the barn, right? It's running wild and free. I mean, then and you got to you got to love the competition. And, you know, again, it's a you know, you and I have talked about some of the concerns about regulatory capture as you know and and preventing this competition you know going back to the conversation about China so important that we protect this competition among these models because it's driving down the cost all enterprises is driving down the cost to all inference to consumers and you know all I know is it's going to be an exciting few months ahead so fun fun to have you here fun to do this again and until next time.

86:43 care.