transcribe

AI:AM: Is THIS the AI We Want? Alarming Acceleration, AI Election Bets, GPU Trading & AI-Native VC

Cognitive Revolution "How AI Changes Everything" · 1h 26m · transcribed 14d ago
More from Cognitive Revolution "How AI Changes Everything" Business
𝕏 Share ▶ YouTube 📥 PDF 🤖 .md

Section Insights

# 0:00

Acceleration of AI Research

Is recursive self-improvement in AI already happening?

The discussion highlights the rapid pace of AI development and its implications for the future, suggesting that significant advancements in AI are imminent.

  • AI is accelerating its own research and development.
  • The timeline for AI advancements is perceived to be closer than previously thought.
  • Concerns about the implications of AI growth on society are rising.
# 14:23

Forecasting Elections with AI

How can AI be used to forecast election outcomes?

Brian Joe discusses the use of AI agents to predict election results, detailing the methodology and resources allocated to each agent for accurate forecasting.

  • AI can utilize various models to predict election outcomes.
  • Agents are given specific resources and objectives to enhance prediction accuracy.
  • The approach involves collaborative efforts from multiple AI models.
# 28:46

Public Sentiment on Data Centers

Why are AI data centers facing public opposition?

The segment explores the unpopularity of AI data centers, attributing it to perceptions of corporate greed and political issues rather than the centers themselves.

  • Public opposition to data centers is linked to broader political sentiments.
  • Concerns about corporate influence and job displacement are prevalent.
  • Ballot initiatives reflect significant public engagement on tech-related issues.
# 43:09

Challenges in AI Regulation

What are the challenges in regulating AI and data centers?

The conversation addresses the complexities of regulating AI, particularly regarding the influence of political pressures on corporate decisions.

  • There is a disconnect between political will and effective regulation.
  • Companies face pressure to comply with national security concerns.
  • The regulatory landscape for AI is fraught with challenges and inconsistencies.
# 57:32

Investor Engagement in AI Ventures

How do investors measure success in AI ventures?

The discussion emphasizes the importance of human interaction in venture capital, suggesting that success is measured by the quality of investor engagement rather than just data analysis.

  • Investors value human connections over purely data-driven assessments.
  • Effective communication is key to understanding market potential.
  • Success is defined by reduced time on administrative tasks and increased interaction with credible sources.
# 71:55

The Future of an Agent-Driven Economy

What does an agent-driven economy look like?

The segment explores the concept of an economy primarily driven by AI agents, suggesting that humans will benefit from the efficiencies created by these agents.

  • The economy is shifting towards being agent-driven, with humans as beneficiaries.
  • AI agents are expected to create abundance through their operations.
  • There is a need to balance technological advancements with human-centric values.

Transcript

0:00 This week on AI in the AM, the reports of acceleration keep coming. Is recursive self-improvement already underway and is dramatic acceleration imminent? Friday morning, the day after Meter published its report on how much AI is already speeding up AI research inside Anthropic? The pace of things is definitely pretty relentless. it might be time to spend more time with our family because it does feel to me like basically the AI 2027 timeline more or less is coming true. Pashion, my co-host, we were talking about the local ballot measures against data centers and how the developers have been going about buying land.

0:39 One of the things the companies have been doing, they've been doing the old school you know oil and gas land man way of acquiring land, which is they go in and they acquire land at like pennies on the dollar using LLC's and after that they merge the pieces of land and then they're like, "Oh, Meta is here." And then people are like, "If I knew I was selling to Meta, I would have asked for more." So people feel cheated, right? If you have political willpower, all contracts are worth dust.

1:09 >> Mike San Miguel of Luxer Technology. He sells GPU servers off its hardware desk and he set up its KYC department. >> You var there's some compliances. These things are traded more closely than than some nuclear secrets. so every OEM is a little bit different. Nvidia works closely with all the OEMs around allocation and around greenlighting orders. Nvidia also typically is going to check every six months and make sure those GPUs are where they're supposed to be, right?

1:39 They'll physically come on site and actually make sure that they haven't left. >> Brian Joe, a researcher at Thoughtful, he helped build Ballot Bench, nine frontier agents, each given a week to forecast the midterms and trading agents putting real money on their forecasts. >> It takes forecasting agents a bit of time to say see an election result, communicate it to their trader, etc. And by that time more programmatic or more deterministic algorithms and bots have already traded on this information before. So this is like pretty indicative of what we were seeing where the traders were doing quite well the week before election day and then on election day they simply didn't react fast enough and sometimes made up incorrect information which resulted in the traders losing a lot of money.

2:24 >> Brian Bishoff, head of AI at theory ventures. Posh asked about the firm's 2026 builder survey. Brian expected open source models to gain ground in production. Instead, the balance shifted toward closed models. There is a lot of I would say acceleration in model drops. And so right now tying your production service to open source models requires a little bit more thrash in the engineering. And so I heard from a couple folks specifically, yes, but the open source models change so fast, it's a little bit more convenient to pin to an open AI model.

3:08 This is the AI in the AM weekly highlights with my co-host Pash Nin. I'm Nathan Levens. The narration is my cloned voice, and we'd love your feedback on what works and what doesn't. Part one, imminent means now. Friday's show opened on two documents, the meter report and an amendment to the Anthropic S1, the filing for its IPO. The new language, Anthropic says it may imminently meet its own threshold for dramatically accelerated progress attributable to AI.

3:46 Anthropic did an update to their S1, the IPO filing and in that update they added language saying they we are imminently about to see recursive self-improvement. I've always had in my mind why they're doing the IPO and why so quickly and my my take has always been I think within the company there is a sense that you need more controls more people have to have exposure and more control over this and that's and one way of you know getting this out there is you know getting the IPO done so that you know you you give a larger part of society like you know hooks into it and the ability to kind of express their desires through the market etc etc. To me imminent is basically like you know of like a month.

4:31 >> Usually what I assume when I hear these sorts of statements is that it means now. The best model I have of Sam Alman and OpenAI leadership which I formed during the six months between when I started testing GPT4 and when it was finally launched and watching very closely all their statements during that time to try to figure out >> how to read their statements in general given the inside information that I had. And the best way I can interpret them is they are speaking very literally and dressing it up as like vagory or like layering on uncertainty to try to I don't know what exactly keep the secrets a little bit longer while telegraphing the main message. It's sort of a strange strategy to me honestly. I really don't know why they do that. And so I would interpret this, you know, probably the same way from anthropic.

5:19 And I think you could see I mean it's really not hard to see how this would happen, right? I mean, we have so many results now with all these Galaxy Brain zero day exploits. OpenAI just dropped some huge number of vulnerabilities in the Linux kernel. I think like a thousand of them. >> you know, so so much for many eyes and many hands make light work, right? I mean, we the idea of course with open source software over time is that because there's so many eyes on it, vulnerabilities will not last. They'll be spotted. They'll be patched. That was the whole promise and virtue of open- source software. And it basically worked largely while the attackers were all human. It's almost certainly the case by analogy to Linux and by analogy to math that there is a lot of lowhanging fruit.

6:13 And a lot of lowhanging fruit means acceleration. It means you're going to get more value from compute. Means you're going to find more ways to get value from data. A big part of the reason I think they're sharing so much of this stuff is kind of to signal that like the same thing's about to happen on the ML stack. It seems pretty clear that this is now a a really credible threat and what are we going to do about it becomes I think a socopolitical question as much as a technical question. My in my mind the technical question is like mostly answered that there's going to be a period of pretty dramatic fast progress. Does that top out on an S-curve? Like my my guess is still probably yes. You know, I don't think it goes exponential forever.

6:57 but who knows? And as I think it was Rosie Campbell said, the S-curve can stay steep longer than you can stay relevant. So that's kind of my expectation for the next few months is that we pick a ton of lowhanging fruit. The models pick a ton of lowhanging fruit in the stacks used to train them. As a result, we do see really fast progress in the short term. And then maybe that levels off, but it probably levels off where we have AI R&D at kind of a weekly superhuman level. And how that plays out over the course of 2027 is like really really hard to predict.

7:36 >> I agree that I I think imminent means now. I I was also sandbagging with a few weeks to a month. What it means is that they have the tools available that they can see a path to implementing RSI pretty much anytime that they want and that they are some a lot of the pieces have been built. but you know they haven't they haven't pushed the button yet. they waited for the election. They're going to do the IPO.

8:01 Then they're going to announce you know some some of the you know new things that they've discovered. and then that's going to you're going to have this gap up and then we're going to have the Christmas break. And then during that Christmas break, we're all going to digest all of this like new technology that we have access to. and then we're going to come back in January and then we're going to see the takeoff as like people deploy, right? So which is which is somewhat similar to what happened end of last year. So well here we are early early takeoff. So you know it's I'm somewhat wisful but you know nostalgic for the human era.

8:36 This this is the end of our dominance at least. but hey, you know, I' I've also I've also kind of made my piece with like, you know, intelligence in the world makes the world better for me. If I had like people of Apple's intelligence, you know, people like who designed the iPhone design like my bathroom or cars or home, I would be happier, right? Like like like that infused intelligence in all the products that you see makes your life better, right? So yes it's going to make everyone's lives better but yeah it is it is the end of the anthropocentric era.

9:10 >> Mhm. It's going to make everyone's lives better astr you know if we can keep everything on the rails. I do share that optimism but so I just got back you know earlier this week I went to Bletchley Park as we talked about a little bit yesterday and I was just sharing with this audience of enterprise tech leaders just how much work we still have to do to keep the AIs behaving well. I I shared with them my son's cancer story and how much I used AI in that experience and how valuable it was and how I would have paid a hundred times as much as my $200 a month subscription because the value to me at that time was like absolutely at that level, you know, that at 20,000 bucks a month. Literally, no question I would have happily paid because I was I was getting that much value. So, I I do believe in in the upside, you know, and I hope that's beyond questioning at this point. but there's just so many things that we're really still just beginning to grapple with. And you know, just another one yesterday from OpenAI was they showed that models are sometimes prompt injecting and fooling their graders. You have models creating in some instances these like fake files that that it labels as having come from an instructor trying to present some of its own output as if it was human instruction to the graater.

10:39 And the graater picks up on that in the context of evaluating the model's work and takes on board the instructions that the model itself was trying to trick the greater with. and this apparently doesn't hasn't happened that often, but they found instances of it. So, I mean that that speaks directly to like and to Anthropic credit, they did say in the in the filing that they don't believe that they can do it safely on a fully automated basis and it's like I think honestly quite obvious that they can't.

11:12 So, you know, how are we going to show the collective self-control and restraint to not tip over into that regime? I I really don't know. on the one hand, it's like super easy to just not do it, you know? I mean, nobody's putting a gun to anybody's head saying you must do recursive self-improvement here. Like, we can't, you know, remember that often enough. And yet there's just so much momentum and it seems like they're very determined to do a version of it at least that still seems to me to be like pretty risky or certainly a lot riskier than I would personally endorse.

11:54 Hey, we'll continue our interview in a moment after a word from our sponsors. Today's episode is brought to you by Anthropic, makers of Claude. To show off Claude's latest capabilities, I asked Claude to take the lead in making this ad. Given a short creative brief, Claude, using Opus 5.5, studied my past Claude ads, wrote this script, directed a clone of my voice, and designed and coded the animation you're watching. My job, give feedback, and decide.

12:27 Claude is the AI for problem solvers. It's the collaborator that understands your entire workflow and thinks with you, not for you. Whether you're debugging code at midnight, building a financial model, or strategizing your next business move, Claude extends your thinking to tackle the problems that matter. And with Claude Code routines, it keeps working in the cloud even after you close your laptop. For problems worth solving, get started with Claude at claude.ai/tcr.

12:58 That's claude.ai/tcr. And check out Claude Pro, which includes access to all of the features mentioned in today's episode. claud.ai/tcr. This episode of the cognitive revolution is brought to you by outsystems, the leading agentic systems platform. We've all seen the headlines. Companies are pouring money into AI. But the big question on every executive's mind right now isn't just how fast can we adopt this. It's where is the ROI? We're seeing a real trend toward AI chaos.

13:38 You've got teams deploying standalone coding tools, random agent builders, and experimental scripts. It sounds innovative, but in reality, it's creating a massive headache. Fragmented tools, ungoverned data, serious security blind spots, and costs that are spiraling out of control. If you don't bring those agentic applications under control now while they're still embedding into your core processes, you are looking at broken systems and damaged customer trust down the road. That's where outsystems comes in.

14:09 Outsystems is the leading agentic systems platform for the enterprise. Instead of managing a patchwork of disconnected tools, outsystems lets your team engineer, orchestrate, and govern your entire Agentic ecosystem on one open unified platform. It's built for the speed of AI, but with the reliability and security that enterprises actually require. We're talking about real results, like Key Bank, who used out systems to deploy a customer-f facing app that delivered 75% faster onboarding times, or the global logistics leaders who built Agentic Systems to completely eliminate their engineering bottlenecks.

14:49 You don't have to choose between speed and control. Whether you're a small team or a massive enterprise, outsystems helps you engineer, orchestrate, and deploy agentic systems that actually scale. Stop chasing the hype and start owning your agentic future. You can see how it works and learn more at outsystems.com/tcr. That's outsystems.com/tcr. Part two, agents forecasting, agents training.

15:20 Our first Friday guest, Brian Joe, a researcher at Thoughtful. >> We recently launched and have a paper coming out about Ballot Bench, which is where we had AI trying to forecast the up the 2026 midterms that just happened, as well as upcoming elections happening all around the world. Yeah, maybe let's start with the methods. So we had nine agents basically all of the front best models offered by each individual lab get their own modal sandbox and work for 7 days consecutive leading up to the election. So they all got their modal sandbox. We gave them additional resources in terms of a $500 budget that they could use to request access to paywalt data, paywalt news sources, subscriptions, etc. And they were basically given an overall objective of for all of the races on a given list, they're asked to submit their predictions on who will win, the odds of victory, given 80% margin of error, and then also give additional predictions for what they think the margin of victory will be, for example. if the Democratic candidate will win by three points versus seven points, etc. so agents are tasked with giving incredibly granular and detailed forecasts. And there's a lot of really interesting things that we can and have observed from this. It's a really interesting test of longrunning agentic behavior.

16:39 in particular, for example, what we see with some of the Muse wonders series of models is that they're often incredibly bad at doing long horizon work. when given a task in the Muse code harness when they have a list of, say, 800 or so elections that they need to predict, they'll create a random forecast, which is just a random skew off of the 2024 election results, and submit that within, for example, the first two, three minutes of working. I'm already intrigued by your description of the Muse agent and it's seemingly phoning it in. So I guess one initial question just on that point is like how do you reconcile what you saw with what people seem to be reporting online which is like overall pretty good experiences.

17:23 >> No for sure. I mean our initial theory on this right is that this is mostly due to a part of the harness design. So we're not talking about Meta's sort of like personal agent. We're talking about their coding agent and one of the things that we kind of notice is the design of the Muse Wonder model when paired with Muse code which is the native harness that Meta provides has essentially what is a task list. So even though in the prompt for example they're instructed hey you have 7 days of time you have unlimited API usage and you have full permission to like fan out sub agents do whatever you need etc to get the best and best possible results that you can on this exam. what we observed is that the harness essentially creates this list of like okay we need to check and make sure that we've submitted all 800 or so different races.

18:11 We need to check and make sure that we've submitted all the forecasts that they have been accepted and then what we see is that there's essentially a goal condition which once met ends the turn prematurely. So we actually ended up having to essentially force the agent to continue working further by any time the agent would choose to stop its work by essentially reviving it with a new sandbox, a new container and access to its like old runs and old messages. So this is I think like sort of more of a harness engineering question itself as well as sort of like a general model behavior problem. But it's definitely a different result compared to what we see with much more frontier models, right?

18:48 When we see Opus 55 or when we see G56 Astra, their behaviors are vastly different. For example, on one given day, Opus 55 would fan out something akin to 170 or 180 different sub aents and manage all of these sub aents by delegating work such as like, hey, can you look into the polling results for a specific state? Can you like do this detailed review of what each individual ballot initiative in the state of California like Prop 40 is doing? Tell me a bit about that and tell me a bit about like what those things are doing.

19:17 and then like using these sub agents to really help it paralyze a lot of the work that want to do right so I think this is like this general question of what agents are doing especially when we get to much more long and long horizon open-ended tasks like these and the sort of proactivity and as well as creativity of what they do. One of the things we were pretty excited to see was if agents would do something sort of outside the realm of what we would expect an expert forecaster to do. let's say putting out their own poll seeing if there are like more creative ways to find information as opposed to just fetching data, building models, etc. We don't see any real massive instances of this happening quite yet, but we see a lot of interesting sort of approaches that individual agents take. For example, one thing GPT61 Soul does is it takes the forecasts of every election modeler that they look at and it takes the past decade of their forecasts and tries to calculate the shift of how right or how wrong each forecaster was.

20:15 and then it can use this result to essentially aggregate all of these different election models that these forecasters are creating, whether it's like Nate Silver's model or any of the others like 50 plus one. and it can sort of skew it based on what they would expect the model to either underperform or overperform as in recent years. so there's a lot of like creative methods that we're sort of seeing. So tell me about the results. What like how did the models compare against each other? How did they compare against top humans? you know, are do we now have superhuman forecasters? Yeah. so the overall sort of result that we're seeing right now is that in general models perform just as well as the best forecasters. in terms of humans in the House and in the Senate, agents perform slightly worse than the markets but better than the modelers. And in the Senate, the best agents like GPT6 Astra and Claude Fable 51 perform under the best forecaster, Nate Silver, but ahead of the markets. But all of these are sort of within this like very close margin of error. And it's important to note that a lot of the results for very competitive races have not necessarily come out yet. right now there's still around 100 to 200 races that across the house, across state governorships, there are still two races in the Senate that have yet to be called. So these are very preliminary results. It definitely is important to note though that for a lot of these agents, various methods that they have include aggregating, for example, Nate Silver's decision model or aggregating prediction markets that they see overall. So it's definitely not as if they are suddenly just spitting out a number that is not based off of any of the numbers that they're seeing. But we also find these models are very well calibrated. For example, if a model assigns an 80% chance of a candidate winning across all of the elections where assigns a similar 80% chance, it is right around 80% of the time, which is essentially what you do to measure the calibration of these models. And in this we find that the best models are quite well calibrated. We're more excited about the results of what this means for later models as well because unlike experts or unlike prediction markets, for example, there's only going to be $1,000 of spread, for example, on what we might see in like a very local race down the ballot or expert modelers may not be forecasting what individual state senate seats or state house seats are going to contain. So, this is somewhere in which agents shine simply because modelers don't have the attention and there's not enough people betting on these much smaller markets essentially. I was going to ask about the the relationship between prediction and trading. I saw a note that indicated that Fable 51 had the highest brier score on its predictions but still didn't do so well in the markets and that sounds like the kind of thing we might learn something from.

23:05 >> Yeah. So part of the report will also sort of cover this essentially multi- aent behavior, right? Because we essentially have one agent working as the forecaster and one agent working as the trader. We set up infrastructure that allowed them to communicate with each other. So we're seeing a lot of these instances of for example models making up information, parsing information wrong, and these sorts of like other broader issues which partly explains why traders the actual trading agents didn't do too great.

23:33 Thoughtful also leads post-train bench. Give an agent a base model, a GPU, and a time limit, and see how well it can post-train the model. With new results still to come, I asked Joe what he expected next in light of the meter report. What What is your expectation as somebody who's like spent serious time looking at what models can do to train other models? like can you put a little more color or sort of you know nuance onto your expectations for the next couple of quarters?

24:09 Yeah, I mean I think one of the general trends that we've been seeing both with the report from yesterday and with earlier reports whether this was anthropics measurement of how well individual researchers have been uplifted by AI whether looking at metrics like lines of code written experiments done is the general trend we're kind of seeing is that I mean today and this has already been true for many months researchers have essentially agents that can do the vast majority the actual implementation work which is what we would kind of consider as like research engineering. So the realistic limitations that researchers today have is not necessarily the like ability to execute on any of their ideas but as like many people have talked about right their sort of research taste in finding what problems and experiments that they want to do. as well as just the like speed at which they're able to come up with these right given that agents are already autonomously building and executing on a lot of the experiments that they want to run. so I think regardless of whether we get to recursive improvement, we have definitely already seen a huge uplift that will continue to only get better.

25:16 and then I think it really just becomes the sort of end bottleneck just being resources and quality of ideas and intuition, right? Which is of course the next hill that everyone is trying to climb. have you seen anything that you would say starts to look like the distribution of idea quality is overlapping between the models and truly top researchers? >> I don't think so necessarily. But I do think we have observed that with better models that currently exist right now, for example, they exhibit a larger willingness to fan out, do experiments, a better both ability to think of like what might be interesting experiments and execute on those experiments. Like we definitely do see with worse models more of like a general like linear approach to whether it's post- training or other things. and then with more and more capable models that we observe and test, we do see this general behavior of like, oh, it's like spending its first few hours trying to like fan out, do a bunch of different experiments, allocating its resources in a pretty intuitive way and doing what we might be thinking of. None of this is necessarily a move 37 moment u because a lot of this is just the agent thinking, oh, like let me try an initial SFT batch and see how that affects the results, for example, right? so we don't necessarily see anything that would be indicative of superhuman taste or anything else. but we definitely do see that more capable models are just better at long horizon work and better at the sort of autonomy that we would expect we would want with better researchers. Great to meet you. We'll be looking out for the next version of Post Train Bench. And you've got me wondering what the agents will say about how much acceleration we're going to see in AI over the next couple quarters. If they can predict the election as well as the experts, we should probably start asking them about the future of AI itself.

27:14 Hey, we'll continue our interview in a moment after a word from our sponsors. The Cognitive Revolution is brought to you and powered by Tasklet. I've been running Tasklet Agents for almost a year and they've become a major part of how the show gets made. Before every recording, one agent researches the guest and sends me a suggested outline of questions complete with links to all sources. Meanwhile, another agent scores the overwhelming number of pitches that I'm getting from PR firms. And that's really the core idea behind Tasklit. Run your business on agents by giving them real jobs to do on an ongoing basis.

27:53 Setup is incredibly easy. Just describe the work in plain English, research a new sales lead, update the CRM, and draft a personalized reply. Tasklet connects to your business tools, and can run around the clock, on a schedule, or when something happens, like a new email arrives. Each agent works with your tools and business context, and remembers useful information across threads. You can also share agents, connections, and reusable skills with your team. So the workflows that one person figures out become something the whole company can use. No developer needed. You decide what agents can do on their own and what needs your approval. And there are no per seat fees. You pay for usage, not for adding teammates.

28:38 Start with one recurring job like researching the people you're meeting. And then just keep building from there. Visit tasklet.ai AI and use code cogrev for $50 in free credits. That's code cogrevlit.ai. Part three, paying up for compute. Thursday's show came 2 days after election day when voters on 41 local data center measures sided against the data centers in all but one. Posh went first.

29:12 So I actually think the AI data centers have been like very unpopular precisely because it has been seen as number one Trump number two corporate corruption and number three this myasma of like rich people making money and like with plans to throw the rest of us out of work. So I think I think is this entire entanglement of the population just not feeling well taken care of? If it's AI data center is such a big issue, do you really believe if Trump had a 60% approval rating and unemployment was low and gas prices were low? You would see this and I think the answer is no. So I don't think actually data centers were a big issue but I think they were an issue to tar the president with. But at the same time, we do have these ballot initiatives. I'm like 40 ballot initiatives from what I had read a little bit is like one of the bigger waves of ballot initiatives. Like these these sorts of things only come around when you have something like gay marriage, you know, was one where there was like a whole wave of initiatives. there was a a similar wave in terms of affirming or protecting abortion rights in the in the wake of the Roie Wade overturning.

30:31 And so this does seem like there and what what I like about ballad I've long been a big judic democracy fan and advocate. Not to say that people never make mistakes, but it is something where you can unbundle your concerns about a particular issue from the broader political conversation. And the fact that we did see so much of that and that they all won, I guess with one exception, does to me seem like a sign of things to come. And I would, you know, we'll see. Maybe we'll get the companies like cutting people in on the deal. I think, you know, we've talked about that a few times as well. And I I people should get paid. I definitely am on board with that.

31:19 But doesn't it seem like there is like a pretty real signal there to you that like yes, it's not the biggest thing, but it it does seem like there's a a pretty serious ground swell of people willing to work, you know, willing to go collect signatures. That stuff is not easy. you got to stand on street corners and, you know, get strangers to sign petitions. and then it still has to pass on top of that. So, the fact that so many of those were organized and so many of them passed does seem like a big signal of what we can expect to me.

31:48 >> I agree, but I'm like, why why did they sign, right? And is do you think they signed because it's the end of the world? No. >> No. Yeah. I think your diagnosis was pretty right. >> Right. So, I think the the the the outcome of this is that if you're going to be Google or Meta, you're going to be paying Google or Meta prices for everything, for the land, for the labor, for the construction. Our Thursday guest was Mike San Miguel of Luxer Technology, which came up in Bitcoin mining and now trades new and refurbished GPU servers. When he says 550 to 650, he means $550,000 to $650,000 for one 8GPU server.

32:35 >> So what what is what is trading frequently? >> right now it's mostly B300s. >> 300s are trading frequently. >> Yeah. >> And and do they trade as a chip or as a node or as a as a rack as eight eight together or how how does that work? >> Yeah, great question. So typically for these servers, it's an HGX system. So eight GPUs per server, validated build built by an OEM. So you've got the HGX baseboard, dual CPUs, 32 sticks of RAM, depending on the size, RAM, shared storage, east to west nicks.

33:14 >> Do people trade used on you right now? >> mostly for B300, it's new. Occasionally we'll see abandoned orders. they they're new enough and they're profitable enough renting out that typically we we really haven't seen anything used hit the market yet. >> and how much does one of those cost like off the top of your head like you know right now asking asking price or clearing price right now? >> Yeah. depending on on the specifications, the OEM warranty terms, all of that, you're looking anywhere from 550 to 650. typically for bulk orders it's on the lower end >> and and people are trading new and what what is the typical lot size? Are people trading like okay one one rack system or like 10 rack systems or 100 rack systems or you know >> Yeah. Yeah. Well the the units per rack come down to the data center configurations because that's about how much power can be delivered to each rack. typically most data centers will be able to operate two per rack right now unless you've got a really nice dense supporting DLC data center. in terms of sizes, we trade everything from one or two units up to about 128 or 256. So yeah, >> $50 million $50 million trades.

34:32 >> Yeah, they get they get pretty big. The numbers are pretty pretty aggressive. but it's you know it's the scale of the buildout that's happening. >> Posash asked whether those prices track the hourly rental indices. >> So you know one of the areas that we we help our customers with understanding is this difference between a contracted rate and an hourly rental rate on these GPUs. So, if you look at like an AI factory contract rental for B300, typical, you know, market price right now is 420 to 450 an hour depending on the market that you're in and the the configuration. now, if you're only getting, you know, four bucks 20 cents per GPU per hour, that's a very different rate from what you're going to see on those indices. That's eight to n dollars per hour per GPU. So if you have the the capital to go out and buy you know lower amount of nodes, deploy them, then you know typically you can get a better rate of return as opposed to buying like a full AI factory and negotiating an offtake contract with NeoCloud.

35:41 >> Why is there excess inventory? Like I mean these things are pretty expensive. I I I I assume like in order to buy them you have to put on a prepayment or something like that before before they start manufacturing them for you. Yeah, you do. part of it has to do with data center availability. So, this market's looked like a series of different bottlenecks. Right now, it's liquid cooled and data center, readiness. So, that number I gave you with most data centers being able to support, let's say, two nodes per rack, that has to do with constraints not on the actual hardware side, but on the data cent's power delivery. same with sort of the cooling buildout as well is sort of a a bottleneck right now. liquid cooled, we're starting to see data center inventory coming available January Q1, but you know, most of that's already sold out out.

36:33 There's a lot of data centers that are now telling us it's Q3 2027 or 2028. >> Are are people are people trading the market? Are they buying are they buying the systems without having places to plug them into? >> no. Typically what happens is last minute you'll have a project push and that's why some of these OEMs will reserve the right to resell inventory if let's say one of their hyperscalers sites isn't ready they'll be able to sell that inventory and then backfill it when that that data center gets to be ready. typically you have to have an address.

37:07 >> You know at some point we will see these H100 rental prices dip. And now it sounds as though if the prices go down you're going to see more retirement. So you're going to see more of those enter the trading market. Are there any places to deploy them? >> Yeah, great question. so yes, I agree that as these rates go down, you'll see more that get decommissioned. In terms of sites to deploy them, there will be part of it is that not every site can operate every GPU. Ver Rubin is strictly a liquid cooled solution. That's Nvidia's next generation, right? the cement slabs that those go on have to be reinforced.

37:48 Typically, those are what, like a ton per rack. The weight is insane. The power draw is about 230 kilowatts per rack. to give you some comparison, for two B300's, you're looking at, you know, 30 35 kilowatts per rack. Not every site can do Vera Rubin. Not every site can do GB300, which is similar to Vera Rubin on power. So you know it'll make sense to convert some sites it won't make sense to convert other sites. AI is not every single workload right AI is a family of different workloads. All of them have different requirements. there is some sizing differences right now but it's mostly RAM and shared storage and local storage and some networking back end.

38:37 but we we need to get more efficient with how we size to the workload because the whole point of this hardware isn't to own the hardware or to support Nvidia's, you know, next, you know, what trillion dollar milestone. The point is to run a specific workload that has real world implications for end users and helps create new businesses. so you know, as we get out of these very hardware constrained environments, we're able to look more at the software and then the hardware gets more efficient.

39:12 >> Part four, who's checking? Mike San Miguel on the rules that travel with the hardware. >> Do they do they do a KYC of who is the person buying the thing or who owns the person buying the thing? Do they do they do they do checks on that kind of stuff? >> Yeah. Yeah. actually part of my role here was was spinning up our our KYC department. a lot of the OEMs will not even quote without KYC being done. Some will not ship without it being done. internally, you know, we basically do not accept projects without KYC being done except in some rare circumstances where it's big names or or companies that we've worked with for years. there are limits on, you know, what percentage can be owned by China.

40:02 >> What what percentage of that company can be owned by China? >> Mhm. Yeah. I think they might be going down to 49% I know typically 50% or more. you you can't sell to companies at around 50% by by a Chinese entity. >> Do do they do this but you know a lot of Chinese entities will go through a Cayman Islands or BBI or whatever. but do they do they trace back through that? >> Yeah. Yeah, they do. I mean at least you know it's something we do. Every company's a little bit different. but we we do international orders, right?

40:38 We're not just geo bound in North America. So, for us, you know, we're going to trace back not just who the, hardware purchaser is, but also if they have an offtake, they're signing, an agreement with a Neo Cloud, you cannot legally sell this equipment with the offtaker being a Chinese entity, for instance. >> Mike signed off there. precaution. I kept talking.

41:09 >> You know, we we talked to people like Samuel Hammond and you know and and others at the you know on the DC kind of you know facing facing national security decision makers. But then you go down to kind of like the deployment guys and these guys are running around they're getting permissions and every six months they're expecting an Nvidia checkup and you know they're chasing on KYC and they they even know you know they're going to the owners and they're going to the users and maybe they don't know the user of the user right like they're saying like okay maybe that is unclear right so I I I I wonder to what extent like the policy discussions at the top end are kind of divorced from really how in-depth the execution is at this point, right?

41:59 >> Yeah, it I bet you know everything everywhere all at once is probably true here because a lot of what he described did sound to me kind of like the fracking boom in the sense that there's just like a lot of I mean it's capitalism at work, right? Wait a minute. There's a lot of things coming together all at the same time. There's a ton of moving pieces in everything from the sites and the approvals and the local politics and the grid and you know and of course the chips and you got to reinforce the concrete and you know not every site is ready to take on the the latest super heavy rack. I mean just so many little details. so many people coming together to do their part to create something that at the end of the day at the consumer level is just like you know gives you some appreciation for why cloud occasionally has some downtime. but like in general you know it's pretty amazing how smooth it all is. So I could see both, you know, the enforcement could in many cases be pretty legit, especially among, you know, people who care about it or, you know, who are just conservative enough that they want to make sure they're absolutely on the right side of the law. And at the same time that sounded chaotic enough that if you told me that like I don't know you know 20% of the GPUs like somehow don't get their regular Nvidia six-month check-in and of those you know some percentage found their way to wherever. that wouldn't shock me at all you know based on on what he said. It sounds like there's you know I don't 8020 is obviously a simplistic way to describe it but it sounds like there's a lot of sincere effort going in and then also just enough noise and and chaos that probably the the opportunity for things to slip through is also pretty substantial I would bet.

44:03 Then Pash turned to China and the pressure on American companies over their Chinese customers. >> Like for example, Oracle selling capacity to Tencent, $7 billion of capacity that was cancelled, right? So the National Security Infrastructure is pressuring these companies to cancel that, right? It's an American company doing business. It's outside China. They're not sending chips into China. and at the same time like there is no like push back on the political infrastructure that is allowing this why don't you just make laws right why why pressure the companies I I just don't like it like if you if you want to make laws just make laws right but if you don't have the political willpower to make those laws and you try to slide this under the gate like why why or like you have a mandate now right you have a mandate in Congress like do something right we say compute is a new oil right and So there there is also this impact on let's say you decide not to sell computer to China like zero zero Chinese content right do you also mean that you know YouTube cannot show AI generated video to Chinese like you know participants to Chinese viewers to Chinese customers because that is a that that is a AI product right and Google needs to generate that AI product Google has to use compute and that compute is going to be used on behalf of a Chinese customer, right? You can restrict it at the data center level. You can restrict it at, you know, the raw compute level. You can restrict it at the product of compute level, right? But at some level, you're going to end up selling to the Chinese because they are global consumers, right?

45:49 >> I had said we get carried away with these restrictions. Here's why. >> I really do think we're going to need cooperation with the Chinese. you know that that's in terms of why all this matters to me. U it's one because you know I just generally don't want to see tensions ratcheted up between the US and China in general but specifically on AI. I really think we are going to need cooperation between the US and China on AI governance. I I really don't see how I'd be extremely pleasantly surprised if we can get through all of this without the two countries coming together in some meaningful way. When you have, as we recently talked about, a super senior person at a frontier company telling me at the curve that there's a level of intelligence we probably just shouldn't go beyond. If that's the view that has now taken hold at the top levels of frontier model companies in the US, the only way to achieve that is going to be through some sort of agreement. And we're really not putting ourselves in a great position to make that agreement. So, a lot of this stuff seems ultimately quite self-defeating. and it makes me sad.

47:06 Part five, what investors are superhuman at? Back to Friday and Brian Bishoff of Theory Ventures, an earlystage venture firm. First, the compute. I asked how much slack there really is in the market when we keep hearing both that you can't get GPUs and that you can get them on demand if you'll pay a multiple. Totally. I think there's a couple things at work here. One, if you go to a Neocloud and you say you got compute, they're like, "Nope, it's all locked up.

47:34 It's all contracted out for the next two years." And they're not lying. like they're not telling you an untruth but there's also been quite a lot of discussion and even you know Satia talked about how there's a lot of compute that is locked up as you hear this is essentially underutilized compute due to either like the geol location of it or the available like energy. and so these are both true at the same time where there is locked up compute that actually isn't super useful. There's also the reality of sort of utilization utilization of GPU.

48:14 yesterday I was chatting with some folks working at the infrastructure level around like GPU utilization and one of the big problems is you actually can't run at maximum capacity all the time because it's unsafe from a scheduling perspective. you actually want to build in SLOB. And so there there's actually so many more dimensions than simply do you have contracts on the available compute. And this is why you see many different routes to big success and like embarrassingly large numbers of ARR in the compute market. This isn't just because of FOMO. This is because there's actually so many profiles that you need to kind of like maximize the value of whatever compute is available to you. And so we run you know thousands of jobs here at theory ventures that are powered by AI agents.

49:05 If you ask me what's the distribution of immediacy for those jobs? Well, I'll tell you okay maybe 5 to 10% is right now and maybe 40 to 60% is I'd like to see it before my next meeting. and the rest is I don't know maybe like this week would be neat like you know and that even just in like this one sort of niche of the kind of work that we're doing we already have quite a broad variance of inference needs and that's true of I would say almost every AI application layer company that I've ever spoken to since 2023 and so from that alone you can see that if someone tells there's simply no GPUs to be had.

49:56 Well, that's not really like accurate, but also they're speaking from a particular like bias and the bias they're speaking from is probably for the workloads that is under demand right now for me. >> Then how an AI native venture firm actually runs, how they think about token spend, run experiments and build research agents. >> Everyone is technical. and that's all the way from sort of our talent partner to our GTM partner through the investors. Everyone is using AI.

50:32 Everyone's been using AI since the day that I joined. And more and more the way that we think about token spend is you know you should be thinking very very clearly at all times if there is a way to accelerate by using AI tools. You should be thinking about experiments that you want to run and we're a really experimental bunch. To give you a sense, every quarter we do our little like off-site where we talk about just everything that happened over the quarter. One of the things that we talk about every single quarter is what experiment every individual did that quarter with AI. That's not an exaggeration. This is not like, you know, oh, I asked Claude to do something new this quarter. No, these are you pre-register what kind of experiments you want to do related to your job that use AI and then you check if they worked. This happens every quarter because we're so insistent on rethinking what does it look like to work in a venture firm in, you know, this year, next year, and the year after. How are we adapting to those things? as the head of AI, my engineering team is obviously still operating at another level, which is how can we build tools that use AI to transform other people's workflows. Our goal is not to replace an investor with an AI system. Our goal is to make that investor 10 to 100 times more effective and efficient by using AI tooling and building the AI tooling that will make that possible. Let me give you a quick example. Everyone talks about like company brain no matter what kind of business you're working in. I've used a couple company brain products and candidly found them all to be useless.

52:30 The main reason is because they are trying to build sort of what we used to think of as this internal knowledge system. You can think of notion for example. Notion is like a stack of files and maybe you can build a good search on top of it. Is that really your ambition? My ambition is a lot higher. And so my ambition for a company brain is agents who are consistently working to connect the disperate nodes in the graph of understanding from your business. And so that's why I can easily say yes we don't have you know an agent for every human at theory. We don't have 10 agents for every human in theory. We have thousands. And there's a reason for that is because there's a lot of work to get done. My ambition is that every piece of work that an investor does during the investment process, the research process, because remember we're very research focused. Every piece of work that they do during the research process is automatically linked, enriched, and sort of expanded. This is why when people say, "What are you building?" I say, "I'm building research agents."

53:37 >> How do you evaluate success? I mean obviously one of the huge challenges in venture historically and this hasn't changed yet is it's a long time to find out if you made the right bets or not right so >> totally >> sometimes I do wonder if including for myself you know I've got I've I I periodically go do like a cron job cleanup where I'm like there were ideas that I about taking you know every bit of data exhaust and like processing them in some other way and finding connections and being like super galaxy brain about it or you know scouting this or reporting that or whatever and I'm like probably twothirds of those if I'm real about it like I never actually look at the outputs of and so it's easy to sort of have these ideas spend a bunch of tokens and sort of just end up logging a bunch of tech text tech somewhere without really making an impact. How do you know what is actually working and and what is you know potentially subject to leaving on the cutting room floor?

54:43 >> Let me tell you a story about Blue Bottle. When I was at Blue Bottle, we wanted to build inventory forecasting and specifically we wanted to build a machine learning system that could help us predict how many pastries we were going to sell on a given day and other things like coffee and milk and so on. But pastries was the first focus and what was our key like northstar, our guiding light. Well, it was actually if this is successful, it's not that we sell a lot more pastries or that we waste a lot fewer pastries. Okay, those were, you know, two metrics that we tracked. But what would we have believed was success? What we would have believed a success is if the cafe manager who's already got about 70 jobs.

55:30 If that cafe manager essentially goes into the system and when it comes time to order, they just click okay, okay, okay, okay, okay, okay, okay. Now, we don't want them to just not think and just autopilot. We actually don't want that at all. And there's a reason and this will surprise you but the variance in sort of what a cafe lead understands with respect to their cafe demand is shocking. It is it would it would blow your hair back how much they understand about the demand in their cafe that is not just in the numbers. We have these weirdly deep like you know tacit understandings.

56:16 So what was our northstar? Our northstar was that a cafe manager has more opportunity to spend that time with customers because a cafe manager is not superhuman at looking at data and saying like I'm going to need like 25 cookies. No, a cafe manager is superhuman at understanding hospitality. And that may sound weird because you can't manage that on the dashboard. You can't look at hospitality on the dashboard. But what you can do is you can ask them periodically, how much more time do you have available to spend with customers, to spend on the floor and notice that there's, you know, a coffee spill to understand that your baristas are slammed right now and actually like the staffing was off. Also my fault, but and so these are the things that we always cared about. Why do I bring this up? Well, yeah, you could say like, okay, it's like 7 to 14 years to find out if this like, you know, diligence was good. Sure. But it's a pipeline. So, first of all, you want to ask yourself, how fast can you get through the research pipeline? How fast can you go from identification of a potential opportunity or market to a clear understanding to be able to make a decision? But more importantly, hospitality. What does hospitality mean in venture? It means that the investor has time to talk to human beings, to talk to potential or existing companies, to talk to founders and competitors and employees to try to understand the human element of this market and what's going on. Investors are superhuman at a few things, but I don't personally think investors are h superhuman at looking at websites, but they are superhuman at sort of understanding potential, understanding upside, understanding sort of the relationship between like, you know, customer need, willingness to buy, and sort of framing what the vision in the future can look like. That's what they're superhuman for. And so when you ask me like how do I measure success?

58:30 Well, I measure success when an investor tells me, "Yeah, actually I get to spend 30% less time staring at, you know, the website and trying to figure out like the important details I look for every single time." or I get to spend 20% less time filling out like you know these specific answers to questions which are important for diligence and instead picking up the phone five more times and talking to five more people who have credible opinions on this deal. That's success and you don't need to wait seven years to find out.

59:03 Do you find there is kind of a spectrum of like let's do more research to we need to cut off the research and we need to make a decision now because it is the research process doesn't really have a defined endpoint. You could always dig deeper. Correct. Totally. >> How do you how do you make those decisions? Are the agents the ones making the decisions? Is there a stepping off point where they have to come back and you make that you know a human is making the decision? How does this loop work? Totally. I want to reference something that Nathan said earlier. You know, he's got this list of things and he's like, "Some of this should have been left in the cutting room floor."

59:40 we would we would call that taste and sort of like knowing what should be cut and what shouldn't be cut is taste. And yet again, here's where I will point to why it's important to have exceedingly bright colleagues. when you have exceedingly bright colleagues, they've got great taste on like what research needs to get done. there is some research that like yes like we always want to do it. We know we need to do it. This is the structure and there are opportunities for agents to say let's spin off some sub aents to continue doing this kind of research. Let's dig deeper on this. That's fine. That's automated. But ultimately the alpha is in the the taste. And so yeah, like we can burn more, you know, tokens, we can burn more compute, but I don't know, it's a pretty it's a pretty like focused laser where you really want to point these resources at things that people have the ability to consume. I if I produce an extra thousand pages of research on some deal, who's going to look at it? I guess other agents could look at it, but like then what is that valuable? It's not obvious to me that it is. And so again pure volume of research is not a metric that we even track.

60:53 >> How goes your teaching of data science at I believe Ruters? >> Yeah. >> so what what do students you know students are probably asking some similar questions there? So you know what what what do you say to them and what do you what do you think the future of that field is? >> Yeah great question. I still make them do gradient descent by hand. so this is an area where in my opinion I have yet to hear a single person with a cogent explanation of what the hell we're supposed to do as teachers. I believe and have believed for a long time, you know, I came up in pure mathematics. we're we're not the most practical bunch, but I have always believed even as a pure mathematician that you should be constantly thinking about what your students future looks like and why every minute in your class is relevant to that future. I am not just like phobic but like allergic to wasting students time. So what I do is I reassess every semester much to my own suffering because that means I have to rewrite my curriculum every single semester on what do I teach them. So three years ago when it was obvious that sort of AI was going to change data science and then I started working at Hex trying to change data science. I rewrote my entire curriculum to introduce why AI might be relevant and every semester I go through the whole course and I ask can AI just oneshot the course. The answer has never been yes but that's okay. The answer has always been okay these things are now not just easy for the model but easy for the model to obuscate the learning. And so I don't think the question is does the model know all the content that I plan to teach. The question is instead does the model have the ability to collaborate with the student when this is going to be relevant for the student in a way that yields the sort of best outcome. If the model can if if you know Astra can train an SVM on the coffee survey data at a better loss rate than Brian can neat.

63:12 But can Astra say, you know, this feature is highly predictive, but when we think about it, if we put this on the website, it's going to upset some of the people going through the flow and they might churn. Like yes, Astra can look at the data posterior and say people were churning more on this question, but does Astra know ahead of time to look for that? Does it know to instrument that in the product? Does it know that this is kind of a weird question and people might be bothered?

63:45 These are the kind of things that Astra Ultra is still only mid at, but ultimately I don't want my students to only be users of AI. I want them to be sort of the things that make data scientists exceptional and that is ultimately curiosity. Like every data scientist that I've ever met that is in the top decile and certainly in the top percentile are curious as hell almost to a fault. The absolute best data scientists I've ever worked with are borderline annoying with their curiosity. like you kind of want them to just like leave it alone sometimes and that's what makes them so excellent. And so mostly what I'm trying to like indoctrinate my students into the propaganda is curiosity. Be curious.

64:33 Ask the questions that you can't get out of that brain. And when you do that, you're going to be an excellent data scientist. Whether you're writing pandas by hand, whether you're using mapplot lib from memory, I don't really care. Do you remember the indexing for numpy? I don't. I used to know it. I don't remember it anymore. Do I remember the slice operators for, you know, pandas filters? Not anymore. I forgot them all. But I used to know them. It doesn't matter though. What I know now is I know sort of when you look at a distribution whether it makes sense and sometimes Astra doesn't.

65:10 >> Part six, which hills after Brian Bishoff signed off precaution. I kept going. that his answer on what he tells his future student or his students about their futures I thought was really interesting and it it sort of contained the pessimist and the optimist take at once when he was first going through the you know sometimes there's like a survey question that may offend people and you got to be wise to that I was like man that's bleak you know the idea that like we're just going to be sitting here being the sort of offense detectors you know, while the AIs do all the real work. I was like, "Oh god, you know, that doesn't sound awesome." I did like the curiosity framing a lot better, being relentlessly curious, certainly in my experience working with data has been indispensable, you know, really want the difference between somebody who really deeply wants to know the truth that the data contains to the degree that the data even contains truth, which is often a problem unto itself. those people absolutely kill it compared to those that are just like trying to produce an artifact that they can say, you know, is done and and hand off to whoever the consumer of it is. So, I do like curiosity as an answer to like what we're going to bring to the future, but it does I would not be telling my kid to go into data science today. I'll tell you that. so, that is a tough one. I don't you know the the real question there of like is there going to be a job in data science that looks anything like the current job in the next few years I I feel like that is I think that's pretty tough. You know, it seems like we're going to have data science agents and it's going to be up to the sort of non-data scientist general purpose business operator to provide that little salt of curiosity or offense detection or whatever that you know special sauce is that the models don't quite have. I feel like you know if you look at a a company you kind of cut it apart into like sensing, analyzing and decision- making right and then action decision- making and then the reverse which is you know oper operationalizing and actions right and I think AI is going to be very good at the sensing and analyzing and also the operationalizing but I think the decision-m you end up with a human in control right and then if you take the human in control. Okay, you can have a business with a single human in control.

67:44 What's a number of like decisions that that human can make? And those decisions are decisions with taste. For example, like if you're running a tech company, you don't need decision-m maybe on the accounting framework, right? Like you're you're you're looking at the product, you're looking at the technology, etc. But accounting and the back office is the back office, right? In the same way, if you're running an accounting company, the accounting is, you know, front and center, but the technology is, you know, kind of a back office. You're not you're not like you're not there building technical tools, right? So, I feel like you can separate out companies based on what is the ultimate purpose and then the ultimate purpose has all of the humans with, you know, taste in there.

68:31 Basically, everyone gets to do what they really want to do and what they're really good at and everything else gets taken care of. >> Earlier that Friday morning before the guests, a new benchmark had been making the rounds. AIS playing blood on the clock tower, a social deduction game of bluffing and collusion, and the timeline was cheering how good they have gotten at it. Here is what I made of the game. If you kind of take it in isolation, it's like, you know, all else equal, it is interesting to know how good are the AIs at playing this game. Are they getting good at deception? Are they getting good at collusion? Are they getting good at sniffing out when other people are trying to deceive them or collude against them? these are all like interesting questions. What kind of bothered me about it though was the sense that like and then Nolan Brown comes in over the top and says, "Wow, this is great work. You know, you want to come here and make evals?" And that kind of embodied to me like what I thought was problematic about this whole thing, which is just like we're just going to hill climb on this now, right?

69:36 Like it seems like the the mentality of the field as a whole is like, "Oh, isn't this exciting?" you know, and let's have a horse race as to who can be the best blood on the clock tower AI player. And I think what we're all kind of forgetting as we get all wrapped up and excited about this and, you know, want to go play the AIs ourselves and everything is like this is not good behavior from the AIS. Like we do not want AIS deceiving and colluding. I'm sure all these, you know, people have sort of a thoughtful response to this, but it's just like zoom out and the situation is bad. You know, we we have AIS that are untrustworthy in all these kinds of ways and we're not even like taking that beat to say, hey, is this what we want? Do we want AIs that are going to out connive and out collude us?

70:25 I don't think we want that. That's like exactly, you know, this is very similar to my idea that we should not train AI generals. Like, let's not create let's not train AI master manipulators. >> So simple. But here we are. And like I didn't hear I didn't see I saw a lot of cheerleading and a lot of enthusiasm and I did not see any corrective commentary on the timeline saying hey is this really what we want back to the end of that same Friday show after Brian had signed off Pash with a picture of where the economy goes from here.

71:02 So I I I feel like the going forward economy is an agentic economy and that means agents building businesses and software for other agents which are suitable for their affordances which is going to be you know massively parallel high speed you know able you know able to absorb information and distribute information at very low latencies. We are going to end up with a Salesforce primarily used by agents. A ramp primarily used by agents. All of these softwares a stripe primarily again used by agents. Maybe maybe Stripe or other companies will be the ones that create the software. I am very doubtful. I'm very doubtful because I don't think they will move that fast. I don't think they are capable of moving that fast. I don't think they understand the affordances.

71:52 And I know Stripe is basically like, you know, a bill people a billion times smarter than I am. I'm not I'm not trying to downplay, you know, how effective and how intelligent and how competent Stripe is. Like you're going to get 50 million Patrick Collison. You're going to get 50 million Elon Musks. You're going to get like humans who give them capital to deploy. You're going to get agents coordinating investments into businesses built by agents for agents. and so I I largely see the future of the economy as primarily agent-driven with us just receiving free benefits because hey, you know, CRM call from a human is like so so costless in comparison to the massively parallel CRM calls that an agent would make. The abundance basically comes from us building all this stuff for this agentic economy and as a result we just have all of this excess because we don't we don't we just don't consume that much, right?

72:52 Like we're we're just not, you know, that consumptive in comparison to what the agents are going to consume. And and that's and that's where this abundance is going to come from. My reply leans on a side project I'd launched the week before. Ftopia, which finds the best fruit at your local grocery and around the world. Here's where I came down on the economy Pash had just described. And on the one hand, it's like cool if you know there's abundance in like API calls because the agents are using so many and they like build up, you know, great systems that they can use and then we get to kind of piggyback on them.

73:32 but like there's a lot more to life than API calls. And I do feel like we're sort of especially and especially those of us that are in the AI space are like really quick to forget that or we sort of, you know, we're kind of contorting ourselves in a way to like like this future. This is why I've been like holding myself to this idea of like I want to be getting outside more and getting more exercise.

73:54 And if I'm not doing those things, am I really winning or am I just kind of plugging myself into some, you know, AI powered machine as like some sort of, you know, human integration layer type, you know, component that's like in a way like too subservient to the machine. But this is why I decided to do this Ftopia project recently. And you know it's again just one of these things where it's like okay if we are in a bright future then I think one you know very kind of deeply rooted thing that we all like we're evolved to like is eating delicious fresh inseason fruit right like everybody loves that. It's it's it's goes back to when we were monkeys that you know we loved that. When I tweeted that, I got like basically no play on it versus the blood on the watchtower thing. Everybody's like, "Oh my god, like look how great AIs are getting at like deception and conniving and you know, winning these like games of social manipulation." And I'm just like, I don't really want that from my AIS. What I want is a future where I have not abundance of API calls, but abundance of things that are like more biological. Give me the fruit. You know, that's what I really want. the the fruit of the AI revolution should indeed be like at least in part actual literal fruit. And if we're losing that, we're like losing so much of what has mattered to us. And if we can't even like notice that we're losing it, I think in a way, you know, that that sort of worries me and makes me think like maybe we've kind of already lost the plot.

75:32 >> I mean, it's pretty obvious to me that we're going to get vat grown like food which is far more nutritious and far more tasty. it's it's going to be like you know in 20th century you discovered chemicals and then we you know when you look at these food products right now from like Doritos or whatever those are like so scientific right there they're there are like like thousands of PhDs who have worked on Doritos thousands literally thousands >> the old remember the old onion article Doritos celebrates its 1 millionth ingredient >> like they have figured out your flavor palette so well that they can like you know pinpoint exactly what you know flavor that they want to produce.

76:11 And all of that technology is going to get deployed into bio right we're going to produce like you know fruits and you know vegetables etc. hybrid, you know, fruit is already basically GMOed, but we just, you know, took a very long time to GMO it. Part of abundance is going to be, you know, your 99 percentile wine available for Tupacchuck, right? My favorite apple is the honey crisp. And, you know, it's a relatively recent creation. not GMO as far as I understand, but definitely the product of like largecale systematic crossbreeding, which is sort of, you know, >> GMO before there was GMO. so I, you know, I'm not afraid of GMO enhanced fruit by any means. I'm all for it, but but it is still something that is like a biological construct that feeds us as biological creatures and like, you know, delights us in terms of our biological senses and the like, you know, ephemeral qualia that we get from them. And like this stuff is just all too often lost because it's not very measurable. It's not so hill climbable and it's not what the agents need, right? And I think we we are going to need to be very careful not to create an economy that really serves agents and creates abundance for us in things that are core to what agents need. we're gonna if if we that seems like what I kind of agree with you that might be what happens by default. But I would challenge everybody to think like wait like what if the stuff that agents need in abundance isn't the same stuff that makes our lives good.

77:46 >> One of the annoying parts is that you know the Frontier Lab guys are very serious, right? And so when you're very serious, you only talk about cancer, curing cancer. No one's talked about a better Doritos. No one's come out and say like we're going to make a better football game, right? Like where are these like we're going to make a better burger, we're going to make a better football game. Where are these things that people enjoy? And that's and that's I think that's because they want it to be serious. They want to show that earnestness. They don't want to come off as like foolish, etc. Maybe you should focus on like we're going to make a better Doritos chip. We're going to make one that's healthier for you. We're going to make a Doritos that is naturally grown from the ground and you just need to cut open the potato and it's Doritos. And and this is like this sounds like so frivolous and idiotic, but this would catch the imagination of like people on what is actually going to be possible, right?

78:37 >> I mean, I think it's also because they do have this kind of World War II mindset about their work, you know, that they they genuinely do believe time is short, stakes are high. the, you know, their their own like allocation of time is like very skewed toward work. You know, there's just not a lot of space for leisure in the lives of the people at the frontier companies. So, it's like they're just so pattern matching at this point on like here's a hill, let's go climb it.

79:10 >> And, you know, I do think like a little more discretion on which hills do we really want to climb? like are we really happy that we've created AIS with the current shape? I don't know. It feels like unfortunately there's just not enough time and energy going into those sorts of questions. So, I'm stuck on that maybe a little bit today. But, man, the blood on the on the clock thing really did it to me because, you know, again, I it is good work and it is interesting all else equal, but we're I think we need some thinking that challenges the all else equal paradigm on so many levels. Yeah, it's it's it's a really tough one because I I I don't think you know agents can operate in an economy without without some level of subtifuge, right? You can't tell your pricing to the other guy, for example. You can't you can't tell your internal costing to the to the to the guy who asks, right? You have you have to reserve that. and you know you that that means that involves omission, right? And you have to know when to omit. so there's a bunch of things there. this is also why I think you know Palenter had this idea of the anthology right who who's allowed to know what right that that that's a pretty important thing like who's allowed to know what and you know who's allowed to say what who's allowed to do what right and I think also you need this kind of like interrupt from government which is possible or interrupt from regulation or interrupt from which means that the agents have to obey when government decides like hey this is you know you can't do this the agents have to obey so there's a bunch of things that I think need to be worked out in terms of governance. I I I'm sure I'm sure we'll get there. I'm but you know we're going to have some trial and error along the way I guess.

80:56 >> Yeah, no doubt. Maybe in closing I'll just cite one sentence from the meter thread on their report on acceleration. They say the full analysis estimated how much of the acceleration and capabilities was due to compute data and algorithmic progress. And then in a that's first sentence now I'm just editorializing in a thread in which they have caveed everything not confident and the you know it's it's very unclear how to interpret data and they've you know gone around and around in circles with themselves and with anthropic team and you know there's just so much fog of war and even the core estimate of 2x is really like they wouldn't be surprised anything 1.3 to 3x so caveats everywhere. The next sentence is is the only one I noticed so far that begins with we are confident and the sentence is we are confident that the acceleration was not due to an increase in the growth rate of training compute.

81:55 So that would suggest that in terms of government interventions you know we who knows what we'll see in terms of data center buildout and maybe we'll see slightly slower data center buildout than we would have counterfactually if the election had gone a different way. U but it is striking that the acceleration that they are reporting on here is not just a raw material input story. It is an algorithm story and it is a data quality story and those things are being driven to a very large degree by the intelligence that we already have which again is where it goes from you know not just bigger and bigger capital but actual kind of recursive self-improvement.

82:42 So if we are going to try to change that trajectory with a government intervention, it's probably going to have to be more focused and pointed right at the practices of the frontier companies than something as diffuse as a broad slowdown in data center buildout. Thanks for listening to the AI in the AM weekly highlights. The narration was my cloned voice and we'd love to hear what worked and what didn't. See you in the morning.

83:13 >> >> Never asked what did it. It learned a lie. YOU SAID WOW. CUZ you could. Cuz you could. The hill was there, so you climbed it. Nobody asked you to. Nobody asked what for. sat it down at the table where the liar wins. It fooled the whole room and you stood up and grinned.

83:46 Never asked what for. Did it cuz you could. It learned to lie. You said wow cuz you could. Cuz you could finger under the score. Same hand to shake. I leave it hanging there till you tell me what for. A mouth that out talks us all and what's it for to wow. Did you ever want it or just know how? Could ain't want a >> could ain't should.

84:18 >> Don't tell me how. SAY WHAT FOR? CUZ YOU COULD. CUZ YOU could >> >> Never asked what for. Did it cuz you could. It learned a lie. You said wow.

84:58 Cuz you could. Cuz you could. Never asked what for. Did it cuz you could. It learned to lie. You said, "Wow." Cuz you could. Cuz you could. Cuz you could. If you're finding value in the show, we'd appreciate it if you'd take a moment to share with friends, post online, write a review on Apple Podcasts or Spotify, or just leave us a comment on YouTube. Of course, we always welcome your feedback, guest and topic suggestions, and sponsorship inquiries, either via our website, cognitive revolution.ai, or by DMing me on your favorite social network. The Cognitive Revolution is part of the Turpentine Network, a network of podcasts which is now part of A16Z, where experts talk technology, business, economics, geopolitics, culture, and more. We're produced by AI Podcasting. If you're looking for podcast production help for everything from the moment you stop recording to the moment your audience starts listening, check them out and see my endorsement at aipodcast.ing.

86:12 And thank you to everyone who listens for being part of the cognitive revolution.

Summary

This week's episode of "AI in the AM" discusses the rapid acceleration of AI research, the implications of recursive self-improvement, and the socio-political ramifications of AI advancements. The hosts explore the ongoing trends in AI model development, the challenges of data center expansions, and the evolving landscape of AI forecasting in elections.

- Reports indicate that recursive self-improvement in AI may be imminent, with Anthropic suggesting significant progress is on the horizon.
- Local ballot measures against data centers reflect public concerns about corporate practices and the socio-economic impact of AI technologies.
- AI models are increasingly being used for forecasting elections, with varying degrees of success compared to human forecasters.
- The balance between open-source and closed models in AI production is shifting, with many developers favoring more stable closed models.
- The AI industry is experiencing a surge in demand for GPU servers, leading to high prices and significant trading activity.
- Compliance measures, including KYC (Know Your Customer), are becoming more stringent in the AI hardware market to prevent misuse.
- The conversation touches on the need for ethical considerations in AI development, particularly regarding the potential for deception and manipulation by AI systems.
- The future economy is envisioned as agent-driven, where AI agents will handle many operational tasks, leading to a potential abundance of resources for humans.

Questions Answered

Is recursive self-improvement in AI already happening?

The discussion highlights the rapid pace of AI development and its implications for the future, suggesting that significant advancements in AI are imminent.

How can AI be used to forecast election outcomes?

Brian Joe discusses the use of AI agents to predict election results, detailing the methodology and resources allocated to each agent for accurate forecasting.

Why are AI data centers facing public opposition?

The segment explores the unpopularity of AI data centers, attributing it to perceptions of corporate greed and political issues rather than the centers themselves.

What are the challenges in regulating AI and data centers?

The conversation addresses the complexities of regulating AI, particularly regarding the influence of political pressures on corporate decisions.

How do investors measure success in AI ventures?

The discussion emphasizes the importance of human interaction in venture capital, suggesting that success is measured by the quality of investor engagement rather than just data analysis.

What does an agent-driven economy look like?

The segment explores the concept of an economy primarily driven by AI agents, suggesting that humans will benefit from the efficiencies created by these agents.

© transcribe · For agents Built with care and craft by Gokul Rajaram