Cambridge AI Safety Plan, Instinct CEO on Invest Like the Best, Zuck Hires MongoDB CEO | Diet TBPN

0:00 The most viral video I saw over the weekend was the the parody of Dario Amade on SNL that felt like it really took over the timeline. Martin Casado said honestly downplayed how ridiculous the rhetoric and broader situation is. It was a funny clip. There were a bunch of interesting nuance. I mean, first it's like, okay, clearly the the the AI rhetoric and the public discourse is getting to like a a new height to the point where like a lab leader of a private company can be can even be parodyied on SNL because the audience understands who that is. Yeah. so it's interesting to see that like okay, yes, like Daria's done enough on Oprah and CNN and and NBC and ABC and all the major news networks to be relevant to the broader audience. And because it only really works like you can't like they could never pair.

0:49 There's a lot of people in tech that we know that could wouldn't work as you know SNL punching bags, but he's made it. He's made it. >> It says a lot that the like the sweater vest thing is iconic enough to be like a part of a character. >> I wonder if he'll stick with that or switch it up because it's not it's not his leather jacket yet. It's not his Hawaiian t-shirt yet, but there is such value into leaning into like a consistent thing. But I don't know.

1:13 We'll see. The interesting like wrinkle here is that Dario is at the point where he is enough of a name that they can do this bit and and make jokes and it's it's relevant to the SNL audience. Obviously, it went way more viral on X than I I think it probably did on YouTube. But when Michael Chay is introducing the segment, he sort of stumbles over Daario's name a little bit. Like he's reading it from the qards and he says like and this is Daario.

1:39 And then he pauses and it seems like maybe Dario Amade is not fully a household name in the SNL writers room yet in the sense that they were like, "Okay, who is this guy? Let's look him up. Okay, get him on the Q cards." I don't know. Maybe I'm reading into it too much, but it felt like this was certainly like a crossing the chasm moment, like the first time everyone gets to know Daario. but very very funny and it really illuminates a little bit of like the the complexity of the discourse, the irony of saying, you know, somebody's got to save me from me basically >> going going pretty hard on him in some ways. That being said, he's always like incredibly fascinating to listen to and he has incredible clarity of of thought and he does at times he'll like pause, you know, in in a way that would maybe scare normal people a little bit.

2:28 but the thing that I think he can do that will have the biggest impact is >> I agree. >> no, I actually like the I I like the the whatever kind of shawl sweater thing that is. >> Isn't it Is it Brunella >> probably? Yeah, >> I think someone I think someone clocked it, right? It's Brunello. So, you know, it's nice. It's quite a luxury. And I feel like a power tie, some shoulder padding would go a long way. Anyway, >> the biggest thing he can do is to stop smiling when he's talking about really bad devastating things.

2:57 >> Yeah. >> and so I'm sure that that's something >> Yeah. There's something to be learned from like the way like Fed chairman speak or the way you know the the head of the FDA or the head of the military speaks. We've talked to some of these folks and like they're very austere, very serious, and our show is very lighthearted. We kind of have to like click into a different mode when you're talking to somebody like like a Michael Katzios or someone who's like dealing with the weight of something big and and and puts up a wall that is it is a it is a performance, but it serves a very valuable purpose, which is like I take this very seriously. Or you can go the opposite way, the the Mark Zuckerberg way and rip it and just be like, "Yeah, I'm having fun with AI."

3:40 Which is honestly like it feels like people are like, "Yeah, I kind of want that energy a little bit. Just give me the white pill." But I don't know. >> Yeah. And again, I I don't think that Zach's approach is at all appealing to the people actually pushing the frontier. No. Right. No. like no, no one's going to no one's going to take no researchers like take >> there are some research >> dating dating back to basically Demis being like, you know, that moment where he realizes Zach is just as excited about VR as he is AI, right? Anyways, >> I I I I hope that >> maybe VR will be bigger than AI, Jordy.

4:16 Maybe >> one we wouldn't like that. >> I would love that. >> Wouldn't you like that, John? >> I would love that. but yeah. Anyway, so so from SNL headed straight into dinner with the president >> Sunday. I mean, he wasn't actually at SNL. but yeah, it's like you wake up to that on on Sunday morning. And you have to imagine that Trump sees it, but Trump has been both he's hosted he's hosted SNL multiple times and he's also been made fun of on SNL for years. like when he gets elected, they basically hire an actor to come and be him for four years, for the entire 10 year because they're going to be doing the cold open with him like basically every other week. and so maybe there's a little bit of camaraderie. They're like, "Hey, we've both been through this, Dario. It's not so bad. It's actually kind of kind of funny." Or or maybe they're both like, "Yeah, the news really is fake." like we got we got >> but yeah as as as critical as it is at the same time I think it's something that all founders should aspire to right when you're starting your company and investors say well where do where do you see yourself you know where do you see yourself in the company and >> getting skewered getting on the biggest stage perhaps perhaps >> Andrew Curran says President Trump has personally invited Dario to a private dinner tonight that was yesterday >> so that happened on Sunday night >> this will be the first time they've actually met oneon-one big chance to patch things up or crash the plane into the mountain.

5:41 >> they're actually running it back tomorrow night. Tuesday night there's a dinner in DC for all the AI leaders with the White House again. So I mean the pace of this stuff is really really really picking up. There's there's meetings every every week now basically and yeah I mean there's lots to work through and no one is going to get to put their their head in the sand. Not even the comedians in SNL. will have to chime in as well as the politicians and the lab leaders. and so yeah, despite Daario being exclu excluded from it seemed like he was excluded from that White House dinner with Cinping and and that basically every top tech leader was invited to he was noticeably absent.

6:19 President Trump, you know, followed up with a one-on-one dinner with Daario to hear his side of the story. And I wonder what type of leaks or reporting will come out of that. You would think that that would be going on today. Maybe there will be some news or maybe they'll put out a joint statement. Maybe everything went really well. But I mean both. >> Or joint. You could see Daario sign up for Truth Social >> or UFC courtside or ringside.

6:41 >> Ringside be pretty electric. >> UFC. >> Yeah. >> With with the Dawn. I want to see Dana White Daario. >> Yeah. >> Trump. >> Yeah. >> Just squatted up. >> You think he'd be doing slap boxing before, too? >> Oh, yeah. For for sure. You got to do the full suite if you if you go all in. Anyway, there's also interesting timing that I don't know if this how timed up this was, but every day there's another proposal for how to manage AI safety.

7:07 this one feels like the most formal concepting exercise from a group of top AI researchers. it's published through the University of Cambridge. You know, Sam was talking to was it wasn't Axios, it was he he was interviewed and he was saying that like maybe we're gonna sit down and get beers basically. You remember this whole thing? Why don't you guys just sit down and have some beers? That was the that was the proposal. and he was like, I don't want to leak anything that's going to happen, but we are thinking about thinking about it. And you know, he's like, you can't just >> personal agents are talking about getting together for a beer.

7:42 >> Yeah. but it was very funny framing, but this is like the start of that, a start of more collaboration between the labs, between the academic community. And what's interesting is like they go a little bit further. You know Tyler over there is his criticism has been like everyone comes out with an AI safety proposal and it's basically always like we got to talk about it and this is a little bit more concrete. So the first one is they want to set in place the ability to monitor the level of AI enabled R&D at labs. So what percentage of the code, what percentage of the dollars, what like give us firm metrics on how much of the AI R&D is being done by artificial intelligence, anthropic, open AI, they've all sort of like mentioned this in blog posts that it's happening, but there's also this like vague jockeying because it's better to be further along on the RSI curve for financial reasons. So there's not really an Apple st like no one really feels like they have a strong handle on like whether there's more RSI going on at OpenAI or Anthropic. Like it feels like they both have very different metrics.

8:50 Like OpenAI says like we have a intern level and then Anthropic will say like we had Claude spin up a thousand ideas and like you can't really apples to apples those. The proposal is like let's get some metrics in place so we can actually monitor this thing. then they also have again it's sort of like we need to talk about it but they want the government to create firm plans for responses to potential theoretical bad scenarios. So like if there's a big cyber attack and an agent goes rogue and brings down the internet and hacks into a bunch of stuff like what did what will the government do? What's the equivalent like FEMA? Like we have a response for like there's a big hurricane hits town, you bring in people, there's food, there's shelter, there's like a plan and we don't really have that for like the internet went down because like an agent took it down. same thing for biocurity. they have a couple others even like economic dislocation that type of thing. so they want they want the government they're not proposing the solutions to those yet but they're saying like the go this is where the government should be prepared. So, I don't know. it's a good read.

9:59 >> Did you see that, China has places you can go to to simulate being in a disaster like high s, you know, super high winds and and flash floods? >> Flash floods, things like that. >> Should we have like AI disaster simulations where you can be in a data center that's going rogue and kind of work to to shut it down with friends? Yeah. just a kind of like live action roleplay through it. >> I like that. I like that. I I'm kind of treating Modern Warfare 4 like that. I feel like that's getting me enough cycles of like the simulation of, you know, you're already play against bots who are already shooting shooting AI basically. So, getting some drills in.

10:42 >> Yeah, I know it's it's a joke, but I I wouldn't actually be surprised that a a future firsterson shooter or like one of these open world games is centered around the Terminator scenario. There was there was a game there was a game I forget what it's called but yeah like you drop in with three people and you're on this like planet. There's one that's starship troopers themed so it's like the bug planet and then there's another planet that you can drop into that's like the Terminator planet and you're going up against like they basically like ripped Terminator's intellectual property. Like they look exactly like T.

11:14 >> Why is Tyler not back in school doing the best internship in history? He turned it down. >> I turned it down. He actually turned it down just to grind for this. >> Yeah. >> Things will never be chill again. The doomers who shape the AI safety freakout. The Wall Street Journal has an article here about some early anthropic employees who are considering buying remote land in the United States to ro to relocate to if AI goes seriously wrong. They're getting bunkies. They're going bunker mode.

11:42 Yeah. Employees also trained doomsday scenarios and preparation plans in a private Slack. While anthropic workers have their own office space inside Berkeley's Constellation AI safety community, the wider safety scene has discussed iodine pills, remote islands, and electromagnetically shielded desert bunkers. >> This is like an interesting example of like an actual answer to the to the Teller count question, which is like if you're if you really care about Doom, why aren't you making financial >> decisions? cover up, get a lot of debt, and buy a buy a bunker, buy remote land with a massive mortgage.

12:12 >> Yeah. Cuz usually the answer to the the doomer thing is like, well, I'm not going to be able to get to my bunker because the, you know, bio terrorism will already have killed me or something. >> This is an interesting >> instantiation of this. >> The thing that always the thing that always just like breaks down with these scenarios is like, you know, every every every guy has like sort of played out the scenario in their head, right? And where are you going to be best off? In some like remote area where you don't know any people that actually live there, right? Let's pick a let's pick Wyoming, right?

12:46 >> You really think that the world is like falling apart and you're just going to head over to Wyoming and set up shop in a community that knows you've been building your bunky there? Yeah. Right. And and they're just going to be like, "Oh yeah, like there's a bunker over there. It has everything you need in it to survive." And you know what? But but you know, they paid us to set it up years ago and and and let's not we shouldn't we shouldn't just go occupy said bunker for ourselves.

13:10 >> So the prepper community has a term for this. They say don't be a loot drop. >> It's a loot drop. >> You're a loot drop if you do this. If you go and set up a really luxurious bunker with a bunch of supplies, but you don't actually know the community, you're not integrated into like everyone else. >> Yeah. The fabric. So ideally, it's like you want your team and everyone else to all be in it together and so that you know there's no there's no squabbbling, but yeah, you can just set up a loot drop and then everyone's like, "Oh yeah, I installed a crazy safe over there.

13:39 Let's go get that as soon as the the the crisis happens." Yeah. I mean, there's a whole bunch of fun game theory you can play through there. I would I would personally I think we need to regulate because I would personally I would hate to be in a self-defense situation with with a bunch of rogue agents. And imagine if all the Whimos went rogue and we were just forced because of self-defense reasons. >> You and all your best friends just drive around and just have to shoot them out of the windows of our cars like all day long. Like that would be that'd be a nightmare.

14:07 >> That is a nightmare. You would never you've never even >> if I was in a self-defense situation I would do it but it would pain me. >> Totally. >> Totally. Let's talk about some personal agents. >> Okay. What? >> Noah Shin jumped on Founder of Instinct, broke his silence finally with none other than our friend and legend Patrick >> Oanosy. Patrick pulled out the highlights. He said, "A list of surprising and mind-boggling stats from this conversation. 1 billion a year in transaction volume. Still invite only."

14:38 I can't remember if that's total. I think that's actually total volume to date. I don't think that's like run rate volume. growing 10% a day. >> I read that as $1 billion of annualized that they are at $1 billion of annualized transaction volume. Yeah. But I guess it's maybe even higher. I don't know. >> Yeah. >> Yeah. Interesting. >> growing 10% a day. Spent zero dollars on marketing. they have this, you know, pretty amazing flywheel where people post screenshots all the time.

15:11 That's doing a lot of marketing. A lot of the but a lot of the screenshots are like horrible accidents, but maybe those actually drive curiosity and like people being like, "Well, I'm not going to make that same stupid mistake and the service is probably getting better." You think those actually help? >> I think those help. People are just curious. >> People are curious and they're like, "I want to see it. I'll be safe if I won't give it my login details, but I'll still >> No. And it and it and it can't be said enough."

15:36 >> Yeah. >> A lot of people within tech have experienced AI doing things for them, like making a product. Yep. >> Most people outside have not experienced that yet and it's just a magical thing. >> Yeah, totally. >> 40% of users shared a personal credit card with Instinct within 3 weeks. Compute demand is doubling roughly every week. At 5 to 8% daily growth over the months it takes to bring new compute online. Instinct would reach about 100 million users. On Instinct's own AB test, it matches Opus 5 performance at a fraction of the cost. Noah spends about 40% of his time on compute. Buying compute at the last minute costs three to fourx more. Makes sense. Background work runs on setups that are 3x, 5x, and 8x more efficient on the same compute.

16:21 >> 50% of Instinct's transaction volume is travel. >> That's a crazy. >> Instinct plans to take a cut of what users buy. Take rates. Shopify is 2 and a half to 3%. Amazon is 15%. Apple is 30%. Noah's personal goal is to keep Instinct free for everyone for life. >> some small businesses run their entire back office on Instinct. That's interesting. users who connect at least one piece of sensitive information retain at about 80%. Just give me one piece. Just one piece.

16:53 >> Just give me one piece of sensitive. >> some >> Tell me your darkest secret. >> Some users send more than 90% of their messages. skeletons you have in your >> to instinct by voice. say an app makes 70% of its money from ads and 30% from actually selling you things. Agents could wipe out the 70%. >> But Noah thinks that 30% gets much bigger because people buy more when buying takes zero effort. this is a big question for me, right? Because already >> buying something on a Shopify store takes close to zero effort. Like the checkout is actually down to if I land on the website and I know what I want.

17:31 Yeah, >> I can check out within and and again, Shopify doesn't have an ad business that are not fun to shop for though. There's a lot of stuff, especially if it's like I was thinking about like cable management. I wanted to get some cable management stuff for my desk and like I don't really care about the brand. I don't care about the the flow. I just want like, you know, a sweet. I needed some like command strips, maybe like a little tray, like whatever like the modern solution is to like a couple cables, a power brick, like just go and get something that's well reviewed and like a suite that will figure that. What What kind of voice are you doing?

18:11 Instinct launched to about 200 friends and family. Invites have sold on eBay for around $300. More than 50% of Instinct's traffic doesn't run on iMessage. Instinct has proactively called Noah three times. That's like when when it's doing something and it and it has like a deadline. It's like Noah gave the example of like, "Hey, you really have to sign this like within the next 5 minutes. Are you available?" It'll just call you which is cool. >> raised one on 10.

18:38 >> Yeah. >> yeah, less than a month after raising >> two and a half. So Noah Noah is 23. So very very good summary. >> Also same colleges that I went to. He went to Nor Eastern. >> There you go. This is the whole Instinct pitch deck, by the way. And it's Christian Kyle saying, "My three-month-old son is now twice as big as when he was born. He's on track to weigh 7 and 12 trillion pounds by age 10." Very funny.

19:03 >> Such a viral post. This This is like This went super viral on Reddit. It goes viral like every couple months on Reddit and stuff. It's a It's a wild post. But I did want to set some like some comps around the Instinct growth numbers that are interesting. So, a billion dollars a year in transaction volume. We're not exactly sure if that's annualized or or to date, but it's kind of the same. for reference, it took Stripe around 18 months to hit 1 billion of annualized transaction volume, although Stripe also spent two years building in beta before launch. Instinct got to 1 billion transaction volume in closer to six months. But, it's important to note that with Instinct, the transactions flow through thirdparty cards. So, they're not making any money off of these transactions yet, but it's very clear to imagine where monetization might happen.

19:55 >> yeah. At the very least, they could just launch a credit like their own card. >> Yeah, of course. >> And then they're getting that interchange fee and they don't need to get they don't even they don't even need to get anyone to opt in. >> Yeah. >> And they might be able to >> And it's an easier pitch, too, because you're you're onboarding this AI agent. You're like, I've heard some pretty crazy stuff about this thing. Like, yeah, I'd actually like it if you used your card as opposed to like has basically a computer, a phone number, email.

20:19 >> Yep. Give it a credit card. >> Give it a credit card. >> And so I'm sure they're working on that. And when they do, they're not going to make 3% because you do have to pay Visa and Mastercard, like the underlying rails, but earning 1%, half a percent, quarter percent, even even.1% on, you know, they grew to a billion, you know, you're talking about getting into, you know, tens of millions of dollars of revenue pretty quickly.

20:43 And the team's just 14 people. and so you can sort of see why they raised a billion at 10 billion from benchmark sequoia co there's you know yes you have to believe that that they won't be you know killed by muse and that they will continue to scale but if if you know any of the growth rates sustained for any even a limited amount of time it's going to be a lot of transaction volume great business and you can underwrite it pretty safely and then there's still the there's still the off the the offshoot into Amazon world I I think because Amazon doesn't really have a a strong play there, but could fit in very well with >> Yeah, they have a lot of potential dance partners and ultimately things grow really fast because they don't just like grow really fast for no reason. No, right. So, >> no good.

21:32 >> other people, the chief economist at Apollo is warning that mass adoption of AI agents could trigger a bank run as they optimize user investments. Gary Gendler spent his last few years as SE SEC chair warning that the exact same problem will hit the markets. The pursuit of algorithmic perfection at scale destroys system stability. Personal finance agents making simultaneous optimal decisions could potentially cause a massive flash crash. Agent a causal coordination with no messages passed between them will show up in a lot of places. markets will get the big headlines, but the huge number of clever advisers making very similar choices will impact all of human society. Blah blah blah blah blah blah blah blah blah.

22:15 human laws were not written for this. They were not written for a lot of what is coming. >> You have to apologize. >> It is, is about to start happening very fast. credit to Satrini for basically being the first. >> That was a different thing, but yeah, >> I don't No, it's the exact >> It's similar, but >> No, it's it's idea of like there's a lot of aspects of our economy that are that like like businesses that work because of friction.

22:38 >> Sure. >> That friction in some ways will be removed when you have an always on agent that says, "Hey, you haven't touched this product in >> very long time. Do you still want to be paying for this?" etc., etc. >> I I don't think this is I don't know. My my banker will just hit me up and say, "Hey, I'm going to I'm just clearing this with you. I'm going to sweep funds into >> money market >> money market account or or T bills like it's not like like a good banking experience is already doing this proactively for you.

23:09 >> Yes. >> Now there's of course people that maybe they just are set up with like a simple checking account and they could be moving those into a higher yield account of some sort that aren't doing it and maybe the agent starts doing that. But like like fundamentally a great banking experience they're like doing this on your behalf. when some says, "Dang it, if ordinary people get fair deals and aren't ripped off, it's the end of >> Yeah. I Yeah, because I I I I wonder like the personal AI agents are taking off, but like we're still in like single digit millions of people doing them.

23:44 It's going to take time for people to adopt them and then they need to actually authenticate and add these things, link their bank accounts." And then presumably the first version of this will be the agent coming to you and saying like hey is it okay if I do this? Second order will be like I just did this. I know it's okay you trust me. And that relationship that trust I feel like that's going to take like years to build up and diffuse through. there there are still like there's still plenty of friction in the economy. I think I think it's going to take a while for people to to onboard the onboard to this fully. I mean people were it's the same it's the same idea of like when Shach came out people were like Google search is dead like Google no one will Google search Google's search revenue is going to go to zero like next year and it was like yeah it was a reasonable replacement but it took a little bit longer even though it was growing so fast faster than any consumer product ever incredible revenues blah blah blah blah blah it still wasn't that fast and so it's the same thing where it's like okay if this happens over like a couple of years like banks adjust money market funds adjust like it's not the end of the world potentially, but we'll keep monitoring it. Anyway, let me tell you about Railway. Railway is the all-in-one intelligent cloud provider. Use your favorite agent to deploy web apps, servers, databases, and more. Well, Railway automatically takes care of scaling, monitoring, and security. Zuck, >> what happened?

25:04 >> We believe super intelligence will create significant new opportunities for all people and businesses. Meta already serves billions of people at scale and helps hundreds of millions of businesses reach customers. Today we are starting the next major pillar of our business, Meta Enterprise Platform to help businesses use AI to grow and transform in new ways as well. Meta Enterprise will use our strengths that few other companies have advanced models, leading agents, large scale infrastructure, and years of working closely with many businesses. Initially, we will focus on bringing our full technology stack, including the Muse Agent, Meta Business Agent, Muse API, Muse Code, and more to businesses and developers to help them grow. to lead this effort. I'm excited that Shiron CJ Desai, former guest of the show, will join Meta as chief enterprise platform officer reporting directly to DOC. CJ is an experienced enterprise leader with a track record of building full stack software and delivering results in AI, infra, business applications, and security.

26:00 blah blah blah blah blah blah blah blah. anyways, so here's how I think this is going to go. I think that they created this new business unit. Mhm. >> They still have a lot to prove on if they can >> Sorry, if you scroll down, there's some wild comments. >> >> anyway, continue. >> anyways, they created this new business unit. They're talking about apps and agents and all this stuff. I think what's really going to happen is they're going to do big compute deal.

26:26 >> Mhm. >> And then and then it's going to look like the this sort of unit is doing it. It'll quickly go from let's say 0 to10 billion a year of revenue. They can sort of use that. They don't have to necessarily break out where that revenue is coming from, but they can say like our enterprise platform is is gone from zero to 10 billion in >> Yeah, this was like the joke semi analysis was was sort of trolling with around like oh like all the growth in Google Cloud Platform is selling TPUs.

26:54 It was like a little bit more nuanced than that. Right. but there's a bunch of ways to like wrap up revenue into a into like an enterprise platform that could be tokens, could be doing inference, could be selling compute raw bare metal deals, could be could be actually selling services. we'll be interesting to see where it goes. Leave us five stars. Apple podcast, Spotify, sign up for newsletter tbn.com. We'll see you tomorrow for big event. Goodbye.

27:22 >> Can't wait.