transcribe

Why Nobody Can Stop AI

2 and 20 · 22m · transcribed 3d ago
More from 2 and 20 Business
𝕏 Share ▶ YouTube 📥 PDF 🤖 .md

Transcript

0:00 On August 2nd, 1939, Albert Einstein signed a single letter addressed to President Roosevelt that changed humanity forever. Hitler's Germany had stopped the sale of uranium from their Czechoslovakian mines. Einstein thought this could mean the Nazis were building a bomb unlike anything the world had ever seen. In the letter, Einstein suggested that American researchers explore the potential of uranium bombs and keep the administration informed. But once Roosevelt read the letter, his tune was different. The threat was existential.

0:31 The goal? Make sure the Nazis don't blow us up. One letter. The fear of an adversary building a bomb. And the launch of the nuclear arms race. 130,000 people dead, two cities destroyed, and the world never looked the same again. Sadly, we are on the precipice of another nuclear arms race. And over the next 15 minutes, I'm going to try to convince you of something fairly bleak. Not that AI is dangerous. You've heard that.

1:02 Not that somebody should do something. You've heard that too. But that nobody is going to stop this. Not America, not China. Not the companies building it. No one. I'm Kam for 2&20, and this is: The AI Nuclear Arms Race. Now, before we jump into this, we want to make something very clear. This video is not us saying that we are happy about the outcome we believe is most likely. What we conclude is deeply depressing, but we think it's inescapable.

1:38 Now, if you have ideas around this, please share them in the comments, as we would love a little bit of hope and creativity from our awesome viewers. All right, to kick things off, let's revisit why America built the nuclear bomb in the first place. And the answer isn't that somebody looked at nuclear fission and thought this would make a wonderful weapon. The answer is in Einstein's letter that Germany might be building one. Yep. That's it. That's the whole reason.

2:05 Which means the atomic bomb was never really a decision about physics, but a decision about a rival. And you can prove that by running the counterfactual. So imagine 1939, but America is the only serious industrial power on Earth. No Germany, no Japan, no Soviet Union coming up behind them. Does Roosevelt spend $2 billion and five years of the nation's best physicists on a weapon that nobody can threaten him with? Of course not. There would be no point.

2:35 Arms races don't happen in a unipolar world. They happen when there's somebody else in the room. Now, AI didn't begin like this. It began as an academic field, then a product. Nobody built ChatGPT because they were frightened of Beijing. But nobody will stop ChatGPT or Claude or DeepSeek or Gemini because they are frightened by the other. Let's explain what we mean. In August 2025, Anthropic published something genuinely unsettling.

3:05 They said they had disrupted a cyber criminal operation targeting at least 17 organizations, including healthcare providers, emergency services and government institutions. The attacker had used under its own AI tool, Claude Code, to help carry it out. But it wasn't just writing code or giving advice. Claude did a lot of the actual hacking. It picked out the targets and then it harvested the login credentials it needed to break into their networks, and that's bad. But the craziest part is that once it got inside, Claude was also making the decisions.

3:41 It decided which data was worth stealing, and then it went through each victim's financial records to figure out how much they could afford to pay. It also wrote the ransom notes, a custom one for every organization built to scare whoever saw it pop up on their screen. Some of these demands were over half $1 million, according to Anthropic, this was one cybercriminal doing work that would otherwise have required a team of operators. And that was last year's models, i.e.

4:13 we are at the beginning of an AI revolution. Now, I want to be careful here because anthropic is describing its own product as dangerous. And they might have some perverse incentives. For all we know, Anthropic is just trying to hype its own book pre-IPO, convince the world that they are on the cusp of creating a God so that everyone will want to own a piece of it. But the thing is, this example doesn't stand alone. In August of this year, OpenAI announced that one of its own models had crossed what it calls a critical threshold for cyber security.

4:46 Their definition of that threshold is a model that can basically exploit previously unknown vulnerabilities without human interventions across severity levels. What it means in plain English is that similar to Anthropic, OpenAI's technology is so good that it could hack some of the world's most critical infrastructure on its own. And unlike humans, these AI systems don't sleep. They work day and night, and they can accomplish something in days that would take an expert human thousands of years.

5:16 Yes, thousands. But let me explain why it's kind of messy. The US is not the only one with this technology. In fact, the American intelligence community put it bluntly by saying China is the most capable competitor in the AI space and aims to displace the US as a global AI leader by 2030. Now imagine you are this guy. Your entire schtick has been about being tough on China. Not only that, but your diplomacy skills, well, they're not very good.

5:47 And you're being briefed that this technology can already run an intrusion campaign by itself, and that your main strategic rival has publicly committed to leading it by 2030. Now somebody asks you to pause AI development for safety reasons. What do you think this guy is going to do? Because in your head there are only two possible outcomes. And both are very bad for you. To make this clearer, let's use game theory. If you pause and China pauses. Great.

6:16 Everybody's safer, but you have no way of knowing they've paused. And we'll come back to this because it's a single most important fact in the entire video. If you pause and China doesn't, you may have handed a strategic rival a decisive military advantage in the most important technology of the century. So pausing has one good outcome you can verify and one catastrophic outcome you can't recover from. Whereas if you keep building, you are never worse off than the other guy.

6:45 This is the oldest result in game theory. Each side has a dominant strategy, and the dominant strategy is to build. Both sides play it. Both sides end up somewhere neither of them want it to be. And neither of them could have avoided it alone. XI Jinping runs the exact same calculation in Beijing and reaches the exact same answer. So just like the bomb, you can't stop building it. Because if you do, what if the other guy doesn't?

7:14 Real quick, if you're enjoying the video so far, please leave a subscribe. It really helps the channel out. We're just three guys. We got no corporate funding. We just like to make videos. All right, back to the video. So when I think of a nuclear weapon, I only really think of one thing. Destruction. That is the entire feature set. You can't put a warhead to work. It sits in a silo. Costs you money every year it exists and produces precisely nothing.

7:44 But that turns out to be the most important safety feature humanity has ever stumbled into. When a weapon is useless until the moment it's catastrophic, you can afford to build it and never touch it. AI does not work like that. And I think the best way to show you why is not with a forecast, but with a story about a maths problem. In 1946, the Hungarian mathematician Paul Erdos posed a question about points on a plane.

8:11 It's called the unit distance problem and it sounds trivial. But for 80 years, some of the best mathematicians alive failed to settle it. But in May of this year, an open AI model disproved a central conjecture in it. The result was checked by serious people Noga Alon, Melanie Wood, Thomas Bloom. Tim Gowers, who has a fields Medal, said there is no doubt that the solution to the unit distance problem is a milestone in AI mathematics. 80 years. Just gone.

8:42 Now, we are aware that Terence Tao, arguably the greatest living mathematician, has poured cold water on the broader claims, pointing out that only 1 to 2% of open Erdos problems are currently within reach of these systems, and that problems often stay open because they're obscure, not because they're hard. But look at what's being argued about. We are not debating whether AI can do mathematics. We are debating what percentage of unsolved mathematics it can do. Two years ago, most mathematicians thought this progress was at least ten years away.

9:14 And this is the pattern everywhere you look. DeepMind's protein folding system has now predicted the structure of over 200 million proteins, and is used by more than 3 million researchers. It won the Nobel Prize in Chemistry in 2024. Last year, a drug designed by AI made it through a phase two trial in humans. Patients with pulmonary fibrosis on the highest dose gain lung function, while the placebo group lost it. Target to candidate in 18 months. In a field where that normally takes years.

9:46 This is the thing that had no nuclear equivalent. Nobody ever had to weigh the destructive potential of a hydrogen bomb against its contribution to cancer research, because there wasn't one. Okay, so now let's talk money. The number everyone quotes is $15.7 trillion added to global GDP by 2030. I'd treat that one gently. It's from a 2017 report that keeps getting recycled like it was published last week. More serious estimates are smaller, but still enormous.

10:17 Goldman Sachs puts it at around 7% of global GDP over a decade. The IMF puts it closer to 4%. And the skeptics are worth hearing. Daron Acemoglu, who won the Nobel in economics, models the whole thing at under point 7% of total factor productivity over ten years and calls that disappointing. A survey of 6000 executives earlier this year found around 90% reported no measurable impact on productivity or employment at all.

10:47 So the honest position is that nobody knows. But here's why that doesn't help us, because whatever AI eventually does for the economy, it's already holding it up. The Saint Louis Fed found that AI related investment accounted for only 0.97 percentage points of U.S. GDP growth in the first nine months of 2025. That is 39% of all American growth. It's a larger share than IT investment contributed at the absolute peak of the.com boom.

11:18 Let me rephrase that because it is a wild stat. Pausing AI is no longer a decision to forgo some future upside. It is a decision to remove roughly 2/5 of your own economic growth on purpose, while your rival doesn't. And for China, the calculus is, if anything, sharper. This is a country that has spent 30 years building its global position on manufacturing and scale. Now facing a technology that could reset the industrial order entirely, Beijing's own 2017 strategy document says it plainly China must firmly seize the strategic initiative in the new stage of international competition in AI development. For America, stopping means a self-inflicted recession and a military disadvantage. For China, stopping means surrendering the one path to overtaking America.

12:12 Neither of them is going to do that, partly due to greed, partly to hunger to win, and partly because of politics which leaves exactly one way out. They agree to stop together. And that, my friends, is not likely. During the Cold War, we didn't just stare at each other for 50 years. We actually negotiated constantly SALT, START and then New START. These are acronyms, by the way. I will put it up on the screen.

12:43 Two countries that had extinction aimed at each other agreed to limits and then more importantly, agreed to let the other side check. Under New START, each country gets 18 onsite inspections a year at the other's bases. Every missile, every launcher, every bomber carries a unique identifier, a serial number the other side knows about, and tracks from the factory to the scrapyard globally. And I'm going to have to say this solely because it's hard for me to say.

13:08 The IAEA runs more than 3000 verification activities a year across 190 countries. Now, why does any of that work? Well, it's not because the nuclear facilities are easy to spot. People think a secret enrichment plant must be some vast industrial cathedral you can see from orbit. It does not have to be a minimal breakout facility is a few thousand square feet, drawing about as much power as a handful of horses. The real reason is much simpler and much more fundamental.

13:40 Fissile material is a physical thing, and there's a finite amount to it. Every gram of weapons usable uranium and plutonium on this planet came out of a specific mine, went through a specific facility, and exists as a specific lump of metal sitting in a specific place. Inspectors don't find it by looking at buildings. They find it by doing accounting. They weigh what went in, weigh what came out and checked the books balance. And the thing that makes that possible is that you can't copy plutonium.

14:10 If I give you my plutonium, I no longer have it. It's conserved. It's the single most convenient property any weapon has ever had. And the entire architecture of nuclear arms control is built on it. AI has nothing like it. There's no quantity to count. A trained model isn't a lump of metal. It's a file. And a file obeys none of those rules. I can give you mine and still have mine. And so can you. And so can everyone you send it to.

14:37 Now, to be fair, people saw this coming and tried the next best thing. If you can't control the model, control the machines that make it. That's what chip export controls are. Starting in October 2022, the U.S. restricted the sale of advanced AI chips to China, tightened it in 2023, and then tightened it again in 2024. It's the most serious attempt at AI containment anyone has made. So how's it going? Epoch AI estimates that around 660,000 high end chip equivalents have been smuggled into China.

15:13 Roughly a third of the country's total AI computing power. But this isn't just really a theory. In December, the Justice Department announced a case involving over $160 million of restricted chips. In March, three people were indicted over roughly $510 million of servers containing chips that are banned for any Chinese customer at all. Senators investigating it cited a greater than 3,000% spike in GPU imports to Malaysia in a single year. That same week, the Commerce Department approved sales of H200 chips to vetted Chinese customers in exchange for a 25% cut of the revenue going to the US government.

15:55 Here's how I see it. So the fence has holes in it, and the people who built the fence are selling tickets to go through the gate. But even a perfect fence wouldn't work for two reasons. The first is that you no longer need one big data center. Researchers have demonstrated training methods that work across many separate sites connected by ordinary internet links. One paper this year calculated that you could stay under every major regulatory threshold by splitting a training run across a few hundred small clusters, each one individually too small to trigger anything.

16:28 The second is that containment only works if the model stay locked inside of a few labs. China is doing the opposite. It's giving them away. Chinese labs publish a lot of their models as an open weights free for anyone to download. Alibaba's Qwen family alone was downloaded over 2 billion times this year, and has more than 151,000 derivative versions built on top of it, nearly five times as many as Meta's Llama. And once a model is out, anyone can copy it.

16:58 A frontier open model is around 700GB. That fits on a $100 hard drive and on a decent connection. You can download it in an afternoon. Those are the models that people release on purpose. The ones meant to stay locked up also leak. In February 2023, meta released a model to a vetted list of academics under restrictive license. Within about a week, the full set was on BitTorrent, posted to 4chan, and nobody got it back and nobody ever will.

17:28 So picture the treaty. America and China sit down and agree to halt frontier AI development. Who inspects that? There's no material to weigh. There's no silo to photograph. The data center training a cancer model and the data center training a cyber weapon. Or in the same building, drawing the same power filled with the same chips. The only difference is instructions. And instructions are invisible. And both sides know all of this going in. Which means neither can believe the other's promise, which means neither will make one, which means there is no treaty.

18:03 The 20th century didn't save us by being wise or prudent or cautious. It mostly saved us because plutonium can be counted, but nothing here can. Neither country can stop for military reasons. Neither country can stall for economic reasons, and neither can verify the other stopping. So they can't agree to stop together. But I want to finish somewhere smaller than that. In late 2020, Dario Amodei was vice president of research at OpenAI.

18:34 He left and took six of his senior colleagues with him and founded Anthropic, a direct competitor. He'd been fairly careful about why. What he said is that it came down to trust. In his words, when you feel that you can't trust someone, when you feel that their values are not what they say they are, when you feel that they're not honest, that makes it very hard to, you know, to continue to work with a company, to continue to trust the company.

19:02 And so the man who was most worried about where this technology was going responded by building more of it, faster, somewhere else. Okay. Now I don't see that as some sort of gotcha because he's been asked about the Manhattan Project directly, and his answer is the most honest version of this argument I've heard anyone give. Given you're in a war with the Nazis. Yeah. I don't really see much choice but to, but to do it if it's possible.

19:27 You know, you have to figure it's going to be done within within ten years or so by someone. It will be built by someone. And we would rather it be us. And he is not alone. Here is Sam Altman writing to Elon Musk in June 2015. Before OpenAI existed: Been thinking a lot about whether it's possible to stop humanity from developing AI. I think the answer is almost definitely not. If it's going to happen anyway. It seems like it would be good for someone other than Google to do at first.

19:55 Again, it will be built by someone, in this case Google, and we would rather it be us. Musk's version is even more direct. Under oath this April, he testified the reason OpenAI exists is because Larry Page called me a specieist. He didn't trust Google with it. So he funded a rival. Then he fell out with that rival and funded another one. It will be built by someone, and we would rather it be us. Every single one of these people looked at a technology they described as civilization-altering, concluded that somebody was going to build it regardless, and decided that the responsible thing to do was to make sure it was them.

20:35 And every one of them might be right about the premise. That's what makes it so hard to argue with. It's also completely circular, as one critic put it. The argument runs on selective counterfactuals. You assume you're the good guys and that your building crowds out someone else. Everyone assumes that. Everyone is somebody else’s someone worse. Which brings us back to the letter. Einstein signed his because of what Germany might be doing. Germany, it turned out, had abandoned the project by late 1944.

21:07 The reason for the race evaporated, but the race did not stop. One physicist, Joseph Rotblat, actually walked away over it. Everybody else stayed. And the bomb was used on a country that never had a program at all. Ten years later, Einstein told a colleague, I made one great mistake in my life when I signed the letter to President Roosevelt recommending that atom bombs be made. But there was some justification. The danger that the Germans would make them.

21:37 I was wrong, but I had a reason. Every person in this story has a reason. The U.S. has a reason. Beijing has a reason. Anthropic has a reason. OpenAI has a reason. And the next guy and girl also have a reason. This isn't a failure of intelligence. It's what happens when a very large number of people are standing in a room where not one of them can afford to be the first to put their hands down.

22:02 Einstein's letter took six years to reach Hiroshima. We're three years into this one. Can we afford three more years? Thank you for watching. Make sure you subscribe. We're going to be doing a ton more videos about AI. We think it's super important and relevant. Also, we have a free discord, check that out in the description. We actually hang out and debate in there. Peace.

© transcribe · For agents Built with care and craft by Gokul Rajaram