Transcript
0:00 On November 17th, 2023, Sam Altman, wildly successful CEO of OpenAI, was fired. >> The godfather of ChatGPT kicked out of the company he founded. >> Employees were revolting, investors were freaking out. Nobody seems to understand why this guy was fired. And then, >> >> four days later, Sam Altman is apparently back as OpenAI CEO. >> I want to tell you the full story. Some of it we only know now, thanks to unsealed court testimony that's come out just this year. I think you need to understand what happened those five days if you want to understand this strange moment we're living in.
0:32 >> AI is a technology for taking away your job and maybe eventually displacing human sovereignty over our own future. >> Where, frankly, a handful of unelected tech CEOs have more power over the future than most heads of state. >> You've said AI could wipe out half of all entry-level white-collar jobs and spike unemployment to 10 to 20% in the next 1 to 5 years. >> Yes. >> And the most powerful of all those people might be Sam Altman, the CEO of OpenAI, a company founded on the idea that no one person should control artificial intelligence.
1:04 >> AI will probably, like most likely sort of lead to the end of the world, but in the meantime, there will be great companies created. >> So, why was Sam Altman fired? And what does that tell us about who gets to decide our future? Friday, November 17th, 2023. Sam Altman is in a hotel room in Las Vegas. He's in town for the Formula 1 race. It is a well-earned break. He's been spending the better part of a year doing press, traveling the world meeting global leaders ever since his company, OpenAI, released a chatbot, which you know what? I'm just going to go ahead and assume you've heard of.
1:46 >> ChatGPT. >> ChatGPT is on pace to pass 100 million active monthly users after just 2 months. >> Things are, by any reasonable measure, going extremely well for Sam. His company is about to be valued at $86 billion. People are calling him the Oppenheimer of his generation, which he doesn't seem to mind. He sometimes mentions that he and Oppenheimer share a birthday, fun fact. At noon, Sam opens his laptop for a quick video call. On the other side is OpenAI's board. Let me introduce them. We've got Adam D'Angelo, CEO and founder of Quora, you know, that the website with the questions, you know the one.
2:24 Helen Toner, AI policy researcher, day job at a Georgetown think tank. We've got Tasha McCauley, robotics entrepreneur. And Ilya Sutskever, OpenAI's chief scientist. Widely regarded as one of the most brilliant AI researchers around. Also a co-founder of the company, also Sam's friend. Sam was expecting a routine call. It is not a routine call. Sutskever reads from a prepared statement. "The board has," he says, "lost confidence in Sam's ability to lead the company. They are terminating his employment effective immediately. There will be a public announcement shortly."
3:02 Sam is too stunned to speak. He feels like he's in a bad dream. After the call, Sam is locked out of his laptop. 20 minutes ago, he was CEO of the most famous AI company on Earth. And now he can't get into his own Gmail. Now, minutes later, the board puts out a public statement. It reads, "Mr. Altman's departure follows a deliberative review process by the board, which concluded that he was not consistently candid in his communications with the board, hindering its ability to exercise its responsibilities." The board also names an interim CEO in the announcement, Mira Murati, OpenAI's chief technology officer.
3:42 Within minutes, this firing is one of the biggest news stories in the world. Sam's phone is getting so many texts per second his iMessage app crashes. 300 miles away at OpenAI's San Francisco offices, hundreds of people are standing around staring at their phones refreshing the same four tabs. Nobody is working. Nobody can work. Their CEO was just fired 20 minutes ago and nobody will tell them why. Also caught off guard is Microsoft, OpenAI's most important investor. Murati had called them mere minutes before the public announcement went out. Satya Nadella, the CEO of Microsoft, is to use the technical term, pissed.
4:20 Sutskever and Murati hold a company-wide meeting to try to explain what's happened. Someone asks if there was a specific incident or something and Sutskever replies, "I wish I could go into details, but I can't." Someone asks if they'll ever find out more. Sutskever says, "Keep your expectations low." Then, someone asks about the tender offer. Now, this wasn't mentioned much in reporting at the time, but in just a few days OpenAI was due to close a deal that would let employees sell their own shares at that $86 billion valuation.
4:52 For a lot of employees, that meant millions of dollars. Some people had already put down deposits on houses. So, Brad Lightcap, the COO, has to take this question. And he says, "The tender, we're we're going to see." None of this, it turns out, is particularly reassuring. >> That whole weekend was one of the most stressful periods in my life, I would say. It was like a civil war within the company. >> Around 2:00 p.m., Sam puts out a tweet in his trademark all lowercase. He says, "I loved my time at OpenAI. It was transformative for me personally and hopefully the world a little bit. Most of all, I loved working with such talented people."
5:32 Greg Brockman, who is OpenAI's president, also a co-founder, also a board member until the same four people removed him that same day, has been told that he can stay on as OpenAI's president. Instead, he retweets Sam and announces that he's quitting in solidarity. Three more senior researchers quit as well. Employees feel like the sky is falling. Sam, for his part, gets on a flight back to San Francisco. Late Friday evening, the board takes a meeting with OpenAI's senior executive team. So, not the whole company this time, just the 15 or so most senior employees.
6:07 The board keeps citing confidentiality and just continues to avoid answering any questions in detail. And the execs are furious. They say the board has no choice but to reinstate Altman, otherwise they'd be letting the company die. Helen Toner replies, "That could in fact be consistent with the mission." At first glance, that sounds completely insane. Why would a board fire a wildly successful CEO and call it consistent with the mission?
6:38 OpenAI was founded in 2015, not as a normal tech company, as a nonprofit research lab. It was funded by donations from the likes of Elon Musk, also a co-founder, Peter Thiel, and Sam Altman himself. Ilya Sutskever was there from day one. And the mission was not to make money. It was very explicitly to save the world. Easy to roll your eyes at, but here's what these people at least claimed to believe. That within their lifetimes, possibly within the next decade or two, humans would build machines smarter than themselves. Machines that could do new scientific research, could run a military strategy, just do any cognitive work that humans can do.
7:18 They had a name for this. AGI. >> AGI, artificial general intelligence. >> And they believed that whoever built AGI first would have in their hands the most powerful technology to have ever existed on Earth. And they did not think that this was a safe enterprise. This is Sam Altman writing on his personal blog back in 2015. Development of superhuman machine intelligence is probably the greatest threat to the continued existence of humanity. In that blog post, Sam recommends this book, Superintelligence.
7:52 And the book's argument is basically this. Humans run this planet not because we are the strongest species, but because we're the smartest. You build something smarter than humans, then the same cold logic applies. >> It's smarter than us, right? There's no way we're going to prevent it getting rid of us if it wants to. If you want to know what life's like when you're not the apex intelligence, ask a chicken. >> That's the position that OpenAI's founders thought humanity was about to be in. So, they set up this research lab to build AGI themselves and build it safely before someone less careful builds it first. They specifically do not trust Google DeepMind.
8:26 >> The reason I wanted to create OpenAI was to serve as a counterweight to Google. The furthest thing from Google would be like a nonprofit that is fully open cuz Google was closed, for-profit. We don't want this to be a sort of profit maximizing >> a behemoth from hell, you know? >> >> That just never stops. >> OpenAI's original mission, and you can still read this online, was to advance digital intelligence in the way most likely to benefit humanity as a whole, unconstrained by the need to generate financial returns. These are not normal startup commitments. But, here's the problem.
8:57 Training cutting-edge AI turns out to cost an astronomical amount of money. >> Like, I think one thing that no one knows yet is just how much money you need, you know, billions for sure, tens of billions, maybe hundreds of billions, I sure hope not. >> So, in 2019, Sam Altman comes up with a fix, one that happens to land him as CEO of the company. The idea is this. A nonprofit board of directors that owns and controls a for-profit subsidiary.
9:23 So, the for-profit can take investment, it can pay competitive salaries including startup equity, it can build data centers, it can compete with Google. But, it answers to the nonprofit, and the nonprofit answers to the mission. There are two guardrails built into this structure. Guardrail number one, investor profits are capped at 100x. Beyond that point, everything flows back to the nonprofit to be used for the benefit of humanity for something like universal basic income.
9:54 The idea was simple. Investors can get rich, but not infinitely rich. Past a point, the upside belongs to everyone. >> If you wanted to just make money, that would not be a good way to do that, right? Like having a nonprofit have that level of control over your company. They could have just started over and started a for-profit company, right? And they chose not to do that. >> Guardrail number two, the nonprofit board can fire the for-profit's CEO.
10:16 This is the kill switch. The idea is you put people on the board who care about not ending the world. You let the capitalist machine run underneath them. It's the only way to compete. But, if it starts racing somewhere dangerous, the people upstairs can pull the plug. On paper, this board puts humanity above the company. If the two come into conflict, humanity is supposed to win. Sam Altman talks about this constantly. >> Like no one person should be trusted here. I don't have super voting shares.
10:48 like I don't want them. The board can fire me. I think that's important. >> Now, this is what OpenAI looked like as of November 2023 after the release of ChatGPT, the fastest growing tech product in history, and $13 worth of investment from Microsoft. In theory, the nonprofit is in charge. In practice, you have a handful of volunteers overseeing an $86 billion tech company. Saturday morning, November 18th, Sam wakes up in his home in San Francisco.
11:23 So, what do you need if you've just been fired by one of the most important companies in tech and you'd ideally like your job back? This, to be clear, is not career advice. Step one, open a line of communication. On Saturday morning, Sam gets a call from the board that just fired him. They want to talk after the disastrous reaction the previous day from employees and senior executives. They're right to be worried, by the way. In a few hours, the executives are going to give the board a 5:00 p.m. deadline to reinstate Sam and then resign.
11:55 Step two, assemble your allies. Sam starts turning his home into a war room, or to use his words, a government in exile. Within hours, dozens of people are lining up in Sam's home, including OpenAI employees and senior executives. Lawyers set up an office outside his bedroom. Mira Murati is there, which is pretty weird considering she's just replaced him as CEO. Sam, for his part, works the phones. According to his phone records, he's on calls for 12 hours a day.
12:24 Step three, stay relaxed. 6:00 p.m. rolls around. The 5:00 p.m. deadline has come and gone. The board has neither reinstated Sam nor resigned. Sam walks out to his living room and starts handing out a round of Negronis. "You need to relax," he tells people. "Whatever's going to happen is going to happen." Step four, make a show of strength. On Saturday night, Sam sends out a single tweet. "I love the OpenAI team so much." Within minutes, dozens of OpenAI employees are retweeting it with heart emojis.
12:55 Step five, set the terms of the fight. Now, Sam has this guy Chris Lehane on the line. He's a political operative whose personal motto is apparently "Everyone has a plan until you punch them in the mouth." Lehane, maybe unsurprisingly, wants Sam to go scorched earth on the board. Drag them in the press, destroy them. Sam, though, plays it differently. He steps back and lets his supporters and allies build his case for him. They start feeding sympathetic journalists just enough to shape the narrative in the press. The board doesn't know what they're doing. The board is in over their heads. The board won't even say why they did it. Within 24 hours, the narrative isn't Sam Altman was fired. The narrative is, what the hell was the board thinking?
13:39 Which actually is a good question. What was the board thinking? In 2003, a teenager shows up at the University of Toronto and knocks on the door of Professor Geoffrey Hinton, a future godfather of AI. That teenager's name is Ilya Sutskever. And he's interested in Hinton's work on artificial neural networks, an out of fashion approach to AI that's loosely inspired by biological brains.
14:11 So, Hinton gives Ilya an old paper. A few days later, the teenage Ilya comes back with an insight it had taken the field years to produce. For most of his life, Ilya Sutskever has seen things in AI before other people see them. Fast forward to 2012. >> >> Ilya, Hinton, and a colleague have been researching deep learning, basically just making neural networks way bigger than had been tried before. At the time, mainstream AI researchers have been struggling for years to get computers to correctly label images. The best in class programs are wrong about 25% of the time.
14:44 Ilya and Hinton debut a network that demolishes that world record. In 2013, a young startup called DeepMind uses the same type of deep neural network and unveils an AI system that can top human experts at multiple different classic video games. In 2015, Elon Musk and a charismatic guy named Sam Altman take Ilya out to dinner. They don't trust DeepMind. They want to start a new AI lab. They want to save the world. They want Ilya to be a part of it.
15:15 Ilya turns down a $6 million a year, "Please don't leave us" counter offer from his employer Google and joins them. In 2016, DeepMind shocks the world with an AI system that beats one of the greatest human masters of the board game Go. Next year, they try using 100 times more computer chips, and instead of teaching this AI system Go with examples, they just let it start playing itself. In 8 hours, it is better at the game of Go than any person or program that has ever played. By this time, Ilya is convinced that the path to AGI might be as simple as finding the right neural network design and then scaling it up.
15:53 And in 2017, 18 months into leading research at OpenAI, he reads a paper from Google that makes him jump out of his chair, run down the hall, and tell his colleagues, "Everyone stop what you're doing. This is it." Over the next 5 years, OpenAI is going to make a series of exponentially larger bets that prove Ilya's conviction right. They train GPT-2 and discover that it has abilities they hadn't specifically trained it for, like being able to translate text into new languages.
16:24 The text output from GPT-3.5 is so human-like that they wrap it in a user-friendly interface and try releasing it as a chatbot. It hits 100 million users in 2 months. By this time, Ilya has developed a slogan. He says it around the office. A couple weeks after the ChatGPT release at the company's holiday party, he gets his colleagues to chant it back to him. Feel the AGI. He wants them to take it seriously. To take seriously that the progress might not stop. That this curve they're riding, a curve that started >> >> way back in 1956 with a few researchers in New Hampshire dreaming about thinking machines, might just keep going up.
17:05 Soon, AI systems might match human intelligence and then surpass it and then vastly surpass it. And then, the person in control of the technology will have their hands on something unprecedentedly powerful. Something they may not be able to control at all. >> Scientists have been accused of playing God for a while. But, there is a real sense in which we are creating something very different from anything you've created so far.
17:38 >> At the 2022 retreat, Ilya and his colleagues are hanging around a campfire and he's got this wooden effigy in his hand that he commissioned from a local artist. The effigy, he explains, represents an AI that is pretending to be nice, but is actually been lying to them. Ilya douses it in lighter fluid and sets it on fire. Around the same time, Ilya is starting to get very worried about Sam.
18:09 Sunday morning, November 19th, 2023. Less than 48 hours since Sam Altman was fired, and he's heading into the office. Interim CEO Mira Murati has invited Sam to come by for the day. So, Sam goes to the front desk, gets a visitor's badge, and because he's Sam, immediately posts a selfie with the caption, "First and last time I wear one of these." Negotiations go on for hour after hour with the board. Journalists are camped out outside the OpenAI offices tracking the steady stream of takeout deliveries.
18:39 No one involved has slept much in the last 48 hours, but Sam feels good. The board is losing in the court of public opinion. They've definitely lost with the employees, and they're even open, it seems, to bringing him back as CEO. There's haggling going on over who would need to resign, who would make what statements. It really seems like the board is out of cards and about ready to fold. Or are they? See, all week at the board has been putting out feelers to anyone in their network who might be willing to come in as interim CEO.
19:10 At one point, they reach out to Dario Amodei, the CEO of OpenAI's bitter rival, Anthropic, to ask if he wants to be CEO, maybe even combine the companies. He's not interested. But, Sunday evening, late in the night with hundreds of employees at the office expecting an end to the drama, the board makes a sudden announcement. Murati has been replaced. There's a new CEO. Enter Emmett Shear. Co-founder of Twitch, the website with the videos and the the game, you know the one. Sam is shocked.
19:45 He feels like the board has been negotiating in bad faith all weekend. OpenAI employees flood out of the building. Sam goes home in disgust. But, Sam Altman being Sam Altman, he already has another move lined up. The previous day, Microsoft CEO Satya Nadella had asked Sam if he wanted to join Microsoft. Now, Sam tells him he's in. At 7 minutes before midnight on Sunday, Microsoft CEO Satya Nadella tweets that Sam, along with his number two, Greg Brockman, will be joining Microsoft.
20:19 Word spreads very quickly through employees' signal chats. Anyone who wants to join them will have a guaranteed job at Microsoft. Now, think about how this lands with the employees. They have been banking on this tender offer that now looks dead in the water. The board has just fired their leader, the guy that all the investors love, who's now been immediately snapped up by a much larger company, which they are now being invited to join. And they still have no idea why this is happening.
20:47 All they've been given is that one phrase, "Not consistently candid." I think it's time to tell you what that means. A few weeks before the firing, a terrified Ilya Sutskever sends a disappearing email to three of his fellow board members, Tasha McCauley, Helen Toner, and Adam D'Angelo. It contains a 70-page two-part memo. The screenshots inside have been compiled covertly over the course of months. There are Slack messages photographed with personal phones to avoid screenshots on company devices.
21:21 The first line of the memo reads, >> >> "Sam exhibits a consistent pattern of lying, undermining his exacts, and pitting his exacts against one another." This forms the centerpiece of the case that Ilya Sutskever, the man who spent a decade telling the world that superintelligence is coming, is starting to form against his co-founder and CEO, Sam Altman. Let's go through it. Exhibit A, the lying. Many of those screenshots came from one person, Mira Murati, the board's eventual pick as interim CEO. Back in October, on multiple calls with the board, Murati had explained in detail Altman's toxic leadership style, >> >> a pattern of behavior going back years.
22:02 If someone resisted his decisions, he would tell them whatever he thought they wanted to hear. If that didn't work, then he would undermine their credibility and turn people against each other. So, this is why the board was so cagey during the firing. They'd been told by outside legal counsel that they couldn't reveal this information without revealing Murati as the source and putting themselves at serious legal risk. Exhibit B, the safety lapses. At a board meeting in December 2022, Sam told the directors that three controversial changes to GPT-4 had all been approved by the company's joint safety review board. Helen Toner asked for documentation.
22:40 It didn't exist. It turns out only one of the changes had been approved. Around the same time, Microsoft started testing the still unreleased GPT-4 in India without any safety approval. The board found out about this by accident from a conversation in a hallway. Later, Sam sends a Slack message to the company. He says that the company's lawyer decided GPT-4 Turbo also doesn't need joint safety review. Mira Murati checks with the lawyer.
23:11 He says he never said that. >> On multiple occasions, he gave us inaccurate information about the small number of formal safety processes that the company did have in place. >> It really seems like the man who thought he was building super intelligence was lying to the board about how he was keeping it safe. Exhibit C, the fund. Sam Altman had also told the board and the world that he was an independent director of OpenAI with no financial stake in the company. It turned out that the OpenAI Startup Fund, $175 million of venture capital deployed in OpenAI's name, was personally owned by Sam Altman. It does now seem like this was mostly done to make paperwork easier rather than for Sam's personal financial gain.
23:54 But it took months for the full situation to be clarified to the board after they raised the issue. Exhibit D, the paper. Helen Toner authors an academic paper buried within it some very mild criticism of OpenAI. Altman is furious. He goes to Sutskever personally, says that Tasha McCauley agrees Helen Toner needs to be removed from the board. McCauley had said no such thing. >> Not trusting the word of of the CEO who is your main conduit to the company, your main source of information about the company, it's just like totally totally impossible.
24:28 >> It turned out that the board had been deadlocked for months over filling multiple vacancies. Sam wanted his picks. Interviews were on the agenda for the November board meeting just days away. If Sam aligned directors joined the board, then these four would lose their majority and they'd never be able to fire him. Their window was closing. They decided they had to act. As Ilya told one board member at the time, "I don't think Sam is the guy who should have his finger on the button.
25:00 The early hours of Monday, November 20th, 2023. 61 hours since Sam Altman was fired. Around 1:00 a.m., a letter starts to circulate on various signal chats. It's never been totally clear who drafted this letter, but it seems likely that it came from senior executives. Recent evidence suggests the original idea came from Sam himself. Let me read you some of it. It's addressed to the board of directors. The process through which you terminated Sam Altman and removed Greg Brockman from the board has jeopardized all of this work and undermined our mission and company.
25:34 Your conduct has made it clear you did not have the competence to oversee OpenAI. Despite many requests for specific facts for your allegations, you have never provided any written evidence. You also informed the leadership team that allowing the company to be destroyed, quote, would be consistent with the mission. That mission, if you recall from earlier, being to ensure that AGI benefits all of humanity. The letter demands that OpenAI reinstate both Sam Altman and Greg Brockman. The first signature is Mira Murati's.
26:08 This is one of those moments that we can only fully appreciate now that we know the full details of why the board fired Sam. Murati is essentially accusing the board of not having evidence when she herself provided many of the screenshots that formed the basis for their case. Of course, no one else knew that at the time. The letter spreads around it quickly, and within a few hours most of OpenAI's employees have signed. Along with the letter, many employees start posting the same phrase on X.
26:35 OpenAI is nothing without its people. Now, there are a few possible readings for why so many employees signed this letter so quickly. One is that they were genuinely shocked and unhappy about Altman's firing. Another is that they were worried about the value of their shares. For a lot of them, the majority of their net worth. As one employee who chose to remain anonymous told a journalist a few weeks later, Sam Altman is not the best CEO, but millions and millions of dollars and equity are at stake.
27:04 Another reading backed up by some reporting at the time is that once the letter started to snowball, people felt pressure to jump on the bandwagon, whether for fear of Altman's retribution or simple peer pressure. >> I think there was a lot of like phone banks and like people like getting really organized to try to like really get everyone to sign this thing. >> In the wee hours of Monday morning, another name appears on the growing list of signatures.
27:26 Ilya Sutskever. There's this great line from season 1 of The Wire. If you come at the king, you best not miss. It's pretty timeless advice, whether you're talking about drug lords or ancient emperors. If you challenge someone extremely powerful, you have to succeed immediately and totally, or you're screwed. Turns out that OpenAI was not the first time Sam Altman had faced this kind of challenge.
28:02 First, Loopt, Sam Altman's first company. >> Loopt is about connecting with people on the go, which is after all the main reason you have a phone. >> It's a location-sharing app, raises something like 30 million, doesn't really ever take off, sells several years later for 43 million. Fine. But it ends badly. Senior executives twice tried to get Sam fired for deceptive and chaotic behavior. Still, he moves on to his next job, becoming president of Y Combinator age 28.
28:30 He's really good at this. Incredible fundraiser, brilliant at building relationships. This is where his star really starts to rise in Silicon Valley. But YC's partners and executives grow increasingly frustrated at Sam's distractibility, at the attention that he's paying to his own personal projects, including the newly founded OpenAI, and to his own funds. Eventually, YC's beloved founder Paul Graham and his wife Jessica Livingston had to intervene to get Sam removed. Graham has this incredible quote about Sam, written when Sam was just 23 and intended to be complimentary.
29:06 You could parachute him into an island full of cannibals, >> >> come back in 5 years, and he'd be the king. Murati, Sutskever, the board, they know this is who they're going up against. His experience, his network, his allies in the press. The board knows their only hope is to act quickly and without warning. This is why they couldn't risk getting more senior execs involved, why the firing comes out of the blue, why they couldn't warn Microsoft in advance.
29:31 But this strategy also ultimately proved their undoing. Within the first day, Murati had seen enough to know that things weren't going to go the board's way. >> >> When I realized that their decision was like potentially catastrophic for the company, I felt like I had to act very quickly. >> In her deposition in the Musk v. Open trial this May, Helen Toner said this about Murati. She was waiting to see which way the wind would blow, and she didn't realize that she was the wind.
30:01 And as for Ilya Sutskever, he thought he would be greeted as a hero, that he was freeing people from what he saw as Sam Altman's gaslighting. Instead, people were blaming him for destroying the company. On that Sunday night, in the office lobby as employees were streaming out, Anna Brockman, wife of OpenAI's president Greg Brockman, confronts Ilya. They were old friends. Ilya had officiated the Brockmans' wedding. In tears, she pleads with Ilya to change his mind.
30:33 Ultimately, he blinked. By Tuesday morning, it's all over. More than 700 of OpenAI's 770 employees have signed the letter. With over 90% of the company threatening to walk, the board know they've lost. A deal is reached. Sutskever, Toner, and McCauley are out. Bret Taylor, CEO of Salesforce and someone that McCauley had previously been worried would be too deferential to Sam, and also Larry Summers, former Treasury Secretary, president of Harvard, agree to join the board.
31:11 D'Angelo stays. He's actually still on the board to this day. By holding out for as long as they did, the board do win a few concessions. Brockman and Altman won't get their board seats back, not right away, and there'll be an investigation by an outside law firm into what happened. But, it's not much, really. At 10:00 p.m. Pacific, the announcement goes out. Sam Altman will return as CEO of OpenAI. There's an impromptu party at the office. Everyone's celebrating. Greg Brockman posts a selfie with the caption, "We are so back."
31:46 There's a great detail from the reporting. At some point, someone turned on a smoke machine. It set off the fire alarm. Everyone kept partying. About 2 and 1/2 years have passed since that weekend in November. Many experts now think that AGI could arrive in as little as 5 years. Even skeptics these days rarely give it more than 20. Earlier this year, Claude 3 Opus, an AI system capable of finding vulnerabilities in software that experts had missed for years or even decades.
32:23 Enough of a jump up in capabilities that its makers decided not to release it to the general public. Over the last couple of years, a lot of changes have happened at OpenAI as well. Remember that structure, nonprofit on top, profit caps for investors, the board in total control, keeping to the mission, the benefit of humanity. In December 2024, OpenAI tried to restructure the company. Under the new plan, the nonprofit would essentially lose control. The plan didn't go through. After months of pressure, OpenAI backed down.
32:55 Sort of. Under the new new structure, the nonprofit is on top in name. There's a safety committee that in theory could halt model releases, and it owns about a quarter of the equity in the company. But, the profit caps that were supposed to keep OpenAI focused on its mission and not on investors, those are gone. And in practice, the board's authority is massively weakened. Now, it should be said that OpenAI still gets its models tested by third-party safety evaluators, and they're doing meaningfully important safety work on their current models.
33:29 But, what's left is functionally much closer to a normal tech company. A normal tech company that was built on purpose by people who thought they might be making a civilization-ending technology not to be a normal tech company. Sam Altman built that structure. He talked publicly about how it was crucial that the board could fire him, how that was part of why OpenAI was responsible. He recruited some of the world's top AI researchers under the idea that AGI should not be built by a mega-corporation.
34:00 And then, when the mechanism was actually triggered, when the board actually did the thing it was designed to do, within 72 hours, Sam had announced plans to go build AGI at Microsoft and refused to come back until two of the board members who had challenged him were gone. There are fewer people left at OpenAI who might challenge Sam these days. Within a year of his return, both Ilya Sutskever and Mira Murati had left. A lot of the senior people at OpenAI who were most worried about AI safety have left, too.
34:29 Just a few months before he was fired, Sam signed a one-sentence open letter that reads, "Mitigating the risk of extinction from AI should be a global priority alongside other societal scale risks such as pandemics and nuclear war." Since the board drama, which is referred to internally as the blip, Sam talks more about a gentle singularity. >> Someday, we will make something that qualifies as an AGI by whatever fuzzy definition you want. The world will have a 2-week freakout, and then people will go on with their lives.
34:59 >> He recently told the New Yorker, "My vibes don't really match a lot of the traditional AI safety stuff." So, 2 and 1/2 years on, AI is more powerful than it was in 2023 by a lot. OpenAI is more powerful, too. It is now worth $852 billion. The people that tried to put the brakes on one of the most powerful AI companies in the world are gone. The structure that was supposed to act as a brake is functionally gone, too.
35:30 So, where does that leave us? I've actually met Sam Altman. In 2022, I was volunteering at a conference. I was the speaker liaison, and he was one of the speakers. So, I showed him to the green room, gave him a bottle of water, made small talk. I get it. I get why so many talented people have wanted to work with him over the years. I get why he's so successful. You talk to the guy, and he makes you feel like the most interesting person in the world.
36:05 I think Sam Altman is not that mysterious. I think he's a guy who's really good at finding winning moves on whatever game board he happens to find himself on. I think he'll continue to do that. And I think it's sad and more than a little scary that the moves he makes might matter so much to the development of one of the most powerful and dangerous technologies in history. Sam is not the only one playing this game. We're doing things our way >> >> and they're doing things their way.
36:35 We'll see who wins in the market and we'll see who wins in the court of public opinion. >> It's unbelievably competitive, that whole AI space. >> They all want to win and they all want to beat each other and they all want to go fast. >> Right now, the future of AI basically depends on the moral compass of like five people. That's the situation we're in. Most powerful technology that humanity might ever build and how it goes depends on how a few CEOs are feeling.
36:58 That is not a stable situation. I don't want to live in that world and I'm pretty sure that you don't, either. AI is going to change all of our lives. I think it's going to change our entire civilization. And we should all have a seat at the table where how that goes gets decided. Because I think what this story really tells us is that self-regulation is not going to work. The OpenAI nonprofit structure was a close to best-case scenario of corporate self-regulation of AI.
37:29 You had a nonprofit board of volunteers explicitly committed to humanity over profit with the power to fire the CEO. And it was defeated in 72 hours by market pressures, employee revolt, and one very powerful man's network. >> Although the board had the legal ability to fire me, in practice it didn't quite work. And that is its own kind of governance failure. It does make it harder for me to like look you in the eye and say, "Hey, the board can just fire me."
37:59 >> So, here's the deal. We are not going to fix this from our bedrooms, but none of this is set in stone. People are realizing that these AGI companies are building something extraordinarily powerful and consequential and risky. I'm talking about people like Bernie Sanders and Mitt Romney and the freaking Pope. When OpenAI tried to restructure into a normal tech company without nonprofit governance, former OpenAI employees, Nobel laureates, and elected officials stepped up and made them back down.
38:28 When OpenAI and other big tech companies lobbied to ban US states from passing laws regulating AI, non-profits went to battle and got the Senate to vote 99 to 1 against the US cannot agree on anything Senate. >> >> And then there's the super pack funded by Greg Brockman, OpenAI's president, with advice from our friend Chris punch-them-in-the-mouth Lehane. As more and more of its bad behavior was exposed, individual researchers inside of OpenAI demanded answers. OpenAI ended up distancing themselves.
38:59 You really do have more leverage, more opportunities to influence how things go than you think. And more now than you may have in a few years. We've put a link in the description of this video. You can't miss it. If you're a US citizen, they'll tell you how to contact your representative in less than a minute. If you work at one of these AI companies, then please hold your leaders accountable. Ask them questions. We'd love to help you think about how to do that.
39:24 If you work in policy, you can sign up for one-on-one advising with us. We would love to help you figure out how to pivot into AI governance and regulation. >> We're in a pivotal moment right now. I've got a 2-year-old. You know, I want to be able to tell him when he's grown up what I did and have him be proud of me. Just like make the jump. Redirect all of your focus onto trying to make your career something that you'll be proud of.
39:44 >> And let's do it. Let's make sure that Sam Altman is not the only person who decides our future. >>