transcribe

Over 10% Chance AI Kills Us All

Mastra · 46m · transcribed 7d ago
More from Mastra Business
𝕏 Share ▶ YouTube 📥 PDF 🤖 .md

Section Insights

# 0:00

Introduction to Agents Hour

What is Agents Hour about?

Agents Hour is a weekly show hosted by Shane Thomas and Abby Ayer, focusing on AI news, guest insights, and problem-solving discussions.

  • The show airs every Monday at noon Pacific.
  • Listeners are encouraged to participate by sharing their insights or problems.
  • The hosts aim to provide valuable knowledge and updates in the AI space.
# 9:13

Current Events in AI

What significant news is being discussed this week?

The hosts discuss a controversial resignation from Anthropic, highlighting concerns about AI safety and responsibility.

  • A post from a former employee of Anthropic gained massive attention, raising alarms about AI development.
  • The resignation points to broader issues within AI companies regarding ethical practices.
  • The conversation emphasizes the need for responsible AI development.
# 18:26

History of AI Governance

What is the historical context of AI governance discussed?

The discussion traces the evolution of AI governance views from early optimism to current concerns about safety and ethical implications.

  • Dario Amodei's journey from OpenAI to founding Anthropic reflects a shift towards prioritizing safety in AI.
  • There is a growing consensus on the need for external evaluations of AI systems.
  • The conversation highlights the importance of ethical considerations in AI development.
# 27:39

Industry Perspectives on AI Development

What are the industry leaders saying about AI safety?

Industry leaders are calling for a cautious approach to AI development, balancing innovation with safety and responsibility.

  • There is a recognition of the potential dangers of rapid AI advancement.
  • Leaders are urged to consider the business implications of AI risks.
  • The dialogue includes humor to lighten the serious nature of the topic.
# 36:52

Threat Intelligence in AI

What are the current threats associated with AI technology?

Recent reports detail how AI is being misused for cyber attacks and other malicious activities, highlighting the need for vigilance.

  • AI tools are being exploited for cyber attacks, including data theft and surveillance.
  • Specific groups are using AI to enhance their capabilities in malicious operations.
  • The importance of monitoring and regulating AI usage is underscored.

Transcript

0:12 Heat. Heat. N. Fire.

0:42 Got you. Would you like to be a guest on the show.

1:17 Visit mastra.ai/guest. Do you have a hot takeache, a problem you cracked, a cool product, or a demo worth watching? Share them right here on Agents Hour. It's Monday noon. The time is here. Shane and I'll be loud and clear. Pacific vibes, we bring the heat. AI agents can't be beat. AI agents now.

1:52 Let's go. Losing guess the big show. Solving problems we do live. Staying focusing agents that work for you. Making moves we see it through. Every week a brand new show. Tag along and watching knowledge grow a news and guest. The big show answering questions. We arrive every Monday. Come alive.

2:28 We want reviews, but only if it's a five. We share the drama, we got the drive. Stay in the loop. It's the place to be. Shane and I be setting you free. We're here to stay. Tune in. Mondays make your day. From the news to the problem solve I will get involved.

3:29 Every week in AI, something insane happens >> and there's so much drama. Every Monday, we break it down live. >> We do the news. We bring on guests building in the space >> and we go deep into the stuff that actually matters. >> Agents Hour every Monday, noon Pacific. >> Follow. Don't miss it. Peace. >> Peace. This is Agents Hour with Shane Thomas and Abby Ayer. >> Hello everyone and welcome to Agents Hour. We are here in person today which is we don't do this every week. We're actually on location in San Diego. So we'll talk a little bit about that. But for those tuning in, thank you for watching the show. I'm Shane Thomas. I'm one of the founders of MRAA. I'm here with Abby, one of the other founders.

4:19 What's up, dude? What's up, >> dude? How are you? >> I am doing well. Glad to be here and glad to be doing this in person in our definitely makeshift studio for the day. >> Yeah, our nice little Airbnb studio. >> And yeah, with that, I guess a few things on our mind today. We're going to do the news like we do every week. We don't have any guests today, but we do want to recap a Master launch from last week. So, we'll talk a little bit about that. And I think if you are one of the people tuning in, you're going to know the topic we're probably going to be talking about in the news. We have like one big topic. There was one big thing that kind of became a whole bunch of things. There's a lot of back and forth between, I would say, Titans of the Space on pacing the frontier. And so, that'll be that'll be a topic we talk quite a bit about today.

5:09 >> Skynet is coming, y'all. That's what we've learned. Skynet is here for us. >> It's coming. It's coming for all of us. But before we do that, why don't we talk a little bit about something we launched last week. I guess it was maybe Wednesday. And I'm going to share my screen here. This is kind of cool. We we launched Monster Factory. We put it on Product Hunt to see what people thought. Turned out people liked it. We got the best product of the week on Product Hunt last week.

5:42 So, thank you all for those of you who did give us an upvote. And if you didn't, you know, you should do so because it does help us. We're like number two for the month so far. So, maybe we can get top product of the month with your help. So, please go give us an up vote on Product Hunt and get us to that number one spot for September. That'd be awesome. >> It was such an impactful launch for us. So much good feedback. obviously I think many people want they many people want to know about this topic software factories many people want it themselves and you know with monster factory you can have one you don't need to rent one you can own it and you know I think the product hunt launch reflects that and we have more things coming around factories it's we're not done yet.

6:34 Yeah, I think it definitely hit a nerve. I saw a lot of people saying it shouldn't be called software factory. Software factory is a dumb name. A lot of people saying, you know, a lot of people like to the term like automating the SDLC. A lot of people saying it should just be called like engineering teams because it's just that this is what you do. You just engineers have always improved their work and automated things. That's why we created CI and all these other tools, right? That's why DevOps became a thing. This is just I guess like a superpowered automation of sorts. But you know just like in in anything in AI you know there's all these dumb words for things we didn't want to call it a software factory and you know we launched as a software factory but it's called monster factory if you want to build a engineering team a social media factory a marketing factory like which is what we intend to let people do to do any type of SDLC or you know any life cycle of work that's what we're aiming to do. Funny enough, I was talking to Dex over the weekend.

7:38 We're just talking about dumb words and AI. you know, people in the latest YC batch are making domain specific harnesses, which, you know, in our YC batch that was just called verticalized agents. So, don't kill the messenger. We didn't create these words, but we are living in them. >> Yeah. I think that that is one of the things we realize is like we don't get to pick the terms that are going to stick, but we will live the terms once the industry decides that this needs to be this is the way people are going to refer to it.

8:10 >> Yeah. >> I mean, yeah, I can think of so many over the last 18 months of of terms where I wouldn't never have wanted to call it that, but here we are. >> Yeah. >> All right. so yeah, if you do want to check out master factory, if you're curious, you want to spin it up, you can just run npm create factory, you can test it out. If you go to the master website, click on the drop down, find master factory, you can deploy it in a few clicks to master platform. If you don't want to run it yourself, you want to test it out, give us some feedback. We would appreciate that. That's how we continue to improve it.

8:48 >> Yeah, we got some really good feedback over the weekend. we just told the user, open a GitHub issue, the factory will fix itself. So, that's the best way to contribute. >> And Ruben says, you know, it's been a pretty uneventful week. And maybe that's a great segue to >> kind of the topic of the the hour, I suppose. So, why don't we go ahead and just jump right into the news? >> Let's do it.

9:34 Welcome to Agents Hour. It is September 14th. It is a little bit around noon Pacific time. We started a little late today. We are on site in San Diego and we are going the to do the news like we do every week. And it was an eventful week. there's kind of one big topic. So, why don't we jump right in and you know, let's not beat around the books. Let's jump right in. Let's get to the topic.

9:57 Can see a little preview of some of the stuff we're going to cover. >> Hold on. >> Let's make sure this is actually good. >> All right. >> Technical difficulties. >> Yeah, we're in a new we're in a new studio setup in Riverside changed some things up on us. So, we're trying to figure out the some of the camera switching here, but I think we got it. We're good. >> Let's Yeah, let's just have this one.

10:32 >> All right. I think we're going to go. >> Okay. >> Not sure we can do that. So, all right. We're here. Let's do it. Pacing the frontier. Should we pace the frontier? What the heck does that mean? This is the topic. So, it all started with this post from someone, you know, unnamed person. No one really knew who this person was. They didn't have hardly any followers. They had never posted on X before. Just to be clear, this post got 171 million views. That is insane for someone who's never posted on X. It's honestly like algorithm breaking in a way. So we can talk about why that is, but Ned says, "I resigned from Anthropic today. I spent the last three years doing pre-training research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving super intelligence and gambling with our lives. More thoughts below." And then he goes on to talk about some of the reasons. He doesn't get into a lot of details, but basically talks about how people are not taking this seriously enough. even people inside Anthropic think that this is going to happen or it's it's possible and no one's doing anything about it. So that was that's how it all started. And then Evan Hubbinger, which is someone who is like head of I'm trying to know exact title.

11:58 Alignment science lead is Evan's title. >> What does that even mean? >> So he working on AI alignment, but Evan says, "Jacob is correct here. We really do earnestly believe AI could kill all humans." exclamation point kind of like he's excited about it. I personally think it is greater than 10% within the next decade. I believe anthropic is trying its best but we do not yet have a plan to solve alignment for super intelligence and are not clearly on track to.

12:30 So, this was the thing. I think it the original post was one thing, but this was the thing that got people extremely kind of bent out of shape for good reason because why like where did you come up with 10%. That's kind of important. Like what is the percentage chance that you think that we actually have we're all going to die and humanity? The way they talk about this is like extinction level ramifications. So like Jacob went on CNN, talked to Anderson Cooper. one thing I'll just say I just noticed very much from him is it's all highlevel not no details.

13:12 you know the the threat to our existence isn't here today. But if the if we continue down self recursive self-improvement then AIs will have ways to improve models themselves and then destroy the whole world which you know if you go on CNN like I feel that's just like such a doomer thing because the people watching CNN are not in this industry deeply and I guess it you know it makes sense to keep things general but it's a lot of FUD you know at least that's my opin opinion. I do not think a 27year-old researcher who pro who was like 2 years old during 9/11 should have any like any you know point of view here but maybe he knows something that we don't you know >> yeah I mean I think we should and we'll get into this a little bit we are not detectives we did our own research you should do yours but there's some kind of interesting sketchy things about how this became so super viral and how many people jumped on it so quickly for for this to even happen. A lot of things had to go perfectly right with the algorithm and so it a little like skeptical. I think in the first 15 minutes there was like three major news article reposts. There was a Wall Street Journal article that actually came out a few minutes before the post itself. So this was all like coordinated in some way. How like but obviously the other thing is it clearly resonated with folks too. And I think one of the reasons is just like human psychology, everyone like fear spreads faster than anything.

14:49 So if you can spread fear and panic, people will spread that faster than any good news. Like bad news spreads faster than good news, right? And so I think there was probably this bent up like everyone's on edge. Everyone hates data centers, you know, if you like talk about the just general perception, especially across like the US. So I think this is like there there's a lot of pent-up demand for people want looking for reasons to hate AI and this was like fuel for the fire so to speak.

15:17 Now let's continue. and the other thing about this Jacob Coxin is he did work at OpenAI for I think three years but Ananthropic I think he was only there for like six between six weeks and three months like it wasn't that he wasn't there that long. So it is interesting that he so quickly left and he is treated like he was anthropic for a long time but he actually like couldn't have had much impact and I would question how much you can even understand about a company in just six weeks and maybe it was a little longer.

15:49 I think it's still to be determined. so then this person Parker comes out and this also I'm only sharing posts that got a lot of you know, a lot of views because there there was so many posts around this topic. Tried to find ones that resonated with a lot of people. And it says, "This post looks like a start of a very sophisticated and wellunded PR operation. So this person with minimal followers and no previous activity goes to the Wall Street Journal, publishes an exclusive within hours. It has tens of thousands of reposts.

16:20 and then according to Grock, some of the repost accounts were like a lot of like head of AI policy, head of AI futures projects, a lot of like people that are pretty big into spreading AI doom narratives across, you know, the industry. And then a lot of, if you track down where the funding comes from, it's a lot of like people that are funding these AI doom NGO type organizations. And this person doesn't really have that much of a resume.

16:51 So, there's just a lot of if you if you kind of dig into it, it's interesting. It makes you at least think that maybe this wasn't completely organic. There was maybe someone like putting their finger on the scale, so to speak. and there's another one, similar topic. We'll just skip this. Bernie had an AI bill that, you know, Bernie Sanders had an AI bill wanting to stop all AI. No AI progress at all. Completely stopping. And one of the things in this is, you know, you get 20 years in prison if you try to advance AI at all. That's interesting part of the thing. And of course, if you do want all AI to stop, I guess maybe this is reasonable, but also seems a little wild of a claim. My thought is that they put out this really like crazy stop everything and then maybe there's a middle ground. And I think maybe that's what they're going through. Like here's the extreme scenario. Now we get some real regulation that finds some kind of middle ground. I think that's the idea is like you try to tip the scale by going to the extreme. And then this was the big response. So Daario from Anthropic puts out an essay that says we must pace the frontier. I've written a new essay on why the AI industry should slow down with a three-part plan for doing so. Anthropic is unilaterally committing to the first of these steps.

18:17 Do you want to talk Abby a little bit about what were those three steps and you know like what is anthropic committing to to at least start? >> Yeah sure. So well before that I'd like to just talk about like history of Daario's views here. you know so you know to go back in time here. So Daario was at OpenAI. you know, one of the first interviews I was researching was him on Lex Freedman's podcast. And this was back like 2016ish or you know, early days or even later than that. And that podcast was really cool like it was very much optim a lot of optimism and a lot of technical deep dives into AI in general. And there was mention of having governance then and then from founding Anthropic in like 2021 you know Anthropic is a PBC a public benefit corporation it still is you know he he won't you know Anthropic's mission is to do this to create to create AI in a safe a safe environment and then you Fast forward to 2022, you know, most researchers were doing RLHF and, you know, Anthropic was one of the first, research teams to be like, we should take humans out of this loop and have the models critique themselves.

19:56 but you need to have that in a very ethical way. so they were like making sure that, you know, this research is happening both at a fast pace but in a safe way. in 2023 he had his first judicial you know committee hearing where he did start talking about the misuses of AI. So there are biological threats, cyber security threats and he was advocating for external evaluations and he maintains that today. We're going to get into the the things that he he wants but even back then so it's not like some new thing. So 2023 he wants he he he he said all the problems that could exist and he his his first thing is hey we should have external evaluations on all frontier systems prior to it deploying in a mass quantity.

20:51 later in 2023 he this is where things kind of differ right like he starts thinking about how AI can get poisoned by commercial incentives and you know then they should have this rubric of AI safety levels ASL similar to a biosafety level BSL1 to BSL4 that same type of thing there now we're in 2024 that ASL framework has evolved and it's something that is part of anthropics culture. safety of AI, no dooming yet just and I think I would respect everything up until this point I do respect a lot because it's saying there are potential issues here are some guard rails and things we can do. but then you know this last year this year we have we have obviously gone in exponential rates of growth and I think that it's moving too fast for the ASL to contain. He went on CBS to talk about his first was let's say the first doomer take u where he went on CBS Sunday morning he talked about that you know the growth rate is so fast that you know the safety protocols are not catching up in time and the raw scaling will just outpace the actual safety that can come with it and that leads us to today you know we must pace the frontier the latest thing so a couple things that were established one there needs to be global coordination and that's a big contrarian view for a lot of Americans which is like US needs to own the AI boom but Dario's advocating for global coordination so across many governments should own the both the governance and the progression of this and so if you're a democratic nation maybe all major labs need to have safety baselines.

22:54 Second thing is this embedded evaluator concept where there should be third-party audits and government government regulation on on any AI deployed and then lastly if there should be like you know there's not only compute constraints there's these scaling constraints people hate data centers all this stuff and that is an advocate an advocation of pausing development or slowing down so That's the timeline. Now, when I go through the timeline like that, I'm not hating on Daario. Like, it is a natural progression of how this thing evolves. I think a lot of people saying that he's a doomer and he sucks and stuff is just because you're hearing what he's saying today, but if you do the research on his how his opinions have changed, it makes sense. So, like I'm not against him or anything, but the problem is I think in America we're very just worried about the Chinese labs and so any pausing is like telling builders to stop moving and making moves, which I think is what everyone's really tripping about. What do you think?

24:06 Yeah, I think if you were just to look at what Daario has said and you're kind of if you're a complete completely new to AI, you have not been paying attention other than you know it exists and maybe you've heard about it, seen some ads, this is going to really scare the out of you. It just is, right? if you combine this with the previous topics, right? And of course, Jacob's going on all these shows, Dario's now going on CNN. I mean, everyone is kind of sounding the alarm. And so yeah, I think pe a lot of people are going to be scared. I think Daario is actually coming from a good place. I think he genuinely does care about safety. I think the challenge that I have with Daario is I think he thinks that, you know, it's almost like main character syndrome in in a way where it's like he's the one or they're the ones that can do it and it should be in the hands of it's like the power should be in the hands of a few, which is always a scary thing. If you think about just governments and control in general is just if you want to like control it, well then how do we know that you are not corruptible, right? Typically spreading control is a better way to prevent that kind of thing. It's not perfect. These none of these systems are perfect, but centralizing all the power behind one like government, especially governments, multiple governments, centralized power does seem challenging to pull off and potentially scarier than the alternative where everyone has super intelligence, right?

25:32 so I think that's the challenge that I see is I think Daro is coming from the right place, but I think he thinks that he's the guy and only he can do it. And that that's the thing that challenge that's challenging to me. >> He does have the receipts though, like we're going to get into the other people who agree with him, but at least he has the receipts from the beginning and we'll see what the other other dudes are saying.

25:55 >> So Elon says Dario was right. And the funny thing is before this everyone like some people that are really pro Elon flipped sides at this point. They were so anti- whatever Dario was saying then Elon said it and they're like well maybe he has a point. It's so funny. You see the people that like you know flipped their position when Elon chimed in. But Elon also has a history of you know this is why he funded OpenAI at the beginning right he was worried that you know dominance by I think Google was a risk.

26:25 if Google controlled all the AI that that's a risk that having one company control everything and so open open AI was supposed to be a nonprofit supposed to be open. so I do think like Elon has some of the receipts as well at least as far as like how he he thought about it. I think he's always been worried that AI could potentially be negative, right? And we've all seen the movies, right? Everyone knows what it could potentially do. so then we have Clem Clement from HuggingFace said it's now clear that alignment is critical and won't be solved behind the closed door of a handful of Frontier Labs. So today we are launching the open alignment initiative led by Tom Wolf Hugging Face and asking to be part of the embedded evaluators program that Daario just committed to. So, let's make AI safer by making it more transparent. So, more people with credibility coming in. Sam Alman even came in and said, "I agree with Daario that we need to pace the frontier. This has been a primary topic of discussions we've had at OpenAI in recent weeks.

27:27 Committing to having independent evaluators with employeeike access is a great idea and we will do the same. We will have more to share soon." So, all you kind of have the big three. I would say the big three are all in agreement that this needs to happen. And then this came out. I thought this was kind of funny. You can't spell Daario Amado without AI doomer, which is just funny that that works out that way if you rearrange the letters. got a so so let's pause and just like it's kind of funny even if this is a very serious topic. It's it's a little humorous. We should we can all we should be able to laugh even in times of grave danger potentially. we should be able to laugh.

28:10 and then so David Sachs came out with a post and said, "Dario has written that we need to pace the frontier and Sam has agreed. People may be surprised by my response. Go ahead. You guys are the frontier by any reasonable metric. Market share, revenue growth, model capability. The two of you have a duopoly on frontier intelligence. You've also claimed the lead is widening because of recursive self-improvement. I don't see what you see in the lab. If the unreleased models are scary enough that you think you should slow down, I support your decision to be responsible.

28:42 So on top of this, he also says, most of all, stop pretending the motivation to slow down is purely altruistic. You face massive product liability exposure if your products enable a truly damaging cyber attack. The market already punishes models that behave in unpredictable or unauthorized ways. After the hugging face episode, it is simply good business for OpenAI and Anthropic to trade some raw power for reliability and predictability. Call it alignment if you want. It is also just giving customers what they want.

29:12 and then there's a bit more talks about Bernie Sanders's take and basically says if you you should go ahead and do it and if you do, you'll buy goodwill for the next conversation. If you don't, we'll know this was just another bid for regulatory capture or an election season SC up. So, that of course got a ton of views as well. A lot of people resonated with that. meanwhile, this came out breaking. China's ZAI has raised 5 billion. The interesting part is where the money is going.

29:45 They're saying 60% of the net proceeds will fund next gener generation GLM models in what ZAI calls its fully self-training system. Sounds like precursor self-improvement to me. So the question is if China's already investing to do it and they're investing a lot, can we actually pace the frontier? Maybe. But I think that's why a lot of people are worried because you can't actually control what everyone does. We can only control what we do. And so that's some concerning facts. Gavin Baker came out and said wild 24 hours and a lot of different proposals have been made. The only tangible new fact is OpenAI and Enthropic are going to have embedded thirdparty evaluators. We talked about that. Dario floating meter as a possibility for one of those third party eval evaluators.

30:38 Gavin has a good summary of the events. We've kind of gone through them all. So, I'm not going to reiterate them, but if you're looking for a good highle summary, Gavin's is a pretty good one. Yan Lun jumps in and says, "You know, pessimists arc is right. Dario was already claiming that GPT2 was too dangerous to open source back in 2019. I made fun of them then. Everyone should make fun of them now." So, that's Yan Stake.

31:05 >> Legendary. unrelated or somewhat related to pacing the frontier. You know, Tibo from OpenAI said that they're pausing signups to their 200 pro plan. Mostly I think this is compute related, but it the question is like are they just trying to improve or make sure that there's enough compute to go around? Are they running into scaling problems? Is this you know one of the reasons they want to like slow down the frontier a bit? Is it is it connected or not? I guess that's you know a bigger question.

31:37 I'm not sure. What do you think? >> Yeah, my take is there is definitely some strain on capacity, but also like I mean this came out before all the doomer stuff or at the same time. So, it's kind of it's kind of weird like I guess we should have all bought a couple more subscriptions because now we can't. but there's definitely something going on with capacity. Fable had the same issue, right? there was so much demand that they had to knock it off the subscription and so you know this is unfortunate. I think there's a lot of backlash in the developer community but what do you expect? You know we've been subsidized for so long.

32:22 >> Yeah, I think there there's just going to be some backlash to that. Mark Zuckerberg chimes in says AI development needs to move faster. So he's he decided he's taken the opposite take of the other the big three so to speak. He's maybe the if there's a Mount Rushmore of like AI like frontier model lab figures you know Mark Zuckerberg is like not quite on the monument but he's he's trying to get on it I guess. And so he just says no we need to keep moving faster. So I thought that was pretty funny. I guess how do we want to s summarize this? What is your overall takes in should we be pacing the frontier? Should we you know what, you know, for people like us, we're builders, we're out here using the things. Does this affect us? How should we be? How should we respond when the you know, my mom or my grandma says, "I saw this CNN article. Is AI going to kill us all?" Like, what do you think?

33:18 I'll I'll give my take. And and you know, I will caveat this by saying we're not in the labs, right? We don't know what they've seen. We only know what we see in the outside world. And so, you know, that that's our take, but what do you think? >> So, funny enough, I went on a date last week and when we talked about my job, the first question she asked me was like, "Do you think the AI do you think AI is going to kill us?" And she's not in in tech at all. So, like that is the general sentiment because the news comes from these, you know, the CNN's and everything. so from that perspective, it it's kind of like we're not at the point of this extinction level thing. and a lot of people are worried about it. So I like just feel bad because a lot of people get anxiety from these news articles and things and it's just not an it's not an anxious point today. for builders though you know if the models stopped getting better our life doesn't really change that much because we are building with the models at every part of its like growth. So if it stopped today, you know, we may not get anything better, but we're already building harnesses and the tooling around it to try to make it good enough. so I don't really think it affects us as much, you know. What do you think?

34:35 >> Yeah, I do agree with that. If everyone decided, hey, we're not we're going to define what the frontier is and we're never going to pass it. I think there's still so many un like we we basically haven't figured out all the things we can even do with it today. There's a a lot that could still be done. Now, the challenge is I don't think it's actually going to happen. I don't think people are going to really slow down.

34:57 I would say, you know, we've seen the hugging face incident. And if you really dig in, there's some interesting things there, but also like I kind of would have expected that outcome given what the models have been trained on, what they would do. I think if we saw like a dozen hugging face level incidents, then I I would start to get more worried. I think only seeing you know a few of these that have come out I do think that the gap from going from there to like full scale AI takeover controlling everything all these external systems without any kind and of course there's no like perfect off switch to this but it's all comput and data centers right I don't think that we're at this point where this whole thing is going to be taken over in an uncontrollable way in the next couple years now maybe in the next decade could it happen I'm not an expert it's possible But I see like the models that they are seeing must be so much better than the models we have if they're worried about it because I can't get my model to do a simple task sometimes where I'm just like, you know, you're just like missing obvious things, but yet I I think this model is going to full-on take over the world and kill all humanity. That seems skeptical to me, but again, maybe we don't see all the models that they have cooking in the labs. So, I guess our tip to everyone is invest in a bunker and then you'll be fine either way.

36:22 >> Yeah, I I will say I'm not investing in a bunker. I'm not overly concerned yet. I think we should be careful like this stuff is dangerous. We should have some safety guard rails. I think there are I am worried about too much overregulation in that like one controlling body controls all progress and everything has to go through that because I think if we slow down too much you become like some of these other overly regulated countries who can't really be competitive and then I think if you're in the US you'd be concerned that well is that actually going to slow down other countries that are doing the same things and don't we want the best intelligence to protect ourselves?

37:01 potentially. >> I'm selling pre-sale tickets 2036 to OB's Bunker. You can Venmo me Cash App, you know, but if you're really worried about it, I got you. >> Yeah. All right, let's let's get off the the Doom topic, even though this next one's kind of related. Safety and practice. Enthropic published a detailed threat intelligence report. It covers how people misuse claude for cyber attacks, influential influence, operations, surveillance, biology, and building weapons and how they found and stopped them.

37:34 And you know, Zihad, I think is how you pronounce that, had a pretty good take on at least a summary of all the all the things that were in the report if you don't want to read the full report, but there was, you know, here's some TLDDR from it. There's a suspected Russian state- linked group using Clawude across fishing intrusion, data theft, and ma malware development, including rebuilding malware after security products detected it. so there's shiny hunters linked hackers used AI agents to scan 1.8 million Android apps for exposed secrets and help run breaches across multiple companies. A China based group built an automated exploit setup that could research vulnerabilities, build offensive tools, and keep working against target with little to no supervision.

38:19 so one consultant used it to build Lacana 360 for Mali's intelligence service, a system designed to monitor roughly 25 million SIM cards across the country, including calls, messages, voice interception, watch lists, and automated intelligence dossas. a whole bunch of things. There's a Yemen group, another Russia attack. they talk, I think they talk about the, distillation from Kimmy, from deepseek zi. They talk about all the different how moonshot was basically distilled from a lot of these like Chinese models were distilled from Frontier and that is one of their one of Dario's takes and one of Enthropic's takes is if we're moving the intelligence forward and then we're getting distilled by these Chinese companies we're actually helping them move faster. so which I can agree with like that.

39:08 >> I think the wildest thing was Anthropic claims that Moonshot, DeepS were routing requests to Claude instead of their own model for some users without telling them which I think people have said that's that's not true or there is true whatever but that is quite interesting. And then if you see all these like we you know weapons six weapons programs are using anthropic etc. Like all these AI personas on dating apps u people trying to do these griffs with old people like yeah dude safety is important you know and I think anthropic is looking at this list and saying we don't have an ASL to protect us from this right now so we probably should slow down because we have no guardrails for this.

40:01 OpenAI on September 10th said, "GBT live one is now available in the API. Bring chat GBT's natural back and forth to your app with voice agents that listen while they speak and work with the models and harness you choose. So now you can get real-time voice right directly in your app." app. And I think this is something that a lot of people have wanted for a while because they interacted with, you know, chat GBT and the voice was pretty good comparably to a lot of things you could build yourself. And so I think a lot of developers will like that they can maybe just use their voice mode directly.

40:35 Cursor introduced projects. This is from September 10th. It's a new way of working in cursor. Rather than creating a chat for every task, you work with a coordinator agent in a single persistent thread. Like bot, your agent is always on. Proactively manage is work with sub aents and improves over time. On September 10th, we also had DeepSeek V4.1 Flash. Smarter, faster, more efficient. So, it's the smallest model in their new architecture family.

41:06 And >> it's good. >> Yeah, it's pretty good. If you look, it's it's like >> if you don't get rate limited, it's great. >> Yeah. For And for how small it is, too, right? It's like it's impressive. It's impressive that you can get such good levels of intelligence on these smaller models, >> but you don't know if it's hitting Claude or not, but you know. >> Yeah. Yeah, that that is true. Maybe it's just, you know, it's it's like the Scooby-Doo thing. You pull it off. You pull off the hood and it's like, oh, it's just actually Claude. Maybe that's why. Cognition released Devon voice.

41:36 Your favorite AI software engineer just got a landline, so you say it, Devon ships it. So, it's powered by GBT Live. and our new sweet 2 model. So there you go. this is tangibly AI related, but I think if you if you remember back, Tailwind had a lot of they had some layoffs because no one was going to the Tailwind docs anymore because AI was just reading the docs for you, right? This was probably 18 months ago or so, a long time ago at this point. And so it's kind of full circle. It feels like Tailwind is, you know, kind of winding down, so to speak, and becoming part of Shopify because a lot of people use Tailwind and don't even know it these days, which is, you know, unfortunate for the Tailwind team because they built something that everyone uses, but no one pays for.

42:21 >> At least they have a home now to keep continue working on it. >> Yeah. So, congrats to them. And I think that it is good because again, so many people are building web apps these these days. So many agents are building web apps these days and a lot of them are using Tailwind under the hood. All right, this last one. What is this, Obby? You want to tell us about it? >> The death of agent plugins maybe. so yeah, in the latest MCP spec, you know, we talked about MCP's v2. This is another feature which is super cool because you can, define extensions that load agent skills, alongside the MCP server. So if you already have an MCP server, you can then bundle your skills within the server and then you can grab skill metadata. You can then use it as like you know how skills work is agents have to load skills into context to then use them. Now you can do that through MCP. So why ship an agent plugin when you're already shipping MCP servers? Which also makes me think why anthropic was not involved with agent plugins? And I would agree that this is a way better delivery mechanism in my opinion.

43:31 >> Yeah, I think so. I mean, we will see now we have like two standards trying to do some of the same things. We'll see who who ends up in >> MCP is having a revitalization right now. >> Yeah. >> And this is a perfect like add some more fuel to that fire. >> Yeah. I think you know a lot of people kind of lost sight of MCP even though enterprises didn't. It's still very well, very widely used across the enterprises that I that I talked to. But I think people, you know, it was very trendy and then it kind of diminished and people, you know, were saying it's not actually that good of a spec. But, you know, with some of the new stuff they've released and the two 2.0 spec plus some of this, it's having a bit of a re revitalization. I think, you know, MCP is back, at least as far as like hype terms, as we kind of alluded to earlier. There's all these hype terms and I think MCP is it's still it's still got a little hype behind it. But that is it. That is the show today.

44:27 Thank you for tuning in to Agents Hour. We do this every week. We talk about the news. We give our takes. Sometimes we're right, sometimes we're wrong. You be the judge. We'll let history decide. but you follow us at MRAA if you want to see what MRA is talking about, all the things we're shipping. We ship a lot. Check us out on YouTube so you don't miss any episodes. Please give us that subscribe, that like. That helps other people find the show. We appreciate that. I'm Shane Thomas, SM Thomas 3 onx.

44:54 That's Abby at Abby on X as well. And with that, I guess we will see you next week on Agents Hour. >> Peace. >> See you. >> Yo, that show. We were live in the zone. They agent Shane and I be on the throne. Did you give us that review? Only if it's a five. Jump on the tube. Make sure to like and subscribe. New so fresh. Yeah, we keep you in the loop. Get so fly. They bring the whole troop. AI on the rise. Don't miss this power. Welcome to the show. It's AI sour. Did you just drop in? Is this your first time? Make sure to follow us on X and go like and subscribe. Yeah. Learn the principles and patterns in our books. The master.AI site. Give it a look. New so fresh.

45:43 Yeah. We keep you in the loop. Guess so fly they bring the whole troop. AI on the rise. Don't miss this power. Welcome to the show. It's AI sour. This is the end. We all wrapped up. Another showdown. Another one coming up. AI agent sour done but the news doesn't cease. Shane and Abby, we out of here. Peace.

Summary

The episode of Agents Hour features Shane Thomas and Abby Ayer discussing the latest developments in AI, including a significant launch from their company, MRAA, and a controversial post regarding AI safety and existential risks. The hosts delve into the implications of recent statements from AI leaders, the public's reaction to AI fears, and the ongoing discourse about the pace of AI development.

- MRAA launched "Monster Factory," which received positive feedback on Product Hunt and aims to automate software development processes.
- A viral post from a former AI researcher raised alarms about the potential risks of AI, claiming a significant chance of existential threats.
- Key figures in AI, including leaders from Anthropic and OpenAI, are advocating for a "pacing the frontier" approach to AI development, emphasizing the need for safety evaluations and regulations.
- There is skepticism regarding the motivations behind calls for slowing AI progress, with concerns about market competition and regulatory capture.
- The discussion highlights the balance between innovation and safety, with contrasting views from industry leaders like Mark Zuckerberg advocating for faster development.
- Anthropic's report on AI misuse outlines various threats, including cyberattacks and surveillance, underscoring the importance of safety measures.
- The hosts emphasize the need for transparency and collaboration in AI governance to mitigate risks while maintaining progress.

Questions Answered

What is Agents Hour about?

Agents Hour is a weekly show hosted by Shane Thomas and Abby Ayer, focusing on AI news, guest insights, and problem-solving discussions.

What significant news is being discussed this week?

The hosts discuss a controversial resignation from Anthropic, highlighting concerns about AI safety and responsibility.

What is the historical context of AI governance discussed?

The discussion traces the evolution of AI governance views from early optimism to current concerns about safety and ethical implications.

What are the industry leaders saying about AI safety?

Industry leaders are calling for a cautious approach to AI development, balancing innovation with safety and responsibility.

What are the current threats associated with AI technology?

Recent reports detail how AI is being misused for cyber attacks and other malicious activities, highlighting the need for vigilance.

© transcribe · For agents Built with care and craft by Gokul Rajaram