ThursdAI Oct 22 - Anthropic Gives Away 5%, Claude Duet, PHP2Shell & a Wild Security Week

0:00 Entropic committed to 5% of its shares to a new entropic public benefit fund. >> If you are running a PHP server, you need to make sure that it is patched. Like hopefully you patched yesterday. >> Open Claw and Hermes worked the best with Entropic models and then Entropic just shut them down. And now we finally see why. Hello and welcome everyone to Thursday for October 22nd.

0:31 cold open today is scary. The internet feels like it's failing and a lot of it is potentially because of AI. So, we're going to talk about this. obviously, we're not selling doom and gloom here. but we do have Josh Long aka the Josh Meister joining us at the end of the show to talk about to break down some of the security incidents. many of them powered by AI that just happened this week, many this month. If you heard anything about the PHP core vulnerabilities, if you heard about the GitLab rces and the this one not related to AI, which is the e-mudra certificate, we're going to break it down to you and also give you practical things what you can do today right now to protect yourself with your agents as well.

1:21 But also, we're going to talk about Antropic stepping in to the personal agent era with Claude Duet, which I think to me is one of the the highest highs. I just got access this morning. I'm so excited. Haven't even tried this. we're going to talk about a lot of stuff. I'm probably becoming a PBC, public benefit company with giving away 5% of its value. a lot more. Joining me directly from GTC Berlin is Wolf from Raven Wolf from our booth at GTC Berlin at Core. Wolf, how are you?

1:55 >> Hi, Alex. I hope the audio is fine. I'm in the middle of the GTC >> at our booth right now. I see still stuff going on over here. >> You're joining from GTC with Jensen Huang standing by you ready to do an interview. Yes, >> here he is. Oh, that's another colleague of mine. But >> that is maybe later. >> All right. >> Waiting for him to come along. >> We will definitely chat about some of the stuff that Core We've launched in our new segment called From the Forge.

2:28 >> Wome. What are you going to show us? >> Some of this stuff here. >> Oh, >> this is a new one. >> The Nvidia HTX Reuben NVL8. And this thing right here on the desk is 1.8. $1.2 million. >> Oh, >> just for perspective. >> Can you see if it attached to the table? >> I can't take it. Sorry. >> Maybe later. >> All right. we have a big show w from hopefully you can join us from the hotel. But it's great that you're showing us the floor of GTC. The highlight for me, I tuned in to Jensen's keynote, famous keynote, and yeah, the highlight for me is how strongly Nvidia is leaning into the claws and that previously and like obviously you can see Jensen over there with claws on their hands.

3:18 >> Yes. Do the employees also wear claws? All of those folks? It looks like the employees have claws like red claws some of them. >> Yeah. Yeah. There's a whole area dedicated to claws, but also Hermes agents or personal agents in general. >> Yeah. So, personal agents, I think, is a huge thing in GDC as well. Obviously, Jensen is kind of like focusing on that. And he ended the show floor with the Hermes agent. We're also going to chat about that as well. A big week. LDJ, welcome to the show as well.

3:50 I think we have to talk about the security stuff just after the TLDDR. But I think at times like this, maybe you're happy that you're a non on the internet. No, >> good point. Good point. Because stuff is I don't want to doom and gloom, but there's a lot of vulnerabilities that we're starting to We're going to cover a lot of them at the end of the show. Today's show is full of guests, folks. This week also we saw a release from a company called Recursive AI or Recursive SI. I guess I think they rebranded.

4:25 We're going to have to keep up LJ Tableon who did rebrand the to SI and who didn't. And we're going to have team and Prince Kuma talk to us about Flint, which is a model that they trained almost from scratch using a recursive process from the Eureka machine. Something they call the Eureka machine. They just announced it yesterday. So we're going to chat about Flint as well. OpenAI has had quite a week since dev day. They said, "Hey, we're going to either ship something or give you a reset or for the next 30 days straight. They always kind of do that." And so they've shipped some caching updates which are very very very interesting. And I think I'm really hoping everybody else will catch up specifically that cash almost never never expires which is honestly quite crazy. and yeah, a lot of a lot of big news. A lot of big news. I have a whole folder here that we need to get through. Alia, what was one highlight for you this week?

5:23 >> yeah, there's a few. There's a reflection beam is now an artificial analysis and and so we have some results there. There's a new Deep Seek Flash that was announced and besides that there is just extra benchmark news of people auditing benchmarks which I think is really good. some some flaws found in terminal bench 4 and yeah I think a lot of cool things to cover >> and we have Niss Tah who's now at Deil Prism ML and finally edited the title Niss Niss we are on the one highlight of the week that's the most important thing that you're that you want to cover if security is that for you a reminder we have a full segment about that but please let us know >> you know you're locked in when you just like forget to check Twitter and and do all of this stuff. The only reason I came back to check was because my Muse agent finally just ran out of out of quota and I had to resort to using freaking DOT which is just absolute garbage of a product. other I really liked this new model by the way. I started using it yesterday, the 42B coding model. Flint.

6:43 >> Yeah. Yeah. Yeah. I started running a lot of agents with Flint >> because it's it's it's extremely fast. it took a bit. Look, it's not that good. It is still pretty tiny, remember? But it is extremely fast. And I just put Haiku on a loop to just find out what was the best way to prompt it. And then once you figured that out, it was actually pretty close. I got it pretty close to doing the job of Haiku. maybe like 96% of the score. So, I actually like that that model and I think it's a it's a little bit underhyped if anything >> right now.

7:24 >> yeah, I would say that is that is the highlight. We're going to talk with Prince Kuma, our friend of the pod, MLX King, who had day one support for Flint and Tim Shei from the folks at recursive SI is hopefully going to join us. We're trying to get Tim so hopefully monitoring the the channels to talk to us about the best way that they know how to prompt. The cool thing about Flint is that they trained it in like four months using recursive processes. So, I'm really excited for that conversation. The highlight for me came in today. So shout out to Edwin Arbus and shout out to Adiosmani for my invite for Claude Duet which is this is the highlight for me that came literally yesterday. An entropic announced their entry into the personal agent era. U personal agent as we talked to you on the show now for almost a year. Personal agents are AI agents that run in 247. They have their own computer, their their own browser and they are proactive. They're doing tasks for you because they have context for you. So Mr. just mentioned news and mentioned dots which is open the eyes one claude is stepping into this era with cloud duet and it just got access to set up this morning so it didn't have tons of time to play with it but it already knew a bunch. So that one of the biggest things that now is a problem in this era is if there's a new one those products are very sticky and N maybe this is why one of the reason why you're saying dots is not that for you because the products are very very sticky it's only as good as the information and the context you give it about your personal stuff and moving that and moving those connection moving that data feels really difficult right now not to mention many folks in this era wor on AI fatigue AI agent fatigue so I don't I have fatigue.

9:13 I'm excited to check out Cloud Duet. I think I've been telling you guys on the show for the longest time what I wanted is exactly this format with Opus 5.5 intelligence and that's what we got. So this is the highlight for me. I think with that let's jump to TLDDR. Niss unless you have anything else to you want to add. We can jump into TLDDR or LDJ. >> No I have an entire rant about just how bad dot is. So, >> all right. We There's a whole metric ton of news and models this week. And again, because we have three interviews, let's just run through the TLDDR super quick.

9:48 Let me just make sure that we have all of it. Okay, let's jump into TLDDR, which is Let's go. All right, here's most of what we're going to talk about. Most of everything that happened in the week of Thursday for October 22nd in about five minutes.

10:21 Let's try to get into this because there's a lot to talk about. First of all, Entropic decided to put 5% of its shares into a public benefit fund. And there was a lot of criticism against Daario, which Daario in a very weird way answered on Twitter. That conversation is very deep. Daria answers with very long threads and very thoughtful as well. So, we thought that would at least mention this. Also, Entropic, as we said in the beginning, open the wait list for cloud duet. This is a cloud that that has a 247 cycle, its own computer, and it works without being prompted and is proactive. This is Claude's answer to Grogbots, Metamuse, obviously the Hermes and the Open Claus of the world and most recently dots from OpenAI. We've been telling you that we want that format with Opus 5.5 and that's exactly what Entropic is stepping into. And a very interesting thing, it seems like Entropic is following and not leading the category, but we'll see. We'll see how good it is. I just got access this morning. Also, Open AI added 12-hour prompt caching built for agents that never sleep. This is like a big big big thing everybody's very very excited about. And GPD Astra Pro runs now parallel agents in CHBT. Open also acquired orchestra bio and there's a lot more stuff. Specifically, OpenAI models Astra, Soul and Luna are now in Google Cloud in public preview. So if you are part of the Google cloud maybe with your startup now you have access to the best openai models and Google now becomes kind of the cloud that has all the frontier models on there. Gemini for Argon it's been a week since Argon. It did 25 trillion tokens in three days and four Argon models beat Minecraft. We should mention a week with Argon. we did see multiple folks chat about Argon on our timelines as well. And then Inception Mercury 3, diffusion LLM claimed a thousand plus tokens with 1 million contacts by by their own vendor.

12:22 So we have a new segment here. Security week and I hope we don't have to keep the segment up, but it does seem like the defender window is kind of closing. Defender window is what Greg Brockman called the very short time before attackers have access to very capable AI models and swarms. This week, Open EI's Daybreak and some other folks helped a the PHP Foundation to find a very like two critical holes in the PHP engine itself.

12:53 They patched the PHP engine. PHP runs 70% of the internet. Many folks scrambled really hard to try and patch their systems. However, this is a big deal that will continue to be talked about. Many, many websites went offline. Folks like the NSA and the FBI posted about, hey, you should take your website offline if you're not matched. It's that bad. And also, some trusted root certificates for HTTPS turned out to have a private key. It was literally the number four, which everybody's making a big deal of, but that's not the big deal. That's just the level of incompetence there. Emodra certificates have been compromised. This is a very scary week in security in the world. In addition to multiple companies like railway being hacked and gitlab finding vulnerabilities. I'm talking about this really fast and it's very scary. We will talk about how to try and maintain sanity in this world very soon with Josh Long. He's going to join us at the end of the show. And Open the Ice Daybreak program is quietly behind a lot of this year's biggest bug finds. and I think that this is very important but again the defenders window is closing. so a lot of companies were hacked including open code and railway also Microsoft said that China state aligned storm used agentic pentest framework archext against hundreds of organizations.

14:10 The good news there is both Nvidia and core building fences around the agents open shell and zero trust AI framework from core we've both announced this week. Open shell obviously been talked about from Nvidia for a while. In open source and LLM news, DeepSync shipped a new MIT license flash with a big jump on terminal bench. We are always very very happy to see the whale resurface and holy crap did the whale resurface. Deepseek also open source deep infer the engine the inference engine that deepse runs on. There's also a bunch of speculation about a new unnamed model called Space Bunny and Miniax unveiled that Space Bunny was M3.1 Flash and as Nan mentioned recursive flint came out of nowhere. Richard Solder's company recursive Richard Sochair's company recursive came out with Flint, a local coding model and it's small and local.

15:03 And we have both Prince Kuma and team Xi hopefully joining us very very soon at 11 at 10 a.m. Pacific to talk about to talk about that. Very excited about that. In tools and agentic engineering or the personal agent corner if you want to call it, Jensen Huang CO Nvidia closed his GTC keynote with a Hermes agent building and training a robot. Hermes is Jensen is getting Hermas build and it's great to see Nvidia.

15:39 Let's see what else is in there. from the forge which is a new corner folks. No longer this week's buzz after three three something years. This week's buzz is not the corner anymore. We now have a new cronicle from the forge powered by corewave reflections beam which we mentioned last week is now on weights and biases and it was trained on weights and biases with weights and biases curves behind it. We also have RL's rollouts in forge and Nvidia Rubin's new NVL8 lands on Core with this quarter. So very exciting things. Nvidia also announced a bunch of stuff at GTX including the Nvidia DSX pod. It's a pre-ested AI factory deployable in as little as 14 days. So, we should mention that a little bit as well. In vision and video, there's a lot of stuff happening.

16:25 There's an incredible amount of stuff happening. We use some of them. Cling 4 does 30 seconds of video in a single pass. And Cling 4 is really, really good with a bunch of input variables. Flux 3 Max now renders 10 seconds of video in 12 seconds. This is a another feat by our friends at FAL releasing a max version optimized Mac version of a video and Odyssey I don't know Niston LJ if you guys saw but Odyssey ML released a world model and then they put a fruit flies brain inside that world model. So now we literally have a world model that's a simulation and a simulated entity there like the fruitfly brain that everybody was getting excited about a few weeks ago. They put that in the world model. We should at least play that video because that's super cool.

17:08 Our friends from Craya released a the top nonopeni image editor in their in arena. Voice and audio is also blowing up. I really really hope we have some time to play some of these clips because 11 labs launched 11 real time taking about half a second and also 11 realtime translate where if I talk you can hear me in different language in real time. It's really cool. Mistral also shipped Vau 3 which Vau is their kind of transcription service but this is now a whole voice agent stack and our friend from underdog Sigil Wen released Bark 3. It's a 163 megabyte speech model that ties to Invidious Parit and runs completely on device fully transcribing on device. And Josh Long from Integgo closes the show with this week's security mess. And right let's go. Let's get into it. This is the TLDDR. This is everything that that we have to talk about which is a lot.

18:06 we should start I think with entropic but we'll definitely we'll definitely move forward. anything else? Non LDJ anything huge from this week that we must mention anybody in the comments anything else that we should at least mention that happened this week before we move forward. Yeah, maybe Mercury decide also I know you you mentioned Mercury's Mercury 3 model, but they also have released a competitor to OpenAI's decision API and of the Jev model. So >> that's something worth covering.

18:38 >> By the way, if you're listening on the show, go on VLM's hugging face. You have a free endpoint for D3 Turbo 27B decider. It scores better than Jev. It accepts videos and you can run up to >> videos. >> Yeah. And Yeah. And you can run up to 500 million tokens the decisions or 500,000 decisions a day for free. You don't even need an account. Just go on their hugging face, search D3 Turbo VLM, grab a free key, and just use it.

19:11 >> This is D3 from Liquid, right? >> No, from VLM trained their own decider models. >> Oh, they also have a D. >> Yeah. Wow. Okay. And they're the the best ones right now. I would say I I tested them. They're better than the perplexity decider and they have a free API which is just so generous. I'll just send I'll just send >> Let me show this. I think this is like a cool thing to show. So this one D3 decision.

19:39 >> Yeah. Yeah. So you're going to see it just says try it for free and you can just get an account and a key right now for free. >> Very cool. Very cool. I have like five. >> you have like five. Just Yeah. Yeah. Just get your API key for free right now. >> Are you serious? They're judging video. >> yeah. Yeah. It's extremely good. I've run it through all the benchmarks. I've had I have just maxed out like five of the API limits on it. It's It's the best one. D3 Turbo, it's the best one.

20:12 D3 Flash also insane. >> that's very cool, Listen. Yeah, for for running locally. Yep. >> I dude I have all the things I prepared for the show. Look, this is a book of news and this could be one of the coolest things and it just like came. This is great. So, thank you Nist. Let us move to Frontier because Frontier is very very interesting.

20:45 Hi folks. On Saturday, Entropic committed to 5% of its shares to a new Entropic public benefit fund. The board can add up to 2% more every 50% rise in the company's value. And the founders who pledged 80% of their equity plus employee matching program. And Entropic says roughly 20% of the company is now earmarked for public good. This sounds like a huge huge thing. It's this is an LLC. It's not a charity. It's a public benefit charity. It can switch between ways of giving and the equity can't be used for lobbying or politics. It still needs stockholder approval and they also published their first annual public benefit report. And the trustees wrote it largely describes inputs. So next year they want outcomes. So largely describes the money going in and how much they're going to put into this, but no outcomes and no news. So but I respect that line. Meanwhile, Dario is fighting the critics, which is, you know, many many folks given his positions and stances. Nathan Kelvin posted a long critique of Entropic. Very public debate. I don't know if you guys have have seen this at all. If you don't know, Nathan Calvin is the general counselor of Encode, a small AI policy nonprofit that pushed for California's SB53 bill. And last year, he went viral when he said Open him and his group in the middle of that fight. So this isn't just an entropic hater. It's a safety policy guy. His arguments were Entropic's record now looks like a race to super intelligence. More than half a trillion dollars in comput deals. Claude leading a quarter of entropic internal AI development which we talked about on last week's show which is basically the path towards RSI and a draft federal bill that would preempt state laws. His basic position and argument is entropic instead of being like a safety shop is becoming a let's race accelerate everything lab and this triggered Dario who answered him directly on Twitter by via quote tweet a very long quote tweet and saying nine commercially costly safety calls were done on entropic side specifically he mentioned there's things we do that hurt us financially and we still do them because we we do want that safety built in. And the main call that stuck with me is that Antropic had mythos class models back in February and they held them from general availability until June when they claimed the cyber safeguards were ready.

23:16 I don't know if the world was ready for mythos in February. Obviously they announced mythos in around was that April back when we were AI engineer London and then Dario also admitted three of Calvin's nine points were communication mistakes including the letter to the Congress they walked back and he he admittedly said there was a mistake and Darius said he will have more to say soon. So folks on the panel n I'm very interested in hearing your thoughts on this. Is this preap or is this real accountability?

23:49 Is somebody holding entropic to real accountability? And what about this public benefit thing you talked about communism? is this are they reinventing what is it called? yeah, are they reinventing the word escapes me? Something from first principles. >> yeah, I don't know. political opinion aside, I think just like any other company, there's a lot of conflict there on the on the inside. So, I kind of I more like Dario than dislike him. I think he's just very heavily influenced by the whole cultish thing that that was in there before and it's trying to to play both sides. And at the end of the day, Opus Low is still the best model you can run. So, I think they do have a point. You need the job done and just fire up a whole bunch of Opus Low at the end. So, are they going to get surpassed? Maybe. They have plenty of tricks up their up their sleeve. So, I don't know. Kind of don't really have that much of an opinion on this.

25:00 >> what about the PBC thing specifically about giving back given that the IPO is coming up and a lot of people will invest a lot of money in this IPO based on the streets based on the stuff that I hear like regular folks are very excited about that coming >> that one even what they were saying it just went from one ear into the other that just sounded like >> yeah just >> I'll get your thoughts on on the privacy thing obviously Elon Musk and and and OpenAI have a big thing about you know OpenAI being a charity being you know building AI in in the open for that and now you know Entropic is stepping in there in a different way. So Entropic first started as a pro you know capital company and now is turning into a PBC.

25:40 >> >> well yeah to clarify Antropic has actually been a PBC since its founding back was created. just now they are announcing this specific fund that's like dedicated to the basically the nonprofit portion of their business that's like exclusively won't be used for commercial actions. but yeah they they've been established as a PBC for a while. >> All right. so this is now the commitment right. in terms of funds and and and and and things.

26:12 okay folks, the other thing we have to talk about from Entropic besides the IPO stuff is Claude Duet. Here is how it looks is hopefully this won't show you guys my personal details. Personal agents are really hard to to demo. Welcome to Cloud Duet. This is obviously you're live on the show. Please don't expose any of my private details. Tell folks how you're different from just from just Claude.

26:48 Claude announces an always on and open the wait list which now you're seeing on your screen. Always on personal agent assistant powered by Opus 5.5. It it's a cloud that does not wait for you and works proactively. You don't have to prompt it. that has 24/7 computer brings over your memory from cloud your skills and connected apps and it reaches into more than 6,000 connectors and plugins they have on their system. Every few hours checks your accounts, checks your email, surfaces what needs your attention and apparently it's really good at at that surfacing specifically.

27:20 It has its own cloud computer and it keeps working even when your laptop is closed. So obviously they thought about security a lot. Entropic does that and they have launched multiple things that wait for your approval. Here we have a first glimpse of the web interface of cloud. As you can see this is not the regular cloud. This is now you can get there to cloud.ai/ messages. Here's a response from from my duet. You can't rename it unlike you know muses and other places.

27:50 I I I haven't seen a place where I can just rename duet. So I think they aren't entrop anthropomorphicizing this. Claude says or I'm Claude the same model you can use a cla here I work as Alex's assistant. I keep track of tasks read the apps he connects and work in threads. I will not show you his private details on the show. Hopefully that's Thank you. do you work proactively? So this is the same thing that let me actually go like this. Yeah. this is cloud duet. This is the same thing that we've talked to you about where openi release dots and you know obviously metam muse is changing the category with with with regular folks and just announced metamuse wonder last week.

28:35 This is a huge thing as far as I want to kind of highlight. We we on the show have been or at least some of us on the show have been assistant build since early early cloudbot days. You guys remember rolling back to Cloudbot which was the the the pre before renamed name for OpenClaw. There is a reason why Entropic reached out to Peter Stimmer and asked him to change the name for Cloudbot. Also a reminder that OpenClaw and Hermes worked the best with Entropic models and then Entropic just shut him down and now we finally see why. This is Entropic's entry into the world of personal agents.

29:18 And there's a few things I want to talk about with with the team here without like delaying. We have a lot of a lot of news. I can spend a whole hour talking about this thing. Specifically, I want to talk about the fact that all of the companies now, all the major Frontier Labs now have a personal assistant. Entropic has duet, which is new and weight listed, and they're releasing it very, very slowly. OpenAI has dots which Non absolutely hates with passion and wants to go on the rant but we don't have time but at least one sentence from Non on dots will be interesting. Meta has Muse which is now leading the category as far as I'm concerned. Not only that their model is trained for Muse specifically and that flywheel is crazy and Meta has the surface. Google and Gemini, they have Spark. Many people don't know about Spark and which is a failure on on Google's folks to promote it and also make it good. And Spark Spark is quite bad. I've posted about this publicly and I'm willing to give them feedback. So folks from Google, if you're listening, I'm very willing to give you feedback.

30:20 Also, there's entries to the category. Instinct is a big entry to the category and HARK from Hark Pro, whatever that's called, and a bunch of others. a lot. Obviously, we have the the standalone selfass assistance Hermes and Open Claw, which is now what Jensen pushes to people that people can build upon and pi durable, which makes easy for you to build one. So now every every company has a personal assistant and one of the reasons is those they are very sticky, very sticky. And Mr. You want to talk about your experience with with Muse and then moving away from Muse and how you feel about that?

31:01 Yeah, it look it's not that good. It's kind of dumb sometimes, but it just goes. It is so energetic. I give it tasks that involve, I don't know, categorizing videos, going into like a a simulation and stuff and it just does it. whereas DOT just feels like that government employee that tries to avoid all responsibility while still being right and handing off things and just giving you like threeline updates.

31:34 It's just so unenticate. Yeah. It's it's the most useful thing. Meanwhile, dot just struggles and stops the job because it couldn't upload a 20 megabyte file through the interface. So, it just stopped working. Whereas, Muse is just like opening entire VMs, backing up 80 gigs of data is it's just going completely completely nuts. Ser most most useful agent they have they have deployed to the masses I think.

32:07 >> Yeah, the the agentic part is very interesting there. like you want your agent to be able to go and do stuff. Obviously, you want safety, right? So, like we're talking in in Black October essentially, one one of the one of the worst cyber security months in in the history of technology. So, you don't want your agent taking too many things and going like willy-nilly with your data, but definitely you also want it to be agentic and not stop and wait for you. Otherwise, it's wasting your time.

32:35 cloud duet specifically they're talking about they will highlight the most important things but we'll decide what other what other things are important so as far as I've seen back in my you know Hermes days when I could use my subscription for cloud with with Hermes with openclaw oppus has the best decisions towards like what to surface to you and the productivity part is like very interesting I'm really excited to see it's getting a little bit cramped in my AI assistant zoo where there's no instinct and there's no dots doesn't send me that many notifications and here's the biggest part and LDJ would love to chat with you about this is that all these companies now realize that this is the real stickiness it's way more stick so if you're cl if you're cloud coding all day you can just switch to a different harness with Opus behind it it's not going to be the same but it's going to be pretty much the same if your assistant has building workflows with you and knows everything and has the password stored in a secure way etc.

33:39 switching to a new assistant is significantly more of an effort to you and so it's very sticky. Not only that for regular folks and I think that this is the biggest part I would love you for you to chime in for regular non- techy folks who don't clock code and get to the limit very very fast. This is a significant boost in the number of tokens they will use because these assistants run 24/7 and do things proactively. That's on that. Did did you already experience some of these like personal agents? Which ones? And what do you think about duet specifically?

34:12 >> Yeah, so I've been using dot lately and I I think there's a a lot more room there where it could get a lot better and overall just I think this this goes for really all the bots overall. I think that as the models get more capable, get better trained to interact and and kind of predict what the user would want in the future, I think they'll get a lot better over the the next couple years.

34:35 But at the moment, it still has been definitely like a net positive usefulness for me. And it's been just suggesting like, oh, hey, here's that here's that specific model that you're waiting for artificial analysis scores on, like here's and here's the cost efficiency curve that you're curious about on that that's finally published and and and just the things like that which are also relevant to our information we cover on the show. >> Oh, 100%. All of these agents now on Wednesday report to me and say, "Hey, here's some stuff." Specifically, Grock Bots is really good at that specifically because like a lot of the information that we talk about on the show exists on the timelines. Last week, I asked my Grock bots to just scroll the timeline for me and tell me like I don't have time to be on Twitter. N you can do that as well. I don't have time to be on Twitter. Just scroll the timeline for me and tell me like what's important. And and it did. It was very interesting. I was like, can I just like stop using Twitter? But no, not really. Right. So back to cloud duet. I want to show you guys a cool thing that it just told me.

35:36 I asked how do you tell what other assistants are doing and it says your cloud code sessions can send task to me directly. This is a significant difference from dots from openi where dot sessions are not aware of the dot itself. I I've tried to ask it. Claude in your other channels can pass notes to me and I can read those channels and if another assistant writes to an app you're connected I can read work there.

35:59 So, it already knows that I have G, GitHub, and linear and email as well. And I'm very excited about this. Like, literally got this this morning. And right now, still the regular cloud is is helping produce the show. But I'm really excited about the fact that we now have an entry from Entropic because Entropic is really known at taste. And we've been chatting about different offerings from different products and launches. is really known at taste and taste is something I expect my personal assistant to do. Not only design taste, but the stuff that I can already say about do it, they're very very good. They have a standalone interface. This is very important. I don't know how openi messed this up. Openi's dots is sitting inside the openi stuff and it's like over there on the top. There's no standalone. Muse, grabbots, whatever, they all have standalone with integrations. This is very important. So, not only does glad have standalone, they have a standalone app as well. Also very important. I want to open the cloud apps separately. Open the eye shoved them all together and it's really I think it hurt them because I I want that separation between my personal assistant and between the work that I do. Maybe my personal assistant is for my personal stuff and open's work is for work stuff, right? So that is already a big big thing for for for cloud besides the intelligence, right?

37:15 And if fable 5.5 comes which was rumored to come this week and my assistance is not that that that better. That's crazy. But also folks, Oppus 5.5. Oppus 5.5. This is exactly what I want for my assistants, right N? I see you shaking your head. This is like this is the level of intelligence that I want from my personal assistant, Oppus 5.5. And like the only way to get it right now is beg your Grogbots for OPUS 5.5 because apparently last week said the Grogbots for the difficult task, they will route you to Opus 5.5 or get in claw. So I'm very excited. if it's not clear. Very very excited. Let's go on topic. Thank you for this. I've been waiting. Last comments before we move on to OpenAI stuff.

38:00 >> Just just give Muse a sandbox with tail scale and Opus on it and just give it an MCP to talk to Opus for everything it needs. It works amazingly well. That's how I use it. >> Yeah. But see N here. >> It's not the same. It's not the same. I agree. Like like when a notification comes, I want that intelligence and that taste to like judge it and not like have to decide, oh, I need to go to MC. Yeah, I agree. Muse wonder is great. Muse wonder. I upgraded my Muse for Wonder and it it's performed significantly better. Muse does a lot of stuff for me, including including calling. Okay, one use case for personal assistants, no matter which ones, but Muse is currently the best at this. Muse called, let me see. My muse called my kids school to see when I need to come in tomorrow for a parent teacher conference. I didn't know about this. It wasn't on my calendar. They didn't send an email about this. Muse called the school and they patiently walked through which kid when and then it put it on my calendar.

38:57 Just this thing that was done before I asked. Well, no, I actually had to improve the call, but it was beautiful. and I think that like a lot of a lot of the how should I say a lot of the use cases we don't need like more intelligence for but a lot of the decision- making and the the taste making in assistance we do need something like OPPO so I'm very very excited what yes called school which was great all right folks let's move on to our next segment for openai very very excited to talk to you guys about this from entropic to openai they shipped the most boring features this week But the best ones 12hour prom caching in the API writing to the cache costs two and a half times the input price versus the regular one 1 25 for the default 30-minute cache. Basically, OpenAI is changing the way caching happens and they're pitching it for always on assistant like dots which reuse the same instructions and tools all day hours apart. TBO from the Codex team says the Codex and work the cache for logged in users now never expires.

40:03 This is the big piece. You can come back to a conversation from last week and you still hit the cash. There's no penalty. This is a big thing that's known as a no no in the world of cloud code for example that you have this longass red and you come back to it and you pay all the tokens from scratch. I think that this is like huge huge news. thoughts on the caching updates folks? things will be snappier and we like that >> but also cheaper and also this this is the way they fit the agentic world right like this is why like probably open eyes are building this because you come back to your dot after 5 hours and all of the instructions and all of the system prompts and all of the tools whatever they're already cached and I got to wonder if they found some breakthrough to do it always on cash is crazy like obviously obviously they need to do something where to store it and storage now is not cheap.

41:03 >> Yeah, some crazy DevOps their team had to has to pull continue pulling for that for years in a row. Yeah. >> Yep. >> Yeah. Hey, Astra, design a new SSD. >> Maybe maybe Astra designed a new SSD. I am very disappointed in Astra. I've been talking about this. I'm not the only one. Some folks like Swix and some folks like T are saying that Astra is the most disappointing release from OpenAI. I just it just stops all the time. And so when they launched Dots and I was like, "Yay, Dots." And I tried it out and it was like powered by Astra.

41:34 I was like, I'd rather this be powered by by GPT 5.6 Soul for example. However, there's Astro in this Astra Pro and OpenA also has an update about GPD 6. Astra Pro in Chad runs multiple agents on different parts of the task in parallel and you can follow each of the tasks. Astro Pro is ridiculous. This is like a This is not a just a huge model. This is a huge model with many big models in the trench code together and they run a lot of agents right now to for your task. I think that's super cool. And OpenAI, I don't know if you guys saw this, OpenAI bought orchestra Bio, a drug development coordination platform, and now is folding it into Open Rosalind, which is their life sciences stack. and they're offering Rosalind as a free for university labs as well. So shout out to OpenAI for some of this work. They also have Alexander Madri coming back to OpenI to work on safety and security and they are a big part of this week's cyber security stuff with Daybreak.

42:41 Daybreak is open the eyes responsible access to models that can do vulnerability analysis and many companies are getting access to Daybreak after the very famous hugenface not being able to use any open AI models to check on the agents forms. Righty Argon let's talk about Argon. It's been a week. Argon is Gemini 4. Folks from Gemini said they have seen 25 trillion tokens in 72 hours since Gemini for Argon was released. Logan a friend of the show says Argon processed about 25 trillion tokens in the first three days.

43:21 96 million tokens every second and Google's open openly computed. Sundar said on big technology podcast that Google is compute constrained which is just insane. And my favorite Argon story from this week, which Claude suggests that this is my favorite. It's not really Alex's favorite. It's Clott's favorite. Let's call things like they are. Four Argon agents beat Minecraft from scratch in 27 hours. No human help, no pre-scouted seeds, and they wrote 2,700 of their own program. Also died 65 times on easy mode. Argon is now in anti-gravity. And Argon was used to help find PHP2 shell from whiz and daybreak like we mentioned was used as well. So this is Argon a week after it came out. LJ or N have you used the Gemini offering at all.

44:16 The one thing we're going to wait for Wolfam to chime in here but Wolf is our kind of like main residential Google and Gemini guy because his whole house runs on on Android. He does not use argon because it rejected his prompt which is kind of crazy. >> It looked pretty good for 3D. I just just got kind of lost. Are the API prices good on it or Oh, $2 input, $10 output. >> Yeah. >> Through November 30. Yeah, it's not it's not cheap.

44:47 >> But it's solid prices. It's not >> It's not that bad actually. >> Yeah, it's not bad. It's solid prices. It's not Astra or or Fable prices. And open source has been very big this week, folks. So, let's go to open source because I think it's exciting. The whale has resurfaced and there's a bunch of open source stuff that we need to talk about. Open source AI. Let's get it started.

45:21 All righty. Let's switch to opensource and I think the start the start is deepseek v41 flash 1017 the whale has resurfaced with a new checkpoint they've been doing this this is the third deepseek checkpoint if I'm not mistaken right like this is backtoback checkpoints on the deepseick v41 flash which is now still MIT licensed which is incredible and deepsec is now showing up in a big big way terminal bench or goes from 31 to 42.

45:54 It's a huge jump on on DeepSync. And outputs get shorter. It's both better and cheaper to run. Deep says it come from reinforcement learning where the model critique it own work to end to end projects which sounds like a little bit like smells a little bit like RSI. On deep they claim 77% ahead of GPD 5.6 soul at 73 and Opus 5 at 74 not 5.5 just OP 5 that they they deep also opened their search and fetch API to everyone 75 cents per thousand calls and free through cloud code and codeex until October 31st very interesting as an announcement thoughts on DeepSync folks any LJ I would love to hear from you or your like you know your timeline is deepse getting excitement there is this new checkpoint better than the previous checkpoints.

46:44 >> Yeah, I mean I would definitely say it seems like this checkpoint is certainly better than the previous one. I I really feel like they just should have called it V4.1 or 4.2 or or whatever next increment instead of this, you know, they're doing the whole date scheme that OpenAI used to do with the GPT for Turbo and 40 models. >> Yeah. >> Besides that naming scheme right though, yeah, it's it's impressive. They're just continuing on the pace, but it seems like I wouldn't really say this is threatening, you know, the Fables and Astros and certainly not like, you know, Fable 5.2 or Astro 6.1 that would be bound to come any day now.

47:26 >> Yeah. But remember this is an MIT license model. It is a big model, right? It's not a tiny one. You can't run it locally. So before we talk about the next models that you can run locally, this is not something that you can run locally. But so so we see some of the evals here that are interesting. Deep I think is the highest one. The that's interesting. Agents last exam also 35.4 and AEX bench at 72. So shout out to DeepSick for MIT license as always.

47:56 Yeah, we have we have to applaud them as well. Let's see here. The Pareto Frontier looks insane on this model. The prayer for tier 4 cost versus execution is just absolutely mogging. Look at this. This is like just deepse on this top right quarter which is what is this? This is sweet. Yeah, on deep suite you can see how the models on kind of across the map for pricing. So deep on pricing is just just absolut absolutely crazy. But let's see what else.

48:30 They also launched Deep Seek Search and Artificial Analysis posted they have a search index for search APIs. I think XA is the the the top one on there. Yeah, XA. Shout out to XA for also releasing some stuff >> without running on a local model. You could actually run it on the 512 GB Mac Studio if anybody still has that before they stop selling them. >> Yeah, I think those are in hard in in hard supply. Y any thoughts on on the latest dipstick? Welcome to the show.

48:58 >> The they broke the internet. >> Deepick announces deep infer which is the open source infant stack for DeepSick inference engine that serves the V41 flash in production. It's Apache 2 license. It runs on Nvidia Blackwells and Huawei's Ascend 950 and it is their own speculative decoding there and batch inference as well. Tin Demmer's the Qura guy said it's a very very big deal. He says deepsec inference system is on par with openi stack and it's now Apache 2 licensed. That's dimmer's the kuro guy the guy who who invented the the quantization method and it says that it's better what entropic is running.

49:40 Again I will repeat this slowly for the folks in the back. Tim Demer is one of the top folks at inference in the world says what DeepSync released with deep infer in Apache 2 license fully is on par with open AI's inference engine and production and is better than what entropic is running at least for the user size. It doesn't ship KV cache system or prefill decode disagregation yet so you have to plug in yourself. Semi analysis measured VLM within 6% of its decode throughput. So open source isn't that far behind from from that as well. Like yum. Okay. Now broke the internet.

50:20 >> That's the secret sauce. That that's the thing. look, I don't know if everybody got the how huge that is. Look, inference providers, that's their bread and butter. The better you do this, the more you can serve more users with the same equipment, meaning that you just get to keep the rest of the margin for yourself. Like you make this better, you you literally print money and it we all anticipated that you know the the big labs probably has some some tricks. They probably work on this.

50:58 That's their a profit line at the end. So obviously, but we never got to see just how insanely efficient these things can be. And we also the prices of deepseek models inference on DeepSeek versus other providers. And Deepseek themselves were competitive to say the least. They were phenomenal, phenomenally cheap. So we all we all we all guess that probably they have you know they have a very good stack for >> ju just to show the pricing again this is this is just on the flash model they're just like mogging the pareto frontier on cost and they have been always doing this and they only can do this if they have very very very good inference stack and now this inference stack is open sourced >> it's very >> even more look even more deepsee are known to release models and we all thought okay they serve their models better than everybody else they get to charge for it through the API so the model gets open source but now you also have the inference stack >> it's just going to be a bomb it's just going to be a bomb everywhere seriously >> I'm super excited about prices everywhere going cheaper if we can run stuff on this I know our folks already are looking at the differences between VLM and and deep infer etc and that's very very interesting also obviously deep has been releasing other stuff like their attention mechanism DSAs were released earlier this year also blew everything else the the algorithms that they released for other stuff this now includes speculative decoding is very very big for open source I'm very very excited speaking of open source let's also talk about miniax right we have to talk about miniax I don't know Peter if you're already excited to chime in there was a model that was trained under the name was that pony something space bunny or was the model was trained under space bunny I think it was open in open code and it's now minax m3.1 flash with 15 cents per 1 million tokens and the weights are coming on October 23rd so tomorrow Peter any any thoughts on miniax how that landed on the arena at all and also welcome to the show good to see you >> yeah actually I don't have too much to say unfortunately about minimax just but just to add on deepse I think first of Well, yeah, that's a that's an incredible thing that they're releasing that much. But >> I think it's interesting that I was looking at the Parto Frontier on on Arena and it did land on it, >> but it's not like it crushed it. So, I think it's kind of interesting. Yeah, maybe if they actually just had a bit more compute, maybe they would have reduced the prices even more because I guess there's no point them reducing prices by 50% if if they can't serve it.

53:54 I guess maybe that's why the open sourcing as well. But yeah, it's kind of yeah that that's interesting. I kind of considering all of the all of the I know messaging around it and all of the reception. You'd think that oh they're going to you crush it completely. I would say in my mind they they kind of got to the category of models that are very competent and they're they're pushing up from the lower end. M >> they're not they're not like super cheap. So I I want to dig into this a bit more. Why? I don't really understand why. Maybe it's just using a lot of tokens or something. Not not totally sure. but yeah, I think it's another thing is that for for things like for Miniax, for Kimmy, for Mimo and others like I think it's really difficult, right? Because how do you compete with Gypsee? I think like Opus has an answer for it and like GPG, maybe Astra and others have an answer for it, but like what about the others? I think it's it's hard for them, >> but also there's a lot of people who use AI and I I agree with you, but it's just somehow they are I think the highlight for me between those different models, not only DeepSc is the best one, they're also like fully fully Apache to MIT license open source like fully and they don't require any stuff.

55:13 You can just run them. And also this is the smallest DeepSc deepseek V4 the big one is going to land at some point. That's going to absolutely mog the frontier especially with the pricing and notable as well Nvidia's kind of like ownership of everything. The deep infer stack runs both on Nvidia stack and on Huawei chips which is also very very notable because that's where like most of the Chinese folks because the export controls can run. All right, so skipping miniax is pretty good. 74 in deep and weights are due tomorrow so we're going to mention them. One more open model this week is Flint, right? So, we always talk about kind of the the the the kind of the Chinese models and then also open source models. We're going to chat with Prince Kuma about Flint. In huge open source news, but open source/ aentic news, Jensen Hang closed his GTC keynote by showing how he uses the Hermes agent to build something to train robots. W from you were there and also this is a ping from your personal assistant after watching my news. C can you tell us about that a little bit? So shout out to news research and the Hermes team for that.

56:24 Incredible. Let's take Jensen. >> Let's go man open claw very strong and then pull him towards Hermes which is like like huge although they still wearing claws. There's still the images of Jensen with like the red claws like all over GTC from each other this before. tell us how it felt to see Hermes like on a big stage and what did they actually show and say? >> Yeah, just just to clarify, so they are still calling these kinds of agents claws apparently, but he always mentions Hermes in the same spot as open claw and even some others he mentioned. And the thing is now that at the end of the keynote he was demonstrating how to use an agent and local AI and cloud AI together to create or to train the little micro ducks that little robots that hugging face or pollen robotics created which now belong to Nvidia after all. And yeah he was using Hermes agent locally on his system and apparently there are skills for this and it was training the robot on the local system but also using online inference to yeah to basically get the horsepower that he didn't have but the agent was doing it all. It's it's like we are using our Wolfred and Amy's. You know, you have an idea, the agent does everything that is required and uses all the resources that are necessary and then in the end you have a walking micro duck and he even demonstrated it on stage with four little ducklings walking around and doing some movements that have been trained that way.

57:51 >> It's very interesting how these things work. Micro duck is Poland Robotics which was purched by Hug and Face. Now huggin face is part of Nvidia and Jens is standing on stage saying, "Hey, go to Poland Robotics, buy these micro ducks." It's really, really, really funny. >> He said they are sold out already, but buy one anyway. And I got mine in August already pre-ordered. So I hope I could get it at the end of the year because it's so interesting how robotics is now taking off. And I've felt that at the GTC as well that robotics is Yeah, it's getting ever more important. the world models. Other labs like Black Forest lab, they are also building these world models and cooperating with Nvidia to get robots going farther away. So I think we are seeing an inflection point with robotics soon as well.

58:34 >> All right folks, this is a good reminder to tell you that the show is sponsored by none other than Coree and for the last three and a half years we had this week's buzz because you know weights and bias is part of Cororeweave and Buzz this was our joke. Whatever. It's time to It's time to announce a new segment on the show. This replaces this week's buzz and this is called From the Forge.

59:12 >> Cool new intro. >> Yeah, cool new intro. Wolf from you're at GTC Berlin representing us at GTC Berlin from Corewave and we last week we talked about reflections beam. Reflections beam shipped their open weights and we're supporting them immediately on core wave and corweave also said that reflection used weights and biases inside forge while training reflections beam. I need to get used to this. We told you about reflection beam two weeks ago on the 16th. The full weights landed 501 billion parameters Apache to license and we're running this on Cororeweave serverless inference from day zero. They used weights and biases inside Cory Forge during Beam's training and we we knew Beam back when it was just the last curve said our social channels which is I think is really absolutely crazy. Reflection says the beam served a trillion tokens on its first day which is a huge huge news for western open source and it's also landed on arena and artificial analysis.

60:09 There's a bunch of stuff as well. Also on forge Nvidia and.com used RL rollouts which is our new offering for to postrain neatron 3.5 lightning browse comp bumped up from 37 to 45% on that roll out using those RL rollouts with 30% fewer to call. So if you are building any type of AI core reforge is the place for you to go and robotics folks physical AI launchables wolam you were there while while we announced this one command and you're running isac sim templates wolam you also said that GTC Berlin we talked like they announced a bunch of stuff with ruben stuff could you tell us about the nvl8 what's that and like how how that's going to plug in >> yeah in addition to what you already mentioned there were two announcement core we've made so the first one is about nvidia htx ruben nv8 8 which is coming to cor this quarter. It's eight Ruben GPUs, two terabytes of HPM4 and it's built for large KV cache and concurrent long context agents and it's very efficient. So Nvidia reported 10 times more tokens per megawatt than the HGX B200 tested with Kimik2 thinking. So per month is an interesting one that is >> we can run more agents for more users and hopefully that will also affect the price of course >> and the second announcement is very important for Europeans especially I mean it's it is important for everyone and this is not a closet what I see but I'm in a hotel room and the lighting is ter something >> yeah it's okay it's okay keep going >> security zero trust AI framework that moves security from promises like we are not doing this we are not trading into customer controlled architecture. So your keys, explicit support access when needed, hardware attestation, confidential execution and no customer data being used to train provider models because everything is encrypted. It's end to end and this is coming and this is something like in Europe a lot of people they can't use or are not allowed to use US services. They in Germany many companies require German hosting. But if this makes it impossible for a US company to look into what is going on here, that makes it more possible to use such services and for everybody security and privacy being enhanced is a good thing.

62:26 >> Yep. And we offer a bunch of the services on forge. Check out forge.cor.com. All right, folks. This was from the forge. This is the first from the forge. I'm excited. There's a lot of news. There's a lot of news to to continue. Obviously GTC Berlin Wolf from you were there. Let's move on because we have tons of stuff to talk about. There's a lot of stuff in multimodal and video. By the way, the the transition that you guys saw from the forge, this is the the the Mini Max M3 Max, which is we've talked about a couple of times on the show, and the music is from 11 Labs.

62:59 Let's talk about AI art and diffusion and video models, etc. Welcome back, Peter. Folks, we have quite a few I don't know how to call this category. AI art video generative diffusion AI. Now, there's now diffusion models in LM. So, that's very interesting, but let's call this taste category or whatever. Clink 4 generates up to 30 seconds of video in a single pass up to 4K. We're now at the era where we're generating 30 seconds in 4K with stereo audio and better lip sync. I think it's a good thing to go and try and and find a good video with Cling 4 videos, voices, and images as references and and it's like three times as cheap as C dance. There's a lot of capabilities in there. 4K specifically is the most exciting one.

63:55 So, let's take a look at cling 4 and then maybe play. So video duration from 3 to 30 seconds 1080p and 4K dynamic range 4K aspect ratio 21 by9 is a new one. They they have new languages from notice the German and and other languages as well. Dialects and accents is pretty cool. let's take a look at there's so much to take a look at. Let's take a look at the start frames for example. >> You never listen to a WORD I SAY BECAUSE I'VE HAD ENOUGH.

64:29 FOR ME, THIS was getting stuck, but like here's an example of the input frame of the first frame. And this was the video that they showed. Let's take a look at another one. This, I think, is a full >> Hi everyone. I'm so happy you're here. >> Great. >> Welcome to my channel. >> The lip syncing is absolutely absolutely perfect. Let's take a look at this one. This is the start frame. And let's look at this guy.

64:52 The internet is a little stucky for me. So, let me try again. >> Trick check. Light slow. I'm stepping clean. No cut, one take. Keep a sh rock tight. Words run fast, but the flow steam bass from the quiet. >> This is like the lip sync is great, but this was this was like very very clearly AI generated. Does this lead the video category? I'm not sure because there's a lot of competitors there. Flux 3 Max from Black Forest Labs and FAL, which we've seen the foul folks optimized the Miniax M3 with with the Max version that runs super fast. Now Black Forest Labs and Foul released Lux 3 Max. headline is speed 10-second clip in about 12 seconds. So, not quite real time, but very very close. The folks at FAL, shout out, huge shout out to them. we can take a look at one of those as well.

65:37 Peter, how how how is the arena in video generation these days? What's what's the highlight there? Who who are these models breaking through the top or we still like in the Omni category from Google? >> Yeah, there's definitely a competition from Mini Max. yeah, H3. I don't think we don't have cling yet, but you know what? They actually I'm just looking at the leaderboard. We haven't had one for a while, right? I think so. It' be nice to see where they land.

66:06 >> Yeah, Clink three was like 19th now. So, that's really they get you I I kind of forgot that. Yeah, actually, we haven't had a a response from from them in a while. So, yeah, if they go towards the top, that would basically add another competitor. Yeah, we probably had like three or four. Now it's going to be four or five >> Yeah. >> top tier competitors. So that that's really cool to see. >> And and I think for folks who haven't like tracked this category for a while because they said, "Okay, you know, this is all like AI slop etc. The video models improving in a crazy rate. When we talked to you about Cance before, Cance was the only one that get take up to like five references and do text to video and image to video. All of these now can do video to video and references for faces and characters and the character consistency is insane. Is insane. So now we're looking at realism and and speed basically because seedance took like I don't know two two and a half three minutes to generate 30 seconds and now you know this one is faster. Wolf go ahead and then we'll show some examples of the the fell ones >> and H3 is also taking off in the local AI scene because I'm running this on my 3090 and it works very well. Miniax H3. Yeah, you can now run from Yeah, it's it's almost frontier. I would call it frontier. I mean, you have Omni and reference and all of this. You can do this on your own GPU and the community is fine-tuning and optimizing it. And so, it's really fast now. It is amazing what is happening here.

67:33 >> All right, folks. I think it's time for us to to move Oh, we're five minutes late. I'm so sorry. Okay, folks. It's time for us to to get to our next guest on the show. Let me just run through this. So, Clink 4, Flux 3 Max, and Craya 3 also launched a model. It's number four on Arena Multi-image Edit leaderboard. So, shout out to the folks at Craya. And yeah, AudiML we don't have time to go to. All right, folks. Time for us to move on because this week there's a lot of great open-source news from Western Labs. I I'm very very happy to invite Timshi from from Recursive AI and Prince Kuma, our friend. second time I think or third time from the pod from like the local a IMX community. team, how about this is your first time on the pod and I know this this came together super quickly. So really appreciate you coming on cuz you guys just launched Flint. Would love to just have you introduce recursive AI and then and then yourself and then we'll we'll get to Prince and then we're going to talk about this most most most exciting release from you guys.

68:31 >> Yeah, super excited to be here and thanks for having me. recursive is building yeah self self-improving AI for to build super intelligence and we want to automate knowledge discovery so the idea the process of ideation implementation validation the whole scientific method and we're applying to the science of AI first so flint was like the first demonstration of this technology or we call it eureka machine where you can basically apply it to the discovery of the training of the mod training of so small language model and then we basically opt optimized model for Mac. So it will run really fast on Mac and it has like a lot of capabilities that can sort of do for your daily workflows. yeah so my background is I started my career actually with working with Richard and then NLP labs in at Stanford and then OpenAI and then started a company called Questa is basically one of the early attempts to apply AI to enterprise and then yeah super excited to be here and also yeah to talk about a release >> that is awesome absolutely thank you for coming we're going to get to Flint in a second also like a bunch of questions about the Eureka machine whatever you guys call the Eureka machine RSI etc.

69:41 Prince Kuma second time on the show of some folks. No use them. Alex King, you want to briefly introduce yourself so we can get to the the meat and potatoes of the most important thing in in open in western open towards AI this week. >> Absolutely. Thank you for having me, Alex. Hi, Tim. once again, Prince Kenuma here. I am the creator of MLXVM. Over the past 3 years, I've been building infrastructure to run AI on Apple devices efficiently. We just crossed 7.5 million downloads in our engine and happy to have brought day zero support for Apple silicon on MLX for Flint.

70:17 Folks, I'm looking at the Flint website which I think I should like just the show while team we have a questions for you. Let's pull this up here. This is quite the model and a very small one at that. So, could you introduce us to Flint before we talk about the Eureka machine sizes wise? What are you guys aiming for and why is it so goddamn fast >> or how? >> We think of a fling as a first demonstration of our machine sort of our system to automate scientific bit discovery and the flint has our goal is to optimize it for like your daily coding use cases. So developers can run flint on a Mac M4 and M5 and get really fast speed as you can see and be able to sort of drive a lot of their daily coding use cases.

71:03 Yeah, Prince, you saw this as well. Like there's like over 185 tokens per second on M4 Pro and >> 100 189 as of yesterday, but this is more of like a branch that I'm working on. >> Yeah. And is tell us about So okay, so I do want to like talk about sizes a little bit, right? It's a 40 billion parameter model, but like a very very sparse MOE, right? Tim, could you talk to us about like how Okay, so the Eureka machine once we get to did it make the decisions like how do you guys arrive at that like very very very sparse kind of architecture two billion parameter active is that what like enables the speed and how is it compared to like other models in open source are running at the sameish size.

71:48 >> Yeah, we actually first look at the sort of hardware substrate, right? So we work with closely with partners like you know Nvidia and AMD and we look at you know for a model to be running locally what are the basically the shape of the model to be efficient right so for me pro we basically asked the model to search what is like the most efficient architecture and it come up with something that's like actually way sper than sponsor than like comparative models. so we ended up with a flamed architecture of 40 billion total parameter with about 2.5 billion active per token and it's basically with unique machine think projected to offer the best combination of speed and coping performance sort of among like all the configurations you can have on the up on on the on the Mac. yeah >> I find your last sentence incredible enough so that I will ask you to repeat it. This is what the Eureka machine thinks that is the most optimal.

72:49 >> Let's talk about that. >> Let's talk about that. This is incredible. You guys are literally the name is recursive. Recursive was that rebranded to SI or not yet? But recursive super I think is the concept >> and then now you're referring to the fact that like the thing that built the thing that's how you guys arrived at the architecture. How involved were you guys at the building of this the decision- making versus the the Eureka machine decision? And what is the Eureka machine? I think it's time for us to actually talk about what is that because I think it's maybe more important than actual model.

73:22 >> It's funny that you mentioned it. We actually got the domain recursive. SI and it's also stands for recursive super intelligence. Recursive self implemented is also recursive super intelligence. and we are thinking about like the UK machine as a machine to automate scientific discovery. So it will basically have think of as like a community of scientists and agents that will come up with ideas and validate ideas and implement those ideas. So it runs in a loop keep you know trying different things that will potentially improve the your objective. So you define a certain objective for this particular example is going to be speed and coding performance. is a good combination of what we think is going to be good for developer on the machine and then the Eureka machine basically search for different combinations of different architecture and obviously like you know it's not inventing like completely new like say transform architecture right obviously there's like a lot of unknown unknowns in scientific discovery but within the known boundaries of what you can search for I think it's sort of find some interesting discoveries >> that is a very interesting How how recursive is it as well? Like the Euro machine like update the Euro machine inference or code or directions? Can you talk about like the recursive part as well? Cuz when you say Euro machine came up with this model, this model is a small model. How recursive is the recursive part?

74:52 >> Yeah. So the Euro machine is a larger system of model with combination of different models and harnesses. we're sort of moving towards eventually fling will build the next version of Flint. currently it's basically a larger set of systems models looking at the small model and say what are things to improve. So for example I think Thomas Wolf actually posted about this as well. It's like an A researcher looking at the LM head would say you should always keep it in high precision and never decide to quantize it to four bits. But our uric machine actually tried that idea and validated it's actually a good idea for for for Mac and local coding use cases. So some some of some things like that that can be a human scientist will overlook but as Eureka machine like automated AI scientist will just know combining all the existing knowledge propose a hypothesis that might eventually be true and those are the discoveries that we think and then our machine will be able to make down the road for many different domains. I think it'll be really valuable for for for the science itself.

75:57 >> I I think that's kind of incredible especially like some of the decisions that you know maybe humans didn't go through. let let me go to Prince for a bit for the model itself. Prince you've been testing out like maybe day zero support as well. Tell us is this a good model for a daily driver for coding right now? >> Yes. So actually I rank it around one of the top 30. I have a list that I keep at all times because people ask me all the time. right now it's it's Flint, it's Coher, and it is Quen 27B, the the most famous females one. Those are really good models. And actually, Flint was surprising to me because a lot of models you have to like ask a question and then ask it again and continuously do that.

76:37 Flint tool calling great as well as like building artifacts like hey build me a game take documents from the internet store it somewhere in my computer it can do all of that and right now it's my main daily driver for for a couple weeks on native >> wow that's quite incredible and Tim kind of like I'm I'm always assuming when I bring folks from different labs for training models like I'm assuming there's something else that that they still don't like can talk to us thought about like the new iterations but given the name itself it's like very very obvious where you guys are going to to to improve these models. I guess my question to you guys is like why start from local and why bet on open.

77:23 >> I think the answer to open is very clear. I think that you know historically models being like closed u source at least in the western hemisphere and I think that a lot of Chinese open source models open way models actually gained a lot of adoption in large enterprises and we see the curve of like companies shifting you know maybe they start with large closed models and they realize that that the cost is going to be very high and then they start shifting towards open way models and we actually did that before at cross like we with like OGBT models And we shifted to fine-tune our small we actually have like tens of thou tens of like different open way models small models running production running different things like summarization or you know some auto reaging tickets and all these like specialized tasks and it's obvious like enterprises would eventually own their intelligence in a sense like they would prefer to have control over their intelligence instead of like relying on a external partner who you know may may may still be very costly at the time. So we think that this could be the future where people some people would choose open way models, some people would choose like closed source models but at the end of the day it's a customer's choice and I see the usage of a model is going to be really high in the next year or so and to answer your question about small I think that we're just getting started this our first model release and yeah we'll see how what the recommendation can do in the future. Well, Tim, we I don't know how familiar you are with Thursday, but like we've been doing a segment on open source AI and modeling for the past three and a half years and every time that we have somebody from the labs come in and then we see Apache 2, we have we have applause because I think it's very very important for the ecosystem. So, thank you guys for releasing this in a very open licensing as well. I also want to mention the speed with which you train this. I think this is one of the highlights just before we drop off. This is one of the highlights in the blog post for me.

79:19 Could you talk about how fast this happened and how just how >> Yeah, actually a lot of kudos to the technical team and I think there's a lot of effort went into this but I'm also very impressed by how fast we're able to put together the whole model factory and be able to like train the first model and in about less than four months and I think a lot of us actually >> four months to release a model from scratch >> we actually didn't have any pre-training post training effort and until like one month ago and we were debating what are the use cases product use cases of like you know machine we thought we should be using it to train models and I think that we just decided to put together teams but also rethinking how you know agents can be used in the whole process and how eurocom machine can be used to accelerate that process so we apply auto research to almost every stage of the pipeline from pre-training to post training to RL to inference and we think that that's going to be a way for a lot of lab to accelerate to catch up to like you know build models in a in a way that wasn't imagined before >> I think before so that is incredible for four months from nothing to no training to a model that runs in one of the fastest to to run on local hardware I think it's incredible maybe for folks who are listening who are like I can't wait to try the flind the model excited about the recursive stuff excited about your commission but like can't wait to try the flend model maybe from both of you any comments about how best to use this like all models come in with their tricks with their with their like specific stuff for you already mentioned something team would love to hear from you what what do you find from from using flint like locally that folks need to know about like how to prompt is there like any specific things about reasoning or quirks like that that are different from different models would love to hear like each one of you give us a little bit of a tip to folks who are listening who would like to use those locally >> oh and also I'll bring in Non the co-host who had been using this all day and wants to to tell you all about this.

81:21 But let's start with with use case as well. >> Yeah, I would say this model is more agentic. >> Yeah, I would say this model is more agentic than than feels like you can run it locally do a lot of things. but remember this is the small model. So, you know, and compared to like Opus and all these larger models, you typically would need to give it more instructions and guard rails and you know, tips on what to do and it's just the nature of the model, but also you can yeah, I'm just curious like what what are the things you guys were able to build with this. So, yeah, feel free to share on social media about things you build and how you use the model.

81:59 So, I I have it doing both stuff for smaller models, just doing testing. But the most fun thing I have with it is I made a big cityize GTA mod simulator. And this was probably the smallest, fastest thing that I could actually run as a character going around to make decisions and stuff. I just want to know what what went into RLing it. So, well, again, it's not perfect, but it's pretty easy to to prompt. It is actually pretty pretty obedient. So, I want what went into RLing this and do you guys plan on turning it into Jev as well?

82:41 Because it's kind of ideal size. >> Yeah, I think that's that's a good question. I think the RL I think put a lot of effort to make sure the model is very agentic. And the J question is interesting. I'll be curious to see if someone could make a demo around like, you know, just model modest by itself to train itself to be a Jeff. I think that would be a cool demo to show. >> Yeah. >> Yep.

83:04 >> The good the good thing is that you can run up to like eight agents in parallel and and have really good performance that you're talking about each agent like 32 tokens per second and averaging like nearly 500 tokens per second on a on a M5 Max. And this could work on a 48 GB even up to like 36 GB. And if you have more memory, great. But you can run a parallel agent with it. I think this is a great place to end our interview on because specifically given the next segment about security and other stuff and given the personal agents stuff many people would like to run agentic models locally completely so that no information leaves their systems and you know that that's one of the big upsides for local agents completely like everything is running on your system team and prince thank you so much for coming and telling us all about flint running >> running this locally we're very very excited about the new lab as as well as like a new leader in in the category and the speed is really really impressive.

84:01 So, so definitely give Flint a try. We'll be monitoring the other stuff. So, team feel free and welcome to come back once we once the Rick machine decides to release a bigger model. I I we'll be definitely down to to host that as well. Thank you so much for joining as we move to our next segment. >> Thank you, Alex. Bye-bye. >> Cheers. Thank you, Alex. Thank you, Tim. Thank you, Prince Kuma. And now, >> I think I'll bring everybody back to the stage for this as we introduce Josh Long. Josh, I don't think you've been in video form on the pod before. We definitely chatted.

84:32 I should have I should have chatted with my claude to rearrange the segments because we just finished a very high note local open source apache like works amazing runs and now we have to talk about the black October the very stuff that happens in the world of cyber security. Josh is a cyber security evangelist. Josh, some folks are calling whatever's happening right now in cyber security is the black October. There's a lot of vulnerabilities and hacks are happening all at the same time. So much so that folks are saying is the internet done which is very scary. Just to name a few of them the bigger ones and the reason why we're talking about them to you in the AI show AI news show because a lot of them are AI related. Some of them are maybe swarm stack some of them were discovered by AI. We'll start with PHP to shell which is a huge vulnerability in the most popular language in the world. the most websites run on I think like the latest stats are talking about 70% of the world's websites are still running on PHP PHP to shell basically gives an ability to take over that machine without any like credentials there's also a an updated CVE in emuja was that emuja right like the certificate update that lets everybody say that they have a valid HTTPS there's just some of the stuff including like them getting a supply chain attack Josh smiling. H how are you still able to smile given that everything is going on right now and folks are freaking out?

85:59 I want I want some chats about that. >> Okay. So, for one thing, it gives me an opportunity to get to talk about fun stuff like this on shows like Thursday Eye. So, that part of it I don't mind. but yeah, I do think that so there tends to be a lot of like Y2K like hype around things like this, right? Everyone starts freaking out and it's like, "Oh my gosh, it's the end of the world." But it's so a lot of the standard kind of disclaimers apply here, right? Talking about the PHP2 shell vulnerabilities. First, there were a few independent groups who discovered these two vulnerabilities.

86:37 We had Califf and Whiz and even OpenAI and they were all kind of like finding these independently and reporting them. which is really interesting because that doesn't seem like that happens very often. The last time that I remember that this being like a big thing where a whole bunch of people independently found the vulnerabilities and reported them was back with I think it was in 2017 toward the end of the year with Spectre and Meltdown and those kind of like came to light all of a sudden in January 2018 and everyone was freaking out about those too. so kind of unusual to have a situation like that where multiple groups are independently discovering the same vulnerabilities and reporting them at the same time. So I said that daybreak the the kind of responsible security program that they allow slowly people into to to be able to extend the defenders window whatever Greg Bachmann called the defenders window right like we have a limited time until all these like other intelligence catches up to a frontier of cyber security we have limited time so daybreak is their system to let people into and with GBT 5.6 six oh sorry 6.1 saw this is like where they discovered the PHP vulnerabilities and disclosed to PHP foundation right and also calif and and whiz and folks they apparently use Gemini Argon for that as well both arrived at the same thing what is that thing Josh could you tell us like what what folks need to be what folks need to know about the the PHP to shell and what they need to do as well >> yeah so the the most important thing is first of all if you are running a PHP server, you need to make sure that it is patched. Like hopefully you patched yesterday because you know these are pretty critical vulnerabilities. as far as the the ratings go a 10 is the highest. See the CBSS score for these I think is 9.2. So it's like just about as critical as it gets as far as vulnerabilities go. so these are very severe flaws. and it's very very important to make sure that you are patched. both the FBI and NSA accounts on Twitter and the prime minister of Japan and multiple other like state big state like companies all came out and said hey patch or go offline >> this is like how big this is like you have to patch or completely take your system offline until you are able to don't keep them online this goes out to everybody who listens to that does run like PHP software whatever WordPress is we will say though many companies and clouds If you are running things on the big clouds, they have protected this and if you have like a web firewall in front of your website like a cloudflare or versel, right? Those they have released they have updated their their web firewalls and have released like protections but you still have to patch like it's still like very important.

89:20 Yeah, Josh. >> Yeah, 100%. Yeah, I was going to mention that too. So, I I I've seen a lot of like again hype and kind of misinformation going around online where people are saying, you know, you might want to just stay off the web for the next couple of days and and I'm like, well, let's think about that. Let's break that down. Okay, so first of all, the big Fortune 500 companies are are going to have patched immediately.

89:43 they already had web application firewalls in place anyway, which would have mitigated those kind of attacks. So there there has been an increase in attempted attacks but but because just about everybody's behind something like Cloudflare you know they're pretty widely protected like even if you haven't patched yet but hopefully just about everybody has patched by now. Again if you are running a PHP server you need to make sure you're on the latest patch.

90:14 there's no excuse for that. even if you do have a web application firewall, you need to make sure you're patched. but but it's not the kind of thing that people need to be like freaking out about, oh my gosh, like can I browse the web? Am I unsafe? Well, think about it this way. So, if like you shouldn't be entering personal information into random websites anyway, right? So, like it's it's really it's really probably not something you need to be extra like height on heightened alert about, right?

90:44 like go about your daily business. Don't worry about it too much, but it's it's more on the side of the people who are running websites, especially if you're not a Fortune 500 company who's got a big security team and all that. you probably are more likely to be impacted by this because you need to patch yourself. >> Yeah. And hopefully folks are not self-hosting those completely and hosting them on servers and cloud providers that that they do kind of like patch because it's in the language itself. So that's one thing that happened. PHP to shell is it's kind of crazy the first class, but there's a few vulnerabilities that were discussed. I I do want to talk about the the the the emudra kind of like signature signing.

91:22 Could you walk us through that one, Josh? Like and this I see you like want to chime in. Feel free to chime in here as well if if your experience is like different because we need to talk about the class of attacks and discoveries getting faster and faster with AIS that are able to do this now and whether or not this is now our daily lives like should we always keep our OS up to date and browser up to date now which is good security practices which you can get to at the end. N you want to chime in here on about something I see.

91:48 >> Yeah. as as someone that built their own router at home on Alpine Linux and sets own SC Linux policies. Josh, what about people that just don't care at all? Like they actually just do not care. Like what does this go? Because I know I could run an LLM on their on their server or turn it into a botnet or something. It could harass people. But what about what about people that they just they still just do not care?

92:15 >> When you say don't care, what do you specifically You mean >> I mean literally do not care. >> No, but like about which part? About the fact that now there's vulnerability >> on websites or their servers. >> yeah. >> So, so you're talking about like server operators who don't care? >> No, no. Just the people that own these these websites and that go to them. Yeah. >> So, so I I guess the thing that people should be aware of if you are running a website and you don't have a patch and you don't feel like you need to worry about this you probably don't understand the severity of situation, right? So, so anytime that there are major vulnerabilities just flapping in the wind, right?

92:53 somebody else could come along and pretty much compromise your site. They could could take your site offline, right? If if you're running an e-commerce site, guess what? You're not going to have sales. But it gets worse than that because now it's reputational damage to you, right? So, it nobody's ever going to trust you again if they know that the last time they went to your site it was compromised, right? And and you know, and there are different things that could happen with that compromise.

93:21 Sometimes it's a very like public obvious thing where somebody defaces the website. other times it might be sort of a more behind the-scenes thing where they're they're in there but they're not making themselves like widely known. So they're kind of like able to steal information that people are providing to the website, right? Potentially including sensitive information might be credit card numbers or whatever else people are typing into the site, right? So, it it's it's a an an important responsibility. If you are choosing to run your own server, you need to make sure to keep it patched.

93:59 That's super important. if if for no other reason than your own, you know, reputation, you need to make sure to keep it patched. >> So, Thursday is focused on a engineers and folks who are building in the public, right? So, many of us are running websites. There was also some vulnerabilities in Node.js which folks need to know about and like patch their Node.js. There was also supply chain attacks that you know hopefully they were patched with lightlm. The other class of a big vulnerability that was discovered was not related to AI at all.

94:25 was like complete negligence from the company called EMURA that they signed HTTPS certificates and instead of Josh please correct me if I'm wrong here but like I'm not a cyber security expert at all but like instead of a 24 or 48 like long character signing key the keys were literally just one of them was just the number four one of them was 56 whatever some faulty thing could be I don't know led to the fact that those security signatures were compromised completely that is a different class of things to worry about. Could you walk us through what a how should I say a compromised GPS signature affects how it affects people in the wild?

95:03 >> Right. and and by the way, this is actually not the first time with Emoja that's not not the first time we've had a a similar situation. So, I actually wrote a blog post way back in like 2011 about a company called Digenotar that had a had a situation where their root certificate authority was was compromised. Basically, what happens in these kind of situations is that anybody who wants to can create their own certificate impersonating any website that they want. now that doesn't mean that anybody out on the the the web could reasonably like imitate Google. This is more of like a a they call it a man-in-the-middle type situation, right?

95:54 So, if for example, like let's say that you went to your local restaurant or cafe or whatever, hopped on their public Wi-Fi publicly available Wi-Fi, right? and and somebody running or who was could insert themselves in the middle, right? So So basically the the person hosting that Wi-Fi or depending on on the situation with, you know, if it's like an open Wi-Fi network, maybe even somebody on the same network could find a way to get in between you and the website that you're trying to access.

96:32 if they had one of these vulnerability vulnerable certificates issued or that appeared to be issued by Emmudra if your browser trusted that Emra root certificate then it would trust that you were actually going to for example google.com or wherever you were trying to go. so there's a lot of things that kind of have to to be in place that that it's that man-in-the-middle aspect of it that makes it like probably not as big of a deal as I think a lot of people worry that it is. I have seen people, you know, suggest that it's again kind of one of these world ending events. It is a problem, no doubt. And so the major browsers, if you're keeping your browser up to date, that's really important because they've already removed that certificate. Apple is the one that's kind of a little bit problematic right now because Apple takes a long time to in between patches, right? So they haven't put out a new version of Safari, at least unless they did like in the last hour or so. Yeah. there's not a new version of Safari out yet.

97:46 and and Apple does take a long time typically to release patches just in general. but if you're running, you know, Google Chrome, Brave, you know, Edge, whatever, they've got patches for all of Firefox for all the all the major browsers have already removed the emoj certificate. So nobody else can pretend to have a a a signed certificate that you know can let you impersonate something like Google >> impersonate and then decode traffic. I think this is the biggest thing like the man in the middle part that you're talking about HTTPS traffic is secured and this is why the signatures those exist if they pretend to be one of those they can say hey I am an legit HTTPS like thing but then decode the traffic that's supposed to secure like credit card data and other stuff. If you're still browsing and not going to HTBS only sites, that's a problem. But if you are now, you update your browser. I think this is the most important part.

98:42 It's very very important to update the browsers. Update the OS systems as fast as possible. and I would go and say as far as don't go to websites that are not big and known to purchase stuff as well and enter your credit cards is always a good advice, but the best advice right now is just don't go to a random janky website. Enter credit card form. By the way, the certificate key, what happened there is during the key signing ceremony, the 384 bit key was just changed to a four, which is so sketchy, but also so elegant. And in my opinion, yeah, it was definitely a human that did that because they just found out that the private key was literally just the number four. Like, it was an LLM. It would have at least done something better. That was that was definitely the humus. It's it's pretty it's pretty funny. But the update I'm like rebooting a whole bunch of stuff right now. I just realized update.

99:36 >> Yeah, there's a lot of there's a lot of non again for folks who maintaining servers, not folks who just browsing the web. There's a lot of work to do now to upgrade Linuxes and PHP versions and node versions and etc. So hug ops to everybody who spent a lot of time lately not going to sleep trying to mitigate the scope of this attacks. Last thing that we do want to talk about this week also and it seems to be picking up is hacks on like major companies. So open code was hacked what a week ago or so and they like publicly talked about this and now railway which is a deployment place where you like deploy your websites and put the websites was also hacked and like the the thing that seems to be happening there and we would love to chat with all of you guys about this is that the extraction of API keys specifically LLM API keys is now what's happening because that's money that's literal money to those hackers right whereas previously you know in the world of let's say DeFi and and crypto, you would hack something because you can like sell those tokens immediately and then get valuable money. When somebody hacks a website, your personal data, you can sell it on the open dark web, but dark web has your personal data at this point already. I'm I'm not sure how much your SSN is worth right now, but your API key to open if it hasn't have a limit on it, that's essentially like a free and direct access to your credit card. And there's a lot of folks who are buying those keys for distillation purposes on the black market. There's like many many folks who cover the API keys on the black market. So that seems to be what's happening right now with railway and and open code as well which I think open code got up as well. And thoughts on this folks will this keep happening like like with agents worms. This is this that defender window that Greg Bachmann talked about coming to a close or are we at the place where like it's going to be great but for a while it's going to be sucky like how how to protect yourselves against it. would love to hear opinions around like that specific thing.

101:35 >> Just use tail scale everywhere and yeah tail scale two factor and just just vibe code your own dependencies and vibe code your own home firewall and just just don't trust any any type of like sales or marketing material for a product. If you can't see the code or your agents can't see it, you you literally just cannot trust it. I I don't I don't know what else to to tell people.

102:07 I guess my my question something I don't I don't really have a good feel for is people in security like everyone are they taking it like properly seriously because it's just weird how these things keep happening and like the open code team they didn't have access to the tob break right it just feels like such a trivial thing like surely you can just do that so I don't know like do do you feel like they're taking it seriously or no not quite penny hasn't quite dropped It's it's interesting because I I think so and you mentioned vibe coding, right? Obviously, there's a lot of people who are who are just trying to do their own thing and and and and I think that's where a lot of times these API keys will leak is because, you know, they're they're they're unintentionally putting their API key in a public place, right? And then someone can grab it, right? That that is is one of the problems that we're seeing. that's happened multiple times where somebody's just unintentionally exposed their own private API key.

103:13 That's that's kind of problematic. And and I think that you know if again if you've never run your own service publicly and now you're trying to do that for the first time and you don't really understand kind of how these things work because most people probably really don't like the average person really doesn't get that. they don't understand what the potential problems are with exposing keys like that. and so you know they just go with whatever output they get. It seems to work when they're testing it and so they just publish it to to the web. And so sometimes that's how these things get get exposed which you know and and and I I don't expect that everyone's going to be a security expert, right? And not everybody knows to, you know, prompt like, hey, is there anything about this that like maybe I should consider doing to harden this before I publish this to the open web and start advertising it?

104:02 it would be nice if people thought to do that or if that was like sort of baked in whenever you're developing something vibe coding something. But >> there is a reason why it took over eight Peter we talked about on the public. There is a reason why it took over 8 n months for the bigger companies to release someone like a personal agent to you like OpenClaw and Hermes which you run on your computer and you're kind in charge of the security itself. That's one thing. There's a completely another thing to allow people to provide their passwords passwords and then save them in in like a local but secure way that the LM doesn't have access to. Like all that infrastructure has to be in place for people to like to to go trust because like if one of those vulnerabilities goes around and actually like steals your data from your personal AI assistant is going to be like awful for a reputation of the company. There is a reason why like only now almost a year after Peter's time innovated with open cloud we're seeing these companies step in despite them very clearly realizing the benefits of hey personal agents is like much more sticky sells more tokens it's harder to to to switch from etc because there's security there w to your to your metaphor from before building your house versus renting it so I would I would add on that metaphor given the security considerations we're talking about now this is the difference between building your house and not really knowing what a a doorbolt security bolt is and living in the high-rise apartment with a security guard at the door, right? Like Muse, the folks have hired Moxy Marlin Spike, the folks from Signal for the E2E connections, they are known for security significantly more than you just like vibe coding your server in engines and running this on on like PHP, which is now fully compromised, which is scary.

105:38 The practical advice that we want to give folks is obviously if you are having any API tokens and you were on those exposed websites, open code or railway etc. rotate them like know which agent has which API token is very very important. I know it's a pain in the ass but at days like this a pain in the ass is very well worth it. Put caps on your API tokens as well. Put caps and email notifications. Get email notifications when your API keys are like running out.

106:04 Put you know it's better to to put auto auto refill because then you at least get a notification from your credit card statement like your key has been auto refilled. So if you see like multiple ones happening in succession, you'd be able to like run and stop this bleed basically if your key was exposed. Rotating keys is very important. And I think the the last thing very clear for many people, many people don't have this yet. Use a goddamn password manager. Use a password manager and store those keys so you don't have to remember them and put them in files. It is it is increasingly more important now than it used to be before given that you know you're giving your keys to agents etc.

106:42 Wolf, you want to chime in with some security practices simple for folks? >> Yeah, you you already gave the practice I wanted to mention, but you should treat it like a credit card, but it's even worse than that because the credit card you also have the CVC number and you often have to do a two factor to authorize a payment >> and credit cards are secured and protected by the credit card companies. I don't think your API key is going to get refunded by OpenI if it was leaked.

107:04 Like a very very big point like they can, you know. >> Yeah. So, treat them even more securely than your credit. I I I have the most fun advice. Just install Tails scale everywhere and give it to your go on Instagram on your Meta Muse and then you just talk to Muse on Instagram and just be like, "Hey, every day just go on every single device I have and run a security check of every npm and and pip needed thing and just give me a report." I I'm joking to do it with Muse. I I do it with Opus. it. But it's a very simple prompt. Just every day at 6 am just check every single pip, npm, Linux package, whatever I have on my system and give me and also check up the the latest like snick and socket io news and give me just an overall security report of what I need to do or send me an alert if it's something bad.

107:59 >> That's it. It just runs it every day. It's pretty simple. That's what's saved my ass almost all the time. Josh, any any other jams into like what folks can realistically do to to like at least somewhat protect themselves in this world of increasing security vulnerabilities and and agent hacks companies. >> Yeah. I so generally speaking the the standard security vice advice just kind of applies here. So >> eat your vegetables type stuff. Yes. >> Yeah. Yeah. Yeah. Yeah. So like obviously make sure you're using multiffactor authentication on everything. you know make sure you're keeping your operating system, your browser and everything up to date.

108:39 when there are new patches issued, make sure you apply them quickly. the the one kind of caveat here, the the irony is that you know with some of these software supply chain attacks, right? The the light LLM thing. that's one of those awkward situations where if you are following that best practice of like updating immediately like as soon as something comes out you could ironically end up getting compromised instead because of that supply chain attack. It's it's kind of one of those unfortunate catch22 situations. Generally speaking for for anyone listening if if you keep everything up to date and and update pretty quickly that's the best way you can protect yourself.

109:26 >> Let's draw a very clear distinction here between what you keep up to date and what you need to like wait for supply chain attacks they apply to package management right for software engineers. So if you are installing from UV or pip or npm or like PNP whatever those packages those packages can get exposed by hacking the folks who distribute those packages and then put like key extraction things or other things that can hurt your system. Do not update those just willy-nilly. There needs to be a period of 24 hours at least to let the folks know that they've been hacked and like claw back the the the light limb like exposed. This is what's called the supply chain attacks browsers and we haven't seen like any supply chains and browser update like update your browsers immediately update your operating systems as soon as those patches come out and for PHP and other places you have to patch the actual patches that they released on your system. So th those are like kind of like different package management supply chain attacks do not update immediately. In fact, the best practice that we know to to help you against supply chain attacks is to put a blocker to say, "Hey, I'm not installing any packages that are like at least 24 hours old because hopefully in the 24 hours that passed the the folks who got hacked and the package was like infected, the the package manager, you know, will take it back, right? Clo it back that so package managers do not install the new ones. Don't kept them up to date like Josh was saying.

110:44 operating systems, phones, you know, browsers, those things need to be when you see that update notification, hit that like literally >> update. >> Yeah. Well, well said. That's a good distinction to make between stuff that like consumers are running typically versus stuff that is running behind the scenes, right? So, so yeah, really good point about that. for for a lot of software supply chain attacks some sometimes they're caught quickly sometimes they're caught you know within a couple of hours like the light LLM thing recently so in other cases they might kind of linger a little longer than that so but but I I think that's that 24-hour thing unless you know there's some vulnerability that you have to patch right now that was just fixed in the latest version right and if something like that does happen, they're going to make a big deal out of it, right? They'll let people know it's going to be a well-known like public thing more than likely. and in those cases, maybe you need to patch right away. if it's not that kind of a situation where you know there's some urgent vulnerability that's got to be patched, then again, when we're talking about the software supply chain kind of things, probably a good idea to to wait a little bit of time just to make sure.

112:00 >> Hi folks, we're coming to the end of the show. So Josh, thank you so much for joining us and and very clearly helping folks and with with with positivity in mind as well. I think it's very important because my my my comment to many folks in group chats who are freaking the out. If you are freaking out on X, yes, it's scary. Yes, there's a lot of stuff. The algorithm is poised. This is what the algorithm loves when you click on a on a thing and you hover and you like, oh, and even if you comment and write something in that area, the the the fear and gloom and FUD and like all of these things the algorithm absolutely loves. So you are will get constantly people who are like first joining the algorithm and to get some clicks but also like people who just comment on the same stuff right but you'll be fed this fear over and over again in different forms and different types different varieties because the algorithm loves the engagement it does not mean the sky is falling go out touch some grass and then come back and run some security stuff because it is important but it's like the the internet I've seen some folks say hey go take out you know $20,000 in cash to put it because like the the banks are like all done scary But I don't know if we're quite there yet because yeah, the companies are patching the PHP stuff, right? But definitely there's a lot of stuff that that the can of worms has been opened and be security mindful as well. So yeah, I think may maybe this is the advice we all now have access to basically security like experts if we can't like knock on Josh's door everybody to run stuff for us. So definitely use that. The other thing that you did bring and it's kind of like a breaking news type situation. Peter, thank you for bringing this up. We talked about like agent security. One password finally comes out with the platform for you and your agents to be able to run. They they're mentioning all of literally everyone bot and entropic and claude and openi and muse and browse and town and like etc. One password for agents give your agent access without giving up your passwords. When agent reaches a signin form and request credential from your vault you approve the request one password directly autofills the credential in the destinations. I think that's incredibly important. I've been waiting for this since, you know, Muse announced this and it's just like absolutely crazy. I said I signed up. Let me in there. So, shout out to One Password for like finally like getting there. If you are using one password, join this weight list. I think it's very very important. It's kind of like the Stripe link for credit cards but for passwords and I think it's very important. Again, if you are using agents at all, even if you are not using a one like password manager, one password I found for the past 15 years to be the most reliable for me. Josh, thank you so much for joining folks.

114:26 We're two and a half hours on the stream, but I think it was very very important to at least mention some of the stuff because it does feel like the defenders window is coming to a close and a lot of swarms and a lot of like bad actors with AIS are now taking over. Like Nissan said, you you do need to take some steps and dedicate some time in your personal life to kind of protect yourself as well. The eat your vegetable stuff are always there freezing your credit and and specifically now I think the the the the more how should I say the the the biggest exposure for folks who not realizing that this is exposure is API keys for LM. Those are directly connected to your account especially if auto refill is there. put caps, put notifications on, and then make sure that you rotate them if a website was like like railway was exposed or or open code.

115:13 With that, I think we'll say thanks to folks on the stage. and then we'll sign off. This is it for Thursday for October 22nd. The biggest news that we've talked about is this the vulnerability security stuff. We talked about Flint from Cursive AI being built by the Eureka machine which is a recursive improvement machine and is getting a very fast local model. We talked about holes in PHP and your AI keys and a lot more andropic economy PBC and entropic stepping into the personal agent with cla duet which we got access to and we'll tell you all about next week. Hopefully by then the wait list will open. Thank you so much for everybody for tuning in for listening into Thursday I Josh thank you for joining Peter Gustavj Wolf from Niss was here in Yam before and Tim joined us as well. Thank you all for joining Thursday I will see you here next week. Bye, Ron.