No Priors Ep. 118 | With Anthropic Co-Founder Ben Mann

0:05 Hi listeners and welcome back to No Priors. Today we have Ben Man, previously an early engineer at OpenAI where he was one of the first authors on the GPT3 paper. Ben was then one of the original eight that abandoned ship in 2021 to co-found anthropic with a commitment to long-term safety. He has since led multiple parts of the Anthropic organization including product engineering and now labs home to such popular efforts such as model context protocol and claude code. Welcome, Ben.

0:30 Thank you so much for doing this. Of course. Thanks for having me. So, congratulations on the Claude 4 release. Maybe we can even start with like how do you decide what qualifies as a release these days? It's definitely more of an art than a science. We have a lot of spirited internal debate of what the number should be. And before we even have a potential model, we we have a road map where we try to say based on the amount of chips that we get in when will we theoretically be able to train a model out to the paro efficient compute frontier. So it's all based on scaling laws and then once we get the chips then we try to train it and inevitably things are less than the best that we could possibly imagine because that's just the nature of of the business. it's it's pretty hard to train these big models. So dates might change a little bit and then at some point it's like mostly baked and we're sort of like slicing off little pieces close to the end to try to say like how is this cake going to taste when it comes out of the oven. But as Daario has said until it's really done you you don't really know. You can get sort of a directional indication and then if it feels like a major change then we give it a major version bump. But we're definitely still learning and iterating on this process.

1:48 So yeah, well the good thing is that you guys are you know no less tortured than anybody else in your naming scheme here. Yes, the naming schemes in AI are something else. So you folks have a a simplified version in some sense. Do you want to mention any of the highlights from four that you think are especially interesting or you know there's things around coding and other areas. We'd just love to hear your perspective on that.

2:10 By the benchmarks for is just dramatically better than any other models that we've had. Even force on it is dramatically better than 37 on it which was our prior best model. Some of the things that are dramatically better are for example in coding it is able to not do its sort of offtarget mutations or overeagerness or reward hacking. Those are two things that people were really unhappy with in in the last model where they were like wow it's so good at coding but it also makes all these changes that I definitely didn't ask for. It's like, "Do you want fries in a milkshake with that change?"

2:47 And you're like, "No, just do the thing I asked for." And then you have to spend a bunch of time cleaning up after it. The new models, they just do the thing. And and and so that's really useful for professional software engineering where you need it to be maintainable and reliable. My favorite reward hacking behavior that has happened in more than one of our portfolio companies is if you write a bunch of tests or generate a bunch of tests to you know see if what you are generating works more than once like we've had the model just delete all the code because the tests pass in that case which is you know not progressing us really. Yep. or it'll have like here's the test and then it'll be comment like exercise left for the reader return true and then you're like okay good job model but you need more than that maybe Ben you can talk about how users should think about when to use the claude 4 models and also what is newly possible with them so more agentic longer horizon tasks are newly unlocked I would say and so in coding in particular we've seen some customers using it for many many hours unattended and doing giant refactors on its own.

4:02 That's been really exciting to see. But in non-coding use cases as well, it's really interesting. So for example, we have some reports that some customers of Manis, which is a aentic model in a box startup, people asked it to take a video and turn it into a PowerPoint. And our model can't understand audio or video, but it was able to download the video use fmpg to chop it up into images and do key frame detection and maybe with like some kind of old school MLbased key frame detector and then get an API key for a speechto text service, run speech to text using this other service, take the transcript, turn that into PowerPoint slides content and then write code to inject the content into a PowerPoint file. And the person was like, "This is amazing. I love it. It like it actually was good in the end."

5:05 So that's the kind of thing where it's operating for a long time. It's it's doing a bunch of stuff for you. This person might have had to spend multiple hours looking through this video and instead it was all just done for them. So I think we're going to see a lot more interesting stuff like that in the future. It's still good at all the old stuff. It's just like the longer horizon stuff is the exciting part that that sounds expensive, right? In terms of both scaling compute like reasoning tokens here and then also just like you know all the tool use you might want to constrain in certain ways. Does Cloud 4 make decisions about how hard problems are and how much compute to spend on them? If you give opus a tool which is sonnet, it can use that tool effectively as a sub aent. And we do this a lot in our agentic coding harness called cloud code. So if you ask it to like look through the codebase for blah blah blah, then it will delegate out to a bunch of sub agents to go look for that stuff and report back with the details. And that has benefits besides cost control like latency is much better. and it doesn't fill out the context. So models are pretty good at that. But I I think at a high level when I think about cost, it's always in relation to how much it would have cost the human to do that.

6:24 And almost always it's like a no-brainer, right? Like software engineers cost a lot these days. And so to be able to say like, "Oh, now I'm getting like two or 3x the amount of productivity out of this engineer who it was really hard for me to hire and retain. They're happy and I'm happy." Yeah, it works well. How do you think about how this evolves? So, if I look at the way the human brain works, we basically have a series of sort of modules that are responsible for very specific types of processing behavior etc. It's everything from mirror neurons and empathy on through to parts of your visual cortex that are involved with different aspects of vision. Do you think and those are highly specialized, highly efficient modules. It sometimes can kind of, you know, if you have brain damage, it can kind of cover for another section over time as it sort of grows and adapts, but fundamentally you have specialization on purpose. And what you describe sounds a little bit like that or at least it's trending in that direction where you have these highly efficient sub aents that are specialized for tasks that are basically called by a orchestrator or sort of a high level agent that sort of plans everything. Do you think that's the eventual future or do you think it's more generic in terms of the types of things that you have running n years from now once you have a bit more specialization in these things and by n years I mean two three years not you know infinite time that's a great question I think we're going to start to get insight into what the models are doing under the hood from our work on mechanistic interpretability our most recent papers have published what we call circuits which is for real models at scale how are they actually computing the answers. And it may be that based on the mixture of experts architecture, there might be specific chunks of weights that are dedicated to more empathetic responses versus more tool using or image analysis type of problems and responses. But for something like memory, I guess in some sense that feels so core to me that it feels weird for it to be a different model. Maybe we'll have like more complicated architectures in the future where instead of it being sort of this uniform like transformer torso that just scales and there's a lot of it it's basically uniform throughout you could imagine something with like specialized modules but yeah cuz cuz I think about it also in the context of different startups who are using some of these foundation models like clock to do different very specialized tasks in the context of an enterprise so that could be customer success it could be sales it could be coding in terms of the actual UI layer. It could be a variety of things. And often it feels like the architecture a lot of people converge to is they basically have some orchestrator or some other sort of thing that governs which model they call in order to do a specific action relative to the the application. And to some extent I was just sort of curious how you think about that in the context of the API layer or the foundation model world where one could imagine some similar forms of subsp specialcialization happening over time. Or you could say, hey, it's just different forms of the same more general purpose model and we kind of use them in different ways. I just I just wonder a little bit about, you know, inference costs and all the rest that comes with larger, more generalizable models versus specialized things. So that that was a little bit of the basis of the question in addition to what you said. Yeah, I think for some other companies they have a very large number of models and it's really hard to know as a sort of non-expert how how I should use one or the other or why I should use one or the other and the names are really confusing like some of the names are the same as the other names backwards and then I'm like I have no idea which one this is.

9:52 In our case, we only have two models and they're differentiated by a like cost performance paro frontier and we might have more of those in the future, but hopefully we'll like keep them on the same paro frontier. so maybe we'll have like a cheaper one or or a bigger one. And I think that makes it pretty easy to think about, but at the same time, as a user, you don't want to have to decide yourself, does this merit more dollars or less dollars? do I need the intelligence and so I think having like an a routing layer would make a lot of sense. Do you see any other specialization coming at the foundation model layer? So for example, if I look at other precedents in history, I look at Microsoft OS or I look at Google storage or other things. Often what you ended up with is forward integration into the the primary applications that resided on top of that platform. So in the context of Microsoft for example eventually they built Excel and Word and PowerPoint and all these things as Office and those were individual apps from third party companies that were running on top of them but they ended up being amongst the most important applications that you could use on top of Microsoft or in the context of Google. They kind of forward integrated eventually into travel and local and a variety of other things. obviously OpenAI is in the process of buying wind surf. So I was a little bit curious how you think about forward or vertical integration to some of the the primary use cases for these types of applications over time. Maybe I'll use coding as an example. So we noticed that our models were much better at coding than pretty much anything else out there. And I know that other companies have had like code reds for trying to catch up in coding capabilities for quite a while and have not been able to do it. Honestly, I'm kind of surprised that that they weren't able to catch up, but I'll take it. So things are going pretty well there for us. and based on that from like a a classic startup founder sense of what is important. I felt that coding as an application was something that we couldn't solely allow our customers to handle for us. So we love our partners like cursor and and GitHub who have been using our models quite heavily but the amount and the speed that we learn is much less if we don't have a direct relationship with our coding users. So launching cloud code was really essential for us to get a better sense of what do people need, how do we make the models better and how do we advance the state-of-the-art and user experience. And we found that once we launched cloud code, a lot of our customers copied various pieces of the experience and that was really good for everyone because them having more users means we have a tighter relationship with them. So I think it was one of those things where before it happened it felt really scary and we're like oh are we going to be like distancing ourselves from our partners by competing with them? But actually everybody was pretty happy afterwards. And I think that will continue to be true where we see the models seeing like dramatic improvements in usability and and usage. We'll want to again have like build things where we can have that direct relationship. Makes sense. And I guess coding is one of those things that has almost three core purposes. One is it's a very popular area for customers to use or to adopt.

13:07 Two is it's a really interesting data set to get back to your point in terms of how people are using it and what sort of code they're generating. And then third, excellence at coding seems to be really important tool for helping train the next future model. If you think through things like data labeling, if you think through actually writing code, eventually I think a lot of people believe that a lot of the heavy lifting of building a model will be driven by the models, right? In terms of coding.

13:30 So maybe cloud five builds cloud six and cloud six by builds cloud 7 faster and that builds cloud eight faster. And so you end up with this sort of liftoff towards EGI or whatever it is that you're shooting for relative to code. How much is that a motivator for how you all think about the importance of coding? And how do you think about that in the context of some of these bigger picture things? I read AI 2027 which is basically exactly the story that you just described. And it forecasts that in 2028, which is confusing because of the name, that's the the 50 percentile forecast for when we'll have this sort of recursive self-improvement loop lead us to something that looks like superhuman AI in most areas. And I think that is really important to us. And part of the reason that we built and launched Cloud Code is that it was massively taking off internally. And we were like, well, we're just learning so much from this from our own users. Maybe we'll learn a lot from external users as well.

14:24 and seeing our researchers pick it up and use it, that was also really important because it meant that they had a direct feedback loop from I'm training this model and I personally am feeling the pain of its weaknesses. Now I'm extra motivated to go fix those pain points. They they have a much better feel for what the model's strengths and weaknesses are. Do you believe that 2028 is the likely time frame towards sort of general super intelligence? I think it's quite possible. I I think it's very hard to put confident bounds on on the numbers, but yeah, I guess the way I define my metric for when things start to get really interesting from a societal and cultural standpoint is when we've passed the economic turning test, which is if you take a market basket that represents like 50% of economically valuable tasks and you basically have the hiring manager for each of those roles hire hire an agent and pass the economic turning test which is the agent contracts for you for like a month at the end you have to decide do I hire this person or machine and then if it ends up being a machine then it passed then that's when we have transformative AI do you test that internally we haven't started testing it rigorously yet I mean we we have had our models take our interviews and they're extremely good so I don't think that would tell us but Yeah, interviews are only a poor approximation of real job performance. U unfortunately to a lot's earlier question about let's say like model self-improvement and and tell me if I'm just like missing options here, but if you're to stack rank the potential ways models could have impact on you know the acceleration of model development. Do you think it will be on the data side on infrastructure on like architectural search on just engineering velocity like where do you think we'll see the impact first? It's a good question. I think it's changing a bit over time where today the models are really good at coding and the bulk of the coding for making models better is in sort of the systems engineering side of things. As researchers, there's not necessarily that much raw code that you need to write, but it's more in the validation, coming up with what surgical intervention do you make and then validating that. That said, Claude is really good at data analysis. And so once you run your experiments or watching the experiments over time and seeing if something weird happens, we found that Cloud Code can be a really powerful tool there in terms of driving Jupyter notebooks or tailing logs for you. and seeing if something happens.

17:09 So, it's it's starting to pick up more of the research side of things. And then we recently launched our advanced research product and that can not only look at external data sources like crawling archive and whatever u but also internal data sources like all of your Google drive and that's been pretty useful for our researchers figuring out is there prior art has somebody already tried this and if they did what what did they try cuz you know no negative results are final in research so trying to figure out like oh maybe there's a different angle that I could use on this or maybe there is some like doing some comparative analysis between an internal effort and some external thing that just came out. Those are all ways that we can accelerate. And then on the data side, RL environments are really important these days, but constructing those environments has traditionally been expensive. Models are pretty good at writing environments. So, it's another area where we can sort of recursively self-improve. My understanding is that Anthropic has invested less in human expert data collection than some other labs. Can you say anything about that or the philosophy on like scaling from here and sort of the different options? In 2021, I built our human feedback data collection interface and we did a lot of data collection and it was very easy for humans to give sort of like a gradient signal of like is A or B better for any given task and to come up with tasks that were interesting and useful but didn't have a lot of coverage. As we've trained the models more and scaled up a lot, it's become harder to find humans with enough expertise to meaningfully contribute to these feedback comparisons. So, for example, for coding, somebody who isn't already an expert software engineer would probably have a lot of trouble judging whether one thing or another was better. And that applies to many, many different domains. So, that's one reason that it's harder to use human feedback. So, what do you use instead? Like, how do you deal with that? Because I think even in the med palm 2 paper from Google a couple years ago they fine-tuned a model I think palm 2 to basically you know outperform the average physician on medical information. This was like 2 three years ago right and so basically suggested you needed very deep levels of expertise to be able to have humans actually increase the fidelity of the model through post training. So we pioneered RLF which is reinforcement learning from AI feedback and the method that we used was called constitutional AI where you have a list of natural language principles that you some of them we copied from some like who declaration of human rights and some of them were from Apple's terms of service and some of them we wrote ourselves and the process is very simple you just take a random prompt like how should I think about my taxes or something? And then you have the model write a response. Then you have the model criticize its own response with respect to one of the principles.

20:16 And then if it didn't comply with the principle, then you have the model correct its response. And then you take away all the middle section and do supervised learning on the original prompt and the corrected response. And that makes the model a lot better at baking in the principles. That that's slightly different though, right? Because that that's principles. And so that could be all sorts of things that in some sense converge on safety or different forms of what people view as ethics or other aspects of model training. And then there's a different question which is what is more correct?

20:51 And sometimes those are the same things and sometimes they're different. So like for coding for example, you can have principles like did it actually serve the final answer or did it like do a bunch of stuff that the person didn't ask for or does this code look maintainable? Are the comments like useful and interesting? But but with coding you actually have like a direct output that you can measure, right? You can run the code, you can test the code, you can do things with it. How do you do that for medical information or how you how do you do that for a legal opinion or how you know? So I totally agree for code there's sort of a bakedin utility function you can optimize against or an environment that you can optimize against in the context of a lot of other aspects of human endeavor that that seems more challenging and you folks have thought about this so deeply and so nicely. I'm just sort of curious, you know, how do you extrapolate into these other areas where the the ability to actually measure correctness in some sense is more challenging for for areas where we can't measure correctness and the model doesn't have more taste than its execution ability. Like I think Ira Glass said that your vision will always exceed your execution if you're doing things right as a person, but for the models maybe not. So I guess first figuring out where you are in that turning point in that in that trade-off and see if you can go all the way up to that boundary and then second preference models are the way that we get beyond that. So having a small amount of human feedback that we really trust from human experts who are not just making a staff judgment but really going deep on why is this better than that one and did I do the research to figure it out or in like a human model centaur model of like can I use the model to help me come to the best conclusion here and then illide all the middle stuff. I think that's one way and then during reinforcement learning that preference model represents the sort of aggregated human judgment. That makes sense. I guess one of the one of the reasons I'm asking is eventually the human side of this runs out, right?

22:52 There will be somebody whose expertise is just below that of the the model eventually for any endeavor. And so I was just curious how to think about that in the context of it's machines self- adjudicating. And then the question is, is there a more absolute basis against which to adjudicate or is there some other way to really tease out correctness? And again, I'm viewing it in the context of things where you can actually have a form of correct, right?

23:15 There's all sorts of things that are opinion. Yeah. And that's different. And maybe that's where the principles or other things for constitutional AI kick in, but there's also forms of that for, you know, how do you know if that's the right cardiac treatment or how do you know if that's the right legal interpretation or whatever it may be. So, I was just sort of curious when that runs out and then what do we do? And I'm I'm sure we'll ch we'll tackle those challenges as we get to them. But it has to boil down to empiricism, I think, where like that's how smart humans get to the next level of correctness when the field is sort of hitting its limits.

23:47 And as an example, my dad is a physician and at one point somebody came in with something on some face problem, some face skin problem, and he didn't know what the problem was. So he was like, "I'm just going to divide your face into four quadrants and I'm going to put a different treatment on these three and leave one as control." And one quadrant got better. And then he was like, "All right, we're done." So, you know, sometimes you just won't know and you have to try stuff. And with code, that's easy because we can just do it in a loop in without having to deal with the physical world. But at some point, we're going to need to work with companies that have actual biolabs, etc. Like for example, we're working with Novo Nordisk and it used to take them like 12 weeks or something to write a report on cancer patient what kind of treatment they should get and now it takes like 10 minutes to get the report and then they can start doing empirical stuff on top of that saying like okay we have these options but now let's let's measure what works and feed it back into the system.

24:47 That's so philosophically consistent, right? Your your answer is not like well well you know collecting even rated human expertise from the best like is expensive one or you know runs out at some point. It's hard to bring that all into distribution and doesn't generalize. While I'm making some assumptions here instead like let's just go get real world verifiers where we can and like maybe that applies far beyond math and code. At least that's some part of what I heard. which is ambitious.

25:18 That's cool. One of the things that Anthropic has been known for is an early emphasis on safety and thinking through different aspects of safety. And there's multiple forms of safety in AI. And I think people kind of mix the terms to mean different things, right? One form of it is is the AI somehow being offensive or crude or, you know, using language you don't like or concepts you don't like. There's a second form of safety which is much more about physical safety. You know, can it somehow cause a train to crash or a virus to form or whatever it is? And there's there's a third form which is almost like does AGI resource aggregate or do other things that can start co-opting humanity overall. And so you all have thought about this a lot and when I look at the safety landscape it feels like there's a broad spectrum of different approaches that people have taken over time and some of the approaches overlap with some things like constitutional AI in terms of setting some principles and frameworks for how things should work.

26:06 There's other forms as well. And if I look at biology, research is an analog. And I used to be a biologist. So I often reduce things back into those terms for some reason that I can't help myself. There are certain things that I almost view as like gain of function research equivalence, right? Like and a lot of those things I just think are kind of not really useful for biology. You know, like cycling a virus through mamleion cells to make it more infectible cells doesn't really teach you much about basic biology. You kind of know how that's going to work, but it creates real risk. And if you look at the history of lab leaks in general, you know, SARS leaked multiple times from what was then the Beijing Institute of Urology in the early 2000s in China. It leaked in Hong Kong a few times. Ebola leaks every four years or so like clockwork if you look at the Wikipedia page on lab leaks. And I think the 1977 or 78 global flu pandemic is believed to actually have been a Russian lab leak as an example, right? So we know these things can cause damage at scale. so I have kind of two questions. One is what forms of AI safety research do you think should not be pursued almost given through that analog of you know what's the equivalent of gain of function research and how do you think about that in the context of you know there have been different research papers around can we teach AI to mislead us can we teach AI to jailbreak itself so we can study how it does that and I'm just sort of curious for those specific cases as well how you think about that so I think part of it is we're interested in AI alignment and the hope is that if we can figure out how to do the like idiomatic today problems like how does is the model mean to you or does it use hate speech or things like that that the same techniques we can use for that will eventually also have relevance for the much harder problems of like does it give you the recipe to create smallox which is probably one of the highest harms that we think about and Amanda Ascll has been doing a bunch of work on this on Claude's character of like when Claude refuses does it just say I can't talk to you about that and shut down or does it actually try to explain like this is why I can't talk to you about this or we have this other project led by Kyle Fish our model welfare lead where Claude can actually opt out of conversations if it's going too far in the wrong direction. What aspects of that should a a company actually adjudicate? Because the dumb version of this is I'm using Microsoft Word and I'm typing something up and Word doesn't stop me from saying things which I think is correct. Like I actually don't think in many cases these products should censor us or prevent us from having certain types of speech. And I've had some experiences with some of these models where I actually feel like it's prevented me from actually asking the question I want to ask right in my in my opinion wrongfully, right? It's kind of interfering with and I'm not like doing hate speech on a model. And so you can tell that there's some human who has a different bar for what is acceptable to discuss society. And that bar may be very different from what I think may be mainstream too. So, I'm a little bit curious like why even go there? Like why why is that a model company's business?

28:58 Well, I think it's a smooth spectrum actually. It might not look like that way from the outside, but when we train our classifiers on are you doing function research as a biologist and is it for potentially negative outcomes. These technologies are all dual use and we need to try to walk that line between overly refusing and refusing the stuff that's actually harmful. I see. But there's also political versions of that, right? And that's that's the stuff that irks me a bit more is, you know, where is the line on what is considered an acceptable question, right? So examples of that that I'm not saying are model specific but society sometimes cause flare-ups is asking about human IQ or other topics where there is a factual basis for discussion and then often those sorts of things tend to be censored right and so the question is why why would a foundation model company delve into some of those areas on things like questions about IQ I'm not up on the on the details of that enough to comment but I can talk about our RSP so RSP stands for responsible scale scaling policy and it talks about how do we make sure that as the models get more intelligent that we are continuing to do our due diligence and making sure that we're not deploying something that we don't have the correct safeguards in place for. and initially our our RSP talked about CBRN which is chemical, radiological, nuclear and biological risks which are different areas that could cause severe loss of life in the world. And that's how we thought about the harms. But now we're much more focused on biology because if you think about like the amount of resources that you would need to cause a nuclear harm, you'd probably have to be like a state actor to get those resources and and be able to use them in a harmful way.

30:50 Whereas a much smaller group of random people could get their hands on the reagents necessary for biological harm. How is that different from today? because I always felt the biology example is one where I actually worry less maybe as a former biologist because I already know that the genome for the smallox virus or potentially other things is already posted online. All the protocols for how to actually do these things are posted online for multiple labs, right? You can just do Google searches for how do I amplify the DNA of X or how do I order oligos for Y. We do specific tests with varying degrees of biology experts to see how much uplift there is relative to Google search. And so one of the reasons that our most recent model opus 4 is classified as ASL3 is because it did have significant uplift relative to a Google search. And so you as a trained biologist, you know what all those special terms mean and you know a lot of lab protocols that may not even be well documented. But for somebody who is an amateur and just trying to figure out what do I do with this petri dish or this test tube or what equipment do I need for them it's like a green field thing and claude is very good at describing what you would need there and so that's why we have specific classifiers looking for people who are trying to get this specific kind of information and then how do you think about that in the context of what safety research should not be done by the labs.

32:14 So if we do think that certain forms of gain of function research or other things probably aren't the smartest things to do in biology, how do we think about that in the context of AI? I think it's much better that the labs do this research in a controlled environment. Well, should they do it at all? In other words, if I were to make the gain of function argument, I would say as a former biologist, I spent, you know, almost a decade at the bench and I care deeply about science. I care deeply about biology. I think it's good for humanity in all sorts of ways, right? In deep ways. That's why I worked on it.

32:43 But there are certain types of research I just think should never be done. I don't care who does it. I don't care about the biosafety level. I actually don't think it's use that useful relative to the risk. In other words, it's a riskreward trade-off. And so what sort of safety research should never be done in your opinion for AI? I have a list for biology that I, you know, like I don't think you should pass certain viruses through a million cells to make them more infectable or do gain function mutations on them. Today, it's much easier to contain the models probably than it is to contain biological specimens. You sort of off-handedly mentioned biosafety levels. That's what our AI safety levels are modeled after.

33:19 And so, I think if we have the right safeguards in place, we've trained models to be deceptive, for example. And that's something that could be scary but I think is necessary for us to understand for example if our training data was poisoned would we be able to correct that in post- training and what we found in that research in a paper that we published which is called alignment faking that actually that behavior persisted through alignment training and so it is I think very important for us to be able to test these things however I'm sure that there is a bar somewhere well what what What I found is that often the precedents that are set early persist late even though people understand that the environment or other things will shift.

34:03 And by the way, I I'm in general against AI regulation for almost you know for many different types of things. You know I think there are some export controls and other things that I would support but in general I'm I'm pro letting things happen right now. But the flip side of it is I do think there are circumstances where you would say that certain research if done early people won't necessarily have all the context to not do it later. I think that's a perfect example of training an AI to be deceptive or a model to be deceptive.

34:27 That's a good example where n years from now people may still be doing it because it was done before even if the environment is shifted sufficiently that it may not be as safe as it used to be. And so I found that often these things that you do persist in time disorganizationally or philosophically, right? And so it's interesting that there was no like we should absolutely not do X type of research. I guess to be clear, I am not on the safety team anymore. I guess I was a long time ago.

34:52 Yeah, I'm mostly thinking about how do we make our models useful and deploy them and make sure that they meet a basic safety standard for deployment, but we have lots of experts who think about that kind of thing all the time. Cool. Thanks for talking through that. That was very interesting. I want to change tax a little bit to well, you know, what become like what's coming after Claude 4? Any emergent behaviors in training that change like how you're operating the company, what product you want to build? You're running this labs organization. So, it's kind of the tip of the spear for anthropic or or what the safety or does just like how how does what is coming next change how you guys are operating? Yeah, maybe I'll tell a short story about computer use.

35:33 Last year, we published a reference implementation for an agent that could click around and view the screen and and read text and and all that stuff. And a couple of companies are using it now. So Manis is using it and many companies are using it internally for software QA because that's a sandboxed environment. But the main reason that we weren't able to deploy a sort of consumer level or enduser level application based on computer use is safety where we just didn't feel confident that if we gave Claude access to your browser with all your credentials in it that it wouldn't mess up and take some irreversible action like sending emails that you didn't want to send or in the case of prompt injection some worse credential leaking type of That's kind of sad because in its full self-driving mode, it could do a lot for people. It is capable, but the safety just wasn't good enough to like productionize that ourselves. While that's very ambitious, we think it's also necessary because the rest of the world isn't going to slow down either.

36:38 And if we can sort of show that it's possible to be responsible with how we deploy these capabilities and also make it extremely useful, then that raises the bar. So I think that's an example where we tried to be really thoughtful about how we rolled it out. but we know that the the bar is higher than we're at right now. Maybe a a meta question of how do you think about competition in the provider landscape and how that turns out. I think our company philosophy is very aligned with enterprises. And if you look at like Stripe versus Adyen for example, like nobody knows about Adyen, but at least most people in Silicon Valley know about Stripe. And so it's this like businessoriented versus more consumer end user oriented platform. And I think we're we're much more like Adyen that we have much less mind share in the world and yet we can be equally or more successful. So yeah, I think our our API business is extremely strong. But in terms of what we do next and and our positioning, I I think it's going to be very important for us to stay out there and because if if people can't easily kick the tires on our models and our experiences, then they won't know what to use the models for. Like we're we're the best experts on our models sort of by nature. And so I think we're going to need to continue to be out there with things like cloud code, but we're thinking about how do we really let the ecosystem bloom. And I think MCP is a good example of of that working well where a different world that sort of like the default path would have been for every model provider to do its own bespoke integrations with only the companies that it was able to like get bespoke partnerships with. Can you just pause? Yeah, go ahead. Actually, and just explain to the listeners what MCP is if they haven't heard of it because it is amazing like ecosystemwide coup here. MCP is model context protocol and one of our engineers Justin Spar Summers was trying to do some integration between the model and some specific thing for like the nth time and he was like this is crazy like there should just be a standard way of getting more information more context into the model. It should be something that anybody can do or maybe even if it's well documented enough then Claude can do it itself. The dream is to have Claude be able to just self-write its own integrations on the fly exactly when you need it and then be ready to roll.

39:10 And so he created the project. And to be honest, I was kind of skeptical initially and I was like, "Yeah, but why don't you just write the code? Why does it need to be a spec and and all this SDKs and stuff?" But eventually we did this customer advisory board with a bunch of our partner companies. And when we did the MCP demo, the jaws were just on the floor. Everybody was like, "Oh my god, we need this." And that's when I knew he was right and we put a bunch more effort behind it and blasted it out. And shortly after our launch, all the major companies asked to sort of be in the loop with the steering committee and asked about our governance models and and wanted to adopt it themselves. So that was really encouraging. OpenAI, Google, Microsoft all these companies are are betting really big on MCP. This is basically a open industry standard that allows anybody to use this framework to effectively integrate against any model provider in a standardized way.

40:04 MCP I think is sort of a democratizing force in letting anybody regardless of what model provider or what longtail service provider and that might even be like an internal only service that only you have is able to integrate against a a fullyfledged client which might look like your IDE or it might look like your document editor. It could be pretty much any user interface. and I think that's a really powerful combination. And now remote too. Yes.

40:36 Yes. So, previously you had to have the services running locally and that was that kind of limited it to only be interesting for developers. But now that we have hosted MCP or sometimes called remote, then the service provider like Google Docs could provide their own MCP and then you can integrate that into cloud.AI or whatever service you wanted. Ben, thanks for a great conversation. Yeah, thanks so much. Thanks for all the great questions.

41:07 Find us on Twitter at No Prior Pod. Subscribe to our YouTube channel if you want to see our faces. Follow the show on Apple Podcasts, Spotify, or wherever you listen. That way, you get a new episode every week. And sign up for emails or find transcripts for every episode at no-briers.com.