Not Another AI Podcast Ep. 09: 10x Number Anthropic Showed Us (And Why They Told Us Not to Switch)

0:00 my team went out to the anthropic base camp and there were five really good senior engineers sitting at the same table working on the same project. We stood up a repo within 5 minutes we had a major merge conflict that was difficult to resolve. We resolve it and by minute 7 our conversation became like okay how do we modularize what each person is working on so we can actually work on a project simultaneously without like all of us stepping on each other.

0:32 >> >> Welcome back. Today we want to talk about collapsing the stack using AI and many directions it will take us. We have sepal here and also Shanti and me P B P B P B P B P B P B P B P B P B P Banka Jen. Let's get into it. Sep you want to introduce yourself and kind of talk about the topic a little bit.

1:05 >> Thanks Pyanka. I'm Sep. I lead engineering data and intelligence for AMA health solutions primarily which is our business that takes data products to market. At heart I'm a builder and operator. I've spent my career at the intersection of technology, data, product, right? and primarily my background has been financial sector which is capital markets and past few years I've been in healthcare right and at AMA my team is responsible for AMA intelligent platform our developer and API ecosystem our private AI infrastructure and number of data and intelligence product that we've been launching in recent past a big part of my work today is thinking about how we take incredibly valuable assets that we have at the AMA to market such as clinical content standards s physician data etc. from simply distributing data and content towards delivering intelligence. So this is why I'm excited to be here because what particularly is interesting for me is that we're not doing this in a startup or a green field environment. We're doing it in a established organization such as AMA.

2:11 >> Yeah. 175 years, right? How old is >> 175 years of legacy? That that actually puts us in a very unique position, Bianca, right? Like we've got a tremendous data assets. and and and they have like like they say your data and your assets is key in the world of AI, right? And that they can unlock a lot of potential. That's the that's the piece that I am most excited about. So today I want to share mostly from practitioner perspective is like what we've learned building these systems and platforms and putting AI into the production but also what kind of value and learning you get when you try to take these kind of assets to market.

2:49 Yeah. So I'm excited to be here. So when you talked about when you were when you and I were talking prior and talking about collapsing the stack using AI, my thought went to you know all of us let's think about a a normal like a B2B organization let's say think about like a mid-market or an SMB 20 to$100 million B2B organization the stack that they have in terms of you know like a marketing stack the the commercial stack, you know, starting from Salesforce to HubSpot to amplitude to but if they're B2C, they definitely have some product component, they have amplitude and they have on the other on the DevOps sides, you have tools on the on the you know finance side, you have so many tools starting from QBO to Netswuite to maybe payroll is through ADP whatever like there's so many applications that you are using each in each function is using three or four or five applications to get job done. and is that how you were thinking collapsing that kind of stack using AI or what were you thinking?

4:04 >> Yeah, I I mean that's one part of it and well I think like you said jobs to be done right that is a phrase that sticks with me a lot more now that I look around the market. I think AI is collapsing the distance between intent and outcome. So less and less software is about how many applications you have but can your software get the job done for you which is you're able to describe your intent of what information you want for example tell me what is my highest performing account or tell me the number of leads that I generated through XYZ source that could mean going across your salesforce going across your other lead genen systems but but and then providing that context so it's not just about system of record. It's about context and then bringing a model/intelligence together to answer a specific question. That is how I think about when I think about collapse and stack. So I think you're moving more and more from number like your unit being just how many applications you use to how many jobs got done and and that is an interesting way I I see things have been collapsing a lot more. And then >> you are talking about collapsing is the time between this this this job needs to be done to when it's done versus do I really need 20 applications. So and and and I think that's the next piece of it meaning when you really start to look around what all these applications did there is going to be an efficiency gain to be achieved if you start looking at surfaces because it really then becomes about your data an intelligence layer that is able to get to it and if these applications have a unique workflow that allows you to bring a data in then that application is critical to your infrastructure.

5:59 In most cases, if your application was just a dashboard or an information sharing mechanism because that was meant for mainly humans. Now you're describing in an agent is actually looking for the context. So you you're kind of even consumer is changing from that perspective. Right? So so those are the type of applications are going to get retired from the stack is how I think about it because previously it was like we spend decades collecting data first then building apps on top of it then dashboards on top of it and then there was a human who would take a look at this application and go in another application and take action. I don't think that kind of workflow stands anymore right and the workflows are changing by the minute here. Yeah.

6:41 >> Danti, how are you thinking about this topic? a couple of different ways. One is when you compare what we're doing today with what models can do on their own with the harnessing we used to have to put around them, a lot of those middle layers got collapsed by model capability. So we were focused on like, oh, I've got a build rag and what does that mean? And now it's like, can I just point my model at some context? It will figure that out for me. Or, oh, I need all of these MCP servers with these special tools. like no we can figure that out with some skills and and what are skills are just some extra directions on how to do a thing.

7:18 >> So a lot of the orchestration the agentic looping the rag pipelines the special querying got compressed down and became features of the model itself. The model can handle some of those pieces. I mean it helps with having the right tooling in place and some harnessing is still necessary. but yeah, there there's a lot of pieces that you don't have to build anymore. Model will do that with good model. And when we say what's a good model now, it's well, can it make tool calls really well? Is it smart enough to orchestrate a whole bunch of things?

7:50 >> Yeah. Yeah. And I really like how many CL connectors Claude has. I mean four for me 4.7 4.7 I think I feel like 4.5 or 4 o 4.5 or 4.7 started changing a lot of >> these these things where suddenly oh you know stripe like claude has an internal connection to stripe already I don't have to do anything just point to it and it kind of goes and looks at all these transaction the o exist I just need to enable o and then boom, right? And can actually go and consume it like a human would line by line.

8:30 >> Yeah. >> And look at stuff. >> do you >> I was say Pianca, do you do you find yourself getting like super frustrated at it when you're like why do I have to find an API key or token? Like why isn't this an integrated tool? I should just be able to log in and you hold on to that. Like why? Come on. Like I know you can do these things >> asking me for that API key and I have to say Shanti I think I was just having this conversation this morning with my team and somebody gave me an example of gum loop a developer said oh a CFO will be able to go and do this whatever this was we were talking in gum loop and I'm like >> do you go and see the gum loop's requirement it says go figure out the API let me ask let me ask a CFO who knows what where their API is for a QBO itself right like tell me if a single non tech non-technology non-developer person will know how to track that API information but you know that so that's an interesting point and why I think it's important to keep that structure for them for the purpose of security that it is not available in the way that it can be available like just go talk to my systems but it is interesting how scary API is or how inaccessible just the API key itself.

9:52 >> I I think that's a important distinction you're making Priyanka there is that enterprises need that trust and from a scaling perspective. so yes it'll be great for models to do all of that and then you want an authority or some sort of a provenence that defines what's allowed to do what's not allowed to do. So can you b use a credit card on my behalf? probably it needs an authority in that scenario that those kind of things are important u and and and you're right the 4.7 opus was the time when it started changing but it's also hardness right it's not just a model yeah the hardness is the architecture around model right that is how you're systemizing all of these things and things come together which is which is the great part of this is that's why I think about architecture being your key component because your models will continue to get better But if your architecture almost assumes that's going to happen and focuses on connecting the dots like Shanti was saying that I can point it to a folder now. So that's an integration challenge that we've now solved for and because I think one more trend that one more way I think about it is we spent a decade making enterprise data centralized so that humans can act on it and have right information at their fingertips. Now, now you're really making them legible for models or legible for the API system and and >> I I have a significant push back on that Sundep. I don't know a single company who has a centralized source of data. Where is it? It's like a unicorn that never existed will never exist. Everybody talks about it.

11:31 >> But you know the direction, right? You know the direction we were going. Obviously, even with models, it's going to be like that, right? Unfortunately, we didn't even get to the centralization of data. And now the problem is making sure so if you have you don't have that data infrastructure and you don't have this way of contextualizing and making that layer intelligence or intelligible enough for models to kind of act on it, >> you're going to struggle anyway. So yes, you have to think through stack like it's a data problem, it's a context problem and then then your models will act. obviously on a personal level we can for smaller workflows point them to a folder and and have it enough context to kind of act on it. and this this is where architectures are becoming much more important. I think even in terms of people by the way Bianca when I think about collapsing stack and I'm interested in hearing your take also on talent side of the things is also from skills required from people are changing. So yesterday I was having a chat with one of my engineers and we were kind of thinking through this is like is is traditional hierarchy of product part makes any more sense or how do we start thinking about it and we we're experimenting like some people are good at prototyping and getting stuff off the ground. Some people are good at building meaning deploying it to the production. the the next level is people some people especially product people are good at scaling and adopting and taking it to the market and then there are few people who tinker around the edges and they perfect the whole thing that you got together now it's not traditionally you can say hey it's a software engineer job or a product job so that's also something that we're kind of thinking through as from a bringing a part together if that makes sense I are there any trends like that you're seeing Shanti particularly >> oh I yes I think There are a ton of trends in that direction and I'll stick with our collapsing theme like it's the collapsing team, right? The there's a team size that you know used to be like scrum teams of eight like the two pizza rule whatever they were going with at the time but I really think that team should be about four where you've got half of a product owner, half of a tech lead and two full-time developers. the way to per work stream and if you got multiple work streams you break them up into teams of four humans three like roles that they're playing in and the reason I think that makes sense is you can't have that many people in the codebase at the same time anymore you will just end up with merge conflicts too the agents can make really widespread changes so unless you're very specific and my team went out to the anthropic base camp and there were five really good senior engineers sitting at the same table working on the same project. We stood up a repo within five minutes we had a major merge conflict that was difficult to resolve. We resolve it and by minute seven our conversation became like okay how do we modularize what each person is working on so we can actually work on a project simultaneously without like all of us stepping on each other. And then I went off and started building like a video for the thing with AI and some music like let let me get out of the codebase a little bit like can somebody one person's working on a sound module. So, we really did start breaking it up into smaller pieces, but any one of us could have done probably 80% of those things, but it was there.

14:58 We're having some fun. >> But that smaller team can be really focused. And now what I found is it is very difficult if you're the the developer in charge of delivery of a product. You cannot be split across two products when that's your main like you need to focus because you have to be able to answer the agents questions and your context has to be what am I trying to have this do and you really need all of the business context of what are we trying to accomplish with this why are we trying to do that is this change that's being suggested a reasonable change not a reasonable change and you've got to guide you've got to provide that architectural direction engineering direction like you need expertise you can't just like throw it at the wall and Like the model will do it. It is smart, right? The model will do something and it'll look pretty good, but that's not enterprise software.

15:48 That's vibecoded slop. And the line keeps moving between like, oh, but it works, so we're going to use it. Like, you can use it a little bit. And then at some point, you're going to be like, oh, where did all our data go? Well, you see, they threw it up in Superbase. Oh, what's Superbase's data retention policy? Oh, nobody's ever read that, have they? All right. that that that's that's why I say it said that prototypers is a different skill than the than the builder because yes you need to think in systems it becomes even more important the systems principles in the AIF era of agents and I'm sure the harnesses will continue to get mature and better that that lot of the software engineering principles will be baked into it but yeah Shanti you you hit the nail on the head there and >> so you are >> yeah thank you so to interrupt sep I You guys are talking about collapsing team and then collapsing stack. Collapsing stack was almost like collapsing AI stack. And then there was a third competent which is collapsing the business stack >> of applications where we have >> oh yeah >> and then how does how does AI fit into that where I understood Sep from your earlier comment anything which is not really doing anything but reporting on it like let's say Tableau if I can build if I can connect to if I can connect my cloud co-work to stripe and salesforce then I can build my booking report in my booking potentially bookings report and claude in a dashboard so I don't need Tableau so that's that's where I can you know there could be one stat which goes away or or things like that >> man yeah go ahead >> all that all that Salesforce hate and I was about to say we don't need CRM systems because we will build our own >> now we do need Slack so see Salesforce still has one product that we like just not one that they created >> and and the pod I'm hearing from Salesforce this becomes a system of record because you've been doing years of work to kind of populate it. So again, it's a pivot for them too, right?

17:50 But but you're correct. Yeah. >> If you're a smaller enterprise and starting today, you would think about your CRM stack and what that looks like right now. >> Yeah. We we actually recently built our own CRM system. We're just like we this way it'll do what we want it to do. It'll be more agentic. It'll actually rather than just being a place where your boss can check on did you do the thing you were supposed to do? how many leads are there could actually provide proactive advice on here's what you should do next here are the next actions that you want to take today >> so it's functioning I know Shanti a few episodes ago we talked about you guys are building your CRM yep >> is it in production has it replaced >> has it replaced most important question has it replaced your prior CRM >> I don't even know that the prior CRM still exists I hope we're not paying for it because but yeah know we we talk about the new one all like sales leads, opportunities, reporting all happens out of the new system. Okay, >> it was a relatively seamless transition.

18:49 it looks relatively nice in terms of UI and UX feel, especially compared to Salesforce because everything looks nice compared to that interface. I I was did some Salesforce app dev and integration at some point in my life and it's just like how can it take so long for like a click you click a button and you're just like waiting and waiting for something to load in the background. >> Yeah. because it's a very very heavy application and here you've built point of >> point of system just custom for yourself and so it also brings up this new world where we are going into where every application every you can imagine at least the mid midcap or SMBs building if they have enough developers in house developing so software SAS or software applications internally versus all that subscription s like they said SAS is dead.

19:46 >> U it's it's possible. It seems like it's possible more so than not because >> Yeah. Well, think about all the money that these midsize companies would spend on customizing something to fit their needs, right? You buy something off the shelf and then spend a ton of money customizing it. >> Y >> and then it mostly fits your need, but it doesn't really understand your business. And now it's so easy to build software that finding niche software that is exactly tailored to your business is simple. And if it's not, you build it yourself. And that cycle is weeks to months. It's not years of building. Now, is it months of testing to make sure it's hardened and does everything right? Yes, absolutely.

20:29 Testing has not gotten easier, >> but you're testing a working system. Yeah, it's gone gotten 10folds harder because the build is easier now. Testing and fixing it, retrofix fitting it is much much harder, much longer. But it is possible. Let's say you can build a CRM system which is well tested in four months with a few developers. >> Oh yeah. >> Yeah. >> Yeah. There were like one of my personal project companies. We've got a app that's out there for understanding car maintenance, car repair status, like diagnostics, does all of those things.

21:07 And we're getting stuck in app store review. And I was talking to the co-founder and I was like, man, I really let the scope on this thing creep because like, of course, I can build those features. Why not? So instead of going live with like four wellproven features, we've got like 20 features and there's so many application pathways that you have to test and go like, well, what happens if you try to get from here to there and you want to pass this data through? Will it go through?

21:31 >> Yeah. >> If we wanted to fully test this, it would probably take three months of dedicated testing to just figure out all of the different ways you could use it. I'm like, or we're just like, try to get it out of the market. Let the market test it. >> Yeah. Yeah. I think there's also a time in future which is of micro apps that you don't build you build specific applications just for that purpose which does only four things unlike our phones >> you know our phones has 100% capability we use 2% of it >> and then 98% sits there maybe and and and thereby the application is heavier how about we start like I think it's that's another trend which is going to happen internal builds more also micro micro apps.

22:16 >> Oh yeah. >> Versus, you know, heavier fully functional build that lives forever. Micro apps which has which probably ends of life very easily or quickly. I don't know. >> one of the challenges in this direction we are straying little off if you want. No, I was just saying that's that's great point that micro apps is talking about because that's where I think workflows become more important and that becomes your secret source or or a real mo around it. If you have a workflow that can tap into the ecosystem that you're talking about that system that you're now creating can also have a learning loop going on and all of those things can come together which is which is which is interesting. I and and I'm going to go back to your first point by the way Panka one of the things that I wanted to say is on the collapsing stack on has started to change behaviors on our consumer side also. So what one and then we'll come back to the point you were making but but but what I was seeing is for example physicians in healthcare ecosystem usually have subscribed to number of journals for example for years and and New England Journal of Medicine etc. And now there are systems like open evidence and others who are coming together to kind of collect all of that information and make it available in an interface where you can literally ask questions and it'll respond with citations etc. Right? But that that's one model to be done. You take that to next level which is where we were thinking about our assets that we've kind of generated over a number of years. that can be just an API that can be plugged into an EHR system or or or some other software that health tech community is building.

24:02 Now the workflow has shifted again. They're not reading the PDFs or the long journals that you've been producing, but they're actually working in their workflows and they need to look up certain guidance because a certain patient with a certain condition is coming for a visit. It's right there, right? And >> yeah, it's like a giant control F which exists in any application. You can just go find any CPT code or anything it can kind of give you a intelligent journal on top like don't this does not fit it and maybe it actually automatically selects for you what CPT code you need to select >> it's a transition there's going to be human in the loop initially but then as systems get confident there are going to be more automation especially in the administrative workflows where the rest yeah 100% >> more tagging and the physicians don't have to hopefully spend all this administrative time which none no physician physician I've ever talked to ever likes this like 90 80% of my time is going into this filing this and doing this versus now the system is doing it itself and they can spend more time hopefully taking care of the patients than >> and that's we are big on that by the way like we we have we ran a huge campaign yeah a number of years ago on physician burnout and honestly that's one promise of AI I get most excited about is reducing the admin burden on physicians Because especially with ambient listening and how notes are getting created how they get documented it's more time for physicians to now actually talk to patients and and and changing their workflow and that has created numerous opportunities for us.

25:37 So that collapses the stack right now. Now now previously our data would go into some other software and some other software and then finally make its way somewhere. Now that distance is shortened new workflows are emerging. So from from collapsing stack perspective I think it's a changing the way the data and context was getting consumed b it's creating new workflows al together which is new opportunity and new way of reimagining all of this. >> Yeah. >> Right. Right. I also have a side topic that I've been thinking about lately.

26:11 we all know so going back to we are all building a lot of us are building in cloud core work. some of us are also using SLMs and and local you know local models. I I was thinking I've been thinking about this and then I I heard or read about Claude doing the experimentation on usage based stuff. I I I see some usage based credit in mind. Invariably you exhaust your subscription then you have to get usage based or credits and then obviously the true cost. So I have a couple of thoughts and I wanted your thoughts on it and this is more about costing and then that's where we'll end. One is what is the future look like when all this real cost of real cost of tokens I mean when the tokens are really fully costed what would the cost look like and what would the situation look like? What do you guys think and where do you think we are headed to?

27:10 >> Yeah >> I don't think the real cost is probably coming by the end of the year or early next time. Yeah. >> Shanti, you want to go first? >> Yep. >> sure. I actually had a talk with the Anthropic Enterprise team recently and they showed us some inside data on what our usage would be if we were paying on the consumption straight consumption plan. So enterprise plan versus where we are on the current team plan on subscriptions.

27:37 >> Yeah, >> it was a large difference. So much so that even they acknowledged you probably don't want to change now. We're like, "No, not if we don't have to." So, >> so >> was it 10 times, 20 times, 100 times? >> 10. >> 10 times. Okay. >> Yep. Yeah. 10. >> Yeah. >> And that's so that was it's interesting to see and just under getting that understanding. We've done experimentation for some of the stuff we have in production on with different models where we'll try DeepSeek Flash. And the deal is you can run things the same prompts through DeepSeek Flash, you know, four, five times. Have a sixth version that says which of the these are the best five.

28:20 Still cheaper because it's more than six times cheaper, right? It's 10 20 times cheaper. Will that become something people want to do? Well, it's as free as your time and your compute. Can you send those all out in parallel and look at them? Maybe. Maybe you can't, but the actual cost if what if that's what we're seeing in the token based pricing if that's the real cost >> which I was going to I was going to caveat I don't think even that is a real cost even that is heavily discounted >> based on external funds that's an idea they have but it's not fully loaded cost but imagine I mean all the valuations that these frontier model companies have right like do you do you >> yeah Yeah.

29:05 >> Well, and there there's talk that the the $2 trillion anthropic valuation is low. That their that their real actual number might be higher than that. Like that's that's crazy. >> That's probably covering almost most of the knowledge work. That's how they're kind of valuating that. But that's correct. >> Yeah. I mean I I think what they're saying is the model the model can do it. >> Yeah. Yeah. But but right if it's not a fully factored in cost like Piana was saying because because it's heavily discounted you're going to the bill is going to be even larger and to your point Shanti then then those open weight models become even more critical for your long-term thinking from an infrastructure perspective. So I I think we should all as practitioners keep an eye on how this ecosystem evolves.

29:56 Obviously there's regulation and those kind of things that you have to watch out for but but here at AMA also we've taken a bet that we are we have a private AI infrastructure that we maintain just because some of our data assets are mission critical from a security perspective. So we do maintain a open weight models environment through which we try to serve up a lot of applications particularly applications around our proprietary data sets. and that's how we started but now we're seeing more and more need for a lot of the internal automation use cases which we would have just plugged cloud in or something else to see hey is it is it a good ROI for this kind of a use case or should we just have it call the open weight model API rather and and still does the job decent enough so you don't need a frontier model to do this kind of stuff and and that >> so something clarifying you're saying you are testing with the open internal models and they are performing at par >> with a frontier. So not at par for all use cases but for mo majority of automation use cases tool calling and doing summarization task and those sort of things right getting graphs reviewed and those sort of things yes they're doing pretty good and and a it gives you a predictable build meaning even if you go with cloud provider to have a GPU infrastructure if you're not actually running hardware that's fine the other trend I'm watching out for is like especially with small language models >> as your developers machines are getting larger and larger. Some people are experimenting if I can run it on my local >> and have less >> and then it's all contained in a box which is which is really cool. That could change the game too. Yeah.

31:41 >> One of the developers that I work with wrote Space Invaders in a single prompt with Quen 3.827 yesterday. >> There you go. >> And it didn't take that long. It's fully working. I lost in the first round pretty quickly, but you built a game with a single prompt. >> Wow. >> Yeah. On fully local. >> Wow. >> Yeah. >> And if if you look at the new flash models, those flash size models, Deep Seek, GLM3, 53, Quen 38, no. Yeah, Quen 38 Flash. yeah, they're all in that like 300 billion parameter range, which is almost exactly the size that'll fit on a single GB300.

32:23 I don't think that's a like mistake that they're like it can fit on a single GPU. Now, can you get that GPU? Not really. Kind of a expensive one, but >> you >> Well, we can rent it. You can rent it in your AWS just like any other virtual machine you're renting. >> Oh, don't don't rent your GPU machine through AWS, please. I like I'm nothing against AWS when you're listening Amazon folks, but that is the most expensive way to get a GPU like >> but the point being yeah it's arbitrage right >> run pod run pod for your experiments somebody else for when you want to go to production they're they're not their uptime isn't great for production but you want an experiment on a GPU they are your best bet like three bucks an hour seven bucks an hour for a GB300 I bought one recently for like five still still gives you some predictability, right? Yeah. It's not like you open, you know, you're paying by the hour, not by the token cost. So, >> yeah.

33:19 >> Well, and actually I was going to ask when you're running these openweight models, are you like do you have your own data center hardware or are you renting it in the cloud? >> So, we started with renting in the cloud and now we're kind of looking at options on bringing it inhouse especially as the scale is growing. >> So, interesting everything is going now on prem. I mean we this is first time in my career I would say that actually renting like buying is starting to make sense than renting. Yeah. Otherwise it would all it was always like yeah economical scale. Why would I want to host that myself? It's just going to cloud.

33:53 >> Yeah. >> If if we had to pay by the way but that's the trend today. >> Yeah. If we had to pay full consumption price, it would be very cost-effective for us to like build our own machines for hosting model inference instead. It it's that expensive per month. >> That's where I think we are going. So, end of end of this year, I I don't know the timing. What What did the anthropic guys tell you, Shanti? When are they going to do they have a timeline for when they're going to usage based pricing? Enter enter enterprise pricing is already usage based.

34:29 >> It is usage. >> So if you're on the enterprise plan and not not team and not individual then you're paying usage based pricing. >> Yeah. Yeah. But we are on teams. Most of us are on teams and individual. >> So the >> when is that going away? When is that going away? >> When you hit 150 employees. >> Okay. >> Or if you or if you need detailed audit logging that you don't want to keep yourself.

34:52 >> Yeah. But I think it's going away before Shanti. I think >> they're not saying that publicly. So, if it's if it's h if it's happening, that's not something they want to publicize. That would probably actually push people to a different provider more quickly who isn't doing that right away. That would be a that would be a really big deal if they got rid of all the subscriptionbased pricing, >> especially consuming kind of like Uber did. I think it's going to happen.

35:23 >> Oh, I think it'll happen. I I agree >> sooner than we think and I do think that the solution at that point would be a combination of what we just talked about onrem or on cloud but SLMs your internal harness your internal arbitrage of which which prompts you send to FMY models which which prompt you send to your local >> you know whatever >> and that intelligenting is also an important part of your stack. So yeah 100 >> yeah like we we use like internally we've been messing with light LLM for that smart routing and internal routing you can do the same thing with open router cloudflare has a open gateway so same idea you can kind of as long as you can sort of classify workloads in a smart way then you can route them to different models and that's what I really want to do because when you think about you're running an organization not everybody knows was what model they should use. So they just default to the smartest model. I don't want to have to hit the button twice. And one of the most interesting things we actually learned in the anthropic base camp was the way they think about it is no. Start with the lowest level of intelligence.

36:33 See if you can get the thing done. Then start adjusting your prompt and fixing yourself to try to get it done. And only when that fails do you step up to the next most intelligent model because Opus will infer what you meant in the prompt. So it will make up for bad humans and haiku won't make up for bad humans. So you can just keep stepping up and deciding like when does my human intelligence need to kick in and can I solve this problem or not? And you know who's really good at helping you write better prompts? Opus.

37:03 >> I hate that. I hate that approach. And I recently ran into this at co-work with co-work. I was working on something. It was taking longer. It gave me something with error. And I tried to shift it to at that point to fable or 4.7 or opus and it said all your context will be lost from this thing and so do you really want to do that? And I'm like wait a minute I've built a lot of context here. I don't want to do that.

37:30 Okay, I'm going to just like you know it was taking long. So I don't like that incremental approach from them. I don't know the solution for it, but I don't like necessarily. I think it makes logical sense from a user perspective. It doesn't make any sense. >> Oh, they're talking about it more for building like automated workloads, too. Like less for the dayto-day and more for I need to build this into a system that's going to run a thousand times for the I'm going to run it once, I just need it to work right. They don't care which model you choose all that much >> because it's, you know, a one-time cost.

38:04 It's not a huge deal. like I I like thinking about that, but I don't do it that way. I start at Fable until I run out of Fable credits. As long as long as this is a situation where I'm okay with the data retention policy, I'm starting at the top and only when I run out of money do I start working my way down. I will say the safety classifier is incredibly annoying. So, and Opus 5 has it also. So I was doing some Opus 5 work around social media data and one of the workloads has some posts in it and I was building a post review or classifier. So you can see the post and then classify what theme is in the post and it wouldn't let me it would not autocommit that code because like I feel like this code might be dangerous. My safety classifier won't let you autocommit. You have to run git commit on your own. like what of all of the things like you wrote the code, you did the thing, but I can't do git commit here.

39:04 >> Yeah, there are some I mean for for good reason they have some some of those sort of constraints put in with good heart I think. >> Yeah, >> but there are these things that you'll you'll we'll run into but well this this has been a good good conversation. I really appreciate this. but we'll wrap here and we'll come back next time and chew on some more important interesting stuff. Thank you Sep. Thank you, Shanti. >> Yeah, no problem. Pleasure. Have a good one.

39:29 >> And we can keep going. We should catch up again. This is great. Thank you so much.