No Priors Ep 56 | With Baseten CEO and Co-Founder Tuhin Srivastava

0:05 hi listeners welcome to another episode of no priors today alad and I are catching up with toin rava the CEO and co-founder of base 10 which gives teams fast scalable AI infrastructure starting with inference they're one of the players at the center of the battle heating up around AI Computing welcome to him hi thanks for having me good to see you guys let's start at the beginning for any listeners who don't know what is base 10 and how did you

0:28 start working on it base 10 is is a infrastructure product so we provide fast scalable AI infrastructure for engineering teams working with large models currently we're focused on inference and we want to do a lot more after that but you know for the past say four and a half years actually oh that's a long time for for last four and a half years we've been you know cutting our teeth and trying to build this thing I think it's been pretty rewarding

0:52 over the last 12 months seeing the market kind of show up and you know everyone get equally excited about a infrastructure we started this Hest because firstly you know we thought ml was pretty cool in in 2019 we thought it was gone somewhere and we wanted to build a PIX and shovel business and kind of solve the problems that we were running into I think the side note here is that I wanted to S accompany with my friends you often say that base

1:17 10 isn't no code it's efficient code like why does that difference matter that wasn't always the case I'd say I'd say like you know those times when we had Elements which were definitely a bit no Cody I think what we've learned over the last three or four years years is you know code is just incredibly powerful and Engineers want to write code even in its best form you know you want to build like really really tight

1:38 abstractions but I think the the ability to turn the knobs under the hood is is very very important I think no code kind of makes that a lot harder I don't think it removes it but it makes it a lot harder so what we what we do is just build very strong intuitive abstractions that try to make the easy things super easy and but still make the hard things possible so you you can get a lot of value really quickly but I

2:04 say unlike a lot of other infrastructure products that' been built over the last 10 years we're trying to solve against the graduation problem which is that you know that we're able to support teams as they grow in scale and just to sort of make it a little bit more versal for our listeners like what what are the types of applications that run on base 10 like what's the scale of the platform do you have a favorite application everything

2:26 from you know tiny side projects on weekend all the way to companies that are pretty AI native you know we've we've supported Foundation model companies we work with companies like descript where AI is very very Coe to the product experience we power a lot of AI features that patreon is shipped but I'd say some of the more interesting use cases from our perspective actually or my perspective at least are either you know the really small teams that

2:51 we're giving a lot of Leverage to so that they can ship things very quickly so a really good example of that might be a company like plan AI which is basically building a SDK for coal centers is how I describe it but you know they're able to ship models and you know collocate workloads so that they can get you know sub 300 millisecond or sub 200 millisecond responses without you know months and months of infrastructure effort I think

3:19 on the other hand it's like I'm it's really exciting to see you know companies become AI enabled because that's where we see a lot of the value is going to be over the next decade is you know at a company like picnic Health which has actually been around for a decade and starting to do a very very interesting thing with the CPUs of data that they've gathered over the last 10 years and like supporting those use

3:39 cases I think their model is called picnic GPT which extracts information from medical records and you know to me that's those are the really exciting use cases where you know you're get you're giving leverage to companies that are good at like they're good at the Domain that they are working in the model might be proprietary the data might be proprietary but the infrastructure doesn't necessarily need to be proprietary and we can give them we can give them just an easy way to

4:06 deploy that stuff without many many people months it's become like in in in Vogue to compare the size of your GPU cluster like people are spending a lot of money on gpus we hear about 600,000 h100 equivalents and lots of venture rounds being raised often to train you know large models in some domain or another or even you know more and more expensive posttraining are are training and inference workloads different yeah I think so I think they just have very

4:34 different almost like slas for the customer like you know the things that matter for inference things like you know your your clusters somewhat collocated with where you're doing your work whereas whereas training and stuff like that matters a bit less it doesn't really matter where that training is happening as long as all your gpus are somewhat together you know even the GPU classes themselves like you know the for training networking is a very very important piece to have networking

5:06 on the racks themselves where influen matters a little less because you're doing a a little bit more on individual gpus and less of across gpus I think from a user perspective there's a lot more workflow I'd say in inference that's repeated across customers as opposed to you know you guys work with a bunch of companies training models you know the stateof the-art really is give me some SSH keys and let let me go at it whereas with inference is

5:33 definitely repeated workflow where people are trying to get similar things out of the inference infrastructure whether that be work whether that be you know version management whether that be the way they deployed hooking it up into cicd you know cold starts and so on and so forth so I'd say it seems a bit more repeatable today how the problem is being solved by customers I'd say the hardware requirements are quite different they're probably a little less

5:58 to to some degree but I think you know resiliency and reliability matters a lot more you know you know downtime is unacceptable from an INF perspective notes get terminated all the time from a training perspective you were quite early to this market and I think you folks pioneered a lot of the sort of early ml sort of infrastructure for these sorts of use cases and applications what has been the most surprising thing or what did you least

6:23 expect relative to how things evolved I think I can answer that question the two different altitudes I think you can answer that question from like a market perspective which is that you know I think El you have some old writing which is you know markets basically markets or all that matter I think we've felt that very viscerally in some sense which is that you know you can build all this cool stuff and then when markets show up you feel

6:49 that and that really pushes you know the customer forward and the needs for your product for I I think so that's one thing which is like the acceleration we th we saw through the end of 202 and in 2023 you know definitely took us by surprise like if I I can I can be really honest and say that from 2019 to 2022 was was pretty quiet you know we had we had happy customers but you know the demands weren't necessarily

7:16 there I think from like a practitioner perspective how fast some of these teams move has really shocked me which is like I I think what is really clear in Ai and early stage and in general and like I think the Enterprises are waking up to realize this right now is that speed is actually your number one advantage things are moving so fast that you know there is if you're not if you're not competing on speed you're going to be left behind

7:46 and so there's actually a lot of propensity to buy versus build I'd say which is like people people are happy to you know to buy technology where I say in the past people were pretty hesitant to buy infrastructure we we talk with companies all the time where where we think that oh you know they probably have something built out where you know they're a lot less sophisticated than you think and they're handling a lot more scale so they need

8:12 to be able to have the infrastructure to support that so I think that's that's probably one of the larger things that we've be surprised by which is you know how fast people need to move to be relevant I think the other thing which is just like how GPU needs have evolved going from you know at the end of 2022 most of our customers were using T and atgs you know after that it changed to a100 now it's going to

8:38 h100s like the compute needs aren necessarily going down they're really going up especially as these Services scale up you guys were just highlighted for leading on the independent artificial analysis. benchmarks for highest throughput and lowest latency serving congrats can you give us some intuition for what is driving that like you know what makes inference harder to hard to run fast inference is you know I think there's like multiple things which are quite Difficult about inference

9:08 you know I think the there's the workflow headaches which we've talked about a bit and I could talk more about there like the scalability and reliability Bott next I think the stuff you're talking about is performance optimization which is really like how long does one generation take I think you know there's a lot lot of work and research that's being done to run Generations as fast as possible to get the maximum through put and the minimal

9:30 latency I think historically historically well it's funny when I say historically because I mean over the last six months but over the last six months over the last six months a lot of work has been done in the research Community to get basically these things to move faster stuff like speculative decoding which came out I think can't even remember when it came out sometime the last six months it started really being used for for us you know what it means is

9:58 like how much how much how well you can use the GPU how you can scale across multiple gpus and how you can honestly like be really really up to date with the latest things that are happening in open souls and research you know we've partnered with a company with with Nvidia with a company we have partnered with Nvidia and and and really like worked really closely with their L llm engine called trt LM and that's

10:29 that's actually driven a lot of the performance games that we've worked with and you know we've contributed to that we've fored that but you know the the hard thing there a lot of the optimization you're doing is pretty low level and there's no real abstraction so you either have to learn how to use open SCE very well rewrite some of these kernels by yourself you know if you look at something like opening eye like what do people complain about a lot of

10:50 the time it's speed it's speed and like that's probably you know one of the core performance advantages of Open Source is that you can get these smaller smaller smaller models to run faster and I think you know that will continue to be a massive Focus for us going forward as well on on the on the benchmarks I think you know it's pretty crazy how that's evolved as well I think you know we've gone from like state-ofthe-art being 90 tokens a second

11:19 then you know it got over 100 now it's over 200 now we're talking for some people up to 300 400 and I think that's continue that's going to continue to be like a very very important place to innovate and we we think over time it will get somewhat commoditize the performance that especially especially for language models to be honest I think you know more and more of that stuff should run locally to to some degree I think and but being

11:48 on top of it and making sure that we're we're kind of attached to the state-ofthe-art is you know a if if we're not existential risk to the business and so you know we H we have to do it how much optimization have you been seeing for other types of models so diffusion models some of the language the Texas speech models you know other areas like that I'm just sort of curious there seems like there's different types

12:10 of optimizations happening across different Foundation model types as well so I was curious you know what's what state-ofthe-art there and how you're thinking about it I don't have the metrics like on hand but we're do we're seeing we we're seeing and you know also pushing limits there as well so just yesterday for example you know we were able to get Wisper running so there's whisper there's faster whisper and there whisper on trt which again is an Nvidia thing I I think what what we

12:36 are seeing is that there's more and more focus on bringing these experiences to real time as possible and so like you know one of our customers is a company could gamma AI storytelling storytelling software they use staple Fusion image models to generate images you know getting that not waiting four or five seconds there and still having high quality images is again core to their business and making it very very fast and and easy to use and I think you

13:04 know we are seeing like definitely from a customer requirement perspective we're seeing that I think you know have we been able to juice as much there not yet but I think we're getting that speaking of the applications you know driven by these models still being generally startlingly slow including the like really amazing capable ones from chat gbt to cognition to you know things like like Pika and mid journey and like the you know in a in a way that

13:36 consumers have not seen in many years we are waiting you know seconds for for interactions is your view of that like it'll change because the models are getting smaller it'll change because more smaller models will get more powerful people do distillation people just get better at running these things like we'll get better Hardware what's the what's the pass to like not waiting 10 seconds a generation 2 minutes a generation you know if you look at so

14:04 many of the gains of running mrw F let's take that as an example you know the step function gains come from you know a few things like first one is running on h100 and not running on a100 so I think you know it's Hardware gets better and better like you get this almost like leg up and so as as you know hopefully those prices go down and that gets more available that'll be thing hopefully the h200 comes up next and we do that I

14:32 think the next piece is around software optimization so stuff like continuous batching and dynamic batching and speculative decoding which basically makes it easier to either paralyze or batch process a bunch of things or makes each individual generation faster or off offsets it offloads it to another model there's lots of stuff in that place I think these models also just going to get smaller to be honest like I I think that's like the the really really powerful small model that does one

15:01 thing that's pretty exciting I think is it I think that's like stuff like Obama's very interesting but I I think if you saw soulcraft has this where they they basically announced that Cody's now running locally for a lot of customers I think you know that's a pretty actually exciting proposition we have to figure out where we S award like that it's not necessarily you know amazing for cloud providers like ourselves because you know we want everything to run on

15:29 but I think things do get smaller things do get more efficient and we have more distilled more powerful model sharper models to do yeah it's it's it's un the unbundling of models how how and when do customers choose to deploy their own models or open source models or fine-tuned open source models on their own infra versus use public model endpoints or like what guidance would you give people this is a general Trend we right which is you got open a you got

15:54 anthropic you have an API that works really quickly you're like that's too slow all that's too expensive or you know I I don't need something that powerful so we often see like if if customers have a lot of money they end up going paying for private deployments in Azure they still have the stun on this issue then they go to open source models when you're starting out with open source models you're going to go to a shared endpoint there's lots of

16:18 great shared endpoint providers but there are things that Ma that might matter to you which that Shar endpoint providers can't give you firstly you know you want your own slas so you don't want kind of this noisy neighbors problem that when my if me and elad have two competing apps and thead apps get slammed I don't want my my app to slow down my model calls still need to be F so in that case you want might

16:43 want might want dedicated compute but there's also like data and privacy stuff maybe you want you don't want you know your data running on the same infrastructure data going through the same infrastructure that other folks infrastructure is going to and honestly maybe even at some scale cheaper to run it yourself and they're the three things I think then when you go to larger companies comes no BR like shars don't really work for large companies not going to work they're not definitely

17:09 not going to work for Enterprises a lot of times you're going to want a you might want mrra 7B not coming off a shared endpoint provider maybe not even on a dedicated endpoint you might want it self hosted within your within your own AWS and gcp and that's kind of actually where we see a lot of the world going as well which is that especially larger customers they're going to have actually pretty good compute deals and they're going to have their own

17:33 spend you know their own credit system or spend commit with the marketplaces and actually running it on the infrastructure solves a lot of problems and has a massive cost advantage to them as well and so I think there's like three stages to it which is like you started shedding from certain point you go to Dedicated in the cloud but I think for some customers that's not enough either and you want to go into your own cloud one prediction

17:59 on the Enterprise side would be that you know if chat GPT only launched 15 16 months ago and GPT 4 just came out a year ago then most Enterprises are still in a planning cycle and they haven't really adopted AI at any real scale which means for infrastructure providers like Bas 10 it's like a huge opportunity that's about to come right already you're crusting this giant wave and the wave is about to get 10 times bigger

18:22 potentially what do you view as the timeline for really Mass scale Enterprise adoption of AI and where do you think things will be in terms of order of magnitude usage overall a year from now two years from now I'm just sort of curious like what your view of that future is I I think it's a good question I think we've been so wrong with time with time scales here but I do think like what we see it right now is

18:43 that when we go to talk to Enterprises I say a lot of them you know like for honestly what we're seeing now is that like co-pilots especially Coen stuff that actually already made its way in the Enterprise like most Enterprises we talk to like when you say when you ask them how advanced are you in the AI strategy they'll tell you well everyone uses co-pilot and that's like the big first big Fay I think the next

19:09 piece then is like using like openi or or anthropic in some in some way well I think that is going on right now and I think people are starting to experiment with that I I think like the fear I would say like I someone was just telling me that I think it was fisa e mo like tens of millions of dollars for ML investment or AI investment over the next 12 to 18 months that's kind of

19:34 frightening to me in some degree like I think it's great it's great for us you know I I I love to hear it as a business Builder here but it's also like you know to me that means that the the pressure I see coming from the above and and you know that's kind of like the I'd say the ml trap we we fell into in 2018 to 2020 when cios were buying software and it wasn't really attached to real user

19:57 value or product value and so like I would actually say that we're probably overestimating how big Enterprise will get in the next 12 to 18 months but we're underestimating where it will be in you know three to five years from now and I I think 10x is like a pretty massive underinvestment like you know like yeah like I'd say what we see like we're working with a customer right now that has four engineers and has by the

20:23 end of this year we'll have you know mid hundreds of thousands of dollars I annual spend you know that's for one use case with sub thousand users for them and that's and they're already cash flow positive as a business which is like which is insane to me in general but like I can't even imagine what these workloads are going to look like when we get to the Enterprise when you start to think about you know take a

20:48 like customer service and chat bot is probably like the number one place where people think about efficiency the volume that some of these customers is you know my brother's the head of AI is Sunrun which is like a public company with that does solar panels and I think that's like another really good example of a company where there's so much opportunity for so much opportunity for AI just to eat away processes and the volume is you know just so much

21:17 higher than what we're thinking from like a traditional business perspective with harder requirements so which even drives spending High yeah I think one reset in stance that people should have on on spending here is that traditionally like you know people were looking at software companies you got really concerned as a software investor if your cost of goods sold was affected by a lot of like data

21:48 processing basically and so you know you have this like expectation that your average SAS business at maturity might have 80% gross margins and and and I I think like you know now people understand that the the training businesses have a big upfront capex investment that you know may or may not pay off but I I think one of the things that you're pointing out is that you can actually spend a lot on the inference and the core intelligence and that

22:19 actually you know end up with a very valuable business on the other end with perhaps fewer people and so I think people have talked about that shape of company but they don't really think of it as a norm yet and you know I at least in in my portfolio we are seeing more efficiency on headcount and like a lot more compute spend and I I know for some of the base 10 customers that compute spend for

22:42 inference is actually like the you know one of the largest items on the p&l oh we were working with this customer and we basically yeah and this was like a challenge for us so we we asked for an upfront like payment for the year of compute and you know the the CTO and came back to us and said hey like like I I I appreciate what you're doing but no like this is you know our like after payroll this is our

23:11 second biggest expense for the year like we're not going to do that and I think that's you know somewhat indicative of like how much spend there is here is that you know there and it's it's probably also somewhat indicative of how big you know I personally think the market can can be once we start seeing Mass scale adoption but I do think you know I I think that's a good point Sarah which is that you know I think it's somewhat

23:40 of a reset in terms of you know I don't know if this looks like normal SAS business I actually I I know for sure it does not I and I think you know even like traditional multiples it's really really hard to think about and you know what what's crazy about it though is I think the most efficient businesses through markups and through software optimization can actually D like Drive pretty healthy margins and still have these really aggressive consumption

24:06 contract and I think that's I know I think that's rare I I I can't think of I feel like you guys see more businesses than I do I hope and then but I hope but but but like you guys will be able to like chime in on on that more like is that is that unique to this industry where else have you seen that it's been a while since I've seen so many companies ramping so quickly like

24:33 and sometimes they were fake ramps so like you know in the internet wave of the 90s it was kind of startup selling to each other and kind of bootstrapping off a venture capital and then there was giant Telecom build outs on like a fiveyear cycle that caused huge Revenue uplift and then suddenly there was a glut and things dropped dramatically here it feels like things are ramping really fast off of products that are a couple months old which

24:58 sometimes suggests that there's not defensibility and so then the question starts to become okay how do you build defensibility and what does that mean and how do things get commoditize and do they and you know so there's a couple different markets where suddenly you see three companies all go from Zer to five or Z to 10 million of Revenue in a year yeah and then you're like okay there's three of these companies and they all ramped at the same time the

25:20 same amount and so there's enormous demand but what does that mean in terms of do they cannibalize each other can three more entrance come in and do the same thing like what what is the basis for competition in that market and so I think there's a lot of that happening too which is at least for me pretty unexpected and I think it's just because we have such a big technology capability shift that suddenly you can do things

25:40 you literally couldn't do a year ago you know it's kind of amazing I think it's particularly exciting when you go and apply that to I think you L you've cut your teeth on a bunch of different Healthcare initiatives Healthcare is like a you know a really interesting place where like you look at you know Nuance if you remember Nuance Technologies like you know they they had the strangle hold over this market for years and honestly it always looked like they were kind of

26:05 struggling all along the way as well and then and then whis and Whisper comes along and you know you see like that market now of like you know note taking for for the medical thing it's insane how fast it's gr and like there clearly is real value there and then I think the question actually goes maybe this does with like assass business again when you're like okay what's the workflow and is the power in the sensibility of the workflow you're

26:30 powering yeah it's a really good point and then the other thing I think that people often forget is that many markets are not monopolies many of them are oligopolies you know that's payments with stripe and Aden and PayPal and all these things right and so it's also possible some of these Market structures are oligopoly markets and then it's possible it actually ends up being win or take all and there's some Network effect or data effect or but if

26:52 you look at some of these types of companies like healthcare to your point is a great example where it's deal driven right you have large appointments with big customers and you lock them in for multi-year deals and if you're actually able to lock down customer bases and effectively you you can fragment into an oligopoly Market more easily than if you have renewals every year right so I think also part of it is just like what's the contractual

27:12 structure of a market and people really don't talk about that kind of stuff but I think it's really fascinating to think about through the lens of what actually is a sustainable business in each each one of these categories you know beyond Healthcare it's like where where are these businesses going to get disrupted I think like Financial Services obvious one and they they've funnily enough I think they've been at the cusp of this stuff in the past I don't

27:33 actually think they were in the cusp of it as as much as you know you'd think like I you know if if you think about a lot of this the Big Data stuff like 10 years ago you know the hedge funds were all over that right they're like hey there there's Alpha there's Alpha here and and I know that you know some of them are starting to look at large models and language models and and whatnot but I do feel like they're

27:56 actually being a bit lagged in terms of their adoption of of these things it might actually be because they were so deep in the other sphere in the in like the old ammo world that's hard to kind of really quickly turn things around it's just such a different capability set that I think like old school machine learning or where you're just effectively doing regressions and just pulling out patterns and data is kind of different from some of the

28:16 generative stuff in terms of what it does and what it can do for you and one of the things I've been thinking about recently related to what you just said is what are the companies that just don't care about this and that may be a very good thing because they're defensible right in the era of AI eating everything like what can't be eaten and therefore maybe those are really good things to get involved with or to work

28:37 on because you're not threatened by a dozen different new startups that becomes really hard with like when when you start to think about some of the demos we seen couple I thought my job was safe until yesterday yes one of my partners asked me how long I thought Venture Capital was going to last in terms of like you know an agent based automation taking over because he was like all excited that he got out of software engineering

29:03 at exactly the right time before his skills became useless but I tried to give him I tried to give him a real answer which is I think on the early stage at the early stages a lot of the data doesn't exist right like you'd have to capture real world data you have increasingly meetings over Zoom but you would want to capture a lot of information about people so much of it is access and the information about

29:27 who is like leaving and like a 100x engineer and entrepreneurial and product oriented and it works with velocity like a lot of that is not collected today right so you have this big inputs and it's there's no digital Trail for it and so you have this big inputs problem I think the decisioning like if you think about like what is actually structurally predictable maybe if you have all that data the like people are the most identifiable piece but the

29:58 maybe you can maybe you have a a model that is doing continuous learning and can and learn like meta structures like a lot is talking about like oh this is a market that operates as an oligopoly where these are the core core drivers of you know differentiation and these the dimensions of competition and such but I think that that feels quite hard when you're investing in a technology landscape that is always changing right and so like you're

30:26 kind of always out of distribution and you don't have the date on the people and like I don't know how you make decisions on whether products are any good because you'd have to have all the customer point of view or you'd have have taste maybe models would have taste so I think Sarah is saying that her job is defensible which I think is what everybody says no no no not my job I am I'm just saying this morning I gave it a

30:49 good think and I was like should we hire people to go work on this and I was like nah that doesn't feel like a tenable problem this year but I I commit to if it is feasible like we're g to be first but I just I feel like it'll be like you know another six months or so so that means it'll be next month you're given what it's actually in about 20 minutes it was a company launching wait

31:10 I gotta ask you one more question because like you you know you're working closely with Nvidia you work with the hardware providers people are really interested in this topic now of like I I mean generally like do you believe in Hardware heterogeneity right there are some strong opinions on this from you you know data breaks and others here and do you like you know do do you still see the same Supply demand Dynamics around GPU shortage from

31:41 your customers that you did maybe beginning of last year I think the chips have just changed so like I think there is like before there was a shortage of it just felt like of everything like if you wanted anything except a T4 you could not get it that that was that was hot I think right now there there's two things we're seeing is that one like most of the like it is now possible for us to

32:05 acquire compute for us to acquire compute pretty quickly we have big span though and so that that is like that everything should be kind of conditioned on that is that we are you know we're making long-term commitments with providers so we're able to that gives us negotiating power I think customers are still struggling with availability for the most premium chips and I and I think you know whether that's h100s or 8100s I think even when there is availability you're you're

32:35 often times looking for like three to six weeks of negotiating with Cloud providers and then your rep calling in favors in exchange for something or the other like the amount of times that with the cloud provided we have escalated conversations just to get people moving faster is unreal and so I think customers are still running into it I do think it's getting better and I do think that it will you know seems like it it will go away I think the

33:02 heterogenity argument around different things I mean it's probably a good thing right like if there is more than one one provider of chips but but that that being said I I personally think that it's pretty overstated how easy it is to to run something that looks like Kudo or Cuda in some form on an AMD chip seems Seems like a challenge to me I know a lot of people there are there are people who who believe that they've got

33:30 it done I think the amount of time we we spend debugging bad notes we get in a place where that you have a lot of information about existing infrastructure that's that's challenging as it is I can't imagine what that'll be on on these ships that are untested and so like I think over time yes I hope so that' be great I think shortterm you know it's really hot for me to see how we make investments Beyond in

33:58 especially when there's a c where there's like customer a crunch on the other side from customers who are like hey we need this now and I don't think I don't think we want to the other thing we don't want to do is that you know what Bas 10 doesn't give you it doesn't give you Ro access to gpus it's conditioned on this inference problem today that you know like you if I gave you a GPU to use on base 10 there's not

34:20 that much you could do on it except inference just just in terms of the access control that we give to you that being said like you know our customers do end up you know fiddling with NVIDIA drivers they do end up you know like installing new versions of pytorch and and you know have like custom Docker images and I think running those things on things that aren't Nvidia and especially you know doing them in in an abstracted way like

34:47 I think it'd be easier if I was building a service on top of these MD chips and saying take this service but you know our customers do interface with those with the GPS in some way and I think building an abstracted service where this heterogeneity it's like that just sounds very very challenging I'm sure there are people much smarter than me who could figure it out but I think for us like I think that would just add

35:07 a lot of complexity and really slow down how fast we can move but I hope I think you know there it is a good world when there is more than one option how do you see customers thinking about build versus buy and how do you think that's going to be evolving over time speed is the only thing that matters in this market and and and what what this means is that you know if if you if you spend

35:27 time fiddling around with your infrastructure and your service goes down when you launch it I think that actually hurts the end user experience a lot and it's something you just don't want to mess around with and we see this from customers you know even customers where AI is their core core thing is that they are understanding that what what they what is proprietary to them is models data and workflow what is repeated for them is infrastructure

35:56 and I think like the smarter like the amount of times that we have seen in the last 12 months that we're going to build this ourselves which is very much like how infrastructure Engineers were thinking a decade ago only to come back three months later it's like I have a you know Docker dumpster fire somewhere is like I you know we can't count like it's our super qualifier like have you built it yourself is how we

36:20 know someone's going to be a great based in Customer because they they empathize with the pain and they know that this is going to allow them to move a lot we had we had a company with a four person AI infrastructure team that been building this for two years migrate all their workloads over the base 10 and 36 hours and you know I think that is a pretty amazing case study for them which is like holy crap we can now take

36:46 these four engineers and focus on what is actually our competitive differentiate Advantage and the way we think about our business is not you know we don't need to scoop everything something off every single customer either like we offer options where you can run this in your own in your own environment and you know pay us a license fee and you know like I think there it is very very cost effective the way that a honestly other providers

37:11 are doing this like I think it's crazy to be honest to try to build this yourself especially at the scale that some of these customers are operating at I was I was looking at one of our one of like the customers that we chatted with this morning who was who was tinkering tinkering around they said they were doing a billion tokens a day this is a you know a a Sixers chat chatbot company that has a billion tokens a day going through

37:35 them like to to build the infrastructure that supports that with the elasticity reliability and and the performance and then build the product experience around that that's impossible for a six person team and like I think you should try to take away the things that you know other people can do just as well if not better and that's my take and I think that's kind of what I see the market covering around to as well which is that speed is a competitive Advantage

38:01 let's let's spend out like we can spend our way we can buy that competitive Advantage without a long build cycle this an awesome conversation thanks for doing it toan thanks for joining thanks Sarah thanks a lot find us on Twitter at no prior pod subscribe to our YouTube channel if you want to see our faces follow the show on Apple podcasts Spotify or wherever you listen that way you get a new episode every week and sign up for emails or find transcripts

38:26 for every episode at no- pri.com