Field Notes from Token Town — Sarah Sachs, Notion

0:05 Sarah, you have been operating at the frontier of applied AA for quite some time and there's so much to learn from your journey. Thank you for doing this. uh to kick us off uh could you please tell us all in the room about your career path thus far and how it's gotten you where you are today and then just one more question given the less traveled frontier newness of all of this tell us how notions applied a team is structured and do do your recommendations make their way to the product and engineering or does your team carry out them at all >> it's a lot of questions >> thank you Um, so I'm Sarah. Um, I lead a lot of our AI teams at Notion. You have to excuse me, I'm losing my voice. Um, yeah, so I've been at Notion for about two years. Um, I've always worked in machine learning. I went to Brown University, um, as an undergrad. Um, did research in computer vision there as well as at UC Berkeley. Um, then I was at Google, um, on the location history machine learning teams. Um, did some startup stints. um worked at Robin Hood um for about three years um led the machine learning team there um and then came to notion because I felt really strongly that the idea of having a system of record for enterprise work how to utilize AI for that was really exciting um and that's kind of the journey um the journey that we've been on you'll have to remind me your other question >> no the other question is uh we want to understand how the notions play a team is structured and to follow on on that uh typically whatever the recommendation that you Does it make its way to the product and engineering or does your team carry out the work?

1:46 >> I see. Yeah. So I mean all of notion is an AI product at this point. Um a very large percentage of data edited in notion is edited by agents or by agentic outputs MCPS or API. almost half of all searches on notion um and AI enabled workspaces are done by agents, right? And so humans and agents are our customers and notion is meant to be the place where um enterprise work happens and um the best place for that collaboration and that doesn't necessarily need to be by humans. Um so as a result when you ask how the team is structured you know there are people my teams particularly who specialize in our agent harness model selection usage based pricing strategies um tool orchestration how we structure our tools but everyone at notion is contributing in some sense because agents are our customers just as much as humans are.

2:45 >> Um so your next question was do we influence the product? >> That's right. Yes. Yeah. Yeah. I mean, um I think uh and actually so Swix was I was on Swix's we we had a long chat with our co-founder Simon admissions co-founder um with Swix and Allesio that was released last night and we talked about this a little bit where I firmly believe that um you want to empower everyone in your company to understand the user journey that you're aiming for and you need to have clarity of product vision.

3:19 Um it's very easy to write code now. Um it's about what you build. Um and as a result I most of our great ideas come from individuals um on the team. Uh we're lucky because Notion employees use Notion pretty much more than anyone else. There are very few companies that use Notion more than Notion. There are some and we love them um but very few. And so a lot of our best products come from friction that we have in how we work. Um and uh really like we pretty much um the way that any idea is developed is by putting it in development almost immediately um among our internal use cases at notion. I saw how notion is using notion when Ivan and actually gave me and Chris a demo on notion. Yeah, >> I mean it's it's it's the easiest sales thing where like someone's like how are you using custom agents and I'm like why don't I just show you how I run this one workstream and I'll show you the Slack channel and I'll show you the meeting notes that are generated and how the agent files the tasks and you know it listens in Slack to file new tasks and it's it's very easy for us to demo because it's how it's how we work. Um, and you know, when I was at Robin Hood, it was a lot harder to think about automating option support flows without trading options, right? So, um, it's definitely really fun as a product and an AI engineer to work on something you use.

4:36 >> Lovely. Getting into details, uh, you open your excellent recent essay that prompted this chat. Thank you. and describing a model upgrade where per token price stays the same but the cost per task grows 3x because output tokens alone and the rate card doesn't really exactly lie it just hides a unit that matters at the high level what does it look like doing this as often as consistency you do how do you really measure cost quality and latency at the task level shifting from the token to the specific capability that you have to execute on It's a great question. I think rate cards are sales instruments.

5:17 >> Um, and you know, everyone's objective is to be as cheap as possible and, you know, max out a benchmark, right? Um, that doesn't mean that you should adopt it or believe it. Um, I mean, obviously the rates you should believe, they're contractual, but um, so for instance, let's use web search. Let's not just think about compute. Let's think about a web search API. Um you can our harness can call on web search API inside notion and you might think that the fastest and cheapest web search API is the best web search API.

5:52 Um but part of these um subprocessors they're actually figuring out how to search, right? And if you do fewer searches, your trajectory is cheaper. So even if something is twice as fast and twice as expensive, if it does the quarter the number of searches, you're still cheaper on the more expensive platform. Um and that's also true with thinking. Um you know for instance um opus um is a very very capable fantastic model. It's really bad at correcting itself. Um if you tell it I actually think you did that wrong. Please improve. It's a lot more stubborn. And so if you think about um the type of model you'd want to use to build an agent, right? Um maybe that's not um the right configured model that you'd want to do for that. And so um to answer your question about how we choose um one of our primary investments is in our evaluation team. So we have a team of data scientists um software engineers and something that we call model behavior engineers that work really hard to create a representative data set of what our agents need to be good at >> and then from that benchmark costs for different use cases and different use cases have different cost needs.

7:01 Onboarding we're okay spending more to make it as fast as possible. Like when you start notion AI, we will pay more so that it's snappy, right? However, if you're running an asynchronous task, we care much more about its ability to self-heal on failure than we do about latency. And if we're using usage based pricing, we want to make sure that you feel like you're getting the value on your dollar. If it's a free onboarding trial, we want to make sure that we're getting the value on the dollar. And we might value different customers differently, right? Um and so model selection is an art um that is necessary um in such a broad landscape.

7:36 >> Lovely. Just follow up on that that notion's fascinating automodel features almost 75% of the traffic and just want to understand a little bit more on what are the invariants you enforce so downstream product behavior stays safe stable as the model gets switched to the front. >> Yeah. Um this is actually difficult. Um, it's also just different. Okay, so the question as I understand it is how do you have a model feature, an auto feature that can change models? There's different levels of complexity to that.

8:06 There's like selecting the right auto model. Um, and that's hard because especially when it's um right now a lot of people presume it's a very cheap model that's not working well and that's just bad marketing from Notion. Auto is um really the best model that we think you need. Um it's not a cheap model. um underneath unfortunately um because the cheap models don't do as well as what we think is the minimum level of care that we want to give our customers interfacing with our agents um upgrading them for um like our corner chat our single agent run isn't that hard um because we have a set of evals that we know we want to work well what's difficult is when people configure custom agents that are running asynchronously in the background and they are configured to work with certain performance um that can be very hard for things um like switching between models because transcripts don't port over very well from each other. So we have to do a lot of engineering to make ourselves interoperable between models. Um and those abstractions can be quite difficult as model providers want more and more customization in their model to prevent you from being able to switch between models. Um, one of Notion's selling points is that we're the Switzerland of AI models, which is to say that a lot of our customers don't want to have to choose or make a bet on open AI or anthropic or open source, right? So, one of our core competencies is being able to give them the frontier today, even if it wasn't the frontier yesterday. And so, a core investment is that tech technology that can seamlessly switch between them even within a single chat.

9:42 >> Love it. Love it. One of the thing that we understood is uh that you implemented a notion scale that model switches every two to three weeks. Uh like to understand how your team designed the guard guardrails to degrade gracefully when the model switches. >> Well, hopefully we never degrade when the model switches should only be an upgrade. Um the reason that we probably have that snippet um you know we switch models every two to three weeks is is multiffold. one, new models are being released way faster than they used to be, right? So, it's not really um I mean, it's nice that we can update with the newest models, but that has a lot more to do with the pace of research than notion. Um I think what's more interesting is that um oftentimes when we switch models, it's not necessarily just like updating set um you or upgrading GBT 52 to 54, right? That's not that's not like what we're doing when we say we're switching models.

10:39 because we've developed a new harness or new implementation or there specific tools that we think work better with a different model and we're constantly evaluating. We have you know over 300 tools um that our agent just for notion alone is referring to and you know the whole company is updating and working on the different um implementations and structures of those tools. So um it's a configuration change. Yeah, >> of course we got to talk about pricing.

11:09 actually was here a few minutes ago and then Christian and I actually were discussing a pricing and then he mentioned >> uh and notions pricing also has changed from seat to uh even including everything as part of the seedbased model that we have been uh using for quite some time and now recently shifted and a lot of usage based pricing and a said you pay for what you work the what work you get done uh usage based pricing for agents is anything but simple because cost of a task is a known ahead head until it gets completed, right? And what principles guided the pricing shift uh in terms of the product and measurement capabilities that made it possible to charge in a way that customers want?

11:52 >> Yeah, I mean the shift is um primarily our launch of our asynchronous custom agent product. Um the idea is that an IT administrator or an evangelist in the company company can create a custom agent that lives in your slack channels that lives in linear that lives in notion and is um autohealing your workspace for you in a way that you configured and so um that value doesn't grow by the number of seats that value grows by how much that agent is doing right um so in and of itself it just kind of makes more sense as a business model but even more concretely different users adopt AI guide to a different amount and it was becoming um we were nerfing ourselves like there were a lot of capabilities for instance um our autofill capability if those of you are not oceanion users you can have an oceanion database you can have AI autofill that database you can imagine for some databases um you really don't want um an entire agent harness running on opus and doing searches to update a database feature right but then we had some amazing power users and enterprises that were asking for more enhanced autofill and it was Absolutely, we can provide that to you. But um for some workspaces, you know, if we're charging $20 a seat, this is going to cost a lot more. And we were in a place where we had power enterprises saying, "Can we please pay for this, right?" And we couldn't afford to serve it to everyone, but if people want something and they're willing to pay for it as a business, we should provide it to them. Um so it was kind of an obvious decision.

13:21 >> Lovely. Lovely. Let's get to the events uh that we discussed a bit. Go uh Sarah uh notion signal is uniquely valuable to labs uh because notion is fundamentally an horizontal product and uh definitely the foundation model labs and products used across every knowledge work task imaginable as horizontal as it gets. So want to understand a little bit about how notion makes a val scorecards different from a lab that can generate from its own usage data is one thing if you can quickly touch upon >> of course um so what makes notion unique is that we aren't targeting unlike typical SAS models where you're targeting a vertical coding legal healthcare um we're trying to target a specific horizontal space of tool making for customers um and collaboration and collaboration ation and enterprise collaboration is kind of a broad category. Um and we have a lot of different usage like we have customers a lot of customers in technology. We have a ton of customers um in APAC regions, right? Um we have different use cases, healthcare customers, legal customers, um startups, super super large companies, the Pokemon company, right?

14:36 Um we have a variety of users and as a result um you know for the data that we can gather, we can create benchmarks in a way that's really representative of general enterprise knowledge work. Um and that's very distinct um for a provider of um for a company of our size um to have the amount of data that we have. And I think also um there's an element of like commitment to rigor and frontier capabilities. So there was a point where we had saturated our evals.

15:04 Every time a new model came out we would be like okay it's like 100% pass rate. It's better than the last one. That's not helpful anymore. Um I mean it's helpful in that we don't regress. So you know that's why I kind of laughed. Hopefully every model upgrade we're not catching regressions, but >> it's also important for us to understand what do we want to be good at? Um what is the frontier of what we want to do?

15:27 And by the way, this eval set is the same eval set that if we ever were to fine-tune a model, if we ever were to say what does not need to be uniquely good at, this representative eval set should be harness agnostic and should be the same thing that we would use to train or evaluate that. Right? It's a core competency that every company needs to own to differentiate themselves in our perspective. Um, and so it's heavily invested in and in order to scale your company, it's very important in the same way unit test and CI have been important in software engineering for a very long time.

16:02 >> Love and just interested in the view about uh as I increase in applied a uh company shipping. Can you share your views on how some of these vertical models like Javi cursor versus the horizontal models the evolves that's done under notion? >> What is the question particularly? So the vertical models that are evolving in uh cursor and Harvey comparing those to the evolves that happen at the horizontal ones at notion. >> The evals how are the evals different?

16:30 >> Yes, >> of course. Um those eval are targeting particular competencies and capabilities when it comes to subject matter expertise. >> Um enterprise work like what does notion need to be good at? it needs to understand um how to orchestrate lots of different tools and think about how to do work in processes. So like imagine that you're onboarding a new chief of staff to a company that's trying to streamline communications and streamline the process of a launch calendar, right?

17:00 Um the competencies of that are understanding conflicting information, understanding how to do um enterprise search very well, understanding the relevancy of different enterprise search sources, uh being able to use notion very well obviously for the notion agent and being able to do like long horizon thinking on a variety of verticals um as opposed to a particular compens competency on a specific vertical. >> Last question. So uh there's a tweet on my we have been discussing internally uh I I'll just read out the tweet. Uh Mitos appears to be the first class of models trained at scale on blackwells.

17:41 Um then will be Vera Rubin's uh pre-train pre-training isn't saturated reinforced learning works and there is so much computing coming online soon. just uh when we reread your essay uh we felt like it co-authored on along three lines. One is definitely a thoughtful practitioners with a key eye on scaling customer value uh technologist, economist and strategist. So how are you thinking about next phase of a adoption and what does it mean for your work?

18:13 >> Yeah. Um I mean I think kind of going back to where we started. I think the way everyone works is different and so notion's commitment is to be the best collaborative place for where people go to do their work and we're changing people to be inclusive of agents because people are doing their work through agents. So um being able to orchestrate agents directly from notion right we just co-launched with enthropic on their manage agent product being able to in your task database assign a status like claude please try a first try um at this bug looking at this purity right um and so when we think about like how capabilities grow um there are primitives that notion firmly believes are important so regardless of who or what is collaborating the system of record of notion is built on top of primitives. It's built on top of the concept of a transcription or a meeting note, a database, um a page, and those pages can have elements that are nested in each other and can be shared. And those primitives we believe are fundamental collaboration points regardless of who's collaborating. Um if you think about on your coding agent, if you look at your skills file, that's markdown. Like notion is right for markdown, right? But how do you share your individual coding agent workflow, right? Um there's this kind of saying um which is a little bit ironic now given I guess reliability but um you know cloud code is really successful because of GitHub. It's successful because of version controlling and collaboration right. Um we believe that we can bring that to enterprise work in general as enterprise work is being done by agents.

19:48 Um because there's a lot of enterprise thirst for how do we adopt agents like we really um see that and there's the Fortune 500 which have a bunch of Palunteer um consultants that are helping them or they're fine-tuning me strong models on their own workspace and they're investing hundreds of millions of dollars into this. But what about the Fortune 5 million who maybe don't even have a software engineering team, right? um who want to understand how to adopt these AI workflows but they need to be HIPPA compliant they need permissioned access to data their data is sprawled in different places uh they want to control cost right um and that's where we believe if that is where work is moving um we've been building for this kind of the whole time um and we're thinking about how to structure um what we build to work in that space and those are the users that we're targeting and trying to make and serve accessible which is why cost is important right um because not everyone wants to pay top dollar reasoning for everything they do. Um and we want to give them the opportunity to select and choose. So we have to build for the frontier and we're committed to that because there are people that token max on notion and we want them to token max on notion and we want to make them billionaires um you know through Swix's talk. By the way, that slide about the trillion tokeners, I was on it and I'm not a billionaire. So that's why you guys heard me laugh out loud. I was not yet. I was like name number seven and hasn't hit the Wells Fargo yet. But um you know we want to make everyone a token maxer. We do. And at the same time we have customers that don't want everyone in their company to be a token maxer for email triage, right? Like you don't need to token max or email triage necessarily. And we all still have budgets, right? And the money isn't coming out of thin air. A lot of it's being funneled to Frontier Labs because they want you to pay by token. and tokens aren't necessarily the best way to process everything. Some things should happen on CPU. The GPU isn't for everything. And you know, respectfully, Philip, because inference engineering on GPUs is important, but GPUs aren't for everything. Can we make it agree, right?

21:56 And so, how do we configure all of these things together so that this AI psychosis can be productive um and actually create work and value? >> Lovely. Lovely. Yeah. >> Any questions from anybody? >> Models from the frontier labs. >> Yeah. >> What is notion's thought process or approach towards this? >> Yeah. I think right now um so we're model agnostic in that there's no point in time in which notion says we're an open I we're an open AI house. We're a base 10 house. Like you'll never see that from us. And that's an important selling point to our customers because that horse race changes more frequently than contracts, right? and no one wants to make themselves obsolete contractually. Um, in terms of our approach to open source, um, right now my observation is that models are being developed for more and more reasoning capacity and the price on that is increasing. Um, and for that high level of reasoning capacity, that price is probably appropriate. However, models that were um, fantastic at a lot of tasks are being deprecated. Um, and so all that's left from first-party models are absolute frontier capabilities at premium rates.

23:14 There are cheaper models that these labs provide. Um, that tends to be cheaper because of latency um or it tends to be cheaper because of um quality. But there's a no man's land in the middle. There's a no man's land of where the Frontier Labs were six months ago, right? like where's our GPT5 open source model, right? Um that's where our customers like if you think about a triangle of cost, latency, and quality. Um you can put lots of models everywhere in the triangle. Um but there's a whole area when it comes to a certain amount of capability um and speed of where we were six months ago that saturated a lot of knowledge work.

23:58 And I don't think that should be controversial to say not everything needs to be that hard, right? And that whole area of knowledge work has been saturated, but first party Frontier Labs are trying to charge you for tomorrow's hardest tasks um on what you were doing yesterday. And if what you did yesterday didn't work with yesterday's model, that's fantastic. And we're all developing more because we have more capable models. But also, what you were doing yesterday, you might want to keep at yesterday's price because it didn't get better tomorrow, right? It can't get better. We've saturated what knowledge can do, what knowledge models need to do. Um, and that's where I really believe in open source and I really believe that more people need to be embracing it because I think this market is really inefficient right now. I think that um there's like an igopoly of three to maybe four people and they just need to be a little bit cheaper than the person who's slightly better than them and they'll win market share. And by the way, that is also true with inference, right? I also think that inference models are being charged um in a way where they're trying to have you commit to provision throughput or commit to long amounts of traffic because they also know that if a frontier lab tomorrow comes out with a reasoning model that's cheaper, they also might lose traffic. And so everybody is trying to structure you away from looking at what's best today because they're trying to structure you to commit to what was best when you signed your contract. But your customers shouldn't have to be privy to that. And I think open source is a release valve to an inefficient market. But it is not the only answer.

25:30 >> Hi Sarah. Uh, one thing I really love about using Notion is all of the different modalities. It's not just text. I have structured data. I have images and audio and video and all that sort of stuff. How do you think about native multimodality when you're building agents? Yeah, I think right now to be honest it's underinvested um for instance like we actually use base 10 um to stream meeting notes. So when you do transcription, our primary subprocessor is base 10. When you trigger an agent from your meeting notes, we're triggering it based on the tokens that have been transcribed, not on the MP3 files, right? Um so we haven't explored a lot of that modality in terms of like vision or video.

26:15 Obviously like we index charts and graphs um so that you can search over them, but that's pretty much the level of maturity we have today and that's an area of an investment. But I think that your question about modality isn't just like pixels versus tokens, but about the different types of structured data. Um, and I think that that's where every company has to really think about um building their product in a way that agents can use them well, right? Um, so for instance, notion is investing very heavily in our MCP and our CLI. Um because um we will rebuild how notion works for users underneath so that it works better for agents because of what I mentioned earlier because traffic is coming from agents more and more. And so when I think about our modalities, part of it is like we need to invest in making them speak really well to agents.

27:08 A year and a half ago, all of our agentic work happened in this really complicated XML that was totally lossless and how it converted into notion blocks and notion representations, right? But then we had to teach these models how to speak really particular notion markdown. And at the end of the day, it was nerfing the models and making them overfocus on that. What if we just had a slightly less expressive version of regular markdown that the models knew how to speak and then had some deterministic code that transferred it well and then maybe it can't do everything a notion block can do but we're still able to bring our customers frontier capabilities. Um the second point on that which you know I'll say facitiously used to be kind of our biggest disadvantage is that notion is like a blank page when you look at it and can be really daunting to a lot of users because it's just primitives right and it's like okay I see these people on Tik Tok or YouTube that are like here's how I hacked my life with the best notion dashboard ever and then you see a blank page and you're like I don't know how to you know slashcomand my way into that right and that was a huge growth issue for us two years ago. However, having a very unopinionated UX or rather strong opinions about having a simple UX and just primitives let us leaprog into the agentic tool calling world because those primitives are exactly what AI is building on top of. There's no complexity of well the user sees this button right or it's or it's um our CSS is so beautiful and our React components do it this way. We have way less of that than a lot of our competitors because our primitives are simply that they are primitives. Even the way the agent works like its memory is docs and databases, right? It's using itself and enhancing itself to work well. And we believe that if we invest really well in those primitives and the ability to edit those primitives well, um the agent can be unleashed and you have to tell it to do less.

29:02 >> This is really great fantastic speech. Uh so this is Helena and I'm from Z. AAI the AI lab behind the GM 5.1 it's kind of it's an open source model. Um so my question is um as a foundation model company we actually really pursue and achieve the AGI but we're really saying the definition of AGI should belong to all the users and as we say notion is really great fantastic AI applications.

29:33 So maybe you guys could be know more about what users really want. So uh I'm really going to ask what is your perspective to define what is AGI and how we're going to achieve the AGI at the final stage. That would be an awesome sound bite if I could give it to you. Um the question is like what's our version of when we achieve AGI or not? Um, we are not in the business of committing to saying that we delivered AGI. We are in the business of delivering work and value to our customers. And most of our customers do not care if we've hit a definition of AGI or not. They care if their ARR is increasing, if their SAS sprawl is decreasing, and their seatbased pricing makes sense to them.

30:21 Right? And so I actually find a lot of that conversation slightly distracting for applied AI companies. It's very flashy. It's very interesting and it deres no user value. Um what I will say is that we are at a place where we're no longer treating our AI like an assistant but rather a co-orker that owns entire processes. And I think that that is a leap that is dramatic. Um and in a sense um we are treating it like a co-orker with full knowledge, right? And I and I think um there's no point in which we'll say we're done and it's achieved. Um because like I said, maybe AGI is being able to do that really cheaply. I mean, obviously that's not the definition of AGI, but like that might be what's best to our customers for the work that they're doing today. And and I think those are the those are the hard questions that we're focused on answering. And we leave it to the Frontier Labs to um decide when they get to send the tweet that they achieved AGI. It won't be coming from us.

31:19 >> All right. It's it's just a very simple question. So just wondering because notion is used by people and agents. What are the categories of agents that are most common >> common commonly using uh notion? >> So there's what's most common today and what we want to be really good at and those are slightly different because part of building frontier AI applications is teaching your users what's possible, right? Um I will say there are a couple of components where we believe we need to do very well. One of them has been and will always be enterprise search. Um, that was, you know, one of our first AI investments was notion Q&A. We continue to staff a team fully on agent trajectories on building context. And I firmly believe in order to do an action well, you need to find the right context. So that will always be true. However, that's not the only work that we do. So for instance, a key use case of the custom agents that we ship today is bug triage. You'd be surprised how often issues come up um and there's like three PMs or PGM just dedicated to figuring out who it belongs to and what the context of that issue is. Um that's been a huge adoption use case. So we have a lot of customers um where you can just like drop something into a Slack channel, a custom agent is listening, it looks at its own constitution or teams directory, forwards that issue to that team and files a task to that team, right? Um, and if that team says, "Actually, this belongs to the mobile team." It updates itself next time. Next thing you know, your company's not dropping any bugs. It doesn't matter where they show up. Um, another one um that I really like um has to do with how we operate our teams and how we communicate progress. So, if anyone's an engineering leader here, they know like writing updates about what your project has done is really painful and really important depending if you're reading or writing them, right? Um and and that's that's not just in engineering, that's everywhere. Um and so one of the like hero use cases um that we certainly have one of our highest adoption of is the idea of being able to form a custom agent that understands a project or a concept and looks everywhere and can write an update. um the one that I have configured will also when I write my updates for my CTO weekly um if there's like OpenAI flex tier what's the status of shipping that and it can't figure that out in the task database in Slack and meeting notes it just um pings the person that it thinks is most familiar asks it for an update in Slack and then updates itself and then sends me a notification after three hours when people have or haven't responded and it's done the most research it needed to do right um these are processes it's not one person's particular job But these are processes that companies are saving hundreds of hours on a day by automating agents to do. Yeah. Um just the question in case everyone else didn't hear it's like you know a month ago I write I wrote a rant on X um saying like welcome to token town. It's the wild west. You know this is how the market is working.

34:13 If you're not one of these three companies here's how to navigate it. Right? because I think people kind of all assume that we all, you know, there's other people and there's a lot of industry working around this, right? And not everyone is negotiating in the trillions and trillions of tokens a month. Um, and so the question was like what's the foresight of what we're not talking about next? I think the next thing is that it's not just models. You know, there's the trifecta of quality, cost, and latency, but AI products have other trifectas. There's autonomy, user intervention, and ease of use, right? H how do you how do you create a secure Oh, sorry. It's um autonomy, user intervention, um access to external data and exfiltration, um um and ease of use, right? That's another trifecta that's very difficult to manage. Data exfiltration and security. Um, you know, that's why you see people freaking out about the idea of their employees using OpenClaw, right? Um, because there's a whole element about actually building AI systems for real work. Um, that is a lot more than what the model's capable of.

35:21 It's what the model has access to, how it's versioned, um, how it's interpreted by different users, how it handles conflicting instructions. Um, and I think that people that are starting to use these models and bring them into production are learning a lot of these lessons the hard way. Um, and it's that's not just about um like reasoning capabilities, right? Um, it's about how you orchestrate how these models talk to each other and and act safely. And I think that we're going to see the idea of an AI security engineer and data exfiltration. Um, we're hiring for that.

35:57 if you know anyone or you want that job. Um, but that is a field and it's an incredibly incredibly important field, much more important than cost because a security incident kills your company. A bad quarter may or may not, right? And I think that I I'm afraid as a user, I'm afraid um that people are going to learn that the hard way and they're underinvesting in some of those problems because they're just thinking about capability.

36:27 Thank you Sah. Thank you Raman. Thank you.