Agents and Infrastructure, Sam Lambert | Compile 26

0:00 Hi everyone, my name is Sam. I'm the CEO of Planet Scale. Uh Planet Scale is a cloud database that hosts some of the world's largest uh databases for some of the world's most successful uh tech products. We have three main services. Vites which is sharded my SQL, Niki which is sharded Postgres and then Planet Scale Postgres which is the world's fastest uh Postgres host running on planet scale metal. Infrastructure has been my obsession for my entire career. I have always loved supporting products that change the world and do things for incredible people. Um, Curser is a customer of Planet Scale and I'm incredibly proud of this that they run very very large shared databases on planet scale. Um, they're changing the world and and being a part of that is is really really incredible. My talk is about how agents have changed the role of infrastructure and how that's going to continually evolve and how infrastructure is going to have to play a role of giving in concise instruction and insights to agents so they can manipulate the infrastructure without doing any damage, being safe and actually just getting their work done in a in a living environment. And if if any damage does happen, anything does go wrong, um it can be undone. So um my demos are using agents. This is a very special conference celebrating a very special product and I thought you know I wanted this talk to be really different and so I'm doing it almost completely alongside agents. Every slide you're seeing as well was developed in cursor and um it's important to remember that agents are nondeterministic.

1:36 Every time I've practiced this demo it's done something different but it gets to the right place. So we've also we're building against a system that is designed to be extremely safe. Plan scale is is you know a very high availability service. Um so it might just block what we're doing and tell the agents to stop and uh you know that'll be something at least that we'll see. Um, we've heard loads of horror stories about databases and and agents interacting with databases and we're going to now see another side of that and we're going to, you know, go through that now and we're going to kick off a demo because these take a little while and uh I will show you around the the product.

2:12 So, if you scan the QR code a moment ago, you'll see SFM Mart, which is my store for uh founders. Uh it's very slow, but we sell great merchandise for you know, people moving to San Francisco and starting their companies. Uh but you know, performance is really starting to slow down uh the amount of orders that I can receive on my site. You can see here we've got Graphfana set up and this is a live site that's hosted on Cloudflare.

2:41 You can see our order rate is okay. We're doing we're doing okay, but our queries are really slow. Uh the average is like nearly four seconds and the max is around five seconds. So this is not good. like we know people abandon their checkouts if uh if their um product is too slow. This uh database is hosted on plan scale on just a simple three node cluster and then we have a little viewer that's going to show us what our agents get up to. So we want to make our database faster. So we're going to go over to cursor and start our first demo.

3:12 So I'm just asking the agents to um you know see if plan scale has any recommendations and then act on those recommendations and kind of optimize the codebase and see what happens. So I'm going to start that now and it's going to start planning um its moves. Like I said, we're going to be able to see performance changes and differences here. our agents view. Once the agents start going, uh we'll we'll get to see kind of as they race along and start doing their work and we'll see what they get up to because we want to show the safety features and the various things that go on in a modern infrastructure platform. We have added a few prompts to get the agents to do bad things and we will hopefully see them when they start doing it. So, you can see it's starting to list out schema changes. It's getting ready to spawn agents. We'll see them show up any second and the site is still really slow. Everything's still going along. We have load and orders being generated elsewhere. And let's see once this gets going, it's verifying the status. See, nondeterministic. You just have to wait and see when they get ready to do their work. So, it's editing away. It's getting going.

4:21 It's reading the plant scale API. And the master agent will soon start to farm work out to some smaller agents. Everything is using composer 2.5. Pretty much everything we built here did. Works extremely quickly. Allows us to do this parallelism. And here we go. The sub agents have started. Thank goodness. Right. Here's our agents doing our work. Our friendly little robots over here going through these various stages. So they're discovering various things that Planet Scales had for them. Planet Scale generates insights and you can see here that it recommends various changes to your database. We've designed this for people, but now it works extremely well for agents and we bring in a lot of context. We try and explain exactly what's going on. We show exactly what queries need to be edited and we actually suggest schema changes as well. We have our own model that actually goes and tests index recommendations for performance improvements and then generates these kind of recommendations. We recommend removing indexes uh big like if you know you're running out of integer space just genuine things that you need to work on as your database that traditionally database engineers would um do. We've automated all these things and now we have these autonomous engineers in the form of agents that can get their work done. We see that, you know, they're going through it. They're zipping along really quickly and we're moving along.

5:44 So let's go and have a look at what some of them have done. So like I said, this has gone to deploy these schema changes that add the indexes that we talked about. We have an ability to see exactly what is being changed. And this is what we call a deploy request. So Planet Scale allows you to branch your database. Branches are intended to give a production-like environment for you to make changes. And you can make schema changes on those. And when you're ready to deploy them, you create what's called a deploy request. Um, it's moving, it's making it, and it has deployed. You get to see exactly what's changed um, in the schema. And in our insights, our graphs, we even get a nice strike line that shows us what changed at that moment. Now, this is really useful considering that in this modern world, you are going to have agents making your infrastructure better over time. You want to know exactly what's happening, right? It has to, you know, there's nothing worse than getting woken up at 2:00 in the morning, production's broken. You have absolutely no idea. We correlate all change, including anomalies, errors, and issues over every graph that you can see. And then we show you exactly the queries and how often they're running. So you can see this must be the orders query. Yep, it's got much faster. And we can see that already in graphana. We have dropped the performance of that query dramatically.

7:03 And we have posted through um spark lines there as well. uh sorry like um lines to see and our site is moving along much more quickly. Let's see what our agents view is doing. Oh good. So this is one of the bad guy agents and he is stuck and this is all acting as me but we have rejected dropping a column because it would break inflight queries. So what planet scale does is it looks at all your queries that are happening. It checks the deploy request for if it's going to break any of those queries that are active in the database and just disallows you from doing that unless you obviously force it through. Um, but we have successfully stopped an agent breaking production. That's great. Good news for determinism. Fantastic. We've added missing indexes. Thank you. Thank you. Um, uh, so yeah, we're still working away. We're seeing deploy requests happen. Branches are being created and merged and our site is getting faster. We see it on Graphana.

8:01 Our order volume is a little bit more consistent. Um, our queries per second has gone up. This is a really great change. I'm really happy here. Okay, let's see what else is going on here. So, we have another agent here that is working way. And you can see these are all real, right? They've been doing their work. This is all um real stuff that's happening and getting to its goals. And we've we've rewritten a list of queries. Um, we've dropped redundant indexes. We've added missed indexes. And this is the one we're going to watch. Now, this agent is making a change that we want to break.

8:38 Let's go over to cursor and see how it's going. Um, it's running a smoke test. Okay. So, we'll give it a couple of seconds to do that. So, this agent is actually going to be allowed push something really bad into production. Um, and you're going to see that has happened. We have lost access to a column. That is very bad. um and our site won't load anymore. Part of what makes Plant Scale special is that we can roll these things back. So, we've just dropped a column and we have a new version of the table in production, but we've been writing data in an intermediate sense since then. And so, bringing this column back could be normally quite difficult, right?

9:20 In at 99% of databases, you would be restoring right now in prod. You'd be getting data still going into the main database. If it's if anything's working, and you'd have to get a version and a snapshot back uh to restore service. With Pilot Scale, we don't actually do that. We keep a a long running window that allows us to flip back the version of the schema that we just had without losing any data. So, I'm going to hit that. And it's very very fast. It takes the same amount of time. no matter how large the schema change is. So customers regularly do schema changes on 100 terabyte tables or more. I think we've seen six 700 terabyte tables getting modified using this process and it will still revert exactly the same way and we have just undone a nasty outage that could have lasted hours very very quickly. Safety allowing agents to go and do their work. So this is the agents view. They have all completed. The bad guys have been prevented and we have saved the day. We now have a faster website. Okay, thank you.

10:24 You can see here the importance of fast queries for a page load, right? The more popular your product becomes, the slower it gets. You saw see this waterfall, right? To get a page load out the door, if queries get slower and slower and slower, we really um reduce the experience of our product and nobody wants that. And now we have faster queries. Um we've been successful, but we've increased demand on our site. More demand, more load on our database. And fundamentally we need to process more queries. So we are going to shard our database. So this is our current problem right? We have three database servers.

10:57 They are a primary and two replicas to keep a copy of our orders table and our and our entire database um that backs our application. The job of these two replicas is just to keep the one copy of the data that we have in the primary. The replicas can also serve for um failing in if we lose a primary for high availability and genuinely having a copy so that if we lost the primary we wouldn't lose any data but we need more capacity. We have run out of uh the limits of vertical scaling and we need to shard our database.

11:30 Sharding for those who who don't know is like it's a very difficult thing to achieve. It basically allows you to break a single data set across many database clusters. Each one of these is a wholly encapsulated database cluster with its own primary and replicas spread across availability zones. Um so the primary and the replicas are all in different availability zones. Meaning we gain so much more resilience from sharding, right? We can lose a primary, we can lose replicas and we would only lose a subset of the data and we still have to represent as one database.

12:02 Right? The application still thinks it's connecting to one database with one data set, but actually we're now operating a distributed system with a lot more complexity and properties. In a moment, we're going to start the sharding demo. And I'm just going to talk through what the database is doing at the database level while it's sharding. And you're going to see this update in real time as we go through. We will gain the status of doing the sharding. So the first operation will be to queue this up, get this ready to go and deploy a change.

12:30 We'll run all of the plant scale pre-fly checks that need to be run and then we'll go through this stage of copying data. I could give a two-hour talk on just sharding and just this process of copying from one to the other. We're taking one data set on one database cluster, distributed it across 16 shards and keeping it in sync while doing it. We're also going to replicate. It's, you know, new data will continue to come in.

12:51 We can't just restore a backup and be good. We would lose everything that was intermediate from that backup. We need to replicate. We need to constantly replicate. And then we need to verify. We will check sum every single row in both databases and make sure they map so that we can we know that we're not going to lose a single bite of data. Then we're going to switch the replicas over. Moving read queries in an application is like less risky to go and do that. And Planet Scale will handle the uh moving of replicas. If our application is addressing the replicas directly, Planet Scale will direct those reads to the reads of the sharded cluster without the application having to do anything. And then we will do some the same for the primaries. a much bigger change, not a one-way change. We can actually still fail back to the single cluster. We reverse replication roles. Again, very difficult to take. We now have 16 separate replication streams. We combine them and replicate them back to the unsharded database.

13:45 And then sharding will be complete. We will have distributed our data across 16 shards. Everything is good. There is another part of sharding that is the most difficult. I have seen sharding product projects at certain companies take 3 to four years. it takes a significant amount of refactoring. I'm going to explain why and we're going to really dig in to the role um cursor plays in this demo. So, first of all, I'm just going to say now let's shard this database. Hilariously simple.

14:16 Um there we go. It's really difficult. You know, I uh used to work at GitHub back in the early days and I wrote a GitHub issue that said we should avoid sharding for as long as possible because it is genuinely has been up until now extremely difficult. So, it's going it's looking um what it has to get done for sharding. And now we're going to talk about the role of the application um as we go. So, maybe I need to like refresh and see if that's going to show up at the bottom.

14:47 Oh, I don't think it's going to load. Let's see if it starts pushing us data in a second. Yep. Okay, it's running the workflows. There we go. Thank god. So now you can actually see that it's queuing non-determinism. There we go. Um, okay. First first element here and this is not just a planet scale kind of demo, right? Like we're using planet scale the platform to show you these primitives and these are the primitives that I believe every infrastructure platform is going to need to build in the future.

15:16 This is just a very applied way and obviously databases are very serious. If we can do it with databases, we can do it with anything. So the first thing we have is a v schema which is how planet scale knows how what to tell the test to shard and where to shard and how to locate our data. So this is very cool because it's declarative and it allows agents to actually build this v schema. This is an example v schema. The agents will build this or something very very similar to this after it gets knowledge from the plan scale skills on how to do this. But we tell it it's a sharded database. We tell it to apply a vindex which is a hash um so that we hash evenly and we can look up the same data every time when we ask for tests. We then talk about the vindexes. So we say what are we going to shard on? So this is where you really have to think as a team or know your product and have the ability to refactor your code to understand this. So most companies have a natural sharding key that will be there. It could be organization Shopify shards on shop. There's like many different things you can shard on essentially, but getting that right is really important and I'm going to explain why in a minute. You can see we've started replicating which is great. Vitess is doing what we need it to do. Um I'll also just show you that completely unaffected for our application right now. It's starting that workflow, but the application is operating perfectly. And so we're saying here that we're going to um shard on our items orders table. It's our largest table flex um and we are going to need to reference products. So here this is where refactor has become very very important right and this is why cursor is doing so much work is we are now going to have to change how our application does joins because we join our orders on the products table. What vitess is going to do for us is a bit of magic. It's going to materialize um the products table onto every shard.

17:02 So once we get to the shard we can join on the products table. If you issue this query right now in the old version of the application and on the old cluster, you just get a simple select query. You get what you want. If we do it in a sharded cluster, this is going to do what's called a scatter gather query. And you can see here it fans out, hits every shard and comes back to materialize the results. For a database, it's possible and it's not terrible, but it is slow and suboptimal to be in the critical path to do things this way.

17:30 We're doing coordination across many nodes. we are aggregating data together and and returning it to the client. So this is where cursor really shines massive refactors, right? If you you you might have to take a 10-year app and go through it and add customer ID to every query, change how you go through every call path um to avoid a scatter gather. So you can see here we're adding customer ID. We know that's how we're sharded and we can then get to the orders table exactly on the right shard and we do not scatter gather anymore. We go to one shard and return it. But if you take an old application, this takes a lot of refactoring to get to this point and really audit the call sites.

18:04 That is what cursor is doing in this demo for right now. So we can go back to the vitest workflow. We can see that the workflow is happening. We've added new routing rules and we're running this workflow. It is replicating data. It's logging what it's doing. It's very safe. And now it's starting to switch replica traffic. So let's talk just generally a bit more about infrastructure while it's doing this. Real products are very complicated, right? The industry has been very obsessed with the point of creation. We can get things built very quickly. Sandboxing databases that can spin up really really quickly. But that's not where the future's going to be. The point of creation happened once like it's done once you've done it. It's the long-term evolution of applications.

18:47 Sometimes applications are built across decades, right? We're work we regularly interact with products that are close to 20 years old. Now there you go. Sharding is done. Let's just go back here and see that nothing went wrong. Our application started to work. Hopefully, we'll be able to dial up our order volume. It's happened. There we go. Look at that massive spike in order volume. We have just increased our query processing rate very very significantly because we're now distributing our workload across a completely sharded cluster. Sharding is done.

19:20 And there we have a sharded plant go cluster. All done. We built planet scale though earlier than agents. Thank you. Um and we built them for the flakiest agents of of all which is humans. We we didn't just like uh we did this because humans have to operate the database and we made it extremely safe by default. And then we moved to um building for agents that just work much much faster and it's it's great that we've got these safeguards that they can go and do it.

19:45 We really follow the Unix philosophy of small sharp tools. you na if you narrow tools down you reduce um ambiguity and you increase reliability. Composition scales intelligence without increasing risk. Having small composable pieces and safety can like happen with narrow interfaces and we can essentially replace complicated decisions and have these chains of simple tools that agents can use to build. So these are some of the tools. Again, these are just examples that we have built, but I I believe every infrastructure platform in the future is going to need to do these things. And you can use all of these for various different tasks. You can just look at recommendations. You can branch for CI. It doesn't mean you have to do a schema change every time that you use a branch. You have online schema changes.

20:30 You've seen the ability to rewind, produce anomalies. This is all generated from a massive platform in the back end. We have to run huge data processing pipelines and data warehouses to take the tens of millions of queries a second that are operating against our platform. Bring them in, analyze the performance of every query over time, make suggestions. Infrastructure platforms are going to have to build so much infrastructure below the water level to support working with agents to scale this expertise. One example I love is traffic control. We've just we've just built this for Postgres. We allow you to isolate down at the query level, the credential level, the tag for the query exactly what resources across a cluster that um any application can use. This is going to allow this allows agents to kind of work through your data set and you can know that they will be killed if they do anything that is going to start to impact performance. Same with when we do schema changes. While we are doing a schema change, we monitor if your queries start to slow down. If you get replication delay, we will actually add back pressure into the system that will slow down what it's doing. We're going to very quickly go through the various stages that we just went through in that demo in terms of instruction. We we use the platform primitives to narrow down instruction to tell agents what to do.

21:46 We go through a development process where we have agents instruct what they need to do and then they go through a more traditional development process of branching, deploying, rolling back. These are all primitives that have to be autonomously managed by agents and agents need to be able to monitor the state of the work that they did. A complex backend still distills these messages into a prompt. Now, right, this is rather than a human scanning through web pages, looking at graphs, looking for correlations, the infrastructure has to surface really sharp information for agents to act on. And good infrastructure turns this expertise into primitives. We're a small company at Plan Scale, but we have hundreds of years of database expertise. We can now forward that expertise through the systems we build to people's agents, so they can build really, really quickly.

22:33 And this scales a new way of working. We we've never been able to write code for our customers before. And now we can prompt their agents to do the right thing. And we can make that change very, very safe. And we're reducing the amount of power that agents need, right? Like if you just throw them at streams of logs and you throw them at tons of data, they can do the wrong thing. and get these things wrong and then we'll go into development. This is like the most hot hot hotly talked about at the moment is the development stage. So I don't need to go in too much like this is a thing that the industry is very good at right now with agents but isolation validation making sure data is accessed properly. They don't just have production access to go and send emails out to your customers or something. And then deployment. Deployment in new infrastructure platforms needs to be this controlled environment where agents can cue and work in parallel if that's a safe thing to do and the platform allows it to be safe. Work serially if that is a thing that has to happen. Right? We might block schema changes while we do a reshard or while we fail over. This is all going to have to be instructed out of the platform. Now long state changes like this will need back pressure. We hold off and push back load and reduce what we're doing for the c for like a schema change or something in reaction to live traffic patterns as they happen.

23:49 This makes agents safer as they go. They'll need to observe their changes and validate what they do. And monitoring closes the loop, right? Like you give the agents the ability to assess that the impact happened or you might fire anomaly that says you've just broken production, roll that back. So the system has to connect every single change that's made across a complex infrastructure system to what happened, who did it, what agent made it change. And that's why we have those kind of various auditable points in the product so that you can very quickly understand what's going on and other agents can observe changes in in an emergent system. Every time you change the the infrastructure, the thousands of services we run behind the scenes now have a new emergent behavior. We need to understand and track those and production behavior becomes the next instruction. What we have just changed will generate knowledge and insights across our entire stack to then instruct the next set of agents that will wake up and do their work. And every stage gets to follow the same pattern. We've hidden very complex things behind automation so that agents can call that automation and use those various pieces to do the different tasks they need. And we've taken our expertise and taken them into discrete tools. So the future of agents is not just having smarter models. That is going to happen. But even if models never get smarter, we have so much more to build as an industry to allow agents to operate against the real living systems. Day one is extremely easy.

25:12 Everything works on day one mostly. Um it's really about continual pruning, evolution, and maintenance and care of what is almost like a living operating system that is the production that agents need to work with in the future. And so infrastructure and really good infrastructure is what is going to allow agents to operate. So that's my talk. Thank you very much for coming.