Section Insights
Introduction to Databases and AI
What is the current state of databases in relation to AI?
The discussion introduces the concept of databases as essential components in the age of AI, highlighting the rapid development and integration of AI with database technologies.
- Databases are crucial in the AI landscape.
- Rapid advancements in database technology are likened to a 'wild west' scenario.
- AI agents are now capable of creating, writing to, and deleting databases.
File Systems vs. Databases
How do file systems compare to databases?
File systems can be viewed as databases, but databases offer significant advantages in terms of data management and safety. The speaker emphasizes that everything can be considered a database, including simple file directories.
- File systems are essentially databases in a different form.
- Databases provide better safety and management for data.
- Using a database system is recommended over traditional file systems.
The Evolution of Self-Driving Databases
What advancements have been made in database management over the years?
The speaker reflects on the challenges of database management and the evolution towards self-driving databases, which automate optimization and maintenance tasks that were previously labor-intensive.
- Database management has historically been challenging and complex.
- Self-driving databases aim to automate optimization and maintenance.
- Advancements in technology have made previously impossible tasks feasible.
Skepticism Around ClickHouse
What concerns exist regarding the ClickHouse database?
There was initial skepticism about ClickHouse being too good to be true due to its rapid development and advanced features. The speaker discusses the competitive landscape of database research and the challenges of building a robust data system.
- Skepticism exists about the capabilities of new database systems like ClickHouse.
- The database research field is highly competitive with many players.
- Innovative features in new systems can raise doubts about their feasibility.
The Future of GPU Databases
What is the role of GPU databases in modern data systems?
GPU databases leverage the power of GPUs to accelerate query execution, representing a resurgence in interest after previous cycles of development. The speaker discusses the historical context and potential future of GPU databases.
- GPU databases aim to enhance query execution speed using specialized hardware.
- There is a historical precedent for using accelerators in data systems.
- The interest in GPU databases is experiencing a renewal, indicating potential growth.
Transcript
0:00 What is a file system? It's a database, right? Everything is a database. Every month there's a story where someone lost their production database because the agent deleted something. It's the wild west for people that don't know the history of databases. It's just like, you know, taking a cheetah, giving a bunch of cocaine, and putting in a Ferrari. Like, the amount of speed that people are developing these things is insane. That is Andy Pavlo. In the world of databases, Andy is about as close to a god as it gets. the Carnegie Melon professor whose courses taught an entire generation of engineers about databases and as it turns out the AI models too. Andy recently made some waves by joining Clickhouse to found ClickHouse Labs. Now not long ago databases in AI meant vector search and rag but today agents create databases, write to them and sometimes even delete them. Welcome to databases in the age of agents. I'm Matt Turk and this is the Matt podcast.
0:51 Andy, welcome. >> Hey, Matt. How are you doing, man? >> Good. I'm excited to talk about all things AI, agents, databases, Larry Ellison. We have to. And then Wuang Clan. >> We definitely do clan. do we have something about Larry Ellison? I mean, it's like a little early to come out of the gate. Although, he is hilarious cuz he's like the super villain with databases. Like he owns a Hawaiian island. >> Well, that seems to be working out for him.
1:13 >> Yes. >> All right. Well, we can graduate to talking about Larry Ellison later in the conversation. I will say my contract with click house does allow me to talk about Larry Ellison. I'm pretty sure that was in writing. >> All right. Awesome. I want to take it from the the top. So two years ago the topic of databases and AI came up. there was a whole era of vector databases and this whole idea that you need to do do rag in connection with chatbot searches. That was kind of two years ago. I think a lot of people's mental models kind of like stopped there in terms of like this intersection of databases and AI. Clearly, we are in the age of agents now. how is that different? What does that change? I mean the vector databases although you're right that like the in terms of being in the zeitgeist of AI and databases. Yeah. Probably two years ago is when the rag vector data sort of came into play. But the vector databases themselves like a lot of them were around since like 2016. It's just now that like the the use for them became obvious because you can use them to to to to enhance and improve you know a agents doing things. So I think the the vector database market is interesting because they were early in the in the space but but they were really pushing the idea that you want to do this you know a the approximate similarity search on vectors or embeddings instead of doing an exact search that datab systems have traditionally done. And this helps too when you're dealing with natural languages that where the the semantics maybe what you're trying to look for or query against isn't you can't precisely define like find me all the the albums in in you know from from artists in New York City that talk about doing you know certain certain kind of activities right it's very hard to define exactly what that is and the the the magic of approximate nearest neighbor search or these vector embedding searches in a vector database is that you can you can look up things based on the semantics of of the thing you're looking for and just sort of it works out that way because you're not trying to do exact keyword searches. So when you know chatb 2024 2023 when again when everyone became aware that this is this is now a technology that exists the vector databases were in the right position at the right time because they they could augment and improve these these chatbots but like the the moat if you will wasn't that big because the end of the day it's just an index and within a year pretty much every single database vendor had their own vector index and now it's become sort of standard stable takes table stakes like every data system that we're using has their own vector index now it doesn't say that the vector databases can't do certain things better than what the, you know, a traditional data system could do. It's just now that everyone has the that basic functionality. So I think that you still use vector searches to enhance and improve the the chat bots or the agents now. It's just the you don't need to have a specialized one database that that that can only do that. and agents now create databases, use databases, can make changes to databases. So maybe a year ago when data bricks acquired Neon, there was this stat that I believe said that the number or the percentage of agents creating neon databases went from 30% to 80% >> or the databases created by agents was like was larger than everything else.
4:35 >> Unpack that for us. What what does that mean? an agent creating a database or why does an agent need to create a database in the first place? >> The creation of a database and then there's branching of an existing database and I think the database metric was about like the the branches being created by agents. so so what does branching mean? So branching is when you have a you have you have a database system or sorry a database and you want to make a sort of copy on write snapshot of that that the database and have it look like two unique databases. and the reason why you want to do that is you could have your production database that's running your your business, your application, like where all your your you know the system of record, all your changes are going into, but then you want to have either a human developer or an agent be able to start trying out new things, make changes to the application and you want them to operate on the real database because oftent times traditionally you would make copies of the database or have a smaller version of of the database like a sample what would already exist there because it's just too expensive to to copy. things around. So you have like a staging or development database and then you would then hopefully apply your changes to the production database and hope things that once you go to the larger scale everything works out. So what branching allows you to do is allows you to make a snapshot of the real database have all the changes that you make go to the snapshot and not affect the production database. And once you're okay with the changes you've made, maybe not necessarily with the data, but usually for like the schema or the application code, once you know you've vetted on the snapshot, then you can put it live in the production database. So now with these agents, people are spinning up these coding agents to write new features, write new new parts of their application. It's not something humans are doing anymore. And now you want to have these agents, you know, they want to they want to vet their changes on a copy of the database. So you can create branches on those things. often just shows up when you're running testing as well. Like a lot of the times the the the branches are created by you know PR is running in GitHub actions or testing things like that. So you want to run your tester code on a copy of the database rather than the live one. So it's mostly being this branching capability is used for development. So agents are people are using agents to write a lot of new code and so you want them to run in their own sandbox. You make a snapshot of the database. So the branching capability is super useful for that.
6:50 >> Yep. even more sort of simpler level an agent needs to create a database because agents take actions on the world and if you ask an agent to create an application an application at the end of the day steal database. >> Yeah. So, so certainly like if you say like if you create a database from scratch then yes your your coding agent you give it the promise say a billion application does this then yes it will create create a database right away but most of the most of the branches or new databases if you want to call it that being created are for this development but yes the agents can now create databases for better words you can have them drop databases people often get in trouble with this because it's doing things they shouldn't be doing >> yeah there was last year as well >> every every year there's some some story >> there's like there was some some incident that was very interesting and I got got pretty widely publicized on X for what it's worth lenin that was trying to build something with Replet and I think the replet agent went into the database and deleted a lot of files which then subsequently Replet made the right adjustments you know obviously all of this is the wild west and and people are learning as they go and mistakes will be made >> but what >> it's the wild west for people that don't know the history of databases like Like for years we've had the capabilities to make sure people don't do things they shouldn't be like stupid things like dropping a d table or deleting what records those you know those controls exist it's just everyone wants to reinvent the wheel and people learn the hard way of just doing what's already been done in the past.
8:19 >> Let's expand on that point. So so what what does that mean? What what does what needs to exist in the agentic world for databases that is not currently in place? guardrails, right? Like like like you know you wouldn't you you know you wouldn't bring a new you know have a new new child a new toddler and just like not put up the guardrail so they don't fall down the stairs or you know put their finger in the socket, right? It's the same kind of thing for these agents.
8:43 They think of them as like toddlers. a little more sophisticated than that, but like you don't run wild or do whatever they want on on a database. Now you could sit and babysit it and just say approve approve approve like you know in cloud code or whatever but of course then you can't scale and have hundreds of agents trying to do things at the same time. So you want to set the right permissions and controls so that they can't go into the production database and drop tables that they shouldn't be. And likewise you you don't have to go read things they maybe necessarily wouldn't want to read as well. So if you have the right guard rails and you and through branching or whatever means you want to you know have create these sandboxes you can avoid disasters like you know all you every every month there's a story where someone lost their production database because the agent deleted something did something they shouldn't do with the right guard rails in place you can assure that these things don't happen right this people get lazy they use the same password across all you know entire company organization and then they give the same password to the agent and wonder why the agent started doing things they shouldn't do >> should those be the same guard rails as what currently exists?
9:46 >> Yeah. So I maybe the larger question is like do agents at least now behave differently than than humans? other than just being able to run 24/7, I haven't seen anything that would suggest the fact that like they don't develop the same way software the same way or inaccurate data the same way that humans do. Right? The end of the day, the data system only sort of exposes so much different capabilities. and that's the API that gets exposed to you. So as long as you you know you can constrain those things for human using those APIs you could do the same thing for for agents and the but the level sophistication of what those guardrails could be beyond just like don't run delete or you know or update on a table the the level of sophistication really matters on what sort of product you're using. and I will say this is something where like you know I like open source databases but like the enterprise commercial legacy databases if you will they've already done all this you know for you know for humans in the past so they have a lot of the protection mechanisms you would want and just expose that to agents as well.
10:51 >> Understood. Is there anything that's funly different compared to human is that I think you alluded to volume and scalability. Is that is that does that change everything? The fact that you have possibly billions of agents taking actions and how they interact with databases. I got to be honest, I don't know the answer, right? Like it's still early early to say and I get I mean the volume of course yet right because agents are running 24/7 in a way that humans cannot. But we've we've seen similar patterns in databases before where we've seen increase in volume. And I would sort of say there's been three eras and we're now in the fourth one. I the first one was like at the very beginning of just having a database in general like that you can now start putting in information and run transactions and and store things for for your organization in a way you couldn't before when it was was all pencil on paper. So probably like 70s and 80s you know there was a obvious increase in the volume of of traffic you need to handle in a database. But in that era it was mostly humans interacting with the database but at their jobs because no one would go home and no one had a computer you know necessarily in like the 80s certainly not the level we had now. So that was sort of say one era we had increase in volume. Then in the 1990s early 2000s the internet comes along and now everyone can interact with databases through websites and through the internet at home. So you saw you know sort of spike in in volume that in that level. Then the next era would be mobile phones. So now everyone any any given time has has you know various apps running. They can interact with websites then you know put things and read things from databases. So I would say that would be theory. So now agents I think is the next is the next chapter in the story where now it's every human not only has a cell phone but now they also can have maybe hundreds of agents interacting on their behalf that are then interact you know reading writing data from from a database end of the day when you application is always going to be backed by a database so that's going to it's going to cause the application to read and write information read and write data. So, I don't think we're there yet because every human obviously doesn't have hundreds of agents running 247. But what's interesting about what could happen is the when you think of a human interacting with their cell phone or or their laptop on a website or an application, they're kind of really only doing it when they're awake, right? You can only can only do one thing at a time. Like I can't sit with my one cell phone in one hand and laptop in the other hand, like do things at the same time. You're kind of only limited what the human can actually do. but you can imagine a world where these agents are just 247 always running and acting as if they were humans and interacting things. So like the volume would increase quite significantly. And then furthermore unlike like a like a web app they're usually going to be you could argue like you know how's this agent really different than like IoT devices or cell phones now like IoT devices you're just streaming inserts of of here's the measurements I'm getting. you know with cell phone apps you're usually reading data right but you can imagine these agents do more complex operations where it is a combination of read and writes and then that makes the traffic more more interesting so I think there's going to be potentially a 10 to 100x in increase in volume and the number of queries that are going to hit up against databases and I don't think we're not there at that level yet but I definitely heard from my friends at industry that they're seeing the the sort of thing we've seen we've seen the same thing at at click house what I don't know yet though is whether the sophistication of the interactions are more sophistic or or more or less than what humans can currently do. I've seen some people say well the the type of queries that the agent would generate or SQL queries the the agent would generate is actually more shallow because it's just trying to like maybe read one or two tables at a time try to figure out what's in there versus like if someone uses like a dashboard or like a BI tool like Tableau or Micro Strategy and maybe doing like very complex joins across the different tables. I don't know yet whether agents are at that level or whether they're kind of going at it reading trying to read data against the database in more simple terms. So I mean so your question is like are agents going to change things? I think yes.
14:51 Right now definitely in volume and I think that's going to increase even further. And of course now when the agents are running they're they're own generating their own telemetry. So that you have to store store that and make sense of that and that's another challenge. But in terms of like whether the the actual patterns of what they're doing how different is that yet? That remains to be seen. This is something I'm trying to research as well. Yeah, some fantastic stuff in there that I want to go back to in a minute. But there is a concept of agent memory that seems to be a little bit of a battleground right now with on the one hand >> file systems and on the other hand databases. What is your view?
15:30 >> Oh, what is a file system? It's a database, right? Everything is a database, right? You have a bunch of you have a directory of a bunch of bunch of Excel files. That's a database, right? you have a have of a notebook, a pencil and paper, that's a database. Like when my daughter was like three, we would we would, you know, was trying to teach her what, you know, the importance of databases and I would make her write down, well, we do it together. We'd write down the the temperature every day, you know, in a little little line notebook and then she'd close and go commit to save the data. that's a database, right? It really comes down to what the, you know, what the form of the of the input or the context you want to provide to the to your agents are now. And as far as I know, it's all human readable text, right? So whether or not that human readable text comes from bunch of JSON files you have in your directory or it's extracted from a database system that can then convert that into the correct input form. At the end of the day, like it it doesn't matter.
16:28 I think that there's obviously advantages and safety guarantees you can get when you use a data system that can protect data. Like SQLite is famous for this. Why would you want to use a file system and try to manage whether you're writing things correctly and safely and that you can multiple changes and and commit this versus like just single light just does it right. So like at the end of the day, everybody should be putting everything in a database system, right? Even some file systems like WinFS underneath the covers, it's a database system, right?
16:54 >> Yeah. There was a piece that also like went viral at some point. I think that was entitled I I recreated a database by accident like talking about like file systems. >> Okay. Interesting. Does does the the the world of like multi- aents, sub aents, agent swarms, does that require more of a database? meaning like they sense like computationally very complex and therefore having something that's extremely heavy duty on the other side seems to make sense. In the end of the day, if you need to share state across multiple machines or instances of these agents, whether they're running on the same box or not, like again, the data a database management system provides that that capability is that you could have, you know, multiple multiple readers reading the shared state, right? I haven't seen anything where like agent it's like as part of its memory lookup is you're pushing down like filters and things like that into the database.
17:52 Usually that's provided as part of the context. Again, this going back to the rag or the vector database stuff like you provide the context based on what you've already looked up in against your your the the memory you know in the database files or the database itself. So again from my understanding is that the API for the input to the context is just human readable text and whether that comes from a file system or a or a database system it doesn't matter but the data system provides certain guarantees and luxuries if you will that a file system can provide and shared state is one of them across multiple agents.
18:24 >> Have you seen the agent world converge towards a specific type of database? I mean asking like like in terms of like using it as for memory or having an agent choose one particular data system over another when they build an application >> more the actually both >> so maybe the latter the latter one it obviously they're fed on on the models are trained on what people are writing about and so you know for if I say if I'm building a new application chances are it's probably going to recommend Postgress at this point >> because that's what in the pre-training and I will say I can't say who I've had two database companies ask me how to get their database to be recommended first by an agent over Postgress and I was like that's not first of all that's not what I do that's not research and two like don't talk to me go talk to the guys building the models like that's clearly something that you know they could have sway in it's basically like SEO how do you get them the agent to learn like hey I should use my database x instead of postgress >> I would suspect that your writing is highly indexed by the search engines and and therefore pitching you is a way of pitching the model.
19:33 >> So this is a very much a first world problem and I'm not trying to say like a humble brag but like one problem I do have is when I ask questions about databases it cites my me back to myself cuz all my course materials >> which has to be somewhat satisfying. >> Yes. Well, no, but it's like no because like say when I was teaching database courses, I would say, you know, show me I want to learn about data systems that do this version of it kind of index people or whatever and I would say so now I have to say like I already know which ones do do this. So show me the ones that don't do this. And then I have to say also don't cite >> don't let Andy Pablo or anyone caring me. But then the problem is that our >> doesn't reply. You know I know you're Andy right?
20:13 Yeah actually says like I I know who you are. It says I've said something like that but perplexity doesn't do this. But the problem is also too is like says our all our course material is open source so anybody can just take and use it. So then it it says okay well you can't site Carnegie Melon University. Let me site another university. But it's just my it's my slides being re used at another course. So I gota be like very specific about what what not to return back to me. so I think that I mean to your original question like I think for OTP I haven't tried this in a while but usually it spits back hey we should use Postgress unless you say something like very specific like you know oh I'm going I'm going to run on embedded devices then then it'll say use duct DB or SQLite or something like that or I only care about key values I might come back and say like use Rox DB but in terms of grow it almost always says use something use postgress and then it might recommend like do you want a hosted version or not right if you have further chats for OLAP I I be interesting to see if there's any kind of ranking on this it's actually not a bad idea maybe somebody's already done this but like that one it definitely would come back and and maybe you have to refine it further and say like I'm look my workload looks like this my data looks like this and then I think it can give a more nuanced answer I haven't tried this in for about a year or so but I've definitely seen before like it say oh you know do you have any money yes okay you could use snowflake or data bricks do you have no money okay you could use duct db or you know do you want to run on prem you could use click house things like that so and then your second question was about agents >> well which one should they recommend so there that's that's the that's the answer to which one they do recommend but >> which one should I recommend >> yeah and where I'm going with this partly is you know this question of of of speed and that's where you know click house shines and we'll talk about click house more later this perhaps a concept of u you context layer ontology which seems to be pushing towards the graph databases are they you know what what what what makes sense >> I I mean I have strong opinions about the graph databases I don't think you need one I think that can be I have not seen any benchmarks that show graph databases the native graph databases outperform relational databases systems when those relation systems have the right constructs in SQL or the API so that you can do graph traversal all on the server side, not go back and forth.
22:38 SQL the SQL standard in 2023 added support for property graph queries. Oracle supports it. Postgress was supposed to support it this year. It's coming out post 19, but I think it got delayed or rolled back. I don't know why. but anyway that's beside the point. No I I think that I mean the context layer again depending on the requirements of what the how many agents you have how much whether you have shared state or not across multiple agents.
23:08 Postgress often is is you know should be the first choice for many of these things for OTP. If you're doing analytics that's a whole another story. >> Let's talk about MCP. >> Yes. So over the last year or so it seems like everybody every database provider added an MCP server to their to their offering. Thoughts on that like pros and and and cons and I'm I'm coming from the you know year or two ago there was like this whole like tech to SQL kind of movement where everybody was trying to make LLMs work on top of databases especially for an analytical purposes and that that sort of didn't work. Are we there now? So Texas SQL is like natural language to SQL. So that this is something people have been trying to do since the 70s, right? and every so often there's you know maybe a there's a you know some kind of breakthrough in natural language processing. People say okay now finally it can happen right it hasn't happened except for LLMs. I think the I've been skeptical of the academic literature and research publications on various frameworks that do text to SQL because often times the benchmarks are showing spider and a couple other ones like these are well-known benchmarks that you know the models are trained on. So like of course this regurgitate the the correct answer. I actually was talking to somebody from a very large bank yesterday who said they now have a text to SQL interface. but in the initial implementation of it they were doing something off the shelf and then they they further refined it a little bit further inhouse but it was it was about 60% accuracy meaning like the the the the LLM or the agent could produce the right SQL query produce the right answer 60% of the time. Right? And that's roughly what I think I've seen before in other some other reportings.
24:51 But they said through this providing a a a more rich semantic layer, a context layer over multiple years and millions millions of dollars to make this work, they were able to get it up to like 99 like 5% which is insane. I think that is I think that's that's a is a viable thing now if you invest the time to have the right semantic layer and the context information you need. So the the model is has the sorry the agent has enough information to how to derive the right SQL query from the question you're asking. I think again that's a a large bank with infinite money and they pull this off. I think if you just download something now a random person will find that these existing tools are not going to be at that level of proficiency and accuracy. but it doesn't mean we can't get there right. so now in terms of MCP, the the the model context protocol, like I get why they did it. Like I think this came out of anthropic and then everyone sort of quickly adopted it as as the as the standard. and every time there's every so often there's a hacker news post saying like MCP's dead or whatever.
25:54 Like it's I don't think it's going away. It's basically a REST interface, right? A standard way to to specify what you want the the the request to be from the agent to whatever, you know, data system you have. And then that's that that data system that receives the MCP request, it's their job to convert it into whatever the correct API calls is to get the data you need. And in the case of database systems is going get you writing the SQL query to go get the data you need. But usually it's either you know often times it's a strict translation of what is being passed to you and not like trying to derive the semantic meaning of the MCB request because the MC request has to be somewhat well well defined.
26:32 I have no problem with MCP. I think that like it's not it's not this magic bullet that solves all the world's problems. It's just a digital interface, right? that you can now interpose or interoperate these different agents and different systems to collect data you need. So I'm all for it because it's just you're using the data more rather than some some homegrown thing. So yeah, fantastic. It's great. Could it be improved? I mean like I think it's it's inefficient because you're sending human readable text over, you know, over the network to go retrieve data. There's obviously better ways to do this with binary protocols, but like in the end of the day, it's it's good enough, right?
27:07 And it seems to be the standard everyone's using. >> Should it be that security layer that you were alluding to earlier? >> Yeah. So, like you could like it's but it's no different than like the if the request came from like SQL from a traditional application client with using JDBC or ODBC like you know the network protocols or terminals for for these different data systems. It's no different if it comes from that versus, you know, a rest request with over with MCP. The end of the day, like, okay, I I got this request. I need to understand who's asking for it. What are they asking to do, and should they be allowed to do it? Like, so the the the guards we were talking about before, they're the same if it's coming in a SQL versus same as coming in MCP.
27:48 >> Okay. So that's one part of the conversation that's databases as a memory layer or execution layer for for agents. let's let's flip the discussion into what AI can do for databases. So clearly we live in a world where AI can code just about anything. Is that true of databases as well? >> Yes. So to give one anecdote, I teach data a course on data minimum systems at Carne University. a year ago the the agents couldn't do our entire project.
28:23 So the projects would be like we give you a scaffolding with database system and you have to implement you know the indexes the query engine and things like that. It could do some of it not all of it. I think it was opus for whatever that whatever anthropic put out last year then that just opened the floodgate. the agent basically do all our assignments right without very little prompting and of course there is a lot of training data for it because all our projects are open source they're all on GitHub not just students at Carnegie Melon University but also students outside of the university we let them use it so there's a lot of training data for them to implement things and so I think agents basically could implement anything you know you would want to build a dat system now you know obviously you have to prompt it the right way and hold its hand and make sure you you have you you generate the right design or produces the the the implementation based on the design that you want. the end of the day I think these agents are very capable to be able to do this.
29:16 >> So they could build an entire database because so my my experience of building database as a venture investor is that it's a 10ear journey of pain where nothing much happened for three years and you have some of the smartest people in the world getting together to solve what seems each time like insurmountable problems. So we're now saying that he can build the whole thing. >> Yeah. The old adage from from data systems is that it takes 10 years but a data system you can build the first 90% in 3 years and then the remaining 10% takes the next 70 years or seven years. so yeah know I I I I think that the agents are very capable you know with enough tokens of course and then with enough guidance people can you can build vibe code entire datab system and there's certainly companies that are doing this now and pretty much every single company is using agents to help develop things. So one of my again I love databases one of my side projects is the database of databases dbdv.io IO.
30:10 and one of the things we do now is we keep track of every single data system that I know about. And for the open source ones, every single night we pull down all the latest commits on GitHub and we then we track to see which ones are actually being, you know, co-signed by clawed or or codecs and things like that. And obviously people turn that feature off. You don't know whether it's actually been gener. But most people don't do that. And at this point I think like 60% over 60% of of the of the open source database systems are being have commits from coming from agents.
30:43 >> And so does does that beyond the writing also apply to the running of it. So going to that self-driving database concept that's something that was a a big project of yours 10 years ago I believe we've Yeah. >> Yes. So walk us through that journey. What was not possible then that has become possible today. >> Yeah. So so when I started at Caring University, one of the things I did was my first years I I would go visit companies and sort of see what sort of challenges they were facing with databases. And the over the the overarching theme I saw over and over again was like just you know running these systems, maintaining them and optimizing them was a huge struggle. And this is not like a huge revelation for me like people have been trying to do this for for decades. I mean since the the the creation of the relational model and relational data in the 1970s people have been trying to do you know autotuning for indexes partitioning keys sharding keys and tuning knobs and so forth like Microsoft research did a lot of work in the early 2000s in this auto admin project they had a bunch of tools allow you to to manage and can optimize data system automatically >> and for context of this because like there's thousands and thousands of possible configuration of a database >> right so there's there's like what indexes should I have in my table to speed up queries and obviously if you have too many indexes the rights go slower and you run out of disc space.
32:01 There's the various knobs that control the runtime behavior of the system. How much memory you use for, you know, one memory pool versus for indexes and so forth, right? This is beyond the the the capabilities of any one human to reason about. And then if you start looking at, you know, if you have thousands of databases, you can't have, you know, it's just not possible for anybody to manage these things. So we spent some time looking how to build a automated tools using machine learning and then you would what you may call AI now but using automated methods machine learning techniques to to automate this and we sort of had two research tracks we had one where we sort of took an opaque box view of the database system where we said we can't change the internals what can we do just with what the APIs that this D system exposed so like basically How do you optimize automatically existing databases my SQL, Postgress, Oracle and so forth? And then another track of research was if you build a data from scratch assuming it was going to be controlled by machine learning you know tools or now what we call agents how would you design the system slightly different and a lot of things we learned from trying to optimize my post we end up end up building in our own system. so the challenge always was then for these these automation tools was just having enough training data of like production databases and being able to take the things you learn optimizing maybe one deployment and apply it to to another.
33:26 Like as I was saying before, most people will have in, you know, they have a staging or development environment and then they have the production database. And so you obviously don't want to try random things on the production database that may cause you to slow down and and degrade performance while you're trying to figure out how to tune it. So you would vet things on the on the production sorry the the staging database and then try to apply to the the production one. But often times like it's never the same hardware, never the same workload. So that that was a big challenge we were always facing just getting enough training data to feed into the models that you can then make good decisions on the production databases. And then the LLM show up and that really changed how you can approach the problem because they are basically trained on blog articles, documentation, you best practice guides that are all available on the internet of how to tune these databases. And so on the research side, what we found is the LMS can get you maybe 85% of the way there in tuning certain things. And obviously there's weird corner cases bespoke algorithm can can handle those. But like our models often time would take hours and hours to train assuming you had enough training data to produce like you know the the pristine optimal configuration but the elements can come in like in 15 minutes produce something that was good enough and that's good enough for most people. So are we there yet? Are we in a world where database can largely be sort of self-tuned self optimized? I mean the answer is yes but of course it depends on what how much improvement you actually want like I'm saying like the the LMS can get you 80 85% of the way there is that is there always more performance to squeak out yes and some people are always going to care about the things and then therefore the the automated methods we built in the past would matter a lot more but for most people actually one of the things I was surprised so much was like when we had the startup and trying to do this auto tuning stuff for post my SQL is how many >> that was autotune >> autotune yes was how many people were running with the out of the box configur uration from Amazon or or Google like in their cloud platforms and they would tell us oh we thought Amazon was tuning this for us like no they're not right like so there's enough people that are maybe running with a default configuration which is that thing's just terrible don't do that where like an LLM could come in and take care of some low hanging fruit and provide some meaningful improvements for people in a short amount of time.
35:43 >> So in the rest of the software world we've seen the the nature of the job of a developer profoundly change. If you're a student at CMU today and you're super excited about the world of databases, so yes, you can work for the click house or cockroach DBs of the world. But outside of that, what is a what is the a job today that requires deep knowledge of databases if we're in a world where agent are going to be like calling them managing them and then they the database themselves will be self-driving. So I would say like so I would what I would tell my students always was like you for my courses like even if you're not going to go off and become a database system engineer like go work on the internals of click house or cockroach or postgress the end of the day everyone's going to interact with the database system and the if you understand the internals of them how they're implemented how you know what the data system is trying to do or not do for you for given queries you're just going to be in a better position to understand what's going on in in in in the world in in tech stacks. and now do you need my course to figure that out? No. I be I'd be naive to think so that like the agents, you know, they've sucked all my stuff in so they can regurgitate everything I would teach anyway. So, but at the end of the day, again, whether you're learning it from from a traditional university course or self-taught or having LM teach you things. If you understand what a data system is trying to do, then it just it sort of removes most of the mysteries and it helps you understand like what you know what's going on in in your application. why are queries running slow or not running slow just you're in a better position to understand what's going on and so I you know if if you would if you if you pursued a computer science degree or engineering degree and focus on datab systems I don't think I think the era of like you're going to write system from scratch by yourself I think that I'd be naive to think that that still still going to be around but the end of the day the database companies are still hiring people to work on these things because you need someone needs to understand what the agents are actually doing it and make sense of it.
37:42 >> But to fully ask the question like we in an era where there's going to be just no humans in the loop in the way databases are managed and deployed. >> I would say that the traditional role of a DBA as we've known in the past I think that's going to be relegated and a lot of that can be automated. Doesn't necessarily mean those roles go away. They just now get to pursue more higher minded activities like data modeling always important. and again agents can do a lot of that too but you still need someone to understand like yes does this data model capture what our organization or business is trying to represent in in the database all those things are super important still just like does someone need to be like tuning the knobs or picking exact indexes for tables I think that ships out you don't a agent can do all that great to switch topics a lot of people in the database world I assume pretty much anyone is very familiar with your work but like for for people who are not so you're you have been tenure professor at CMU. You have had very popular classes. I think part of the one beautiful thing that you did is that you took a lot of CMU courses that you were teaching and then you put them online for free.
38:54 >> So when I started at Carnegie Melon, I was the only data systems professor there. There was another professor there, but he does more like graph mining, data mining stuff. And so when I started in 2013, I was like, okay, well, I got to figure out how to get tenure. And competition is not the right word, but I'm competing against MIT, Stanford, and Berkeley. And I'm the only Davis person there. So like, all right, we'll just put everything everything for free on the internet. and you know that, you know, help promote our research, what we're trying to do. it also had the the side effect of like I would end up being more prepared for classes because I would, not that I was a phone home anyway, but like I was like make sure I had my stuff together because I knew it was going to be recorded and watched because I don't want to say the wrong thing. and I would be more professional. I say less crazy things which is people are like shocked when I say this that like my course even now is considered watered down like even for them that's that's bizarre. so yeah like it was and like I think obviously the benefit is like not everyone can go to Carne University. It's not cheap.
39:51 It's a it's a private school. not everyone can get in but like why should like that education not be you know held back from other people. So it doesn't cost me anything. I just put it on YouTube and make everything available to everyone else. And we were joking about Wuang Clan at the beginning of this conversation. I think I I heard or read somewhere that you had a DJ spinning wooden clan at the beginning of class. What is that story?
40:17 >> So So I it started in 2019. I was like, "All right, well, let's have fun with this." And you know, when you set up the cameras and things like that for your for the lectures, there's obviously time to take the you know, set things up the wires. You know, we were we had to set all this up, too, right? There's time. So, I'm like, "All right, well, there's this awkward silence where the students are coming in, they're staring at you.
40:34 There's nothing really going on. There's nobody saying anything. So, like, let me get a DJ to sit next to me and just play beats and make make play music next to me the whole time, right?" And then like, you know, then you start talking to them while during the lecture and you find out like, you know, they live pretty interesting lives. Like, one guy, you know, was had a bunch of ex-girlfriends trying to hook up with him. One guy would loan a bunch of money to people he shouldn't be loaning to. So, just like we just talk about it in the class, plus databases >> as one does.
40:59 >> As one does. Yes. And then I think one year we had I did a little joke where oftentimes the top database conferences are always at the beginning of the semester at the end of the semester. So I I have to travel. So like the the first week of classes I'll be traveling to go to a conference. So you don't want to cancel class because it's super important for students to know what the course is going to be out the first week because they're trying to figure out classes they want to take. So I would I would film it remotely. So one year I was I was in LA for for a conference and then at the very end of the lecture cuz I filmed it. I was like, "Oh, by the way, the most important things you need to know about databases." And I just listed all the members of the Wuang Clan on the first album plus Capidana who was in jail at the time. And I said, "This is the most important thing you have to know about the databases." And then when it came at the end of the semester, the final exam, the very first question was list all the members of the Wuang Clan on the 36 chambers plus who who was in jail. And you have to get exact spelling. And if they got that question right, they would have got 100% correct on the exam.
41:56 >> Yeah. Not not killer, but killer. >> Yes. Right. So, someone got that wrong. They they they missed the H when they they when they should have had it. so, no one actually has ever got that. and obviously I can't do it again because students are expecting it. >> Okay. Amazing. All right. So, that takes us in some way to Click House and and and Click House Labs. You joined Click House recently to start Click House Labs.
42:21 >> Yes. >> So, why start a lab in a commercial company versus doing that in academia? I mean without going too much into politics the the nature of research in university is kind of tough right now. The funding in the United States is not what it used to be. There's also a larger question of like what does it mean now to have a PhD student when an agent can kind of do a lot of things right? Do you need to have as many PhD students working on stuff when like an agent can produce things and work with me as the professor to to write papers and things like that?
42:55 So I still I mean still there's certainly value still educating students and I like doing that but just wasn't clear to me like how would I raise money to have many students that I would want. So I had end up had some companies reached out to me about like you know hey can you interest in going leave and doing some some interesting things with us? but it was always a bit more like, hey, you want to come be an engineer with us or like kind of watered down versions of what I wanted to do on on the research side. and then when I talked to the Click House people, it almost seemed too good to be true because I was like, "Hey, you know, you know, they're like, hey, come do research with us."
43:36 I'm like, "Okay, this is what research is. You understand this?" They're like, "Yeah, yeah, go do this." I'm like, "Okay, do you understand like it things might not always pan out correctly because this research? I don't know." Like, "Yeah, great. do it, do this. and it's it's insane because they're, as you know, they're, you know, they're not an early stage startup, but they're a startup. They're not a public company. And normally you only see companies, at least in the database world, set up research labs when they already get established, right? And for them to take, you know, this kind of risk or have the the luxury of having someone like me come in to do peer research, is is is amazing. and they're they're very serious about it.
44:16 so like the opportunity almost seemed good too good to be true and so far, you know, it's I'm I'm two months into this and it seems to be working out. I'm sort of in this observation period now where I'm going through and understanding the stuff that they built so far because they haven't really published it published or written about what they've done. They have some blog articles which are super interesting, but they've done some amazing work that I think you know, I certainly can't take credit for for doing, but like they've done some amazing work and my job right now is sort of helping them turn those into publications. we we can disseminate and put out into the world as well as start ramping up longer term more explor exploratory things and be very clear like even when I was at the university you know in database systems my work is very much applied so it's not like I'm doing theory work that has no one you know only 10 people in the world can understand it's not directly related to anything that's going on the click house you know the the the kind of stuff I want to do is directly relevant to what click house wants to do not just on the click system itself but also now they have a postgress offering there's a bunch of stuff we've done in the past on postgress and now we can start getting looking at real data trying to put some of these things in production workloads and help people you know put things out there so that to me that's like super exciting to have like direct impact of like instead of me writing the paper and then hoping someone comes along and like takes it and adopts what we're doing we we can do the research and evaluate whether it's a good idea or not on real workloads on real production systems And if it does make sense, then we can push it out in a short amount of time. So that part is super super seductive to me that like that I had this opportunity.
45:53 >> Okay. Fantastic. While we're at it, let's talk about click house a little bit in which for for disclosure, we are very proud investors here at First Mark. I think I read somewhere that in 2016 you were concerned that clickaus was possibly vaporware because it was a little too good to be true. Yeah. >> Can you unpack that for us? >> Yeah. So, so as you know like I mentioned like you know my database databases my job I think as a researcher trying to understand what else is out there because database is a really interesting research area because not only am I again using the term competing loosely I don't mean that in a in a in a pjorative sense but like you know but you are in kind of competition with other researchers try to get ideas out and published. So not only am I competing with other researchers at other universities, you're also competing against the the the big tech companies, the Microsofts, the Google's and Amazon of the world. Plus there's all these startups that have their own data systems. Like there's not there's not a lot of operating system startups.
46:53 There's a lot of data system startups as you know in as an investor. So my job is try to figure out when when a new system gets announced to see like what are they doing that's differently than than my own research? What are they doing better? what can I learn from from how they approach the problems that they're trying to solve? And so when the the announcement of or the release the unveiling of Click Alice in 2016 came out, I was like I know how hard it is to build a data system and for this thing to appear from nowhere out of nowhere and have like have all these capabilities like the vectorzed execution, the columner stuff, the compression things, the compaction stuff like it it almost seemed too good to be true or it's a fork of something that already exists. But at the time in 2016, there wasn't an OLAP engine that was open source that would have all these capabilities, right? So, so that threw me for a loop to I was like, oh, so I just assumed it was fake and then sure enough, it's real obviously and has a lot of benefit.
47:47 >> So, what what are some of those benefits in particular? it's it's very fast, right? It's very real time. why is that possible? Why do databases typically struggle with that, especially analytical databases? and why are they able to do that? >> So, so I would say the architecture of click house the the the the core fundamentals is by 2026 standards is not novel by 2016 certainly was the the nature of like you're going to store data in this columnar layout. So instead you know storing all the attributes all the data for for a single column together continuously versus like in a row store you store all the the values for all the columns of of a single tuple continuously like storing things like that manner using vectorzed instructions on CPUs to to process mult multiple pieces of data at the same time within you know one single instruction versus like having to do four loops and and run things less efficiently. the the built-in compression mechanisms, the the way they doing a log structure storage where you're appending a bunch of changes and then because you want to ingest as fast as possible then eventually in the background you'll compact it and store it in a more efficient manner. A lot of that those ideas of the core architecture come from the work done at at snowflake and prior to snowflake there was another system called vector-wise or x100 sorry the monb x100 which then became commercialized as vector-wise from CWI which is the same school where ductb was was built so a lot of the ideas that people have been developed for these column store systems and analytical systems and and real time analytical systems have been sort of floating down.
49:32 but nothing was open source in 2016 at least as far as I know that did all these things. So when click house came out of the game like hey we have all this stuff based on all this research that that people have done before. It was amazing right and since then you know ductb has something similar polers from the guys in in Amsterdam firebolt started off as a fork at click house and they rewrote a bunch of stuff inside and now they have all these things like data bricks has their own version engine. So everyone has this sort of this this architecture that is predicated or based upon the work done from snowflake and and the vector wise but now but again 2016 that was super novel now it's sort of table stakes and one of the things I think click house has done really well over the years is just keep improving the performance even further and expanding what workloads they can actually can support >> and what what use cases are you seeing for click house from the inside what do you see customers do.
50:28 >> So I think the going back to the agent stuff, the agent stuff seems really interesting too that like people are using this to observe the information the telemetry that these agents are generating and be able to react in them in real time and ask questions about them. There's a bunch of stuff done on the financial markets as well, fraud detection, anything where you want to be able to ingest data very quickly and then ask questions about it, complex questions about it very very quickly and get back things in in milliseconds rather than seconds like that. I think the combination of doing those two things is puts click house I think in a unique position >> and click house has been on a journey to expand its product offering right so it started with this very powerful OLAP analytical realtime database and now it does it bought langu observability it bought other things like how do you from the inside understand this evolution I mean does it all fit together.
51:27 >> I mean, somebody asked me like why why did I join Click House? again, I'm only two months into this and it's kind of like asking like, you know, like you just got married and somebody asked you why did you choose your wife or husband? Like, oh, because they chose me, right? it's a bit more than that. No, like honestly, before I decided to join Click House, they were doing things at the business level like so this is Click House Incorporated, not Click House the system. They would do things at the business level that I would have done if I was running a data startup like they were like having a postcost offering not just a single you know OLAP engine that's really good not starting in the OTP space having the the the monitoring the visualization the observability stack not just for traditional BI or traditional data but also now the the agents and the LLMs like they're doing all the things to expand out different verticals that build AC you know on their on their platform like all All that that to me that they all make sense in tracks.
52:23 >> Maybe segueing into the state of the database market in 2020. We've alluded to a bunch of this already but like to put it all together. so one thing that click house has done as you mentioned was adding a posgress service a managed posgress service. it seems that it's a clear trend. So data bras bought neon I think snowflake met an acquisition >> crunchy last year >> that convergence of anical databases and transactional databases is that the future is that driven by commercial and business reasons so you can sell more to the same customer or is there an industrial sort of product logic to it >> but there's two things like should a database company offer both a a OTP transactional operational data system and a OLAP system or at least whether or not they're separate systems or not that's that's up for larger discussion.
53:21 Should you do that? Yes. I mean going back to what agents allow you to do like obviously maintaining these things is still a challenge but the the bar of entry of of deploying a Postgress offering is is is certainly been reduced. I another conjecture I have in my mind is like is it better for a an analytical company a startup that starts with an analytical database to then add later on add a OTP offering versus a OTP company then later add OLAP offering I think the again I realize I'm saying this and I'm biased because of my click house but like you look at like what data bricks did you look at like what's what what click house has done to me that appears that's the right path to do this like if you want to start if you want to start a new data setup start doing analytics first get traction there then you can then add the OTP offering but I I can't prove why just from the from the business perspective it seems that's to be the the the smarter play now the question is like okay should you have a single API single interface that or a view a single a single logical endpoint that allows people to run transactions and analytics on what is perceived as a single database instance or should you have sort of be separate services. Click house right now is separate services. What the you know but they have a fast path to get the data out of the the old side from Postgress into click. What what data bricks announc announced this year was their their LTOP engine or LTA story for their lakebase where you can run your transaction transactions on Postgress or Lakebase or Neon and then have the OLAP engine be able to read the that data directly so that you don't have to make you know two copies of it in the way click house does or other systems do. So that idea is not new to have like this this sort of two engines but like one copy of the data. There was a previous startup in the 2010s called splice machine which actually I was an adviser for because the guy the founder was CMU alumni and he asked me to be to be an adviser for them. but they were because they were running HB plus spark SQL but they had a single copy of the data that was a rose store. the I mean people have been wanting these HTAP system or these hybrid systems since the very beginning. it just they always the challenge has always been like how do you you know how do you how do you reconcile the fact that like some of the data might be in still in the row store versus the column store and and like the major commercial vendors like Oracle SQL server they'll they'll store two copies one the ro store and the com store it's called fraction mirror approach and they have an internal mechanism to synchronize these things but the the the challenge has always been why these this sort of hybrid approach has never taken off is because the stakeholders at organizations or companies for these two different sort of categories of workloads have always wanted you know their own best thing like the OTB team the people running the applications operational side they don't want something that's kind of does okay at operations and can do okay analytics they want the best operational data system for better or worse that that's postgress now right and best is not always in terms of performance could be a variety of reasons why it's considered the best and same thing for analytics you don't want a system that kind does okay in analytics and does okay on oper operational workloads. I want the best analytical system. And so that's always been the business challenge, the go to market challenge for for these for these hybrid systems. I think though data bricks might be in in a good position to potentially build pull this off because they already have that sort of market from the from the analytical side, you know, machine learning side and the the data warehouse side and now you just adding in this OTP side and so Click House, you know, could potentially produce pursue the same thing. That's getting kind of the weeds of of data system architectures. But I think everyone has always wanted this. The challenge though often isn't isn't just isn't always just pure engineering.
57:35 There might be the organizational reasons or human reasons why you can't achieve this. >> Yeah. Super interesting. So to play it back, you could abstract away the complexity to the user, meaning that you could have presumably an NLM that translates your query your like a yeah into exactly a router. however from a fundamental architecture perspective it's very hard to combine and from an organizational perspective it may not make sense because people want the best of breed not >> but postgress is the best of breed now in in many cases. So like betting on your front end is that as a postgress deployment that is compatible with postgress that might overcome that that issue. So to talk about other parts of the of the database market, we talked about vector databases. So like the is that thumbs up or thumbs down?
58:24 >> What is thumbs up, thumbs down like in terms of like is that a good business to get invest in or like >> are they going to be around now that you know everybody else and the brother has a vector search capability. One of the things that happened before when I was a professor is like I would always get you always hear rumors about who's doing well, not doing well through a combination of like either the investors or like former employees or students that maybe go to internships or whatever like or like maybe interview some places and and and they come back. So you get sort of bits of information from from everyone to kind of piece together what what the data landscape looks like.
58:55 Problem is now being click now I see I see everything like as an investor you see everything too. The one vector database company that I know is doing very well is Turbopuffer. and they are hyper specialized in doing vector search and at a cost performance ratio that's much better than than than everyone else. so I so I I don't think that the vector databases are going to going go away. I think that they'll evolve in two ways.
59:25 they'll have to become either a sort of general purpose system like a Postgress like a you know my SQL where they become the system of record where you're storing your the you're storing like the the the the two the original tupils plus the embeddings or the vectors for them or they become like an elastic search where there's like a a separate system they have a copy of the data that's being pulled from the operational side. and in that case they can live sort of comfortably as being this like additional thing you add on and if you want so the raw best performance of vector search you some cases you may have to go to one of these specialized systems. So I don't think that's that's going to go away. I just don't think I've seen predictions of like oh postress is going to die at the hands of a vector database. That's not happening right that's that's not happening.
60:13 >> Very much the opposite right. Yes, >> graph databases we mentioned at the beginning. So, you know, not to pick on them, but like Neo4j has been around for 20 years now. >> Sure. Yes. >> And this was supposed to be the the moment, right, for graph databases, AI. So, what's happening there? >> I say I have an outstanding bet with somebody on Hacker News where they said that by the year 2030 the graph database market was going to be was going to overcome the larger relational database market. And if this becomes true, then I will wear a shirt that says I love graph databases and I will use that as my driver's license, my my university ID. I'll put on my website till to the day I die, right? I'm pretty comfortable. It's 2026. We got four years to go. I that this is not happening. no, it's always been a niche market. And I think that because my perspective on the on the research side, the research shows that if you do certain things and implementing the engine, which Click House does do, DuckDB some of does does some of this as well. There's certain there's things you can do that allow you to do the traversals of graphs which essentially just joins self joins on on a table. You can implement those things very very efficiently. and you can easily outperform Neoforj and that's kind of like kicking that's like like Neoforj is like saying you're faster than Neoforj is like saying I'm faster than you know somebody maybe like that's that's you know that's in a wheelchair right you can run fast that's it's it's a low blow. So, but I was just saying like all the graph databases I think like even the best ones you're just not going to really system that has a bunch of these optimizations that that are in in the research and actually appearing in some of these systems now you're just you're going to lose what you will lose against a graph database if you're doing the graph traversal with the client side and the server side meaning like I you know I got to figure out what the the next node I want to go look at and go back to the client and that size the next node to go traverse. If you're doing that back and forth, yeah, they'll beat they'll beat you guys. But like I said, the SQL standard now supports property graph queries. Oracle has this, right?
62:07 They were a big pusher of this this extension of SQL allows you to do that traversal on the server side. So like graph databases, I think a horrible idea. There's no reason why anyone would actually want to use them. >> GPU databases. Yes, >> I know you have a special interest there. There was a cycle when the generation appeared then went away. There seems to be a renewal. What is a GPU database and what is your prediction? So GPU database is a data center system where the the execution end for queries is all floated to a GPU running on PCIe or running in you know in the same box or another box. so I mean the the history of people trying to build accelerators for data systems goes back to the beginning of data systems.
62:50 The 1970s they were called datist machines. So people would build specialized hardware to run sorting and and query execution operators. And that obviously died out in the early 1980s because by the time it take you to like design and fab new specialized hardware, you know, Intel or Motorola put out the next CPU or the hardware got better and just the gains you were getting went away. So hardware accelerators for databases basically died out in the 1980s. There wasn't a lot of activity in in the 90s, 2000s. you saw sort of the rise of people trying to do FPJAS for databases and every so often that comes back now. some of the cloud vendors do a little bit of these things but usually like to filter things on the nick on the network side of things coming in. so where GPU databases again the there was a bunch of systems in the 2010s that were trying this. We did a seminar series at the university.
63:42 We invited all the GP database guys come and give talks about you know what they were doing how they were you know why they were faster than existing systems and the big challenge at the time was like with those systems you had to put the entire database on the on the inside the memory of the GPU because if you had to go back up through PCIe it was just way too slow and a bunch of those startups sort of fizzled out some of them are still around but they're sort of specialized for for doing visualizations and then there was in the last year or so, Nvidia's basically gobbled up a bunch of these GPU database companies that were kind of like struggling along. And I was a I was an adviser for one of them called Voltron. but they they also picked up Heavy DB and so so Nvidia is all all going all in on this now. So it remains to be seen whether like the idea that you're going to build a CPUon database system like long-term whether that's going to still hold. I've heard again mixed reports like this is public Microsoft has you know their they have offerings now in the cloud that can be accelerated you know for for your data system can be accelerated GPUs for analytics. Another major database company I can't say who they are they looked at the economics of GPUs decided it wasn't worth it. so one of my former students now is is a professor at University of Wisconsin. They're now on leave at Nvidia. They have a project called Cirrus DB, which is not necessarily a new data system, but it's a layer in between an existing system and like the C and so they it supports like you know taking duct DB queries and running that down on on the GPU. I think they can do this in in Doris or Starox and data fusion and so at click house we've been potentially looking at this as well, but it's research. I don't like it's interesting to see whether how much you have to do translation between how click house expects things and how CUDA wants things to to be the data layout and so forth. how do you organize memory or share memory between the these different components? TBA remains to be seen whether this actually makes sense but certainly the there's a lot of research energy behind this and I I'm publicly I can say this like Nvidia is obviously pushing this because it'll sell more GPUs, right? because you know it's hard enough to get new CPUs.
66:00 Everyone's comput bound or memory is hard to get. I said the computing hardware is very expensive hard to get now and GPUs of all the things is the most expensive hardware to get and now you're going to say your entire base is going to run off GPU. I don't know if that makes sense at least in the short term. But if the performance improvements are quite significant and some of the research shows that is maybe it makes sense. Is there an emerging category or maybe a niche somewhere within a category that people don't talk about enough yet >> of databases?
66:29 >> Yeah, >> I mean you can always build new data systems for new hardware and but my track record on this is terrible in terms of like we've done a bunch of research on experimental hardware and it always gets canceled or like it's even not that experimental. It's like you know Intel had this Optane persistent memory stuff. We did my we did a bunch of research on building systems for that because like if you assume now your DRAM is persistent you know like mean like you pull the plug you don't lose anything that changes how you fundamentally build a data system. We did a bunch of work on that and then Intel killed that product line. We were doing another research on processing and memory hardware. So think of like DRAM sticks with like CPU cores directly on the on the DIM. So the data system now can say okay instead of pulling things from memory printing into my CPU caches and then I can compute things on them I'll just send the query down to the dim itself and run it there.
67:20 We were doing a bunch of work on this on this thing called opm that got bought by Qualcomm and got killed last year. so that that didn't work out. So there's always a bunch of work you can do on data system or new hardware. I would say that actually I I don't know the answer right this is one of the things I'm trying to figure out figure out at click house is like and we talked about this in the very beginning like are agenic workloads significantly different than than what humans or what existing applications do now and if so why or how and could you how would you change maybe the development of a data system to to to take better advantage of this that remains to be seen how you know how that works. I think there's always a bunch of problems in query optimization I think are interesting. That's remains the hardest part about data systems.
68:08 Incremental materialized view is another big challenge. Again, I but the these are not like things that no one else has thought of. People trying to do these things for for decades. so I think on the the AENX stuff is probably the most interesting and relevant thing to me right now is like what changes with these workloads what changes in the system architecture and I be honest I don't know the answer just haven't seen it yet. >> I'm asking because it generally feels like the database market is specific moment in its in its history meaning that there was a an explosion of activity in the 2010s. there's you know SQL versus NoSQL that that all evolution then there was the emergence of data bricks snowflake now click house but it seems that things have have slowed down a little bit in terms of explosion I guess you know stagnant maybe yeah no but we've been through this trend before right there's a lot of activity relational databases 1970s 1980s and then the 1990s again people sort of you know the market sort of solidified around the these major enterprises, the Oracles, the you know the IBMs, Terodatas. and then if you know if you if your only viewpoint of databases were from from those kind of companies and yeah it looked like it was been stagnant for years. but as you said a lot of activity in 20002 2010s I mean relative to AI then everything looks stagnant because there's in the history of computer science there's been nothing like that before like it's just like you know taking a cheetah giving a bunch of cocaine and putting in a Ferrari like the amount of speed that people are developing these things is insane so that everything looks looks slow or or dead or stagnant at the bat but the end of the day I think like the the volume is going to matter a lot. I think that one interesting question is is to your point of like cost and and efficiency like squeaking out the best you can the the best performance you can get for the hardware that you have trying to reduce that cost that's always an interesting challenge that you know could pursue but like the end of the day I don't think there's going to be a a massive change in what data looks like that requires us to throw everything away that we've we've known about databases in the same way like you wouldn't come up with a new notion of arithmetic or mathematics to replace 1 plus 1 equals 2. The relational model itself is the foundation of how you want to represent data. And it's just you can you can vary the implementations of that. And so there's certainly a lot of work to to make that these systems more efficient for this. but I don't think you're going to throw everything away and like you know you know the agents need some kind of data system that you've never even conceived about. Now the end of the day it doesn't make sense. so I don't think that part changes. I think I said always new hardware. I think there's certainly you know improvements that can be done for SQL. There's always going to be people trying to replace SQL. I think that might be a lost cause. although SQL could end up being like how in the same way that like you know people don't write assembly anymore. SQL might end up being like that because text to SQL works so well.
71:21 I think the agent is super interesting and I think getting better performance is always going to be a at least from my perspective a fun challenges things we we can pursue. You can imagine a crazy world where you say I don't need a general purpose data system anymore for every single application. I want to vibe code exactly a data system that does this for this one thing can then be hyper specialized. You kind of do this now with code generation or just in time compilation for for for some aspects of queries and some systems do that click house postgress umbra out of from the Germans but like hypers specialization and making that be sustainable might be a bigger research question going forward >> so it's this I don't know if it's a paradoxical kind of situation but on the one hand it's a bit of a stagnant industry right now in terms of evolution at the same time As we've hopefully established through the conversation, the the the layer itself is as important as ever, which is one of the reasons why your friend Larry Ellison is the own close to the richest man in the world or was >> he was he's down as of today.
72:28 >> I mean, Oracle is is, you know, a lot more than just a database company, but he's still the the the the corporate. >> I mean, the headquarters is a shape of a database. So, like so like >> Yes. Yeah. Yes. Yes. Yes. but but if nothing else that shows like the fundamental importance of of of databases in the >> I would say like no one is trying to make a serious attempt to replace Linux right I mean yes there's there's niche operating systems for different environments different hardware and things like that but like no one's trying to build a a in a significant scale replacement for Linux entirely like freebase that's predates from the BSD or from Berkeley it predates Linux so that's that's why it's still around but like but for databases there's always new ideas or people think they can do things better than others. So like I wouldn't say it's stagnant. I I would just say that like the a lot of the energy is going to be focused on Postgress in the short term. and there's certain things we can do to improve it. but meanwhile I still think there will be these sort of specialized engines and I don't I don't know what the next workload is. Again, it's not clear to me agentic workloads are significantly different, right?
73:33 Because they're modeling things that that are how he was design applications like the vector lookups that significantly different than what was in the past. You that you needed additional operators. You didn't need additional indexes to do those things. It's not clear whether what I don't know what the next thing is, but I'm there will be something. There always is. >> So, maybe to finish on on that. So the next the next five years. So we don't know necessarily what that looks like. But so in the in the meantime, does it mean the the current company the click houses of the world like just keep getting better and established because the fundamental need exists and that's not a new wave of smaller startups?
74:07 >> So so when you say better though like there's two two notions of better. There's like can can existing query run faster. Yes, we click house has room for improvement. We can improve that. Other stuff can do the same thing. But like I said, the architecture that that a high level or what click house is doing is similar to what snowflake did in 2013 data bricks is basically copying existing some of the existing ideas as well like everyone is kind of doing the same thing at least for analytics and what really matters and one of the things that snowflake did right was part of the success is like it's the stuff around the data system like the user interface developer experience like you know building to to ingest data interoperate with other other things that part they they did really well. So it's almost like the the things around the scaffolding of the foundational of a database management system that actually matters a lot and one of the things that clicks is actually you know this not to me it's not research but this is one of the things they are doing better over time and helps helps expand the reach of what what you can use the system for. So the core kernels of the system like that certainly be improved but the fundamentals of what clicks is doing versus other systems at the smallest level they're they're pretty similar but the stuff around it is is matters a lot and I think click house does some things certain well things can do better snowflake does some things well can do things better same data bricks and the other other other guys >> well thanks for spending time with us congrats on the newish job that you also described in in a app analogy, right?
75:36 What what was it? >> I mean, it's like when you look at Run the Jewels, it's like taking LP and and and Killer Mike, like those guys were established on their own. You put them together, you know, it's like it's like people putting put peanut butter and jelly together for the first time, right? So, like I like to think me and Class getting together is is is like that. >> Okay. Well, this is now officially on the record the most hip-hop referenceheavy episode of the Mad Podcast.
76:00 >> I'm glad you found that in in there in in the blog article. That was my It wasn't a test. That was my cuz cuz you know I wrote the blog article was I was a floating it out there to see what Click House would would be okay with. Obviously I didn't want Larry Ellison in there but like you know want to see like hey you know like if you get Andy you're also getting other other baggage.
76:21 >> All right Andy this was fantastic. Thank you so much. >> Yes. Thanks Matt for the helping. >> Hi it's Matt Turk again. Thanks for listening to this episode of the Mad Podcast. If you enjoyed it, we'd be very grateful if you would consider subscribing if you haven't already, or leaving a positive review or comment on whichever platform you're watching this or listening to this episode from. This really helps us build a podcast and get great guests. Thanks, and see you at the next episode.
Summary
- Databases are evolving with the integration of AI agents that can autonomously create and modify databases.
- The rise of vector databases has changed how data is queried, emphasizing approximate similarity searches over exact matches.
- Guardrails are essential to prevent agents from making destructive changes to production databases.
- The market is seeing a convergence of OLAP and OLTP systems, with companies like ClickHouse expanding their offerings.
- AI can significantly aid in database management, optimizing performance and automating tasks that were traditionally manual.
- The role of traditional database administrators is changing, with a shift towards higher-level data modeling and strategy.
- The future of databases may involve hyper-specialized systems tailored to specific workloads, although the foundational principles of relational databases remain relevant.
- The overall database market is not stagnant; rather, it is adapting to new technologies and demands, particularly in the context of AI and agent-driven applications.
Questions Answered
What is the current state of databases in relation to AI?
The discussion introduces the concept of databases as essential components in the age of AI, highlighting the rapid development and integration of AI with database technologies.
How do file systems compare to databases?
File systems can be viewed as databases, but databases offer significant advantages in terms of data management and safety. The speaker emphasizes that everything can be considered a database, including simple file directories.
What advancements have been made in database management over the years?
The speaker reflects on the challenges of database management and the evolution towards self-driving databases, which automate optimization and maintenance tasks that were previously labor-intensive.
What concerns exist regarding the ClickHouse database?
There was initial skepticism about ClickHouse being too good to be true due to its rapid development and advanced features. The speaker discusses the competitive landscape of database research and the challenges of building a robust data system.
What is the role of GPU databases in modern data systems?
GPU databases leverage the power of GPUs to accelerate query execution, representing a resurgence in interest after previous cycles of development. The speaker discusses the historical context and potential future of GPU databases.