Section Insights
Introduction to Agents in Accounting
What is the role of agents in accounting and how does Basis approach it?
Mitch, co-founder of Basis, introduces the concept of agents for accounting, emphasizing the need to understand what an agent is and how it functions within the context of accounting tasks.
- Basis is pioneering the use of agents for real accounting work.
- Understanding agents is crucial for grasping their application in accounting.
- The discussion will cover product development, deployment, and internal operations.
Understanding Agency in AI
What does it mean for an AI agent to have agency?
Mitch explains that an AI agent requires context and information to make decisions, highlighting the balance between developer input and the agent's autonomous decision-making capabilities.
- AI agents need context and historical data to function effectively.
- The balance between developer control and agent autonomy is essential.
- AI's superior memory and processing capabilities can enhance decision-making.
The Importance of Ontology for Agents
Why is a coherent ontology crucial for AI agents?
Mitch discusses the necessity of a stable ontology for agents, as they lack the inherent memory systems humans possess, making it vital to create a structured environment for them to operate in.
- Agents require a coherent ontology to navigate their operational environment.
- Human memory systems are complex and not replicable in AI agents.
- Building a stable world for agents is essential for effective interaction.
Building for Agent Competency
How should tools and environments be designed for AI agents?
Mitch emphasizes that as models improve, the environments for agents must be designed with a coherent ontology to allow for diverse workflows and effective evaluations.
- Designing tools for agents requires consideration of their unique needs.
- A stable environment is crucial for effective evaluation and performance.
- Agents should be treated as a new type of customer in the development process.
Deploying Intelligence in Organizations
What challenges arise when deploying AI agents in organizations?
Mitch outlines the transformative impact of deploying AI agents on organizational structure and workflow, highlighting the need for support in integrating these agents into existing processes.
- Deploying AI agents changes the nature of work and organizational structure.
- Organizations need guidance to adapt to the integration of intelligent agents.
- Hiring practices should reflect the unique requirements of building coherent ontologies.
Transcript
0:04 I'm Mitch. I'm one of the two co-founders of Basis. As Matt just mentioned, we build agents for accounting. I think we're the frontier company at the moment that is deploying agents that do real accounting work in real firms. Instead of going talking a lot about accounting and and kind of going deep on that, I thought I would maybe take a step back, talk about what we are doing to build basis for what I like to call the AGI era. And I'm going to talk about that both from product perspective, from a deployment perspective, and internally inside of basis. But just to start, I think it might be good for us all to get on the same page as to what an agent is because I suspect that not everyone in this room fully really clicks in their head, even if they don't want to admit it, and that's okay. I don't think it clicks in the heads of even lots of people who started basis in the beginning. So let's take a very basic example. So this is an accounting example but you could do anything. So imagine that I have let's say I have some bank transaction that comes in you know people have heard of like categorization and stuff in the past right I have some bank transaction that comes in you have there a brain assume it's just like a like a genius right this is it's like the smartest AI in the world and you ask the smartest AI in the world okay you have this transaction and inside the transaction there is a date there is probably some text that says hey here is you know the description if anyone's ever seen you know on their bank statement right some weird text and you say hey pick one of these accounts and you say this to this really super genius, what is the super genius going to think? So, if you've ever actually done this work before, which probably unlikely many of you have, it's going to think, wait a second, is this company a construction company? Is this company e-commerce company? It's going to think, wait a second, I'm not familiar with like, you know, the Bob's Berser vendor name or something in the description. What can I do with that? And so it's going to want to do things because it is a genius. It knows that you are not giving it enough information to do its job. But because you said, "Hey, transaction, pick one result." You don't give it the choice.
2:08 It has to do the job. And so because it has to do the job, it would then go and do it. But if you could ask it, it probably wouldn't be happy. And the key here is it's being smart. You are being dumb. You are not giving it the options. The way to think about agents is agents are where you give a model something that's really smart agency. The agency to make decisions not just about what the output is, right? If you go back to here, here it's deciding what it is, but how to get there. Because if you just say what it is, it's going to say, well, I need more information. I might want to know what's the name of the company.
2:44 What do they do? I might want to go look, oh, in like previous history, have there been any similar transactions, right? Maybe the exact string, maybe some other information. I might want to go to the internet and look up things. And so all these different things require agency. So that is the way to think about agents. And and I think this balance of you, and by you, I mean effectively the de the developer deciding what it should do versus it deciding what it should do. That's the constant balance. The it being smarter than me. I wasn't kidding. Like I actually would argue that GPD5 in many ways is probably smarter than me. It knows more about most subjects. It can read a 100 pages in less than a minute.
3:24 I don't know if anyone can do that here. It has a far larger working memory than I do. By the way, to understand working memory, think about like a waiter going to a table and taking everyone's orders without writing it down. How many orders could it take? I don't know, maybe seven. Think about how many you could remember. If you were to count up the amount of tokens that is, it's like somewhere in the order of a couple hundred tokens, right? GBD5 has a 400,000 token context window. It can keep a lot more in it working memory than we can. It almost certainly can write a better Python script than I can perfectly. I mean, just now GBD5 just got 11 out of 12 on the gold IMO. So, it's smarter than me at literally everything. Why do I even have job postings on my site? Why are there job postings all around? What's going on?
4:07 The key here is that it's not effectively embodied. It doesn't have an ability to have medium-term memory or sort of long-term memory, right? It doesn't learn continuously over time. I think, you know, given people are here, people have probably read, you know, symposent talked about this. Dark a good blog on this maybe like a month or two ago. And because models right now don't have ways of continually learning, that is one of the big maybe the biggest gap between whatever you want to call it today versus literally it can just go do everything. And so because of that, because of that gap, the solution has to be context. Why does the solution have to be context? Imagine if you go and ask, let's say you have a coding agent, people are probably familiar, and you go and you give it some sort of task, and you say, "Hey, add a filter to some dashboard. How does it know what repo you're talking about? How does it know what page you mean when you say the word dashboard? How do you know which components you wanted to use for that editing? How do you know what's like the pattern that you want? How do you know what the word tags even means?" is like tags refer to the actual table in the database. Is that the name of the component? Is that the name that people just use? It doesn't know any of this.
5:14 But when you speak to a coworker, it's obvious. If I say to a coworker, hey, can you go fix that thing? I don't need to ask her what the repo is. They know. Agents by default don't. Which is why the most important part of agent building is coherent stable ontology because you're building a world for the agents to live in because they can't recreate that world themselves. Humans have this memory system. In fact, we don't know exactly how it works. People think sleep has a lot to do with it. You compress memories down. There's a reason that you can't remember much of when you were 5 years old, but you can remember a decent amount of when you were yesterday because evolutionary systems have told us that what happened yesterday is somewhat important to you. what happened when you were five is not that important. That's a memory compression system. Agents don't have that. Not in a way that is at human level or anywhere close. I'm going to try to quickly go through three different things. Internal agents, how we do it, and this is where I think about building a company. Two, how we build product.
6:10 And three, how we actually deploy the intelligence externally. All of which will go back to the same pattern of cohesive ontology for agents to interact. So our internal agent team we call Atlas. The job of Atlas is to figure out what is the right context architecture and context management for the information internally. Right? You think about your docs and and you know in any place you work right now if you took a paragraph in some internal doc and you deleted it, people probably would never even notice. Why would they not notice? Because they have memory systems. Either they've read it before or someone told that to them or they just kind of know it. With agents that's not true, right? If I have some ontology and I go and I delete a paragraph that is like crucial to understanding their world, now they'll do something totally different. In fact, it's much closer to thinking about a codebase. If I were to go delete a paragraph in my like main.py file, my code would break. If I delete a paragraph in my internal context, my internal agents are going to break. And the blast radius as agents take on more and more is gigantic. You think about hundreds, you know, running at once.
7:12 Building coherent context is crucial. And having the right integrations and agent development is important. And you might ask, why do this ourselves? Why is there not a vendor for this? Because this is the core competency. It's sort of like saying, why is there not a vendor for writing down the practices of your company? That's what this is. You're defining what it means to work at Basis, but you're defining it in a way that is legible to agents, not just to humans. How basis builds product. Couple things that I think matter a lot. Number one, you want compounding systems. So we think about building horizontal layers that as the models get better, as your context management improves, the environments that you're building allow it to do more and more types of work rather than saying, "Hey, I'm going to go and build for workflow A versus workflow B, where's workflow C." All this kind of relates to the ontology, right? It's not literally just the context. It's the type of tools that are there. It's the type of data that has access to. It's the world that you are defining. Evals, evas, evals. I know there's been some like discourse on Twitter that some people want eval.
8:13 You're not a serious person if you're doing production stuff for real customers who are paying real money and you don't have mass amounts of evals. That is the key to building extremely competent agents at scale. It's not possible otherwise. I don't I will disagree with you if you if you think that's wrong. You have to build for the agents, right? Going back to our first example there when the agent has those tools you're not building the tool for like when you're writing the tool and you're defining it. Traditionally when you were doing this the architecture you had essentially two stakeholders. You had the customer who was using the platform and you had maybe other engineers who would in the future come and edit this information. So that's why you wanted to componentize things right abstract out functions things like this.
8:57 Now you have a third customer. The customer is the agent. So you have to define your tools as part of this ontology in a way that is coherent, that makes sense, that is somewhat stable because it is very hard to have good evals if you're constantly changing the environment, right? Imagine if every single day you walked up the laws of physics changed by 5%. It'd be pretty hard to not like stub your toe all the time. And last, applied research matters. I've heard some people say that doesn't matter. You know, we don't care about research. going from I call an LLM to I can automate massive massive amounts of some like some knowledge work that is research that is not a solved problem so there's lots of research that we do whether it's about memory whether it's about context compression whether it's about reducing rot whether it's about sort of tool ontology context engineering post- trainining environment engineering etc etc etc lots and lots of research and eval are key to this let's go talk about deployed intelligence so we built the first deployed intelligence team in the world to my knowledge and by that I mean the first team with the name deployed intelligence and I think the key here is realizing that if we believe that AI will get here faster than most people realize and you can argue AI is already here but the fusion will be slow because when you don't have really good medium-term memory systems you need to set up the environments right and even if you did have good medium-term memory systems you still don't automatically diffuse throughout human processes and so you need to have have as a core competency the ability to deploy and make the intelligence usable every single day. And the thing is deploying intelligence is different than what it meant to deploy software because you're now changing a core nature of the way people used to work. Used to work is you have labor and they go and use tools.
10:44 Now suddenly you have these agents that can take on some amounts of labor. So how does that change the way your organization is structured? How does that change how many people work on a certain project? How does that change what potential revenue lines you can go for in the future? It actually changes everything. And most of your customers are probably not experts in this because it's pretty on the frontier. So, you need to help them. And that's why I think having a deployed intelligence team is really critical. And and I think I just want to think to call out. I know it might sound like FDES in like a balancer context, but this isn't about writing bespoke code. It's about helping people put it together bespoke processes, right? Imagine that you were going to an organization and saying, "Here's 300 new employees. What do they do?"
11:24 Not just 300 new employees. 300 new employees that are smarter than all their employees at everything, but don't really have great memory. It's weird. It's alien. You need to help them. Okay, last just a plug. Who we hire? I think this is another thing people don't talk about enough, which is that because of the unique constraints of agent building, you don't you want to have maybe different profiles of people. I'm spec specifically going to talk about engineering here, but this applies, I think, in all areas. So, number one, if you're building coherent ontologies, you have to be system thinkers because you're not just saying like, hey, let me go hack this thing on the side. You're actually building a coherent system over time. By the way, that's true both in the systems you're building for the agents that you're deploying, but also for the systems of the code itself because what about all the coding agents you want to be operating on your codebase? That better be pretty coherent, too. So, you want engineers, not coders. That's the way I like to think about it. Two is you need or three I guess you need people who are really curious who are really curious about what the data is what they're building on top of I promise you you will not be able to build good ontologies and good systems if you don't even understand why certain data is many to one you need to understand the data to build good abstractions even your variable naming will be incorrect and that all that stuff adds up and you also need to be curious about using the frontier using agents and lastly you need to be collaborative because we're all on the frontier. We're working together. You meet interdisiplinary. So that's that's the end of my talk. Plug come work at basis.
Summary
- Agents require the ability to make decisions based on context, not just provide outputs.
- The balance between developer-defined tasks and agent autonomy is crucial for effective AI deployment.
- Coherent ontology is essential for agents to understand and interact within their operational environment.
- Internal teams, like Basis's Atlas, focus on context architecture to ensure agents function correctly.
- The development of products should prioritize compounding systems and extensive evaluations (evals) to enhance agent capabilities.
- Deploying intelligence involves changing organizational structures and workflows, as agents can take on labor traditionally performed by humans.
- Hiring for agent development requires system thinkers, curious individuals, and collaborative team members to build effective ontologies and systems.
Questions Answered
What is the role of agents in accounting and how does Basis approach it?
Mitch, co-founder of Basis, introduces the concept of agents for accounting, emphasizing the need to understand what an agent is and how it functions within the context of accounting tasks.
What does it mean for an AI agent to have agency?
Mitch explains that an AI agent requires context and information to make decisions, highlighting the balance between developer input and the agent's autonomous decision-making capabilities.
Why is a coherent ontology crucial for AI agents?
Mitch discusses the necessity of a stable ontology for agents, as they lack the inherent memory systems humans possess, making it vital to create a structured environment for them to operate in.
How should tools and environments be designed for AI agents?
Mitch emphasizes that as models improve, the environments for agents must be designed with a coherent ontology to allow for diverse workflows and effective evaluations.
What challenges arise when deploying AI agents in organizations?
Mitch outlines the transformative impact of deploying AI agents on organizational structure and workflow, highlighting the need for support in integrating these agents into existing processes.