Managed Deep Agents - Conceptual Video

0:00 In this video, I'm going to go through why I'm so excited about managed deep agents, all the problems I think it solves, and how you can use it. I do really think that this will make it far more easy than before to build powerful agents and actually run them at scale. And so, I'm excited to dive in. I want to get started by talking about what exactly an agent is. So, the core part of an agent is the business logic that you provide. This is what makes one agent different from another. So, prompts, skills, tools, sub-agents, all of these are really business-specific logic that goes into defining how the agent will act. This is something that you have to provide. This isn't something that any framework or any provider or any model can give you.

0:41 You have to provide this to the agent. The next layer is the harness. So, the harness is what's responsible for context management. It takes all of your prompts and skills. It It's in charge of orchestrating them and passing them to the model and getting responses back and and and running in a loop. We've built an open-source model-agnostic harness called deep agents, and this is really powerful for getting started building agents with a really seamless experience. But then there's a whole 'nother part.

1:10 There's the infrastructure. So, you've got your business logic, you combine it with your harness, but you still need to run it in production. And so, there's a whole runtime, sandbox, and a bunch of other infrastructure that's needed to actually go from running something locally to serving it to thousands or millions of users. And so, what is so hard about deploying agents? What's so hard about this infrastructure layer? There's a number of things. First is the runtime. So, when you're running an agent not on your laptop, but in a web server and you're serving production traffic, you need things like durable execution and fault tolerance, which basically means that if it errors halfway through the run, you don't start over, you can just resume from there.

1:47 You need things like streaming to make sure that you've got this continuous stream of events that that a front end can listen to, queuing and run cancellation and rollbacks for when you have a ton of bursty traffic coming in. And so it's this whole production runtime that you need when moving from your laptop to the cloud. Another thing you need is sandboxes. So you don't need them always, but if your agent's going to be writing and running untrusted code, which a lot of agents are these days, you're going to want sandboxes. So that's another component.

2:15 Context management. So context management is done by the harness, but where does that context live? Where does the context of instructions and skills and other rag type things actually live? You probably want it to live someplace where it can actually be accessed and maintained by non-developers. Often times the people best suited to edit this context may not actually be in GitHub. Evals. Evals are really hard. So evals are something that you're going to want to do to benchmark what how your agent is performing, to compare different model choices that you're considering.

2:48 If you're thinking about changing a prompt, you're going to want to run some evals. And it's really hard to build and write and maintain these. Auth is really hard, knowing who is calling your agent and then also what they're authorized to see and do. UX is really hard. This is actually one of the most underrated things I think about building agents. So you've built this agent, but how do you expose it to users? And then finally memory. When your agent's running, what is it remember?

3:12 And how does it remember? And how is that stored and how is that surfaced back to the agent in future runs? So these are all the hard things that come with deploying agents and going from something that you're just running locally to something that you're running in production. And so what managed deep agent does is it takes that business logic that you provide and it bundles up the deep agent's harness with infrastructure and provides a really seamless package to get both of those in one, and that makes it really easy to just go from your business logic to an agent running in production. So again, to repeat, it bundles up deep agents, our harness, with a bunch of infrastructure, and and go into what that is, and makes it really seamless to run.

3:55 So, what exactly is that infrastructure? If we take a big higher-level look at what managed deep agents does, this is roughly how it's architected. At the core of it, you've got the agent loop, and this is deep agents. So, this is the agent harness itself. The first thing we do is we're going to run this agent loop in LangSmith deployment. So, LangSmith deployments will give you all the things that you need for runtime. So, durable execution, fault tolerance, streaming, all this comes out of the box for LangSmith deployments. And so, this is really the core of how agents are deployed.

4:25 You've got a bunch of your business logic. And so, tools and middleware, these are loaded up in the agent loop in LangSmith deployments. What about the other stuff? What about the context? So, this is where it gets kind of interesting. So, we actually store instructions and skills and other context in what we call context hub. This is a nice little UI where people can go in and edit the context, and then it's automatically reflected in the agent. You don't have to do another redeploy. It doesn't actually live in the code, it actually is stored in the context hub. And that's really powerful again, because you can edit it and have it immediately accessible to the agent.

4:54 So, this gets pulled into the agent loop without having to redeploy. The other part of context is memory. And so, this goes both directions. So, the agent loop can write to memory, and then the agent loop can also pull from memory. And this is also stored in the context hub, so that you can audit it and see what's going on. Sandboxes are really useful for running untrusted code. We connect to LangSmith sandboxes and give agents an easy way to again write and execute code.

5:22 Schedules are are are also pretty interesting. So, agents will be more and more proactive in the future. They'll run in the background, whether on schedules or on events, and do things and then surface them to you. And so, schedules are part of the Schedules are basically cron jobs. They kick off the agent with a certain input at a certain time, and then they run. And so, schedules are naturally built into LangSmith deployments, and they live there.

5:44 This is where we start to get into some things that are in the managed deep agents package itself. And so, one of the things is connectors and an easy way to connect external services via MCP. So, MCP super common and super popular protocol for connecting to all sorts of different servers, Slack, Notion, Linear. And so, managed deep agents support specifying a bunch of remote MCP servers and connecting to those. Managed deep agents also adds in a concept of off. So, it will be able to identify who is calling in to the agent so that you can make sure they are who they say they are and then and then do things with that. And then in front of that, we have a concept of channels. So, channels relate back to the the idea of UX and how UX is hard. You you might not always want like a a web app for your agent. If you do, by the way, we've got a bunch of endpoints in LangChain deployments that make it super easy to spin up some some web app endpoints, but you might want to bring it to other places where people already are, other channels where other people already are.

6:41 And so, that's what channels are. The first one we're launching with is Slack. So, you can easily create a Slack bot and bring it in there. And then lastly, evals. So, evals kind of sit separately of this, but these are a super important part of developing and building agents. We use Harbor, which is an open-source eval format for all of our evals. So, this is a a standard industry standard industry leading format for running evals and and and we make it super easy to build Harbor evals for all of your managed deep agent deployments. So, with that conceptual overview, hopefully you understand what managed deep agents is.