Agent Observability: Gain Insights into Tool Calls & Run Stats

0:00 If you've built an agent that can do tasks autonomously, you might be wondering about the actions your agent is actually taking. A powerful agent can take different paths depending on inputs. It can make different tool calls and also call different sub aents. But so far there hasn't been a great way to get aggregate visibility into what your agent is doing in Linksmith. So today we're launching two new levels of visibility to gain insights into tool calls as well as agent trajectories. And you don't have to be using Langraph to take advantage of this. Just to show you how complex an agent might get, here are the various paths that Carl, who's our customer support agent, might go down. So, let's explore what these new metrics look like for Carl. Let's go down to the tool call section of our pre-built dashboard for Carl. Agents are only as powerful as the tools that you give them access to, and powerful agents often have multiple tool calls. These can indicate the actions that an agent has taken. So you might want to know things like which tool is getting called the most often. Or if you launch a new feature that uses a new tool, you might want to know what the usage of that new feature looks like. These tool call related graphs help show exactly that. We can see here that the most popular tool that's used by Carl by far is this search memory tool.

1:29 This helps Carl get context about Langraph, Lang Smith, and Lang Jane, as well as different playbooks that he can use. Other things about these graphs that are really helpful is that you can find which tool is taking the longest time. This might indicate where your users have to wait for a while to get a response. And the other thing that's pretty neat is you can see which tools have caused the most errors. The nice thing here is you can actually click into the graph and go into the runs for this project. So I can see here there is this error and my next step might be to go in and actually debug what was going on here. The next type of metrics that we've added are metrics about run types that help show agent trajectories.

2:19 This can be really insightful to help understand the paths that your Asian takes at a high level. You can see that all of these graphs are filtered down to depth equals 1. And so what this means is that it'll include any runs that are the immediate child runs of the trace. To show you what that means, if I go back to my trace over here, all of these runs here are the immediate child runs to the trace. They do have nested runs, but those are not included in these charts. If you're using langraph, these runs might be indicative of distinct langraph nodes. So, for example, if you have an agent that can take three different initial actions depending on input from a user, you can use these charts to see the distribution of runs. So, let's take a look at this in the context of Carl.

3:15 You can see here that the most popular node that Carl goes into by far is this augment unthread event which makes a lot of sense because this event is used to help bring contacts into a ticket. Things like your org ID or workspace ID. The other things that you can see in these charts are pretty similar to what we had in the tools charts above. So things like which run or which node is taking up the most time as well as the error rate grouped by run name and both of these might indicate areas where you'd have to spend more engineering time in order to improve the reliability. So all of this is live today. Would highly encourage you to give it a try with agents that you're building. We want to do a ton more around visibility in agents. So, please let us know thoughts and any feature requests in our Slack community channel.