Day 4: Harness Engineering - Tools and Tool Calling

0:00 In this video, we're going to be talking about the second component. By the way, this is component two, not component one. Yeah, forgive me for copy and paste. Yeah, this is component two. The tools. The second component that make up the harness, and that is the tools. Now, what are tools? Tools are a set of functions that the exposes to the model. The anest gives your model access to these tools. The model can then request a tool by emitting a structured tool call. It emits a structured tool call that suggests a tool with the arguments that the tool should be called with. The harness then executes the tool and returns the result and that result is passed back to the model as we've seen the loop do. The loop is going to take care of passing the result back into the model. So that's basically what tools are. Tools are the model's surface area into the world. Every gap from the first lesson that is execution, fresh knowledge, environment and all that, a tool helps to close it. A tool that the anest provides helps to close any of those gaps that we discussed when we were talking about the gap between translating our models into aentic assistant. And yeah, that's basically all you need to know about the tool.

1:16 They are just functions that the model can describe how they are to be called. The anest then calls them and they execute within an environment and their results are passed back to the model to continue its conversation as orchestrated by the loop. Now what does a good tool design look like? First you need a set of small and well-n named tools that compose very very well. Yeah, small tools focused on a particular action well named so that the model can recognize them and recognize their function and tools that compose together. For example, you have a bash tool and a read file tool. These are two small tools well named and they compose together. This is better than having like 15 specialized tool. There's a lot that can be done with a simple bash tool and something like a read file tool or a write file tool. Also, you need your tools to produce clear structured outputs. You need them to produce clear structured outputs. Even if they're going to be outputting errors, it needs to be clear and well structured. This allows their output to be easily read.

2:23 The model has to be able to differentiate between success and failure from what the tool is giving it. The model has to be able to know whether the tool failed or it succeeded and be able to read the result clearly, whether it's a success result or a failure result. Also, your tools need to be sized so that the model can use them very very well. They need to be sized for efficient use. So, let me break this down. Your tools should be sized in a way whereby if you want to perform a simple operation like read file, it doesn't have to call five different tools. Each tool should do enough that the model is not drowning in tiny operations. You want to read a file just to read a file. Don't have a tool that is called get file name. You have to call get file name first. Then you have to call get file extension. Then you have to call name file which taxs on get file name and get file extension and joins them together. Then you now have read file content all just to read a file. No, don't over split your tools.

3:24 When an operation is simple, create a simple tool around it. Don't break it down and make it more granular. Don't go crazy on how much you're splitting down your tools. And as you're doing that, also don't make your tool do so much that we cannot pinpoint the actual step where something went wrong. For example, you create a tool called analyze web page. And that analyze web page is doing so much. It's fetching the web page. It is passing the content. It is detecting forms. It is filling forms. It is getting form outputs. It's calling APIs.

3:58 It's querying databases. It's querying knowledge sources. And it's just doing so much that when such a tool fails, the model can't clearly say that this is the step that failed. This is the part that failed. It will just say the tool failed to just say the entire tool failed. And we cannot like detect which exact point that tool failed. So focused tools that are well sized for their specific operations. Don't create robust tools that do too much. Composability over monoliths. Composability over monoliths.

4:30 When it comes to defining your tools, make them sized for efficient use. not too granular that we now start calling 10 different tools to perform one operation and not too bloated that when they fail we can't actually pinpoint the step where something failed. So those are some nuggets you should keep in mind for constructing your tools. Now to wrap things up let us do our case study looking at an agent in the wild and that'll be cloud code. We're still going to be going with clot code. Now cloud code has a tool surface that is small and deliberate. In cloud code you get to see small focused tools like read, edit, bash, glob, gp and many more. Now compare this to an agent with like 50 bespoke tools. Almost every time you have another peculiar operation you want to perform, you have to create another bespoke tool for that particular operation. But cloud code gets to do a lot with these simple tools. Read, edit, bash, globe, grab. It gets to do a lot with this set of simple tools because the cloud code anest design bet is that a competent model with sharp generalpurpose tools beat any constraint model with specialized tools anytime.

5:49 Anytime. Small sharp general purpose tools beat specialized tools anytime. So, make sure that you create focused tools. Just as we've mentioned our best practices, focus tools and do your best to stay away from creating very very customized tools. Like I said, always favor composibility. The way CL code has decided to create its tools this way is an actual decision that they decided to go with and it shapes the entire feel of the product. Even a tool like open claw or rather a harness like open claw. I remember one of Peter Steinberg's interviews where he was talking about asking openclaw to do something that he didn't actually build a tool for it in open club. He just has all a lot of these a bunch of these general purpose tools and he just asked his open claw agent to I don't know was it do a particular research or do something on his computer. I think he wanted to just prompt his agent from his telegram to do something on his system. But he had not built any specialized tool for that and the agent was still able to get it done and by the time he looked at the logs he realized that the agent was able to use a lot of his base tools, compose them together, create a plan and be able to get that task done. That is the power of having focused general purpose composible tools over highly bespoke ones. So yeah, going forward as we build our file system, we have a chapter dedicated to file system. We build our file system. We have a chapter dedicated to having a tool registry. So we're going to be building our tool registry and we're also going to be adding our bash and code execution tools. And as this course progresses, you're going to be seeing how we are building all these focused tools and how they are giving us the best performance from our harness.