Day 10: Harness Engineering - Case Study: Claude Code

0:00 So, throughout previous lessons, we've been looking at a real harness, like Cloud Code. A real harness out there in the in the real world, and comparing or rather looking at how our components exist in this harnesses. We looked at Cloud Code, we looked at Cursor, we looked at Codex 2, we also looked at Codex. But now, this video is for you to see everything all at once within a single popular harness. And I've chosen Cloud Code because it's the most popular.

0:32 Cloud Code, Cursor, I don't know which I I don't have the numbers, but I I believe currently Cloud Code is more popular than Cursor, so we're going to go with Cloud Code. So, we're going to be looking at each of the components that we've looked at and how Cloud Code implements these components. Just just to kind of decompose a real-world agent into the components that we just discussed. So, first is the loop. First is the loop.

0:55 By the way, if you like this diagram, yeah, I do too. Gemini did this. This was from Nano Banana. I was trying to draw out this, then I just remember that Nano Banana is quite good at creating diagrams like this. I just basically took my notes and dumped it into Nano Banana, and Nano Banana took my notes and gave me this diagram. I had to do a little bit of tweaks, but yeah. I still got something good, and that's why you see this AI symbol here.

1:22 Anyway, yeah, just giving you background just in case you want to create these type of diagrams. So, first we have the loop. We have the loop. And like we know, the loop is the agent's outer orchestrator. Is the agent's main orchestrator. Mostly uses the react pattern. That's the most popular pattern. That's also the pattern we're going to start with in this harness build in this course, I will continue to improve on it. And what it does is that it keeps calling the model, it keeps calling the model until the task is done, until there is there's always a condition. There's always a condition that needs to be met.

1:57 So, we have to check. So, we say keep scrolling until the task is done. So, there has to be criteria called a completion criteria that will continue to check after every turn. If it is met, the result is going to be returned. If it has not been met, then we keep calling this react. action and observation process. So, that's what the loop does. It calls the model, decides action, executes tools, and inspects the tool, checks whether the result is gotten or a step budget has been hit because that's one thing that you also want to make sure that you have a step budget. You don't want this loop to continue forever. So, for every task, you want to give it a step budget like 15 15 or 20 or whatever you feel is reasonable for >> >> the subtasks to be completed and for you to get something reasonable. And if it doesn't complete it in those number of turns, it will just package what it has and return it as the result so that that can be passed to the next execution, which is fine. That one will just take off from where the previous one stopped. So, that's the loop. Next, we have tools. We have tools. Cloud code has simple tools. That is very simple tool surface as we've discussed. Tools like read, bash, glob, grep, a simple task management tool for writing out your plans when you give it a task. It also has an MCP connector and Cloud's main ideology is to have simple tools that are composable so that you can get so many customizable tasks done using simple tools. It's easy to compose simple tools than to have all these bespoke tools and you have to keep creating a bespoke tool for every single task that you want to perform. So, these are Cloud's basic tools and it also adds more of these base tools that are easy to compose. That is basically how Cloud operates. I can always add your own skills. More on that later. So, these tools are basically sharpened for specific tasks and like I said, easy to compose.

3:47 Then we have the context. We have the context. And Cloud implements this with different strategies. There's a system prompts that we are familiar with. We also have the conversation history. We have the tool results, which are important. Then we have this thing called agents.md, agents.md, which is one of the simplest ways that Anases implement long-term memory. It's basically a file, so it's not like you have like a vector store and rag processes and all that. No, it's just a simple file where instructions for the task is going to be defined. And also this file gets updated with new information that needs to be persisted as the task runs. Now, Claude on its own actually implements this as claude.md.

4:25 That is the agents.md version that Claude uses. But you can still use agents.md in Claude. you can then use your claude.md to refer to agents.md because agents.md is actually the main standard and we're going to be building out a system for that in our Anases, too. Agents.md is the main standard used by Open Code, used by Codex, and different other Anases out there. But you know, Claude has to be stubborn, so they went with claude.md, but you can always refer to agents.md within your claude.md so that you can continue to use agents.md, which is the standard. Then you also have at files can also pull files in with your at references. And yeah, this is how contexts are assembled in Claude Code.

5:08 Then we have our environment. Yeah, the Claude Code environment, which is basically your local machine. Basically your local machine, your file system. Claude has all the tools for reading and writing to your file system. it can also run shell. It can also run your shell. That's a bash tool. It's running your shell. we've talked about the MCP connector, which also comes from tools. As we have looked at earlier, the environment is just a playground for tools. Yeah, environment is basically just a playground for tools. And you can make that environment remote, you can make it scoped, and you can also make it your entire system, which is not advisable, but sometimes just got to do what you got to do.

5:42 you also make sure that you are getting destructive operations, which is something that Claude also does to make sure that your actual system is protected from any kind of shenanigans that your model might decide to do. Then we have memory. Then we have memory, short-term and long-term. Like I said, agents.md is one of the clean ways to implement simple long-term memory. Put your instructions, put instructions that need to persist, context about the project you're working on, and stuff like that.

6:10 you put that in your.Claude directory. The.Claude directory in Claude where you can dump information like that. And you can have different layers of agents.md. We're going to be looking at that later. You can have different layers of agents.md. You can scope them to your repo or you can have them at user level. You can even have them like at the system location on your system so that every single project that you're running can actually inherit from those instructions.

6:35 I think I saw that with open code. But we're looking at Claude Claude code here, so I believe it does something similar. You can have like system-wide instructions for every Claude code instance that you spin up. Then we have the observability and log system. Claude code will show you every tool call in the UI so that you can know what is going on. It also tries to collect its own information because it needs to debug when you ask it why something failed. But basically the one you see in the UI is basically the tool calls that it is making. Every read operation, every edit operation, every bash operation is shown to you as it happens.

7:14 So basically this is Claude code and this is how it implements every single component that we've looked at. I want to believe at this point you're getting kind of theory fatigued. So in the next section we're going to start building out our honest.