Anthropic Just Released A New System For Using Claude Code

0:00 There's a huge problem with using AI agents. Whenever you give them a task, they write a lot of code and then call the task done. But when you actually use what they built, you find they've missed a lot of the things you asked for. That's why the most important thing you need to do is check whether the agents actually did what they say they did. There are a lot of ways to do that, and Anthropic's Claude code team has shared their own. With their setup, they put out more than twice as many changes as before, and the share of changes they had to undo dropped by over 40%. They shared the ideas their system is built on, and we turned that system into a workflow you can follow to get the same quality of output. If this is your first time, we're a software company and this is our channel AI labs. And in this video, we're going to go through each step of their process and why it matters, and show you the prompts we used so you can build it for yourself.

0:46 As the models get better, reviewing what your agent builds matters even more than it ever did before. They've gotten so good that you can just give them a whole list of features and let them work on their own for hours. When you come back, every feature is built with out you watching any of it. The Claude code team sees that as a problem because more features being built faster also means there's way more to review. Across Anthropic, the number of changes to review every day went from about 2,500 to over 30,000 in just 4 months. But even though the agents are good enough, you still need to check that things are working right. And with the amount of code they're writing each day, the chance of a mistake slipping through gets higher, too. So with this much to review in so little time, you need a way to make reviewing easier.

1:30 Review is really important because if a mistake gets to your apps' real users without a check, it becomes a much bigger problem. The team says that instead of saving time, you end up spending twice as much time as you would on the review itself, first checking the changes and then fixing the bugs that slipped through. If you rush the checks, you just push that time to later when the fix costs more. So review is a step you can't skip, and the real question is who does it. To answer that, the team first changed the question they ask when they review. Instead of asking whether someone read the code, they started asking what could go wrong with the change and how to catch it. That puts the focus on how the thing you're building should work. The Claude Code team also changed when the checks run.

2:10 Now, they happen while Claude is building instead of waiting until the end. You can already see this when you build with Claude Code because it goes through some steps to check that your app is working before it tells you the task is done. So, by the time you look at a change, it's already been checked and fixed at least once. With that in place, you can spend your time on the parts that are most likely to break, and the rest of the team's process is about how to do exactly that. But, before we move forward, it would be great if you subscribe to the channel and hit the hype button. This small gesture of support goes a long way for us. Now, the checks Claude runs while it's building are much smaller. You need deeper checks to see that the app doesn't only work the way you'd expect, but it also holds up when people use it in different ways.

2:50 That's why deeper checks are the first step in the Claude Code team's own process. Before a change goes anywhere, they ask Claude to run something that proves it works and to show them the result instead of just saying it's done. That can be the app's tests or a screenshot compared with the design. Boris Cherny, who is the creator of Claude Code, says that the best way to get Claude to work well is to give it a way to prove it did the work right. And the team's advice is to give your agent a list of checks it can run. Each of those checks is called an eval, and it is a fixed test that tells you whether the work is done. It has to be strict enough that bad work fails it. We've shown many ways to give Claude evals in our earlier videos. Claude Code's team mentions their own way to set up these checks, and we turned that into a skill.

3:34 In the demo app we were testing on, we had a specs folder with all the features in it. Each feature had its own folder named after the feature itself, and inside each folder, we had an rd.md file that listed the requirements for that feature. You can ask Claude to create a skill that turns each feature into an eval.md file. This is the file that has the actual method the agent will use to check its work. If the feature has a design, it should first make it as a simple web page called a mock-up instead of building straight on the app. Then it saves that page as the design the app gets checked against. Claude can keep working on the real app until it matches that page, and we can quickly see how the feature will look without wasting too many tokens. You should also tell it that once it has written the evals, it needs to get them reviewed by you, and without that review, it shouldn't build anything. Then ask it to create a hook that makes sure agent doesn't mark the task as done until the evals have passed. Claude then creates the skill for you. It writes the skill in your dot Claude folder and creates a template next to it that gives all the eval files the same structure. So, for each new feature, it starts by running the skill.

4:40 It opens the folder for that feature and writes the requirements document along with the eval file. Then it uses the hook to check whether the requirements are set up properly. It goes back and forth several times checking whether the evals are written properly, and it creates a mock-up that you can look at. After it has finished everything and checked the evals, it changes the feature's status from draft to in review. Then it tells you every decision it made on its own and opens the files for you to check. You can check how the design looks on there. It also shows every other screen the app can be in along with the same design on mobile, so you can check the smaller screen layout as well. You need to review both the eval.md and the mock-ups. If you have any changes at this point, you can make them by asking Claude. If you don't, you can just approve them, and it will mark the feature as reviewed and ready for building. Now, this skill, along with all others that will be showing here, are all in AI Labs Pro, which is our community. So, if you found value in what we do and want to support the channel, this is the best way to do it.

5:39 The link's in the description. Even after Claude builds the feature based on the evals, you still need to review it, and the agent that reviews it shouldn't be the same one that built it because that agent made every decision in the change and ends up agreeing with itself. So, the Claude code team's second check is a separate reviewer agent. The reviewer checks every change before a person looks at it. Since the team started doing this, far more of their changes get a review that catches something real. That review agent is only available on the team and enterprise plans for now, but a lighter version already comes with Claude code as the code review command. When you run it, a separate agent reads everything that changed and reports back the mistakes that would break your app. You can also choose how deep it checks with an effort level. At low effort, it only reports what it's most sure about, and the higher levels check more of your app. To tell that command what matters in your app, Anthropic's article on this says to write your rules in a file called review.md. By default, this file isn't read unless you ask Claude to do so. So, if you want the code review command on your computer to follow your rules, they need to be part of your Claude.md, but instead of adding them directly to that file, it's better to put them in another file. Then you need to link that file inside Claude.md. This way, whenever you open a new session, the review.md is also loaded along with the Claude.md file. This file should have what counts as a real failure in the app and how serious each problem is.

7:03 It should say what to ignore and how to report the review, along with all the ways you want the app tested. You can ask Claude to create this file for you, and you'll end up with a review.md file in your project's main folder. Every review you run on your app starts from it, and since it's linked from your Claude.md, the code review command reads it, too. With the review rules written down, the next step is a loop that builds the feature and reviews it. You can ask Claude to create a review loop skill that builds a feature and then checks it with its eval.md. It uses sub-agents with fresh context windows.

7:34 This way, they don't have the same context as the main agent, so they don't just agree with what the main agent did. It should keep looping until all the evals have passed. From that prompt, Claude makes the skill along with three agents, and the main agent runs them. The main agent acts as the orchestrator, which runs the loop and starts each agent in its own context window. The first agent is the builder, which builds the feature from the requirements document that the right eval skill wrote. After the builder is done, the verifier starts and checks the app against the evals from that same skill.

8:06 It checks whether everything the evals covered has been built and marks each eval as passed or failed. Then the reviewer goes through the app to find anything the evals missed and it uses the rules in your review.md. To use the skill, you run the review loop command and then name the feature you want to build. It starts by writing a state.md file that keeps track of the loop. This file shows which round the loop is on and what's still failing. After that, the first round starts and the builder builds the feature. While the builder works, the hook from the right eval skill stops the main agent from calling the feature done too early. Then the verifier and the reviewer start and the loop keeps fixing issues in rounds. When it hits a decision it can't make on its own, it shows you a progress report and asks you. The loop only stops when all the evals pass and the review doesn't find anything new. But before we move on, let's have a word by our sponsor, Unstoppable Code. You probably switch between different coding agents, terminals, and desktop apps all the time, feeling frustrated by all the juggling and context switching.

9:06 Unstoppable is an agentic IDE that lets you orchestrate a team of coding agents by running Claude Code, Codex, and Cursor all in the same desktop app. One thing that sets Unstoppable apart is their multi-agent workflows, which chain together prompts and scripts to automate your entire dev life cycle. We built a code reviewer that iterates with both Claude and Codex until the PR gets a passing grade and another scheduled task for fixing bug reports automatically. The terminal, the browser, and the diffs all sit in the same window, so you can see everything at a glance and switch between agents seamlessly. The best part is that with their free plan, you can run Codex and open models without paying a thing or plug in the subscriptions you already have. Download the desktop app and start building at unstoppable.ai.

9:52 Full link in the description. But fixing each issue doesn't stop it from coming back because Claude doesn't remember mistakes from earlier sessions. So it makes the same mistake on the next feature and you have to fix it all over again. That's why when the Claude Code Teams keeps finding the same mistake, they change the instructions Claude reads and Claude stops making it. We do the same with the review.md file from earlier. At the bottom of it, there's a section for rules learned over time.

10:18 When a review misses a bug that shows up later, a new rule goes into that section. It also adds a rule when the review keeps flagging something that turns out not to be a real problem. That way, the next review doesn't make the same mistake again. The review loop keeps track of this for you. When the same problem comes up in a later round, the loop matches it to the earlier one. At the end, it suggests a one-line rule for that part and only adds it if you approve. But, a careful review on every change costs time and usage. So, the Claude Code Teams rule is that a change needs to get checked harder when a mistake in that feature could do more damage. And they don't choose on their own if a feature needs a full review or a quick one. They have an agent that sorts every change for them, which they call a classifier. You can sort your own changes as well, and all it takes is one question, which is what happens to a customer if a feature breaks. A mistake that only makes something look wrong is low risk. If it stops a feature from working, but nobody loses anything, that's medium because you can still fix it later. But, some mistakes can make a customer lose money or get locked out, and some can show them another person's data. You can't take those back once they happen, so those changes are high risk. You can also turn that question into a skill that rates each feature on this scale for you. Before you build a feature, you need to run the skill to check whether it's low, medium, or high.

11:33 Then, when you use the right eval skill, you pass it that rating, too. So, a high-risk feature gets stricter evals than a low-risk one. But, even on the changes that do need the full review, there's one more problem. The builder and the reviewer agents don't know when to stop, so they can keep going back and forth trying to find and fix issues. The Claude Code team once had a change that went back and forth more than 50 times like this. The agents kept going until someone manually stopped the loop. When the team looked at what the reviewer kept reporting, about 59% of it had been there since the very first round. So, the reviewer was mostly finding the old problems again and again. And on your plan, every one of those rounds uses up your limits.

12:12 People's main complaint about Anthropic's article was that it only works if you have unlimited tokens. So, the team added three rules, and the first one is that after three rounds, the reviewer stops listing more problems and tells the builder to rethink how it's building the feature. If the feature is still failing after three rounds of fixes, more work in the same way wouldn't get it there. The second is that after the first round, the reviewer only checks what the last fixes changed and doesn't report anything it has already reported. That stops it from checking the whole feature again and finding the old problems one at a time.

12:44 And the third is that when the reviewer finds a problem, it says what kind of problem it is. That way the builder fixes that kind of problem everywhere it shows up and not only in the one spot the reviewer found. With those three rules in place, more of their changes passed review on the very first try. Our review loop works the same way, but each of our rounds does more work. It builds the feature and checks every eval in the browser, and then it reviews the code.

13:08 That's why we set it to stop after five rounds instead of three. The loop can also stop before it gets to five when the same problem is still there after two rounds of fixes. But even when the loop finishes and every eval passes, some mistakes will still get through. So, you need two more things, which are a way to find those mistakes and a way to undo them fast. The Claude code team can undo almost anything they release in minutes, and that's what lets them read less. To find the mistakes, they use the product themselves every day, so they often spot a problem before their customers do. You can do the same by using every new feature yourself the way a customer would. But you already know how the feature works, so you'll miss some things. To catch those, you can send a fresh agent that never saw the app being built to use the live app like a first-time customer. To set up that fresh agent, you can ask Claude to create a skill that shows you the app from a new user's point of view. It should use sub-agents that are blocked from reading the app's code, so they don't know what the app is about. They should go through the app the way a first-time user would and find every place where someone might get stuck.

14:10 From this prompt, Claude creates a first-time user skill and the agent that the skill sends to use the app. It also gives the test a temporary database, so it doesn't touch your real data, and it creates a hook that stops the agent from reading the app's code. The agent would normally get around that block with terminal commands, so the hook stops it from reading the code that way, too. To run the test, you just use the skill's command and tell it which feature you want it to check. Based on your prompt, it opens the app and starts an agent that knows nothing about how the app was built, and then it waits for that agent to finish. When the agent is done, the skill reports everything the agent found and suggests fixes you can apply to make your app better. Finding the mistakes is one half, and the other half is undoing the changes that caused them. For that, you can use Git, which basically saves every version of your app. With Git, each feature gets its own copy of the project, which is called a branch. If any feature in your app causes issues later on, you can just undo it from Git.

15:06 To find and undo the change that broke something, you can ask Claude to create a skill called fix a bug. Basically, when a feature isn't working the way you need it to, you give the skill the problem, and it finds the cause and how to fix it. It tries to make the bug happen again by going through the same steps, and then checks whether it's guess about the cause was right. After that, it reports the problem and fixes it. For this skill to work, you also need to tell Claude in your Claude.md to use Git and make its own branch for each feature. It should also save every version that works. That way, when a feature isn't working as expected, Claude can go back through the saved versions to find the change that caused it and roll back to an earlier version.

15:45 That brings us to the end of this video. If you'd like to support the channel and help us keep making videos like this, you can do so by using the Super Thanks button below. As always, thank you for watching, and I'll see you in the next one.