He Finally 10x Claude Code With This Method

0:00 People have been using loops to get AI agents to build a lot of things on their own. And the reason that works is the structured workflow behind each loop. That's why people let their agents work on big tasks for hours at a time, and they're getting good results. There are a lot of ways to set one of these workflows up, and one of them is Karpathy loop. So, we set up the Karpathy loop on our own apps to see how well it actually holds up. It held up well, but we found one issue that the loop can't see on its own. To fix that issue, we made one change to the loop.

0:28 If this is your first time, we're software company, and this is our channel AI Labs. And in this video, we're going to show you what the Karpathy loop is, how to set it up on your own apps, and the change that fixes that issue. But before we go into the workflow, let's go over Karpathy's method itself, because this loop is built on it. If you don't know loops, we already have a full video on them on our channel. You can watch it so that you understand loops better. Now, Andre Karpathy made a project called Auto Researcher a while back, where an AI agent kept working on training a model and ran the code again and again until the model actually improved. The agent did this in a loop, so there was no need to direct its steps. Now, in Auto Researcher, the agent was only allowed to change one file, and that was the file that trained the model. Every round was one experiment where the agent made one change to that training file, and then trained the model for a few minutes. After that, a second file gave the model a score. If the score got better, the agent kept the change. But if the score stayed the same or got worse, the agent undid the change. The agent wasn't allowed to touch the scoring file. This was because if the agent could change the scoring file, it could have just made the score easier to beat instead of making the model better.

1:34 And a third file called program.md told the agent how to run each round. And program.md was basically an instructions file that Karpathy wrote in plain English. Karpathy wrote that instructions file and let the loop run for 2 days. In that time, the agent ran 700 experiments and found 20 changes that made the model train faster. Now, this was an experiment that showed how agents can keep going and check every single possibility, which wouldn't be possible if you controlled every step yourself. This is because controlling 700 experiments yourself takes a long time, but the agent ran them on its own and found different methods that worked.

2:09 The CEO of Shopify also ran the same kind of loop and got good results. He ran it on one of his own models while he slept, and the agent ran 37 experiments. In the morning, his model was performing 19% better. So, Karpathy's method showed that when an agent is just given a goal, it can keep trying things and get better on its own without you directing every step. This is when a lot of people started getting into loop engineering.

2:32 Now, before you set up a loop, you need to know when you should even use one in the first place. If you use loops for the wrong kind of task, you waste tokens and might not even get what you want. Loops are really helpful when they're placed in the right scenarios, but the right scenario depends on what you're actually trying to get done with the task. For a loop to be worth setting up, the task needs four things. The first is that you repeat the task often.

2:54 A loop takes time to build, and you only get that time back when you keep repeating the task. So, for a job you do once, one good prompt is enough. If you set up a loop for that kind of job, you'll spend more time on the setup than on the task itself. The second is your usage limit, because loops use a lot of tokens. In a loop, the agent reads your project again and tries a new fix on every round. That uses tokens even on the rounds that fail. So, a long loop on a $20 plan hits the usage limit before the work is even done. The third is that the work can be checked without by something that gives a clear score. In an app, that's done by small pieces of code that try a feature and confirm it works. The last is that the agent can actually run what it built and see what breaks, so it knows what to fix in the next round. We use different types of loops in our workflows, but only for tasks that have a clear score we can measure, and we never let a full app get built in one loop and call it done. We only use loops to build one feature at a time or a simple version of the app that the loops checks can verify. But before we set things up, it would be great if you subscribe to the channel and hit the hype button. This small gesture of support goes a long way for us. Karpathy has shared a lot about how he works with AI agents. So we took what he shared and turned it into a setup we could use in our own workflow. In order to make his loop work for our own app, we set up a few skills and agents so that we could automate this job. We'll explain what each part does in depth so that you can ask Claude to build each one for you as well. The first thing we built was a skill that's really important called project context. This skill is specific to whatever you're building and contains everything there is to know about it. It acts as a memory bank for what the app does, which pages it has, the conventions it follows, and the things it should avoid. So when the model is working, it can pull context from there. The project context skill grows as the app grows. So it's always up to date with the latest version. We use a skill instead of a files where this information would normally live because only the skills short description stays in window all the time. That means the agent always knows it's available, but only loads the full details when needed instead of carrying all of them into every task. So whatever you're building, you can ask the agent you're working with to set up a skill that stores all of this context for you.

5:06 After that, there's a main skill that drives the whole thing so that the loop can run without you watching it. It's the build skill and it has all the instructions for running the loop on each feature. The first job of the build skill is to write the checks and it doesn't do that on its own. It uses another skill called write checks. Write checks is similar to the test author agent we used in our previous videos, but it's tailored to this project and workflow. This makes sure that the checks are written before any feature is built so that the agent has something to verify against instead of judging its own code. Once the checks are written, the build skill lists what every check tests in simple words. This is where you need to go back and forth with your agent so that you can make sure the checks cover all the problems that commonly happen during the work. Once you give it a go-ahead, a small program called approve checks locks them in. The approved checks program moves the checks into a locked folder, and a rule in Claude code settings blocks the agent from editing that folder. So, the loop can't make its own checks easier. Once you approve the checks, the build skill commits them. This adds another layer of protection in case the agent tries to change them. Then, it runs the loop, but the build skill doesn't build the features in its own context window. To keep each feature's context separate, it gives every feature to the feature builder agent. The feature builder agent starts fresh each time and builds the feature as instructed. The result of every round is stored in a file so that we can use it later on. When every feature is done, the build skill gives out a report, and all the loop's instructions like the fixed rules and how to work are written in program.md.

6:35 This follows the same format Karpathy used, too. That makes our setup complete for running the loop this way. But, before we run this setup, let's have a word by our sponsor, Upstash Box. Agents can build and fix real software now, but your laptop can only run one at a time. Because running more than one means they'll either collide with each other, slow everything down, or worse, they'll quietly read or change files on your machine. That's why you'd want a sandbox, and it's exactly the problem Upstash Box fixes by giving each agent its own sandbox. Each one is a real computer in the cloud with its own shell and file system. So, an agent can work on your repo without ever touching your machine. We spun up 10 of them, each with its own job. One built a new feature, another chased down a bug, and none of it touched our laptop. Because each Upstash sandbox saves its own state, they kept working for hours, even overnight. It's a whole software factory, and none of it lives on your machine. There's a free tier, you only pay for what you use, and it scales to zero when nothing's running. Have multiple agents powering your workflow at upstash.com, links in the description. Now, since we were working on a project that was already half-built, we prompted Claude to use the build skill and asked it to build a feature that lets guests order food for pick up from the restaurant's website.

7:47 But that feature wasn't in our feature list yet. So Claude added it and wrote the rules for online ordering into features.md. This was the file that we were using for feature tracking. With the rules written down, Claude gave us all the checks it was going to make. The first list had 10 checks. We wanted the guest's email as well. So we asked Claude to add it. That made 11 checks and the new check made sure the email was a real address. When we approved the checks, Claude committed them. Then the feature builder agent built the feature and it took about 6 minutes. At the end, the report showed that all 11 checks passed in the first round, so the loop didn't need another round. But the loop also caught something the score didn't show. The checks only tested the ordering rules.

8:28 So the loop had written those rules, but not the order form that guests use on the website. So the loop built the order form and connected it to those rules before it marked the feature as done. But this kind of loop has one problem, which is that the agent has habits. In a loop, if it sees that a particular way wasn't working, it's going to improve it in that loop, but it won't remember that for the next time it runs that loop. Our loop has the same problem because every feature starts with a fresh feature builder agent and the same files for instructions. So a mistake that a model made in one round is going to carry over to the next. The fix for that is a second loop that reads how the first loop ran and changes how the first loop works. We built our own version of that second loop as a skill called auto loop.

9:09 In Carpen these method, a person writes program.md, but in our setup, the auto loop rewrites the how to work part of program.md for us. To do that, the auto loop reads results file, which stores every round the loop ran. For every round, results file also shows whether the round was kept or undone, what the agent tried, and which checks still failed. The auto loop basically checks for problems that keep coming back like the same mistake agent was making or the same gap showing up in two features.

9:36 Then it writes down each habit with the rounds that show it. Then the auto loop rewrites how to work part in the program.md, so the next features are built with the new instructions. But the auto loop can't edit the checks. If the auto loop could edit the checks, it could make them easier to pass. Then the rounds would stop failing without the habits ever getting fixed. Now, to test the loop within the loop, we ran one on another project. It was a project management app that didn't have any way of people combining and sharing their workflows. To run it, we asked Claude to create another skill called auto loop.

10:08 The auto loop skill basically runs the same loop with the same feature builder agent and write check skill that the previous setup was using, but it runs that loop inside another loop. This is different from the build skill that we used previously because the build skill only runs one pass. Every feature gets the same instructions and the agents build it, but nothing the loop learns carries over to the next feature. The auto loop skill also runs the loop one feature at a time, but after each feature, it reviews and updates the instructions, so the next feature starts from the method that actually worked.

10:40 This helps make your loop much better. The whole purpose of this skill is actually to improve the loop itself and make changes in the instructions. So, whenever the loop runs the next time, it works better than the previous run. So, to add features to our app, we ran the auto loop skill and told it what we wanted. We asked for a way for many people to work together, so they could share focus sessions and run projects together. The auto loop skill then started by adding those features to the tracking file. Just like before, the auto loop skill first lists the checks.

11:10 You can change them if needed or ask your agent to start building directly. The first feature was a shared database, so everyone's work could live in one place. The database passed all 10 of its checks, but the app never saved anything to it. So, the auto loop added a habit to the instructions, which was to connect each feature to the app in the same round that makes its checks pass. Then the auto loop spotted a second problem with mentions, which let you assign a task by typing someone's name after an at sign. Mentions passed its checks, but an older part of the app still read the at sign the old way.

11:41 That's why the auto loop added a second habit, which was to find every place the app already does a job and make each one follow the new rules. Both habits worked because the next features came with their screens already connected. To see the checks in action, the builder got two rounds of shared projects wrong. In the first round, it broke mentions, so the mentions failed and the round was undone. In the second round, it estimated each person's share the simple way, so three people got 33 each, which only adds up to 99. That check failed, and in the third round, the builder fixed it and passed all 10 checks. When it is done, it gives a report telling you where the agent went wrong and how it fixed that, along with what the workflow looked like. So, whenever you're working on the next feature, you don't have to face the same issue again.

12:25 Now, all the skills that we created in this video, along with other resources, are all available in our community, AI Labs Pro. So, if you found value in what we do and want to support the channel, this is the best way to do it. The link's in the description. That brings us to the end of this video. If you'd like to support the channel and help us keep making videos like this, you can do so by using the Super Thanks button below. As always, thank you for watching and I'll see you in the next one.