Section Insights
Introduction to Agent Loops
What are agent loops and how can they be beneficial?
Agent loops are systems that automate the process of writing prompts for AI agents, allowing them to learn and adapt independently. They can be categorized into deterministic and non-deterministic loops, each suited for different tasks.
- Agent loops can enhance AI efficiency by automating prompt writing.
- Deterministic loops have known outcomes, while non-deterministic loops do not.
- Choosing the right type of loop is crucial for effective AI task management.
Implementing Non-Deterministic Loops
How can non-deterministic loops be effectively utilized?
Non-deterministic loops rely on tests to verify the completion of tasks. By writing tests beforehand, the AI can autonomously work on features while ensuring they meet the required standards.
- Predefined tests allow the AI to autonomously verify its work.
- The agent can roll back to previous versions if issues arise.
- This approach minimizes the risk of breaking existing features.
Creating a Skill Improvement Loop
How can you ensure a skill is functioning correctly?
A skill improvement loop can be established by creating a command that triggers an agent to assess and enhance the skill's performance through iterative testing and adjustments.
- Skill improvement loops assess and enhance the quality of AI skills.
- The process involves multiple rounds of testing and adjustments.
- Documentation of improvements helps the agent learn from past issues.
Orchestrating Multiple Agents
What is the role of orchestration in managing multiple agents?
An orchestrate command coordinates multiple agents to review and improve outputs across various dimensions, ensuring a comprehensive evaluation and enhancement of the application.
- Orchestration allows for coordinated reviews by multiple agents.
- Each agent focuses on specific aspects like factual correctness and style.
- This process results in a significantly improved final product.
Workflow Improvement Loop
What is the purpose of a workflow improvement loop?
The workflow improvement loop goes beyond task repetition by analyzing and suggesting enhancements to the overall process, thus improving the efficiency of the loop itself.
- Workflow improvement loops focus on enhancing the entire process, not just individual tasks.
- They utilize an iterate command to orchestrate improvements.
- This loop involves agents that build and score implementations based on defined quality rubrics.
Transcript
0:00 You've probably already heard about agent loops since everyone is hyping them up a lot right now. And you might think they're just a way for these AI companies to get you spending more on their products since these loops chew through a lot of tokens. But that only happens when you're using the wrong type of loop for the job. As you already know, we're a software company and we've been experimenting with these loops in our AI coding tasks. Along the way, we've mapped out the different types of loops and which use cases each one is actually suited for. So, out of all the loops we've set up, we're going to share the ones we found genuinely useful.
0:31 We'll also show you how to set up each one and how each loop is going to impact your workflow. Before we get into the different types of loops, let's quickly recap what loop engineering actually is for those of you who are new here. We won't go deep here, but if you want the full breakdown, we covered it in a previous video on the channel. The core idea of loop engineering is that you stop being the person writing the prompts that drive the agent and you turn it into a system that writes the loop itself. Instead of spending your time setting things up and writing long, carefully structured prompts just to get it to build, you let the agent handle everything on its own. It learns as it goes, grows from the problems it hits along the way, and figures out what it needs to do next. That's what an agent loop really is. In that previous video, we split loops into two types based on the outcome you get, the deterministic loop and the non-deterministic loop. A deterministic loop is the kind where you already know the outcome. So, the agent has a solid way to check its own work against it and it keeps working until it gets there. A non-deterministic loop is the kind where you don't, so there's no solid way for the agent to check its work, which means you need other ways to handle it. But that was a broad split and we showed just one way to set each of them up. They can be built in a lot of different ways and each setup changes what you can do. The first type is one most of you have probably already used.
1:46 It's basically the building block of every other loop and the clearest example is the goal command. We call it the stateless loop and basically it means the loop doesn't hold on to anything or improve itself as it works. There's no part where it learns from what happened and gets better. That's exactly what makes these the simplest loops there are. The one you've probably already seen of this type is the Ralph loop, and it's stateless because it never kept any memory. It just ran the same task again and again, and the moment it saw the task was finished, it stopped. As you might already know about the goal command, it is the best example of the stateless loop. To use it, you tell it what you want to build right after the goal command. From there, Claude sets that as the goal and starts working on it. Then every time the main agent decides a task is done, it uses a smaller model to double-check the work.
2:31 In Claude code, that smaller model is Haiku. It checks everything the agent did against the requirements you gave in the prompt, and if the task isn't fully done, it reprompts the agent to finish what it missed. But there's a problem with this loop. It leans entirely on the model to decide whether a task's done with no standard to measure the work against. That's why it works best on features whose requirements you can check in some hard concrete way. One way to do that is with tests. Like we talked about in our previous video, we write the tests before asking Claude to build any feature. That way, if Claude changes that feature in a way it shouldn't, the tests throw an error and tell Claude its implementation is off. And once we've got tests for every feature, we can hand the agent real autonomy and let it work without worrying it'll break the other features or build the one we want the wrong way. Once you've written the tests, you can ask Claude code to set the goal as getting that feature to pass all the tests. And it keeps writing code, running the tests to check itself, and keeps going until every test passes.
3:27 Once they all pass, that means the feature's built correctly, and Claude will mark the goal as complete. Since the agent's working on its own, you'll want to add one line to your Claude.md file. That line tells the agent to save every working version of the app. That way, if it breaks the app somewhere down the line, it can just roll back to the last version that worked and carry on from there instead of trying to undo changes from memory. But before we move on to more types, let's have a word by our sponsor, Minimax. Minimax just dropped M3 and it's the first open weight model to hit the frontier on three things at once, coding, a 1 million token context, and native multimodality. So, we plug the M3 API straight into Claude code and gave it one real job, research the top electric cars on sale now and build a live comparison dashboard. M3 took it from there on its own. It browsed the web, pulled real specs and prices, then shipped a working dashboard from scratch. You can search different EV brands, browse their latest models, and everything updates in real time. On autonomous browsing, it beats Opus 4.7 and its million token context held every page plus the whole code base in one window. But, agent runs like this burn a lot of tokens and that's where the Minimax token plan comes in. Pick token plan for fixed costs or pay as you go for flexibility. Text, image, speech, and music share the same token pool with the highest quotas. Plans start at just $20 per month. So, hit the first link in the description and get an exclusive 12% off. The stateless loop we just walked through holds no state. It does everything on its own from the instructions with no self-improvement in the process. The next type works the opposite way and we call it the learning loop. A learning loop works differently.
5:07 Instead of just getting a task done and stopping like a stateless loop or the gold command, it focuses on improving something you'll use repeatedly, whether that's a skill or a workflow. The way it does that is simple. It runs the skill, observes how it performed, and then improves it based on what it learned, keeping a complete record of every lesson along the way. So, that when you run the skill, when you're actually using it, the agent knows what caused issues in past, so it won't lean towards that. You can put this kind of loop to work for a lot of things. For example, on the community website of ours, we built multiple skills to handle different repeated workflows while putting the site together. But, building a skill raises an obvious question, which is how you'd even know whether it's working the way it should. So, to answer that, we set up a full learning loop. We did it by creating a skill loop command that triggers the loop. This command contains instructions to call a skill improver agent and keep calling it until there is no more improvements left. This skill improver is actually an agent we created, which improves the skill by assessing its quality, testing it across multiple areas, and watching for the issues that come up. To use it, you just run the command and pass in whichever skill you want to improve, and it gets to work. This loop runs in multiple rounds. In each round, it runs a set of tests and checks after making its changes. Then it launches a separate Claude session that works only on the prompt you pass it, running in the background without stopping to ask permission for anything, and reporting the output back. Inside those sessions, it runs the implementation two ways, one with the skill and one without, so it can measure the actual impact the skill has. That comparison lets it pin down exactly what needs improving, and it makes those changes directly. But, the most important part is the learning.md file it creates. This file gives the agent a way to know what works and what doesn't, and it lives inside the skill itself. It's basically an improvement journal that documents everything the agent learns in a structured format. It records what it tried and what the result was, both with the skill and without it, then lists the lessons it picked up across all the rounds it worked through. And that's how it keeps going round after round until the skill is refined into the best possible version of itself. You can use the same setup to improve any workflow you've got. In our previous video, we showed how to build a loop agents, one that handles the implementation and another that reviews the work and reports back fixes for the implementation agent to apply. There's a problem with that setup, which is that a single review agent is handling every aspect of the review on its own. But, a review is never about just one aspect. It always comes from different perspectives, and that's too much ground for one agent to cover alone. It's better to split those across different agents, because when multiple agents review across multiple dimensions, they cover the blind spots any single agent would miss, and that makes the review way more complete. The idea is close to the LLM council that Andrej Karpathy released, which is a council of multiple agents that talk to each other and argue over a topic you hand them. Using the reasoning of several models to land on the right answer. For creating a multi-agent loop, you need to create multiple agents. So, for example, we created four agents in the loop we set up. The first checks for factual correctness, and it comes with tools like web search, so it can ground itself in real sources. The second is a domain checker agent, which checks whether whatever is being reviewed is actually relevant to what we're trying to do. The third is a safety critic agent, which looks at the safety issues like sensitive content along with security risks and policy violations that could cause problems down the line.
8:26 And the last is the style critic, which makes sure the content is clear and well-written and tailored to the style we're after. You can use these agents for any task, whether it's coding or not. What ties these four together is an orchestrate command that we created. contains the detailed instructions for how it should manage and coordinate all four agents and handle the feedback each one reports back. To start the loop, you run the orchestrate command and ask it to review whatever you want, and it spins up all the agents for the process.
8:54 The orchestrate command runs in multiple rounds, too, spinning up every agent in each round. The main agent applies all the fixes reported in round one, then spins them all up again for the next round. By the end of the final pass, you're left with the app in way better shape. If you'd rather the agents communicate directly, you can use the agent teams workflow we covered in a previous video, which gives you more of the LLM council experience without one agent handling all the communication.
9:19 But, we chose the orchestrator because one agent needs to hold the context of the previous rounds to coordinate the workflow properly. Another type we reach for often is the verification loop. It uses multiple agents as well, where one does the implementing and the other scores that implementation, and the implementer's whole job is to get that score as high as possible against a set metric. To set that up, we created a command that coordinates the entire loop, running the whole review workflow on its own. As you already know that cursor has thermonuclear review. It's actually a really powerful review skill that checks how clean and healthy the code is, so it stays easy to build on later. It audits all of the code and hands back an in-depth review with non-negotiable standards, so you're guaranteed the highest quality review it can produce. To do that, it runs a dynamic workflow. The review has to span a lot of categories and a dynamic workflow is the best way to handle that, since it fans the work out across multiple sub-agents that each take on a different aspect at once. Like we mentioned earlier, we created two agents that act as the players in this loop.
10:19 The first is the implementer, whose job is to read the PRD and then build the required functionality. The second is the thermonuclear code reviewer, and its only job is to hand back a review score. Since all it does is review and score, it doesn't have tools for editing. To trigger it, you run the review loop command. by understanding what the app is meant to build, then kicks off a thermonuclear review for round one. That first review flags the issues it finds, including a critical one that's stopping the app from even starting. It records the findings in a JSON file and spins up the implementer agent to fix them. The loop keeps going from there, but keep one thing in mind. Because it's reviewing across so many dimensions, it takes a really long time and eats up a lot of tokens, since the dynamic workflow driving it fans the work across a whole set of sub-agents at once. So, we wouldn't recommend it unless you've already built the whole app at a larger scale and want it thoroughly reviewed.
11:11 You can also build this same loop without the dynamic workflow by using a normal reviewer agent, which takes less time and burns through way fewer tokens. And if you're enjoying the video so far, subscribe to the channel and hit the hype button. This small gesture of support goes a long way for us. Out of every loop we've shown you so far, none of them had a separate step for improving the loop itself. But that's really a core of what a loop is meant to do. That's where the workflow improvement loop comes in. What this loop does is go a step beyond just repeating the task. Instead of just running it again and again, it looks at the process itself and suggests improvements to the workflow. Now, you might think the learning loop from earlier already does this. But, there's a real difference. The learning loop improves a skill, one piece inside the process. This one improves the loop itself, the whole process you've set up.
11:57 The entry point is an iterate command that acts as the orchestrator for everything that happens during each run. There are three agents this time. The first is a builder agent, which handles the implementation and delivers one of the app's requirements on each run. The second is a scorer that checks that implementation against a rubric we defined to act as the app's quality guardrail and scores the work out of 100. And the third is the process optimizer agent, which is the one that actually handles self-improvement.
12:24 Normally, a loop runs through the same cycle where it plans, implements, verifies, and repeats. But, this agent adds an extra step of going back over the loop iteration and suggesting ways to make it better. To use it, you just run the command iterate all. By all, we mean we're implementing the entire app broken into parts inside a single workflow. The loop starts by spinning up the builder agent, then the scorer evaluates what it built against the rubric and records the score in a JSON file that tracks every round. Then, the process optimizer agent kicks in, going through the conversation to spot anything that could improve the workflow, making sure the app is built to a high quality, and the right steps are being followed. So, by the end of this workflow, you don't just walk away with a built app. You walk away with a workflow that's been tested and refined, with every step validated as one that actually needs to be there. That brings us to the end of this video. If you'd like to support the channel and help us keep making videos like this, you can do so by using the Super Thanks button below. As always, thank you for watching, and I'll see you in the next one.
Summary
- **Stateless Loop**: A basic loop that executes tasks without retaining memory or improving over time; ideal for tasks with clear, measurable outcomes.
- **Learning Loop**: Focuses on continuous improvement of a skill or workflow by observing performance and making adjustments based on past outcomes.
- **Multi-Agent Loop**: Utilizes multiple agents to review a task from different perspectives, enhancing the thoroughness of the review process.
- **Verification Loop**: Involves an implementer agent and a reviewer agent, where the implementer aims to achieve a high score based on set metrics, ensuring quality through detailed reviews.
- **Workflow Improvement Loop**: Goes beyond task execution to suggest enhancements to the entire workflow, ensuring that each step adds value and is necessary.
- **Dynamic Workflows**: Employed in verification loops to distribute tasks across multiple agents, though they can be resource-intensive.
- **Iterate Command**: Acts as the orchestrator in workflow improvement loops, integrating implementation, scoring, and optimization for a refined process.
Questions Answered
What are agent loops and how can they be beneficial?
Agent loops are systems that automate the process of writing prompts for AI agents, allowing them to learn and adapt independently. They can be categorized into deterministic and non-deterministic loops, each suited for different tasks.
How can non-deterministic loops be effectively utilized?
Non-deterministic loops rely on tests to verify the completion of tasks. By writing tests beforehand, the AI can autonomously work on features while ensuring they meet the required standards.
How can you ensure a skill is functioning correctly?
A skill improvement loop can be established by creating a command that triggers an agent to assess and enhance the skill's performance through iterative testing and adjustments.
What is the role of orchestration in managing multiple agents?
An orchestrate command coordinates multiple agents to review and improve outputs across various dimensions, ensuring a comprehensive evaluation and enhancement of the application.
What is the purpose of a workflow improvement loop?
The workflow improvement loop goes beyond task repetition by analyzing and suggesting enhancements to the overall process, thus improving the efficiency of the loop itself.