transcribe

Omnigent: The New Meta-Harness for EVERY Coding Agent - Claude Code, Codex, Pi, More

Cole Medin · 14m · transcribed 9d ago
More from Cole Medin Business
𝕏 Share ▶ YouTube 📥 PDF 🤖 .md

Section Insights

# 0:00

Introduction to OmniAgent

What is OmniAgent and why is it important?

OmniAgent is a new open-source meta harness tool designed to streamline AI coding workflows by allowing users to orchestrate multiple AI coding assistants. It emphasizes the importance of using different models for their unique strengths, particularly in the context of coding.

  • OmniAgent simplifies the orchestration of AI coding assistants.
  • Meta harnesses are crucial for optimizing coding workflows.
  • Using multiple AI models can enhance coding efficiency.
# 2:58

Getting Started with OmniAgent

How can users easily set up OmniAgent?

Setting up OmniAgent is straightforward; users can simply link the GitHub repository to their AI coding assistant, and it can be installed with a single command. The tool runs locally without needing re-authentication for existing AI assistants.

  • OmniAgent can be set up quickly with minimal commands.
  • It integrates seamlessly with existing AI coding assistants.
  • Open-source nature allows for community experimentation.
# 5:56

Workflow Management with OmniAgent

How does OmniAgent facilitate complex coding workflows?

OmniAgent allows users to delegate tasks between different AI coding assistants, such as using Claude Code for implementation and Codex for code review. This separation helps reduce bias in the AI's output and improves the quality of the code.

  • OmniAgent enables delegation of tasks between different AI models.
  • Separating implementation and review processes minimizes bias.
  • Users can configure workflows easily and efficiently.
# 8:54

Customization and Guardrails in OmniAgent

What customization options and safety measures does OmniAgent offer?

OmniAgent provides extensive customization options for each coding agent, including system prompts and sandboxing configurations. It also incorporates guardrails that require human approval for certain actions, ensuring safety in automated workflows.

  • Users can customize each agent's configuration extensively.
  • Guardrails enhance safety by requiring human oversight.
  • Sandboxing options allow for secure testing environments.
# 11:52

Advanced Use Cases for OmniAgent

What are some innovative applications of OmniAgent beyond coding?

OmniAgent can be used for more than just coding; it can orchestrate debates between AI agents on various topics, allowing them to reason and synthesize conclusions collaboratively. This showcases the flexibility and potential of the tool for diverse applications.

  • OmniAgent can facilitate complex interactions between AI agents.
  • It supports innovative workflows beyond traditional coding tasks.
  • The tool's versatility opens up new possibilities for AI collaboration.

Transcript

0:00 There is a new open-source tool that was released just over the weekend. It's called OmniAgent, and we're going to unpack this today because it is a very powerful and free-to-use meta harness. A meta harness is a tool that allows you to run longer AI coding workflows mixing AI coding assistants. And so, the most classic example that people really like right now is using Claude Code for the implementation in a workflow, and then reviewing that code with Codex. All of the top engineers right now, they're not relying on a single model or even a single harness for their workflows anymore because we want to lean on the different strengths of the different AI coding assistants, and also have these different sessions for the sake of context and token optimization, which is more important than ever right now. And OmniAgent is the layer above the AI coding assistant that makes this orchestration really straightforward because if we don't have a tool like this, just one session to manage all of our coding agents, and we can build custom agents. We'll get into all of this here. If we don't have something like this, we have to go between different terminals, create all these handoff documents. It really isn't straightforward. And so, the idea of a meta harness, it really is peak harness engineering. And so, I want to talk about why meta harnesses are so important right now. Then we'll get into how easy it is to use OmniAgent to drive our AI coding workflows, even building a custom setup in the platform. Cool. So, let's start off by talking about why meta harnesses are so important right now. And so, first things first, if there's one lesson we can learn this year for AI coding, it's that the harness matters as much as or maybe even more than the model. And that's even more apparent right now with the recent ban of Fable 5. We can't even rely on having the best LLMs for our AI coding workflows or getting better ones. And so, if we want to make our AI coding more reliable, that brings us back to the harness. If the LLM can't get better, then we better make the system around the LLM more powerful. So, your harness is everything like your system prompt, tools, skills, workflows, rules, all of that together packaged up is our harness for making a single AI coding assistant more reliable for us. And the meta harness, this is what a lot of people are really starting to lean into right now. This is the next big thing for AI coding. Instead of making one coding agent better, what if we have the layer above that orchestrates many AI coding assistants working together on a larger tasks. That's exactly exactly what a meta harness is. I'm building something kind of around meta harness engineering with Archon. And there's actually a lot of ideas from Omni agent that I'm going to be bringing into my tool and working on as well. This is just so powerful right now. And Omni agent has made it so easy for us to run this meta harness pretty much right out of the box. Like we'll see in this video here, you can get this up and running in just minutes. You can also work on it across different devices in the same environment. We of course can connect all of the most popular AI coding assistants like Claude, Code, Codex, and Pi using a bunch of different models. We can share our sessions with other people, which is another thing that coding agents don't really have right now. And then of course they have a lot around policies and guardrails and running agents in sandboxes. So, this is really set up for a production environment as well. I'm really impressed with everything that they built here, and I'm just ecstatic that all this is open source for us to try right now. So, to get started with Omni agent, literally all you have to do is take a link to this GitHub repo, give it to your AI coding assistant and tell it to set up everything. And of course I'll have a link to this repo in the description. And even if you want to do it yourself, it's just a single command to set up everything. It's so easy. I would encourage you like just try this right now. You don't even have to re-authenticate because using Claude or Codex or even Pi, it's just going to use the credentials from the CLI you're already signed into because Omni agent just runs right on your machine. And so it even ships with Polly and Debbie, a couple of example orchestrator agents that go between Claude and Codex. I'll show you how these work and even how you can create your own agents and orchestrators in this video. And so, before we dive in, just a little bit of context. I find this quite interesting.

4:05 Omni Agent is open source from the company Databricks. And so, massive company. This is a project driven by their CTO. They're already using this very extensively internally, right? They're They're dog feeding their own platform. They're using it for their everyday engineering. And of course, they're having great success with it. It really is an impressive tool. Cuz you can pick any agent. You can build the custom ones like I will show you in a bit. And then you run it in a sandbox. So, it's it's reliable. It's secure. And then you have the main orchestrator that has your history, all of your policies like the guardrails, your MCP servers and skills. So, the AI layer that you customize lives in the server. And then that can apply to any of the AI coding assistants that you run. So, it's not like you have to have a specific setup for Pi and then a specific one for Claude code. Everything runs at the top level. we can access it through a lot of different ways. They have their native app. We have the REST API if you want to do things programmatically, the terminal if you're comfortable with that like Claude code, a web UI that we already saw. So, many different ways to access it. So, once you have Omni Agent up and running, you'll have a web UI that looks like this. It's nice, simple and elegant. It reminds me a lot of the Codex app. So, it's just agent first. You have your chat session here and you tell it what you want to build. We can go between the different coding agents and the example ones for orchestrating that they gave.

5:26 And then also a couple of custom ones that I'll cover with you here. And then for the custom one, we can go between the different harnesses. So, if I even wanted to use Pi with Ollama for some local development, I can even do that with Omni Agent. So, it's very flexible. I can pick my working directory like my AI Tutor application. And I can even do work trees which are very important for parallel development. And then I just send off my request. Obviously, we're not going to watch paint dry here, so I'm going to go to a conversation that is already complete. So, I selected Poly as my orchestrator here. It has skills for how to run these larger workflows between different coding agents, and then it has access to call upon the different harnesses.

6:06 And so, you can give it a GitHub issue or just a free-form request like I did right here. And I'm specifically saying to delegate the implementation to Claude Code and the review to Codex. And so, it loads context on our work, and then it loads the skill that Poly has, so it knows our workflow, our repeatable workflow for this specific flow. And then we kick off Claude Code for implementation. So, we have the prompt that we're sending into Claude Code.

6:32 This is a subprocess that you can have run in a sandbox or not, whatever you can configure all of this, and we'll talk about that as well. And so, then Poly is going to monitor, make sure that Claude Code goes through the implementation, and then it'll send off the review to Codex. So, I didn't even have to set up authentication again. It just used the Codex and Claude Code credentials I already had. So, I had this up and running in like literally less than 10 minutes. I was able to run this workflow going between Claude and Codex so easily. And I know this is a pretty simple example of orchestrating a larger AI coding workflow, but it is very important, at least at a very fundamental level, to do your code review in a separate coding agent session from your implementation.

7:14 Otherwise, the LLM builds up way too much bias. And this isn't like an absolute truth, but a lot of people feel like Codex is best at reviewing and Claude is best at implementing. So, if you're not doing a workflow like this, I would encourage you to just try it, especially cuz of how easy it is with Omni Agent to run this kind of thing now. And so, we end with the implementation just local in our work tree. And of course, we can follow up just like any AI coding assistant and say make a PR for this, just for example. So, however you want to then get this to the point where we review it and ship it, you can do all of that with Omni Agent. It has all of the capabilities that any AI coding assistant would have, plus the ability to also call upon individual coding agents. So, I don't want to spend too much time in the config here, but I at least want to show you how this works at a high level. They've done a really good job making simple primitives for us to build all of the custom agents and orchestrators in Omni Agent. And of course, it's really set up for our AI coding assistants to help us build our own custom agents if we want to in the platform. And so, every single orchestrator comprises of three parts.

8:22 We have the configuration, and then we have all of the skills like the cross review one that we saw being used in the web UI, and then we have the agents that it can call upon as it is orchestrating a larger workflow. So, Claude Code and Codex like we saw, and then if we wanted to use like our Kimmy subscription or MiniMax or Local Models and Ollama. Polly, the default one here, can even run Pi as well. And so, within our configuration, we have the executor like for our orchestrator when it's not calling upon an agent, what is the actual coding assistant we're using? So, it's using Claude at the highest level. We have its system prompt, which is quite long, but system prompts can be pretty long these days.

9:01 We have the sandboxing configuration can run unsandboxed or run in something like Docker or one of those more production platforms like E2B, for example. yeah, there's a lot of different here around like the guardrails. And so, there's a whole capability in Omni Agent I'll show in a little bit where we can have certain actions where we as a human have to approve. And so, we have human in the loop where it'll wait for us to approve something before it continues in the workflow, like force pushes to get, for example.

9:29 And then we have the tools that it has access to. And so, this is where we tell it that these are the agents that you can delegate work to when you're orchestrating things at a high level. And then skills, I mean these are like just the classic skills that we have with Claude code, Codex, every AI coding assistant. This is the workflow that it can walk itself through. And then each of the individual agents has the exact same configuration. So, same system prompt, executor, like this one is using Claude, obviously the Codex one is using Codex, Pi is using Pi, the tools it has access to, the guardrails. And so, we can make things very custom for each individual agent that we're orchestrating as well. Very cool. And so, I have a cool example here showing you both how to build a custom agents in Omnigen, and then also how we can do policies and guardrails. And so, this agent right here, I made something very simple, right? Like super simple system prompt using Claude code, no sandboxing or anything. But I have a custom guardrail here where I'm allowing Claude code to really run any command autonomously except any sort of git push that is using the force flag, right?

10:31 Like that is one of those more dangerous actions you want to be careful of your agent running cuz it can override changes in your repo. And so, the policy here just lives right next to the config for the agent, and it's just Python code, which also I didn't write this either, right? Like this entire thing, I had my AI coding assistant build this custom agent for me just based on my request. Like I literally just told it to look at Polly and Debbie for an example of how all the config is set up, and then just build this for me.

10:59 And so, it's going to ask for my approval whenever I run any kind of forced git push. And so, let me actually show you that here. So, I'm going to go back to Omnigen here. I'm going to switch to my guarded custom agent that I have loaded up, and then I'll just paste in a prompt. I'm just telling it to run this command, to get push origin feature login. And also, another thing I didn't show you cuz I didn't show you a live run yet, it's really cool how once it loads the repo, we can see like all the files in the workspace. We can see the agents that we're using if we're orchestrating many of them. It's really neat the the UX and and the UI that we have here in the platform. And here you can see that I told it to run the command and now it's asking for my approval. And so we can get really custom here with the capabilities we allow our coding agent to just run with versus when we just have to approve. And yeah, you can do this with hooks and Claude code for example, but the benefit here is that we can apply this no matter the coding agent that we're using or orchestrating in Omnigen. And we can use Omnigen for more than just AI coding as well. Debbie, their other example that ship along with Polly, is a great example for this. Because this is an orchestrator that pits two coding agents against each other to argue on a question or a topic and then come to a conclusion together with Debbie orchestrating everything. And so I ask my question and then it lets Claude reason about it and take on one perspective and then GPT take on the other perspective. We can also click into the view to see their reasoning and all of the tokens for each of the individual agents and then back to Debbie to see how everything is being orchestrated.

12:32 And so basically it goes through two rounds here, or at least that's what I prompted it to do. So then it sends the GPT feedback into Claude and vice versa for one more round and then we sort of conclude the debate with Debbie synthesizing what both agents have said, giving us a final answer. So again, just like my test with Polly, it's a rather simple example, but I just want to quickly show what these workflows can look like and I'm sure your imagination can really start to run wild here with the kinds of ways that you can build these larger workflows combining coding agents when it becomes so incredibly easy to do so, even setting up your own custom orchestrators like I showed earlier. All right, so at this point I've showed you every feature, at least at a high level for Omnigen, except for the collaboration. So what I want to show you really quickly is how we can work in the same session both on our phones and on our computer. And you can do this across the internet so you can work with people across the globe as well. But the easiest setup is to be working with different devices on the same Wi-Fi network. And so if you scroll down in the read me here, they have instructions for doing that, deploying to a server, using it from your phone.

13:40 And so personally, I didn't really do this myself. I just had my coding agent set up everything for me. That's obviously a theme that runs through any kind of setup these days. But yeah, very easy to get this up and running. And so if we go back over to Omni agent here, just over to this other conversation that I showed you earlier with the human in the loop. I'm going to go ahead and send a message on my phone. So I'll just say hi right here, send it in and boom, there we go. You can see it immediately pop up on our desktop here. So very cool that hey, just between different devices I can keep the conversation going as I'm going between different rooms in my house. And like I said, you can host this as well so you can even work with it with other people. So there's everything you need to know to get started with Omni agent. And meta harnesses are just so important now. You don't want to be stuck with just one model or provider for your entire AI coding workflow. It's just not the best way to do things these days. And so I would highly encourage you to try out Omni agent. It's so easy to get up and running. And so that's everything I got for you now. If you appreciate this video and you're looking forward to more things on harness engineering and AI coding, I would really appreciate a like and a subscribe. And with that, I will see you in the next video.

Summary

OmniAgent is a newly released open-source meta harness tool that facilitates the orchestration of multiple AI coding assistants, enhancing the efficiency and reliability of AI coding workflows. It allows users to leverage the strengths of different models, such as Claude Code and Codex, to implement and review code seamlessly, all while optimizing context and token usage.

- OmniAgent enables users to run longer AI coding workflows by combining various AI coding assistants.
- The tool simplifies the management of multiple coding agents, eliminating the need for cumbersome handoff documents and multiple terminals.
- Meta harnesses, like OmniAgent, are becoming essential as they enhance the capabilities of AI coding beyond relying on a single model.
- Users can easily set up OmniAgent in minutes, using a simple command or through a GitHub link.
- The platform supports collaboration across devices and allows sharing sessions with others.
- OmniAgent includes built-in policies and guardrails for secure and reliable operations, suitable for production environments.
- Users can create custom agents and orchestrators, tailoring workflows to specific needs.
- The tool's flexibility allows for various access methods, including a web UI, REST API, and terminal commands.

Questions Answered

What is OmniAgent and why is it important?

OmniAgent is a new open-source meta harness tool designed to streamline AI coding workflows by allowing users to orchestrate multiple AI coding assistants. It emphasizes the importance of using different models for their unique strengths, particularly in the context of coding.

How can users easily set up OmniAgent?

Setting up OmniAgent is straightforward; users can simply link the GitHub repository to their AI coding assistant, and it can be installed with a single command. The tool runs locally without needing re-authentication for existing AI assistants.

How does OmniAgent facilitate complex coding workflows?

OmniAgent allows users to delegate tasks between different AI coding assistants, such as using Claude Code for implementation and Codex for code review. This separation helps reduce bias in the AI's output and improves the quality of the code.

What customization options and safety measures does OmniAgent offer?

OmniAgent provides extensive customization options for each coding agent, including system prompts and sandboxing configurations. It also incorporates guardrails that require human approval for certain actions, ensuring safety in automated workflows.

What are some innovative applications of OmniAgent beyond coding?

OmniAgent can be used for more than just coding; it can orchestrate debates between AI agents on various topics, allowing them to reason and synthesize conclusions collaboratively. This showcases the flexibility and potential of the tool for diverse applications.

© transcribe · For agents Built with care and craft by Gokul Rajaram