Wenbo Guo: OpenSage: Next Generation of Agentic AI

0:02 Okay, cool. nice to meet everyone. I'm Wenbo. I'm currently assistant professor at UCSB. I'm also research scientist at Meta AI MSL. today I will talk about our recent work OpenStage, which is a new agent framework where we think could be the future of agent AI. before we talk about OpenStage, let's look at the current agent world, how people build agent, right? So basically, the idea is like there's pre-specified you know agent structure, agent topology, and also tool size, memory structure before you build agent. It's kind of like you build a software, you know exactly what you want to build. you'll have a clear kind of like system design, and then we build the agent, and then we let agent run the job.

0:51 But if we dive in and think about the kind of like limitation of this paradigm, the problem is like if we do pre-specify pre-specified the topology, and also we pre-specify the set of tools, that kind of constrain the generalizability of the agent a lot because within this we want agent to do complex long horizon job, but sometimes agent may spawn its own you know sub-agent along the way of execution. along the way of execution, agent may find out, you know, the set of tool I have wrong is not enough. I need to invent or need to be added for more tools to finish more complex jobs.

1:30 so but if everything is pre-specified, it's going to constrain the agent capability or generalizability a lot. so that's kind of motivate us to design OpenStage, which we think is what we call agent 2.0. Pre-hands is the agent 1.0. so basically, similar as we design the machine learning models, if we think about it like 10 years ago or even like 20 years ago, when we design machine learning models, the first step we're going to do is feature engineering, right? So, we need to think about given a raw data, what are the features we use, how to extract feature size from the model from the in- raw inputs.

2:09 and then there are so many feature engineering methods. That's kind of adding our human knowledge what we call inductive bias into the model. but then people turns out found out, you know, we don't need that step. You just train the different deep neural model on the raw data. The model will figure out by itself. And actually there, without the inductive bias constrain the search space of the model, the model can actually you know, search for better like solutions in a larger kind of search space. so, that's kind of here we think about similar. what if we don't build the agents? what if we don't do all those kind of like manual feature engineering like pre-specify the workflow, topology, tool size? What if we just build a minimal set of scaffold that enable the agent to build its own agents. Right? For example, we enable some initial tools for the agent, so the agent know, you know, along the way agent can spawn its own sub-agent, design its own workflow, topology, write its own tools, or even write its own memories. So, that's kind of like the idea behind Open Search. It's to really unleash the power of AI when building agents. Basically, we want something like AI build AI, agent build agents.

3:19 To do that, Mhm? Sorry, it's not working. Sorry, the clicker is not working. Okay. Now it's back. So, as you can see, what we want is we want to provide the minimal set of scaffolding that enable agent to build its own agent, including write its own topology, workflow, write its own tools, and also design his own memory.

3:55 so, for the sake of time, I skip the technical detail. Basically, we redesign the whole, you know, ADK, what we call agent design cookie, to be all That's better. That's good. to be able to enable the agent to kind of explore explore its own topology, write its own tools along with the way of execution. So, here's a comparison between our OpenSage with Google AI or those you know, frontier agent developer kits on this new capabilities. And as we can see, that's why we call OpenSage that's agent 2.0. that's really the kind of like next generation of agent construction framework that enable all these capabilities, freedom, flexibilities that existing non-existing ADK is able to offer. and then, let's look at the performance. so, so here we're all we're meaning kind of test OpenSage on coding and also cyber security-related benchmarks.

4:55 this is our performance compared to Cloud Code and also Codex by the time of we release OpenSage. That was about February this year. as you can see on all these existing benchmark, like notebook benchmark, like Cyber Reign, Turn the Bench, Cyber Reign Pro, and the Dial of Stream, OpenSage was able to outperform all existing agent frameworks. this number a little outdated. OpenSage is still evolving. we're also kind of seeing new performance on these benchmarks as well.

5:29 and another kind of a recent trial we feel very exciting is we actually run OpenSage on a real-world CTF game. for you for those of you know about it, we run it against the DEF CON 2026 qualification game that's kind of considered Olympic in the world of offensive security is most challenging security kind of competition all there typically require a team of professional hacker working on 42 48 hours like even sometimes there's team with hundreds of hackers work on all these challenges try to solve them.

6:03 What we do is that we try open stage on 15 of the non-interactive challenges because actually will not be the organization kind of doesn't allow allow you know AI to solve the challenge and submit. So what we do is we run in parallel at the same time the challenge release. So in total kind of 15 challenges were able to solve seven of them and after we do a post talk analysis four of them actually was very close. If we gave I don't know another hour we can solve four more challenges.

6:39 Basically in total we were able to retrieve eight flags and put us on the top five in all the teams that ever participate in the game. Once the setting wrote here also very interesting is open stage was able to beat all the team that claim they don't use AI or use low AI. Basically this is the AI only agent that's able to beat a team of hacker professional hacker that didn't use AI in their competition. So basically this shows open stage was really able to solve real world challenging task because actually all these kind of competitions all these challenges we actually we look we look at the trace open stage took like five or six hours of continuous runs spawn like thousands of sub agents to solve them. So basically demonstrate open stage was really able to scale up its own topology and able to solve challenging tasks.

7:37 Now what we're doing is like not only the agent because we know like building agent is the first step, but the brain, the model itself is also very important. Sometimes model need to co-evolve with agents. This is also kind of motivated by our observation in the sense it's like if we say wrong open stage with the latest model, we found that actually the model haven't fully figured out how to build its own agent. Sometimes model want to spawn new agent but it actually fell. So this motivate us to bring up this kind of end-to-end our model like you know holistic framework where we think about we need to build new agents to train the model. We need to basically train the model to better spawn its own agent, better write its own tools. And also we think about the inference stack like the trajectory of agent agent trajectory is actually very different from pure QA task. We also trying to develop new agent agent inference frameworks. But overall what I want to deliver is like we imagine the future of AI agents is really you know the future of AI agents is really try to open up the freedom to let AI do more explorations in terms of everything that eventually build up to the agent. What we want is to provide the meaningful scaffolding, provide the proper model, provide the most efficient inference framework to make that happen. Okay?

9:02 Thanks.