Build agents with Gemini API (I/O Connect ‘26)

0:04 My name is Mariano. I'm a product manager at Google DeepMind. I take mostly care of genetic capabilities for the Gemini API, and today, we're going to be building agents with the Gemini API. But before jumping into that, I would like to go a bit around what brought us here. OK I want to just take a moment to think about how much we have shipped since 2024, when we started with Gemini 1.5, how we moved through different models.

0:33 Some of them really loved like Gemini 2.5 and now finally landing in Gemini 3.5 Flash. One of my favorite models at the moment, to be honest, is my go to model for mostly everything on my day to day. But we have been not just shipping at a relentless pace on the frontier models, but also in the open models road we have shipped this year. Gemma for I think, one of the most anticipated models of the open models.

1:02 And it's been not just these models that we have been shipping, we have been also going through different media models. All the models I want to make here a special shout out to Nano banana two. And in the audio models, the text to speech ones. Something that I really like to do, for example, is I consume a lot of media, I like to learn by listening. And I have built a couple of cool apps and cool agents myself that will check my inbox using now the new Workspace integration to AI Studio, that we will talk in a bit, and then I will create little podcasts or little notes to listen in the morning while I'm preparing my coffee or whatever.

1:43 So yeah, if you haven't tried it, give it a try. They are really good. Also a testament of how good they are because I mean, I'm saying it while I'm wearing a Google t-shirt, so probably I'm a bit biased, but let's also talk about the customers who are using it. We have a lot of the most important industry companies using it. Companies like Cursor, GitHub, Figma, Vercel. It's all over the place. Gemini 3 has been really well adopted and really well received by the whole community.

2:19 One of the main reasons it's been so well received is its multi-modality capabilities. And what is the multi-modality capabilities is how it can take multiple inputs from system instructions, classic text, images, audio, video, speech, URLs, whatever you want to throw at it. And it will give you multiple types of outputs like text, image, audio, video, function calls. So you can execute the stuff on the client side or code.

2:50 One of my favorite things to do is to just play with, hey, what if I ask for it to be something all this as an output. What if I ask it to be something like. Video with bow as an output. It's really great. You can really mix and match and create really cool things. And when you want to build your ideas with Gemini, you have to go where. To AI Studio. I hope most of you have used AI Studio.

3:16 I hope most of you are actually daily users of AI Studio or the API in AI Studio. You can. You have a playground where you can test all of the new things that we are constantly shipping and talking about new things. Let's see what's new in AI Studio. One of the most exciting things for me, my background is mostly on web development, mostly as a full stack developer for web applications. So when we shipped the full stack Node.js applications, for me, it was a big thing.

3:51 It was a big milestone. Now you can go and build more than just the forefront, more than just a couple of landing pages. And you can build the whole thing, in a Studio without leaving it. It will automate your Node.js server side runtime, setup it. You will have full access to the NPM ecosystem. So you will not have any kind of weird blockers there. And it has a built in secret management for server side API keys, which to me, it's a game changer because something I always hated was authentication and managing secrets.

4:28 I don't know how many of you have been a lot in the frontend world, but those two things, usually you end up getting into way too many third parties and you end up yeah, everything gets a bit messy, and mostly when you are by coding or coding with the coding assistant. There is also a new native Firebase integration that this allows you to do. What I was just complaining about. It will give you an view an integrated Firebase authentication translating signing with Google.

4:58 You can just have it out of the box, which is one of the best things ever. Because let's be honest, what is one of the main ways that you use to sign up to applications. You sign up with Google, you are probably all signed up with Google at least a couple dozens of applications. I don't want to go by 100. Then you will also have your automatic provisioning of Firestore. That is our NoSQL database. And we will soon have Cloud SQL.

5:28 And this is something that I was just talking about. This is an integration with Workspace. I gave a hint about it when I was talking about my morning podcast about my inbox. You can easily integrate your applets to Workspace so you can connect Gmail, Docs, Drive, chat, most of the Workspace applications to your applet. This allows you to create a lot of things and to work smarter with your own corporate data. We all know how hard it is sometimes to integrate all of these things together.

6:03 Here you can do it with an automated OAuth, which means no Cloud Console configuration required. If you have been around trying to do this kind of integrations that it can become messy when you have to go to the Cloud Console to configure something and you might just drop off. Now you just keep going and you don't have to leave. AI Studio, and one of the things that I think is a game changer is now getting better and better is the generative native Kotlin apps via natural language.

6:37 And what is it. You can go and prompt your Android application, and it will generate a native Kotlin and Jetpack Compose application as quickly as you can describe it. I think we have surpassed the 1 million applications created since we released it last month. Philip great. We am not lying to you. It's been. And I think one of the things that always took me down to doing any kind of mobile development, I'm a big frontend guy.

7:06 So yes. Was the whole configuration, the setup, having to set everything in my development environment, et cetera something that I did is you can have everything on your browser. There is 0 configuration. You have an emulator right there that will just run your application. And then when you want it, you can download it, install it in your Android phone, or you can even just publish it so you can test it with one click OK. You have a one click Publish workflow that I think it's great.

7:38 For me, every one click Deploy. It's what I'm looking for on any dev tool when I am developing something. And let's not forget about the enterprise sector. There is also new enterprise ready deployment workflows, so you can deploy directly to Google Cloud Run without having any dedicated server infrastructure you can deploy to your project. It has support for Google hosted projects and Firebase App Hosting. So we have covered for any of those enterprise requirements, but also want to make a good remark on the small things that the team is shipping.

8:20 All of these UI and UX quality of life benefits that you are getting nowadays. One thing I hate when I am coding something by coding something is there is one word that I want to change, or there is one title that I want to change the style, a button, whatever. What do I have to usually do. I have to start writing hey, the title that is in the landing page in the second section behind the hero.

8:52 Can you make it larger. Can you make the logo larger. No, not that logo. The other logo. Well, one of the improvements that we have now is the focus on annotation modes, where you can go click on something and prompt a change to that component specifically. You don't need to be playing around with your prompt to be able to pinpoint to something. You can just go and do it. Then another big one for me is design previews that what is it.

9:19 When you make a prompt and you have the model that starts thinking and reasoning and generating what happens there. You are there just waiting. And maybe the first result that you get is not something that you like. And let me know if any of you never had this. You finish the first version and the next thing that you say is, oh, can you make it more I don't make it pop, make it dark mode, make it whatever.

9:48 OK with design previews, what you have is while the generation is still happening, it will bring you a couple of previews with different styles and you can pick one. So while the generation continues, you already have picked your design style so you don't have to be fixing it afterwards and having to go again through all the generation, and you start getting an idea of what will you get by the end. It's going to be a bit of Lo Fi, but you will get it.

10:16 I will show you in a minute. And then finally, something that we internally call tab, tab, tab, because tab is a way to go is when you are starting to write something. I don't know if you've ever had this writer block. You have an idea, but you don't know how to express it. Or you start writing in a blank input and you're not really sure on how to continue. Well, now you have a. Luckily you have LLMs helping you to move forward.

10:46 You start writing, you give it a couple of milliseconds to be honest, and you will have some autocompletion to help you finish writing your prompt and enhance it in real time. So how do I get started with all of these great things. To be honest, I hope all of you have already get started. I hope this is not news for anyone but you. Just visit AI Studio. You get your API key.

11:17 It's a pretty straightforward process. That API key will help you with anything related to the API, and then you have to do no other setup. You have a generous free tier, and once you are ready for it, you can set up your billing. And you can also set your cap, your spend cap. This is something also really important. So you don't have any surprises having said it all, sometimes you have been coding something.

11:50 You have been building something, crafting something on AI Studio. And there is a moment that you want to move into your local environment because you are used to your Ide, you are used to your coding agent, you are used to coding in anti-gravity in your computer. So something that we have added now is a one button way to export what you are currently doing on AI Studio. So anti-gravity it will be export without losing any context, without losing any history.

12:20 It will be neatly structured. You will not lose any secrets. You will not lose anything. It will ask if you want to export the secrets to. But if you don't want to lose them, you just click and you just go, OK, I have been talking a lot about AI Studio and how great it is. So what about if we go and check a bit how it works. All right. Here we have AI Studio.

12:51 This is, for example, if I start writing, I want to build an app. It wants personal fitness goals and meal plans. OK I don't want that. But something I do want is a stretch tracker. And why. Because I'm in my seconds and all my joints start to be painful. So I have a prompt already done where I ask it to build me a nice stretch daily stretch application, and I will just ask it to build it.

13:25 Once I started to build it, it will start thinking it will start generating. Let's give it a second. So it starts. Here we are. While we are waiting, you will see we are running Gemini 3.5 Flash. Best model right now and we will. It will start first by generating all of these design plans. It should come at any moment now letting me hey, how would you like your application to actually look like.

13:58 Because there are many ways that it can look like for example, high density, vibrant palette, elegant, dark, sleek interface, immersive UI. OK, we have a lot of different things. I will go with elegant dark, I will select this design and now the generated application will be with that design. I don't want to make you wait for the generation, so I have already run it before with exact same prompt.

14:31 And here we have the daily stretch application. As you can see, the daily stretch application is literally working here on an emulator in my web. In my web browser, I have not set literally anything. It just work. I can come here, I can move here. I have even my tennis pre-match for my knee that is hurting. I can get into the different stretch and I can ask it to start stretching, and it will give me a nice video and instructions on how to do and how I should be feeling the different movements.

15:10 I think this is something pretty neat. It will keep also track of what I've done before and I can. Yeah, I can use it on my everyday. And how do I use it on my everyday on my phone I just go here and click Install. If I just connect my phone via USB, I can just download it into my phone and have it there ready to be used, which it's great. I remember last time I tried to do web.

15:39 Sorry, mobile applications. It was probably in 2015. It was not that easy. Then you can also, with a simple button, publish to the test to test it on Play Console. I think this is one of the underrated features, is how easily you can get it in the Play Store for testing, to sharing with your friends, and have everything being tested, because end of the day, we want to build also for others, it's not just for us.

16:10 And finally, I want to show you in the Code Ed where you can export to anti-gravity. Here you can check. Once it will be included, it will be included by Gemini API key because the app I asked it to do some generations like for example, this image is generated with Nano banana. So yeah it needs an API key. But yeah I think that's it from me. Let's go back to the slides. I really hope that this gives you an idea on how basically I work.

16:47 I really hope that you can go home, build some neat applications and yeah, let me know what you build. Now we will get into the stage to Philip, who will be talking about how to build an agent with the API. Thank you folks. Hi, everyone. My name is Philip. I'm based out of Germany, so if you hear some German accent, that's why.

17:20 And I'm working on the Gemini API team, mostly on the interactions API and managed agents. And I'm very excited to share that the interactions API is now general available since Monday evening, I think. And for the ones of you who never used the interactions API or never heard about the interactions API. It's our new unified interface to use Gemini model and Gemini app agents, and I guess every one of US has seen a shift towards more authentic use cases.

17:48 We no longer just prompt. A model with Hello, how are you. Or can you look up on Google Search something for me. We really have those agentic use cases where we define more tasks and goal, which we want the model to achieve. And that's why we built the interactions API. And you can see on the left side a code snippet on how you can use it. It looks very familiar to other LLM APIs. So if you are using competitors.

18:13 You should be very easy for you to adopt. And it's also very well aligned through all of the different modalities Gemini app supports. We are going to take a look at it. But to look at the code snippets. So we have either a model or an agent. So it's very easy to define. Do I want to use Gemini app or do I want to use an agent like anti-gravity or Deep Research. And then you define your tools or your inputs and what's very new and unique about the interactions API is this environment parameter, which we are going to demo and use.

18:40 And then also in addition to it, we saw a need for longer horizon tasks. Agents run longer. And when we are running them on our clients, it's very easy because we are basically on our own machine. We don't need to connect to some back end. We don't need to manage some state. But with APIs and remote agents it's a little bit different. So we added support for async executions or background as what we call them. So you basically create your request, you get back an ID, and then you can use that ID to pull and get status update or just stream event updates.

19:14 And then also we added support for server side context management where you just need to send new inputs to the API, and the API keeps track of it. Of course it's optional. So if you still want to keep track of your state, you can do this. We are so definitely give it a try. And to keep it short we look at a few code examples, and then we are going to play around with it and build some cool agents.

19:37 So server side state, how does it work. Very simple. We have a model, we have an input and we send the input. And if we want to do a follow up, we send a new input and provide an ID to the previous turn. So you don't need to manage this weird user model, user model terms. You basically just provide what's new. Same for multi-modality. So Gemini app supports audio generation, image generation, audio understanding, and all of those follow the same pattern.

20:06 So we have our model for audio generation. We need to use the Gemini app TTS one. We have our input. What do we do. We want the model to generate. And then since we want to generate audio we have the new response modalities which basically tells the API, hey, I want to receive back an audio file. And then we have some configurations. So we want to use the query voice or in this case in US. So it defines a little bit what extends or which we want to generate.

20:35 Same pattern goes for image generation. So we have our model Nano banana Pro. We have our input. And then we have some configuration on what we want to generate. And the same goes for tool use. And very cool about the interactions APIs that you can now combine first party tools with third party tools. So we already had the Google Search Google Maps for a very long time in the Gemini API. But one limitation was always you cannot use Google Search together with your own custom functions.

21:03 And now with the interactions API, you can easily mix and match whatever you need. So if you need an agent or an application which requires real time data from Google Search, you can define the tool, type Google Search, and then you can provide your own custom functions. For example, if you want to create some kind of internal incidents. The code snippet here is a very basic agent, I would say, which searches the web for new CVEs on React, and then also uses custom function calling to integrate or call it into the internal database, and something potentially a bit confusing in the beginning, but at the end very helpful for building authentic applications is that with the interactions API, we moved away from this typical user model user model turn concept.

21:51 If you have used models since the beginning of ChatGPT and throughout the years, you always had this turn based interactions where you provided a user input and then you got back a model output and you get back a user input and so on and so on. And throughout the whole last of the year, we saw models no longer just generate a simple response. We got reasoning, which is a complete new complex tape, I would say. And then we also get all of those agentic integrations where models started to call tools.

22:20 We provided two results from the environment, not from the user. And with the interactions API, we moved into a steps data model where each unique interaction or part of the interaction becomes its own data type. So you have a type for a user input, but then you also have a type for reasoning, for function calls, for function results, which should make it much easier for you to build with it, but also for the model to reason about it.

22:43 So if you're using coding agents, it's much easier for them to understand. OK, when I build an agent loop, I need to provide a function result after my function call and not need to wrap it in this weird user turn and then provide some additional data and announced at Google I/O we added support for the anti-gravity remote agents. So one important piece for us at Google was that we unify all of our agents stacks basically, and the anti-gravity Ide and now anti-gravity 2 and anti-gravity CLI all bring this anti-gravity agent harness, and this agent harness is now integrated into the Gemini app API.

23:20 And agent harness here does not mean it's the same agent. So it's not the exact same agent, but it's the same technology which empowers the model to do certain things. But there might be slightly differences in terms of which tools are available or how the system instructions work. With the anti-gravity Ide, you have a coding agent, which you mostly want to use to do coding tasks with the remote anti-gravity agent. You can define your system instructions completely freely, and it's not like predefined as a coding agent.

23:50 And of course, it supports tools and remote environments. And what are remote environments. Remote environments are basically isolated sandboxes managed inside Google Cloud, which the agent can use to run commands. Create files do work basically without the need for you to do anything on your local machine. And those environments are persistent. So if you run the first interaction to say, hey, please install pandas or other dependencies, you can follow up with another request saying, hey, OK, now use pandas to analyze some CSV file to do some data analysis and you don't need to manage any of the infrastructure behind creating the sandbox.

24:29 Starting the sandbox. That's all abstracted away from you. And of course, when I do data work or building agents, normally I need some files, some tools, some libraries to use, and you can basically provide them inside your environment when you create it as a GitHub repository, as a GCS bucket, or directly as inline files. And then those files will be mounted into your environment. And we support private GitHub repositories and private GCS buckets. So if you have some sensitive data that's all possible and available to integrate.

25:04 And it's all nice. But if I need to provide all of those resources configurations every time I send a request, it would be very painful. And it also would be very easy to make mistakes. So we created around the anti-gravity agent, the agents API, which lets you define an agent and an Agentforce, is really more a combination of files rather than some weird complex Python loop. It's really. You can define an M.D. file or system instructions, which gives a role to your agent, which has certain patterns, certain ideas you can provide skills through, all of the ways it can be inside gcc's GitHub repository.

25:43 And then you provide an environment. The environment can be remote, which basically means the agent gets a clean environment, or you can use an already existing environment. So if you think about I want to create a GitHub code review agent, I want this agent to be able to use the GitHub CLI to comment on my pull request to maybe open a new pull request. So I would like my environment to have the GitHub CLI installed. And the easiest way to install that CLI is basically I ask the agent, hey, please install the CLI and then I take the environment ID.

26:17 So basically that's a unique identifier for the environment which has the GitHub CLI installed. Use that environment to create my agent, and every time I create a new agent, it will have this pre-created environment with the GitHub CLI available, and it can use it for all of the tasks. And to make all of that easier for you. We created a small experimental CLI for the Gemini API. So that's not a coding agent. It's not like Cloud Code, not like Gemini CLI, not like anti-gravity CLI.

26:46 It's an API CLI to use together with the API. So you can run a CLI command which calls a Gemini model or run a CLI command which creates an agent or helps you create an agent. And in addition to this, we have agent skills for the interactions API, for the agents API, and also for the Gemini API CLI. And now let's look at some easy ways for you to get started without writing a lot of code.

27:14 And as Mariano already said, the easiest way for you to start is AI Studio. So you can go to AI Studio eight Studio or I dev and we are no longer in this build area. We are in the playground area and inside the playground area. We have the model picker on the right side where you now have agents and inside of agents you can select the anti-gravity Agent Preview or you can select deepresearch if you would like to use deepresearch.

27:44 And when you select anti-gravity, you get basically six cards with different pre-built agent templates. They are all open source, so if you would like to change something or if you would like to get inspired by them, you can pick one of them. And to start simple we use the anti-gravity preview. It's currently not available in free tier. We are working on it, so hopefully in the next two weeks, you should be able to use it through some free tier capacity.

28:09 But until then you need to connect. Your API key should take one click and then we have some example prompts. And what I like to do in the beginning is like, hey, I ask my agent, and how does the repository look. You are working in tell me a little bit about it. And when I run this command, what happens behind the scenes is that the environment is created. So we start the sandbox for the agent should take around 8 to seconds for the first time.

28:38 Yes and then the model starts reasoning it sees OK, we need to investigate and then it starts running and calling tools. So it runs a Bash command and to explore the environment reasons more. And then we should get back a very quick overview of where we are running in. So we are running in an Ubuntu. We have Linux installed. We have AMD virtual CPU cores and a dedicated cores gigabytes of memory. We have Python installed pip installed curl.

29:06 Great OK. So we have a little bit of our information about the environment. And to show you a good example on what you can build with this, we have created this AI talk show applet. Maybe let me quickly go back to where we are. If you want to test it yourself inside build gallery. There is the first one is this I talk radio so it's available for you to test. And there is a free tier. So there's a daily allowance of free talk shows.

29:34 So you can basically generate and you can sign in with your Google account. So basically all the things Mariano has shared earlier we used to build that applet. And I want talk radio show about how well Germany is doing at the World Cup. And now, similar to our previous request, we made a single API call. This API call now starts the environment and it should start soon. Reasoning and then we should see in a second that it uses different skills to build it.

30:10 Yes we are now we are first installing packages and then doing other stuff. Since this will take a little bit of time, we will look into how all this works and looks so the beauty of I would say applets or AI Studio build is that all of the code is like for you available to look at. So we can look at, OK, what do we really have. And in the I talk show radio we have this agent folder.

30:32 And the agent folder is basically our talk show radio agent. And it's not a complex Python loop with a lot of tools we need to call. It's really mostly flat files and markdown files. So we have an agent file, which tells our agent, hey, you need to turn any content into a radio show. You should work in the Workspace directory. And then there's a workflow describing OK, first do some research, then write some script. Then you need to generate speech, generate music, mix them together and generate a cover image.

31:05 And to do all of those different things we have skills. So if we look into how to generate speech, for example, we have the TTS generation skill which explains how you can generate TTS. And then we have a Python script which let me scroll a little bit. Yes uses the interactions API with the TTS model. So our anti-gravity agent, which runs inside a Gemini API, now has skills to call other Gemini models to do certain tasks.

31:35 And all of what it required is basically a single Python file and markdown files to give him all of those new capabilities and to keep us no longer waiting. I pre-created an example that's basically the result of it. We have a title, we have a nice thumbnail. We can listen into it, a correct physical order to preparing your breakfast Bowl. Or are we flirting with culinary chaos. Welcome to AI talk radio. I'm Paul. Broadcasting from our London Studio today, we are exploring the great breakfast debate serial first versus milk.

32:12 First, we will objectively break down the pros and cons of both methods. Let's go to our lines. We have David calling from Boston. David, you're on the air. Yeah thanks, Paul. Look, it's got to be cereal first. It's basic logic. Cereal is the independent variable. You pour it, you see the volume, then you add milk to complement it. Plus, if you pour dry cereal into a Bowl of milk, it splashes everywhere. It's a total mess.

32:43 And the mashed potato analogy, you wouldn't put gravy in a Bowl and drop a lump of potatoes on top right. A striking analogy, David. Cleanliness and portion control are certainly strong arguments. Let's hear the counterpoint. We have Sarah on the line from Sydney. Sarah, welcome. Hi, Paul. OK, look, David has it all wrong. Milk first is. It's a total game changer for texture. You can listen to that example, if you want to hear why Sarah thinks milk first is better.

33:16 We only have a little bit of more time left, so I would like to show you some more examples on what you can build. But before we get into it, I know it's like it always sounds very cool and very easy to use, but to show you what really happens behind the scenes, we have this applet here to show you what's really going on. So I have a prompt it really will call the API. The prompt says, hey, write a Python script which prints the Fibonacci number and execute it and then return back.

33:41 When I click Execute, we will send a single Gemini API call. We start the sandbox which will load all of the skills. So the model is now context for the skills. And then we should start seeing soon calls between the Gemini model and our sandbox. Normally when you build an agent, the model generates some kind of a function call, which you need to parse and then execute on your client side. And then you have this loop.

34:06 That's why we call agents a loop basically between your client environment and the API, where you go back and forth and with managed agents, you simply just send a single request, which we have seen in the earlier version. And also the AI talk radio is available in our agent managed agent section. So I can use the same agent which we have inside that applet, which generated a talk show radio directly from AI Studio because it's just files.

34:35 So we on the right side. Maybe I was too fast. We have an environment section. When you are in the agents and you have sources and in here you can see, hey, it's the same agents and it's the same skills. And you might be thinking, OK, my agent calls the Gemini API. How is this secure. How will the agent not abuse the credentials for it. And to solve that, we built a network proxy around it, which basically allows you to define what URLs or domains the agent can access.

35:08 And also B allows you to provide header heat of transformation. So instead of providing your credentials inside the sandbox, you provide them on the proxy level so the agent will never receive your real credential. It will just know that I can call the Gemini app API. And when it makes this outbound call to the Gemini app API, the proxy basically injects your credentials and then the model receives the result from it. So it thinks it can use the API, but it never has access to the credential, which means you can easily rotate them.

35:37 You can easily remove them or you. You don't need to worry too much about what the agent will do because it does not have those real credentials available. And to help you get started, use the coding agent skill. So inside the Gemini API docs, we have a coding agent setup where we have a single line install commands for any coding agent. You can then just prompt it. Hey, help me build a Gemini agent. And if you want to walk through it yourself or look at the docs yourself in a Gemini API, we have this agent sections on the right where we have all of the important information on how does it work with the environment.

36:13 How can I create my own agent. And then there's also a new guide for the interactions API for you to get started. Great So what did we learn today. Hopefully so going from idea to production is faster with AI Studio. So really give it a try. Start there. Export to anti-gravity, stay there, build your own applications, whatever. Use the interactions API for generic workflows. If you have feedback for it, please reach out to me law I'm very happy to listen and hopefully improve it for you.

36:46 And then lastly, agents are no longer those very weird complex Python loops or JavaScript loops. It's really we just write files and then we provide those files to build our own agents. Thank you.