How to vibe code securely (without getting hacked)

0:00 AI hallucinates and creates security holes. How is anyone using this to write production code. Aaron well, developers aren't perfect either. So what we can do is shift our focus from who or what wrote the code to how stable and secure it is. So here's how the best teams are using it. Welcome back. I'm here with Aaron, a developer relations engineer at Google who has been thinking about security in AI assisted development.

0:35 Exactly I want to help people use AI without inadvertently shipping something easy for the wrong people to exploit. So how do we build a process that catches security flaws, regardless of who or how the code was written. So to do that, we bake in pervasive security. The Dora research project has a good write up about it if you want to know more. But think of a coding agent like a brilliant intern. High potential, but you'd never give them the keys to the Kingdom without some guidelines in place.

1:03 And how do we do that in practice what we should focus on small batches, context engineering, access control and sandboxing and external verification. All right. First on your list is small batches. I guess that's where the problems start. How does it work. Well, Dora shows that small batches are the key to stability when writing code. Here's the research report where you can read more about it. Small batches give you two things.

1:33 They give you easier, more focused review, either for a human or an agent, and they give you clearer intention and test coverage. So one test per small new commit reflects a user behavior. So even if an agent wrote the code, it should be reviewed. Yes just because you use an agent, there's no reason to throw your existing processes overboard. In fact, agents are really just an amplification of existing processes, whether those are good or bad. So who reviews the code and when.

2:06 Well, either a developer or an agent can review the code before it gets committed, so you can do the refactoring right there with your agent. This makes pr's less of a burden, and it's even making trunk based development more feasible for some teams. You can progress from developer to agent review as you grow more familiar with the agent's capabilities are. And you also mentioned writing tests for each commit. How does that work. So you can think of the test as narrowing the agent's focus.

2:36 It tells the agent exactly what to solve for. Plus, it gives us a real safety net so we can refactor and change things later without worrying about breaking anything. So first you write a test, and then you write your code to get the test to pass, and then you add any other refinements. Exactly and you may have noticed that agents tend to plan upfront. You give the agent the feature or fix you want to see. The agent breaks it into smaller red green refactor loops.

3:04 That could be a lot of tests. Aaron it could be but consider that each test covers a specific behavior. Writing tests does two things. It lets you know the code you wrote does the job. And it makes sure you don't break it when you introduce new things. Got it. And how do you actually implement all that. You can either be really specific in your context files like this, or you can import a skill into a coding tool.

3:29 Using a skill like this is especially helpful if you want to standardize this approach in a team of developers. All right. So after we created a small batch of work. The next item on the list is context engineering. Is this more than just a long system prompt. Much more. You're setting the rules of engagement across your project. Think of it as giving your AI intern a style guide, a set of security requirements, and the context of what you're building.

3:57 OK, and what kind of context. Always validate inputs or sanitize SQL. But keep it tight to avoid context collapse and too much token consumption. Think of context like a meal. It should be nutritious and reasonably portioned. Not enough or too much. And we're going to get sick. Only give the agent what it really needs. All right. And how do we put the agent on a context diet without losing the big picture. Well, I like these two techniques for keeping the context tight.

4:30 Keep a Gemini app context file in each directory or at skills. All right. And could you walk us through these two techniques. Sure a Gemini app file in a specific directory tells the agent the local House Rules. For example, your auth directory would have specific rules for session handling, but the payment directory has rules for how to use a third party API. Now, adding skills is the other way. You're giving the agent a specialized plugin of instructions that it only triggers when it's relevant to the task at hand.

5:02 We defined our small batches and that context is set. Next on the list is access control and sandboxing. If the agent has local file access, couldn't it run malicious commands or delete or modify other parts of the system. Can't we just use a sandbox. Well, sandbox is intended for preventing accidents, but the way it's often implemented is not a security control. What many platforms call a sandbox is often just a Docker container. Wait, a Docker container isn't a secure sandbox.

5:33 Well, not by itself. So containers only isolate the user space, but can still make many system calls to the host kernel. An attacker can escape the container and access your host kernel through them. A true sandbox like gvisor sentry diverts the vast majority of syscalls in an isolated execution layer. So if prompts and containers aren't enough, what's the real solution. Good old defense in depth controls external to the agent itself, so you can apply the principle of least privilege with granular OAuth scopes.

6:04 You can require a developer's confirmation for sensitive terminal commands, and you can have a way to verify the skills and code our agents use are not themselves malicious. So basically we need an external Skywatcher to watch the watchers. Yeah that's right. OK, so that brings us to the last item in the workflow, which is verification. Clarification this is where we check the coding agents homework. Yes and this is different from letting the agent write its own tests.

6:35 So because here we're using external systems for peer review we can use deterministic analysis or probabilistic. OK let's start with deterministic analysis. These are the code analysis tools we had before I coding. Do we still need them. Absolutely see they give us consistency and predictability in what we're testing. We use Sas to analyze code line by line and SCA to check your dependencies. Apps use a lot of outside libraries, so it's kind of a waste of power to have an agent scan them manually itself.

7:11 It's faster and more reliable to just let specialized tools handle the heavy lifting. So these tools should run automatically in the Ci pipeline. Exactly or even in the Ide. But their main problem is they tell if there seems to be a vulnerability, not necessarily if it's exploitable without configuration. This leads to a lot of false positives. And it creates friction for developers. Got it. And as a developer, can I ask the AI for help with fixing them too.

7:41 Yes, you can just give your coding agent instructions to fix the issue, and because you have good test coverage from working in small batches, you'll know the agent's security fix won't accidentally break something else in your application, which does happen. What about vulnerabilities that are hard to find with line by line analysis. Well, right. Things business logic flaws or certain authorization bypasses for those you can instruct an agent to emulate an adversary. So essentially red teaming and not just vulnerability scanning.

8:14 Beyond individual vulnerabilities, this helps you see how a bad actor might abuse your architecture as a whole, which is a very powerful way to test your app. Aaron, this was a lot of new material for me at least. Could you give us a high level summary Sure Here it is. So first is small batches. Use test-driven development and keep changes tiny, testable and reviewable. Then context set the rules of engagement with local context files or modular skills.

8:46 Then limit your agents scope with access control and sandboxing. And finally verification. Don't trust blindly. Use traditional SCA and SaaS tools to catch the gaps and try out red teaming agents to emulate adversary behavior. Code with vibes but verify with tools. That's a great checklist, Aaron. You've clearly spent more time thinking about this than most of us. Thank you for sharing all this with us. And as always, thanks for having me, Martin, and thank you, everyone, for watching.

9:16 If you have any questions for Aaron or me, please let us know in the comments. Also, do let me know what you thought of today's episode. I read every single comment. Until next time.