AI Evals Explained | How to evaluate AI Agents?

0:00 Here is something nobody really tells you. When you actually start to build AI agents. The hardest part isn't building the agent, building the agent is probably the fun part. The hard part is knowing whether the thing that you just built is actually any good. Think about it. You wire up an AI agent and you give it some tools, you run it once, it does something that looks kind of magical and you go, yes, it works, but does it?

0:24 Because it worked that one time that you tried, it's not the same thing that it actually works in production. And that gap, that little voice in the back of your head going, but does it actually work? That has a name. It's called evaluation or evals for short. And I'm going to be straight with you. This is one of the most important things that you need to understand if you ever want to put an AI agent into production.

0:48 So in this video, I want to walk you through what AI EVALs really are, why they matter so much in exactly how you figure out whether your agent is doing a good job or not. And I'm going to do this in the most beginner friendly way that I can possibly get. So if you've never touched evals before, don't worry, you're in the right place. So here's what we're going to cover. First, what an AI eval is, how is it different from the kind of evaluation that we've been doing in machine learning for years?

1:15 Second, what are the metrics, the actual numbers that tell you how good your system system is and which ones to use for different tasks like summarization, classification and translation. And third, the part that ties it all together, the workflows that you can actually run, which starts with something called as a golden data set. And look, this isn't just theory that I'm throwing at you. So many people right now are spending so much time learning to build agentic AI systems, which is great.

1:43 That's where the whole field is going to. But the teams that actually get these agents into production, the ones that are really building coding agents and real customer support agents, they all have one thing in common. They're not guessing or doing vibe evals, they are actually measuring them. So by the end of this video, you'll have a good understanding of evals better than most people who are trying to build AI agents today. So if some of the terms that I spoke about sounded intimidating, don't worry about this.

2:12 But by the end of this video, it won't. And a quick thing before we get started, everything that I'm going to be discussing right now. And all the resources will be available in the description below. So you don't need to scribble down everything. Just watch it and you can grab all of that later. And for anybody who's new to my channel, hi there. I'm Iswarya Srinivasan. I spent over 10 years in machine learning and AI. I've got my Master's in Data science from Columbia University and I've worked as a data scientist at companies like Microsoft, Google and IBM.

2:41 I've also led developer relations at one of the top startups called Fireworks AI. And currently I'm co founding the Gen Academy, which is an AI skill building platform that's all about the real stuff that teams are building right now in production. And evals are exactly the kind of things that we teach. We are not just teaching individuals, but we are also doing AI training for enterprises. So, you know, all of this is coming from a credible source and somebody who has actually done this in practice.

3:08 Okay, so now let's get started. What is an AI eval and why does it matter? Honestly, it is simpler than it sounds. An AI eval is just a structured way of measuring how good your AI outputs really are. That's pretty much it. So instead of eyeballing a couple of answers and going like, yeah, that looks fine to me, you actually set up a little repeatable process that gives you an actual score. And that score really answers one question.

3:35 The only question that matters here is my AI agent getting better or did I do something to break it? See, every time you change a prompt or you swap a model or you tweak your retrieval and eval tells you whether you helped that agent or whether you silently broke something. Now I can already hear some of you going, wait, isn't that benchmarks? And that's a totally fair question because I'm sure you would have seen those leaderboards, things like SWE bench for coding, TAO bench for customer service agents, terminal bench for command line stuff, and Agentic Analysis Index that basically mashes a bunch of these together into one score.

4:13 So let me clear this up for you because it really matters. Benchmarks do test the model, but they are a generic test. The same test is being used for every single model. And AI evals tests your specific agentic system on your task with your data. And those are really different things. A model can be sitting right at the top of every leaderboard and still completely fall apart for your specific use case. Because guess what? That benchmark never saw your data.

4:42 It never saw Your prompt or your users. So benchmarks are great for picking which models to start with, but evals are really how you find if that thing actually works for you. And we are here for the second thing. Now, here is why this matters so much for AI agents specifically. An agent isn't just one answer. It's a whole chain of decisions. It picks a tool, it calls it, it reads the result, it decides what to do next, and on and on and on.

5:09 There's a lot of places where these things can quietly go sideways. So when your AI agent breaks in production, you need to know where, you need to know why. And let me tell you, you're not going to figure that out by reading transcripts once, one at a time, at 2am in the morning. There's actually research this year showing roughly one third gap between how models score in the lab and how they do in real deployed work.

5:35 So AI evals are how you close that gap before your users actually feel it. Okay, so that's basically what AI eval is. But here's a thought that I'm sure some of you are probably having. If you've done machine learning before, you're sitting there and going, hang on, we have been evaluating models for decades, so what's the big deal? And you're probably right. So let me show you what actually changed. Because there's a whole reason why AI evals is trickier than it used to be.

6:01 So yes, evaluation is not new. We've been doing it for machine learning and even for statistical models. But I'll be honest with you, old way was a lot easier. And it's worth understanding why. In traditional machine learning, the output was usually really clean. It's either a category like spam or not spam. A number like a predicted house price, a simple yes or no. And because the output was that clean, grading it was also clean. You had a test set with the right answers and you compared that with the model's prediction against those answers.

6:34 And then you can easily calculate metrics like accuracy, precision, recall, F1 score, et cetera. It was very objective. Everybody agreed on what correct meant and you could actually slap a hard number on it. Think of it like a multiple choice test. The answer is A, B, C or D. There is an answer key. Now you just count up how many times the model got the answer right. Pretty easy, right? Now here is what changed. With large language models and AI agents, the output isn't a tidy category anymore.

7:04 It is an open ended text or a whole sequence of action that your AI agent is taking. And most of the time there isn't just one single correct answer. Like if you ask for a summary for an article, there could be hundred good summaries and hundred bad ones. And two great summaries can look completely different from each other. So you can't really check it against any answer key because there is no answer key. And that is the shift that we're seeing right now.

7:30 This is the thing I really want you to understand. We went from grading a multiple choice test to to grading an essay. And grading an essay is just harder, right? There's no single right answer. You need a rubric, and somebody has to actually use their judgment. That's exactly why AI evals are more messier and more qualitative than the ML evals that you might be used to. But here's the good news. Once that rubric mindset clicks for you, everything else will become simpler.

7:57 So let's talk about what actually goes in that rubric, which is metrics. Metrics is one of the places where a lot of people get confused because there's a dozen of them and it all feels like Alphabet. So let me make this simpler for you. There's only really a handful of the core metric that you need to reach for again and again. The big ones are relevance, which is just did the output actually answer what was asked?

8:21 Then we have faithfulness, which is also sometimes called as groundedness, which is is this actually true and backed by real data, or is the model making it up? That one is your hallucination check. Then we have correctness, which is did it match the expected answer when there is one? And then we have coherence, which is does it actually read well? So relevance, faithfulness, correctness, coherence. These are the top four metrics. But here's the thing. The right metric totally depends on your task.

8:51 Let me show you what I mean. Say your agent is doing summarization. Then you care about faithfulness, which is did it stay true to the source without inventing stuff? Then we care about coverage, which is did it actually capture the main points? Then we have conciseness, which is did it Keep it tight. Now let's say that your agent is doing classification. Good news. This one looks like an old school machine learning. Right? You're right. Back to things like accuracy, precision, recall, and F1 score.

9:18 Because the output in this case is going to be a clean label. But let's say you're doing translation. Then you care about whether the meaning carried over, whether it really reads naturally in the other language. As well. And then you'll see metrics like blue and comet show up here. And if you're doing rag, which is Retrieval Augmented generation, you measure things like faithfulness and answer relevance because that catches the two totally different ways a RAG system breaks bad retrieval or bad generation.

9:48 So a quick pause because I want to walk you through something that I built this week that actually changed how I work, and I think a lot of you will find this useful. A bit of the context First. Between my devrel role running Gen Academy and making content, I'm constantly building demos every single week on different topics, different frameworks, different SDKs. And it's not just code. Every demo requires a readme, a slide deck, a script and a social post.

10:13 Now doing all of that every week is genuinely a lot. So I built a demo generator for myself. I set it up once and now every demo is just one command in my terminal. I typed the topic, I mentioned the toolkits I want to use, and out comes a full demo package. And I built this using Mistral Vibe. If you haven't seen it yet, Mistral Vibe is Mistral's AI terminal native coding agent, which is powered by their Mistral Medium 3.5 flagship model.

10:39 It runs in your terminal and your IDE. It understands your full code base and helps you write, test refactor and deploy using natural language. The feature that made this whole build possible is custom subagents. You define specialized agents for targeted tasks, give each of it a specific scope, tools and permissions, and a main skill delegates to them on demand. Let me walk you through how I set it up. Custom skills in Vibe are defined as Skills MD files.

11:07 It's a markdown file with YAML front matter that declares the slash command and the workflow behind it. I wrote one called Generate Demo inside it. I tell Vibe to ask me clarifying questions first because Vibe has a built in multi choice clarification applications and then delegates to my sub agents in order. One does the research, one writes the actual demo code, runs it in my terminal to confirm that it works. One drafts the readme and the slide outline and each sub agent has its own TUML config file with its own system prompt.

11:37 The doc writer reads from a voice file where I've written out my tone rules so that the output actually sounds like me and not a generic AI copy. Now once all of this is setup, I just type Generate demo in my terminal and the whole workflow runs. What used to take me a full afternoon now takes me a few minutes. So if you Are a developer building a lot or you're in developer relations, or you're teaching and shipping demos.

12:01 Go check out Mistral Vibe. The CLI is included in Vibe Pro or Team, and you can also directly launch Vibe tasks if you want to start in the cloud. You can also teleport from your CLI to Vibe midway through session. My link is in the description along with the full configuration for this setup, so you can fork it and build your own. Okay, now back to what we were building. Now let me zoom out for a second because here is something which is going to save you a lot of confusion down the road.

12:27 Under the hood, all of these metrics really fall into three buckets and it just helps to know them by name. The first bucket is classic overlap metrics. These are the old reliable ones from natural language processing. Word things like Blue for translation, Rogue for summarization, and Meteor. And all they really do is check how much of your output word overlaps with the reference answer. They are fast, they are cheap, and they are dead simple. But, and this is a big but, they are basically blind to meaning.

12:59 So two sentences that basically say the exact same thing in different words can be scored differently. Now the second bucket is semantic metrics. So things like BERT Score, Blue RT and comment, instead of just matching words, they're actually comparing meanings using embeddings. So they can handle paraphrasing way better than the overlapping ones. And then the third bucket is model based metrics. And this is where you literally use an LLM to do that grading for you. So this is your LLM as a judge plus newer named methods like geval.

13:33 And these are the only ones which are flexible enough to grade open ended messy creative outputs with with no reference answers at all. Which, let's be real, is what most actual agent tasks look like. So you see the pattern, right? Different tasks, different metrics. And more and more these days you see a model doing the grading. Which brings me to the single most important habit in this whole video. This is the thing I really want you to walk away with.

14:00 Don't start by grabbing metrics off a list. Please don't do that. Start by building a golden data set. And a golden data set, honestly is not complicated. It is just a handful of example inputs paired with what a great output looks like and what a bad output could look like. Things like your common cases, your annoying edge cases, and every failure you've already watched blow up. Think of it like an answer key that you've been writing for your own specific task.

14:28 So start small 50 to 100 example is genuinely plenty to get going and you're obviously going to grow it over time. Here's the part that really clicks for people. Once you've got that golden data set sitting in front of you, you work backwards from it to figure out your metrics. You just look at your examples and you ask yourself, okay, what would make this output good and what would make it bad? And those answers, the quantified way of judging those answers, are your metrics.

14:58 So the data set comes first and the metrics fall out of it. So now you got your data set and you know what to measure. The next question is, how do you actually do that grading? There are four ways to do that and let me walk you through them. So the first one is human evals, and this is exactly what it sounds like. A person, ideally somebody who knows the domain, sits down and grades the output against your rubric and that is your gold standard.

15:22 Nothing beats it for quality. But the catch is that it's slow, it's expensive and it does not scale it. You're not going to have a human grade every single request. So you save this one for small and careful sample and you treat it like a source of truth. The second one is user evals. And this is also a real signal from your real users in live production environment. Did they do thumbs up or thumbs down for your answer?

15:46 Did they accept it or did they regenerate it? Did they edit what you gave them? Did they finish the task or not? Did they come back the next day with the same question? Again, this is a ground truth that actually matters because it's real people using your product. The downside is it can be noisy and you only get it after you've actually shipped, so it cannot be your safety net. The number three is code based eval.

16:11 Sometimes people also call these as programmatic evals, and these are just simple checks that you write in code. Did the output match the expected value? Is the JSON valid? Did the agent call the right tool with the right arguments? How long did it take? How much did it cost? These are cheap and they are instant. You can run them automatically on every single change. But the catch is they only work when you have clearly checkable answers, which by the way, is exactly why they pair so beautifully with classification tasks.

16:44 And the fourth one, and this is the one that completely changed the game, LLM as a judge, you take a really strong and large model and you use it to grade your outputs against your rubric. And this is what finally lets the Essay style, grading actually scale. It can judge the open ended stuff that plain code never could. Like is this helpful or is this grounded to source? And it can also do that for thousands of outputs.

17:11 There's one rule though, and it's a real one. Your judge model has biases. It tends to like longer answers and it tends to like whichever response it happened to see first. So you only trust your LLM judge after you've checked its grades against a human on sample. Keep it honest and it becomes your absolute workhorse. So those were your four graders. Now the obvious next question is how do you put all of this together into something that you can actually run week after week instead of doing it once and forgetting about it?

17:42 And that, that's called loop. That's the last piece that we'll talk about. AI evals are not a one time report card that you can run right before you're launching and never touch it again. They are actually a loop, something that you run over and over again. The easiest way to think about it is like automated testing for software. You don't just test your code one time and then just walk away, right? You run your tests on every change forever.

18:08 It is the exact same idea. So in software you do CI CD and in AI agents you do CICD plus continuous evaluation and continuous monitoring. Okay, so let me give you that cycle, which is nice and simple. You start by deciding what good even means and you have to get really specific about it. Then you build the golden data set that we talked about. Then you pick your metrics by working backwards from the dataset. Then you run a baseline, which just means that you run your current system across the whole data set and you get a number.

18:41 And that number is actually your starting line because without that baseline it feels better. Is not really data, it's just ybal. So you start from your baseline and try to keep improving on top of it. And then after that comes my favorite step. And it's the one that everybody rushes right past. You go and look at the failures and you group them. Are they all coming from retrieval misses? Are they all coming from formatting screw ups?

19:06 Are they all coming from same weird edge cases? Because that grouping tells you exactly what to fix in your AI agents. Then you fix it usually by tweaking your prompts or by your retrieval or just your tool definitions. And then this part is crucial. You rerun against the same exact data set. Did your numbers go up? And just as important, did you accidentally break something that used to work earlier? That right there catching thing that you broke without even realizing that is the entire reason why the loop exists.

19:42 There's actually an interesting concept in AI agents called whack a mole. I'll add a link about it below, so definitely go and read about it. So now, coming back to the loops, you just keep going with it because this honestly never stops. Once you are in production, you're always watching your live traffic because here's the fun part. Your model provider can quietly update the models underneath you and your behavior just shifts overnight without you touching a thing.

20:08 So your real world failures flow right back into your golden data set and you run the whole thing again. And the good news is that you don't have to build any of this from scratch. For running the evals themselves, there's tools like Prompt Foo, which lets you test prompts and run evals right from the command line, and a library like Ragas, which gives you those rag metrics and straight out of the box. And for watching what's happening, introduction, the tracing and the monitoring side, you have tools like Langsmith, Langfuse, Arise, and Braintrust.

20:41 So look, the teams that are really winning in this game, they are really treating evals like continuous testing. They're always running with it. They're catching the problems before user even see them. And the teams who are actually getting these things broken, they're actually running it once they're shipping it. And then they just find about every single bug from angry users. So don't be that team. Okay, so I know we covered a lot. If you take away just one thing from this whole video, let it be this.

21:08 You will never ever know if your AR agent actually works by watching it succeed one time. You only know by measuring it on your data again and again. And that is what AR eval is. And that is the skill that separates people who are just building demo agents from the people people who are actually shipping them. Now if this video lit something up in you and you're sitting there thinking, okay, I actually now want to build AI agents right away, let me tell you how to go deeper.

21:36 As I said, I'm co founding the Gen Academy with Arvind Narayan Murthy and it's an AI educational institute. Our entire mission is to train people to become top AI builders in the world. And it's genuinely built for everyone. On a modern day, both the technical builders, the software, software and AI engineers, and not technical folks like product managers, PMMs, GTM folks, people who are working in sales, etc. In today's world, everyone can be a builder Our flagship program is a six week mastering agent to KR certification.

22:06 Every single week we are conducting live classes and we are teaching live. Something that I'm super proud of is that we have partnered with companies like Nvidia, OpenAI, Fireworks, AI, Pinecone, 11 Labs, 11 Llama, Index, LangChain, MEM0replit and so many more. So you're actually going to be getting credits from these companies and we are also going to be bringing their top AI experts to come and do guest lectures. One more thing, this certification is usually covered under the learning and development budget at most companies, so you can actually get this reimbursed by your employer.

22:39 I'll add all the details about the certification in the description below so you can go check it out. I really hope this was helpful for you and if it was, please do make sure to subscribe to my channel. I post twice a week and do a lot of these deep dive AI explainers and also share my own journey building a career as a AI leader here in the US as an immigrant from India. And if you have any questions, please feel free to drop that in the comments below.

23:02 I actually read those and I try to answer as much as I can. Okay then, I'll see you in the next one.