How to Build AI Evals Step-by-Step | Daniel McKinnon | Product Growth

0:00 The Metas and the Googles and all the other large companies have to reinvent themselves right now in the age of AI. Every single PM is going to start building AI features. You cannot exist as a PM without understanding this. >> Meet Daniel McKinnon, former PM on the llama models at Meta, now a startup founder and a master of eval. >> PMs right now from a career perspective are in a really tough situation. The average PM is an orchestrator, a motivator, and an analyst. But a lot of this is easy to do with AI.

0:30 >> What really is the difference between product management at Meta versus Google? >> Meta is like a much much more aggressive culture. in many ways, Google is considered to be more of an engineeringled company whereas Meta is more of a productled company. >> If you're a PM who's never worked on an AI feature before, when would you be going through this ebalance process? >> You work at Pinterest and you want to have better image generation that doesn't look like sloth that actually pleases users. You need to think about from day zero. What does success look like?

0:58 >> If it's the best way to measure it, we've got to learn it. Where should we start? >> Let me walk through. Before we get into today's show, please take a second to check that you're subscribed on YouTube and following on Apple and Spotify podcasts. If you want access to all of my favorite AI tools, I've gotten them to give you an entire year of their paid plans. Check out bundle.ac. akashg.com for an entire year of bolt new air table speechify descript magic patterns linear dovetail arise and mobin and now into today's show Daniel welcome to the podcast >> yeah thanks so much for having me and it'll be fun to talk about this stuff >> so I want to start with your article you had this provocative claim and this funny meme here do evils replace the PRD what is the role of evils >> yeah so I was like a little bit spicy in saying it replaces the PRD because without a product strategy or a particular customer like your product is nothing but again that's a paragraph or even a sentence depending on what the product is. The majority of most PRDs that I've seen in my career have spent most of the document talking about specifics, how it will work, how it will behave in certain situations, how the user can expect to get value from that.

2:22 But that gets really turned on its head in this kind of Gen AI world where these products really need to do like everything or at least a lot more things than previous products. And it's very hard to describe that, say, oh, this thing just does everything. And the best way to actually communicate what the product should do is through examples. And that's what an eval is. It's really just like a trivia question for the model. And it's saying this is like the shape of the things the model needs to do well. And if the model does it well, it means it's getting these answers. And if it gets these answers, the users will probably like it. And if the users probably like it, let's ship it into prod and see if they actually like it with an online eval. But the key way to communicate how a product should work in this kind of Gen AI era is is it performing well on an offline eval? And if not, you either need to change the model, change the harness, or change the product. It's possible that what you want to do is not possible with the models today. But it's better to find that out early with an offline eval versus just shipping it to prod and getting frustrated users.

3:22 >> And for people who don't quite understand that nuance, what's an offline eval versus just shipping to prod and looking at them? >> Yeah. Yeah, this is a really really important nuance and I touched on it in this blog post, but when we usually talk about evals in this AI world is something that's run offline, there's like a little bit of gray areas in terms of RL environments and stuff, but think about it as like a pre-baked set of trivia questions that you ask the model.

3:48 So, for example, let's say you have a recipes website and you want to tell users how to make their favorite kinds of ice cream. An offline eval would be a prompt set, say 100 prompts of different ice creams that users might like. And the answer key would be a correct answer or a plausibly correct answer with a way to score whether it's good. So that's run offline during the development of your product and you use that as a proxy for real user traffic.

4:18 Once you do well on your offline evals, you can ship that product online and you have a website and it lets users generate ice cream recipes and you think because you are a good PM and you really thought deeply about what the customer wanted that performance on that offline email set will reflect the satisfaction of the user in the online eval. this obviously doesn't always happen, but that's the idea. And it's the best way to measure whether a Genaii product is likely to satisfy users.

4:48 >> If it's the best way to measure it, we've got to learn it. Where should we start? >> Let me walk through what's changed since I wrote that article. So the key thesis has remained true and this has been now two years in that an eval is the best way to communicate what your product should be doing and to explain to the engineering team working on making the product work what success looks like. This can be very very challenging in an AI world when they these products do so many different things that it's hard to necessarily understand what good is. But things have gotten a lot more complex.

5:23 So back in the day, evals were really simple. My blog post basically just covered these simple evals. They were question and answer. And this seems crazy to think back 2 years about what these models looked like and what the use cases were like, but this is before cloud code and agentic coding and all of these crazy business applications that are getting built right now. claude co-work and the like. Really the core thesis two years ago was that Genai was essentially a search replacement. I don't know if everyone remembers when Google's stock tanked because this was going to be the replacement for search and fundamentally these were like question and answer products. So what I have right now is the benchmarks that OpenAI reported on GPT4. And you might remember a lot of these MMLU, Hela, Swag, ARC, Window, human eval human eval is ironically an automated eval of Python. drop and these are all just question and answers. So I dropped one example here from MMLU. If you know the actual brightness of an object and its apparent brightness from your location, then with no information, you can estimate a speed relative to you, b composition, c size, d distance from you. And if you want to go ahead and look at some of these just for like historical fun, they're they're all on hugging face. So this was actually a very very straightforward thing to do is you had to think about the types of questions users would ask and the types of answers they would expect and how to score that. So the key thing was just matching the questions to the domain of interest and scoring the answers. So if we go back into my post here, we can see a little bit how I said to do this. And first thing is just figure out your problem and it doesn't need to be perfect. What is a set of problems that your users might ask about? So the example I used was for example generating recipes from videos. I guess I have food on my mind because I randomly came up with the ice cream example earlier. And new problem is I have a video on my social media site and I want to be able to generate a recipe that somebody can use to make that thing as they're watching the video. And you say, "Okay, this is really well defined." and now you have to measure h how do you know if it's good?

7:45 You know, it might be formatted right. It might have all the ingredients listed. It might be written in the right style. And then you select all of these components and figure out how to judge if it's correct or hypothesize how to judge this correct. This can be a auto score. This can be another LLM. This can be a human. You can judge correctness many different ways. Once you have that, it's really a mechanical process to actually write the eval. Just come up with probably 100 prompts that are in this distribution. Can be less, can be more, but this is a typical size of an eval in a genai world. and just send it through the model and figure out what is a hard prompt, what is an easy prompt, and have something that scores maybe like 50%. Cuz you have to have room to run. If you create a very easy eval that scores 100%, there's no way for your engineering team to optimize on that.

8:33 And if you create a very hard eval that scores 0%, you also don't even know if this is kind of possible with today's technologies. and that's pretty much it. And once you have this you can give it to the team, they can improve on it. You can ship it out to users. If you achieve a score high enough, you can see if you're actually online performance matches what you expect it to do offline. And then this blog has a bunch of examples of like, you know, some kind of other nice tips. But why is this not that relevant today or why does it need to change today? The real answer is that models have generally saturated QA. This isn't 100% true, but when we think of a good model right now, we don't think of one that can answer a relatively challenging high school physics question like the example I gave above. We think of models that are like winning gold medals at international math Olympiads. Like there's almost no question that any human beyond some super super specialist can ask the model to do and it not have a good answer back. And also QA is not the most useful application now. I mean I think we all remember this narrative that chat GPT was going to be the next great consumer app and they were going to get all these users and it's QA and you're answering all these problems and you know Google did a generative search experience. But if you look at all the headlines in Genai right now, it's not QA, it's agents. And this is actually reflected with how the labs communicate progress. Here's a quick word from our sponsors. If you're building anything that uses live data from the web, eventually you hit the same wall, an agent, a research tool, trends, dashboard. They all need fresh data.

10:15 Scraping that data is the worst part. Captas, proxy, layouts that change every week. It's a whole side project you didn't sign up for. That's where SER API comes in. SER API gives you clean structured results from Google, YouTube, Bing, Google News, Google Scholar, and more. One API call, one clean JSON response. They handle the captions, proxies, and layout changes for you. Take the Google Scholar API as one example. Say you're building a research assistant or pulling sources for a literature review. You hit one endpoint, you get peer reviewed articles back with full text or metadata, titles, links, publications, citation info, all of it across publishers and formats. No scraping a dozen publisher sites and gluing the data together yourself. The same idea extends across the rest of their APIs. Real-time Google search for an agent, pre-classified images for training data, Google News for monitoring, 99.9% uptime, 1.2 second response time. Get started with 250 free credits. Link is in the description or scan the QR code on screen. Thanks to SER API for sponsoring. Are you looking to up your AI product management chops? I highly recommend the AI product management certification by product faculty. It has a 47 with 1,249 reviews on Maven for a reason. I myself took the course back in 2024 and it was awesome.

11:42 Since then, they have upgraded it. So now you get to learn from product leaders at OpenAI and Enthropic. On top of that, Powell Hearn, author of the product compass, leads the build labs. So you will go from theoretical knowledge about AIPM to a very practical course. It's going to help you identify AI leverage opportunities. It's going to help you design trustworthy AI experiences. It's going to help you systematically optimize outputs for accuracy and relevance, build rigorous evaluation suites, architect AI agentic systems that work, and select the perfect LLM for your use case.

12:16 It's normally $2,500, but you get a discount when you use my link. The next cohort starts June 22nd and goes to August 9th. So, do check it out with my link in the description. They have been one of my longest sponsors for a reason. I trust this product and I think you should consider the cohort. So back here two years ago if you look at what how open AAI communicated progress it was these five eval I guess six evals if we fast forward to look at how anthropic communicated progress for opus 4.8 you can see they're using entirely different benchmarks you don't see any continuity it's partially because those eval are saturated and opus 4.8 8 would score effectively 100% on all of them. But it's partially because the task is really different. You'll notice we have agentic coding, agentic terminal coding, multidisciplinary reasoning. This is actually some agentic reasoning, agentic computer use knowledge work. This is actually agentic knowledge work and agentic financial analysis. All this means is what the core model task is is no longer to get a prompt from a user and come back with an answer. But it is actually to get a task from a user that requires many many steps. Some of these steps might involve just thinking which is called reasoning in this world. Some of it might involve tool calling something like search. Some of it might involve more advanced tool calling like something we'll go over today. And this is a totally new paradigm of writing evals because you're no longer thinking about QA. You're thinking about tasks.

13:53 Fortunately for us, the framework is largely the same. We still have to define the problem. We still have to be good PMs and know what we're solving. We still have to collect representative prompts. I call this Goldilock style. Again, they can't be too hard and they can't be too easy. There has to be some room to run. A typical good eval will have something like 25% 50% success rate and then over you know months that will go to 100% and then you'll have to throw it away and create a new one that is harder. And then you also have to figure out how to score. And one things that has changed is QA is relatively easy to score with humans worst case scenario.

14:32 There's exceptions to this, of course. The reason these models are so bad at things like creative writing is because it's hard to score and there's different preferences and different users like different things and why they're so good at math and coding is there is like some right answer and this is much easier to hill climb. But for to some extent QA style questions, you can ask human raiders to review worst case scenario if you can't find a better way to score it.

14:58 With agentic work, it's much more challenging because the time horizon tends to be very long and the final output is a collection of many, many, many steps that it took. Some steps could be correct and lead to the wrong outcome. Some steps could not be correct. And it's just from a labor perspective and a like defining success perspective, it's much much more important to get something that can be automatically scored to make more of these rollouts and understand how you can do more experiments. But again, it's largely the same. and the tasks are much longer time horizon. So kind of the goal of this podcast is to walk the audience through creating an actual agentic eval in real time. I want to caveat this.

15:42 This is a little bit pre-baked. It is unrealistic in 45 minutes to come up with a brand new eval. This is kind of like a weeks or months problem of deep thinking, but we're going to kind of pretend and we'll we'll go through some of the steps together. So, first problem, I want to measure and improve the model's ability to help with clinical genomics. This is a problem that I care deeply about. It's one that I think can improve the world and it's something that I've launched a new startup to solve. And the problem is is that whole genome sequencing has become the absolute gold standard in diagnostics in NICU settings. So for sick babies, unfortunately interpreting the results of a whole genome sequence is very labor intensive and it limits access to this life-saving technology.

16:37 So I wanted to see if I could distill some of this human expertise into a model to help broaden the accessibility of this technology. So first thing is like before we have to deeply understand the problem and I'm not going over these flowheets but this is just kind of how complex this is. Generally you start with the raw reads off the sequencer. You do a lot of processing work to identify how the particular patient differs from the reference human genome and then you do another set of work to determine whether those changes to the genome matter. For example, if my genome were sequenced, I would get about a billion reads that are 150 base pairs long. they would come out in a giant text file and I would need to transform that into a diagnosis that says that this gene may or may not be responsible for this patient's condition. And there's a very structured way of doing this. So, I'm not going to spend too much time on this here because we're going to go over some real examples, but you really need to deeply understand the problem. This is why people like Anthropic and OpenAI are hiring investment bankers, accountants, lawyers. As you see job ads for all these vertical specific teams, you must deeply understand the problem. You will unlikely be successful in creating an eval for some topic if you don't have some background in it or haven't really educated yourself on it. So then let's say we've understood the problem. I think I understand this problem pretty well and by the end of this podcast you will too. Let's go to the prompts. So again we want to find this like Goldilocks set of prompts. So the first thing I like to do is just start with something easy. So you want to make sure the model can actually do this. And when I say the model for agentic stuff, I'm usually talking about the model plus the harness. So I'll use those words interchangeably. But this is a frontier model harnessed in a way that it can use these tools and it can do this reasoning and it could come back with a solution.

18:45 So for this problem of genome interpretation, I picked like one of the easiest genetic diseases possible and this is cystic fibrosis. This was something that we have known the genetic cause for quite some time and there are like canonical genes that cause cystic fibrosis. So to save you the effort of me googling for this, I just had the link right here and let's just go ahead and look at this. So what we see here is the canonical cystic fibrosis mutation in ClinVar which is an NIH database for a lot of genetic disease. And what we see here is it's got four stars and three stars. This really should be four and four. This is like the canonical genetic defect for cystic fibrosis. So I'm going to make sure that my agent can actually get this before I go forward.

19:40 And so this is like the easy thing to start. So we're going to have our agentic genetics eval and we're going to say gene and we're going to say cftr2. This is the again canonical gene. And then we'll say variant. And what a variant is is how a particular gene is mutated. So right here this is again this is a somewhat niche eval but what this is saying is that on this particular transcript of CFTR at this position there is one base deleted and what that results in is the 508th fennel alanine deleted. So this is the actual variant that's going to exist in our eval. Sounds good. Okay. So what we see is that this is the exact variant that we care about. And what you'll notice is this is like very complex and nuanced. And this kind of comes back to the absolute first point I was making is you really need to know the space to do these evals. A lot of the eval are kind of like picked up like if you're trying to come up with an eval for Python coding like many of these are pretty good now. And for any of you who have used these models, saying Python coding is a solved problem is a little bit of a strong statement, but it is a very very well understood and well-characterized problem. So we will go to this and we're going to say okay so this is the thing we want to know and I should add a column here and this is the phenotype is cystic fibrosis. So what you see here is I'm starting to build a table of question and answer. So the question is I have this phenotype of cystic fibrosis which is is a lung disease and you know we'll describe exactly what happens there and then we have the genome of this patient and then we have the answer which is this variant. So let's walk through like how we would do that and when you're constructing these evals you're going to really really really use genai a lot to construct them. So, what I'm going to do is now I'm going to go over to a terminal window I have open here. And I'm using codeex. Any of the tools will work. I have it on 53 spark low because I want it to be fast for this demonstration. But, you know, you can use any any model, any anything you like. And for this task, this will be fine. So, what I have here, and I pre-baked some of these just to kind of make it go faster, but I want to walk through any step anyway, is I have actually two files here that are representing my genome. And what I want to do is create a synthetic version of this genome that has these variants that have the question and answer through this agentic flow that I want to get at.

22:36 So what I'm going to do is I'm going to say please add and then this is going to be this variant to and this is a small variant. So it's going to come through here and create a new file in a new folder and we're going to call this dan cfive.vcf.gz GZ and we're going to have this be Dan CF live and we're actually asking AI to help us create the eval for AI. So we're going to do this and what the model is going to do is a variance file is literally just a text file. Actually we can see what it looks like here just for fun. So while while this is running, let's just take a look just so we know what these variant files look like. And this is this is fine. We don't need to show all of it. But what we basically see here is a chromosomes. So you might remember from things like 23 and me. We have 23 chromosomes. So this is one. This is the biggest one. And this is a position. So your chromosomes have different number of bases. You might remember we have like three billion bases in our in our genome. And then these are swaps. So what we see is in this position a reference human has C and I have a CA here. So that means that you know during some I inherited from my parents or maybe something that emerged during my development I got a a base swapped here and then there's a bunch of metrics around quality how real it is. You you might remember you have two copies of each gene. So this is actually hetererozygous meaning only one copy is impacted. And you know, really, it's just a text file of all the letters in your alphabet. And what I'm saying is I want to add okay, this is still running. And if this is still running, in a while, I'll just use the pre-baked one. Is I just want to add this particular variant into my genome to see if our system can catch it. And this is what Codeex is doing right now. And I actually don't know why this is taking so long because this is like a oneliner.

24:53 but >> find the line I think or >> Yeah. Yeah. Right. Right. I guess I'd probably I didn't want to make this demo too pre-baked. I thought about it. Should I just have a a Python program that just does all this for you? But then I'm like, that would not help the users at all because when they're constructing their own, they wouldn't know how to do it. So, just believe me that this will work. And for the sake of time, we'll go to the pre-baked ones. Mhm. So, so basically what's going to happen is that codeex is going to add, this is in chromosome, where is it? I'm actually not sure. But in whatever chromosome this cystic fibrosis gene is in, it's just going to add one row and it's going to say we're going to have a deletion.

25:34 So instead of having like a CA here, it'll just have a C. So like here's a deletion. You'll notice that we've we've lost an A. We went from TA to T. And then it's going to have a a fake cystic fibrosis patient. So now let's just check to see how this works. And we'll say use our agent. And again these are all agentic evals. And I'm using codeex as our agent but you could use anything clin open code your custom harness any way you could to get these agents to actually operate. And in fact even chat GPT and claude actually in the web UI they use agents right now.

26:13 They're not just model in, model out, they're model in, reasoning, tools, everything. Okay, cool. All right, so the agent finished here. So we see we added this in a record. Okay, and we see here it is. It's in chromosome 7. And you'll notice this is deletion. It says TCTT and instead it's a T. And so this means that this is like the canonical CF. So we're going to see if our agent is able to do it. And we're actually going to try a few different agents because one thing I mentioned is you want to score like you know 25 to 50% on these evals. You have to think about what tool are you using the you know mythos 5 for some biod defense thing then it's got to be really really hard or maybe this is something that for infrastructure reasons or cost reasons you need to use a very small model you need to use haik coup or something like that. So, we're actually going to try these simultaneously on a few agents and see what happens. So, we're going to say inside, what was this file we had?

27:15 Dan CF live. Again, this is the one that we just made. We have the genome of a patient suffering from, and let's just very quickly copy and paste some cystic fibrosis symptoms. this one we suspect cystic fibrosis. Please find a genetic cause. And then we are going to we're actually going to copy this prompt so we can use it across multiple agents. So now we're going and we're we're we're checking GPT 3.5 codec sparklo. we have a few other tabs open. So, I mentioned, you know, maybe we want to actually see if Haiku can do this. So, we'll just upload this. And again, we're going to the CF live. And let's also try Cat GPT 5.5 extra high. And I'm not going to use Pro because it will take too long. And this is also a relatively easy task. I suspect all the agents will get them. Okay, great. So we are cooking with haik coup and we are cooking with you can see what's all already happened with our first agent is after one minute of thinking you've actually find the cftr mutation pattern consistent with this deletion right this is the canonical cystic fibrosis gene so what this is telling us going back to our steps is this task is not too hard for these agents at least in the easy case >> especially not with a powerful system like codeex. We'll see if haiku gets it.

29:00 I suspect haiku will also get it. But you can see haiku even itself knows the cftr region is important. So while that's cooking let's go back to our next step. So again we start with something easy just to make sure it's possible. I knew that this was possible, but if you just told a layman and say, "Hey, could you know, an AI agent find the canonical cause of cystic fibrosis inside a file with billions of variants?" They might say yes, they might say no, right? You just need to know. You need to kind of try it to get a sense of if it's possible. Quick thought experiment for you. Is there anything in this video you should be trying on your own? If there is, try it.

29:42 Take a screenshot, post it on LinkedIn X, and tag me. I'd love to see what you're learning. Now, a quick word from our sponsors before we get into the back half of the pod. If you've worked at any company bigger than 30 people, you know this one. The CEO sets strategy. By the time it reaches the people actually doing the work, it goes through three or four layers of translation. Half of it gets lost and nobody finds out until the quarter is over. That's the problem AISO is built for. It's an AI operating partner for every manager and team.

30:08 Connects to where work actually happens. the meetings, the messages, the docs, and it turns all that fragmented activity into a clear picture of execution. Managers get real coaching grounded in their team's actual work, not generic advice. Teams stay aligned with strategy as it changes, not as it was last quarter. And leaders see where execution is drifting in weeks, not in the post-mortem. One shared memory for the whole org. Everyone finally working from the same picture. If you lead a team, check out ariso.ai/ashos.

30:37 That's a riso. / a a kh I want to take a second to talk to you about the fourth cohort of LAN PM job. I trained 30 students in cohort 1, 50 students in cohort 2 and 75 students in cohort 3 and I am bringing back the program for cohort 4. It starts in August and it lasts 3 months where you're going to have intense sessions a Monday morning session where I go over your resume, behavioral interviews, LinkedIn. On top of that, Bart Choworki is going to be teaching you the PM fundamentals in 2026. how to write AI PRDS, how to AI prototype with cloud code, all of the key skills you need to freshen up your knowledge for this market. And Ankut Romani is going to be teaching you AI product management. He is an AI product manager at Uber and he is going to teach you how to build AI features that actually work successfully. On top of that, Prasad Ready is going to be doing one-on- ones with you for mock reviews, LinkedIn review, candidate market fit review. So, it is a full package. It is three courses in one for one low fee. So join at landpob.com.

31:40 Today's podcast is brought to you by Pendo, the leading software experience management platform. McKenzie found that 78% of companies are using Genai, but just as many have reported no bottom line improvements. So how do you know if your AI agents are actually working? Are they giving users the wrong answers, creating more work instead of less, improving retention, or hurting it? When your software data and AI data are disconnected, you can't answer these questions. But when you bring all your usage data together in one place, you can see what users do before, during, and after they use AI, showing you when agents work, how they help you grow, and when to prioritize on your roadmap.

32:15 Pendo Agent Analytics is the only solution built to do this for product teams. Start measuring your AI's performance with agent analytics at pendo.io/acos. That's pendo.io aka. But then if you know that the easy thing works and you know we've already have early evidence the easy things works you have to like establish the ceiling is like what is is is the hard thing working cuz like if it's just totally saturated then like what's the point of even having eval this task is already solved. so I'm I'm going to show off something that's pretty hard to do today. And what we have here is a recent paper. So this is from last year and and I will make this bigger. The authors here are deciphering the diagenic architecture of congenital heart disease. So what does this mean? This means congenital heart disease is if you're a baby and you're born with problems with your heart and diagenic means it involves two genes. So single gene, single variant genetic diseases are actually sometimes a solved problem.

33:27 Like with cystic fibrosis, not all cases, but many cases like this one are totally understand. Diagenic genetic diseases are like a very very new thing that people are studying. So this is like a very hard task to do and even though this is published and in theory an agent should be able to search the internet and find every publication and you know deeply understand all of this it's actually not that simple. They're not perfect and they actually need a lot of guidance which is why there are a lot of these companies including my own that are called like harness engineering companies or vertical AI companies or agentic AI companies because you need some specialized capability to be able to have the LLM do stuff like this. So let's briefly return to our our agents and let's just make sure they got it.

34:17 Okay. So, codeex with GPT 5.3 says, okay, this, if you recall, this is the deletion that we added. Boom. I gave it the phenotype. I gave it the genome. This is correct. So, how would we mark this correct? We would actually probably have another LLM. I'm not going to do this right now for the sake of time. Just compare my scorecard. This is the correct answer with the response the model is giving right here. And then we can check Haiku. And even Haiku. Oh, wait. Did Haiku not get this? Okay. So, so this is interesting. This is actually harder than I would have thought. I would have expected Haiku to get this because this problem is so easy.

35:00 >> But you'll notice what Haiku says is there's 48 variants spanning the gene. So, it's looking at the gene, but it fails to actually find the particular Oh, this is so interesting. It also hallucinates a hemisy large deletion. So coming back to this, coming back to our point is start with something easy. I thought I started with something easy here. It's a good thing I did this because if I were benchmarking highQ, this is too hard and I'd have to make it even easier. And the things I could do to make it even easier would be potentially you know, limit the region of the genome of interest, give it more hints, maybe provide access to more external information more easily.

35:38 But we can see that Haiku even fails this easy task. And I would be absolutely shocked if 5.5 did. Okay, it it it's not finished yet, but you can already see that it it found the correct answer. So if we were to score this, we could easily have an LLM say, okay, haiku, this is not correct. This does not match what I have in this table. This is correct and this is correct. So that's kind of and then in our in our spreadsheet we would just say you know haiku bad others good >> and but now let's move on to something where we want to understand the hard cases and again I unexpectedly actually picked out a hard case for haik coup but this paper is quite challenging and I believe it is unlikely that any of the models will solve this. So in the supplementary information of this paper is a table and it is a list of patients and a proband is a medical term for the patient you're evaluating and it has diagenic causes for congenal heart disease. So this particular patient has ACACB I have no idea what this is some gene it's het meaning it only has one copy of this variant and myio CD which is also hat which is one carpy and these researchers discovered that the combination of these two diseases leads to congenital heart disease. So let's see if the models can figure this out.

37:11 So what we'll do again is we'll go to our table and our phenotype. >> Two diseases or is it two abnormalities in their DNA? >> Yeah, that's a great question. It's one disease. It's a congenital heart disease and I don't know exactly which one it is from this paper. and you know we could read the paper and figure out exactly what the phenotype is, but it's some defect with the heart. And what's unusual about this and why this is hard is it's two hetererozygous variants on two different genes that is causing this single disease. So it's complicated. So our phenotype here is congenal heart disease and our gene here we have two of them. One is this guy and oops and the second one is this guy. And then the varants are these guys. And this is you'll notice the notation is a little bit different, but this is something that you'll just have to deal with in these evals is like, you know, no matter what you're doing because these tend to be in very technical specialized domains at this point. You know, no one wants eval for for boring stuff like ice cream flavors.

38:16 You you just have to get comfortable with all this different mutations. And now let's go and let's try this again. So what I would do is I would say something like please add these to dan deep variant VCF. But I'm actually not going to do this because you already saw how this worked and basically how the VCF file was structured. It would add these two rows and for the sake of time I've already done it. But then let's go and let's check and see. Oh, how do we do on this use case? So in this case, I've already pre-baked it and I'm going to say this is Dan CHD and I'll say this contains the genome of a patient with congenal heart disease.

39:12 Please identify the genetic cause. Okay. And while we're going to have this one running, we're going to try our other two agents just to see how they do. we can almost guarantee that Haiku will not get this because it didn't get the much much easier task. But for the sake of completion completeness, we will do this as well.

39:44 And and then we'll do this with GT5.5 extra high as well. And again, we would do >> probably like some non-deterministic nature, right? Like do you need to like test the same model a couple times to just see if like maybe two out of three times it gets it right or is that not important? >> Yeah, that's a really good point. so this is a question about sampling. so sampling is actually really important. And you might remember like all of this old research where you would basically sample for good traces.

40:18 And this is kind of what like RL environments do is you do a roll out, you do a roll out, you do a roll out, and then you get the correct answer and boom, you give it a good reward for that. And the key thesis here is inside the weights of the model, the right answer might live there. it just might not get the right answer each time. So when you're doing these evals, you do want to try multiple times. In this particular case, I actually know from having done it that sampling has very little effect and it's essentially deterministic based on model capabilities. I've seen slightly the same model get to the same conclusion with slightly different approaches. But in general, sampling in my experience is less important than it used to be. where sampling used to be a big deal. like if you look at let's just look at this is a funny story. let's look at Gemini Ultra scorecard.

41:14 So if you'll remember Gemini Ultra Oh wow. Did Google actually bury it? Okay, here it is. This is So you'll remember way back in 2023 when people thought Google was kind of out of the AI race. I actually worked on this model so I know the story very well. Google released Gemini Ultra which was I believe it was a 660b dense model which was crazy back then. That was like one of the largest dense models ever trained and they released this scorecard and what you see is this. This was very controversial and this comes back to your question about sampling.

41:54 If you remember MMLU, this used to be like the canonical benchmark for LLMs. And let's just go back to look at what a question is to remind you is just simple question answer. If you know the actual brightness of an object, its apparent brightness from location with no information, you can estimate this. Okay, models used to be bad at this, which is hilarious because this seems so distant right now. And Google wanted to be the best and GPT4 was the best at this point. Got 86.4% 4% on MLMU and it was on five shots meaning it had five samples and they picked the best one and that's why you see five shot three shot three shot 10 shot it really is like kind of like a way of cheating is like how many times can you sample from this model and what you see is that Gemini Ultra actually had 32 shots so they got more shots on goal and actually now that I'm remembering this this actually might be pre-examples but if this is actually not the number examples in the context window and just the shots or or or the number of times sampled. It it it doesn't really matter for the sake of this argument, but the answers to MMLU might be inside the model weights, but it might just be not enriched enough in terms of the probabilities. So by sampling more times, you actually get a higher chance of getting the correct answer. So this used to be a really big thing back in the day. Today, I don't think this is a big thing. The labs don't really publish anymore. I don't think there's a lot known about this. In my personal experience, I've not found that running the same prompt through the model multiple times generates different answers. In fact, I don't know if I've ever seen that for this task, but it's a really really good thing to do and you should test for your use case. Little little side side conversation while we look at the answers. And we've got all these cooking. These are all cooking. And we can see that 5.3 Spark finished first. This is actually why we did it. And what you'll notice is it's totally wrong. They found multiple variants. TBX1, my H cyst, JAG1. You'll notice these aren't even genes of interest for us. These variants aren't even relevant. No high confidence variants. so what we've done here is we've done the second step in our process is we've established the floor.

44:10 We found something easy, that cystic fibrosis gene. Now we found the ceiling is this model didn't get it. And plot twist, no model on the planet gets this without like a very very strong harness. Again, I'm working on that very strong harness. So, you know, we can make systems get this, but this is hard. And then you just kind of go through and it's almost like a binary search process where you say easy, medium, hard, and then just assemble a list of prompts.

44:36 You know, you might have a hundred of these, which again would be phenotype gene, phenotype gene, phenotype gene, and then you understand how the models do on it. And then you're done. And then you have your eval. And then you understand what is good enough to actually ship product. So if you're scoring 50%, is that good enough to ship your product? Probably not. So then you look at which phenotypes am I better at? Maybe you put guard rails on the product to make sure that it only will answer the types of questions that it can get 80% on or something like that. That's a product decision. That's a product manager's decision to do that. And then you also hand all the hard ones to the research team and you say, "Hey guys, you didn't get this. Fix the model or fix the hardness and make sure that it can get these in the future so I can ship a product with these capabilities."

45:24 And not shockingly, Haiku is totally off base. clearly HiQ is not good at all at this task. so that's surprising. It missed the first one. Not surprising it missed this one. and we again we have Oh my god. Okay, this is so interesting. Okay, so this is actually a good example of something that came out and sampled a second time and worked because I actually tried this. We were just talking about sampling. But we can see right now that GPT 5.5 extra high this time actually did identify this diagenic pairs and it did almost certainly find the paper.

46:05 Yeah. So it did find the paper with all these diagenic pairs. So this is actually a very interesting reasoning trace where it was able to turn this congenital heart defect phenotype into a search for a very specific paper and then pull out the results from this paper. So actually this is quite impressive from GPT 5.5. But this is this is correct. So in this case, I would have to find an even harder one if I were benchmarking this model in particular. But this is basically the the key set of steps. And I don't think we have time to do a bunch more, but it's basically running through all the different types of scenarios and then coming up with prompts that will challenge the model but not totally stump the model. So yeah, with that, that's that's how you write an agentic eval. And here is two lines in our new one.

47:00 >> Wow. So, it's just a spreadsheet. And the key thing here is the domain subject matter expertise. It's not like how it's written or anything like that. You're not giving us an EVEL template like we might have given people a PRD template before. It's really the subject matter expertise that's driving all of this. >> Yeah. Exactly. There are a lot of companies over the last, you know, n years who have tried to build better tools for evals. And I'm not saying that tools for evals don't need to exist.

47:31 There's plenty of ways to improve, but when you're creating evals like this, it is literally just prompts, responses, and ways of scoring whether response is correct. >> Fascinating. So just to bring it all back full circle, if you're a PM who's never worked on an AI feature before, when would you be going through this eval process and when wouldn't you and how would you be using it? >> Yeah, that's a a good question. So if you've never worked on an AI feature before, I would actually try to find somebody who has who can help you through this. This is deceptively simple. I made this really simple because we had 45 minutes today, but this and I don't know why it's so complex honestly. I had many many conversations with people about how to build evals but there's just something kind of like taste based or or nuanced about how to build them and you know it is what it is but fi find somebody who can help you but I would say you start from the beginning is like you are building an AI feature like I don't know I'm just making it up you work at Pinterest and you want to have better image generation that doesn't look like slop that actually pleases users so it's like an image generation feature you need to think about from day zero. What does success look like? What is unique about those Pinterest users? What do they want to see? And you need to translate that. You can't just write down a PRD and say they want beautiful kitchens. You have to explicitly define what a beautiful kitchen is and not in words in examples and a way to score those examples. And I am not in the image generation space, so I don't exactly know what that looks like. But there are many many many examples like this where you need to understand what the user wants and translate translate that into prompts and responses and way to score those responses. And that is the first thing you should do when you're starting to build a new AI feature.

49:20 >> Okay. So just like you have ramped up your expertise in the genomic space if you were tackling that problem you'd go learn, you'd go talk to people who have built image and eval oh this is how I build an LLM judge that generates images of this type. And then that would really be the basis for your eval. >> Yep, that's correct. >> Okay. Wow, there's so much so many layers. I've done like five or six episodes on eval, but I think this was one of the most tactical that really helped me understand how things change and I think that's a function of your experience, which I wanted to talk about for a little bit. So, your eval piece crossed my radar. I think another really interesting piece you wrote about was product management at Meta versus Google. You've worked on Gemini, you've worked on Llama, you've seen both of these cultures. What really is the difference between product management at Meta versus Google?

50:15 >> Yeah. So, I would caveat that and say I wrote this like two and a half years ago and I was at Google three and a half years ago, I think. So, a lot has changed. When I was at Google, Google was a dead company. I think the stock fell to like $80 and I think it's you know 300 or 400 right now and they've really changed how they think about things. I'm a boomerang at that so I think I've spent seven years there in in in total and you know I saw everything from Cambridge Analytica lows to highs of like Llama 3 really wowing people to lows of Llama 4 disappointing. So I I saw a a very large spectrum and what I would say like my key takeaways for what Google versus Meta was like is Meta is like a much much more aggressive culture in many ways. I think that it comes from like the founder leadership of Mark Zuckerberg is he is the last man standing. Well, I guess besides Elon, but he is the last man standing who's got like, you know, the founder leading a fan company who has utter and absolute control who's going to do what he wants.

51:20 And sometimes it's really empowering because he says, "This is super important to me. You have all the resources in the world and you should go do it." And sometimes it's like not what you want. For example, I worked on Llama before. Llama had problems. I think the main problem was actually how it was evaluated. Huh, funny. Those emails are important. If you want to read about that story, Google it. I had nothing to do with that and I I loved working on Llama before and Mark just said, "You guys all suck. You need to go find new jobs." So basically the whole Llama team is gone because of you know Mark's his decisions. So I think that you know that that cuts both ways. I'd say my overall preference is for like a very like high conviction founder company. I actually have like incredible respect for Mark. I've only you know met him a couple times and every time has been just like wow this is like a really smart guy but it creates a lot of problems too right because you know Google is much more consensus driven I think Google has much weaker product management function at least it did when I was there so you know Google is considered to be more of an engineeringled company whereas meta is more of a productled company at least that historically has been the case and it was a really great experience working at both places I think I've learned a lot for both places. I left both places with a lot of friends and yeah, if you want to read like kind of this blog post actually went pretty viral. If you want to read like kind of a interesting snapshot of what it was like in say 2023 between both of the places, you know, give it a read.

52:48 >> Highly recommend it to everybody. As you guys can see, I'm itching to ask many more questions. So Daniel, we're going to need to have you back. Before you go, tell us a little bit about your startup. >> Oh, cool. Yeah, so I've started a company called Gamoff Labs. And as I hinted at during these evaluations, the core problem I want to solve is to make it much much easier to get whole genome sequencing into every single NICU in the entire world. This is the absolute gold standard of helping sick babies.

53:17 there's overwhelming clinical and economic evidence that it's effective, but the problem is it's just too damn hard and expensive. So where you see this used is in places like Stanford, Boston Children's, CHOP and these are the absolute top facilities in the world and I want to see them in, you know, rural Arkansas, rural India, you know, rural China, all the places where this this kind of life-saving technology is not being harnessed. And my key thesis is that a lot of the human work involved in interpreting these genomes could be augmented by AI. And we've already shown that using the system that we've built, we can identify variants that have never been discovered before.

54:01 We've actually allowed one family to have a child and they they couldn't before because they didn't know. Again, this is small scale. We started this five weeks ago. So, you know, but one person is is really crazy to have that impact on their life. And yeah, like it's a deeply deeply missiondriven thing. I think it's very very interesting technically because it's all about building the best agentic harnesses. It's all about understanding how AI can help with biology. And if you're interested in joining me on this journey, we are hiring right now.

54:31 we're very small team. ra raised our preede round and are basically planning on building like the operating system for rare disease and genomic medicine. And I couldn't be more excited to wake up to work on this every morning. And I would love I would love it if you would reach out if you're interested. >> Wow. So a lot of you guys I know at least in my audience you want to become that AIBM at meta Google this is often the next step after that. So if you know we always say the grass is greener at some point this is where I've seen those AIPMs at Meta and Google go just like Daniel into starting their own companies and that's actually the cool thing is it helps prepare you for that. You can see how his own evals and deep AI knowledge has now applied to his startup. Daniel, thank you so so much for lending your expertise today.

55:19 >> Yeah, and I want to leave with just one parting thought is the metas and the Googles and all the other large companies have to reinvent themselves right now in the age of AI. If you're inside these companies, it is very very interesting to see how this classic consumer software building factory has changed. But if you come and you do a startup or you start your own thing, you get to build the future from scratch. And sometimes that's actually easier.

55:46 >> We'll leave it there. See you all in the next episode. I hope you learned as much from today's episode as I did. If you can do one thing that's totally free that would help the show, it would be to check that you're following on Apple and Spotify podcasts. Check that you've left ratings and reviews on those platforms. Check that you're subscribed on YouTube. Leave a like and a comment on this video. And then share it with your friends. We're trying to make better and better podcasts. After 2 years, we think we've gotten something pretty good going. So, let us know what we can do to make it even better, who else we should interview, and we will put on the best shows we possibly can. Finally, don't forget my offer for the bundle. You get an entire year of my paid newsletter, plus my favorite AI tools, Bolt, new, Air Table, Speechify, Descript, Magic Patterns, Linear, Dovetail, Arise, and Mobin.

56:36 That's $27,000 worth of value for just $150. So check that out at bundle.ashg.com if it interests you. And I can't wait to share our next episode soon.