0:00 I've taught over 4,000 people from companies like Google, Amazon, and Meta on how to test AI products. And here are the four most common mistakes I see. Number one, you're using off-the-shelf metrics like helpfulness score, coherence score, hallucination score. These are just someone else's generic prompt hidden behind five layers of abstraction. They measure abstract qualities that usually have nothing to do with your product. And good scores on these don't mean your system works. Number two, a really popular technique to test AI products is to use an LLM as a judge. But most people don't measure if the LLM as a judge agrees with a human. If it only matches human labels 60% of the time, it's useless. [music] So, you have to make sure you measure your judge. Number three, if you have a rack system, you need to separate retrieval from generation. If retrieval is broken, no amount of prompt engineering is going to fix the generation bit. And four, you need to audit your whole system end-to-end. I built skills that walk your coding agent through each of these mistakes and many more. They work with Claude code, Codex, or whatever coding agent you want to use. They come from a year of watching thousands of people make these exact mistakes and many more. I left a link to a blog post with the skills and how to use them in the comments.