Prompting Is Over! How to Use Karpathy's AI Method (LLM Wiki + AutoResearch)

0:00 Over the past few months, two of Karpathy's projects have been everywhere. One creates a personal knowledge base. The other conducts research on its own. But very few people have noticed that both are built on the exact same way of thinking. To understand why Carpathy's projects are so useful, you first need to understand how he thinks about working with AI. If you simply copy Karpathy's repositories, you'll end up with two tools. But if you understand the principles they're built on, you'll have a framework for building almost any AI workflow. That framework has three parts: spec, verifier, and knowledge base. Let's start with the first piece, the spec. The word sounds technical, but the idea is genuinely simple. A spec is just a precise written description of what you want the model to do before you ask it to do anything. Think of it like a recipe you write before you start cooking, not a vague intention.

0:57 Something like, "I'll make pasta tonight." A real recipe lists the ingredients and measures the quantities. It puts the steps in order and spells out when the dish is done. When the recipe is sharp, almost anyone in your kitchen can cook the dish, and it comes out the same. When the recipe is vague, every cook produces a different plate and most of them are disappointing. That's exactly what happens with language models. If your prompt is just summarize the latest research on transformers, you've handed the model a vague intention and the output will reflect that vagueness. If your prompt names the inputs, the audience, and the format, you've handed the model a recipe. Add the length, the citation style, and the things to leave out, and that recipe gets even sharper. The output gets dramatically more consistent, and the model has way less room to drift. Karpathy talks about this a lot in his post on what he calls software 2.0 and vibe coding. The pattern he keeps pointing at is that writing the spec is the actual work. The model is just the cook and you're the one writing the recipe. The quality of your recipe sets the ceiling on the quality of the dish. This sounds obvious when I say it out loud, but watch how most people use AI in practice. They type a oneline request, they get a mediocre answer, and then they blame the model. The model isn't the bottleneck in that scenario. The recipe is the discipline Carpathy is pushing is to slow down at the beginning and write the spec like you actually mean it. Name what good output looks like and name what bad output looks like, too. Then name the format and the inputs the model is allowed to use. Once you've done that work, the prompt almost writes itself and the model performs noticeably better. One more thing about the spec that I think people miss. A good spec is also a contract you can review later.

2:52 When the output is wrong, you can go back to the spec and ask which part of it the model failed. If the spec was vague, you can't even have that conversation because there was nothing concrete to fail against. The spec is what makes the rest of the system possible. Quick break. I was sketching an alternate thumbnail for this video inside the new Deposit Photos AI assistant. Sports thumbnails usually need three tools. A stock library, a generator, and an editor just to ship one cover. Inside the AI assistant, I just describe what I want in plain English, and it either pulls from 331 million licensed assets or generates a fresh image. Right in the same chat, I typed cinematic stadium scoreboard at night. Three team crests glowing packed crowd blurred behind dramatic flood lights. Nano banana rendered it and the lighting came out genuinely cinema grade. Then I asked it to warm the grade and tilt the scoreboard toward camera and it edited that same image without starting over. Here's where the editing suite earns its place. I dropped in the GBT, Claude, and Gemini logos, and background remover stripped each one to a transparent layer, so they sit on the board like real team crests. Then I spotted a sponsor banner on the stadium boards I didn't have rights to, and the AI editor wiped it out in one pass. For paid ad campaigns, you can flip the engine over to Bria, which is commercially safe with full indeentification. Everything sits under their standard license, so it's cleared for YouTube, paid social, and client decks. Use this code to get 20% off paid plans. The link sits in the description below. Here's what breaks without it. A sharp spec tells you what good output should look like, but nothing yet checks whether the model actually hit that bar.

4:41 That gap is where the second piece comes in, the verifier. And this is where most AI workflows quietly fall apart. A verifier is just an automatic check on the output. something that looks at what the model produced and decides before any human sees it, whether the result is acceptable or whether the model needs to try again. Back to the kitchen analogy, the spec was the recipe. The verifier is tasting the dish before you serve it. A good cook doesn't just plate whatever came out of the pan. They taste it, they check the seasoning, they look at the texture, and only then do they decide it's ready. If it's not ready, they fix it or they cook another batch. That tasting step is what the verifier does in an AI workflow. It can be a piece of code that checks the format or another language model reviewing the first draft against the spec. It can be a unit test or a regax or it can be a schema validator with a numeric threshold or it can be a second pass that grades the first one on a rubric. The specific implementation depends on the task. The principle is always the same. You don't trust the first draft. You check it against the spec automatically and you loop until the check passes. Now look at what happens when you combine a sharp spec with an automatic verifier. You get a system that can run on its own. The spec tells the model what good looks like. The verifier checks whether the output meets that bar. If it doesn't, the system refineses and tries again without a human in the middle. That loop is exactly what auto research runs. When you read the auto research repo, the pitch isn't research in the writing sense. It's an agent that improves your training code while you sleep. You point it at a small model and a single GPU.

6:28 The spec is a markdown file where you describe what to try. The agent edits the training script. Then it runs a short training job. It checks one number. Did the loss go down? That single number is the verifier. There's no source checking or fact-checking involved. The check is a single number. If the change beats the previous best, the agent keeps it. If it doesn't, the agent rolls the code back. Then it tries a different change left running overnight. That loop can rack up over a 100 experiments before you wake up. This is exactly the spec principle built into a real production system. It's called AI master, the agentic workflow my team and I run our own channels on. Inside it, there's a content agent built specifically for this. You define your channel DNA once, your niche, your host persona, your voice guidelines, your compliance rules, and the system injects that spec into a set of specialized agents automatically. Hook Pilot handles your first 10 seconds. Script Writer takes your brief and voice notes and produces a full script with visual cues and voice routing. AdSmith handles sponsor integrations. AI Producer connects to your YouTube data and tells you what to make next. You brief the video, the agents write the script. The system generates the voice over and avatar footage, and you publish straight to YouTube. Inside A Master, you also get every major model in one window at a lower cost per token than running them separately, covering images, voice over, and video. Plus, the newly released Grock 4.5 and the full GPT 5.6 lineup, Saul, Terra, and Luna are already available here. Our subscribers get instant access. You can build a consistent character and monetize it directly on the platform. Over 12,000 people are already running their content through this. And the results show up in the numbers. Faster output, a consistent voice across hundreds of generations, and characters that get reused instead of thrown away after one video. Every purchase is covered by a 7-day money back guarantee, so trying it out cost you nothing. links in the description if you want to run the same setup on your channel. Once you see it that way, you also see why the same architecture works for things that have nothing. Model training, code generation, document drafting, data extraction, customer support replies, anything where you can write a sharp spec and build an automatic check is a candidate for the same loop. That's why I keep saying these projects feel inevitable once you understand the model. Auto research isn't a creative leap. It's the obvious thing you build once you've internalized spec plus verifier as a pattern. And there's a quieter point hiding in here that I want to name. The verifier is the part that lets you trust the system enough to step away from it. Without a verifier, every AI workflow needs a human babysitter checking each output.

9:31 With a verifier, the human moves up a level and only inspects the cases the verifier flag. That shift is what separates a clever demo from a workflow you actually leave running. A verifier alone still has a blind spot, though. It can tell you the output is good. It can't make the next output better than this one. That's the piece still missing from the picture. The third piece is the knowledge base. And this one is the most underrated of the three in my opinion. A knowledge base in Carpathy sense is a structured place where the lessons from past work accumulate. So the system gets smarter every time you use it. It's not a database in the boring sense.

10:13 It's the memory of the system. The analogy I like here is your personal cookbook. The recipe was the spec. The tasting was the verifier. The cookbook is the knowledge base. Every time you cook a dish, you learn something. The pan was too hot. The garlic went in too early. The cut was uneven. If you write those lessons back into the recipe, your next attempt is better. Over a year of cooking, your cookbook becomes something nobody else has because it's tuned to your kitchen. Your ingredients, your taste. That's what a knowledge base does for an AI workflow. Every time the system runs, it produces an output, a verifier result, and a set of lessons.

10:56 Those lessons get written back somewhere structured. so the next run can use them. Over time, the system stops being a generic model behind a prompt and starts being a model plus your accumulated context. The compounding is the whole point. That principle is exactly where LLM wiki comes from. LLM wiki is on the surface a wiki for language model knowledge, but seen through the lens we've been building, it's a knowledge base in the strict sense. It's a structured place where what you learn about a topic gets stored in a form the model can retrieve later.

11:31 The structure isn't decoration. The structure is what makes the retrieval reliable. That's the move I want you to notice. Most people when they want their AI workflow to remember things, dump everything into a giant text file or a vector database and hope for the best. Carpathy's version is more disciplined. The knowledge is organized. The entries have a shape. The lengths between them are explicit. You can predict exactly what the model will pull back instead of rolling the dice. Okay, so we have the three pieces spec, verifier, knowledge base. Now let's look at what this actually looks like in the real projects because once you see them through this lens, they read very differently. Let's start with LLM wiki.

12:15 It's the simpler of the two to show and it's the cleanest place to see knowledge base in action. When you open the project, it's organized like a wiki with entries, links, and categories. That's not a stylistic choice. That's the knowledgebased principle showing up in the interface. The shape of the data is the contract. Look at how an individual entry is structured. There's a clear top section that names what the entry is about. There's a body that explains it.

12:44 There are explicit links to related entries. None of this is decorative. Every piece of that structure is something the retrieval layer can latch on to when the model needs to pull this knowledge back in later. Now watch what happens when you query the wiki. The system doesn't dump the whole corpus into the model. It finds the entries that match, pulls just those, and hands them to the model as context. Because the entries were structured on the way in, the model's inputs on the way out stay predictable. This is the part I want you to internalize.

13:18 >> >> The work that makes LLM wiki useful happened before any query was ever run. Somebody decided what an entry looks like. Somebody decided how entries link. That upfront discipline is what makes everything downstream feel smooth. You can build the exact same thing for your own domain in a weekend. And you don't need this specific repo to do it. You need the principle. Now let's dig into auto research here. This one is the loop in action. When you open the repo, you see the three pieces we just talked about sitting right there in the design.

13:52 There's a spec at the top, a markdown file where you write out what directions the agent should try. There's the model in the middle editing training code and running experiments. There's a verifier checking each result against one metric, usually validation loss. And there's a keep or revert step that loops back after every run. Now watch a single training run. You point the agent at a small model and a single GPU. The spec tells it what to try, maybe a new optimizer or a different learning rate.

14:22 The agent edits the training code and starts a short training job, usually around 5 minutes. When the job finishes, the verifier checks one number. Did validation loss go down? If yes, the agent keeps the change and moves to the next idea. If no, it rolls the code back through git and tries something else. Left running overnight, that loop can rack up over a 100 experiments before you wake up. Carpathy's own two-day run hit roughly 700 experiments.

14:52 It kept about 20 genuine improvements. And if you have the knowledge base wired in, the system can also carry lessons from earlier runs into the next one. That's the compounding effect I mentioned earlier, and it's what turns a single overnight script into a system you keep running. Look at the two projects side by side. Now, Auto Research leans hardest on spec plus verifier with the knowledge base running quietly in the background. LLM Wiki leans hardest on the knowledge base with simple spec and verifier ideas baked into how entries are written. Same three parts just mixed in different ratios depending on the job. So, both projects run on the same three parts. What changes is which part carries the most weight. And that's why the method outlifts the repos themselves. Say you use a model to draft client emails. The spec is a short dock describing what a good email from you sounds like. The verifier is a second pass that grades the draft against that spec. The knowledge base is a folder of past emails labeled good or bad with notes on why. Same pattern applied to writing code. The spec is your acceptance criteria and style guide. The verifier is your test suite plus a enter. The knowledge base is your past pull request structured so the model can find relevant prior decisions. Once you see the pattern, you stop being impressed by individual tools. The tools are downstream of the model because the model is the thing that actually matters. Write the spec like you mean it. Build the verifier so you can step away. Grow the knowledge base so every run makes the next one better. Do those three things and your AI workflow stops being a clever toy and starts being something you can actually rely on.

16:37 That's the Carpathy method. See you in the next one.