Section Insights
The Burden of Code Quality on AI
How does code quality affect AI coding output?
The quality and organization of code significantly impact AI's ability to interpret and utilize it effectively. Well-formatted and structured code allows AI to explore and understand the codebase better.
- AI relies heavily on the code itself for understanding.
- Code should be self-explanatory and well-organized.
- Poorly structured code can lead to inefficient AI performance.
The Pitfalls of Relying on Documentation
What are the issues with using documentation as the primary source for AI?
Many developers create documentation that AI is expected to rely on, but these documents are often not executable or testable, leading to potential discrepancies between the documentation and the actual code.
- Documentation can drift away from the actual code.
- Conflicting sources of truth can confuse AI.
- Relying solely on documentation is an antipattern.
The Dangers of Conflicting Sources of Truth
Why is it problematic to have conflicting documentation and code?
When documentation and code conflict, AI cannot determine which is the accurate source of truth, leading to confusion and inefficiency in coding output.
- Conflicting information can hinder AI's effectiveness.
- Maintaining a single source of truth is crucial.
- Documentation should complement, not replace, code.
The Role of Documentation in Codebases
What is the appropriate role of documentation in a codebase?
While code should be self-explanatory, certain aspects like architectural decisions and domain-specific language require documentation to provide context that code alone cannot convey.
- Documentation should capture architectural decisions.
- Glossaries can clarify domain-specific terms.
- A thin layer of documentation can aid AI exploration.
Effective Documentation Practices
What types of documentation are beneficial for AI and developers?
A minimal amount of documentation, such as architectural decision records and glossaries, can enhance AI's understanding and navigation of the codebase without overshadowing the code itself.
- A glossary aids in understanding domain language.
- Architectural decision records provide context.
- Documentation should facilitate, not dominate, code comprehension.
Transcript
0:00 There's this really annoying pattern that I see people do everywhere that I think is making their AI coding output worse, not better, and costing them tokens. Now, most of the time when I let AI explore my codebase, the only thing AI has to go on is the code itself. And it the code is supposed to be sort of self-explanatory. It's supposed to be have enough information in the code so that AI can interpret it. That puts a lot of burden on my code, so I need to make sure my code is well-formatted, that it's in the right file systems, that it's organized into reasonable chunks so the AI can look at it, that there's interfaces and implementations and they're separated so AI can just take what it needs. However, a lot of folks are not doing this. They are taking a layer of docs, so they're creating docs for their code and getting the AI to primarily rely on those docs.
0:46 The issue I have with this is those docs, like markdown files inside the repository, they're usually not executable. They're not testable against the code. And so they can drift away from the code. The docs can say one thing, but the actual code, the source of truth, can say another. And this is really bad because if there are two sources of truth and they conflict, then AI won't know which one is the real one. And so I consider this a really, really big problem and a massive antipattern to build this whole layer of docs just to explain what the code can anyway. Now, some people will read this and think that I'm against all docs, that the only thing you should have in your repo is code. No, that's not true. Code should be self-explanatory, but there are some things that code can't explain. code can't tell you what alternatives were considered instead of the code that's there. And so you need architectural decision records in your repo to capture those. It also can't really explain the domain language that you're using to work with your code. Does it know what an order is in your codebase? Probably not. And so a glossary is extremely helpful. And finally, a thin layer of docs just to navigate around the main aspects of your code is really helpful just to speed up AI's exploration of your code. But the concept of docs as the source of truth, as the thing that you read to understand the code, no, I don't like that at all.
Summary
- Over-reliance on documentation can lead to discrepancies between docs and actual code.
- Code should be self-explanatory and well-organized to facilitate AI understanding.
- Documentation can drift away from the code, creating conflicts that confuse AI.
- Architectural decision records are necessary to explain design choices not evident in the code.
- A glossary can clarify domain-specific language used within the codebase.
- Minimal documentation can aid in navigating the code but should not overshadow the code itself.
- The speaker advocates for a balanced approach to documentation, ensuring it complements rather than replaces the code.
Questions Answered
How does code quality affect AI coding output?
The quality and organization of code significantly impact AI's ability to interpret and utilize it effectively. Well-formatted and structured code allows AI to explore and understand the codebase better.
What are the issues with using documentation as the primary source for AI?
Many developers create documentation that AI is expected to rely on, but these documents are often not executable or testable, leading to potential discrepancies between the documentation and the actual code.
Why is it problematic to have conflicting documentation and code?
When documentation and code conflict, AI cannot determine which is the accurate source of truth, leading to confusion and inefficiency in coding output.
What is the appropriate role of documentation in a codebase?
While code should be self-explanatory, certain aspects like architectural decisions and domain-specific language require documentation to provide context that code alone cannot convey.
What types of documentation are beneficial for AI and developers?
A minimal amount of documentation, such as architectural decision records and glossaries, can enhance AI's understanding and navigation of the codebase without overshadowing the code itself.