Once our documentation lived as plain-text AsciiDoc in a Git repository, the obvious next question came up: could we point an AI chatbot at it? The honest answer was "not yet" — having machine-readable source is necessary but nowhere near sufficient.
Structure matters more than tooling
It's tempting to think the hard part of a RAG (retrieval-augmented generation) setup is the model or the vector database. In practice, the hard part is the documentation itself. A retrieval system can only return what was written as a coherent, self-contained chunk. Topics that relied on "as explained above" or buried the actual answer three paragraphs into unrelated context retrieved badly, no matter how good the embedding model was.
Chunking is a documentation problem
This is the part that surprised me most: improving retrieval quality looked a lot more like editing than engineering. The fixes that mattered were:
- Splitting long topics into smaller, single-purpose sections with their own headings
- Making each section answer one question instead of several
- Replacing implicit references ("see the previous section") with explicit ones
- Front-loading the answer instead of building up to it
None of that required new tooling. It required rewriting topics the way you'd want them to read if a reader landed on that one section alone — which, with a chatbot, is exactly what happens.
Where AI assistants actually help
For the writing side, AI writing assistants turned out to be most useful for the unglamorous, high-volume tasks: drafting first-pass release notes from changelogs, flagging inconsistent terminology across hundreds of topics, and suggesting simplifications for sentences that had clearly been edited by five different people over five years. They were far less useful for anything that required understanding the product itself — that part still has to come from the writer.
Where they don't
An AI assistant will happily generate a plausible-sounding procedure for a feature it doesn't actually understand. That's fine for a first draft a human will verify, and dangerous for anything that ships unreviewed. The lesson generalizes: AI tooling raised the floor on documentation quality and lowered the time spent on rote tasks, but it didn't change who is accountable for whether the documentation is actually correct.