When AI agents started reading your docs
My first dev docs were pages built for a browser and a human. I spent years arguing that docs-as-code meant treating the markdown with the same care as the code it describes. Then, in 2024, a reader showed up that nobody had designed for. It is the model your customers type into, and it reads your docs before most of your users ever do.
This is a throwback in the truest sense. I have been on the other side of the doc-consumption problem for years, first maintaining SDKs across languages (the maintenance-retrospective territory I wrote up in Five SDKs, one streaming API) and now at Speechify. The shift from human-first to model-first documentation is the biggest change to developer experience in that whole window. Here is what I used to believe, what actually happened, and what survives.
The way I sold docs-as-code, circa 2020
The pitch was simple and it was right for its time. You wrote markdown next to the code, stored it in the same repo, and reviewed it in pull requests like any other change. It built through CI and deployed like software. The docs could not drift because they shipped with the code that made them true.
By 2020 that approach was standard for open source and for fast-moving API companies. Tooling like Docusaurus, MkDocs, and Hugo turned markdown into a site your developers could actually search. I ran my own workflow that way for years, because it worked. Markdown, Git, a static site generator, and a search index were enough to put answers in front of a developer.
That model has a name now. People call it docs-as-code 1.0, and a good summary of the philosophy sits in the docs as code field guide. The core insight was that documentation has the same failure modes as software, so it gets the same solutions: version control for drift, review for accuracy, and CI for broken links. All correct then, and still correct now.
The one thing nobody framed as a target audience back then was a language model.
When the first reader stopped being human
The change landed quietly. In September 2024 Jeremy Howard proposed /llms.txt as a way to give language models a concise, structured version of a site, because LLMs read differently than browsers do. They benefit from expert-level information in one parse and they cannot ingest an entire HTML page with navigation and ads. The original proposal framed it as helping models use sites at inference time, the same context a coding assistant pulls before it answers.
Adoption followed fast for a community proposal. Mintlify added native llms.txt support in November 2024, making the file the default for thousands of docs sites overnight. Soon after, Stripe, Anthropic, Cursor, and the Python instructor library shipped one too. By mid-2026 the ecosystem reports adoption across companies like Anthropic, Cloudflare, Vercel, Supabase, and OpenAI. It is not a formal standard. It is a convention that is already everywhere, which for developer tools is usually more useful than a standard ratification.
The MCP side came at the same problem from the tooling direction. Whereas llms.txt gives a model a map of your content, the Model Context Protocol gives it a way to query it as a live source of truth. Fern’s writeup on MCP servers for documentation sites makes the case plainly: without a connection to your current docs, a coding assistant fills in from whatever training data it has, which means it answers with your deprecated endpoint. With one, it asks your docs directly and gets the current answer at request time.
The measurement shifted with it. The Developer Relations strategy work that came out this year points at the same number from a different direction. Stack Overflow’s survey puts daily or weekly use of AI coding tools among professional developers at a majority, and the argument in Developer Relations for the AI era is that the LLM is now your first-touch user. It consumes your docs and your API before a person ever does.
Staff engineers are not LLMs
Reading the landscape, it was tempting to treat setup and discovery as free. This is where I got specific fast as an SDK maintainer, because the interactions shaped my view of the API more than any internal review did.
The model makes answers look confident
A human who is unsure hedges. A model that lacks a schema or an auth detail often inverts the whole shape and then continues. The difference is that the quiet-reader failure is invisible until someone files a support issue about the code their assistant generated. Tracing that back to a doc gap is real work. This is the same pattern I keep hitting when an integration breaks in a way that only shows up on a real stream, and I wrote the failure mode up in The invisible work of voice AI SDKs.
Consistency is a compliance requirement
When a human reads docs they build a mental model and forgive small inconsistencies. A model builds its call strictly from what is on the page. If an example shows token in one place and bearer in another, the generated code will mix them. Docs in the model era cannot have a single drift. The writing has to be monotonic from the first example to the reference.
The error page is now the last page the model reads
A human who hits an error opens the docs and checks the response codes. A model gets a failed call and immediately reaches for your error documentation, then extrapolates. If your error page is thin your model guesses why. Error payloads and their docs are a real part of the developer experience, and I have a whole post counting the ways these decisions show up for voice APIs in 5 API design decisions that shape voice AI dev experience.
Docs-as-code 1.0 vs 2.0
The best way to show the change is in the same row as the old version.
| Docs-as-code 1.0 | Docs-as-code 2.0 | |
|---|---|---|
| Primary reader | Human developer | LLM first, human second |
| Main format | Markdown to static HTML | Markdown plus llms.txt, structured data, MCP |
| Discovery | Keyword search | RAG, semantic, and MCP query |
| Faithfulness | Reviewer catches it in a PR | The model is monotonic or wrong |
| Versioning | Docs match the tagged release | Docs must match whatever the model can fetch now |
| Update loop | CI on push | CI plus agents that re-prompt on failures |
The reliable one-paragraph version. Version 1.0 fixed the problem of documentation that had to be read by a person. Version 2.0 keeps every one of those habits and adds the requirement that a model can parse, navigate, and trust it in one pass. Nothing from 1.0 is wasted. Markdown, Git, review, and CI all survived, and the better analysis of the migration is in the docs-as-code 2.0 thread from May that breaks down where the two versions genuinely differ.
It is still early, and the ground rules are contested. llms.txt adoption is real but its visibility signal has limits. Note that some practitioners in 2026 declared it “dead” as a ranking measure, while the practical agent-navigation use kept growing. The honest framing is to keep the interactive docs for the assistant that queries you and use llms.txt as the map for the ones that just want a quick load of the important pages.
What I still believe, because the reader changed
The tone flipped for me when I stopped forcing a distinction between “docs for humans” and “docs for models”. A model reads the same markdown; it just reads it all at once and takes it literally. So the practices that age well are the ones that never depended on a human remembering the good line.
Plain, concrete language, one idea per paragraph, examples that are copy-paste runnable, no prose that rewards a skim. Those qualities serve both a tired developer at 11pm and a model assembling a call at 10x reading speed. The overblown bits die either way. Docs that assume tribal knowledge, that never show the error path, or that let the example fall out of date with the schema? Those were always going to fail. The model just made the failure faster and louder.
The result is that developer documentation is no longer the artifact of one discipline. It is the shared contract between your API, the people who call it, and the tools those people use to call it for them. That is why API design, docs, and DevRel now sit on the same decision table, and why the SDK work I spent years on only matters if a model can find it.
Want a practical start? Ship an llms.txt, keep a monotonic running example in your quickstart, and make your error page legible. Then check the other end of the pipeline. The MCP side of this story keeps moving fast, and I covered the stateful-to-stateless rewrite of the protocol separately, so if you maintain a server this is the running spec to watch: MCP Goes Stateless on Monday.
FAQ
What is llms.txt and why does it matter?
llms.txt is a markdown file at your site root that tells a language model what the site is and where the important pages are. Jeremy Howard proposed it in September 2024 so models could use sites at inference time without parsing navigation-heavy HTML. It matters because coding assistants read this file before answering, and if you do not ship one the model answers from training data instead of your current docs. See the original proposal and the spec site.
Does llms.txt replace my docs or my sitemap?
No. robots.txt controls access, sitemap.xml lists URLs for search crawlers, and llms.txt curates a concise, model-friendly summary of what matters. They do different jobs and you keep all three. The documentation site itself stays the source of truth; llms.txt is the map pointing a model at it. The site-test audit explainer covers the differences in one place.
What is the difference between llms.txt and MCP for docs?
llms.txt hands a model a static map of your content. MCP gives a tool a way to query your docs, API schema, and repo as live sources of truth at request time, so answers reflect your current release rather than stale training data. Fern’s docs and MCP writeup is the cleanest comparison of the two approaches.
Is docs-as-code still worth doing in the AI era?
Yes, and it matters more. Version control, review, and CI are what keep a model from spitting out your deprecated endpoint, because the docs ship with the code that makes them true. What changes is the audience: the doc must be parseable and trustable by a model in one pass, not just navigable by a human. That is the whole docs-as-code 2.0 shift covered in the field comparison.
How do I know if my docs work for AI agents?
Watch the support queue. When an integration breaks and the answer traces back to a doc gap, that is a model hit and it will keep recurring. Then look for monotonic examples, a runnable quickstart, and an error page that covers the failure path. If any of those are thin, the model will guess and the humans will pay. I wrote the maintenance-side of this exact loop in Five SDKs, one streaming API.