content: How to Implement llms.txt for Developer Documentation - #830
content: How to Implement llms.txt for Developer Documentation#830frances720 wants to merge 1 commit into
Conversation
…Documentation Keyword: llms.txt for developer docs Generated by edu campaign skill Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014oMqk2tFowXdvXbKgMHnwx
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![promptless[bot]](https://avatars.githubusercontent.com/in/979718?s=60&v=4){kind=small}
There was a problem hiding this comment.
Documentation review by Promptless (blog content, src/content/blog/technical/implementing-llms-txt-for-developer-documentation.mdx). Great topic and a clear, well-structured implementation angle that reads as genuinely distinct from the existing "llms.txt Is Not What Most Teams Think It Is" post — the four-decisions framing and the closing hand-off to documentation drift both land well. Frontmatter, imported components (BlogNewsletterCTA / BlogRequestDemo), the section: Use Cases value, and the internal link to the drift article all check out against repo conventions.
I verified every external claim against its cited source rather than from memory, and found a set of citation/attribution problems in the correctness dimension — including one fabricated direct quote — that should be fixed before merge. Details are in the inline comments; none require re-authoring, just correcting the sources or softening the claims. The Stripe instructions-section paragraph checked out against its apideck source. Flagging with COMMENT for the correctness fixes; the writing itself is in good shape.
|
|
||
| The compounding problem is caching. AI coding assistants cache `llms.txt` files. A stale version can persist in an assistant's context for days or weeks after you've updated the underlying documentation. By the time you notice the problem, a significant number of developers may have generated broken integrations from it. | ||
|
|
||
| Manual maintenance produces drift. Every API change requires a corresponding `llms.txt` update. In practice, that update gets deprioritized. A 2026 analysis of implementation patterns put it directly: ["hand-maintained, it will drift, guaranteed. The only version I'd trust is one generated from the canonical content at build time."](https://www.pixelmojo.io/blogs/llms-txt-static-vs-dynamic-implementation-guide) |
There was a problem hiding this comment.
Fabricated quotation (correctness). This is presented as a verbatim quote from the linked pixelmojo article, but that exact sentence — "hand-maintained, it will drift, guaranteed. The only version I'd trust is one generated from the canonical content at build time." — does not appear in that article. The source makes a related point in different words (static files decay; a dynamic version pulling from actual content solves it), but nothing matches this wording. A direct quotation attributed to a named source is the highest-trust claim a piece can make, so this needs to be fixed: either replace it with the source's actual wording, drop the quotation marks and paraphrase (still linking the source), or cite the source that genuinely contains this sentence. Please don't ship a made-up quote.
|
|
||
| AI coding assistants move differently from human readers. They read the index to understand what's available, then fetch the sections relevant to the task the developer described. They don't scan menus or backtrack. | ||
|
|
||
| Cloudflare organizes their `llms.txt` [by product](https://buildwithfern.com/post/optimizing-api-docs-ai-agents-llms-txt-guide): Agents, AI Gateway, Workers AI, and so on. A developer building on Workers AI triggers context loading for that section only. The assistant doesn't process documentation for services it isn't integrating. |
There was a problem hiding this comment.
Misattributed claim (correctness). The Fern article linked here does not mention Cloudflare at all, and does not describe organizing llms.txt "by product" with sections like Agents / AI Gateway / Workers AI — it discusses section organization only in general terms. The Cloudflare-by-product detail needs a source that actually supports it (Cloudflare's own llms.txt or docs would be the primary source), or the specific product names should be dropped. Note the existing "llms.txt Is Not What Most Teams Think It Is" post already asserts Cloudflare "organizes all documentation by product" — reusing/citing whatever grounded that claim would keep the two posts consistent.
|
|
||
| The spec defines two files. `llms.txt` is a curated navigation index: an H1 project name, a short summary, and organized sections of links to your most important documentation pages. `llms-full.txt` is the full text of your documentation concatenated into a single Markdown file. | ||
|
|
||
| [Anthropic publishes both](https://llmstxt.org/). Their `llms.txt` runs to 8,364 tokens. Their `llms-full.txt` reaches 481,349 tokens. That gap reflects two different use cases. |
There was a problem hiding this comment.
Wrong source for the token counts (correctness). The 8,364 / 481,349 token figures are linked to llmstxt.org, but that page (the spec proposal, which uses FastHTML as its example) contains no Anthropic token counts. The numbers themselves match what our own earlier post states (llms-txt-is-not-what-most-teams-think.mdx: "Anthropic publishes an 8,364-token index alongside a 481,349-token full export"), so they're internally consistent — but the citation should point at whatever actually substantiates them (Anthropic's live llms.txt/llms-full.txt, or the same source the earlier post relied on), not the spec homepage.
|
|
||
| `llms-full.txt` serves a different pattern: an agent processing your entire documentation in a single pass, or a developer who pastes the file directly into a context window for a focused integration task. For APIs with narrow, well-scoped documentation, the full file may be sufficient on its own. | ||
|
|
||
| Most developer-facing companies with substantial API surface area benefit from publishing both. Anthropic, Cloudflare, and Zapier all maintain both files. The index for navigation, the full file for ingestion. |
There was a problem hiding this comment.
Unverified claim (correctness / audience fit). "Anthropic, Cloudflare, and Zapier all maintain both files" carries no citation. Anthropic-publishes-both is supported elsewhere, but I couldn't confirm the Zapier claim from the sources cited in this piece. Either add a source that shows each of these three publishing both llms.txt and llms-full.txt, or narrow the list to the examples you can substantiate.
|
|
||
| A developer using Cursor types `@docs` and adds your `llms.txt`. The next day, they ask the assistant to integrate your authentication flow. The assistant follows your index's confident pointer to the authentication guide. That guide describes the token format you deprecated six months ago. The integration fails. | ||
|
|
||
| That scenario is repeatable at scale. By 2026, [85% of developers use AI coding assistants](https://buildwithfern.com/post/optimizing-api-docs-ai-agents-llms-txt-guide), and many load `llms.txt` files directly into their context. How you implement the file determines whether those assistants generate working code or plausible-looking code that doesn't run. |
There was a problem hiding this comment.
Minor (correctness wording): the Fern source phrases this as "85% of developers using AI tools in 2026," not specifically "AI coding assistants." The narrower rephrasing is defensible in context, but tightening it to match the source (or citing a source that specifically measures coding-assistant usage) would remove any daylight between the claim and its citation.
2 participants
Keyword:
llms.txt for developer docsArticle plan
Thesis: Implementing llms.txt for developer documentation involves four concrete decisions — which files to publish, how to organize sections for AI coding assistant workflows, whether to add behavioral instructions (Stripe's pattern), and how to keep it current. Most teams make only the first decision and skip the rest.
Target reader: Technical writers and DevRel engineers at API/developer-tool companies who know what llms.txt is and are deciding how to implement it correctly.
Angle: Distinct from the existing llms.txt article on this blog (which covers "what it is vs. what people think") — this article is a practical implementation guide covering the decisions that matter.
Key points:
llms.txt(index) vsllms-full.txt(full content) — serves different AI use cases; large doc sites should publish bothPromptless connection: The article ends by noting that auto-generation keeps the index fresh but the pages it points to still drift — which is exactly the problem Promptless monitors for.
File
src/content/blog/technical/implementing-llms-txt-for-developer-documentation.mdxThis is an AI-generated draft and needs human review before publishing.
Generated by Claude Code