-
Notifications
You must be signed in to change notification settings - Fork 4
content: Your Developer Relations Docs Have a New Primary Audience #801
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | |||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,75 @@ | |||||||||||||||||||||
| --- | |||||||||||||||||||||
| title: 'Your Developer Relations Docs Have a New Primary Audience' | |||||||||||||||||||||
| subtitle: Published July 2026 | |||||||||||||||||||||
| description: >- | |||||||||||||||||||||
| 84% of developers now use AI tools to evaluate APIs. Those tools read your DevRel docs first. Here's what documentation accuracy means now. | |||||||||||||||||||||
| date: '2026-07-29T00:00:00.000Z' | |||||||||||||||||||||
| author: Frances | |||||||||||||||||||||
| tag: Technical | |||||||||||||||||||||
| section: Use Cases | |||||||||||||||||||||
| hidden: false | |||||||||||||||||||||
| --- | |||||||||||||||||||||
| import BlogNewsletterCTA from '@components/site/BlogNewsletterCTA.astro'; | |||||||||||||||||||||
| import BlogRequestDemo from '@components/site/BlogRequestDemo.astro'; | |||||||||||||||||||||
|
|
|||||||||||||||||||||
| 84% of developers now use AI coding tools at work. When one of them encounters your product for the first time, they rarely start by opening your quickstart guide. They open their AI assistant and ask: "How do I authenticate with this API in Python?" | |||||||||||||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Correctness — unhedged restatement of a hedged figure. Same underlying issue as the description: the source figure covers developers "using or planning to use" AI tools. Stated flat as "84% ... now use AI coding tools at work," it overstates the source. The comparable current-use number is 69% (47.1% daily, 17.7% weekly, 13.7% monthly or infrequent). Either hedge it or use the current-use figure — the opening works with either:
Suggested change
|
|||||||||||||||||||||
|
|
|||||||||||||||||||||
| The assistant answers from your documentation. | |||||||||||||||||||||
|
|
|||||||||||||||||||||
| If your docs are accurate, it recommends the right approach. If your docs are stale, it recommends a deprecated authentication method, the developer's code fails, and they're already skeptical before they've written a line that works. | |||||||||||||||||||||
|
|
|||||||||||||||||||||
| That's the new first-touch problem in developer relations. The audience for your docs changed. Most DevRel teams haven't changed how they maintain them. | |||||||||||||||||||||
|
|
|||||||||||||||||||||
| ## AI tools are the first reader | |||||||||||||||||||||
|
|
|||||||||||||||||||||
| The traditional mental model puts the human developer at the center. You write a quickstart for someone who opens a browser tab, reads your steps, and copies your code. Your job is to make that experience clear and fast. | |||||||||||||||||||||
|
|
|||||||||||||||||||||
| That model still describes part of what happens. It no longer describes the first thing that happens. | |||||||||||||||||||||
|
|
|||||||||||||||||||||
| AI coding assistants, including Claude, GitHub Copilot, and Cursor, ground their answers in developer documentation. When a developer asks one of these tools how to use your API, the tool answers from whatever it can access in your docs. If that answer is right, it builds trust in your product. If it's wrong, the trust loss attaches to both the tool and your product. | |||||||||||||||||||||
|
|
|||||||||||||||||||||
| Developer trust in AI tools is already eroding. A 2026 Stack Overflow survey found that while 84% of developers use AI tools, only 29% say they trust AI outputs. That's down from 40% the year before. The most cited reasons: incorrect suggestions and missing context. Both trace directly to documentation quality. | |||||||||||||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Correctness — two separate problems in one sentence. This is the most important comment on the PR. 1. There is no 2026 Stack Overflow survey. Worth noting the rest of the blog corpus never cites a 2025 or 2026 Stack Overflow survey — the only other dated citation is 2024 — so there is no house precedent for this attribution. 2. The 29%/40% pair is a sub-bucket presented as a total. The survey item is "How much do you trust the accuracy of the output from AI tools?" and splits into "highly trust" and "somewhat trust":
The article's 29% and 40% are the "somewhat" rows with "highly" silently dropped. The declining trend is real and your point stands — but both levels are understated. The 46% active-distrust figure is actually the stronger number for your argument. Separately, "the most cited reasons: incorrect suggestions and missing context" needs a source; see the next comment on the missing-context claim.
Suggested change
|
|||||||||||||||||||||
|
|
|||||||||||||||||||||
| 65% of developers in the same survey reported that AI coding assistants missed relevant context during code review or integration work. That missing context comes from gaps in the documentation the AI used. | |||||||||||||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Correctness — this statistic does not exist in the cited survey. I searched the 2025 survey's AI section specifically for a context-related finding. There is no item about AI tools missing relevant context, and no figure of 65% anywhere in it. The "during code review or integration work" scoping is also absent — the actual question is unscoped ("When using AI tools, which of the following problems or frustrations have you encountered?"). The nearest real findings:
So the number is one point off a real figure but attached to a subject the survey never asked about. "the same survey" also chains this to the 2026 attribution in the previous line, inheriting that problem. The real 66% figure supports your argument at least as well — "almost right, but not quite" is a good description of what stale docs produce:
Suggested change
|
|||||||||||||||||||||
|
|
|||||||||||||||||||||
| ## What DevRel docs get used for now | |||||||||||||||||||||
|
|
|||||||||||||||||||||
| Developer relations documentation serves two overlapping audiences that used to behave more similarly than they do now. | |||||||||||||||||||||
|
|
|||||||||||||||||||||
| Human developers read contextually. They notice when something feels outdated. They run a code sample before trusting it. If your quickstart shows an endpoint that returns a 404, they search for alternatives, check your changelog, or ask in your community Slack. They work around problems. | |||||||||||||||||||||
|
|
|||||||||||||||||||||
| AI tools consume documentation literally. They extract information and produce answers as if that information is current. A deprecation notice buried in a footnote gets missed. An outdated code example gets reproduced verbatim. The tool doesn't check when the page was last updated. It reads what's there and uses it. | |||||||||||||||||||||
|
|
|||||||||||||||||||||
| [Documentation drift](/blog/technical/documentation-drift-detection-problem) already costs teams significantly. Compensating for inaccurate docs consumes 15-25% of total engineering capacity, as developers read source code instead of reading the docs and ask Slack questions that docs should answer. When AI tools join that picture, they extend the blast radius. One stale getting-started guide doesn't just mislead one developer. It misleads every developer who asks an AI assistant the same question. | |||||||||||||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Pre-existing, not yours to fix — flagging for the record only. The 15-25% engineering-capacity figure has no external source anywhere in the corpus. It appears in 7 other places, and the citation chain is circular: most occurrences either assert it bare or link back to I have routed the wider cleanup separately — please don't expand this PR to chase it. One genuine nit that is in scope: the house style for this figure uses an en dash ( |
|||||||||||||||||||||
|
|
|||||||||||||||||||||
| 68% of developers cite outdated documentation as their top frustration when working with APIs, according to a Postman survey. That frustration used to show up as support tickets and community Slack threads. Now it shows up first as an AI tool giving bad advice that the developer only discovers after spending time debugging. | |||||||||||||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Pre-existing house-wide error, not yours to fix — flagging for the record. I could not find 68% in any Postman State of the API report. The real documentation figures across three consecutive reports I read in full:
This is not a defect this PR introduced: the identical sentence appears in 8 other places across 7 published posts, several linking it to the Postman 2024 report specifically. Your line copies established house wording. I have breadcrumbed the wider correction as separate work — don't scope it into this PR. But since your version is unlinked ("a Postman survey"), you may prefer to swap in a figure you can cite by year rather than inherit the unsourced one:
Suggested change
(I could not retrieve the pre-2023 Postman PDFs — CDN returned 403 — so I cannot fully exclude a 68% figure in a 2022-or-earlier report. Given three consecutive later reports land at 52/39/55, I'd treat 68% as needing replacement regardless.) |
|||||||||||||||||||||
|
|
|||||||||||||||||||||
| <BlogNewsletterCTA /> | |||||||||||||||||||||
|
|
|||||||||||||||||||||
| ## The measurement gap | |||||||||||||||||||||
|
|
|||||||||||||||||||||
| DevRel teams traditionally measure documentation success through page traffic and developer satisfaction scores. These metrics don't detect documentation drift. | |||||||||||||||||||||
|
|
|||||||||||||||||||||
| A page can have high traffic and a four-star satisfaction rating while quietly containing outdated information. Developers who notice the inaccuracy don't always leave a review. They debug the problem, find a workaround, and move on. The bad rating goes to the support ticket, not the doc page. | |||||||||||||||||||||
|
|
|||||||||||||||||||||
| With AI tools in the picture, this gap becomes more consequential. A page isn't just being read by the developers who visit it. It's being ingested by AI tools that serve answers to many more developers who never visit the page directly. The traffic signal gets noisier while the accuracy signal stays invisible. | |||||||||||||||||||||
|
|
|||||||||||||||||||||
| [Reaching a perfect score on agent-readability benchmarks](/blog/technical/agent-friendly-docs-necessary-not-sufficient) doesn't help if the content the agent successfully reads is wrong. Most frameworks for making docs "AI-native" focus on format: structured Markdown and [llms.txt indexes](/blog/technical/llms-txt-is-not-what-most-teams-think). These are necessary. They solve the delivery problem. They don't solve the accuracy problem. They make it easier for an AI tool to consume your docs. They don't ensure what it consumes is correct. | |||||||||||||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Broken link — this one breaks the published page.
The other three cross-links are fine — Three options: unhide the target post, drop the link and keep the prose, or repoint it. Simplest is to keep the sentence and drop just the link:
Suggested change
Worth knowing: there are already 4 pre-existing published→hidden links producing live 404s elsewhere in the blog, so this is a latent repo-wide issue rather than a convention — but no reason to add a fifth. The |
|||||||||||||||||||||
|
|
|||||||||||||||||||||
| ## What this means for DevRel | |||||||||||||||||||||
|
|
|||||||||||||||||||||
| DevRel teams own the content that sits between the product and the developer. That content has always required keeping up with product changes. In practice, this has been hard. Developer advocates are often closer to the community than to the engineering sprint cycle. They learn about deprecations through Slack notifications or changelog entries, not from the PR that shipped the change. | |||||||||||||||||||||
|
|
|||||||||||||||||||||
| The stakes of that lag have gone up. When AI tools read and repeat your documentation, outdated content spreads faster and with more authority than when developers had to find it themselves. | |||||||||||||||||||||
|
|
|||||||||||||||||||||
| The practical implication: documentation maintenance needs a signal. Periodic audits aren't sufficient. A quarterly review of the docs repo doesn't catch a parameter rename that shipped last week and is already producing wrong AI recommendations. The signal needs to be close to real-time, tied to what's actually changing in the product. | |||||||||||||||||||||
|
|
|||||||||||||||||||||
| [Agent context engineering](/blog/technical/agent-context-engineering) research frames this as a knowledge layer problem. The accuracy of what agents retrieve matters more than how they retrieve it. For DevRel teams, that means documentation accuracy is now load-bearing for product adoption in a way it wasn't before AI tools became the primary first-touch for developer evaluation. | |||||||||||||||||||||
|
|
|||||||||||||||||||||
| ## How Promptless closes the loop | |||||||||||||||||||||
|
|
|||||||||||||||||||||
| Promptless monitors your documentation against your actual product and codebase continuously. When a product change creates a gap between what your docs say and what your API does, Promptless surfaces it before AI tools start serving the stale version to developers. | |||||||||||||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Correctness against our own product — two overstatements. Please fix both. 1. "continuously" is wrong. Promptless is event/trigger-driven, and our own documentation says so explicitly: trigger-based tasks are "event-driven: a trigger fires on new events (like a PR opening) and does not retroactively scan or audit your existing pages" (teach-promptless-a-custom-task). There is no schedule, cron, or interval field in the trigger schema — every trigger type binds to an external event. The only thing in the product that runs on a clock is suggestion archiving, which is queue housekeeping. Deep Analysis does audit docs against source, but a human submits it; there is no recurring mode. A reader can check this against our own docs page in under a minute, which is what makes it worth fixing. 2. The "before AI tools start serving the stale version" race isn't ours to claim. Timing is configurable, not fixed — The honest version is stronger anyway, because it's specific:
Suggested change
Heads-up on a conflicting source you may find: the December 2025 changelog says Promptless Capture "continuously monitors your codebase." The formal doc contradicts it — |
|||||||||||||||||||||
|
|
|||||||||||||||||||||
| For DevRel teams, this closes the loop that's always been hard to close manually: knowing when content that was accurate at publish time has become inaccurate since. The result is documentation that AI tools can read and use correctly, not just documentation that AI tools can access. | |||||||||||||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Correctness — the closer claims back the half of the problem the article just said we don't solve. "documentation that AI tools can read and use correctly" reaches onto the delivery/format side, and Promptless has no capability there: no product feature generates or maintains This also sits awkwardly against line 57, where the article correctly argues that llms.txt and structured Markdown solve delivery while accuracy is the unsolved part. Claiming the delivery half back in the final sentence undercuts the article's own thesis. What we do own on the quality side is real and worth naming instead — Vale prose linting and Doc Detective executable docs-as-tests both verify docs against the running product.
Suggested change
|
|||||||||||||||||||||
|
|
|||||||||||||||||||||
| <BlogRequestDemo /> | |||||||||||||||||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Correctness — the description adds a qualifier the source does not support.
"84% of developers now use AI tools to evaluate APIs" is narrower and stronger than the underlying survey item, which is simply "Do you currently use AI tools in your development process?" The Stack Overflow survey does not measure API evaluation at all, so the API-evaluation framing has no source behind it.
The figure also drops a hedge that materially changes it: the source reads "84% of respondents are using or planning to use AI tools." Actual current use is 68.7%.
Since the description is what gets surfaced in previews and social cards, this is the version of the stat most readers will see.