content: Your Developer Relations Docs Have a New Primary Audience - #801
content: Your Developer Relations Docs Have a New Primary Audience#801frances720 wants to merge 1 commit into
Conversation
… Primary Audience Keyword: developer relations docs Generated by edu campaign skill
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
There was a problem hiding this comment.
Documentation review — Promptless
Thanks for flagging this as an AI-generated draft needing human review; I checked it as one rather than assuming the prose had already been sanity-checked. The writing itself is in good shape — it is on-voice for the collection (second person, present tense, contractions, concrete claims over slogans), the frontmatter matches the dominant tag: Technical / section: Use Cases convention for src/content/blog/technical/, the headings are sentence case, and the argument is well-structured with a clear thesis and no buried lede. My findings are almost entirely factual, not stylistic.
What I checked: every external statistic against its cited primary source; every claim about Promptless against our own product docs; all four internal cross-links against the build; frontmatter against the blog collection schema in src/content.config.ts; and prose against style/client_style_guide.md and the collection's observed conventions.
Verdict: comment, not approve. All four external statistics are wrong, misattributed, or overstated, and the Promptless section overstates what the product does. Details are inline; the four that matter most:
- The 2026 Stack Overflow survey has not published results.
survey.stackoverflow.co/2026returns 404 and the site currently shows the 2026 survey as still collecting responses. All three figures attributed to it trace to the 2025 survey. Three separate claims rest on a source that does not exist yet. - The 65% missed-context figure appears to be fabricated. There is no survey item about AI missing relevant context. The nearest real finding is 66% for "AI solutions that are almost right, but not quite" — a different claim, and the scoping to "code review or integration work" is absent from the unscoped survey question.
- The 29%/40% trust pair is a partial bucket. Those are the "somewhat trust" rows only; the "highly trust" rows (3.1%, 2.7%) are dropped. Real totals are 33% (2025) and 43% (2024). The year-over-year direction is genuine — the levels are understated.
- "Continuously" is not how Promptless works. Promptless is event/trigger-driven, and our own docs say so in as many words. This one is worth getting exactly right because a reader can test it against our documentation.
One build-breaking issue: the llms.txt link at line 57 points at a hidden: true post, which has no generated route.
Two things I am explicitly not asking you to fix here, because they are pre-existing and house-wide rather than introduced by this PR — the Postman 68% figure (8 other published posts repeat it near-verbatim) and the 15–25% engineering-capacity figure (7 other occurrences, circular citation chain). I have flagged both inline for the record and routed the wider cleanup separately. Please don't scope-creep this PR to fix them.
One process note for the article pipeline itself: the pattern here — a real number attached to an invented subject, a year advanced past what exists, a sub-bucket relabelled as a total — is the failure mode that reads as authoritative and survives casual review. Worth a source-verification step in generate-article before the PR opens.
No suggestion in this collection (checked all 240, including closed) overlaps this file, so nothing here was previously declined.
| title: 'Your Developer Relations Docs Have a New Primary Audience' | ||
| subtitle: Published July 2026 | ||
| description: >- | ||
| 84% of developers now use AI tools to evaluate APIs. Those tools read your DevRel docs first. Here's what documentation accuracy means now. |
There was a problem hiding this comment.
Correctness — the description adds a qualifier the source does not support.
"84% of developers now use AI tools to evaluate APIs" is narrower and stronger than the underlying survey item, which is simply "Do you currently use AI tools in your development process?" The Stack Overflow survey does not measure API evaluation at all, so the API-evaluation framing has no source behind it.
The figure also drops a hedge that materially changes it: the source reads "84% of respondents are using or planning to use AI tools." Actual current use is 68.7%.
Since the description is what gets surfaced in previews and social cards, this is the version of the stat most readers will see.
| 84% of developers now use AI tools to evaluate APIs. Those tools read your DevRel docs first. Here's what documentation accuracy means now. | |
| 84% of developers now use or plan to use AI tools. Those tools read your DevRel docs first. Here's what documentation accuracy means now. |
| import BlogNewsletterCTA from '@components/site/BlogNewsletterCTA.astro'; | ||
| import BlogRequestDemo from '@components/site/BlogRequestDemo.astro'; | ||
|
|
||
| 84% of developers now use AI coding tools at work. When one of them encounters your product for the first time, they rarely start by opening your quickstart guide. They open their AI assistant and ask: "How do I authenticate with this API in Python?" |
There was a problem hiding this comment.
Correctness — unhedged restatement of a hedged figure.
Same underlying issue as the description: the source figure covers developers "using or planning to use" AI tools. Stated flat as "84% ... now use AI coding tools at work," it overstates the source. The comparable current-use number is 69% (47.1% daily, 17.7% weekly, 13.7% monthly or infrequent).
Either hedge it or use the current-use figure — the opening works with either:
| 84% of developers now use AI coding tools at work. When one of them encounters your product for the first time, they rarely start by opening your quickstart guide. They open their AI assistant and ask: "How do I authenticate with this API in Python?" | |
| 84% of developers now use or plan to use AI coding tools at work. When one of them encounters your product for the first time, they rarely start by opening your quickstart guide. They open their AI assistant and ask: "How do I authenticate with this API in Python?" |
|
|
||
| AI coding assistants, including Claude, GitHub Copilot, and Cursor, ground their answers in developer documentation. When a developer asks one of these tools how to use your API, the tool answers from whatever it can access in your docs. If that answer is right, it builds trust in your product. If it's wrong, the trust loss attaches to both the tool and your product. | ||
|
|
||
| Developer trust in AI tools is already eroding. A 2026 Stack Overflow survey found that while 84% of developers use AI tools, only 29% say they trust AI outputs. That's down from 40% the year before. The most cited reasons: incorrect suggestions and missing context. Both trace directly to documentation quality. |
There was a problem hiding this comment.
Correctness — two separate problems in one sentence. This is the most important comment on the PR.
1. There is no 2026 Stack Overflow survey. survey.stackoverflow.co/2026 returns HTTP 404, the archive runs 2011–2025, and the site currently displays "The 2026 Stack Overflow Developer Survey is live — take it now." The survey being cited was still collecting responses when this was written. All three figures in this section trace to the 2025 survey (15th edition, 49,000+ responses). Publishing a citation to an unpublished survey is the kind of error a reader can check in one click.
Worth noting the rest of the blog corpus never cites a 2025 or 2026 Stack Overflow survey — the only other dated citation is 2024 — so there is no house precedent for this attribution.
2. The 29%/40% pair is a sub-bucket presented as a total. The survey item is "How much do you trust the accuracy of the output from AI tools?" and splits into "highly trust" and "somewhat trust":
| Highly | Somewhat | Total trust | Distrust | |
|---|---|---|---|---|
| 2025 | 3.1% | 29.6% | 33% | 46% |
| 2024 | 2.7% | 40.3% | 43% | 31% |
The article's 29% and 40% are the "somewhat" rows with "highly" silently dropped. The declining trend is real and your point stands — but both levels are understated. The 46% active-distrust figure is actually the stronger number for your argument.
Separately, "the most cited reasons: incorrect suggestions and missing context" needs a source; see the next comment on the missing-context claim.
| Developer trust in AI tools is already eroding. A 2026 Stack Overflow survey found that while 84% of developers use AI tools, only 29% say they trust AI outputs. That's down from 40% the year before. The most cited reasons: incorrect suggestions and missing context. Both trace directly to documentation quality. | |
| Developer trust in AI tools is already eroding. The 2025 Stack Overflow survey found that while 84% of developers use or plan to use AI tools, only 33% trust the accuracy of AI output — down from 43% the year before — and 46% now actively distrust it. |
|
|
||
| Developer trust in AI tools is already eroding. A 2026 Stack Overflow survey found that while 84% of developers use AI tools, only 29% say they trust AI outputs. That's down from 40% the year before. The most cited reasons: incorrect suggestions and missing context. Both trace directly to documentation quality. | ||
|
|
||
| 65% of developers in the same survey reported that AI coding assistants missed relevant context during code review or integration work. That missing context comes from gaps in the documentation the AI used. |
There was a problem hiding this comment.
Correctness — this statistic does not exist in the cited survey.
I searched the 2025 survey's AI section specifically for a context-related finding. There is no item about AI tools missing relevant context, and no figure of 65% anywhere in it. The "during code review or integration work" scoping is also absent — the actual question is unscoped ("When using AI tools, which of the following problems or frustrations have you encountered?").
The nearest real findings:
| Real figure | Actual survey wording |
|---|---|
| 66% | "AI solutions that are almost right, but not quite" (top frustration) |
| 45.2% | "Debugging AI-generated code is more time-consuming" |
So the number is one point off a real figure but attached to a subject the survey never asked about. "the same survey" also chains this to the 2026 attribution in the previous line, inheriting that problem.
The real 66% figure supports your argument at least as well — "almost right, but not quite" is a good description of what stale docs produce:
| 65% of developers in the same survey reported that AI coding assistants missed relevant context during code review or integration work. That missing context comes from gaps in the documentation the AI used. | |
| 66% of developers in the same survey named "AI solutions that are almost right, but not quite" as their top frustration, and 45% said debugging AI-generated code takes more time than expected. Both are what happens when the documentation an assistant reads has drifted from the product. |
|
|
||
| AI tools consume documentation literally. They extract information and produce answers as if that information is current. A deprecation notice buried in a footnote gets missed. An outdated code example gets reproduced verbatim. The tool doesn't check when the page was last updated. It reads what's there and uses it. | ||
|
|
||
| [Documentation drift](/blog/technical/documentation-drift-detection-problem) already costs teams significantly. Compensating for inaccurate docs consumes 15-25% of total engineering capacity, as developers read source code instead of reading the docs and ask Slack questions that docs should answer. When AI tools join that picture, they extend the blast radius. One stale getting-started guide doesn't just mislead one developer. It misleads every developer who asks an AI assistant the same question. |
There was a problem hiding this comment.
Pre-existing, not yours to fix — flagging for the record only.
The 15-25% engineering-capacity figure has no external source anywhere in the corpus. It appears in 7 other places, and the citation chain is circular: most occurrences either assert it bare or link back to documentation-drift-detection-problem.mdx:31, which also asserts it bare ("a simple diagnostic ... typically finds"). One post attributes it to GetDX, unlinked. Your phrasing faithfully copies the existing documentation-debt.mdx:41 pattern, so this is house convention, not a defect you introduced.
I have routed the wider cleanup separately — please don't expand this PR to chase it.
One genuine nit that is in scope: the house style for this figure uses an en dash (15–25%) in the drift post; this line uses an ASCII hyphen. Cosmetic, and usage is already inconsistent across the corpus, so take it or leave it.
|
|
||
| [Documentation drift](/blog/technical/documentation-drift-detection-problem) already costs teams significantly. Compensating for inaccurate docs consumes 15-25% of total engineering capacity, as developers read source code instead of reading the docs and ask Slack questions that docs should answer. When AI tools join that picture, they extend the blast radius. One stale getting-started guide doesn't just mislead one developer. It misleads every developer who asks an AI assistant the same question. | ||
|
|
||
| 68% of developers cite outdated documentation as their top frustration when working with APIs, according to a Postman survey. That frustration used to show up as support tickets and community Slack threads. Now it shows up first as an AI tool giving bad advice that the developer only discovers after spending time debugging. |
There was a problem hiding this comment.
Pre-existing house-wide error, not yours to fix — flagging for the record.
I could not find 68% in any Postman State of the API report. The real documentation figures across three consecutive reports I read in full:
| Report | Figure |
|---|---|
| 2023 | 52% — lack of documentation, biggest obstacle to consuming APIs |
| 2024 | 39% — inconsistent documentation, biggest roadblock |
| 2025 | 55% — struggling with inconsistent documentation |
This is not a defect this PR introduced: the identical sentence appears in 8 other places across 7 published posts, several linking it to the Postman 2024 report specifically. Your line copies established house wording.
I have breadcrumbed the wider correction as separate work — don't scope it into this PR. But since your version is unlinked ("a Postman survey"), you may prefer to swap in a figure you can cite by year rather than inherit the unsourced one:
| 68% of developers cite outdated documentation as their top frustration when working with APIs, according to a Postman survey. That frustration used to show up as support tickets and community Slack threads. Now it shows up first as an AI tool giving bad advice that the developer only discovers after spending time debugging. | |
| 55% of developers report struggling with inconsistent documentation, according to Postman's 2025 State of the API report. That frustration used to show up as support tickets and community Slack threads. Now it shows up first as an AI tool giving bad advice that the developer only discovers after spending time debugging. |
(I could not retrieve the pre-2023 Postman PDFs — CDN returned 403 — so I cannot fully exclude a 68% figure in a 2022-or-earlier report. Given three consecutive later reports land at 52/39/55, I'd treat 68% as needing replacement regardless.)
|
|
||
| With AI tools in the picture, this gap becomes more consequential. A page isn't just being read by the developers who visit it. It's being ingested by AI tools that serve answers to many more developers who never visit the page directly. The traffic signal gets noisier while the accuracy signal stays invisible. | ||
|
|
||
| [Reaching a perfect score on agent-readability benchmarks](/blog/technical/agent-friendly-docs-necessary-not-sufficient) doesn't help if the content the agent successfully reads is wrong. Most frameworks for making docs "AI-native" focus on format: structured Markdown and [llms.txt indexes](/blog/technical/llms-txt-is-not-what-most-teams-think). These are necessary. They solve the delivery problem. They don't solve the accuracy problem. They make it easier for an AI tool to consume your docs. They don't ensure what it consumes is correct. |
There was a problem hiding this comment.
Broken link — this one breaks the published page.
/blog/technical/llms-txt-is-not-what-most-teams-think has hidden: true in its frontmatter (line 10 of that file). src/pages/blog/[...slug].astro:8 builds routes with getCollection('blog', ({ data }) => !data.hidden), so no route is generated for it — this is a hard 404, not just an unlisted page. I confirmed no redirect covers it in src/lib/generated/redirects.json or vercel.json, and every other surface (blog/index.astro, the .md endpoint, the sitemap) filters hidden entries too.
The other three cross-links are fine — documentation-drift-detection-problem, agent-friendly-docs-necessary-not-sufficient, and agent-context-engineering are all hidden: false and resolve correctly.
Three options: unhide the target post, drop the link and keep the prose, or repoint it. Simplest is to keep the sentence and drop just the link:
| [Reaching a perfect score on agent-readability benchmarks](/blog/technical/agent-friendly-docs-necessary-not-sufficient) doesn't help if the content the agent successfully reads is wrong. Most frameworks for making docs "AI-native" focus on format: structured Markdown and [llms.txt indexes](/blog/technical/llms-txt-is-not-what-most-teams-think). These are necessary. They solve the delivery problem. They don't solve the accuracy problem. They make it easier for an AI tool to consume your docs. They don't ensure what it consumes is correct. | |
| [Reaching a perfect score on agent-readability benchmarks](/blog/technical/agent-friendly-docs-necessary-not-sufficient) doesn't help if the content the agent successfully reads is wrong. Most frameworks for making docs "AI-native" focus on format: structured Markdown and llms.txt indexes. These are necessary. They solve the delivery problem. They don't solve the accuracy problem. They make it easier for an AI tool to consume your docs. They don't ensure what it consumes is correct. |
Worth knowing: there are already 4 pre-existing published→hidden links producing live 404s elsewhere in the blog, so this is a latent repo-wide issue rather than a convention — but no reason to add a fifth. The check-broken-links skill catches these against a local build.
|
|
||
| ## How Promptless closes the loop | ||
|
|
||
| Promptless monitors your documentation against your actual product and codebase continuously. When a product change creates a gap between what your docs say and what your API does, Promptless surfaces it before AI tools start serving the stale version to developers. |
There was a problem hiding this comment.
Correctness against our own product — two overstatements. Please fix both.
1. "continuously" is wrong. Promptless is event/trigger-driven, and our own documentation says so explicitly: trigger-based tasks are "event-driven: a trigger fires on new events (like a PR opening) and does not retroactively scan or audit your existing pages" (teach-promptless-a-custom-task). There is no schedule, cron, or interval field in the trigger schema — every trigger type binds to an external event. The only thing in the product that runs on a clock is suggestion archiving, which is queue housekeeping. Deep Analysis does audit docs against source, but a human submits it; there is no recurring mode.
A reader can check this against our own docs page in under a minute, which is what makes it worth fixing.
2. The "before AI tools start serving the stale version" race isn't ours to claim. Timing is configurable, not fixed — trigger_on accepts opened, first_approval, and merge, and in merge mode the gap demonstrably exists in the shipped product before Promptless acts. Publishing is also human-gated by default ("a suggestion waits for a human to review and publish it"). And site rebuild, crawler refetch, and agent cache freshness are all outside the product. We control when a gap is surfaced, not when it is closed, and nothing downstream is in our scope.
The honest version is stronger anyway, because it's specific:
| Promptless monitors your documentation against your actual product and codebase continuously. When a product change creates a gap between what your docs say and what your API does, Promptless surfaces it before AI tools start serving the stale version to developers. | |
| Promptless watches the events where product changes actually happen — pull requests, commits, feature-flag rollouts, and team conversations — and checks your documentation against each change as it lands. With the default open-time trigger, the doc update is drafted while the code is still in review, so the correction moves with the change instead of trailing it by a release cycle. |
Heads-up on a conflicting source you may find: the December 2025 changelog says Promptless Capture "continuously monitors your codebase." The formal doc contradicts it — screenshots.mdx says Capture "monitors your triggers" — so that changelog line is itself imprecise and shouldn't be used as backing.
|
|
||
| Promptless monitors your documentation against your actual product and codebase continuously. When a product change creates a gap between what your docs say and what your API does, Promptless surfaces it before AI tools start serving the stale version to developers. | ||
|
|
||
| For DevRel teams, this closes the loop that's always been hard to close manually: knowing when content that was accurate at publish time has become inaccurate since. The result is documentation that AI tools can read and use correctly, not just documentation that AI tools can access. |
There was a problem hiding this comment.
Correctness — the closer claims back the half of the problem the article just said we don't solve.
"documentation that AI tools can read and use correctly" reaches onto the delivery/format side, and Promptless has no capability there: no product feature generates or maintains llms.txt, markdown exports, content negotiation, or any agent-readable surface. I found zero references to these as customer capabilities across the product docs tree and all 22 monthly changelog entries. The llms.txt and .md endpoints on promptless.ai are this site's own build infrastructure (starlight-llms-txt in astro.config.mjs), not something shipped to customers.
This also sits awkwardly against line 57, where the article correctly argues that llms.txt and structured Markdown solve delivery while accuracy is the unsolved part. Claiming the delivery half back in the final sentence undercuts the article's own thesis.
What we do own on the quality side is real and worth naming instead — Vale prose linting and Doc Detective executable docs-as-tests both verify docs against the running product.
| For DevRel teams, this closes the loop that's always been hard to close manually: knowing when content that was accurate at publish time has become inaccurate since. The result is documentation that AI tools can read and use correctly, not just documentation that AI tools can access. | |
| For DevRel teams, this closes the loop that's always been hard to close manually: knowing when content that was accurate at publish time has become inaccurate since. The result is documentation whose content AI tools can rely on — verified against what your product actually does — not just documentation they can reach. |
Keyword:
developer relations docsArticle plan
File
src/content/blog/technical/developer-relations-docs-new-primary-audience.mdxThis is an AI-generated draft and needs human review before publishing.
Generated by Claude Code