GTM-4211: validate the generated llms files and chain redirects on every build - #1027
Merged
Conversation
GTM-4211 The llms.txt header carried a benchmark line that did not match our own benchmarks page on any value. It read "Envio 1 min vs The Graph 143 min (Uniswap V2 Factory, Sentio, May 2025)". The benchmarks page has that case at Envio 8s against The Graph 19m, from April 2025, and 143 appears to be the "142x slower" multiplier read as minutes. envio.dev/llms.txt already states it correctly, so the two files disagreed. Changes here: - Header quotes the benchmark exactly as the benchmarks page reports it. - Chain count is interpolated from network-count.json, which is regenerated from the live chain API on every build, instead of a hardcoded "70+" that had drifted well below the real figure. The count is EVM-only by construction, which is why Fuel stays named separately in the sentence. - Showcase entries and standalone pages are now collected. 19 showcase pages and /videos are live and in the sitemap but appeared in no llms file. Showcase reads the same _data.js that renders the pages and standalone pages are globbed from src/pages, so neither needs a second list to keep in step. Both link to the rendered URL, as they have no .md twin. - Relative links in the per-page .md copies are resolved to absolute URLs. The copies are served from the flattened slug URL, not their source directory, so a link like ../../static/img/sync.gif or ../Advanced/hypersync.md resolved to a 404 for anything consuming them. Resolution tries URL space first, matching how the rendered page behaves, then the source tree, and warns on anything it cannot place. - Five pages gained a frontmatter description, replacing the generated "X section of the docs." filler and one empty description. Verified with a full build: builds clean, zero unresolved relative links, zero filler descriptions, and all 19 new showcase URLs plus /videos return 200 live.
GTM-4211 llms-full.txt and llms-full-blog.txt both name llms.txt in their own headers, but llms.txt named neither, so an agent entering through the index had no way to find them. envio.dev/llms.txt already links both. Adds them to Optional.
… on every build GTM-4211 Nothing checked these files after they were generated. The per-chain redirects in vercel.json are a hand-authored snapshot from the crawl-budget work, and chains move upstream, so a target can stop existing with nothing to notice. That is how envio.dev/chains/base came to be a live 404 that /docs/HyperIndex/base still redirects into. Adds scripts/validate-llms.js, wired as postbuild so it sees the real build output rather than a reconstruction of it. Two classes of check: Internal, which fail the build because they are deterministic and a failure means shipping a broken file: - the three llms files exist and are non-empty - llms.txt has a title, a summary line, and no section that matched nothing - no filler or empty descriptions - every docs.envio.dev link resolves to something in the build output - llms-full.txt and llms-full-blog.txt hold exactly the pages llms.txt lists - no relative links survive in the per-page .md copies External, which only warn, so an outage or a rate limit on someone else's host can never block a docs deploy: - every distinct envio.dev/chains redirect target still resolves - every chain in the live API has a marketing page The chain checks skip traces entries, which share a chain id with the chain they belong to and never get their own page, and accept the id-suffixed slug form (plume-98866) so suffixed chains are not reported as missing. Verified by injecting each defect class into the build output and confirming it is caught and exits non-zero, and that a clean build exits zero. On the current build it passes with two warnings, both the known Base 404. Runs in about 8 seconds.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Jordy-Baby
requested review from
keenbeen32 and
nikbhintade
and removed request for
nikbhintade
August 11, 2026 22:01
keenbeen32
approved these changes
Aug 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Nothing checked these files after they were generated.
The per-chain redirects in
vercel.jsonare a hand-authored snapshot from the crawl-budget work. Chains move upstream, so a redirect target can stop existing with nothing to notice. That is howenvio.dev/chains/basebecame a live 404 that/docs/HyperIndex/basestill redirects into.What it does
One script, wired as
postbuildso it sees the real build output rather than a reconstruction of it.Internal checks, these fail the build. Deterministic, and a failure means we would ship a broken file.
llms.txthas a title, a summary line, and no section that matched nothingdocs.envio.devlink resolves to something in the build outputllms-full.txtandllms-full-blog.txthold exactly the pagesllms.txtlists.mdcopiesExternal checks, these only warn. An outage or a rate limit on someone else's host must never block a docs deploy.
envio.dev/chainsredirect target still resolvesThe chain checks skip traces entries, which share a chain id with the chain they belong to and never get their own page, and accept the id-suffixed slug form (
plume-98866) so suffixed chains are not reported as missing.Verification
Injected each defect class into the build output and confirmed it is caught and exits non-zero, then that a clean build exits zero.
.mdlink.mdcopyOn the current build it passes with two warnings, both the known Base 404, which is #1191 in the ui repo.
Runs in about 8 seconds.
Escape hatch
node scripts/validate-llms.js --internalskips the network checks. Also available asyarn validate-llms.