Self-refreshing known-good RPC pool from chainlist (cached) - #11
Closed
frolic wants to merge 2 commits into
Closed
Conversation
Instead of a fixed list, the resolver now pulls from getHealthyRpcs(): a Cache-API-backed pool that health-checks chainlist's mainnet RPCs (+ the committed seed) and caches the survivors, refreshing in the background (stale-while-revalidate via waitUntil) so requests never block on the check. Same worker, no cron/KV — the seed list is the instant cold-start + fallback. Adds a GET /ens/rpcs endpoint exposing the current pool + freshness. Live check: 41 candidates tested, 16 healthy (pool grew 7 -> 16). verify-rpcs now shares the checkRpc logic. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
It's a meta/diagnostic endpoint, not part of the /ens API surface. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Owner
Author
|
Superseded by #21, which brings this forward onto the v2 monorepo API worker ( |
frolic
added a commit
that referenced
this pull request
Jul 15, 2026
Brings **#10** (RPC rotation) and **#11** (chainlist-backed healthy list) forward onto the v2 monorepo API worker, rebuilt around a cron + KV. ## How it works - **`ethereumTransport`** — viem `fallback` across the free RPCs in random order, paid endpoint (`ETHEREUM_RPC_URL`) last. viem retries each transport zero times and advances on any non-user error, so a **429 rolls over transparently** — keeping paid usage, and cost, minimal. Shuffling spreads load (a stateless Worker can't round-robin). - **Cron (`0 * * * *`)** — health-checks the chainlist candidates and writes the survivors to **KV**, once for the whole fleet. - **Request path** — one KV read: ```ts const healthy = (await env.RPCS.get<string[]>(RPCS_KEY, "json")) ?? []; transport: ethereumTransport(healthy, env.ETHEREUM_RPC_URL) ``` A cold/empty list just means straight to the paid RPC — the transport already handles that. ## Why cron + KV rather than refreshing on the request path `caches.default` is **per-colo**. Refreshing lazily meant *every* data center re-ran the full 41-endpoint health-check pass hourly — roughly **2k check requests/hour** against the free RPCs the rotation exists to lean on, which is how you get rate-limited by them. The cron does **~41/hour**, once. And because a cron runs in a single colo, the list has to live somewhere global — hence KV, not the Cache API. ## What that deleted Falling back to the paid RPC on a cold KV removed the seed list, and with it most of the machinery: - `rpcUrls.ts` (7-endpoint seed list), `getHealthyRpcs.ts` (cache + refresh + staleness), `scripts/verify-rpcs.ts`, and a dead `alchemy.run` esbuild stub in the tests. - Gone as concepts: `waitUntil`, the `Age` check, the `refreshing` dedupe flag, and the two competing TTLs. ## Verified on a preview stage - Cron registered: `['0 * * * *']`; per-stage KV namespace created. - A triggered refresh wrote the health-checked list to KV (nodereal, publicnode, mevblocker, regional blxrbdn, …). - **Cold KV** → resolves via the paid RPC. **Warm** → resolves via the KV list. - Typecheck clean; **15/15 tests**, including the cold-KV→paid path in both the transport unit test and the miniflare integration test (which now runs with an empty KV). **Note:** the deploy token needed `Workers KV Storage: Edit` added. Supersedes #10 and #11. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #10. Makes the free-RPC pool self-refreshing instead of a fixed list: the resolver pulls from a cached, health-checked "known-good RPCs" list sourced from chainlist.
Design — no separate worker, no cron/KV
Per the discussion, this lives in the same worker as a shared function rather than a separate service (the resolver calls it in-process, no service-binding round-trip):
getHealthyRpcs(ctx)— reads a Cache-API-backed pool; on a cold/stale cache it returns the committed seed list immediately and refreshes in the background viactx.waitUntil(stale-while-revalidate), so requests never block on the health check.refresh()— fetcheschainlist.org/rpcs.json, filters mainnet https URLs (no api-key placeholders, capped for the subrequest limit), health-checks each viacheckRpc(ENS forward-resolve of vitalik.eth), and caches the survivors.GET /ens/rpcs— exposes the current pool + freshness (generatedAt,checked,rpcs) for visibility.The seed list from #10 is the instant cold-start value and the fallback if chainlist/the check is unreachable — no hard dependency.
Verified live (pr-test deploy)
/ens/rpcs→checked: 41, rpcs: 16— the pool grew from the 7 seed to 16 chainlist-verified RPCs.generatedAtadvances as the background refresh runs;/ens/resolve/vitalik.eth→ 200.api.ensideas.comuntouched.Tests (15 total, +7)
fetchChainlistRpcs(filtering),checkRpc(429/network-fail → false),getHealthyRpcs(cold→seed+refresh, fresh→no refresh, stale→serve+refresh via mocked Cache API + deps).verify-rpcsnow sharescheckRpc.Notes
getHealthyRpcsonly runs on cache misses; the refresh only runs on its own ~5-min cache miss, in the background.🤖 Generated with Claude Code