-
Notifications
You must be signed in to change notification settings - Fork 22
test(e2e): opt-in live Reserve/zapper API validation mode (zrs1) #1088
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -11,9 +11,11 @@ grep -rln "@mobile" e2e/tests # mobile-tagged specs | |
|
|
||
| Harness architecture (mock layer, trust contract, CI split) lives in | ||
| `docs/wiki/domains/e2e.md`; mock mechanics and recipes live in `e2e/CLAUDE.md`. | ||
| Playwright runs 3 projects (`playwright.config.ts`): **smoke** (`@smoke`-tagged, | ||
| Playwright runs 4 projects (`playwright.config.ts`): **smoke** (`@smoke`-tagged, | ||
| Desktop Chrome), **full** (everything else, Desktop Chrome), **mobile** | ||
| (`@mobile`-tagged, Pixel 7 viewport — off CI, `pnpm e2e:mobile`). | ||
| (`@mobile`-tagged, Pixel 7 viewport — off CI, `pnpm e2e:mobile`), and **live** | ||
| (`@live`-tagged, opt-in, excluded from the other three — see § Live API | ||
| validation). | ||
|
Comment on lines
+14
to
+18
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win Qualify the inventory as offline-only or update its counts. The new As per coding guidelines: “When documentation is created or invalidated, delete or merge superseded documentation and update stale claims during the same task.” 🤖 Prompt for AI AgentsSource: Coding guidelines |
||
|
|
||
| 73 specs across 5 top-level dirs: `general/` (8), `index-dtf/` (17), | ||
| `yield-dtf/` (6), `smoke/` (12), `flows/` (30). `index-dtf/` and `yield-dtf/` | ||
|
|
@@ -97,6 +99,21 @@ additional state coverage: [boot](tests/smoke/boot.spec.ts) (home shell), | |
| auction/issuance write and edge-case behavior) and all of `smoke/` have zero | ||
| `@mobile` coverage. | ||
|
|
||
| ## Live API validation (`tests/live/`, opt-in) | ||
|
|
||
| Off by default; enabled per surface with `E2E_LIVE_RESERVE_API` / | ||
| `E2E_LIVE_ZAPPER_API` and run with `pnpm e2e:live`. Not part of the offline | ||
| coverage matrix above — these specs validate Register's API *usage* against a | ||
| real deployment (zrs1 for the planner) and are excluded from smoke/full/mobile. | ||
| Operator guide: `e2e/README.md` § Live API mode. | ||
|
|
||
| | Surface | Spec | Covers | Not covered | | ||
| |---|---|---|---| | ||
| | Reserve API (request-level) | [live/reserve-api-contract](tests/live/reserve-api-contract.spec.ts) | `/current/prices`, `/current/dtf`, `/historical/prices`, `/historical/dtf` (+ v2 candles), compliance (geolocation, per-DTF, wallet), `/v1/discover/dtfs`, `/v1/portfolio/:address`, `/dtf/exposure`, `/dtf/rebalance`, `POST /rebalance/liquidity`, legacy `/zapper/tokens` | auth'd surfaces, write endpoints | | ||
| | Zapper/planner (request-level) | [live/zapper-api-contract](tests/live/zapper-api-contract.spec.ts) | planner `health`, bounded `/api/zapper/{chain}/tokens` probe per chain, `/api/prices/{chain}` per basket (+ reserve-covers-the-gap cross-check), buy quotes per DTF, ungoverned deploy quote | sell quotes, governed deploy body variants, on-chain execution | | ||
| | Pricing UI | [live/pricing-live](tests/live/pricing-live.spec.ts) | overview hero price + plotted chart geometry from live Reserve responses per registry DTF | any DTF whose live basket drifted from the captured chain state (skips with a re-capture instruction) | | ||
| | Zap widget UI | [live/zap-widget-live](tests/live/zap-widget-live.spec.ts) | buy quote through the real widget on Base, submitted tx equals the live quote's `tx` (mocked wallet/receipts) | sell, other chains, real chain execution | | ||
|
|
||
| ## Active fixmes (2) | ||
|
|
||
| - `tests/index-dtf/overview/edge-cases.spec.ts:22` — Market Cap data-type | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -3,6 +3,7 @@ import { DEFAULT_GEOLOCATION, mockApiRoutes, type GeolocationStatus } from '../h | |
| import type { UnmockedLogger } from '../helpers/logger' | ||
| import { MockOverrides } from '../helpers/overrides' | ||
| import { resetFrozenTime } from '../helpers/clock' | ||
| import { describeLiveConfig, isLiveMode, liveConfig, type LiveConfig } from '../helpers/live' | ||
| import type { TxRecord } from '../helpers/provider' | ||
| import type { BoundaryRequest } from '../helpers/requests' | ||
| import { mockRpcRoutes, setMockNow, setYieldReplay } from '../helpers/rpc' | ||
|
|
@@ -26,6 +27,15 @@ export interface BaseFixtures { | |
| // Every handled API/subgraph/RPC request. Tests use this to prove source, | ||
| // identity, parameters, and request counts rather than only rendered shells. | ||
| boundaryRequests: BoundaryRequest[] | ||
| // Live-API surfaces resolved from E2E_LIVE_RESERVE_API / E2E_LIVE_ZAPPER_API | ||
| // (helpers/live.ts). Empty object = the default fully-offline suite. Specs | ||
| // read it to skip when their target is not configured; the API mock reads it | ||
| // to pass a surface through to the real deployment. | ||
| live: LiveConfig | ||
| // Response-contract failures recorded by the live passthrough. Fails the test | ||
| // at teardown, exactly like `unmockedCalls` — a live run that quietly accepts | ||
| // a drifted payload would validate nothing. | ||
| liveViolations: string[] | ||
| // Escape hatch for genuinely exploratory specs: when true, unmocked calls are | ||
| // still logged/attached but don't fail the test. Default false — a committed | ||
| // migration flow must fail on any unmocked RPC/API/subgraph/egress call. | ||
|
|
@@ -44,6 +54,16 @@ export const test = base.extend<BaseFixtures>({ | |
| compliance: [DEFAULT_GEOLOCATION, { option: true }], | ||
| allowUnmocked: [false, { option: true }], | ||
|
|
||
| // oxlint-disable-next-line no-empty-pattern -- {} = no deps | ||
| live: async ({}, use) => { | ||
| await use(liveConfig()) | ||
| }, | ||
|
|
||
| // oxlint-disable-next-line no-empty-pattern -- {} = no deps | ||
| liveViolations: async ({}, use) => { | ||
| await use([]) | ||
| }, | ||
|
|
||
| // Fresh per test — a new instance means overrides never leak between tests. | ||
| // oxlint-disable-next-line no-empty-pattern -- Playwright derives fixture deps from the destructuring pattern; {} = no deps | ||
| overrides: async ({}, use) => { | ||
|
|
@@ -63,7 +83,16 @@ export const test = base.extend<BaseFixtures>({ | |
|
|
||
| unmockedCalls: [ | ||
| async ( | ||
| { page, compliance, overrides, txLog, boundaryRequests, allowUnmocked }, | ||
| { | ||
| page, | ||
| compliance, | ||
| overrides, | ||
| txLog, | ||
| boundaryRequests, | ||
| allowUnmocked, | ||
| live, | ||
| liveViolations, | ||
| }, | ||
| use, | ||
| testInfo | ||
| ) => { | ||
|
|
@@ -122,8 +151,14 @@ export const test = base.extend<BaseFixtures>({ | |
| geolocation: compliance, | ||
| overrides, | ||
| requests: boundaryRequests, | ||
| live, | ||
| liveViolations, | ||
| }) | ||
|
|
||
| if (isLiveMode(live)) { | ||
| console.log(`[E2E] live API mode: ${describeLiveConfig(live)}`) | ||
| } | ||
|
|
||
| // WalletConnect / relay / explorer: fulfill empty (NOT abort) — connectors | ||
| // init eagerly on mount and aborts surface unhandled rejections. | ||
| await page.route('**walletconnect.com**', (r) => fulfillEmpty(r)) | ||
|
|
@@ -216,6 +251,18 @@ export const test = base.extend<BaseFixtures>({ | |
| `test hit ${calls.length} unmocked call(s):\n${calls.join('\n')}` | ||
| ) | ||
| } | ||
|
|
||
| // Live contract drift is a failure of the deployment under validation — | ||
| // never a soft warning, and never silenced by allowUnmocked. | ||
| if (liveViolations.length) { | ||
| await testInfo.attach('live-contract-violations', { | ||
| body: liveViolations.join('\n'), | ||
| contentType: 'text/plain', | ||
| }) | ||
| throw new Error( | ||
| `live API returned ${liveViolations.length} contract violation(s):\n${liveViolations.join('\n')}` | ||
| ) | ||
| } | ||
|
Comment on lines
+254
to
+265
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win Attach live violations before the unmocked-calls throw. The unmocked-calls check at Lines 249-253 throws before this block runs. If a test records both an unmocked call and a live contract violation, Move the attachment above the first throw so both artifacts always reach the report. 🐛 Proposed fix for the attachment ordering if (calls.length) {
await testInfo.attach('unmocked-calls', {
body: calls.join('\n'),
contentType: 'text/plain',
})
}
+ if (liveViolations.length) {
+ await testInfo.attach('live-contract-violations', {
+ body: liveViolations.join('\n'),
+ contentType: 'text/plain',
+ })
+ }
// Every committed test is strict by default. Exploratory work must opt out
// explicitly with test.use({ allowUnmocked: true }).
if (!allowUnmocked && calls.length) {
throw new Error(
`test hit ${calls.length} unmocked call(s):\n${calls.join('\n')}`
)
}
// Live contract drift is a failure of the deployment under validation —
// never a soft warning, and never silenced by allowUnmocked.
if (liveViolations.length) {
- await testInfo.attach('live-contract-violations', {
- body: liveViolations.join('\n'),
- contentType: 'text/plain',
- })
throw new Error(
`live API returned ${liveViolations.length} contract violation(s):\n${liveViolations.join('\n')}`
)
}🤖 Prompt for AI Agents |
||
| }, | ||
| { auto: true }, | ||
| ], | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Specify that custom targets are origins.
resolveLiveTargetine2e/helpers/live.tsstoresnew URL(raw).origin. Therefore,https://host/prefixsilently loses/prefix. Either preserve the path in the helper or document and reject path-prefixed values. Apply the same rule ine2e/README.mdLines 78-84.🤖 Prompt for AI Agents