Skip to content

feat(e2e): semver-based version gates for tests#444

Open
dschmidt wants to merge 1 commit into
mainfrom
feat/e2e-version-gates
Open

feat(e2e): semver-based version gates for tests#444
dschmidt wants to merge 1 commit into
mainfrom
feat/e2e-version-gates

Conversation

@dschmidt
Copy link
Copy Markdown
Contributor

@dschmidt dschmidt commented May 16, 2026

Description

Adds infrastructure for per-test web/oc version gates so individual e2e tests can declare which (OpenCloud server, web) versions they apply to, instead of being indefinitely test.skip'd whenever the bundled OpenCloud image lags behind required web fixes.

This addresses the third "skipped pending image bump" gotcha we've hit on this repo (most recently the unzip extension's #293 fix, blocked on web#2506 + web#2513). Once the bundled web catches up, the gated test starts running automatically — no manual unskip needed.

New API

import { test, expect } from '../../../../support/test'

test('extract zip file preserves subfolder structure', async ({ skipIfWeb }) => {
  skipIfWeb('<=7.0.0', 'needs web#2506 + web#2513')
  // ... test body
})

Full semver range grammar supported (<, <=, >=, >, ^, ~, space-AND, ||-OR) — skipIfWeb is a thin wrapper around semver.satisfies. Examples:

  • skipIfWeb('<=7.0.0', 'reason') — skip on 7.0.0 and earlier.
  • skipIfWeb('>=7 <8', 'broken on 7.x') — skip any 7.x.
  • skipIfWeb('<7.1.0 || =9.0.0', 'two bad windows') — OR-combined ranges.

skipIfOc works the same against the OpenCloud server version.

Layout

  • support/fixtures/versions.ts — fixture module that extends @playwright/test's base with version-discovery fixtures.
  • support/test.ts — single composition point. Uses mergeTests so additional fixture modules can be added with one extra import + argument.
  • Specs import { test, expect } from support/test, no matter how many fixtures land later.

Version discovery

Worker-scoped Playwright fixtures populate webVersion and ocVersion:

  1. WEB_VERSION / OC_VERSION env vars take precedence when set. The fixture validates they are valid SemVer and throws clearly otherwise.
  2. Otherwise: a shared internal worker fixture (_discoveredVersions) does a one-time authenticated probe using the admin user (ADMIN_USERNAME / ADMIN_PASSWORD, defaulting to admin / admin) and scrapes both versions from web's boot-time announceVersions() console log. The log only fires once an auth context becomes ready, so an unauthenticated probe would never see it.
  3. The fixture is lazy — it only runs when a test actually uses skipIfWeb / skipIfOc. Tests that don't gate on versions pay zero cost.
  4. The probe runs at most once per worker. webVersion + ocVersion share the same probe (single login captures both log lines), no module-level state.

Measured cost of the auth probe locally: ~560ms once per worker (only when scraping is needed). Set WEB_VERSION / OC_VERSION to skip it entirely.

Migrated

  • packages/web-app-unzip/tests/e2e/extractZip.spec.ts — drops its test.skip and gates on skipIfWeb('<=7.0.0', 'needs web#2506 + web#2513'). Once a bundled OpenCloud image ships with both prerequisite web PRs, the test starts running automatically.
  • packages/web-app-pastebin/tests/e2e/pastebin.spec.ts — drops the test.skip('scrollTo works on public link', ...) whose prerequisite (fix: preserve query params on public link redirect web#2199) shipped in web v6.1.0; every web version we test against today is newer, so no gate is needed.

Related Issue

Surfaced while landing #443 — the unzip subfolder fix is correct but its e2e cannot pass against the bundled web until web#2506 + web#2513 are in. This PR provides the mechanism to gate that test on the actual running web version.

Types of changes

  • Enhancement (a change that doesn't break existing code or deployments)
  • Tests (adding or improving tests)
  • Bugfix
  • Breaking change (a modification that affects current functionality)
  • Technical debt (addressing code that needs refactoring or improvements)
  • Documentation (updates or additions to documentation)
  • Maintenance (like dependency updates or tooling adjustments)

@dschmidt dschmidt force-pushed the feat/e2e-version-gates branch 9 times, most recently from a4350a4 to bb8091d Compare May 16, 2026 10:40
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Introduces a per-test SemVer-based version-gating mechanism for e2e tests so specs can declare which OpenCloud server / web-UI versions they apply to instead of relying on indefinite test.skip. Worker-scoped Playwright fixtures lazily probe the running stack via an authenticated login that scrapes web's announceVersions() console output, with WEB_VERSION / OC_VERSION env vars as a fast-path override. The unzip e2e is migrated from test.skip to skipIfWeb('<=7.0.0', ...) so it auto-enables once a sufficient web version is bundled.

Changes:

  • New support/fixtures/versions.ts with webVersion, ocVersion, skipIfWeb, skipIfOc worker fixtures and a one-shot boot-log scrape.
  • New support/test.ts composition point exporting a mergeTests-combined test and re-exported expect.
  • extractZip.spec.ts switched to the new test, drops test.skip, gates on web <=7.0.0, and asserts the preserved nested folder structure; semver + @types/semver added as devDependencies.

Reviewed changes

Copilot reviewed 4 out of 5 changed files in this pull request and generated no comments.

Show a summary per file
File Description
support/test.ts Single fixture composition point using mergeTests to compose the versions fixture module.
support/fixtures/versions.ts Worker-scoped fixtures that resolve web/oc versions via env vars or boot-log scrape and provide skipIfWeb / skipIfOc skip helpers.
packages/web-app-unzip/tests/e2e/extractZip.spec.ts Replaces test.skip with skipIfWeb('<=7.0.0', ...) and adds nested-folder-structure assertions for the unzip subfolder fix.
package.json Adds semver ^7.8.0 and @types/semver ^7.7.1 devDependencies used by the new fixture.
pnpm-lock.yaml Lockfile entries for the new semver / @types/semver devDependencies.
Files not reviewed (1)
  • pnpm-lock.yaml: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Adds infrastructure for per-test web/oc version gates so individual e2e
tests can declare which versions they apply to instead of being
test.skip'd indefinitely whenever the bundled OpenCloud image lags
behind required web fixes.

- support/helpers/versionGates.ts: pure versionMatches() wrapping
  semver.satisfies(). Full semver range grammar supported (<, <=, >=, >,
  ^, ~, space-AND, ||-OR). 9999.0.0 is the sentinel for main /
  bleeding-edge; satisfies any >=X / >X range, never satisfies <X / <=X.
- support/fixtures/versions.ts: Playwright worker-scoped fixture that
  exposes webVersion, ocVersion, skipIfWeb, skipIfOc. WEB_VERSION /
  OC_VERSION env vars take precedence; otherwise both versions are
  scraped from the boot-time announceVersions() console log after an
  authenticated probe (announceVersions only fires once an auth context
  becomes ready, so an unauthenticated probe wouldn't see it).
- support/test.ts: re-exports playwright's test with the fixture merged
  in.
- support is now a workspace package so its vitest unit tests are
  picked up by the recursive `pnpm -r test:unit`.
- packages/web-app-unzip/tests/e2e/extractZip.spec.ts switches from
  test.skip to skipIfWeb('<=7.0.0', '...'). Once the bundled web ships
  with the prerequisite fixes (web#2506 + web#2513) the test starts
  running automatically; no manual unskip required.

CI matrix work that exercises multiple (OC, web) variants comes in a
follow-up PR.
@dschmidt dschmidt force-pushed the feat/e2e-version-gates branch from bb8091d to 4978ae0 Compare May 16, 2026 10:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants