You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Eden's PR CI doesn't catch a regression class that fires only when an
EVE PR triggers eden-trusted.yml against an evebuild/pr:<N> image
— it can only be detected post-merge.
Concretely, on an eden pull request the only EVE-touching workflow
that runs is eden_setup.yml: eden setup / start / eve onboard / clean against a handful of
devmodels and one release tag (eve.tag=8.6.0). test.yml is not
invoked on eden PRs at all.eden.yml runs test.yml on push to
eden master, but only against lfedge/eve:16.6.0 — a release tag, so
the rootfs's embedded eve_version is 16.6.0-kvm-amd64, exactly
matching what $short_version constructs from eve.tag-eve.hv-eve.arch.
The two eve.tag value shapes in eden config produce different
embedded eve_version strings:
eve.tag
Image ref
Embedded eve_version
16.6.0 (SemVer)
lfedge/eve:16.6.0-kvm-amd64
16.6.0-kvm-amd64
5973 (PR number)
evebuild/pr:5973-kvm-amd64
0.0.0-pr5973-<short-sha>-kvm-amd64
A test or test-helper change that depends on eve.tag-derived strings
matching the embedded version (e.g. an exact-match --os-version=…,
an ! stdout '<short_version>' assertion) passes against the first
shape and fails against the second. The first shape is the only one
exercised by eden's own CI; the second is only exercised by lf-edge/eve/.github/workflows/eden-trusted.yml when an EVE PR's
build completes.
Incident (illustrative)
df8752b added a ! stdout '{{ \$short_version }}' assertion after eveimage-remove --os-version={{ \$short_version }} in tests/update_eve_image/testdata/.
It passed every eden CI run (eden PR, eden master push) and merged on
2026-05-18. The next morning, every EVE PR run started failing the
EVE Upgrade jobs with:
```
FAIL: tests/update_eve_image/testdata/revert_eve_image_update.txt:56:
unexpected match for `-kvm-amd64` found in stdout: -kvm-amd64
```
The fix (#1187) is one-line-per-file in the testdata; the structural
issue is that nothing in eden's own CI exercised the PR-build path.
Other affected runs from the same root cause, recognizable by the
same `unexpected match` signature:
(Not mutually exclusive. Listed in roughly increasing implementation effort.)
A. Add a `test.yml` invocation to eden PR CI against a release-tag image
Cheapest: wire `eden_setup.yml` or a sibling workflow to call
`test.yml` on `pull_request` with `eve_image: "lfedge/eve:"`.
This catches breakage in `test.yml` consumers (the testdata scripts,
the test binaries) but does not close the tag-shape gap — same
shape as today's `eden.yml` post-merge run.
B. Add a `test.yml` invocation to eden PR CI against a PR-build image
Pick a stable, long-lived `evebuild/pr:-kvm-amd64` image (or pin a
specific EVE PR by ID), `docker pull` it, and run `test.yml` against
it on every eden PR. Closes the tag-shape gap directly. Tradeoff: the
pin needs occasional refresh as PRs close / images age out.
C. Add a `test.yml` invocation to eden PR CI against a freshly-built EVE master artifact
Periodically (or on every eden PR), fetch the latest successful
`eve-kvm-amd64-generic` artifact from `lf-edge/eve` master CI, load
it, and run `test.yml` against it. EVE master builds embed a similar
non-SemVer shape (e.g. `0.0.0-master-` or whatever `make show-ver`
emits for HEAD), so the assertion class trips identically. Tradeoff:
cross-repo artifact fetch adds complexity and a permissions surface.
D. Inverse cross-repo trigger: eden PRs trigger an EVE-PR-style smoke
When an eden PR opens or updates, dispatch a workflow run on an
upstream-of-record EVE PR (or on EVE master) that runs the eden PR's
`tests/` against an `evebuild/pr` image. Mirrors what `eden-trusted.yml`
does today but inverted. Closes the gap fully but is the heaviest option.
Option A by itself doesn't actually close this class of escape —
flagging that so it doesn't get picked as "the easy fix" without
realizing.
Acceptance criteria
A reproduction of #1187's bug should fail on an eden PR before merge:
Problem
Eden's PR CI doesn't catch a regression class that fires only when an
EVE PR triggers
eden-trusted.ymlagainst anevebuild/pr:<N>image— it can only be detected post-merge.
Concretely, on an eden pull request the only EVE-touching workflow
that runs is
eden_setup.yml:eden setup/start/eve onboard/cleanagainst a handful ofdevmodels and one release tag (
eve.tag=8.6.0).test.ymlis notinvoked on eden PRs at all.
eden.ymlrunstest.ymlon push toeden master, but only against
lfedge/eve:16.6.0— a release tag, sothe rootfs's embedded
eve_versionis16.6.0-kvm-amd64, exactlymatching what
$short_versionconstructs fromeve.tag-eve.hv-eve.arch.The two
eve.tagvalue shapes in eden config produce differentembedded
eve_versionstrings:eve.tageve_version16.6.0(SemVer)lfedge/eve:16.6.0-kvm-amd6416.6.0-kvm-amd645973(PR number)evebuild/pr:5973-kvm-amd640.0.0-pr5973-<short-sha>-kvm-amd64A test or test-helper change that depends on
eve.tag-derived stringsmatching the embedded version (e.g. an exact-match
--os-version=…,an
! stdout '<short_version>'assertion) passes against the firstshape and fails against the second. The first shape is the only one
exercised by eden's own CI; the second is only exercised by
lf-edge/eve/.github/workflows/eden-trusted.ymlwhen an EVE PR'sbuild completes.
Incident (illustrative)
df8752badded a! stdout '{{ \$short_version }}'assertion aftereveimage-remove --os-version={{ \$short_version }}intests/update_eve_image/testdata/.It passed every eden CI run (eden PR, eden master push) and merged on
2026-05-18. The next morning, every EVE PR run started failing the
EVE Upgrade jobs with:
```
FAIL: tests/update_eve_image/testdata/revert_eve_image_update.txt:56:
unexpected match for `-kvm-amd64` found in stdout: -kvm-amd64
```
The fix (#1187) is one-line-per-file in the testdata; the structural
issue is that nothing in eden's own CI exercised the PR-build path.
Other affected runs from the same root cause, recognizable by the
same `unexpected match` signature:
Proposed options
(Not mutually exclusive. Listed in roughly increasing implementation effort.)
A. Add a `test.yml` invocation to eden PR CI against a release-tag image
Cheapest: wire `eden_setup.yml` or a sibling workflow to call
`test.yml` on `pull_request` with `eve_image: "lfedge/eve:"`.
This catches breakage in `test.yml` consumers (the testdata scripts,
the test binaries) but does not close the tag-shape gap — same
shape as today's `eden.yml` post-merge run.
B. Add a `test.yml` invocation to eden PR CI against a PR-build image
Pick a stable, long-lived `evebuild/pr:-kvm-amd64` image (or pin a
specific EVE PR by ID), `docker pull` it, and run `test.yml` against
it on every eden PR. Closes the tag-shape gap directly. Tradeoff: the
pin needs occasional refresh as PRs close / images age out.
C. Add a `test.yml` invocation to eden PR CI against a freshly-built EVE master artifact
Periodically (or on every eden PR), fetch the latest successful
`eve-kvm-amd64-generic` artifact from `lf-edge/eve` master CI, load
it, and run `test.yml` against it. EVE master builds embed a similar
non-SemVer shape (e.g. `0.0.0-master-` or whatever `make show-ver`
emits for HEAD), so the assertion class trips identically. Tradeoff:
cross-repo artifact fetch adds complexity and a permissions surface.
D. Inverse cross-repo trigger: eden PRs trigger an EVE-PR-style smoke
When an eden PR opens or updates, dispatch a workflow run on an
upstream-of-record EVE PR (or on EVE master) that runs the eden PR's
`tests/` against an `evebuild/pr` image. Mirrors what `eden-trusted.yml`
does today but inverted. Closes the gap fully but is the heaviest option.
Option A by itself doesn't actually close this class of escape —
flagging that so it doesn't get picked as "the easy fix" without
realizing.
Acceptance criteria
A reproduction of #1187's bug should fail on an eden PR before merge:
shape (an `evebuild/pr:` or EVE-master artifact image).
transitively invokes `revert_eve_image_update`).
`unexpected match for '-kvm-amd64'` signature.
If step 4 doesn't fail, the gap isn't closed.
Related
that currently exercises `evebuild/pr:`)
needs to be invoked on eden PRs to close part of this)