Skip to content

[17.0] pillar: promptly reap powered-off kvm domains - #6155

Merged
eriknordmark merged 1 commit into
lf-edge:17.0from
eriknordmark:backport-6120-17.0
Jul 10, 2026
Merged

[17.0] pillar: promptly reap powered-off kvm domains#6155
eriknordmark merged 1 commit into
lf-edge:17.0from
eriknordmark:backport-6120-17.0

Conversation

@eriknordmark

Copy link
Copy Markdown
Contributor

Backport of #6120.

Reaps the qemu -no-shutdown paused domain promptly instead of waiting out the
graceful-shutdown budget, so stopping or purging a KVM app completes within a
second or two of guest poweroff instead of ~100s (occasionally the full
~11-minute force budget). Included in 17.0 per the next-17.0.0-rc label on the
original PR.

Cherry-picked from master (git cherry-pick -x 38fc3e7). The only conflict was
in kvm_test.go: the cherry-pick's diff context referenced a master-only test
function absent from 17.0, so the region was resolved to add only the
TestDecideKvmState truth table this change introduces. The applied code diff
is identical to the original.

How to test and validate this PR

Deploy a container or VM app under HV=kvm, then stop or purge it. Before this
change, teardown waits ~100s (or up to the full force budget) after the guest has
already powered off; after it, the domain is reaped within a second or two of
guest poweroff and its memory / PCI passthrough are released promptly. A guest
that is genuinely still shutting down (QMP run-state "running") keeps its full
graceful budget. pkg/pillar/hypervisor/kvm_test.go covers decideKvmState
(the QMP run-state to SwState mapping) as a truth table.

Changelog notes

Stopping or purging an app on a KVM-based device now completes promptly once the
guest has powered off, instead of waiting up to several minutes for the
graceful-shutdown timeout.

PR Backports

  • 16.0-stable: No - teardown-latency optimization, not a correctness fix.
  • 14.5-stable: No.
  • 13.4-stable: No.

Checklist

  • I've provided a proper description
  • I've written the test verification instructions
  • I've added a reference link to the original PR
  • PR's title follows the template

Stopping or purging a KVM app (container or VM) took ~100s on every stop,
and occasionally the full ~11-minute force budget, even though the guest
finished its ACPI poweroff in about a second. qemu runs with -no-shutdown,
so on guest poweroff the process does not exit: it pauses in QMP run-state
"shutdown" while still holding its memory and any assigned PCI devices.
KvmContext.Info() discarded that QMP state and reported the (still-RUNNING)
containerd task, so domainmgr could not tell the guest was down and polled
out the graceful-shutdown timers before forcing the domain off.

Info() now surfaces the QMP "shutdown" run-state as HALTING (via a small
decideKvmState helper with a unit-test truth table). waitForDomainGone and
verifyStatus act on HALTING to reap the paused domain immediately instead of
waiting out the budget, and Delete now SIGKILLs the qemu pid if the QMP quit
does not make it exit, so teardown - and the release of memory and PCI
passthrough - is bounded even when the monitor is wedged.

The published state machine is unchanged: HALTED is still reported only once
the process is gone and resources are freed; while a guest is genuinely still
shutting down its QMP run-state stays "running", so the long graceful budget
(e.g. a Windows VM) is preserved. The change is scoped to the qemu/-no-shutdown
backend; NOHYPER, xen and kubevirt do not have the pause and are untouched.

Fixes lf-edge#5916

Signed-off-by: eriknordmark <erik@zededa.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit 38fc3e7)
@codecov

codecov Bot commented Jul 10, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 19.51220% with 33 lines in your changes missing coverage. Please review.
✅ Project coverage is 20.88%. Comparing base (83ec3cc) to head (079953a).

Files with missing lines Patch % Lines
pkg/pillar/hypervisor/kvm.go 22.22% 28 Missing ⚠️
pkg/pillar/cmd/domainmgr/domainmgr.go 0.00% 4 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             17.0    #6155      +/-   ##
==========================================
+ Coverage   20.47%   20.88%   +0.41%     
==========================================
  Files         490      500      +10     
  Lines       91763    93495    +1732     
==========================================
+ Hits        18785    19526     +741     
- Misses      71413    72222     +809     
- Partials     1565     1747     +182     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@rene

rene commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

@eriknordmark , please, let's close this PR in favor of #6156 , which already contains this one.

@eriknordmark
eriknordmark merged commit 0a034d0 into lf-edge:17.0 Jul 10, 2026
35 of 39 checks passed
@eriknordmark
eriknordmark deleted the backport-6120-17.0 branch July 21, 2026 05:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants