Skip to content

fix(server): retry sandbox delete phase conflicts#1905

Open
TaylorMutch wants to merge 1 commit into
mainfrom
1517-delete-retry/tm
Open

fix(server): retry sandbox delete phase conflicts#1905
TaylorMutch wants to merge 1 commit into
mainfrom
1517-delete-retry/tm

Conversation

@TaylorMutch

Copy link
Copy Markdown
Collaborator

Summary

Retry transient sandbox resource_version conflicts while moving a sandbox into Deleting during DeleteSandbox. This prevents a cleanup-time CAS conflict from leaving provider-attached sandbox records behind and causing provider deletion to fail afterward.

Related Issue

Fixes #1517

Changes

  • Added a bounded retry around the server-side sandbox delete phase transition.
  • Kept retry scope limited to persistence Conflict errors before backend driver deletion.
  • Added a deterministic unit test that forces a stale snapshot conflict, then verifies retry preserves concurrent state and transitions to Deleting.

Testing

  • cargo test -p openshell-server set_sandbox_phase_deleting_retries_after_stale_snapshot_conflict
  • cargo test -p openshell-server delete_provider_rejects_attached_provider
  • cargo test -p openshell-server compute::tests::
  • mise run pre-commit passes

mise run pre-commit was run and failed on checks unrelated to this patch: python:format:check wants to reformat tasks/scripts/sync_docs_website.py, and helm:lint reports missing chart dependency metadata for postgresql. Rust format/check/lint completed successfully inside that run.

Checklist

  • Follows Conventional Commits
  • Commits are signed off (DCO)

Signed-off-by: Taylor Mutch <taylormutch@gmail.com>
@github-actions

Copy link
Copy Markdown

Label test:e2e applied for c6f6b7f. Open the existing run and click Re-run all jobs to execute with the label set. The run will execute the standard E2E suite after building the required gateway and supervisor images once. The matching required CI gate status on this PR will flip green automatically once the run finishes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test:e2e Requires end-to-end coverage

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: provider cleanup can fail after sandbox deletion conflict

1 participant