Skip to content

fix(controller): keep pool scale/update/status running on schedule fa… - #1618

Open
Spground wants to merge 1 commit into
opensandbox-group:mainfrom
Spground:feature/public-fix-pool-scale-error-handling
Open

fix(controller): keep pool scale/update/status running on schedule fa…#1618
Spground wants to merge 1 commit into
opensandbox-group:mainfrom
Spground:feature/public-fix-pool-scale-error-handling

Conversation

@Spground

@Spground Spground commented Aug 25, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • What is changing and why?

scheduleSandbox now returns a best-effort result built from the last persisted allocation on failure, so update/scale/status proceed instead of aborting with a nil dereference. Failure events are recorded and conflict errors are returned first to preserve RetryOnConflict retries.

Testing

  • Not run (explain why)
  • Unit tests
  • Integration tests
  • e2e / manual verification

Breaking Changes

  • None
  • Yes (describe impact and migration path)

Checklist

  • Linked Issue or clearly described motivation
  • Added/updated docs (if needed)
  • Added/updated tests (if needed)
  • Security impact considered
  • Backward compatibility considered

…ilure

scheduleSandbox now returns a best-effort result built from the last
persisted allocation on failure, so update/scale/status proceed instead
of aborting with a nil dereference. Failure events are recorded and
conflict errors are returned first to preserve RetryOnConflict retries.
@github-actions github-actions Bot added component/k8s For kubernetes runtime size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Aug 25, 2026
@Spground Spground self-assigned this Aug 25, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d3cefed1b8

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +196 to +198
if errors.IsNotFound(err) {
return nil
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Clean up state when the inner Pool fetch is not found

When a Pool disappears between the initial Reconcile fetch and this retry-closure fetch, returning success bypasses the allocation and scale-expectation cleanup performed by the outer not-found path. Both stores are keyed only by namespace/name, so if a Pool with the same name is recreated before the queued deletion request runs, that request fetches the new object and the old state is never cleared; stale creation expectations can then block the replacement Pool's scaling until timeout, while stale allocations are exposed to its first scheduling pass. Perform the same cleanup here before returning.

Useful? React with 👍 / 👎.

allocation, err := r.Allocator.GetPoolAllocation(ctx, pool)
if err != nil {
logf.FromContext(ctx).Error(err, "Failed to read pool allocation for best-effort schedule result", "pool", pool.Name)
allocation = map[string]string{}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

空map会不会导致后面scale有问题?

toDeletePods, err := r.doRelease(ctx, pool, batchSandboxes, pods, allocAction.ToRelease)
if err != nil {
return nil, err
return r.bestEffortScheduleResult(ctx, pool, pods), err

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

前面的allocation/release 如果存在部分成功/失败的情况,这里返回的状态会不会导致后面的scale出现问题?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

component/k8s For kubernetes runtime size/M Denotes a PR that changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants