[lifecycle]: skip initialization timeout for nodes running workload pods - #52
[lifecycle]: skip initialization timeout for nodes running workload pods#52lukeraphael wants to merge 1 commit into
Conversation
…d pods The initialization timeout deletes a registered NodeClaim whose node never becomes Initialized. A node that is serving non-DaemonSet pods is not stuck in that sense, so deleting it would evict running workloads. Such nodes are left alone and re-checked every 5 minutes in case their pods drain away. Assisted-by: Claude Code:claude-marshmallow-eap
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
![exa-heron-staging[bot]](https://avatars.githubusercontent.com/in/3694514?s=60&v=4){kind=small}
There was a problem hiding this comment.
Heron review
global-review-orchestrator-guardian
Verdict: ❌ Reject · 1 finding · head 7863c7fafa · openai/gpt-5.6-luna · 3m 5s · session
| Severity | Rule | Location | Finding |
|---|---|---|---|
| 🛑 blocking | high-confidence-regression |
pkg/controllers/nodeclaim/lifecycle/liveness.go:178 |
Static mirror pods are incorrectly treated as workload |
Last updated 2026-08-24 03:30 UTC · reply @heron dismiss <reason> to dismiss on its comment thread
| if err != nil { | ||
| return false, err | ||
| } | ||
| return lo.ContainsBy(pods, func(pod *corev1.Pod) bool { |
There was a problem hiding this comment.
Static mirror pods are incorrectly treated as workload
🛑 blocking · rule high-confidence-regression · confidence 0.88
The new workload predicate excludes only DaemonSet pods, terminal pods, and terminating pods. It does not exclude node-owned static/mirror pods, even though this repository's scheduling contract explicitly treats those as non-reschedulable (pkg/utils/pod/scheduling.go:38-51) and defines IsOwnedByNode as the static-pod check (pkg/utils/pod/scheduling.go:174-176). A node can therefore have only a kubelet-managed mirror pod while still remaining uninitialized (for example, its bootstrap taint or an extended-resource registration is stuck); this predicate returns true, causing the timeout path at lines 151-154 to requeue forever instead of deleting the stranded NodeClaim/instance. Add !podutils.IsOwnedByNode(pod) (or use the repository's corresponding active/reschedulable classification) and add a mirror-pod regression test. This was validated by tracing the changed predicate to deleteNodeClaimForTimeout and comparing it with the existing static-pod exclusion used by the node lifecycle scheduling/disruption paths.
Suggested fix:
Exclude node-owned static/mirror pods from the predicate (for example, add && !podutils.IsOwnedByNode(pod)) and cover a node with only a mirror pod in the initialization-timeout test.
Heron review global-review-orchestrator-guardian · fingerprint fde4ce520718 · reply @heron dismiss <reason> to dismiss
1 participant
Summary
NODECLAIM_INITIALIZATION_TIMEOUTdeletes a registered NodeClaim whose node never becomes Initialized.Test plan
go test ./pkg/controllers/nodeclaim/lifecycle/(full suite)