Skip to content

Merge node finalizer removal and node deletion into one step#1097

Merged
gardener-prow[bot] merged 11 commits intogardener:masterfrom
gagan16k:cleanup
Apr 24, 2026
Merged

Merge node finalizer removal and node deletion into one step#1097
gardener-prow[bot] merged 11 commits intogardener:masterfrom
gagan16k:cleanup

Conversation

@gagan16k
Copy link
Copy Markdown
Member

@gagan16k gagan16k commented Apr 10, 2026 •
edited
Loading

What this PR does / why we need it:
Merges the separate deleteNodeFinalizers and deleteNodeObject phases into a single deleteNodeObject step that atomically removes the node finalizer and deletes the node object. This removes a gap in the deletion flow where the finalizer removal could be skipped while node deletion still proceeded, leaving a node stuck in terminating state.

Additionally:

  • Adds a machine DeletionTimestamp guard in reconcileClusterNodeKey to prevent node finalizers from being re-added while the machine is being deleted.
  • Refactors updateNode to use switch statements, removing early returns so taint/label/condition changes still trigger machine reconciliation when the finalizer is absent.

Which issue(s) this PR fixes:
Fixes #1080

Special notes for your reviewer:

IT logs 
Random Seed: 1775816871

Will run 10 of 10 specs
------------------------------
[BeforeSuite]
/Users/I765230/go/src/github.com/gagan16k/machine-controller-manager-provider-aws/test/integration/controller/controller_test.go:47
  > Enter [BeforeSuite] TOP-LEVEL @ 04/10/26 15:58:11.465
  STEP: Checking for the clusters if provided are available @ 04/10/26 15:58:11.465
  2026/04/10 15:58:11 Control cluster kube-config - /Users/I765230/go/src/github.com/gagan16k/machine-controller-manager-provider-aws/dev/kube-configs/kubeconfig_control.yaml
  2026/04/10 15:58:11 Target cluster kube-config  - /Users/I765230/go/src/github.com/gagan16k/machine-controller-manager-provider-aws/dev/kube-configs/kubeconfig_target.yaml
  STEP: Killing any existing processes @ 04/10/26 15:58:13.757
  STEP: Checking Machine-Controller-Manager repo is available at: ../../../dev/mcm @ 04/10/26 15:58:13.926
  STEP: Scaledown existing machine controllers @ 04/10/26 15:58:13.926
  STEP: Starting Machine Controller  @ 04/10/26 15:58:14.097
  STEP: Starting Machine Controller Manager @ 04/10/26 15:58:14.113
  STEP: Cleaning any old resources @ 04/10/26 15:58:14.119
  2026/04/10 15:58:14 machinedeployments.machine.sapcloud.io "test-machine-deployment" not found
  2026/04/10 15:58:14 machines.machine.sapcloud.io "test-machine" not found
  2026/04/10 15:58:14 machineclasses.machine.sapcloud.io "test-mc-v1" not found
  2026/04/10 15:58:14 machineclasses.machine.sapcloud.io "test-mc-v2" not found
  STEP: Setup MachineClass @ 04/10/26 15:58:14.798
  STEP: Looking for machineclass resource in the control cluster @ 04/10/26 15:58:15.994
  STEP: Looking for secrets refered in machineclass in the control cluster @ 04/10/26 15:58:16.181
  STEP: Initializing orphan resource tracker @ 04/10/26 15:58:16.517
  2026/04/10 15:58:20 orphan resource tracker initialized
  < Exit [BeforeSuite] TOP-LEVEL @ 04/10/26 15:58:20.46 (8.995s)
[BeforeSuite] PASSED [8.995 seconds]
------------------------------
Machine controllers test machine resource creation should not lead to any errors and add 1 more node in target cluster
/Users/I765230/go/src/github.com/gagan16k/machine-controller-manager/pkg/test/integration/common/framework.go:649
  > Enter [BeforeEach] Machine controllers test @ 04/10/26 15:58:20.46
  STEP: Checking machineController process is running @ 04/10/26 15:58:20.46
  STEP: Checking machineControllerManager process is running @ 04/10/26 15:58:20.46
  STEP: Checking nodes in target cluster are healthy @ 04/10/26 15:58:20.46
  < Exit [BeforeEach] Machine controllers test @ 04/10/26 15:58:21.088 (628ms)
  > Enter [It] should not lead to any errors and add 1 more node in target cluster @ 04/10/26 15:58:21.088
  STEP: Checking for errors @ 04/10/26 15:58:21.278
  STEP: Waiting until number of ready nodes is 1 more than initial nodes @ 04/10/26 15:58:21.443
  < Exit [It] should not lead to any errors and add 1 more node in target cluster @ 04/10/26 16:00:07.166 (1m46.078s)
• [106.706 seconds]
------------------------------
Machine controllers test machine resource deletion when machines available should not lead to errors and remove 1 node in target cluster
/Users/I765230/go/src/github.com/gagan16k/machine-controller-manager/pkg/test/integration/common/framework.go:678
  > Enter [BeforeEach] Machine controllers test @ 04/10/26 16:00:07.166
  STEP: Checking machineController process is running @ 04/10/26 16:00:07.166
  STEP: Checking machineControllerManager process is running @ 04/10/26 16:00:07.166
  STEP: Checking nodes in target cluster are healthy @ 04/10/26 16:00:07.166
  < Exit [BeforeEach] Machine controllers test @ 04/10/26 16:00:07.558 (392ms)
  > Enter [It] should not lead to errors and remove 1 node in target cluster @ 04/10/26 16:00:07.558
  STEP: Checking for errors @ 04/10/26 16:00:08.389
  STEP: Waiting until test-machine machine object is deleted @ 04/10/26 16:00:08.561
  STEP: Waiting until number of ready nodes is equal to number of initial nodes @ 04/10/26 16:00:32.715
  < Exit [It] should not lead to errors and remove 1 node in target cluster @ 04/10/26 16:00:33.267 (25.709s)
• [26.101 seconds]
------------------------------
Machine controllers test machine resource deletion when machines are not available should keep nodes intact
/Users/I765230/go/src/github.com/gagan16k/machine-controller-manager/pkg/test/integration/common/framework.go:717
  > Enter [BeforeEach] Machine controllers test @ 04/10/26 16:00:33.267
  STEP: Checking machineController process is running @ 04/10/26 16:00:33.267
  STEP: Checking machineControllerManager process is running @ 04/10/26 16:00:33.268
  STEP: Checking nodes in target cluster are healthy @ 04/10/26 16:00:33.268
  < Exit [BeforeEach] Machine controllers test @ 04/10/26 16:00:33.737 (470ms)
  > Enter [It] should keep nodes intact @ 04/10/26 16:00:33.737
  STEP: Skipping as there are machines available and this check can't be performed @ 04/10/26 16:00:34.068
  < Exit [It] should keep nodes intact @ 04/10/26 16:00:34.068 (331ms)
• [0.801 seconds]
------------------------------
Machine controllers test machine deployment resource creation with replicas=0, scale up with replicas=1 should not lead to errors and add 1 more node to target cluster
/Users/I765230/go/src/github.com/gagan16k/machine-controller-manager/pkg/test/integration/common/framework.go:745
  > Enter [BeforeEach] Machine controllers test @ 04/10/26 16:00:34.069
  STEP: Checking machineController process is running @ 04/10/26 16:00:34.069
  STEP: Checking machineControllerManager process is running @ 04/10/26 16:00:34.069
  STEP: Checking nodes in target cluster are healthy @ 04/10/26 16:00:34.069
  < Exit [BeforeEach] Machine controllers test @ 04/10/26 16:00:34.633 (564ms)
  > Enter [It] should not lead to errors and add 1 more node to target cluster @ 04/10/26 16:00:34.633
  STEP: Checking for errors @ 04/10/26 16:00:34.818
  STEP: Waiting for Machine Set to be created @ 04/10/26 16:00:34.982
  STEP: Updating machineDeployment replicas to 1 @ 04/10/26 16:00:37.613
  STEP: Checking if machineDeployment's status has been updated with correct conditions @ 04/10/26 16:00:37.94
  STEP: Checking number of ready nodes==1 @ 04/10/26 16:02:44.31
  STEP: Fetching initial number of machineset freeze events @ 04/10/26 16:02:45.723
  < Exit [It] should not lead to errors and add 1 more node to target cluster @ 04/10/26 16:02:46.379 (2m11.743s)
• [132.308 seconds]
------------------------------
Machine controllers test machine deployment resource scale-up with replicas=6 should not lead to errors and add further 5 nodes to target cluster
/Users/I765230/go/src/github.com/gagan16k/machine-controller-manager/pkg/test/integration/common/framework.go:813
  > Enter [BeforeEach] Machine controllers test @ 04/10/26 16:02:46.379
  STEP: Checking machineController process is running @ 04/10/26 16:02:46.379
  STEP: Checking machineControllerManager process is running @ 04/10/26 16:02:46.38
  STEP: Checking nodes in target cluster are healthy @ 04/10/26 16:02:46.38
  < Exit [BeforeEach] Machine controllers test @ 04/10/26 16:02:46.758 (379ms)
  > Enter [It] should not lead to errors and add further 5 nodes to target cluster @ 04/10/26 16:02:46.758
  STEP: Checking for errors @ 04/10/26 16:02:47.088
  STEP: Checking number of ready nodes are 6 more than initial @ 04/10/26 16:02:47.088
  < Exit [It] should not lead to errors and add further 5 nodes to target cluster @ 04/10/26 16:04:48.367 (2m1.608s)
• [121.987 seconds]
------------------------------
Machine controllers test machine deployment resource scale-down with replicas=2 should not lead to errors and remove 4 nodes from target cluster
/Users/I765230/go/src/github.com/gagan16k/machine-controller-manager/pkg/test/integration/common/framework.go:843
  > Enter [BeforeEach] Machine controllers test @ 04/10/26 16:04:48.367
  STEP: Checking machineController process is running @ 04/10/26 16:04:48.367
  STEP: Checking machineControllerManager process is running @ 04/10/26 16:04:48.367
  STEP: Checking nodes in target cluster are healthy @ 04/10/26 16:04:48.367
  < Exit [BeforeEach] Machine controllers test @ 04/10/26 16:04:48.919 (552ms)
  > Enter [It] should not lead to errors and remove 4 nodes from target cluster @ 04/10/26 16:04:48.919
  STEP: Checking for errors @ 04/10/26 16:04:49.961
  STEP: Checking number of ready nodes are 2 more than initial @ 04/10/26 16:04:49.961
  < Exit [It] should not lead to errors and remove 4 nodes from target cluster @ 04/10/26 16:05:15.653 (26.734s)
• [27.287 seconds]
------------------------------
Machine controllers test machine deployment resource scale-down with replicas=2 should freeze and unfreeze machineset temporarily
/Users/I765230/go/src/github.com/gagan16k/machine-controller-manager/pkg/test/integration/common/framework.go:872
  > Enter [BeforeEach] Machine controllers test @ 04/10/26 16:05:15.654
  STEP: Checking machineController process is running @ 04/10/26 16:05:15.654
  STEP: Checking machineControllerManager process is running @ 04/10/26 16:05:15.654
  STEP: Checking nodes in target cluster are healthy @ 04/10/26 16:05:15.654
  < Exit [BeforeEach] Machine controllers test @ 04/10/26 16:05:16.025 (371ms)
  > Enter [It] should freeze and unfreeze machineset temporarily @ 04/10/26 16:05:16.025
  < Exit [It] should freeze and unfreeze machineset temporarily @ 04/10/26 16:05:19.451 (3.426s)
• [3.797 seconds]
------------------------------
Machine controllers test machine deployment resource updation to v2 machine-class and replicas=4 should upgrade machines and add more nodes to target
/Users/I765230/go/src/github.com/gagan16k/machine-controller-manager/pkg/test/integration/common/framework.go:881
  > Enter [BeforeEach] Machine controllers test @ 04/10/26 16:05:19.451
  STEP: Checking machineController process is running @ 04/10/26 16:05:19.451
  STEP: Checking machineControllerManager process is running @ 04/10/26 16:05:19.451
  STEP: Checking nodes in target cluster are healthy @ 04/10/26 16:05:19.451
  < Exit [BeforeEach] Machine controllers test @ 04/10/26 16:05:20.369 (918ms)
  > Enter [It] should upgrade machines and add more nodes to target @ 04/10/26 16:05:20.369
  STEP: Checking for errors @ 04/10/26 16:05:20.705
  STEP: UpdatedReplicas to be 4 @ 04/10/26 16:05:20.706
  STEP: AvailableReplicas to be 4 @ 04/10/26 16:05:27.431
  STEP: Number of ready nodes be 4 more @ 04/10/26 16:07:25.181
  < Exit [It] should upgrade machines and add more nodes to target @ 04/10/26 16:08:00.441 (2m40.072s)
• [160.990 seconds]
------------------------------
Machine controllers test machine deployment resource deletion When there are machine deployment(s) available in control cluster should not lead to errors and list only initial nodes
/Users/I765230/go/src/github.com/gagan16k/machine-controller-manager/pkg/test/integration/common/framework.go:935
  > Enter [BeforeEach] Machine controllers test @ 04/10/26 16:08:00.441
  STEP: Checking machineController process is running @ 04/10/26 16:08:00.441
  STEP: Checking machineControllerManager process is running @ 04/10/26 16:08:00.441
  STEP: Checking nodes in target cluster are healthy @ 04/10/26 16:08:00.441
  < Exit [BeforeEach] Machine controllers test @ 04/10/26 16:08:00.826 (384ms)
  > Enter [It] should not lead to errors and list only initial nodes @ 04/10/26 16:08:00.826
  STEP: Checking for errors @ 04/10/26 16:08:00.986
  STEP: Waiting until number of ready nodes is equal to number of initial  nodes @ 04/10/26 16:08:01.154
  < Exit [It] should not lead to errors and list only initial nodes @ 04/10/26 16:08:32.076 (31.251s)
• [31.635 seconds]
------------------------------
Machine controllers test orphaned resources when the hyperscaler resources are queried should have been deleted
/Users/I765230/go/src/github.com/gagan16k/machine-controller-manager/pkg/test/integration/common/framework.go:972
  > Enter [BeforeEach] Machine controllers test @ 04/10/26 16:08:32.076
  STEP: Checking machineController process is running @ 04/10/26 16:08:32.076
  STEP: Checking machineControllerManager process is running @ 04/10/26 16:08:32.076
  STEP: Checking nodes in target cluster are healthy @ 04/10/26 16:08:32.076
  < Exit [BeforeEach] Machine controllers test @ 04/10/26 16:08:32.465 (388ms)
  > Enter [It] should have been deleted @ 04/10/26 16:08:32.465
  STEP: Querying and comparing @ 04/10/26 16:08:32.465
The following resources are orphans ... trying to delete them
Virtual Machines: []
Volumes: [vol-06c85c7307f3706f5]
NICs: [eni-047bbef8af9b80b1c]
MCM Machines []
 The following resources are orphans ... trying to delete them
Virtual Machines: []
Volumes: [vol-015a19901228e4dc1 vol-02571b17ea7739bcc vol-0163bc770ac357a04]
NICs: [eni-0759ec8e37d1d82ce eni-017e1341458d6ed3f eni-0f52245f87975f713]
MCM Machines []
 The following resources are orphans ... trying to delete them
Virtual Machines: []
Volumes: []
NICs: [eni-0759ec8e37d1d82ce eni-017e1341458d6ed3f eni-0f52245f87975f713]
MCM Machines []
 The following resources are orphans ... trying to delete them
Virtual Machines: []
Volumes: []
NICs: [eni-0759ec8e37d1d82ce eni-017e1341458d6ed3f eni-0f52245f87975f713]
MCM Machines []
 The following resources are orphans ... trying to delete them
Virtual Machines: []
Volumes: []
NICs: [eni-0759ec8e37d1d82ce eni-017e1341458d6ed3f eni-0f52245f87975f713]
MCM Machines []
 The following resources are orphans ... trying to delete them
Virtual Machines: []
Volumes: []
NICs: [eni-0759ec8e37d1d82ce eni-017e1341458d6ed3f eni-0f52245f87975f713]
MCM Machines []
 The following resources are orphans ... trying to delete them
Virtual Machines: []
Volumes: []
NICs: [eni-0759ec8e37d1d82ce eni-017e1341458d6ed3f eni-0f52245f87975f713]
MCM Machines []
 The following resources are orphans ... trying to delete them
Virtual Machines: []
Volumes: []
NICs: [eni-0759ec8e37d1d82ce eni-017e1341458d6ed3f eni-0f52245f87975f713]
MCM Machines []
 The following resources are orphans ... trying to delete them
Virtual Machines: []
Volumes: []
NICs: [eni-0759ec8e37d1d82ce eni-017e1341458d6ed3f eni-0f52245f87975f713]
MCM Machines []
 The following resources are orphans ... trying to delete them
Virtual Machines: []
Volumes: []
NICs: [eni-0759ec8e37d1d82ce eni-017e1341458d6ed3f eni-0f52245f87975f713]
MCM Machines []
 The following resources are orphans ... trying to delete them
Virtual Machines: []
Volumes: []
NICs: [eni-0759ec8e37d1d82ce eni-017e1341458d6ed3f eni-0f52245f87975f713]
MCM Machines []
 The following resources are orphans ... trying to delete them
Virtual Machines: []
Volumes: []
NICs: [eni-0759ec8e37d1d82ce eni-017e1341458d6ed3f eni-0f52245f87975f713]
MCM Machines []
 The following resources are orphans ... trying to delete them
Virtual Machines: []
Volumes: []
NICs: [eni-0759ec8e37d1d82ce eni-017e1341458d6ed3f eni-0f52245f87975f713]
MCM Machines []
 The following resources are orphans ... trying to delete them
Virtual Machines: []
Volumes: []
NICs: [eni-0759ec8e37d1d82ce eni-017e1341458d6ed3f eni-0f52245f87975f713]
MCM Machines []
 The following resources are orphans ... trying to delete them
Virtual Machines: []
Volumes: []
NICs: [eni-0759ec8e37d1d82ce eni-017e1341458d6ed3f eni-0f52245f87975f713]
MCM Machines []
 The following resources are orphans ... trying to delete them
Virtual Machines: []
Volumes: []
NICs: [eni-0759ec8e37d1d82ce eni-017e1341458d6ed3f eni-0f52245f87975f713]
MCM Machines []
 The following resources are orphans ... trying to delete them
Virtual Machines: []
Volumes: []
NICs: [eni-0759ec8e37d1d82ce eni-017e1341458d6ed3f eni-0f52245f87975f713]
MCM Machines []
 The following resources are orphans ... trying to delete them
Virtual Machines: []
Volumes: []
NICs: [eni-0759ec8e37d1d82ce eni-017e1341458d6ed3f eni-0f52245f87975f713]
MCM Machines []
 The following resources are orphans ... trying to delete them
Virtual Machines: []
Volumes: []
NICs: [eni-0759ec8e37d1d82ce eni-017e1341458d6ed3f eni-0f52245f87975f713]
MCM Machines []
 The following resources are orphans ... trying to delete them
Virtual Machines: []
Volumes: []
NICs: [eni-0759ec8e37d1d82ce eni-017e1341458d6ed3f eni-0f52245f87975f713]
MCM Machines []
 The following resources are orphans ... trying to delete them
Virtual Machines: []
Volumes: []
NICs: [eni-0759ec8e37d1d82ce eni-017e1341458d6ed3f eni-0f52245f87975f713]
MCM Machines []
 The following resources are orphans ... trying to delete them
Virtual Machines: []
Volumes: []
NICs: [eni-0759ec8e37d1d82ce eni-017e1341458d6ed3f eni-0f52245f87975f713]
MCM Machines []
 The following resources are orphans ... trying to delete them
Virtual Machines: []
Volumes: []
NICs: [eni-0759ec8e37d1d82ce eni-017e1341458d6ed3f eni-0f52245f87975f713]
MCM Machines []
 The following resources are orphans ... trying to delete them
Virtual Machines: []
Volumes: []
NICs: [eni-0759ec8e37d1d82ce eni-017e1341458d6ed3f eni-0f52245f87975f713]
MCM Machines []

The following resources are orphans ... trying to delete them
Virtual Machines: []
Volumes: []
NICs: [eni-0759ec8e37d1d82ce eni-017e1341458d6ed3f eni-0f52245f87975f713]
MCM Machines []
 The following resources are orphans ... trying to delete them
Virtual Machines: []
Volumes: []
NICs: [eni-0759ec8e37d1d82ce eni-017e1341458d6ed3f eni-0f52245f87975f713]
MCM Machines []
 The following resources are orphans ... trying to delete them
Virtual Machines: []
Volumes: []
NICs: [eni-0759ec8e37d1d82ce eni-017e1341458d6ed3f eni-0f52245f87975f713]
MCM Machines []
 The following resources are orphans ... trying to delete them
Virtual Machines: []
Volumes: []
NICs: [eni-0759ec8e37d1d82ce eni-017e1341458d6ed3f eni-0f52245f87975f713]
MCM Machines []
 The following resources are orphans ... trying to delete them
Virtual Machines: []
Volumes: []
NICs: [eni-0759ec8e37d1d82ce eni-017e1341458d6ed3f eni-0f52245f87975f713]
MCM Machines []
 The following resources are orphans ... trying to delete them
Virtual Machines: []
Volumes: []
NICs: [eni-0759ec8e37d1d82ce eni-017e1341458d6ed3f eni-0f52245f87975f713]
MCM Machines []
   < Exit [It] should have been deleted @ 04/10/26 16:11:17.897 (2m45.432s)
• [165.821 seconds]
------------------------------
[AfterSuite]
/Users/I765230/go/src/github.com/gagan16k/machine-controller-manager-provider-aws/test/integration/controller/controller_test.go:49
  > Enter [AfterSuite] TOP-LEVEL @ 04/10/26 16:11:17.897
  STEP: Running Cleanup @ 04/10/26 16:11:17.897
  2026/04/10 16:11:38 machinedeployments.machine.sapcloud.io "test-machine-deployment" not found
  2026/04/10 16:11:38 machines.machine.sapcloud.io "test-machine" not found
  2026/04/10 16:11:39 deleting test-mc-v1 machineclass
  2026/04/10 16:11:39 machineclass deleted
  2026/04/10 16:11:39 deleting test-mc-v2 machineclass
  2026/04/10 16:11:40 machineclass deleted
  STEP: Killing any existing processes @ 04/10/26 16:11:40.251
  2026/04/10 16:11:40 controller_manager --control-kubeconfig=/Users/I765230/go/src/github.com/gagan16k/machine-controller-manager-provider-aws/dev/kube-configs/kubeconfig_control.yaml --target-kubeconfig=/Users/I765230/go/src/github.com/gagan16k/machine-controller-manager-provider-aws/dev/kube-configs/kubeconfig_target.yaml --namespace=shoot--i765230--demo --safety-up=2 --safety-down=1 --machine-safety-overshooting-period=300ms --leader-elect=false --v=3
  2026/04/10 16:11:40 stopMCM killed MCM process(es) with pid(s): [18738]
  2026/04/10 16:11:40 main --control-kubeconfig=/Users/I765230/go/src/github.com/gagan16k/machine-controller-manager-provider-aws/dev/kube-configs/kubeconfig_control.yaml --target-kubeconfig=/Users/I765230/go/src/github.com/gagan16k/machine-controller-manager-provider-aws/dev/kube-configs/kubeconfig_target.yaml --namespace=shoot--i765230--demo --machine-creation-timeout=20m --machine-drain-timeout=5m --machine-health-timeout=10m --machine-pv-detach-timeout=2m --machine-safety-apiserver-statuscheck-timeout=30s --machine-safety-apiserver-statuscheck-period=1m --machine-safety-orphan-vms-period=30m --leader-elect=false --v=3
  2026/04/10 16:11:40 stopMCM killed MCM process(es) with pid(s): [18743]
  2026/04/10 16:11:40 go run cmd/machine-controller/main.go --control-kubeconfig=/Users/I765230/go/src/github.com/gagan16k/machine-controller-manager-provider-aws/dev/kube-configs/kubeconfig_control.yaml --target-kubeconfig=/Users/I765230/go/src/github.com/gagan16k/machine-controller-manager-provider-aws/dev/kube-configs/kubeconfig_target.yaml --namespace=shoot--i765230--demo --machine-creation-timeout=20m --machine-drain-timeout=5m --machine-health-timeout=10m --machine-pv-detach-timeout=2m --machine-safety-apiserver-statuscheck-timeout=30s --machine-safety-apiserver-statuscheck-period=1m --machine-safety-orphan-vms-period=30m --leader-elect=false --v=3
  2026/04/10 16:11:40 stopMCM killed MCM process(es) with pid(s): [18613]
  STEP: Scale back the existing machine controllers @ 04/10/26 16:11:40.424
  < Exit [AfterSuite] TOP-LEVEL @ 04/10/26 16:11:41.099 (23.202s)
[AfterSuite] PASSED [23.203 seconds]
------------------------------

Ran 10 of 10 Specs in 809.632 seconds
SUCCESS! -- 10 Passed | 0 Failed | 0 Pending | 0 Skipped
PASS

Ginkgo ran 1 suite in 13m49.352376s
Test Suite Passed
Integration tests completed successfully

Release note:

Merged node finalizer removal and node object deletion into a single atomic step during machine deletion

@gagan16k gagan16k requested a review from a team as a code owner April 10, 2026 10:48
@gardener-prow gardener-prow Bot added do-not-merge/needs-kind Indicates a PR lacks a `kind/foo` label and requires one. cla: yes Indicates the PR's author has signed the cla-assistant.io CLA. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Apr 10, 2026
@gagan16k gagan16k added kind/enhancement Enhancement, improvement, extension kind/cleanup Something that is not needed anymore and can be cleaned up labels Apr 10, 2026
@gardener-prow gardener-prow Bot removed do-not-merge/needs-kind Indicates a PR lacks a `kind/foo` label and requires one. labels Apr 10, 2026
takoverflow
takoverflow reviewed Apr 16, 2026
View reviewed changes
Comment thread pkg/util/provider/machinecontroller/machine_util.go Outdated
Comment thread pkg/util/provider/machinecontroller/node.go Outdated
Comment thread pkg/util/provider/machinecontroller/node.go
r4mek
r4mek reviewed Apr 16, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

@r4mek r4mek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR!

Comment thread pkg/util/provider/machinecontroller/node.go Outdated
Comment thread pkg/util/provider/machinecontroller/node.go Outdated
takoverflow
takoverflow reviewed Apr 20, 2026
View reviewed changes
Comment thread pkg/util/provider/machinecontroller/machine_util.go Outdated
Comment thread pkg/util/provider/machinecontroller/machine_util.go Outdated
Comment thread pkg/util/provider/machinecontroller/machine_util.go Outdated
Comment thread pkg/util/provider/machinecontroller/machine_util.go Outdated
Comment thread pkg/util/provider/machinecontroller/node.go Outdated
Comment thread pkg/util/provider/machinecontroller/node.go Outdated
Comment thread pkg/util/provider/machinecontroller/node.go Outdated
takoverflow
takoverflow reviewed Apr 20, 2026
View reviewed changes
Comment thread pkg/util/provider/machinecontroller/machine_util.go Outdated
@gardener-prow gardener-prow Bot added cla: no Indicates the PR's author has not signed the cla-assistant.io CLA. cla: yes Indicates the PR's author has signed the cla-assistant.io CLA. and removed cla: yes Indicates the PR's author has signed the cla-assistant.io CLA. cla: no Indicates the PR's author has not signed the cla-assistant.io CLA. labels Apr 20, 2026
@r4mek
Copy link
Copy Markdown
Contributor

r4mek commented Apr 22, 2026

/lgtm

@gardener-prow gardener-prow Bot added the lgtm Indicates that a PR is ready to be merged. label Apr 22, 2026
@gardener-prow
Copy link
Copy Markdown

gardener-prow Bot commented Apr 22, 2026

LGTM label has been added.

DetailsGit tree hash: 2767ad0e6ec111fd16b72022bb2085195800adb2

@takoverflow
Copy link
Copy Markdown
Member

takoverflow commented Apr 22, 2026

/lgtm

@gagan16k
Copy link
Copy Markdown
Member Author

gagan16k commented Apr 22, 2026

Integration test after changes

IT logs 
===============================================================================================================================================
Random Seed: 1776845890

Will run 10 of 10 specs
------------------------------
[BeforeSuite]
/Users/I765230/go/src/github.com/gagan16k/machine-controller-manager-provider-aws/test/integration/controller/controller_test.go:47
  > Enter [BeforeSuite] TOP-LEVEL @ 04/22/26 13:48:22.866
  STEP: Checking for the clusters if provided are available @ 04/22/26 13:48:22.867
  2026/04/22 13:48:22 Control cluster kube-config - /Users/I765230/go/src/github.com/gagan16k/machine-controller-manager-provider-aws/dev/kube-configs/kubeconfig_control.yaml
  2026/04/22 13:48:22 Target cluster kube-config  - /Users/I765230/go/src/github.com/gagan16k/machine-controller-manager-provider-aws/dev/kube-configs/kubeconfig_target.yaml
  STEP: Killing any existing processes @ 04/22/26 13:48:25.744
  STEP: Checking Machine-Controller-Manager repo is available at: ../../../dev/mcm @ 04/22/26 13:48:26.013
  STEP: Scaledown existing machine controllers @ 04/22/26 13:48:26.014
  STEP: Starting Machine Controller  @ 04/22/26 13:48:26.273
  STEP: Starting Machine Controller Manager @ 04/22/26 13:48:26.286
  STEP: Cleaning any old resources @ 04/22/26 13:48:26.301
  2026/04/22 13:48:26 machinedeployments.machine.sapcloud.io "test-machine-deployment" not found
  2026/04/22 13:48:26 machines.machine.sapcloud.io "test-machine" not found
  2026/04/22 13:48:27 machineclasses.machine.sapcloud.io "test-mc-v1" not found
  2026/04/22 13:48:27 machineclasses.machine.sapcloud.io "test-mc-v2" not found
  STEP: Setup MachineClass @ 04/22/26 13:48:27.294
  STEP: Looking for machineclass resource in the control cluster @ 04/22/26 13:48:29.108
  STEP: Looking for secrets refered in machineclass in the control cluster @ 04/22/26 13:48:29.362
  STEP: Initializing orphan resource tracker @ 04/22/26 13:48:29.853
  2026/04/22 13:48:34 orphan resource tracker initialized
  < Exit [BeforeSuite] TOP-LEVEL @ 04/22/26 13:48:34.469 (11.602s)
[BeforeSuite] PASSED [11.602 seconds]
------------------------------
Machine controllers test machine resource creation should not lead to any errors and add 1 more node in target cluster
/Users/I765230/go/src/github.com/gagan16k/machine-controller-manager/pkg/test/integration/common/framework.go:649
  > Enter [BeforeEach] Machine controllers test @ 04/22/26 13:48:34.469
  STEP: Checking machineController process is running @ 04/22/26 13:48:34.469
  STEP: Checking machineControllerManager process is running @ 04/22/26 13:48:34.469
  STEP: Checking nodes in target cluster are healthy @ 04/22/26 13:48:34.469
  < Exit [BeforeEach] Machine controllers test @ 04/22/26 13:48:35.223 (754ms)
  > Enter [It] should not lead to any errors and add 1 more node in target cluster @ 04/22/26 13:48:35.223
  STEP: Checking for errors @ 04/22/26 13:48:35.476
  STEP: Waiting until number of ready nodes is 1 more than initial nodes @ 04/22/26 13:48:35.724
  < Exit [It] should not lead to any errors and add 1 more node in target cluster @ 04/22/26 13:50:20.089 (1m44.866s)
• [105.620 seconds]
------------------------------
Machine controllers test machine resource deletion when machines available should not lead to errors and remove 1 node in target cluster
/Users/I765230/go/src/github.com/gagan16k/machine-controller-manager/pkg/test/integration/common/framework.go:678
  > Enter [BeforeEach] Machine controllers test @ 04/22/26 13:50:20.089
  STEP: Checking machineController process is running @ 04/22/26 13:50:20.089
  STEP: Checking machineControllerManager process is running @ 04/22/26 13:50:20.089
  STEP: Checking nodes in target cluster are healthy @ 04/22/26 13:50:20.089
  < Exit [BeforeEach] Machine controllers test @ 04/22/26 13:50:20.587 (498ms)
  > Enter [It] should not lead to errors and remove 1 node in target cluster @ 04/22/26 13:50:20.587
  STEP: Checking for errors @ 04/22/26 13:50:21.648
  STEP: Waiting until test-machine machine object is deleted @ 04/22/26 13:50:21.911
  STEP: Waiting until number of ready nodes is equal to number of initial nodes @ 04/22/26 13:51:00.471
  < Exit [It] should not lead to errors and remove 1 node in target cluster @ 04/22/26 13:51:01.209 (40.622s)
• [41.120 seconds]
------------------------------
Machine controllers test machine resource deletion when machines are not available should keep nodes intact
/Users/I765230/go/src/github.com/gagan16k/machine-controller-manager/pkg/test/integration/common/framework.go:717
  > Enter [BeforeEach] Machine controllers test @ 04/22/26 13:51:01.209
  STEP: Checking machineController process is running @ 04/22/26 13:51:01.209
  STEP: Checking machineControllerManager process is running @ 04/22/26 13:51:01.209
  STEP: Checking nodes in target cluster are healthy @ 04/22/26 13:51:01.209
  < Exit [BeforeEach] Machine controllers test @ 04/22/26 13:51:01.7 (491ms)
  > Enter [It] should keep nodes intact @ 04/22/26 13:51:01.7
  STEP: Skipping as there are machines available and this check can't be performed @ 04/22/26 13:51:02.22
  < Exit [It] should keep nodes intact @ 04/22/26 13:51:02.22 (521ms)
• [1.011 seconds]
------------------------------
Machine controllers test machine deployment resource creation with replicas=0, scale up with replicas=1 should not lead to errors and add 1 more node to target cluster
/Users/I765230/go/src/github.com/gagan16k/machine-controller-manager/pkg/test/integration/common/framework.go:745
  > Enter [BeforeEach] Machine controllers test @ 04/22/26 13:51:02.221
  STEP: Checking machineController process is running @ 04/22/26 13:51:02.221
  STEP: Checking machineControllerManager process is running @ 04/22/26 13:51:02.221
  STEP: Checking nodes in target cluster are healthy @ 04/22/26 13:51:02.221
  < Exit [BeforeEach] Machine controllers test @ 04/22/26 13:51:02.724 (504ms)
  > Enter [It] should not lead to errors and add 1 more node to target cluster @ 04/22/26 13:51:02.725
  STEP: Checking for errors @ 04/22/26 13:51:02.972
  STEP: Waiting for Machine Set to be created @ 04/22/26 13:51:03.223
  STEP: Updating machineDeployment replicas to 1 @ 04/22/26 13:51:06.213
  STEP: Checking if machineDeployment's status has been updated with correct conditions @ 04/22/26 13:51:06.714
  STEP: Checking number of ready nodes==1 @ 04/22/26 13:53:06.33
  STEP: Fetching initial number of machineset freeze events @ 04/22/26 13:53:07.86
  < Exit [It] should not lead to errors and add 1 more node to target cluster @ 04/22/26 13:53:08.988 (2m6.264s)
• [126.768 seconds]
------------------------------
Machine controllers test machine deployment resource scale-up with replicas=6 should not lead to errors and add further 5 nodes to target cluster
/Users/I765230/go/src/github.com/gagan16k/machine-controller-manager/pkg/test/integration/common/framework.go:813
  > Enter [BeforeEach] Machine controllers test @ 04/22/26 13:53:08.988
  STEP: Checking machineController process is running @ 04/22/26 13:53:08.988
  STEP: Checking machineControllerManager process is running @ 04/22/26 13:53:08.988
  STEP: Checking nodes in target cluster are healthy @ 04/22/26 13:53:08.988
  < Exit [BeforeEach] Machine controllers test @ 04/22/26 13:53:09.788 (799ms)
  > Enter [It] should not lead to errors and add further 5 nodes to target cluster @ 04/22/26 13:53:09.788
  STEP: Checking for errors @ 04/22/26 13:53:10.345
  STEP: Checking number of ready nodes are 6 more than initial @ 04/22/26 13:53:10.345
  < Exit [It] should not lead to errors and add further 5 nodes to target cluster @ 04/22/26 13:55:05.641 (1m55.854s)
• [116.653 seconds]
------------------------------
Machine controllers test machine deployment resource scale-down with replicas=2 should not lead to errors and remove 4 nodes from target cluster
/Users/I765230/go/src/github.com/gagan16k/machine-controller-manager/pkg/test/integration/common/framework.go:843
  > Enter [BeforeEach] Machine controllers test @ 04/22/26 13:55:05.641
  STEP: Checking machineController process is running @ 04/22/26 13:55:05.641
  STEP: Checking machineControllerManager process is running @ 04/22/26 13:55:05.641
  STEP: Checking nodes in target cluster are healthy @ 04/22/26 13:55:05.641
  < Exit [BeforeEach] Machine controllers test @ 04/22/26 13:55:06.192 (551ms)
  > Enter [It] should not lead to errors and remove 4 nodes from target cluster @ 04/22/26 13:55:06.192
  STEP: Checking for errors @ 04/22/26 13:55:07.432
  STEP: Checking number of ready nodes are 2 more than initial @ 04/22/26 13:55:07.432
  < Exit [It] should not lead to errors and remove 4 nodes from target cluster @ 04/22/26 13:55:38.1 (31.907s)
• [32.459 seconds]
------------------------------
Machine controllers test machine deployment resource scale-down with replicas=2 should freeze and unfreeze machineset temporarily
/Users/I765230/go/src/github.com/gagan16k/machine-controller-manager/pkg/test/integration/common/framework.go:872
  > Enter [BeforeEach] Machine controllers test @ 04/22/26 13:55:38.1
  STEP: Checking machineController process is running @ 04/22/26 13:55:38.1
  STEP: Checking machineControllerManager process is running @ 04/22/26 13:55:38.1
  STEP: Checking nodes in target cluster are healthy @ 04/22/26 13:55:38.1
  < Exit [BeforeEach] Machine controllers test @ 04/22/26 13:55:38.69 (591ms)
  > Enter [It] should freeze and unfreeze machineset temporarily @ 04/22/26 13:55:38.691
  < Exit [It] should freeze and unfreeze machineset temporarily @ 04/22/26 13:55:39.908 (1.217s)
• [1.808 seconds]
------------------------------
Machine controllers test machine deployment resource updation to v2 machine-class and replicas=4 should upgrade machines and add more nodes to target
/Users/I765230/go/src/github.com/gagan16k/machine-controller-manager/pkg/test/integration/common/framework.go:881
  > Enter [BeforeEach] Machine controllers test @ 04/22/26 13:55:39.908
  STEP: Checking machineController process is running @ 04/22/26 13:55:39.908
  STEP: Checking machineControllerManager process is running @ 04/22/26 13:55:39.908
  STEP: Checking nodes in target cluster are healthy @ 04/22/26 13:55:39.908
  < Exit [BeforeEach] Machine controllers test @ 04/22/26 13:55:41.061 (1.152s)
  > Enter [It] should upgrade machines and add more nodes to target @ 04/22/26 13:55:41.061
  STEP: Checking for errors @ 04/22/26 13:55:41.644
  STEP: UpdatedReplicas to be 4 @ 04/22/26 13:55:41.644
  STEP: AvailableReplicas to be 4 @ 04/22/26 13:55:51.054
  STEP: Number of ready nodes be 4 more @ 04/22/26 13:57:45.76
  < Exit [It] should upgrade machines and add more nodes to target @ 04/22/26 13:58:30.3 (2m49.24s)
• [170.392 seconds]
------------------------------
Machine controllers test machine deployment resource deletion When there are machine deployment(s) available in control cluster should not lead to errors and list only initial nodes
/Users/I765230/go/src/github.com/gagan16k/machine-controller-manager/pkg/test/integration/common/framework.go:935
  > Enter [BeforeEach] Machine controllers test @ 04/22/26 13:58:30.3
  STEP: Checking machineController process is running @ 04/22/26 13:58:30.3
  STEP: Checking machineControllerManager process is running @ 04/22/26 13:58:30.3
  STEP: Checking nodes in target cluster are healthy @ 04/22/26 13:58:30.3
  < Exit [BeforeEach] Machine controllers test @ 04/22/26 13:58:30.876 (575ms)
  > Enter [It] should not lead to errors and list only initial nodes @ 04/22/26 13:58:30.876
  STEP: Checking for errors @ 04/22/26 13:58:31.162
  STEP: Waiting until number of ready nodes is equal to number of initial  nodes @ 04/22/26 13:58:31.447
  < Exit [It] should not lead to errors and list only initial nodes @ 04/22/26 13:59:04.208 (33.332s)
• [33.908 seconds]
------------------------------
Machine controllers test orphaned resources when the hyperscaler resources are queried should have been deleted
/Users/I765230/go/src/github.com/gagan16k/machine-controller-manager/pkg/test/integration/common/framework.go:972
  > Enter [BeforeEach] Machine controllers test @ 04/22/26 13:59:04.208
  STEP: Checking machineController process is running @ 04/22/26 13:59:04.208
  STEP: Checking machineControllerManager process is running @ 04/22/26 13:59:04.208
  STEP: Checking nodes in target cluster are healthy @ 04/22/26 13:59:04.208
  < Exit [BeforeEach] Machine controllers test @ 04/22/26 13:59:04.76 (552ms)
  > Enter [It] should have been deleted @ 04/22/26 13:59:04.76
  STEP: Querying and comparing @ 04/22/26 13:59:04.76
The following resources are orphans ... trying to delete them
Virtual Machines: []
Volumes: [vol-0b4ffa8bf3940651f vol-06848eccee7f42d02]
NICs: [eni-0f6702d05f5c4e056 eni-0ee46747bca3a0c7b]
MCM Machines []
 The following resources are orphans ... trying to delete them
Virtual Machines: []
Volumes: [vol-0f3ed23f2a15c2cbf vol-0d28c7ab46f0804fb]
NICs: []
MCM Machines []
   < Exit [It] should have been deleted @ 04/22/26 13:59:24.953 (20.193s)
• [20.745 seconds]
------------------------------
[AfterSuite]
/Users/I765230/go/src/github.com/gagan16k/machine-controller-manager-provider-aws/test/integration/controller/controller_test.go:49
  > Enter [AfterSuite] TOP-LEVEL @ 04/22/26 13:59:24.953
  STEP: Running Cleanup @ 04/22/26 13:59:24.953
  2026/04/22 13:59:45 machinedeployments.machine.sapcloud.io "test-machine-deployment" not found
  2026/04/22 13:59:45 machines.machine.sapcloud.io "test-machine" not found
  2026/04/22 13:59:45 deleting test-mc-v1 machineclass
  2026/04/22 13:59:46 machineclass deleted
  2026/04/22 13:59:47 deleting test-mc-v2 machineclass
  2026/04/22 13:59:47 machineclass deleted
  STEP: Killing any existing processes @ 04/22/26 13:59:47.895
  2026/04/22 13:59:48 controller_manager --control-kubeconfig=/Users/I765230/go/src/github.com/gagan16k/machine-controller-manager-provider-aws/dev/kube-configs/kubeconfig_control.yaml --target-kubeconfig=/Users/I765230/go/src/github.com/gagan16k/machine-controller-manager-provider-aws/dev/kube-configs/kubeconfig_target.yaml --namespace=shoot--i765230--demo --safety-up=2 --safety-down=1 --machine-safety-overshooting-period=300ms --leader-elect=false --v=3
  2026/04/22 13:59:48 stopMCM killed MCM process(es) with pid(s): [3176]
  2026/04/22 13:59:48 main --control-kubeconfig=/Users/I765230/go/src/github.com/gagan16k/machine-controller-manager-provider-aws/dev/kube-configs/kubeconfig_control.yaml --target-kubeconfig=/Users/I765230/go/src/github.com/gagan16k/machine-controller-manager-provider-aws/dev/kube-configs/kubeconfig_target.yaml --namespace=shoot--i765230--demo --machine-creation-timeout=20m --machine-drain-timeout=5m --machine-health-timeout=10m --machine-pv-detach-timeout=2m --machine-safety-apiserver-statuscheck-timeout=30s --machine-safety-apiserver-statuscheck-period=1m --machine-safety-orphan-vms-period=30m --leader-elect=false --v=3
  2026/04/22 13:59:48 stopMCM killed MCM process(es) with pid(s): [3172]
  STEP: Scale back the existing machine controllers @ 04/22/26 13:59:48.21
  < Exit [AfterSuite] TOP-LEVEL @ 04/22/26 13:59:49.252 (24.299s)
[AfterSuite] PASSED [24.299 seconds]
------------------------------

Ran 10 of 10 Specs in 686.387 seconds
SUCCESS! -- 10 Passed | 0 Failed | 0 Pending | 0 Skipped
PASS

Ginkgo ran 1 suite in 11m38.957481167s
Test Suite Passed
Integration tests completed successfully

@gardener-prow
Copy link
Copy Markdown

gardener-prow Bot commented Apr 24, 2026

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: takoverflow

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@gardener-prow gardener-prow Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Apr 24, 2026
@gardener-prow gardener-prow Bot merged commit c403565 into gardener:master Apr 24, 2026
13 checks passed
@gagan16k gagan16k deleted the cleanup branch April 24, 2026 08:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@r4mek r4mek r4mek left review comments

@takoverflow takoverflow takoverflow approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. cla: yes Indicates the PR's author has signed the cla-assistant.io CLA. kind/cleanup Something that is not needed anymore and can be cleaned up kind/enhancement Enhancement, improvement, extension lgtm Indicates that a PR is ready to be merged. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Hibernation fails due to existing node

3 participants

@gagan16k @r4mek @takoverflow