[WIP] refactor: split termination controller

[jmdeal]

Fixes #N/A

Description
This PR splits the node termination controller into three. Each controller has an associated finalizer, with the final controller (Instance Termination) being unable to reconciile until the drain and volume finalizers have been removed.

Controller	Finalizer
Drain	karpenter.sh/drain-protection
Volume Detachment	karpenter.sh/volume-protection
Instance Termination	karpenter.sh/termination

This change was motivated by the increased complexity of the termination controller once additional status conditions were added for drain and volume detachment monitoring. Alternatively, the subreconciler pattern (a la the NodeClaim lifecycle controller) could have been used. However, this approach has some additional observability benefits thanks to the per-controller metrics, and is subjectively easier to test and maintain.

How was this change tested?
make test

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: jmdeal
Once this PR has been reviewed and has the lgtm label, please assign ellistarn for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[coveralls]

Pull Request Test Coverage Report for Build 12252575600

Details

• 459 of 665 (69.02%) changed or added relevant lines in 19 files are covered.
• 17 unchanged lines in 4 files lost coverage.
• Overall coverage decreased (-0.6%) to 80.12%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
pkg/controllers/controllers.go	0	4	0.0%
pkg/utils/node/node.go	9	15	60.0%
pkg/controllers/nodeclaim/hydration/controller.go	17	27	62.96%
pkg/utils/nodeclaim/nodeclaim.go	15	25	60.0%
pkg/utils/nodeclaim/types.go	18	28	64.29%
pkg/controllers/node/hydration/controller.go	24	35	68.57%
pkg/utils/node/types.go	27	42	64.29%
pkg/controllers/node/termination/reconcile/reconcile.go	55	71	77.46%
pkg/controllers/node/termination/instancetermination/controller.go	47	76	61.84%
pkg/controllers/node/termination/volumedetachment/controller.go	62	104	59.62%

Files with Coverage Reduction	New Missed Lines	%
pkg/test/cachesyncingclient.go	2	82.29%
pkg/controllers/disruption/consolidation.go	4	88.55%
pkg/utils/termination/termination.go	4	87.18%
pkg/controllers/provisioning/scheduling/preferences.go	7	86.52%

Totals
Change from base Build 12250796523:	-0.6%
Covered Lines:	9052
Relevant Lines:	11298

---

💛 - Coveralls

[jmdeal]

/hold

I'm separating out the observability changes in this PR so I can get the essential feature change in and prioritize other work. I'll come back to this refactor, the current sticking point blocking this is the rollback story when adding additional finalizers.

[github-actions]

This PR has been inactive for 14 days. StaleBot will close this stale PR after 14 more days of inactivity.