Namespace Finalizers Preventing Deletion in Clusters Step

[t-margheim]

Summary

Kubernetes namespace finalizers are blocking namespace deletion during the clusters step, even though the namespaces appear to be empty.

Affected Namespaces

• ptd-controller-system
• grafana

Context

• Step: clusters
• Operation: Delete namespace
• Blocker: Finalizers preventing deletion

Symptoms

1. Namespace deletion hangs or times out
2. Running kubectl get all -n ptd-controller-system shows: "No resources found in the ptd-controller-system namespace"
3. Despite appearing empty, the namespace cannot be deleted

Investigation Findings

• Namespaces appear empty when checking standard resources
• The issue may be related to namespace-scoped Custom Resource Definitions (CRDs)
• CRDs may have finalizers that aren't being properly cleaned up
• Resources may exist that aren't returned by kubectl get all

Notes

This may not be worth a comprehensive fix if:

• It only affects initial migration or rare teardown scenarios
• Manual intervention once is acceptable
• The issue won't recur during normal operations