Appearance

Ownership and failure recovery

Repair missing children while refusing unsafe adoption and cleanup.

What you will understand

  • Verify child owner references by parent UID.
  • Explain repair after deletion and refusal to adopt unrelated objects.
  • Choose finalizers only for cleanup obligations not handled by garbage collection.

Builds on: Leases and leader election, Build an idempotent operator

Mental model: ownership must survive name reuse

A controller manages a relationship between a parent intention and dependent resources. In this lab, the generated Deployment and ConfigMap carry ownerReferences naming the WebApp’s UID and identifying it as controller owner. The UID matters because a WebApp can be deleted and recreated with the same name. The replacement is a new parent identity. A controller that adopts any child with the expected name could overwrite a resource belonging to someone else or a previous lifecycle.

The Python operator checks existing child ownership and refuses unrelated objects. This is a useful safety property even in a tiny controller. Labels such as app=m16 help find resources, but labels are mutable selection metadata, not proof of ownership. If an operator reports an adoption refusal, inspect both the current parent UID and the child’s complete ownerReferences before considering any repair. Do not remove the check simply to make a demonstration proceed.

Stable names do not grant adoption

A matching owner permits normal reconciliation; names and labels alone do not.

Ownership viewStep 1 of 3
Read parent identityThe operator uses the current WebApp UID when deriving child owner references.. Current WebApp UID A, Declared desired state, active. Operator, Checks existing controller owner, idle. Existing child, Expected name, idle. Different UID B, Unrelated lifecycle, idleCurrent WebApp UID A: Declared desired state
Current WebApp UID ADeclared desired stateIn focus
Operator: Checks existing controller owner
OperatorChecks existing controller ownerContext
Existing child: Expected name
Existing childExpected nameContext
Different UID B: Unrelated lifecycle
Different UID BUnrelated lifecycleContext
Current WebApp UID ADeclared desired stateIn focus
OperatorChecks existing controller ownerContext
Existing childExpected nameContext
Different UID BUnrelated lifecycleContext
Animation controls
Speed
0.5×
1×
2×
Explore a scenario
Normal behavior
Unrelated object blocks adoption
Read parent identity

The operator uses the current WebApp UID when deriving child owner references.

Read the complete visual sequence
  1. Read parent identity

    The operator uses the current WebApp UID when deriving child owner references.

  2. Inspect existing child

    A child with the expected name must already have the matching controller owner before it is managed.

  3. Apply only a valid relationship

    A matching owner permits normal reconciliation; names and labels alone do not.

Unrelated object blocks adoption

An existing child belongs to another lifecycle.

  1. Same name is found

    The expected child name is occupied by a resource whose owner UID is B.

  2. Identity mismatch is detected

    The current WebApp UID is A, so the operator refuses to apply over that child.

  3. Investigate before takeover

    Resolve the intended ownership explicitly; removing the safety check risks overwriting another resource.

About this model

The teaching controller intentionally refuses adoption; production adoption policies require additional explicit design.

Walkthrough: repair and deletion are different reconciliations

When the ConfigMap is accidentally deleted while the WebApp still exists, the next pass computes the same desired child and recreates it. Its new UID proves replacement; the parent’s UID remains unchanged. When a Deployment is deleted, the next pass recreates that child too, but application availability may be disrupted while Pods return. Reconciliation provides a route back toward desired state, not instantaneous immunity from destructive changes.

Deleting the parent is different. The teaching operator skips resources with deletionTimestamp and relies on Kubernetes garbage collection for its Kubernetes children. No external system is provisioned, so there is no external cleanup finalizer in this example. A production operator managing a cloud database might need a finalizer to coordinate retention or deletion before the parent disappears. Such cleanup must be idempotent and must handle the external resource already being absent. A finalizer cannot turn a nontransactional pair of systems into one atomic transaction.

Repair a child, then delete the parent

When the parent is removed, garbage collection removes owned children; the operator does not recreate a deleted parent.

Lifecycle viewStep 1 of 3
Accidental child deletionWith the parent still active, a missing child is drift from desired state.. WebApp, Current desired intent, idle. Operator, Recreate missing child, idle. ConfigMap / Deployment, Owned Kubernetes children, active. Garbage collector, Removes eligible dependents, idleWebApp: Current desired intent
WebAppCurrent desired intentContext
Operator: Recreate missing child
OperatorRecreate missing childContext
ConfigMap / Deployment: Owned Kubernetes children
ConfigMap / DeploymentOwned Kubernetes childrenIn focus
Garbage collector: Removes eligible dependents
Garbage collectorRemoves eligible dependentsContext
WebAppCurrent desired intentContext
OperatorRecreate missing childContext
ConfigMap / DeploymentOwned Kubernetes childrenIn focus
Garbage collectorRemoves eligible dependentsContext
Animation controls
Speed
0.5×
1×
2×
Explore a scenario
Normal behavior
External cleanup is required
Accidental child deletion

With the parent still active, a missing child is drift from desired state.

Read the complete visual sequence
  1. Accidental child deletion

    With the parent still active, a missing child is drift from desired state.

  2. Reconcile repairs drift

    The next operator pass creates the same desired child name with a new child UID and the same parent UID.

  3. Parent deletion changes intent

    When the parent is removed, garbage collection removes owned children; the operator does not recreate a deleted parent.

External cleanup is required

Kubernetes garbage collection cannot perform a database provider API workflow.

  1. External resource exists

    A hypothetical production extension manages infrastructure outside the API owner graph.

  2. No finalizer coordination

    Deleting the parent without an implemented cleanup barrier can abandon that external resource.

  3. Implement the obligation

    Use an idempotent finalizer protocol with external identity checks; a string alone supplies no cleanup code.

About this model

The actual lab creates only Kubernetes children and needs no external-resource finalizer.

This manifest illustrates the parent relationship through the real WebApp API. Owner references on the children are generated by the controller; do not copy a UID from documentation into a hand-written dependent manifest.

apiVersion: labs.kv.example/v1alpha1
kind: WebApp
metadata:
  name: m16
  namespace: kv-lab
  labels:
    lab: m16
spec:
  replicas: 1
  message: repaired-by-reconciliation
kubectl --context kind-kv-lab -n kv-lab get webapp m16 -o jsonpath='{.metadata.uid}{"\n"}'
kubectl --context kind-kv-lab -n kv-lab get configmap m16 -o jsonpath='{.metadata.ownerReferences}{"\n"}'
# Delete only this generated, disposable ConfigMap.
kubectl --context kind-kv-lab -n kv-lab delete configmap m16
python3 operator.py --once
kubectl --context kind-kv-lab -n kv-lab get configmap m16 -o yaml

Practice: collect evidence at each boundary

Use the m16 local exercise. Record the parent UID, child UID and child owner UID before failure. Delete the generated ConfigMap, run one reconcile pass, and record them again. Expect the child UID to change, its owner UID to match the unchanged WebApp UID, and the page content to match current spec.message. This proves repair toward current intention rather than restoration of a particular old object identity.

Parent deletion is demonstrated again in the m20 platform capstone. Before trying it manually, save the desired manifest and ensure the objects contain no data you need to preserve. Delete the WebApp, wait for dependent removal, and inspect events if deletion stalls. Background garbage collection is asynchronous. A child can remain briefly visible after the parent’s delete request returns. Do not immediately declare ownership broken because two terminal reads caught different points in the deletion sequence.

Pitfalls: cleanup policy belongs in the API design

A database operator needs a documented retention decision: should deleting the custom resource preserve data, require a separate approval, or remove storage? ownerReferences alone cannot express every data policy. A broken finalizer controller can block deletion, while a missing finalizer can leak an external resource. Repeated cleanup attempts must not delete a newly recreated external object with the old name; carry and verify external identity as well as Kubernetes UID. Conditions should identify the failed cleanup step without placing credentials or sensitive data in status messages.

For recovery, pause destructive automation when necessary and understand what will reconcile next. Restoring an old custom resource into a live environment can cause the operator to enact old intent immediately. A backup procedure must include a plan for controller startup and ownership reconciliation, not just an API import.

Check your understanding

A child has the expected name and labels, but its owner UID differs. Should the operator overwrite it?

No. Those facts do not establish that it belongs to the current parent. The lab operator refuses adoption so an operator or user must investigate identity and choose an explicit recovery action rather than silently seizing the child.

Next, relate reconciliation pressure to resource pressure. A reliable loop still depends on enough CPU, memory, API capacity and storage to finish its work.

Go to the source

Reviewed 2026-10-27 · Kubernetes 1.37 · Related local exercise

Search the book

Search loads when you open it.