Appearance

GitOps and drift reconciliation

Make declared ownership and live recovery agree.

What you will understand

  • Trace Git revision through render, diff, sync and health.
  • Explain Argo CD self-heal and prune as separate choices.
  • Choose an explicit field-ownership takeover for the local drift exercise.

Builds on: Watches and server-side apply, Ownership and failure recovery

Mental model: the source of intent is another dependency

GitOps stores reviewed desired configuration in a source repository and uses a controller to compare that source with live cluster state. Argo CD renders the selected revision, compares desired resources with the destination, and applies changes according to application policy. The controller also evaluates resource health. A successful sync says something about configuration convergence; a healthy application requires workload and dependency evidence. A Git commit is not an atomic transaction across every resource in the cluster.

Drift is a difference between the intended source and observed live state. It can come from an emergency edit, defaulting, an operator, an autoscaler or an unauthorized change. Treating all differences as malicious is as unhelpful as ignoring all differences. Define which actor owns each field. For an HPA-managed workload, continuously forcing the Git replica count can create a fight between controllers rather than a more reliable deployment process.

Revision, render, diff, sync, health

A completed sync does not prove ready endpoints or successful application requests.

Flow viewStep 1 of 3
Render a known revisionThe delivery controller obtains source and constructs desired Kubernetes resources.. Reviewed Git revision, Desired source, idle. Render manifests, Helm / Kustomize / YAML, active. Compare live state, Detect declared drift, idle. Sync operation, Write selected resources, idle. Workload health, Separate readiness evidence, idleReviewed Git revision: Desired source
Reviewed Git revisionDesired sourceContext
Render manifests: Helm / Kustomize / YAML
Render manifestsHelm / Kustomize / YAMLIn focus
Compare live state: Detect declared drift
Compare live stateDetect declared driftContext
Sync operation: Write selected resources
Sync operationWrite selected resourcesContext
Workload health: Separate readiness evidence
Workload healthSeparate readiness evidenceContext
Reviewed Git revisionDesired sourceContext
Render manifestsHelm / Kustomize / YAMLIn focus
Compare live stateDetect declared driftContext
Sync operationWrite selected resourcesContext
Workload healthSeparate readiness evidenceContext
Animation controls
Speed
0.5×
1×
2×
Explore a scenario
Normal behavior
Source cannot render
Render a known revision

The delivery controller obtains source and constructs desired Kubernetes resources.

Read the complete visual sequence
  1. Render a known revision

    The delivery controller obtains source and constructs desired Kubernetes resources.

  2. Compare and act by policy

    A difference is evaluated under configured sync, self-heal and prune behavior rather than assumed to trigger every action.

  3. Evaluate health separately

    A completed sync does not prove ready endpoints or successful application requests.

Source cannot render

A source or rendering failure prevents new desired manifests reaching the API.

  1. Revision is selected

    The controller knows which source revision it intends to deploy.

  2. Render fails

    A missing value or invalid source prevents a valid desired manifest set.

  3. Live resources may keep serving

    Existing workloads can remain healthy while delivery is blocked; diagnose source rendering rather than restarting Pods.

About this model

Argo CD is an optional add-on; the mandatory local exercise models the ownership loop with kubectl.

Walkthrough: automatic sync has independent controls

Argo CD automated synchronization, self-healing and pruning are related but distinct settings. Self-healing controls automatic correction of certain live-only deviations under automated sync. Pruning allows removal of tracked resources that are no longer in desired source, with additional safeguards and options relevant to empty applications and destructive changes. Enabling one behavior should not be assumed to enable every other one. Inspect the installed Argo CD version and application policy before predicting whether a manual edit will be reversed.

SSA changes the coordination behavior at the API field boundary. A controller applying a field already owned with a different value can conflict. --force-conflicts is an explicit takeover, not a harmless way to silence a warning. The local m19 exercise first declares a ConfigMap using kv-gitops, introduces drift with an ordinary patch, and then intentionally forces the declarative manager to recover its field. That force is part of the teaching scenario’s approved ownership decision, not a recommended blanket Argo CD policy.

Drift repair includes an ownership decision

The lab intentionally uses force-conflicts for this disposable field, restoring the source value and ownership contract.

Ownership viewStep 1 of 3
Declare the source valuekv-gitops applies data.release as git-declared.. kv-gitops manager, Declares release field, active. Manual patch, Changes live release, idle. ConfigMap m19, Observed data.release, idle. managedFields, Changed-field ownership, idle. Explicit takeover, Force only this lab field, idlekv-gitops manager: Declares release field
kv-gitops managerDeclares release fieldIn focus
Manual patch: Changes live release
Manual patchChanges live releaseContext
ConfigMap m19: Observed data.release
ConfigMap m19Observed data.releaseContext
managedFields: Changed-field ownership
managedFieldsChanged-field ownershipContext
Explicit takeover: Force only this lab field
Explicit takeoverForce only this lab fieldContext
kv-gitops managerDeclares release fieldIn focus
Manual patchChanges live releaseContext
ConfigMap m19Observed data.releaseContext
managedFieldsChanged-field ownershipContext
Explicit takeoverForce only this lab fieldContext
Animation controls
Speed
0.5×
1×
2×
Explore a scenario
Normal behavior
Controllers fight over the same field
Declare the source value

kv-gitops applies data.release as git-declared.

Read the complete visual sequence
  1. Declare the source value

    kv-gitops applies data.release as git-declared.

  2. Observe manual drift

    A patch changes the field and may transfer ownership; inspect both live value and managedFields.

  3. Restore with explicit authority

    The lab intentionally uses force-conflicts for this disposable field, restoring the source value and ownership contract.

Controllers fight over the same field

Repeated takeovers can produce oscillation instead of convergence.

  1. Two managers declare replicas

    A delivery controller and autoscaler both insist on different values.

  2. Each recovery undoes the other

    Broad forcing repeatedly changes live state and ownership.

  3. Split responsibilities

    Remove the field from one declaration or define a supported collaboration policy rather than forcing indefinitely.

About this model

The drift exercise uses an ordinary patch; conflicting SSA by a second manager is a separate API operation.

This ConfigMap is the entire desired resource in the local exercise. The lab models reconciliation with explicit kubectl commands; it does not silently install an Argo CD service or connect to a remote repository.

apiVersion: v1
kind: ConfigMap
metadata:
  name: m19
  namespace: kv-lab
  labels:
    lab: m19
data:
  release: git-declared
# Run from the downloaded m19 directory.
kubectl --context kind-kv-lab -n kv-lab apply --server-side --field-manager=kv-gitops -f resources.yaml
kubectl --context kind-kv-lab -n kv-lab patch configmap m19 --type=merge -p '{"data":{"release":"manual-drift"}}'
kubectl --context kind-kv-lab -n kv-lab get configmap m19 -o yaml --show-managed-fields
# Intentional takeover of this disposable lab field only.
kubectl --context kind-kv-lab -n kv-lab apply --server-side --field-manager=kv-gitops --force-conflicts -f resources.yaml

Practice: observe state and ownership before restoring it

Run the m19 local exercise. Expect release to change to manual-drift, then return to git-declared after explicit declarative recovery. Inspect managedFields before and after the patch because an ordinary update can transfer ownership of changed fields. A separate manager performing a conflicting SSA declaration has different behavior from a patch that already changed the field. These are distinct operations, not interchangeable ways to demonstrate a conflict.

For an optional real Argo CD extension, read ARGOCD-OPTIONAL.md from the download. First choose a supported release and check its upstream-tested Kubernetes versions; a pinned install manifest alone does not establish compatibility with this course’s Kubernetes 1.37 cluster. Keep source credentials, destination permissions, project restrictions and application scope explicit. Test rendering before sync and use an immutable reviewed revision when reproducibility matters. Record both the desired revision and observed health. Repository access failure, manifest-render failure, admission rejection and an unhealthy deployed workload need different diagnosis paths even though they can all prevent a delivery from completing.

Pitfalls: emergency changes need a return path

An emergency live change may be reverted by active self-healing unless the source or reconciliation policy is updated deliberately. Pause the relevant automation only through the team’s incident procedure, preserve the exact live change, and incorporate or revert it in source afterward. Broad ignore-differences rules can hide meaningful drift; broad pruning permissions can delete resources that were incorrectly tracked. Restoring an old cluster snapshot while GitOps points at a newer revision can immediately change the restored state again.

Choose the application boundary to match ownership. GitOps can own the WebApp custom resource while an operator owns generated ConfigMaps and Deployments. Having both controllers declare the same child fields produces avoidable conflicts. This boundary is exercised in the final platform capstone.

Check your understanding

The app is Synced, but no ready endpoints exist. Has GitOps proved service availability?

No. Sync compares and applies desired configuration; readiness and actual requests are later evidence. Inspect application health, Pod conditions, EndpointSlices and dependencies before claiming the service is available.

Next, define who is allowed to supply that desired configuration and which artifacts it may reference. Git history alone is not a complete tenancy or software-trust boundary.

Go to the source

Reviewed 2026-10-27 · Kubernetes 1.37 · Related local exercise

Search the book

Search loads when you open it.