Appearance

Request lifecycle and admission

Separate identity, permission, mutation, and validation before persistence.

What you will understand

  • Trace an API write through authentication, authorization, and admission.
  • Choose between a webhook and an in-process validation policy.
  • Diagnose rejection without confusing it with controller failure.

Builds on: Release configuration and safe delivery

Mental model: a successful HTTP request is only the beginning

An API write passes several distinct boundaries. Authentication establishes the caller’s identity from a credential; authorization decides whether that identity may perform this verb on this resource. Admission then evaluates the proposed operation. Mutating admission can supply or change fields, while validating admission can reject the resulting object. Successful persistence makes desired state visible; it does not prove a scheduler, controller, or kubelet has fulfilled it. A Deployment can be admitted successfully while all of its Pods remain Pending.

This separation makes incident evidence useful. A forbidden response points toward authorization or a policy rejection whose message explains the denial. An invalid object points toward schema or validation. A webhook timeout names a dependency on the write path. A stored Deployment with unavailable replicas requires downstream investigation. Do not restart application Pods to fix an API admission outage: the rejected desired state may never have reached storage at all.

Identity, permission, admission, persistence

Admission accepts the proposed object; storage records desired state. No Pod readiness is implied by this response.

Flow viewStep 1 of 3
Establish identityThe API server authenticates the credential; a rejected credential never reaches object policy evaluation.. kubectl, Authenticated user credential, idle. Authentication, Who is making the request?, active. Authorization, May this user create?, idle. Admission, Is this proposal acceptable?, idle. API storage, Accepted desired state, idlekubectl: Authenticated user credential
kubectlAuthenticated user credentialContext
Authentication: Who is making the request?
AuthenticationWho is making the request?In focus
Authorization: May this user create?
AuthorizationMay this user create?Context
Admission: Is this proposal acceptable?
AdmissionIs this proposal acceptable?Context
API storage: Accepted desired state
API storageAccepted desired stateContext
kubectlAuthenticated user credentialContext
AuthenticationWho is making the request?In focus
AuthorizationMay this user create?Context
AdmissionIs this proposal acceptable?Context
API storageAccepted desired stateContext
Animation controls
Speed
0.5×
1×
2×
Explore a scenario
Normal behavior
Identity is allowed, object is denied
Establish identity

The API server authenticates the credential; a rejected credential never reaches object policy evaluation.

Read the complete visual sequence
  1. Establish identity

    The API server authenticates the credential; a rejected credential never reaches object policy evaluation.

  2. Authorize the operation

    Authorization evaluates caller, verb, resource and scope. It does not inspect all application policy requirements.

  3. Admit then persist

    Admission accepts the proposed object; storage records desired state. No Pod readiness is implied by this response.

Identity is allowed, object is denied

A content policy rejects a request after authorization.

  1. Allowed verb

    The caller can create Deployments in this namespace, so the authorization boundary passes.

  2. Policy rejection

    Admission finds the required team label missing and denies the request.

  3. No stored proposal

    No new desired-state object reaches storage; restarting a kubelet cannot repair this failure.

About this model

Decoding, conversion, defaulting and detailed validation stages are summarized; this is the security decision sequence for a write.

Walkthrough: mutation has consequences for validation

Consider an organization that requires every new Deployment to carry a team label. A mutating webhook could inject a default label; a validating rule then checks the final proposal. Mutation occurs before validating admission, but the complete API pipeline also includes decoding, defaulting, conversion and validation associated with the resource. The diagram deliberately summarizes these implementation details. Mutating webhook reinvocation is configurable; implementations must tolerate repeated calls and must not assume a durable ordering between independent mutators.

A webhook is an external HTTPS service with certificate trust, availability, latency and timeout concerns. Its admission review response permits, rejects, or mutates the request; it should not perform irreversible external work as though admission already committed. A ValidatingAdmissionPolicy instead evaluates CEL inside the API server. It is useful for expression-based object rules without an external round trip. A policy needs a binding to select enforcement scope and validation actions. Neither mechanism replaces RBAC: permission to request an operation and permission for the proposed contents are different decisions.

Mutating webhook and CEL validation

The required label now exists, so validation accepts the write. CEL evaluation does not require a separate webhook service.

Policy viewStep 1 of 3
Mutation is repeatableA selected mutating webhook supplies a missing label. It must tolerate reinvocation and avoid irreversible side effects.. Deployment proposal, team label absent, idle. Mutating webhook, May add team label, active. Validating policy, CEL checks final proposal, idle. Policy binding, Namespace scope + Deny, idle. Accepted object, Persist after checks, idleDeployment proposal: team label absent
Deployment proposalteam label absentContext
Mutating webhook: May add team label
Mutating webhookMay add team labelIn focus
Validating policy: CEL checks final proposal
Validating policyCEL checks final proposalContext
Policy binding: Namespace scope + Deny
Policy bindingNamespace scope + DenyContext
Accepted object: Persist after checks
Accepted objectPersist after checksContext
Deployment proposalteam label absentContext
Mutating webhookMay add team labelIn focus
Validating policyCEL checks final proposalContext
Policy bindingNamespace scope + DenyContext
Accepted objectPersist after checksContext
Animation controls
Speed
0.5×
1×
2×
Explore a scenario
Normal behavior
Webhook unavailable
Mutation is repeatable

A selected mutating webhook supplies a missing label. It must tolerate reinvocation and avoid irreversible side effects.

Read the complete visual sequence
  1. Mutation is repeatable

    A selected mutating webhook supplies a missing label. It must tolerate reinvocation and avoid irreversible side effects.

  2. A binding activates enforcement

    The binding selects namespaces and requests Deny action; the policy expression checks the resulting object.

  3. Validation observes the proposal

    The required label now exists, so validation accepts the write. CEL evaluation does not require a separate webhook service.

Webhook unavailable

A fail-closed mutator makes a dependency outage visible as a failed write.

  1. Call is attempted

    A matching request requires the mutating webhook before final validation.

  2. Timeout with Fail

    The external webhook cannot be reached and failurePolicy Fail rejects the operation.

  3. Storage unchanged

    The new object is not persisted; investigate webhook endpoints, TLS and API-server reachability.

About this model

One optional mutator is illustrated; real clusters may have multiple admission plugins and selected webhook reinvocation.

The following complete example is an optional cluster-scoped exercise. Save it as team-policy.yaml; it targets only Deployments in namespaces labeled training-policy=enabled. Label a separate disposable namespace before experimenting. The binding is what enables denial.

apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingAdmissionPolicy
metadata:
  name: kv-team-label
spec:
  failurePolicy: Fail
  matchConstraints:
    resourceRules:
      - apiGroups: [apps]
        apiVersions: [v1]
        operations: [CREATE, UPDATE]
        resources: [deployments]
  validations:
    - expression: "has(object.metadata.labels) && 'team' in object.metadata.labels"
      message: "metadata.labels.team is required"
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingAdmissionPolicyBinding
metadata:
  name: kv-team-label
spec:
  policyName: kv-team-label
  validationActions: [Deny]
  matchResources:
    namespaceSelector:
      matchLabels:
        training-policy: enabled
kubectl --context kind-kv-lab apply -f team-policy.yaml
# An authorization check does not execute this admission policy.
kubectl --context kind-kv-lab auth can-i create deployments -n kv-lab
# Clean up the optional cluster-scoped objects explicitly.
kubectl --context kind-kv-lab delete validatingadmissionpolicybinding kv-team-label
kubectl --context kind-kv-lab delete validatingadmissionpolicy kv-team-label

Practice: predict the boundary before inspecting the result

Use the m14 local exercise for safe API ownership and deletion experiments. The optional policy above extends that investigation; the downloadable lab does not install a highly available webhook service. Compare a server-side dry run with a real apply in a disposable selected namespace. A server-side dry run still traverses applicable admission. Matching webhooks must declare sideEffects: None or NoneOnDryRun and suppress any side effects for the dry run; an incompatible declaration makes the dry-run request fail. Expect the missing-label request to fail before a new Deployment is stored. Add the label and expect admission success, then independently inspect rollout status.

Record the verb, resource, namespace, response message and whether an object with the expected UID exists. These facts distinguish a rejected request from a controller that has not converged. Existing objects are not automatically scanned and deleted when a new validation policy is introduced; its matching operations govern subsequent requests.

Pitfalls: protect the write path

A fail-closed webhook can prevent matching writes when its service, DNS, certificate, or endpoints fail. A fail-open configuration trades availability against policy enforcement when calling the webhook fails; failurePolicy: Ignore does not override an explicit allowed: false response. Select narrowly, use short timeouts, monitor rejection and latency, and design an emergency recovery route that still respects the organization’s approval boundary. Excluding a webhook’s own dependencies can prevent circular recovery failures. Do not interpret a healthy webhook Pod as proof that the API server can reach its HTTPS endpoint or validate its certificate.

Check your understanding

RBAC allows a Deployment create, but admission rejects its image. Is authorization broken?

No. Authorization allowed the caller to attempt the operation. Admission evaluated the proposed object’s contents and rejected it before persistence. Fix the declared image or applicable policy; granting broader RBAC would not satisfy the image rule.

Next, trace the same request through managed fields and a watch. Keep the API response, persisted object, and eventual workload health as three separate pieces of evidence.

Go to the source

Reviewed 2026-10-27 · Kubernetes 1.37 · Related local exercise

Search the book

Search loads when you open it.