Appearance

Autoscaling mechanics and feedback

Calculate a replica recommendation and follow the delays that shape it.

What you will understand

  • Calculate CPU utilization from usage and resource requests.
  • Explain HPA tolerance, missing metrics and stabilization.
  • Separate replica scaling from node capacity and application throughput.

Builds on: CPU throttling, OOM, and eviction

Mental model: the signal has a denominator

For a CPU resource-utilization target, the HPA compares measured usage with requested CPU. If each relevant container requests 100 millicores and consumes 80 millicores on average, its utilization is approximately eighty percent of request. A limit of 500 millicores does not change that utilization denominator. Raising the request while holding usage constant lowers utilization and changes the HPA signal, even if actual traffic and latency have not improved.

The basic recommendation is the ceiling of current replicas multiplied by current metric divided by desired metric. With three replicas, eighty-percent utilization and a fifty-percent target, the simple calculation recommends five replicas. This is a teaching calculation before tolerance, missing data, readiness handling, scaling policies and stabilization are applied. HPA is a periodic feedback controller, so the exact count at a particular instant can differ from the raw arithmetic.

Utilization divides usage by requests

The basic formula yields five replicas before tolerance, missing metrics, stabilization and bounds.

Chart viewStep 1 of 3
Compute utilizationEighty millicores usage divided by one hundred millicores request gives eighty-percent utilization.. Average usage 80m, Measured CPU per Pod, idle. Request 100m, Utilization denominator, idle. Current 80%, Target 50%, active. HPA recommendation, ceil(3 × 80 / 50) = 5, idle. Scale subresource, Desired replicas before limits, idleAverage usage 80m: Measured CPU per Pod
Average usage 80mMeasured CPU per PodContext
Request 100m: Utilization denominator
Request 100mUtilization denominatorContext
Current 80%: Target 50%
Current 80%Target 50%In focus
HPA recommendation: ceil(3 × 80 / 50) = 5
HPA recommendationceil(3 × 80 / 50) = 5Context
Scale subresource: Desired replicas before limits
Scale subresourceDesired replicas before limitsContext
Average usage 80mMeasured CPU per PodContext
Request 100mUtilization denominatorContext
Current 80%Target 50%In focus
HPA recommendationceil(3 × 80 / 50) = 5Context
Scale subresourceDesired replicas before limitsContext
Animation controls
Speed
0.5×
1×
2×
Explore a scenario
Normal behavior
Request missing
Compute utilization

Eighty millicores usage divided by one hundred millicores request gives eighty-percent utilization.

Read the complete visual sequence
  1. Compute utilization

    Eighty millicores usage divided by one hundred millicores request gives eighty-percent utilization.

  2. Compare with target

    The desired utilization is fifty percent and current replica count is three.

  3. Round the simple result

    The basic formula yields five replicas before tolerance, missing metrics, stabilization and bounds.

Request missing

CPU utilization cannot be inferred from usage alone.

  1. Metrics arrive

    The resource metrics pipeline reports CPU usage.

  2. Denominator unavailable

    A relevant container has no CPU request, making utilization undefined for that Pod.

  3. Inspect conditions

    Do not substitute zero; investigate HPA metric conditions and configure meaningful requests.

About this model

The arithmetic assumes identical complete ready-Pod observations; real HPA processing adds conservative adjustments and policies.

Walkthrough: recommendations pass through guardrails

The resource metrics API needs a working metrics provider, commonly Metrics Server, and containers need the relevant resource requests for utilization calculations. Missing metrics and not-yet-ready Pods affect the algorithm conservatively. A missing request can make utilization for that Pod undefined; an unknown metric is not zero demand. Inspect HPA conditions and events instead of assuming a replica count that stays constant means no scaling pressure exists.

Stabilization uses recent recommendations to reduce oscillation. Downscale stabilization can retain a larger recent recommendation while demand briefly falls; policies can separately bound how quickly replica counts change. minReplicas and maxReplicas bound the permitted range. These controls reduce churn but also delay reaction. Adding a replica then requires scheduling, image startup, readiness and endpoint propagation before useful capacity arrives. If all nodes are full, HPA can request more Pods while throughput stays unchanged.

Demand becomes useful capacity after delays

Image startup and readiness complete before the extra replicas become usable endpoints.

Flow viewStep 1 of 3
Constrain the recommendationHPA bounds recommendations and applies configured behavior. During downscale stabilization it can retain the highest recent recommendation, reducing oscillation while delaying scale-in. Scaling policies separately limit the permitted rate of change.. Metrics sample, Periodic observation, idle. HPA behavior, Bounds + stabilization, active. New Pods, Desired replica count, idle. Node capacity, Requests must fit, idle. Ready endpoints, Useful traffic capacity, idleMetrics sample: Periodic observation
Metrics samplePeriodic observationContext
HPA behavior: Bounds + stabilization
HPA behaviorBounds + stabilizationIn focus
New Pods: Desired replica count
New PodsDesired replica countContext
Node capacity: Requests must fit
Node capacityRequests must fitContext
Ready endpoints: Useful traffic capacity
Ready endpointsUseful traffic capacityContext
Metrics samplePeriodic observationContext
HPA behaviorBounds + stabilizationIn focus
New PodsDesired replica countContext
Node capacityRequests must fitContext
Ready endpointsUseful traffic capacityContext
Animation controls
Speed
0.5×
1×
2×
Explore a scenario
Normal behavior
No schedulable headroom
Constrain the recommendation

HPA bounds recommendations and applies configured behavior. During downscale stabilization it can retain the highest recent recommendation, reducing oscillation while delaying scale-in. Scaling policies separately limit the permitted rate of change.

Read the complete visual sequence
  1. Constrain the recommendation

    HPA bounds recommendations and applies configured behavior. During downscale stabilization it can retain the highest recent recommendation, reducing oscillation while delaying scale-in. Scaling policies separately limit the permitted rate of change.

  2. Create intended replicas

    The workload controller creates Pods; those Pods still need feasible nodes.

  3. Wait for useful capacity

    Image startup and readiness complete before the extra replicas become usable endpoints.

No schedulable headroom

A higher desired replica count can leave useful capacity unchanged.

  1. HPA asks for more

    Metrics justify a larger target within maxReplicas.

  2. Pods remain Pending

    Nodes cannot fit the additional requests or required placement constraints.

  3. Demand stays queued

    The intended extra Pods provide no ready endpoints yet; scaling nodes or changing constraints is a separate concern.

About this model

Node autoscaling is a separate optional mechanism; its timing and supported capacity depend on the provider.

This HPA is an optional extension to a Deployment named m17-web, which you must create with CPU requests first. It does not install a metrics provider or generate realistic load. The local OOM exercise remains independent so it works without that add-on.

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: m17-web
  namespace: kv-lab
  labels:
    lab: m17
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: m17-web
  minReplicas: 2
  maxReplicas: 8
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 50 # Percent of requested CPU, not CPU limit.
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300
kubectl --context kind-kv-lab get apiservice v1beta1.metrics.k8s.io
kubectl --context kind-kv-lab -n kv-lab get hpa
kubectl --context kind-kv-lab -n kv-lab describe hpa m17-web
kubectl --context kind-kv-lab -n kv-lab get pods -o wide
# top requires a functioning resource metrics pipeline.
kubectl --context kind-kv-lab -n kv-lab top pods

Practice: predict a recommendation before changing configuration

Use the m17 local exercise for resource-limit evidence, then calculate a hypothetical HPA recommendation on paper. Start with four replicas, a request of 200 millicores, average usage of 100 millicores and a target of fifty percent. The signal is on target. Now halve the request without changing traffic or usage. The utilization becomes one hundred percent and the simple recommendation doubles. This reveals why right-sizing and autoscaler settings must be reviewed together.

If you install a metrics provider and run the optional experiment, record timestamps for demand change, metric observation, replica recommendation, Pod creation and readiness. These are observations for your setup, not portable latency guarantees. Check CPU usage distribution as well as the average; one hot shard can suffer while average utilization looks acceptable. For queues, consider an appropriate external or custom metric with a defined scaling relationship instead of assuming CPU captures all workload demand.

Pitfalls: more replicas can amplify dependencies

Horizontal scaling can multiply database connections, cache misses or outbound API requests. A saturated database may worsen when more workers arrive. CPU limits can cap the measured usage even as request latency grows, masking unsatisfied demand. Missing metrics, maxReplicas and unavailable node capacity each create different ceilings. Do not have GitOps continuously reset spec.replicas while HPA manages it; establish field ownership so two reconciliation loops do not fight over the scale target.

With multiple configured metrics, the HPA evaluates recommendations according to its documented algorithm; errors can constrain downscaling rather than silently becoming zero load. Inspect each metric status when one healthy metric appears to disagree with the final replica recommendation.

Check your understanding

CPU usage is unchanged, but requests double. What happens to utilization-based HPA input?

The utilization percentage is roughly halved because request is its denominator. The raw scale recommendation can fall even though demand did not. Actual scaling still depends on tolerance, available metrics, stabilization and configured policies.

Next, combine the feedback loop with cluster capacity. A replica recommendation is an intention, while schedulable resources and application bottlenecks determine useful capacity.

Go to the source

Reviewed 2026-10-27 · Kubernetes 1.37 · Related local exercise

Search the book

Search loads when you open it.