Kubernetes v1.37: Scaling Workloads to Zero with HorizontalPodAutoscaler

CPU and memory metrics are derived from the resource usage of currently running Pods. The HorizontalPodAutoscaler (HPA) reads these values through the resource metrics API, which requires at least one active replica to expose the data. When the replica count reaches 0, the workload no longer has any Pods, so the resource metrics endpoint disappears. Consequently, the HPA loses its signal and cannot determine whether additional capacity is needed, making it impossible to trigger a scale‑up from a zero state.

Object and external metrics solve this limitation because they are independent of the workload’s Pods. An external metric such as a queue length, backlog size, or custom business indicator exists in a monitoring system (e.g., Prometheus) regardless of whether any consumer Pods are running. The HPA can continuously query the External Metrics API, evaluate the metric, and decide to create new Pods even when the deployment is at zero replicas.

• Metric availability at zero: External sources (queues, databases, message brokers) continue to emit values; resource metrics cease.
• Deterministic scaling logic: The HPA can map a concrete threshold (e.g., 30 queued tasks) to a replica count, as shown in the Kubernetes v1.37 example that scales a queue-worker Deployment from 0 to 10 replicas.
• Separation of concerns: Workload owners can tune scaling behavior without modifying the application to expose internal resource usage when idle.
• Cold‑start handling: The HPA’s downscale stabilization window (default five minutes) prevents rapid oscillation, while the external metric remains a reliable wake‑up signal.

In practice, an engineer would configure a metrics adapter—such as the Prometheus Adapter—to expose a series like queue_consumer_lag through the External Metrics API. After verifying the metric endpoint (e.g., kubectl get --raw '/apis/external.metrics.k8s.io/v1beta1/...'), the HPA definition includes minReplicas: 0 and an External metric target. When the queue is empty, the HPA reduces the deployment to zero; when tasks accumulate, the external metric remains available, prompting the HPA to instantiate the required number of Pods.

This design eliminates the need for a separate buffering layer for request‑driven services, while still achieving cost savings for workloads that can tolerate cold starts, such as batch processors or queue consumers.

As of Kubernetes v1.37, the HorizontalPodAutoscaler (HPA) natively supports scaling workloads to zero replicas. This functionality relies on external or object metrics, as resource-based metrics (CPU/Memory) are unavailable when no pods exist to report them. Scaling to zero is ideal for event-driven architectures, such as queue consumers, where resources should be reclaimed when no work is pending.

To implement this, you must expose an external metric via a metrics adapter, such as the Prometheus Adapter, which translates your monitoring backend data into the External Metrics API.

Prerequisites for Scaling to Zero

• Metrics Pipeline: Configure your adapter with externalRules to map your metric (e.g., queue_consumer_lag) to the API.
• Connectivity Check: Verify that the HPA controller can retrieve the metric before deployment. Run the following command:

  kubectl get --raw '/apis/external.metrics.k8s.io/v1beta1/namespaces/default/queue_consumer_lag?labelSelector=name%3Dworker_tasks'

• Version Compatibility: Ensure both kube-apiserver and kube-controller-manager are running v1.37 or later.

Once verified, configure your HPA manifest by setting minReplicas: 0. The HPA uses the ScaledToZero condition to distinguish between an automatic scale-down and a manual pause. When the HPA scales the workload to zero, it sets ScaledToZero=True, allowing it to continue monitoring external metrics to trigger a future scale-up event. If the metric is missing, the HPA will report ScalingActive=False.

Configuration Example

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: queue-worker
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: queue-worker
  minReplicas: 0
  maxReplicas: 10
  metrics:
  - type: External
    external:
      metric:
        name: queue_consumer_lag
        selector:
          matchLabels:
            name: worker_tasks
      target:
        type: Value
        value: "30"

Always ensure the deployment begins with at least one replica. Note that the default downscale stabilization window is five minutes; modify spec.behavior.scaleDown if your workload requires faster or slower transitions.

When implementing horizontal autoscaling down to zero replicas, maintaining a clear distinction between an autonomous scale-down and a manual administrative pause is critical for cluster stability. Kubernetes resolves this ambiguity by utilizing the ScaledToZero status condition within the HorizontalPodAutoscaler (HPA) resource. This condition serves as the internal telemetry required by the controller to manage the ownership of the workload state.

The reconciliation logic functions through the following state transitions:

• Automatic Scale-Down: When the HPA controller evaluates metrics (object or external) and determines that scaling to zero is appropriate, it executes the scale-down and sets the ScaledToZero condition to True. This record signals to subsequent reconciliation loops that the controller retains ownership of the workload, allowing it to continue monitoring for metric spikes that necessitate a scale-up.
• Manual Pause: If an operator manually scales a Deployment to zero replicas, the HPA controller observes the change but does not set the ScaledToZero condition to True. Consequently, the HPA views this as a manual pause. Because the controller does not own the zero-replica state in this context, it will not initiate an automatic scale-up, effectively preserving the administrative intent.
• Restoration of Capacity: Upon detecting that external or object metrics warrant a scale-up, the controller increments the replica count and updates the ScaledToZero condition to False with the reason NotScaledToZero.

For engineers managing these workloads, visibility into this state is maintained via the standard Kubernetes API. Running kubectl describe hpa <workload-name> allows for the inspection of these conditions. If the HPA reports ScalingActive=False, it typically indicates a failure in the metrics pipeline—such as an unavailable metrics adapter—rather than a state conflict. In such instances, the HPA remains unable to reconcile the state until the metric source is restored or the workload is manually scaled to at least one replica.

Scaling workloads to zero replicas via HorizontalPodAutoscaler (HPA) introduces specific architectural trade-offs, primarily regarding latency and traffic management. While this functionality optimizes resource usage by removing idle pods—effectively eliminating costs for reserved CPU or GPU overhead—it necessitates a careful approach to workload design.

Latency and Buffering Considerations

The primary trade-off when scaling to zero is the inherent cold-start latency. Upon receiving a metric signal to scale up, the cluster must schedule a pod, pull the container image, and initialize the application before it can process requests. Consequently, this model is most effective for asynchronous tasks, such as queue consumers or batch processors, where workloads can reside in a durable queue during the initialization period. Because Kubernetes Services do not natively buffer incoming traffic when no pods are available, HTTP-driven workloads require an external buffering layer (such as a load balancer or message broker) to prevent request drops during the spin-up phase.

Metrics and Operational Ambiguity

To scale from zero, HPAs must rely on object or external metrics rather than resource metrics like CPU or memory. Since resource metrics depend on active pods, they disappear when a replica count hits zero, leaving the controller without a signal to initiate a scale-up. External metrics, such as queue length, persist regardless of the pod state. Engineers must ensure the following to avoid configuration errors:

• Metrics Adapter Reliability: Verify that the metrics adapter (e.g., Prometheus Adapter) successfully exposes the series through the External Metrics API before configuring the HPA.
• State Differentiation: Use the ScaledToZero status condition to distinguish between a controller-driven scale-down and a manual pause. A workload at zero without this condition will not scale up automatically.

Version-Skewed Upgrades

During control plane upgrades, ensure both the kube-apiserver and kube-controller-manager support and have the feature enabled. A version-skewed control plane where one component lacks the feature may interpret a replica count of zero as a manual pause, preventing automatic recovery. Before downgrading or disabling the feature gate, operators must proactively set minReplicas to at least one and scale all zero-replica workloads to prevent service interruption.