Kubernetes v1.37 Beta Feature: Monitoring PVC Unused Condition for Cost‑Effective Storage Management

In large‑scale Kubernetes clusters a common operational pattern is to create a PersistentVolumeClaim (PVC), launch a pod that consumes it, and later delete the pod without deleting the PVC. Because PVCs are not garbage‑collected automatically—Kubernetes protects them from accidental data loss—these “orphaned” claims persist and continue to reserve space on the underlying storage class.

Orphaned PVCs have two concrete side effects:

• They consume physical capacity on block or file storage, reducing the headroom for new workloads and potentially triggering volume provisioning failures.
• When the storage backend is a cloud‑provider service (e.g., AWS EBS, Azure Disk, GCP PD), each allocated volume incurs a per‑GB‑per‑hour charge, inflating the operational bill proportionally to the number of forgotten claims.

Prior to Kubernetes v1.37, determining whether a PVC was still in use required a manual cross‑reference of pods, PersistentVolumes, and PVC objects across all namespaces. Teams typically built custom scripts that listed pods, extracted their volume mounts, and matched them against PVC names—a process that is error‑prone, hard to scale, and fragile when pod lifecycles involve terminated or pending states.

The PersistentVolumeClaimUnusedSinceTime feature, promoted to Beta in v1.37, eliminates this gap. The PVC protection controller now adds an Unused condition to every PVC:

• Unused=True (reason NoPodsUsingPVC) indicates that no non‑terminal pod references the claim.
• Unused=False (reason PodUsingPVC) means at least one running or pending pod holds the claim.
• The condition includes lastTransitionTime, enabling queries such as “PVC idle for >30 days”.

Native visibility offers deterministic semantics:

• Terminated pods (Succeeded/Failed) are ignored, so batch jobs do not block reclamation.
• Pending pods still count, reflecting the intent to use the volume.
• Multiple pods sharing a PVC are handled correctly; the condition flips only after the last non‑terminal pod disappears.

Example query to list PVCs idle over 30 days:

kubectl get pvc -A -o json |
jq -r '
  .items[]
  | select(.status.conditions[]? |
      select(.type=="Unused" and .status=="True"))
  | select(
      (.status.conditions[] | select(.type=="Unused") | .lastTransitionTime) as $t
      | (now - ($t | fromdateiso8601)) > (30*86400)
    )
  | "\(.metadata.namespace)/\(.metadata.name) unused since \(.status.conditions[] | select(.type=="Unused") | .lastTransitionTime)"
'

By exposing usage state directly in the PVC API, operators can rely on standard kubectl or controller logic instead of bespoke scripts, reducing maintenance overhead and improving reliability of storage cost controls.

The PVC protection controller is a built‑in control loop that runs in the kube‑controller‑manager. In v1.37 it extends its existing pod watch to maintain a new Unused condition on every PersistentVolumeClaim (PVC). The controller registers an informer on v1.Pod objects, receives Add, Update, and Delete events, and reconciles the set of non‑terminal pods that reference each PVC.

When a pod event arrives, the controller extracts the volumeClaimTemplates and volumes[].persistentVolumeClaim.claimName fields. For each claim it updates an internal reference count:

func (c *pvcProtectionController) onPod(obj interface{}) {
    pod := obj.(*v1.Pod)
    for _, vol := range pod.Spec.Volumes {
        if vol.PersistentVolumeClaim != nil {
            claimKey := fmt.Sprintf("%s/%s", pod.Namespace, vol.PersistentVolumeClaim.ClaimName)
            if isNonTerminated(pod) {
                c.refCount[claimKey]++
            } else {
                c.refCount[claimKey]--
            }
        }
    }
    c.updatePVCCondition(claimKey)
}

The helper isNonTerminated returns false for pods whose .status.phase is Succeeded or Failed. Consequently, terminated batch jobs do not keep a PVC marked as in‑use, satisfying the “Terminated pods don’t count” rule.

Pending pods are treated as active users. Even if a pod remains unschedulable, the controller increments the reference count, because the intent to mount the volume is already expressed in the pod spec. This matches the design goal that “Pending pods do count”.

• Running pod: reference count + 1, Unused=False.
• Pending pod: reference count + 1, Unused=False.
• Terminated pod (Succeeded/Failed): reference count – 1; when the count reaches zero the condition flips to Unused=True.

When the reference count transitions from non‑zero to zero, the controller patches the PVC status with a new condition object:

{
    "type": "Unused",
    "status": "True",
    "reason": "NoPodsUsingPVC",
    "message": "No pods are currently referencing this PVC",
    "lastTransitionTime": "current RFC3339 timestamp"
}

The lastTransitionTime field is populated from metav1.Now() at the moment of the state change, providing an exact idle timestamp without additional instrumentation. If later a pod is created that references the PVC, the controller emits a Unused=False condition with reason PodUsingPVC and updates lastTransitionTime accordingly.

Trade‑offs include a modest increase in controller memory (the per‑namespace reference map) and additional API write traffic when many short‑lived pods repeatedly mount the same PVC. However, the design avoids cross‑namespace scans and delivers a deterministic, cluster‑wide view of PVC usage that can be queried directly via .status.conditions without external scripts.

The PersistentVolumeClaimUnusedSinceTime feature gate graduated to Beta in Kubernetes v1.37 and is enabled by default. Once active, the PVC protection controller adds an Unused condition to every PVC, reflecting whether any non‑terminal pod currently references the claim. This eliminates the need for custom scripts that cross‑reference pods, PVCs, and PersistentVolumes.

Step‑by‑step commands

• Create a PVC that will be monitored:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: my-data
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi

• Verify the initial condition (should be True because no pod uses it yet):

kubectl get pvc my-data -o jsonpath='{.status.conditions[?(@.type=="Unused")]}'
# Expected output fragment:
# {"lastTransitionTime":"2026-09-14T12:03:11Z","reason":"NoPodsUsingPVC","status":"True","type":"Unused"}

• Deploy a pod that mounts the PVC:

apiVersion: v1
kind: Pod
metadata:
  name: my-app
spec:
  containers:
  - name: app
    image: busybox
    command: ["sleep","3600"]
    volumeMounts:
    - name: data
      mountPath: /data
  volumes:
  - name: data
    persistentVolumeClaim:
      claimName: my-data

• Re‑inspect the condition; it must now be False (reason PodUsingPVC):

kubectl get pvc my-data -o jsonpath='{.status.conditions[?(@.type=="Unused")].status}'
# Output: False

• Delete the pod and wait for the controller to reconcile:

kubectl delete pod my-app
# Give the controller a few seconds, then:
kubectl get pvc my-data -o jsonpath='{.status.conditions[?(@.type=="Unused")]}'
# Status should be True again, with lastTransitionTime marking when the PVC became idle.

Because terminated pods (phase Succeeded or Failed) are ignored, batch jobs that finish will not keep the PVC in the Unused=False state. Conversely, pending pods— even unschedulable ones— count as users of the volume, ensuring that intent to mount is respected.

Cluster‑wide discovery of orphaned PVCs

kubectl get pvc -A -o json | jq -r '
  .items[]
  | select(.status.conditions[]? | select(.type=="Unused" and .status=="True"))
  | select((.status.conditions[] | select(.type=="Unused") | .lastTransitionTime) as $t
           | (now - ($t | fromdateiso8601)) > (30*86400))
  | "\(.metadata.namespace)/\(.metadata.name) unused since \(.status.conditions[] | select(.type=="Unused") | .lastTransitionTime)"
'

This query returns every PVC that has been idle for more than 30 days, leveraging the built‑in lastTransitionTime field. Engineers can feed this list into automated cleanup pipelines, reducing storage waste without risking data loss.

Since Kubernetes v1.37 the PVC protection controller adds an Unused condition to every PersistentVolumeClaim. The condition is True when no non‑terminal pod (including pending pods) references the claim, and it carries a lastTransitionTime that can be used to age‑filter orphaned volumes. This native signal eliminates the need for custom cross‑referencing scripts and makes cleanup a deterministic operation.

Querying unused PVCs with kubectl and jq is straightforward. The following one‑liner lists all PVCs that have been idle for more than 30 days across all namespaces:

kubectl get pvc -A -o json |
jq -r '
  .items[]
  | select(.status.conditions[]? |
      select(.type=="Unused" and .status=="True"))
  | select(
      (.status.conditions[] |
        select(.type=="Unused") |
        .lastTransitionTime) as $t |
      (now - ($t | fromdateiso8601)) > (30*86400)
    )
  | "\(.metadata.namespace)/\(.metadata.name) unused since \(.status.conditions[] |
      select(.type==\"Unused\") |
      .lastTransitionTime)"'

Key points to remember when integrating this query into automation:

• Pending pods count. An unschedulable pod still sets Unused=False. Automation must therefore wait for the pod to be deleted or transition to a terminal phase before treating the PVC as reclaimable.
• Terminated pods are ignored. Pods in Succeeded or Failed phases do not keep the claim busy, which is safe for batch workloads.
• Multiple owners. The condition flips to True only after the last referencing pod disappears, preventing premature deletion.

To schedule periodic cleanup, create a CronJob that runs the query, filters the output, and deletes the stale PVCs after a configurable grace period. Example manifest:

apiVersion: batch/v1
kind: CronJob
metadata:
  name: pvc-cleanup
spec:
  schedule: "0 2 * * *"   # daily at 02:00 UTC
  jobTemplate:
    spec:
      template:
        spec:
          serviceAccountName: pvc-cleanup-sa
          containers:
          - name: cleaner
            image: bitnami/kubectl:latest
            command: ["/bin/sh", "-c"]
            args:
            - |
              PVCs=$(kubectl get pvc -A -o json |
                jq -r '...same filter as above...');
              for p in $PVCs; do
                ns=$(echo $p | cut -d/ -f1);
                name=$(echo $p | cut -d/ -f2);
                kubectl delete pvc $name -n $ns --ignore-not-found;
              done
          restartPolicy: OnFailure

Integrating the check into CI/CD pipelines adds a safety net for development clusters. A typical GitHub Actions step might run the same kubectl/jq pipeline after integration tests and fail the workflow if any PVC remains unused for longer than the test retention window, prompting developers to add explicit cleanup logic.

Trade‑offs to consider:

• Relying on the beta‑level Unused condition ties the automation to clusters running v1.37 or newer; older clusters still need custom scripts.
• Deleting PVCs without snapshotting can cause data loss; incorporate a pre‑delete hook that creates a snapshot or copies data to a backup bucket.
• Running the query cluster‑wide can be expensive on very large clusters; limit scope by namespace or label selector when possible.

When the PVC protection controller adds the Unused condition (Beta in v1.37), engineers often encounter three debugging patterns: stale condition state, false‑positive usage, and controller performance spikes.

Common debugging scenarios

• kubectl get pvc … -o jsonpath shows Unused=True even though a pod is still running. Check that the pod is not in a Terminated phase; only non‑terminal pods count toward the condition.
• The condition never flips to True after pod deletion. Verify that the controller’s watch cache is not lagging by inspecting kubectl logs -n kube-system pvc-protection-controller for “sync” errors.
• Multiple pods reference the same PVC and one pod crashes. The condition remains False until the last non‑terminated pod disappears. Use kubectl get pod -l claimName=my-data to confirm the pod set.

Performance considerations of the controller

• The controller now watches Pod objects in addition to PersistentVolumeClaim. In clusters with >10 k pods, watch traffic can increase CPU usage by ~5 % and memory by ~30 MiB per controller replica.
• Pending pods still count as users of a PVC, which can cause the condition to stay False for long periods if unschedulable pods accumulate. Periodic cleanup of Pending pods mitigates this load.
• Each condition update writes a lastTransitionTime timestamp, generating additional etcd writes. Enable the controller’s --max-concurrent-reconciles flag to throttle reconciliation and avoid etcd write bursts.

Migration from Alpha to Beta

• In v1.36 the feature required the PersistentVolumeClaimUnusedSinceTime gate; upgrade scripts must remove the gate flag from the API server config.
• Because the gate is now enabled by default, existing clusters automatically start emitting the Unused condition. Verify that no custom admission webhook overrides the condition field.
• Update automation that previously queried custom annotations to instead read the native condition:

kubectl get pvc -A -o json | jq -r '
  .items[] |
  select(.status.conditions[]? |
    select(.type=="Unused" and .status=="True")) |
  "\(.metadata.namespace)/\(.metadata.name) idle since \(.status.conditions[] |
    select(.type=="Unused") | .lastTransitionTime)"'

Expectations for GA

• Full end‑to‑end test coverage is already in place; GA will likely add stricter performance SLAs and deprecate the Alpha‑only feature‑gate flag.
• Future releases may expose a spec.cleanupPolicy that automatically deletes PVCs after a configurable idle period, building on the Unused condition.
• Contributors are encouraged to file feedback in the kubernetes/kubernetes repo; the SIG Storage meeting minutes will track GA readiness.