Hardening Kubernetes v1.37 Container Storage: Bind Mount Options & emptyDir Permissions

When managing Linux filesystems, Virtual File System (VFS) bind mount flags and directory permission modes serve as foundational controls for securing shared resources. These mechanisms allow administrators to override default behaviors at the kernel level, effectively mitigating risks associated with untrusted binary execution and unauthorized file manipulation within shared storage.

VFS Bind Mount Flags

Bind mount flags restrict operations on a specific filesystem mount point, regardless of the underlying object's permissions. These flags are critical for hardening workloads by enforcing security policies at the boundary of a container or process namespace:

• noexec: Disallows the direct execution of binaries. Even if a file is marked executable (e.g., via chmod +x), the kernel will deny execution, preventing the runtime of arbitrary malicious payloads.
• nosuid: Prevents the set-user-identifier (set-uid) and set-group-identifier (set-gid) bits from being honored. This effectively mitigates privilege escalation vectors that rely on executing binaries with elevated permissions.
• nodev: Prevents the interpretation of character or block special device files. This ensures that a compromised process cannot interact with raw hardware devices via the mounted filesystem.

The Unix Sticky Bit (01777)

While standard permissions (Read, Write, Execute) manage access scope, the sticky bit (mode 01777) provides granular control over file deletion in shared directories. When applied to a directory, the kernel restricts file removal and renaming: only the file owner, the directory owner, or the root user may delete or move a file. This is the standard security configuration for shared scratch spaces like /tmp, as it prevents non-privileged processes from deleting files they do not own, even if the directory itself is globally writable.

By implementing these flags and modes—such as configuring an emptyDir volume with noexec or the 01777 mode—engineers can enforce the principle of least privilege, ensuring that multi-tenant or multi-container pods cannot interfere with one another's data or execute unauthorized binaries within shared writable volumes.

Kubernetes v1.37 introduces two Alpha feature gates—VolumeBindMountOptions and EmptyDirVolumeMode—that allow bind‑mount flags (e.g., noexec, nosuid, nodev) and explicit permission bits (including the sticky bit) on emptyDir volumes. Because they are Alpha, the gates must be enabled on both the API server and the kubelet before any pod can reference the new fields.

• API server: start the server with --feature-gates=VolumeBindMountOptions=true,EmptyDirVolumeMode=true. The flag is evaluated at admission time, so the API server will reject manifests that contain bindMountOptions or mode when the gates are off.
• Kubelet: use the same --feature-gates flag on each node. The kubelet validates the pod spec after it is scheduled and will refuse to start a pod that requests the Alpha fields on a node where the gates are disabled.

Enabling the gates alone is insufficient; the underlying container runtime must also support the CRI mount_options field. Runtimes advertise this capability via the runtimeFeatures API. If the runtime does not list MountOptions, the kubelet will reject the pod with a clear error, and the scheduler will avoid placing the pod on such nodes because the node’s feature set is reported in the NodeStatus object.

Practical steps for a cluster administrator:

1. Verify runtime support:

   kubectl get node -o jsonpath='{.status.nodeInfo.runtimeVersion}'

   and consult the runtime’s documentation for CRI mount_options support.
2. Update the API server manifest (or static pod) to include the feature‑gate flag.
3. Restart the API server, then update each kubelet configuration with the same flag and restart the kubelet.
4. Confirm the node reports the feature:

   kubectl get node <node-name> -o yaml | grep -i mount_options

Example pod manifest using both features:

apiVersion: v1
kind: Pod
metadata:
  name: hardened-pod
spec:
  containers:
  - name: app
    image: alpine:latest
    command: ["sleep","3600"]
    securityContext:
      readOnlyRootFilesystem: true
    volumeMounts:
    - name: work
      mountPath: /tmp
      bindMountOptions:
      - noexec
      - nosuid
  volumes:
  - name: work
    emptyDir:
      mode: 01777

When the cluster meets the gate and runtime requirements, the kubelet enforces noexec/nosuid at the bind‑mount level and applies the sticky‑bit mode to the emptyDir, providing native compliance with hardening benchmarks such as CIS Kubernetes and NIST SP 800‑190.

Kubernetes v1.37 introduces bindMountOptions and an emptyDir.mode field that let platform engineers enforce Linux‑level hardening directly in a pod manifest. The options map to VFS mount flags (noexec, nosuid, nodev) while the mode value follows standard Unix permission notation, where 01777 adds the sticky bit (t) to a writable directory. Both features are gated behind VolumeBindMountOptions and EmptyDirVolumeMode and require the container runtime to expose the CRI mount_options capability.

Example 1 – Bind mount options on an emptyDir

apiVersion: v1
kind: Pod
metadata:
  name: hardened-bindmount-pod
  namespace: default
spec:
  os:
    name: linux
  containers:
  - name: hardened-app
    image: alpine:latest
    command: ["sleep","3600"]
    securityContext:
      readOnlyRootFilesystem: true
    volumeMounts:
    - name: temp-storage
      mountPath: /tmp
      bindMountOptions:
      - noexec
      - nosuid
  volumes:
  - name: temp-storage
    emptyDir: {}

• readOnlyRootFilesystem: keeps the container image immutable; the writable /tmp is the only mutable location.
• bindMountOptions: applies the MS_NOEXEC and MS_NOSUID flags to the bind mount, preventing execution of binaries and ignoring set‑uid/set‑gid bits.
• emptyDir: {}: creates a node‑local directory with the default 0777 permissions; the mount flags provide the security hardening.

Example 2 – Sticky‑bit emptyDir mode

apiVersion: v1
kind: Pod
metadata:
  name: hardened-emptydir-pod
  namespace: default
spec:
  os:
    name: linux
  containers:
  - name: app-container
    image: alpine:latest
    command: ["sleep","3600"]
    volumeMounts:
    - name: shared-tmp
      mountPath: /tmp
  volumes:
  - name: shared-tmp
    emptyDir:
      mode: 01777

• mode: 01777: sets the directory permissions to rwxrwxrwt, enabling the sticky bit so only a file’s owner (or root) may delete or rename it.
• volumeMounts.mountPath: exposes the hardened directory at /tmp for all containers in the pod.
• emptyDir: works with any medium (disk, memory, HugePages) and now respects the explicit mode without requiring an init container.

After deployment, kubectl exec into each pod to verify the enforcement: attempts to run a script in the first pod are blocked with “Permission denied”, and ls -ld /tmp in the second pod shows the trailing t indicating the sticky bit is active.