Skip to content

minio: honor securityContext.fsGroupChangePolicy (OnRootMismatch) for large PVC mounts #312

Description

@philblckwd

Describe the bug / feature request

The bundled MinIO subchart hard-codes only a subset of pod securityContext fields in the StatefulSet template. Setting minio.securityContext.fsGroupChangePolicy in parent chart values has no effect, because the field is never rendered.

This causes long MinIO pod startup times on large PVCs: kubelet uses the default fsGroupChangePolicy: Always and recursively chowns the volume on every mount (including node drains / rolling upgrades / reschedules). On large object-store volumes this can take many minutes and surface as VolumePermissionChangeInProgress / offline MinIO drives, risking erasure-set quorum loss during maintenance.

Versions we use

Component Version
milvus-helm chart 5.0.22
Milvus v2.6.20 (also seen with v2.6.18 on the same chart)
Bundled minio subchart 8.0.17 (as packaged in milvus 5.0.22)

Still present in newer chart lines we checked (e.g. milvus 5.0.25) — the MinIO StatefulSet template still only emits runAsUser / runAsGroup / fsGroup.

Current template behavior

In charts/minio/templates/statefulset.yaml (minio 8.0.17):

securityContext:
  runAsUser: {{ .Values.securityContext.runAsUser }}
  runAsGroup: {{ .Values.securityContext.runAsGroup }}
  fsGroup: {{ .Values.securityContext.fsGroup }}

So this values override is silently ignored:

minio:
  securityContext:
    enabled: true
    runAsUser: 1000
    runAsGroup: 1000
    fsGroup: 1000
    fsGroupChangePolicy: OnRootMismatch

Expected behavior

minio.securityContext.fsGroupChangePolicy should appear on the MinIO pod template.

Suggested fix

Match the official MinIO Helm chart pattern and render the full securityContext (minus enabled), e.g.:

{{- if and .Values.securityContext.enabled .Values.persistence.enabled }}
securityContext:
  {{- omit .Values.securityContext "enabled" | toYaml | nindent 8 }}
{{- end }}

And document / default in charts/minio/values.yaml:

securityContext:
  enabled: true
  runAsUser: 1000
  runAsGroup: 1000
  fsGroup: 1000
  fsGroupChangePolicy: OnRootMismatch

OnRootMismatch is the Kubernetes-recommended policy for large volumes: skip the recursive ownership walk when the volume root already matches fsGroup (typical for StatefulSet remounts of the same PVC). First mount / fsGroup changes still perform a full apply.

Refs:

Why it matters for Milvus

With distributed MinIO (e.g. 4 replicas / 4 drives), a slow remount during node upgrade can keep a drive offline long enough that a second concurrent disruption drops below write quorum and stalls Woodpecker/Milvus ingest. Passing through fsGroupChangePolicy lets operators fix this via values without forking the chart.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions