diff --git a/.claude/skills/bump/SKILL.md b/.claude/skills/bump/SKILL.md new file mode 100644 index 00000000..bce805b2 --- /dev/null +++ b/.claude/skills/bump/SKILL.md @@ -0,0 +1,25 @@ +--- +name: bump +description: Bump the chart version in charts/mlrun-ce/Chart.yaml (patch, minor, or rc) +allowed-tools: Read(charts/mlrun-ce/Chart.yaml) Edit(charts/mlrun-ce/Chart.yaml) Read(charts/mlrun-ce/README.md) Edit(charts/mlrun-ce/README.md) +disable-model-invocation: false +--- + +Bump the version in `charts/mlrun-ce/Chart.yaml`. + +Usage: /bump + +- `patch` — increment the patch digit: `0.11.0` → `0.11.1` +- `minor` — increment the minor digit and reset patch: `0.11.3` → `0.12.0` +- `rc` — increment the RC counter on the current version: `0.11.0-rc.34` → `0.11.0-rc.35` + - If the current version has no RC suffix, add `-rc.1`: `0.11.0` → `0.11.0-rc.1` + +Steps: +1. Read the current version from `charts/mlrun-ce/Chart.yaml` (the `version:` field). +2. Compute the new version according to the argument above. +3. Show the user: "Bumping `` → ``" and ask for confirmation before writing. +4. On confirmation, update the `version:` field in `charts/mlrun-ce/Chart.yaml` in-place. +5. Remind the user: version bumps must be committed before opening a PR, and the PR title must follow `[Scope] description` format. +6. Update the MLRun CE version under Version Matrix in `charts/mlrun-ce/README.md`. + +If no argument is given, show the current version and list the three options with the resulting version for each, then ask which to apply. diff --git a/.claude/skills/pr/SKILL.md b/.claude/skills/pr/SKILL.md new file mode 100644 index 00000000..94777982 --- /dev/null +++ b/.claude/skills/pr/SKILL.md @@ -0,0 +1,108 @@ +--- +name: pr +description: Analyze branch changes and generate a fully filled PR description ready to paste into GitHub +allowed-tools: Bash(git diff*) Bash(git log*) +disable-model-invocation: false +--- + +Analyze the current branch changes and generate a fully filled PR description ready to paste into GitHub. + +## Steps + +1. **Gather context** — run these in parallel: + - `git diff upstream/development...HEAD` — full diff against the base branch + - `git log upstream/development..HEAD --oneline` — commit list + - `git diff upstream/development...HEAD --name-only` — changed files + +2. **Analyze the diff** carefully: + - What components or templates were changed? (check which `templates/` subdirs, `values.yaml` sections, `requirements.yaml`, `Chart.yaml`) + - Were any new values keys added? Do they need to be reflected in the three install-mode values files? + - Were any Secrets, ConfigMaps, or port numbers changed? (potential breaking changes) + - Was `Chart.yaml` version bumped? If not, flag it. + - Were `requirements.yaml` or `requirements.lock` changed? + - Does `charts/mlrun-ce/README.md` need updating (new NodePort, new component, new install step)? + +3. **Detect breaking changes** — flag as breaking if any of: + - A value key was renamed or removed + - A Secret or ConfigMap name changed + - A NodePort number changed + - A sub-chart was upgraded with a major version bump + - The storage credentials structure changed + - Any hook annotation or hook-weight changed in a way that affects upgrade order + +4. Provide an optional PR title following the `[Scope] description` format, where Scope is one of: `['feature', 'fix', 'docs', 'improvement', 'revert', 'breaking', 'ci']`. For example: `[Feature] Add Redis support to mlrun-ce`. +5. **Fill the PR template** — produce the complete filled template below. Be specific and concrete; do not use placeholder text. + +--- + +Apply these checklist rules before writing the output: +- `[x]` — you can confirm this item is satisfied from the diff alone +- `[ ]` — requires human action, judgment, or external system access + +Specific rules: +- "tested" → always `[ ]` +- "documentation PR" → always `[ ]` +- "QA tests / Jira ticket" → always `[ ]` +- "installation verified" → always `[ ]` +- `Chart.yaml` version bump → `[x]` if diff shows version changed, otherwise `[ ]` and add to Warnings +- Multi-namespace values files → `[x]` if all three are in the diff OR the change has no effect on install-mode values; `[ ]` with a note if a new value was added only to `values.yaml` +- README update → `[x]` if `charts/mlrun-ce/README.md` is in the diff OR no new NodePorts/components were added; otherwise `[ ]` + +Output exactly this structure with real content (no placeholder text): + +```markdown +### 📝 Description +<2-4 sentences: what changed, why, and what it affects> + +--- + +### 🛠️ Changes Made + + +--- + +### ✅ Checklist +- [ ] I have tested the changes in this PR +- [ ] I confirmed whether my changes require a change in documentation and if so, I created another PR in MLRun for the relevant documentation. +- [ ] I confirmed whether my changes require changes in QA tests, for example: credentials changes, resources naming change and if so, I updated the relevant Jira ticket for QA. +- [ ] I increased the Chart version in `charts/mlrun-ce/Chart.yaml`. +- [ ] I confirmed that the installation works both on a local Docker Desktop environment and on a real cluster when using the required [prerequisites](https://docs.mlrun.org/en/stable/install-mlrun-ce/kubernetes-install.html#prerequisites). + - [ ] If installation issues were found, I updated the relevant Jira ticket with the issue and steps to reproduce, or updated the prerequisites documentation if the issue is related to missing or outdated prerequisites. +- [ ] If needed, update https://github.com/mlrun/ce/blob/development/charts/mlrun-ce/README.md with the relevant installation instructions and version Matrix. +- [ ] If needed, update the following values files for multi namespace support: + - [ ] [Admin values](https://github.com/mlrun/ce/blob/development/charts/mlrun-ce/admin_installation_values.yaml) + - [ ] [User values Node Port](https://github.com/mlrun/ce/blob/development/charts/mlrun-ce/non_admin_installation_values.yaml) + - [ ] [User values ClusterIP](https://github.com/mlrun/ce/blob/development/charts/mlrun-ce/non_admin_cluster_ip_installation_values.yaml) + +--- + +### 🧪 Testing + + +--- + +### 🔗 References +- Ticket link: +- External links: +- Design docs links (Optional): + +--- + +### 🚨 Breaking Changes? + +- [ ] Yes (explain below) +- [ ] No + + + +--- + +### 🔍️ Additional Notes + +``` + +Then replace each `[ ]` with `[x]` on items you can confirm from the diff, following the rules above. + +After outputting the filled template, add a short **"Warnings"** section (outside the template) listing anything that needs human attention before opening the PR (missing version bump, unsynced values files, potential breaking changes, etc.). + +Between every sentence that ends with a `.` add two new lines to make it more readable. diff --git a/.cursorrules b/.cursorrules new file mode 100644 index 00000000..902549ab --- /dev/null +++ b/.cursorrules @@ -0,0 +1,64 @@ +# MLRun Community Edition — Cursor Rules + +## Source of Truth + +`AGENTS.md` is the authoritative reference for this project. Read it before making any suggestions. It covers architecture, design patterns, template conventions, component dependencies, how to add new components, and common debugging scenarios. + +`CONTRIBUTING.md` covers the development workflow, commit format, and PR process. + +## Preferred Response Patterns + + +- Values changes: show `--set` flags or a patch values file overlay, not edits to `values.yaml` directly +- New templates: show the complete file including the `{{- if .Values..enabled }}` guard and `include "mlrun-ce.common.labels"` call +- Service references within templates: use `{{ .Release.Namespace }}`, never hardcode namespace strings +- After any `requirements.yaml` change: remind the user to run `make helm-update-dependencies` and commit `requirements.lock` +- If a change affects the default installation, remind the user to update `values.yaml` and all three install-mode values files (`admin_installation_values.yaml`, `non_admin_installation_values.yaml`, `non_admin_cluster_ip_installation_values.yaml`) with the appropriate default +- If a change adds a new component, changes a component version, or changes the installation process, remind the user to update `charts/mlrun-ce/README.md` + +## Common Tasks (Claude Code has `/bump`, `/pr` skills for these) + +When a user asks you to help with the following tasks, use the commands below — these are the manual equivalents of the Claude Code skills defined in `.claude/skills/`. + +**Render chart templates** +```bash +# Full chart +helm template mlrun charts/mlrun-ce -f charts/mlrun-ce/values.yaml + +# Single template or directory +helm template mlrun charts/mlrun-ce -f charts/mlrun-ce/values.yaml \ + --show-only templates/ + +# With a values overlay (e.g. non-admin install) +helm template mlrun charts/mlrun-ce \ + -f charts/mlrun-ce/values.yaml \ + -f charts/mlrun-ce/non_admin_installation_values.yaml +``` + +**Bump Chart version** (`/bump` in Claude Code) +Read the current version from `charts/mlrun-ce/Chart.yaml` and increment: +- `patch` — `0.11.0` → `0.11.1` +- `minor` — `0.11.3` → `0.12.0` +- `rc` — `0.11.0-rc.34` → `0.11.0-rc.35` (or append `-rc.1` if no suffix) + +Always show the old → new version before writing and confirm with the user. + +**Generate PR description** (`/pr` in Claude Code) +Run `git diff upstream/development...HEAD`, `git log upstream/development..HEAD --oneline`, and `git diff upstream/development...HEAD --name-only`, then fill in `.github/pull_request_template.md` based on the changes. Check `[x]` on checklist items confirmable from the diff; leave `[ ]` on items requiring human action. Flag missing version bumps, unsynced values files, and breaking changes. + +## Do Not Suggest + +- `helm upgrade --install` without running `make helm-update-dependencies` first +- Adding a new sub-chart to `requirements.yaml` for custom resources — add templates to `charts/mlrun-ce/templates//` instead +- `kubectl apply` for resources managed by this chart +- `Chart.yaml` apiVersion v2 dependency blocks (this chart uses apiVersion v1 + `requirements.yaml`) +- Creating a second credentials Secret — mount the existing `storage-credentials` Secret via `envFrom` +- Hardcoding namespace names in templates — use `{{ .Release.Namespace }}` +- Using `kafka.enabled + strimzi-kafka-operator.enabled` as a combined condition — the template guard is only `kafka.enabled`; Strimzi is a prerequisite, not a co-guard +- Treating `seaweedfs-s3-config` as a SeaweedFS dependency — SeaweedFS *creates* it; Pipelines and MLRun *consume* it + +## Workflow (from CONTRIBUTING.md) + +- Fork-based workflow: PRs target `upstream/development`, not `origin/development` +- Branch naming: `/` — e.g. `feature/add-redis-support`, `fix/CE-111` +- Always bump `charts/mlrun-ce/Chart.yaml` version before opening a PR diff --git a/.gitignore b/.gitignore index cc8f86f3..23624f03 100644 --- a/.gitignore +++ b/.gitignore @@ -4,3 +4,6 @@ charts/mlrun-ce/charts/* **/.DS_Store *.DS_Store **/__pycache__ + +# Claude Code local settings (machine-specific, not for commit) +.claude/settings.local.json diff --git a/AGENTS.md b/AGENTS.md index 3960b950..27681f1f 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -4,7 +4,7 @@ This is a Helm umbrella chart repository for **MLRun Community Edition (CE)** — an open-source MLOps stack. The main chart lives at `charts/mlrun-ce/` and bundles: Nuclio, MLRun, Jupyter, MPI Operator, SeaweedFS (S3-compatible storage), Spark Operator, Kubeflow Pipelines, Prometheus stack, TimescaleDB, and Strimzi Kafka Operator. -## Commands +## Commands to lint, package, and manage the chart: ```bash # Lint the helm chart (requires helm and chart-testing `ct` installed) @@ -18,14 +18,6 @@ make helm-repo-add # Package the chart as a tarball make package - -# Run full local end-to-end test on a Kind cluster (requires docker, kind, kubectl, helm) -./tests/kind-test.sh full # Create Kind cluster + install chart -./tests/kind-test.sh create # Create cluster only -./tests/kind-test.sh install # Install chart (assumes cluster exists) -./tests/kind-test.sh verify # Verify installation -./tests/kind-test.sh delete # Delete Kind cluster -CLEANUP_ON_EXIT=true ./tests/kind-test.sh # Auto-cleanup after test ``` ## Architecture @@ -39,27 +31,82 @@ CLEANUP_ON_EXIT=true ./tests/kind-test.sh # Auto-cleanup after test ### Template Organization (`charts/mlrun-ce/templates/`) -- `config/` — ConfigMaps and Secrets shared across components: MLRun env config, Jupyter env config, S3 credentials secret, Pipelines config, Spark config, Grafana dashboards +- `config/` — ConfigMaps and Secrets shared across components: MLRun env config, Jupyter env config, storage credentials secret, Pipelines config, Spark config, Grafana dashboards - `seaweedfs/` — SeaweedFS-specific resources: S3 IAM config secret, bucket init job, admin UI NodePort service, ingress - `kafka/` — Kafka Strimzi custom resources: KafkaNodePool, Kafka cluster CR, bootstrap alias Service, RBAC, NetworkPolicy - `timescaledb/` — TimescaleDB Deployment, Service, PVC - `jupyter-notebook/` — Jupyter Deployment and supporting resources - `pipelines/` — Kubeflow Pipelines resources +- `spark-operator/` — Spark controller RBAC - `persistency/` — PVC definitions - `aws/` — AWS-specific resources ### Key Design Patterns -**S3 credentials propagation**: The top-level `s3.accessKey`/`s3.secretKey`/`s3.bucket` values flow into a `s3-credentials` Secret (created by `templates/config/s3-credentials-secret.yaml`), which is then mounted via `envFrom` in MLRun API and Jupyter pods. SeaweedFS uses the same credentials via the `seaweedfs-s3-config` Secret. +**S3 credentials propagation**: The top-level `storage.s3.accessKey`/`storage.s3.secretKey`/`storage.s3.bucket` values flow into a `storage-credentials` Secret (created by `templates/config/storage-secret.yaml`), which is then mounted via `envFrom` in MLRun API and Jupyter pods. SeaweedFS uses the same credentials via the `seaweedfs-s3-config` Secret. **Global registry anchor**: `global.registry: &userRegistry` in `values.yaml` uses YAML anchors to multiplex the same docker registry config to both `nuclio.global.registry` and `mlrun.global.registry`. -**SeaweedFS as S3 backend**: SeaweedFS replaced MinIO. The helpers in `_helpers.tpl` (`mlrun-ce.s3.*`) generate the SeaweedFS service URL. Legacy `mlrun-ce.minio.*` helpers are kept as aliases pointing to the SeaweedFS helpers. - **Component enable/disable**: Most components can be disabled via `.enabled: false`. The Kafka setup requires the Strimzi operator (deployed as a sub-chart via `strimzi-kafka-operator`) and custom Strimzi CRs in `templates/kafka/`. ### Values Files -- `charts/mlrun-ce/admin_installation_values.yaml` — admin install -- `charts/mlrun-ce/non_admin_installation_values.yaml` — non-admin install -- `charts/mlrun-ce/non_admin_cluster_ip_installation_values.yaml` — non-admin with ClusterIP +- `charts/mlrun-ce/values.yaml` - base values for all modes, and default installation. +- `charts/mlrun-ce/admin_installation_values.yaml` - use to install cluster resources such as CRDs, RBAC, and operators deployment. +- `charts/mlrun-ce/non_admin_installation_values.yaml` - use to install non-cluster resources such as Deployments, Services, and Ingresses with NodePort. +- `charts/mlrun-ce/non_admin_cluster_ip_installation_values.yaml` - use to install non-cluster resources such as Deployments, Services, and Ingresses with ClusterIP. + +## Quick-Start Dev Workflow + +From a fresh clone to a linted chart: + +1. `make helm-repo-add` — adds all external repos (reads `requirements.yaml`; idempotent) +2. `make helm-update-dependencies` — downloads sub-chart tarballs into `charts/mlrun-ce/charts/` (must run before any lint or template render) +3. `make helm-lint` — runs `helm lint charts/mlrun-ce` + `ct lint` + - `ct` only lints charts with changes relative to the target branch; always run from a feature branch, not directly on `development` +4. Render all templates locally (no cluster needed): + ```bash + helm template mlrun charts/mlrun-ce -f charts/mlrun-ce/values.yaml + ``` +5. Render a single template file: + ```bash + helm template mlrun charts/mlrun-ce -f charts/mlrun-ce/values.yaml --show-only templates/kafka/kafka-cluster.yaml + ``` +6. Schema-validate without a cluster: + ```bash + helm template mlrun charts/mlrun-ce -f charts/mlrun-ce/values.yaml | kubectl apply --dry-run=client -f - + ``` + +## Component Dependency Map + +| Component | Enabled by | Runtime dependencies | Key templates / notes | +|---|---|---|---| +| MLRun API + UI + DB | `mlrun.enabled` | `storage-credentials` Secret, `mlrun-common-env` ConfigMap; mlrun-db (MySQL) is bundled inside the mlrun sub-chart | `templates/config/mlrun-env-configmap.yaml`; rest is in the `mlrun` sub-chart | +| Jupyter | `jupyterNotebook.enabled` | `storage-credentials` Secret, `jupyter-common-env` ConfigMap | `templates/jupyter-notebook/` | +| Nuclio | always on (no `enabled` guard in umbrella) | `global.registry` must be set | sub-chart only — no custom templates | +| MPI Operator | always on (no `enabled` guard in umbrella) | none | sub-chart only — no custom templates | +| SeaweedFS | `seaweedfs.enabled` | PVC for data storage; creates `seaweedfs-s3-config` Secret consumed by Pipelines and MLRun | `templates/seaweedfs/`; `seaweedfs.s3.enableAuth: true` must be set or the Secret is skipped | +| Spark Operator | `spark-operator.enabled` | none | sub-chart + `templates/spark-operator/spark-controller-rbac.yaml` | +| Kafka | `kafka.enabled` | Strimzi CRDs — `strimzi-kafka-operator` sub-chart must also be enabled as a prerequisite; CRs use post-install hooks to wait for CRDs | `templates/kafka/` | +| Pipelines | `pipelines.enabled` | SeaweedFS (`seaweedfs.enabled` checked at render time; adds init container to wait for it), `mlrun-pipelines-config` ConfigMap | `templates/pipelines/`, `templates/config/mlrun-pipelines-config.yaml` | +| TimescaleDB | `timescaledb.enabled` | none; uses its own `-timescaledb-secret` for the DB password | `templates/timescaledb/` — custom StatefulSet, not a sub-chart | +| Prometheus + Grafana | `kube-prometheus-stack.enabled` | none at runtime; model monitoring dashboards are pre-loaded as static JSON ConfigMaps | sub-chart config in `values.yaml`; dashboards in `templates/config/model-monitoring-*.yml` | + +## How to Add a New Component + +1. Add a top-level block to `values.yaml` and update the three install-mode values files: + ```yaml + myComponent: + enabled: true + ``` +2. Create `charts/mlrun-ce/templates/myComponent/`. Every template file must open with `{{- if .Values.myComponent.enabled }}` and close with `{{- end }}`. Add label helpers to `_helpers.tpl` following the `mlrun-ce..labels` / `mlrun-ce..selectorLabels` pattern. +3. NodePort selection — avoid all currently occupied ports: + - 30010 Grafana, 30020 Prometheus, 30040 Jupyter, 30050 Nuclio + - 30060 MLRun UI, 30070 MLRun API, 30093 SeaweedFS Admin, 30094 SeaweedFS S3 + - 30100 Pipelines, 30110 TimescaleDB + - Create a NodePort service only for user-facing UIs or APIs; internal-only components use ClusterIP. +4. Storage credentials — mount the existing `storage-credentials` Secret via `envFrom.secretRef`; do not create a second credentials secret. Expose all port numbers and other tunables as `values.yaml` keys rather than hardcoding them in templates. +5. CRD dependencies — if the component depends on CRDs from a sub-chart, use `helm.sh/hook: post-install,post-upgrade` with an appropriate `hook-weight` on the CRs (see `templates/kafka/` for the established pattern). +6. Keep Secret and ConfigMap names consistent with existing patterns (`storage-credentials`, `mlrun-common-env`, etc.). Add env config to `templates/config/` following the `mlrun-common-env` / `jupyter-common-env` pattern. Each Kubernetes resource that supports limits and requests should expose them in `values.yaml`. +7. Add the component's service URL to `templates/NOTES.txt`, update `charts/mlrun-ce/README.md` if a new NodePort is exposed, and add a section to this AGENTS.md describing the component's architecture and dependencies. +8. Bump `charts/mlrun-ce/Chart.yaml` and run `make helm-lint` before opening a PR. diff --git a/CLAUDE.md b/CLAUDE.md index 43c994c2..6a998595 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1 +1,12 @@ @AGENTS.md +@CONTRIBUTING.md + +## Preferred Response Patterns + + +- Values changes: show `--set` flags or a patch values file overlay, not edits to `values.yaml` directly, unless there is a change with the default value that should be reflected in `values.yaml` (e.g. a new component's `enabled` flag) +- New templates: show the complete file including the `{{- if .Values..enabled }}` guard and `include "mlrun-ce.common.labels"` call +- Service references within templates: use `{{ .Release.Namespace }}`, never hardcode namespace strings +- After any `requirements.yaml` change: remind the user to run `make helm-update-dependencies` and commit `requirements.lock` +- Please make sure to update all values files in `charts/mlrun-ce/` if your change affects the default installation — e.g. if you add a new component with an `enabled` flag, add it to `values.yaml` and all three install-mode values files (`admin_installation_values.yaml`, `non_admin_installation_values.yaml`, `non_admin_cluster_ip_installation_values.yaml`) with the appropriate default value +- Update README docs if your change adds a new component, changes component version or changes the installation process. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 00000000..6caf85f8 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,119 @@ +# Contributing to MLRun Community Edition + +## Prerequisites + +| Tool | Minimum version | Purpose | +|---|---|---| +| helm | 3.6 | Chart rendering, linting, install | +| kubectl | 1.24 | Cluster interaction | + +For Kubernetes storage class setup and cluster prerequisites, see [charts/mlrun-ce/README.md](charts/mlrun-ce/README.md#prerequisites). + +## First-Time Setup + +```bash +# 1. Add all external Helm repos (reads requirements.yaml; idempotent) +make helm-repo-add + +# 2. Download sub-chart tarballs into charts/mlrun-ce/charts/ +make helm-update-dependencies + +# 3. Lint the chart +make helm-lint +``` + +## Development Workflow + +1. Fork your own copy of [MLRun CE repository](https://github.com/mlrun/ce). +2. Clone your fork and set the upstream remote to the main repository: + +```bash +git clone https://github.com//ce.git +cd ce +git remote add upstream https://github.com/mlrun/ce +``` +3. Branch from `development` — the default branch is `development`, not `main` or `master`. +4. Branch naming convention: `/`, e.g. `feature/add-redis-support` or `fix/CE-111`. +5. All PRs target `upstream/development`. +6. Bump the version in `charts/mlrun-ce/Chart.yaml` before opening a PR (format: `major.minor.patch` or `major.minor.patch-rc.N`, e.g. `0.11.0-rc.1`). +7. If you change `requirements.yaml`, run `make helm-update-dependencies` and commit the updated `requirements.lock` alongside it. +8. Each change should support Local Mac installations and native k8s as explained in [prerequisites documentation](https://docs.mlrun.org/en/stable/install-mlrun-ce/kubernetes-install.html#prerequisites). + +## Commit and PR Title Format + +All PR titles (and squash-merge commit messages) must match: + +``` +[Scope] Short description +``` + +Allowed scopes (case-insensitive): `Feature`, `Fix`, `Docs`, `Improvement`, `Revert`, `Breaking`, `CI` + +Examples: +- `[Feature] Add Redis sub-chart support` +- `[Fix] Correct SeaweedFS S3 endpoint in mlrun-common-env` +- `[Breaking] Remove legacy minio helper aliases` + +The format is enforced by `.github/workflows/pr-validation.yml` and auto-applies a label on the PR. Squash-merge titles become changelog entries via `git-cliff` and `cliff.toml`. + +## Testing + +Two levels, run them in order: + +### 1. Lint (no cluster required) + +```bash +make helm-lint +``` + +### 2. Schema validation (no cluster required) + +Renders the chart and validates resource manifests against the Kubernetes API schema without a running cluster: + +```bash +helm template mlrun charts/mlrun-ce -f charts/mlrun-ce/values.yaml \ + | kubectl apply --dry-run=client -f - +``` + +Use a specific values overlay to test a particular install mode: + +```bash +helm template mlrun charts/mlrun-ce \ + -f charts/mlrun-ce/values.yaml \ + -f charts/mlrun-ce/non_admin_installation_values.yaml \ + | kubectl apply --dry-run=client -f - +``` + +## Adding a New Component + +For the full step-by-step guide see [AGENTS.md — How to Add a New Component](AGENTS.md#how-to-add-a-new-component). The summary: + +1. Add an `enabled` flag block to `values.yaml`. +2. Create `charts/mlrun-ce/templates//`. +3. Wrap every template file with `{{- if .Values..enabled }}` / `{{- end }}`. +4. Mount the existing `storage-credentials` Secret for storage access; don't create a new one. +5. Use `helm.sh/hook: post-install,post-upgrade` for resources that depend on CRDs. +6. Update values.yaml and all three install-mode values files: + - `charts/mlrun-ce/values.yaml` - base values for all modes, and default installation. + - `charts/mlrun-ce/admin_installation_values.yaml` - use to install cluster resources such as CRDs, RBAC, and operators deployment. + - `charts/mlrun-ce/non_admin_installation_values.yaml` - use to install non-cluster resources such as Deployments, Services, and Ingresses with NodePort. + - `charts/mlrun-ce/non_admin_cluster_ip_installation_values.yaml` - use to install non-cluster resources such as Deployments, Services, and Ingresses with ClusterIP. +7. Update `templates/NOTES.txt` and `charts/mlrun-ce/README.md`. +8. Bump `charts/mlrun-ce/Chart.yaml` version. +9. Run `make helm-lint`. + +## PR Checklist + +When you open a PR, `.github/pull_request_template.md` is automatically populated with a checklist. Every item in that checklist must be addressed before the PR can merge. Key items include: + +- Testing against a real cluster (not just lint) +- Version bump in `Chart.yaml` +- Multi-namespace values files kept in sync +- Documentation updated +- Breaking changes disclosed + +## Release Process + +Versions follow `major.minor.patch[-rc.N]` (e.g. `0.11.0-rc.1` in `Chart.yaml`). RC builds are published automatically by the `release.yml` workflow on tag push. Non-RC (stable) releases use a separate workflow path. + +The changelog is generated by `git-cliff` using `cliff.toml`. Entries are grouped by scope: Features, Bug Fixes, Improvements, Breaking Changes, CI/CD, etc. The squash-merge PR title becomes the changelog entry — keep it descriptive. diff --git a/Makefile b/Makefile index 2cba3c0b..6415679a 100644 --- a/Makefile +++ b/Makefile @@ -36,7 +36,7 @@ package: ## Package the application .PHONY: helm-lint helm-lint: helm-repo-add ## Lint Helm Chart @helm lint charts/mlrun-ce - @ct lint --target-branch $(HELM_LINT_DEFAULT_BRANCH) + @ct lint --target-branch $(HELM_LINT_DEFAULT_BRANCH) --validate-maintainers=false --helm-extra-args "--timeout 600s" .PHONY: helm-update-dependencies helm-update-dependencies: ## Update Helm Chart dependencies diff --git a/charts/mlrun-ce/Chart.yaml b/charts/mlrun-ce/Chart.yaml index 16ed4151..06d2db3f 100644 --- a/charts/mlrun-ce/Chart.yaml +++ b/charts/mlrun-ce/Chart.yaml @@ -1,6 +1,6 @@ apiVersion: v1 name: mlrun-ce -version: 0.11.0-rc.34 +version: 0.11.0-rc.35 description: MLRun Open Source Stack home: https://iguazio.com icon: https://www.iguazio.com/wp-content/uploads/2019/10/Iguazio-Logo.png @@ -8,4 +8,4 @@ sources: [] maintainers: - name: MLRun email: mlrun@iguazio.com - url: https://www.mlrun.org/ \ No newline at end of file + url: https://www.mlrun.org/ diff --git a/charts/mlrun-ce/README.md b/charts/mlrun-ce/README.md index e3707b48..ed07a7bf 100644 --- a/charts/mlrun-ce/README.md +++ b/charts/mlrun-ce/README.md @@ -268,7 +268,6 @@ $ kubectl --namespace mlrun delete pvc # Remove hostpath(s) used for mlrun (and possibly nfs). Those will be created, by default under /tmp, and will contain # your release name, e.g.: $ rm -rf my-mlrun-mlrun-ce-mlrun -... ``` ### Using Kubeflow Pipelines