Skip to content

SBR docs: update sbdconfig-user-guide with Quick Start, Known Issues, etc#2

Merged
openshift-merge-bot[bot] merged 6 commits into
medik8s:mainfrom
maximunited:fix/docs-sbdconfig-user-guide
Jul 20, 2026
Merged

SBR docs: update sbdconfig-user-guide with Quick Start, Known Issues, etc#2
openshift-merge-bot[bot] merged 6 commits into
medik8s:mainfrom
maximunited:fix/docs-sbdconfig-user-guide

Conversation

@maximunited

@maximunited maximunited commented Jan 27, 2026

Copy link
Copy Markdown
Contributor

Update SBDConfig User Guide Documentation

This PR updates the SBDConfig user guide with new sections and important improvements:

Key Changes

  • Quick Start Section: Added a new quick setup guide with step-by-step instructions
  • Known Issues and Workarounds: Documented two critical issues:
    • Invalid default image path (requires explicit spec.image)
    • Missing SecurityContextConstraints on OpenShift (requires manual SCC creation)
  • Installation Section: Completely updated with:
    • OpenShift-first approach with OperatorHub instructions
    • Complete SCC YAML for OpenShift deployments
    • Updated standard Kubernetes installation notes
  • Configuration Reference: Updated to reflect:
    • image field is now required (not optional)
    • Namespace clarification (uses metadata.namespace, not spec.namespace)
    • Status fields updated to use Kubernetes Conditions pattern
  • Quick Reference: Updated sbdconfig-quick-reference.md to match user guide changes

Summary by CodeRabbit

  • Documentation
    • Expanded the StorageBasedRemediationConfig quick reference and user guide with clearer namespaced configuration workflows and more complete kubectl/oc command examples (deploy/apply checks, inspection, troubleshooting, and watchdog debugging).
    • Refreshed configuration and example YAMLs (production, detect-only/no-fencing, and zone-scoped), including updated condition-based status modeling and metrics on port 8082.
    • Updated the README to recommend OLM/OperatorHub installation and added a note about current make deploy limitations and the intended fix.

@maximunited

Copy link
Copy Markdown
Contributor Author

@mshitrit - FYI

@mshitrit

Copy link
Copy Markdown
Member

I think this PR captures and documents well some of SBR critial issues, which requires workarounds or additional attention.
However most of them should be fixed for the next release (here is the Jira epic capturing them).

I think it would be a good idea either to have a separate PR (in case there are here issues which aren't pending fix for GA), or to wait until GA with this PR - in which case it might be closed or significantly modified.

@razo7 razo7 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I crawled the SBR codebase against this PR and found significant drift. mshitrit's comment was prescient as most workarounds this PR documents have been fixed since January.

What's correct and valuable

  • Changing "cluster-scoped" to "namespaced" ✅
  • Moving namespace from spec.namespace to metadata.namespace
  • Updating status from daemonSetReady boolean to Conditions pattern ✅
  • Adding sharedStorageClass and nodeSelector documentation ✅
  • OpenShift-specific oc debug commands ✅
  • Namespace placeholders in kubectl commands ✅
  • Conditions status detail (types, reasons, messages) ✅

Critical mismatches with current codebase

  1. spec.image removed from CRD — all references, examples, workarounds are wrong
  2. SCC is now auto-managed — controller creates CRB to built-in privileged SCC
  3. 9 documented fields don't exist in CRD: imagePullPolicy, staleNodeTimeout, watchdogTimeout, petIntervalMultiple, logLevel, iotimeout, rebootMethod, sbdUpdateInterval, peerCheckInterval
  4. Field name errors: sbdWatchdogPathwatchdogPath, sbdTimeoutSecondssbrTimeoutSeconds
  5. Missing real fields: maxConsecutiveFailures (2-32, default 7), detectOnlyMode (Disabled/Enabled)
  6. Mount path: /dev/sbd/dev/sbr
  7. Metric prefix: sbd_*sbr_* (context lines, not changed by this PR)
  8. Metrics port: 8080 → 8082 (context line, not changed by this PR)
  9. Repo URL: storage-base-remediationstorage-based-remediation

Given the scope of drift, I would recommend that this PR should either be substantially rewritten against the current codebase (keeping the good structural changes), or closed in favor of a fresh docs PR built from the actual CRD types @maximunited

See inline comments for specifics.

Comment thread docs/sbdconfig-user-guide.md Outdated
Comment thread docs/sbdconfig-user-guide.md Outdated
Comment thread docs/sbdconfig-user-guide.md Outdated
Comment thread docs/sbdconfig-user-guide.md Outdated
Comment thread docs/sbdconfig-user-guide.md Outdated
Comment thread docs/sbdconfig-user-guide.md Outdated
Comment thread docs/sbdconfig-user-guide.md Outdated
Comment thread docs/sbdconfig-quick-reference.md Outdated
Comment thread docs/sbdconfig-quick-reference.md Outdated
@razo7

razo7 commented Jul 12, 2026

Copy link
Copy Markdown
Member

Consider adding TODO for the README:

make deploy has known RBAC kustomize issues (hardcoded namePrefix in role bindings, missing SCC ClusterRole in config/default). The Installation section should recommend OLM instead. See PR #23 for details.

@maximunited
maximunited force-pushed the fix/docs-sbdconfig-user-guide branch from a818d18 to 2e49324 Compare July 12, 2026 15:39
@coderabbitai

coderabbitai Bot commented Jul 12, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 69f6ac81-cced-4d8a-be18-0b7fdc155c40

📥 Commits

Reviewing files that changed from the base of the PR and between 8cf875e and ecf94a0.

📒 Files selected for processing (3)
  • README.md
  • docs/sbr-config-quick-reference.md
  • docs/sbr-config-user-guide.md
🚧 Files skipped from review as they are similar to previous changes (2)
  • docs/sbr-config-quick-reference.md
  • docs/sbr-config-user-guide.md

📝 Walkthrough

Walkthrough

Updated StorageBasedRemediationConfig documentation with namespaced resource guidance, revised configuration and status references, expanded installation and management procedures, storage and metrics details, troubleshooting commands, and refreshed examples.

Changes

StorageBasedRemediationConfig documentation

Layer / File(s) Summary
Configuration and status contracts
docs/sbr-config-quick-reference.md, docs/sbr-config-user-guide.md
Documentation now describes namespaced resources, updated fields, detect-only behavior, conditions-array status, and refreshed configuration examples.
Deployment and resource management
README.md, docs/sbr-config-quick-reference.md, docs/sbr-config-user-guide.md
Installation, deployment, monitoring, deletion, naming, and multi-configuration procedures use namespace-qualified commands and describe resource ownership and cleanup.
Storage integration and observability
docs/sbr-config-quick-reference.md, docs/sbr-config-user-guide.md
Shared storage, StorageClass examples, metrics on port 8082, health checks, log analysis, and ServiceMonitor integration are updated.
Troubleshooting and operational guidance
docs/sbr-config-quick-reference.md, docs/sbr-config-user-guide.md
Watchdog, storage, tuning, security, best-practice, alerting, and reference sections include expanded diagnostics and revised operational guidance.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

Suggested labels: lgtm, approved

Suggested reviewers: clobrano

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title is clearly about the docs updates and mentions key themes like Quick Start and Known Issues, which match the changeset.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 7

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/sbr-config-quick-reference.md`:
- Around line 151-158: Update the maxConsecutiveFailures description in the
configuration reference table to state that the threshold applies to watchdog,
SBR-device, local-heartbeat, and peer-heartbeat failures, matching the CRD API
contract and user guide.
- Around line 59-61: Update the log commands in the quick-reference section to
target a single SBR configuration instead of all resources sharing the
app=sbr-agent label. Use the configuration-specific selector or resource name
established by the deployment manifests, and document the exact label contract
if selecting by label; apply the same scoping to both standard and follow-mode
commands.
- Around line 84-85: Update the “Check metrics” example in the SBR configuration
quick reference to use the documented Service/ServiceMonitor access flow with
kubectl port-forward instead of directly curling the node IP. Keep the example
consistent with the existing metrics documentation.

In `@docs/sbr-config-user-guide.md`:
- Around line 174-179: Update the maxConsecutiveFailures documentation in the
detailed configuration section and quick reference to state that the threshold
applies to SBR-device, watchdog, local-heartbeat, and peer-heartbeat failures,
not just storage I/O failures. Preserve the existing default, range, and
time-to-detection details.
- Around line 297-315: Clarify the “Multi-Cluster Configuration” example that
the two StorageBasedRemediationConfig YAML documents must be applied separately
from their corresponding clusters, or present them as distinct per-cluster
files. Preserve the existing cluster-specific names and namespaces while making
the application context explicit.
- Line 649: Update the ServiceMonitor example in the configuration guide to use
the `metrics` port name instead of `runtime-metrics`, matching the port exposed
by `config/prometheus/sbr-agent-metrics.yaml`.
- Around line 430-435: Update the “Supported StorageClass Examples” section to
clarify that Azure Files is supported only when backed by NFS v4.1 with POSIX
file-locking semantics, or remove the Azure Files entry if SMB-backed shares are
the intended configuration. Do not leave the current generic azurefile-csi
example implying all Azure Files variants are supported.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 333f9fe5-5880-42de-b723-395788ea5dc4

📥 Commits

Reviewing files that changed from the base of the PR and between b916efa and 2e49324.

📒 Files selected for processing (2)
  • docs/sbr-config-quick-reference.md
  • docs/sbr-config-user-guide.md

Comment thread docs/sbr-config-quick-reference.md Outdated
Comment thread docs/sbr-config-quick-reference.md Outdated
Comment thread docs/sbr-config-quick-reference.md
Comment thread docs/sbr-config-user-guide.md
Comment thread docs/sbr-config-user-guide.md
Comment thread docs/sbr-config-user-guide.md Outdated
Comment thread docs/sbr-config-user-guide.md Outdated
@maximunited

Copy link
Copy Markdown
Contributor Author

Following up on this — RHWA-694 (Productize SBR Phase III) is now closed/done, and the workarounds this PR originally documented (invalid default spec.image, manual SCC creation) were resolved as part of that GA-readiness work, along with several API renames (SBD → SBR, field renames) that landed afterward.

Per the option you raised — rewrite vs. close — I went with significantly modifying the PR rather than closing it: rebased onto current main, removed the resolved workarounds and stale fields, and updated the docs to match the current StorageBasedRemediationConfig CRD (see the review thread replies below for the full list of changes). The core user guide and quick reference structure was worth keeping; it was just describing a snapshot of the API from before GA hardening.

@maximunited

Copy link
Copy Markdown
Contributor Author

Added a TODO to the README, but with a correction — I verified this against current main before writing it up, and one half of PR #23's diagnosis is now stale while the other is confirmed:

  • Missing SCC ClusterRole: no longer applies. The controller now auto-creates a ClusterRoleBinding to the built-in privileged SCC at reconcile time (ensureSCCPermissions/ensureServiceAccount in storagebasedremediationconfig_controller.go), independent of kustomize. The static config/openshift/sbr-agent-scc.yaml PR Fix RBAC kustomize configuration for make deploy #23 wanted wired into config/default isn't referenced by the controller at all anymore — it's dead code from before SCC handling moved into the reconcile loop.
  • RBAC namePrefix issue: still real. I ran kustomize build config/default and confirmed leader_election_role_binding.yaml, metrics_auth_role_binding.yaml, and sbr_operator_role_binding.yaml still render their ServiceAccount subject as namespace: system instead of sbr-operator-system, because the subject name is pre-baked with the sbr-operator- prefix and no longer matches kustomize's nameReference transformer. This breaks leader election and metrics-auth RBAC under make deploy, exactly as PR Fix RBAC kustomize configuration for make deploy #23 described.

Added a TODO in the README's Installation section documenting this with the verified evidence, and pointed the primary install path at OLM instead of make deploy until it's fixed.

@maximunited

Copy link
Copy Markdown
Contributor Author

/test ci/prow/4.23-ci-bundle-my-bundle
/test ci/prow/4.23-images

@maximunited

Copy link
Copy Markdown
Contributor Author

/test 4.23-ci-bundle-my-bundle
/test 4.23-images

@maximunited

Copy link
Copy Markdown
Contributor Author

/retest

1 similar comment
@maximunited

Copy link
Copy Markdown
Contributor Author

/retest

The docs previously targeted the pre-January SBDConfig API and workarounds
that no longer apply. Update sbr-config-user-guide.md and
sbr-config-quick-reference.md to match the current
StorageBasedRemediationConfig CRD:

- Remove spec.image (removed in PR medik8s#44); image is now injected via the
  RELATED_IMAGE_AGENT env var
- Remove the "Known Issues" workarounds (invalid default image, manual SCC)
  since both were resolved upstream
- Remove manual SCC YAML; the controller auto-creates the ClusterRoleBinding
- Rename sbdWatchdogPath -> watchdogPath, drop 9 fields that no longer exist
  in the spec (imagePullPolicy, staleNodeTimeout, watchdogTimeout, etc.)
- Add missing real fields: nodeSelector, sbrTimeoutSeconds,
  maxConsecutiveFailures, detectOnlyMode
- Fix mount path (/dev/sbd -> /dev/sbr), metrics prefix (sbd_ -> sbr_) and
  port (8080 -> 8082), and a repo URL typo
- Replace daemonSetReady status field with the current conditions array
- Scope log commands to a single config via the storagebasedremediationconfig
  label instead of the shared app=sbr-agent label
- Fix metrics access example: agent has no hostNetwork/hostPort, so use a
  Service + port-forward instead of curling the node IP directly
- Clarify maxConsecutiveFailures covers watchdog, SBR-device, and
  local/peer-heartbeat failures, not just storage I/O
- Split the multi-cluster example into separate per-cluster manifests with a
  note to apply each against its own cluster context
- Narrow the Azure Files storage example to NFS v4.1-backed shares, since
  SMB-backed shares lack the required POSIX file-locking semantics
- Fix ServiceMonitor example port name (metrics, matching the Service) since
  it does not need to match the container's port name (runtime-metrics)
Verified with kustomize build config/default that
leader_election_role_binding.yaml, metrics_auth_role_binding.yaml, and
sbr_operator_role_binding.yaml still render subjects with the stale
`namespace: system` instead of `sbr-operator-system`, because their
ServiceAccount subject name is already prefixed and no longer matches
kustomize's nameReference transformer. This breaks leader election and
metrics-auth RBAC under `make deploy`, matching the diagnosis in PR medik8s#23
(closed, not merged). Point users at OLM installation instead until fixed.
@maximunited
maximunited force-pushed the fix/docs-sbdconfig-user-guide branch from fd22a10 to 8cf875e Compare July 19, 2026 15:35

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@README.md`:
- Around line 71-89: Update the numbered Quick Start installation step in
README.md so it no longer presents make deploy as the default supported path.
Direct users to the documented OLM/manifest installation procedure, or clearly
label make deploy as a development-only workaround until the RBAC issues are
fixed; keep the existing OLM guidance and installation reference consistent.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: ef5de7d2-9a2c-4806-84f6-d000ad094f69

📥 Commits

Reviewing files that changed from the base of the PR and between 2e49324 and 8cf875e.

📒 Files selected for processing (3)
  • README.md
  • docs/sbr-config-quick-reference.md
  • docs/sbr-config-user-guide.md
🚧 Files skipped from review as they are similar to previous changes (2)
  • docs/sbr-config-quick-reference.md
  • docs/sbr-config-user-guide.md

Comment thread README.md
@maximunited

Copy link
Copy Markdown
Contributor Author

CI failure analysis: transient OOM - not related to this PR

The 4.23-images and 4.23-ci-bundle-my-bundle failures are caused by the Go compiler being OOM-killed during go build:

k8s.io/client-go/informers/admissionregistration/v1beta1: compile: signal: killed

Root cause: The CI resource config requests 1000Mi memory with no limit and no CPU limit. Without a CPU limit, Go 1.26 defaults GOMAXPROCS to the node's full CPU count (~16 cores), spawning too many parallel compiler processes for the available memory.

This is random, not deterministic. The same job passes on PRs #69, #70, #63, and #74 - it depends on CI cluster load and node placement. PR #2 had bad luck: the first 4 runs hit merge conflicts (now fixed by rebase), and the remaining runs landed on memory-pressured nodes.

Other medik8s repos (node-healthcheck-operator, machine-deletion-remediation) avoid this by setting limits: memory: 4Gi in their CI config. A follow-up PR to openshift/release adding a 4Gi memory limit would make this reliable.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

@razo7 razo7 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work here @maximunited
left one NIT

Comment thread docs/sbr-config-user-guide.md Outdated
kind: StorageBasedRemediationConfig
metadata:
name: default
namespace: openshift-operators

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT: Quick Start uses namespace: openshift-operators but every other example in both the user guide and quick reference uses sbr-operator-system (or scenario-specific namespaces like sbr-dev). The kustomize default is also sbr-operator-system.

Either use sbr-operator-system here for consistency, or add a note explaining why Quick Start targets openshift-operators (e.g. OLM default install namespace).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. Changed namespace: openshift-operatorsnamespace: sbr-operator-system in the Quick Start example to match every other example in both docs files and the kustomize default.

Comment thread docs/sbr-config-quick-reference.md Outdated

# Check metrics (agent does not use hostNetwork; use a Service + port-forward,
# not a direct node-IP curl — see Monitoring and Observability)
kubectl port-forward -n <namespace> svc/sbr-agent-metrics 8082:8082

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This port-forward command will fail: the Service at config/prometheus/sbr-agent-metrics.yaml exposes port 8080 (targetPort 8080), not 8082. The agent binary listens on 8082 (DefaultMetricsPort in internal/agent/flags.go:110), so the Service manifest is the bug — but until it's fixed, this command hits a port the Service doesn't expose.

Workaround: target the DaemonSet directly instead of the Service:

kubectl port-forward -n <namespace> ds/sbr-agent-<config-name> 8082:8082

Also: docs/sbr-agent-prometheus-metrics.md (linked from this PR at the user guide) still says port 8080 in 9 places — contradicts the 8082 used here.

Tracked as RHWA-1372.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. Changed the port-forward target from svc/sbr-agent-metrics 8082:8082 (fails since the Service exposes port 8080) to ds/sbr-agent-<config-name> 8082:8082, targeting the DaemonSet directly on the actual agent port. Updated the comment to reference RHWA-1372.

Will leave docs/sbr-agent-prometheus-metrics.md for the RHWA-1372 fix since the Service manifest port is the root cause.

- Use sbr-operator-system namespace in Quick Start for consistency
  with all other examples and the kustomize default
- Target DaemonSet directly for port-forward (ds/sbr-agent-<config-name>)
  since the Service manifest exposes 8080 not 8082 (RHWA-1372)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@openshift-ci openshift-ci Bot added the lgtm label Jul 20, 2026
@openshift-ci

openshift-ci Bot commented Jul 20, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: maximunited, razo7

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-merge-bot
openshift-merge-bot Bot merged commit 9d4fa6f into medik8s:main Jul 20, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants