From de5bf239149f7d4464445f65bd76e46f8855ded0 Mon Sep 17 00:00:00 2001 From: Kasturi Narra Date: Thu, 23 Jul 2026 13:10:55 +0530 Subject: [PATCH 1/4] Add PR job analysis to LVMS doctor skill The LVMS doctor SKILL.md only instructed the LLM to analyze release jobs, skipping PR jobs entirely. This meant no prs-job-*.json files were created, so finalize never ran PR aggregation and prs-summary.json was never produced. The Pull Requests tab showed pass/fail counts but no failure detail cards. Align Step 2 with microshift-ci: analyze PR jobs alongside release jobs, save output as prs-job--pr-.json so finalize finds them and produces prs-summary.json for the report. Co-Authored-By: Claude Opus 4.6 --- plugins/lvms-ci/.claude-plugin/plugin.json | 2 +- plugins/lvms-ci/skills/doctor/SKILL.md | 30 ++++++++++++++++++---- 2 files changed, 26 insertions(+), 6 deletions(-) diff --git a/plugins/lvms-ci/.claude-plugin/plugin.json b/plugins/lvms-ci/.claude-plugin/plugin.json index ab37f590..9b85f6fa 100644 --- a/plugins/lvms-ci/.claude-plugin/plugin.json +++ b/plugins/lvms-ci/.claude-plugin/plugin.json @@ -1,7 +1,7 @@ { "name": "lvms-ci", "description": "LVMS CI Automation", - "version": "1.2.0", + "version": "1.2.1", "author": { "name": "knarra" }, diff --git a/plugins/lvms-ci/skills/doctor/SKILL.md b/plugins/lvms-ci/skills/doctor/SKILL.md index 5c0e2cd2..733f905e 100644 --- a/plugins/lvms-ci/skills/doctor/SKILL.md +++ b/plugins/lvms-ci/skills/doctor/SKILL.md @@ -48,7 +48,7 @@ Compute once at the start by running `date +%y%m%d` and substituting into the pa 3. The script deterministically: - For each release: fetches failed periodic jobs, downloads artifacts, writes `/jobs/release--jobs.json` - - Collects open z-stream PRs targeting `release-management` and fetches their presubmit results + - For PRs: fetches PRs with failures, downloads artifacts, writes `/jobs/prs-jobs.json` and `/jobs/prs-status.json` - Outputs a JSON summary listing all releases, job counts, and file paths 4. Read the JSON output to know which releases have jobs to analyze and how many @@ -59,6 +59,7 @@ Compute once at the start by running `date +%y%m%d` and substituting into the pa - `artifacts_dir` — local path to downloaded artifacts - `url` — Prow job URL - `status` — job result (`failure`, `FAILURE`, `SUCCESS`, `PENDING`) +- `pr_number` — PR number (PR jobs only) **Error Handling**: @@ -73,7 +74,9 @@ Compute once at the start by running `date +%y%m%d` and substituting into the pa **Actions**: 1. Use the JSON summary output from Step 1 to build agent prompts. Do NOT read the job JSON files into the main conversation — the prepare script already printed all job details (artifacts_dir, build_id, job name) and agents receive artifacts_dir directly in their prompt. -2. For **every** failed job across all releases, launch a separate **Agent** (using the `Agent` tool, NOT the `Skill` tool): +2. For **every** failed job across all releases and PRs, launch a separate **Agent** (using the `Agent` tool, NOT the `Skill` tool). For PR jobs, only launch agents for jobs with FAILURE status. + + **For release jobs:** ```text Agent: subagent_type=general_purpose, prompt="Analyze this Prow job and save the report: @@ -84,7 +87,22 @@ Compute once at the start by running `date +%y%m%d` and substituting into the pa Use the Write tool. The file must contain ONLY the valid JSON array — no prose, no markers." ``` -3. Launch **ALL** agents in a **single message** as **foreground** agents (do NOT use `run_in_background`). Foreground agents in the same message run concurrently — this is just as fast as background agents but keeps your turn active until all complete. + **For PR jobs:** + + ```text + Agent: subagent_type=general_purpose, prompt="Analyze this Prow job and save the report: + 1. Run /lvms-ci:prow-job + 2. After the analysis completes, extract only the JSON array from the output + and save it to: + /jobs/prs-job--pr-.json + Use the Write tool. The file must contain ONLY the valid JSON array — no prose, no markers." + ``` + + After each agent completes, save its JSON response to the corresponding file using the Write tool: + - Release jobs: `/jobs/release--job--.json` + - PR jobs: `/jobs/prs-job--pr-.json` + +3. Launch **ALL** agents (all releases + PRs) in a **single message** as **foreground** agents (do NOT use `run_in_background`). Foreground agents in the same message run concurrently — this is just as fast as background agents but keeps your turn active until all complete. 4. Say "Analyzing N jobs in parallel..." in your message text alongside the Agent tool calls. 5. When all agents return, immediately proceed to Step 3 in the same turn. Do NOT stop or end your turn between Step 2 and Step 3. @@ -103,7 +121,7 @@ Compute once at the start by running `date +%y%m%d` and substituting into the pa ``` 2. The script deterministically: - - Runs `aggregate.py` for each release → `summary.json` files + - Runs `aggregate.py` for each release and for PRs → `summary.json` files - Runs `create-report.py` → `report-lvm-operator-ci-doctor.html` 3. Report the script's output to the user @@ -112,7 +130,7 @@ Compute once at the start by running `date +%y%m%d` and substituting into the pa **Actions**: 1. Display the path to the generated HTML file -2. Summarize: failed job counts per release +2. Summarize: failed job counts per release, PR status **Example Output**: @@ -121,6 +139,8 @@ Summary: Periodics: Release main: 3 failed periodic jobs Release 4.22: 0 failed periodic jobs + Pull Requests: + 4 PRs with 6 total failed jobs HTML report generated: /report-lvm-operator-ci-doctor.html ``` From 461770c55d5d6b01a93fe03eb06c5a0dec1322f4 Mon Sep 17 00:00:00 2001 From: Kasturi Narra Date: Thu, 23 Jul 2026 13:19:53 +0530 Subject: [PATCH 2/4] Bump lvms-ci version to 1.2.1 in marketplace.json Match the plugin.json version bump to fix check-version-bump CI. Co-Authored-By: Claude Opus 4.6 --- .claude-plugin/marketplace.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.claude-plugin/marketplace.json b/.claude-plugin/marketplace.json index 59859dab..fbab4fbb 100644 --- a/.claude-plugin/marketplace.json +++ b/.claude-plugin/marketplace.json @@ -58,7 +58,7 @@ "name": "lvms-ci", "source": "./plugins/lvms-ci", "description": "LVMS CI Automation", - "version": "1.2.0" + "version": "1.2.1" }, { "name": "mcp-atlassian", From d2a0523762c326b747adcd76b31a141762e55c76 Mon Sep 17 00:00:00 2001 From: Kasturi Narra Date: Thu, 23 Jul 2026 16:34:37 +0530 Subject: [PATCH 3/4] Fix PR status filter and summary file names in doctor SKILL.md Accept both failure and FAILURE status spellings when filtering PR jobs. Correct finalize documentation to name actual output files: release--summary.json and prs-summary.json. Co-Authored-By: Claude Opus 4.6 --- plugins/lvms-ci/skills/doctor/SKILL.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/plugins/lvms-ci/skills/doctor/SKILL.md b/plugins/lvms-ci/skills/doctor/SKILL.md index 733f905e..fcb827d8 100644 --- a/plugins/lvms-ci/skills/doctor/SKILL.md +++ b/plugins/lvms-ci/skills/doctor/SKILL.md @@ -74,7 +74,7 @@ Compute once at the start by running `date +%y%m%d` and substituting into the pa **Actions**: 1. Use the JSON summary output from Step 1 to build agent prompts. Do NOT read the job JSON files into the main conversation — the prepare script already printed all job details (artifacts_dir, build_id, job name) and agents receive artifacts_dir directly in their prompt. -2. For **every** failed job across all releases and PRs, launch a separate **Agent** (using the `Agent` tool, NOT the `Skill` tool). For PR jobs, only launch agents for jobs with FAILURE status. +2. For **every** failed job across all releases and PRs, launch a separate **Agent** (using the `Agent` tool, NOT the `Skill` tool). For PR jobs, only analyze jobs whose status is `failure` or `FAILURE` (exclude `SUCCESS` and `PENDING`). **For release jobs:** @@ -121,7 +121,7 @@ Compute once at the start by running `date +%y%m%d` and substituting into the pa ``` 2. The script deterministically: - - Runs `aggregate.py` for each release and for PRs → `summary.json` files + - Runs `aggregate.py` for each release → `release--summary.json`, and for PRs → `prs-summary.json` - Runs `create-report.py` → `report-lvm-operator-ci-doctor.html` 3. Report the script's output to the user From 6d9e7b9593924b8f9b654b64584dfa72dc30d23d Mon Sep 17 00:00:00 2001 From: Kasturi Narra Date: Thu, 23 Jul 2026 16:56:35 +0530 Subject: [PATCH 4/4] Revert SKILL.md wording to stay aligned with microshift-ci Co-Authored-By: Claude Opus 4.6 --- plugins/lvms-ci/skills/doctor/SKILL.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/plugins/lvms-ci/skills/doctor/SKILL.md b/plugins/lvms-ci/skills/doctor/SKILL.md index fcb827d8..733f905e 100644 --- a/plugins/lvms-ci/skills/doctor/SKILL.md +++ b/plugins/lvms-ci/skills/doctor/SKILL.md @@ -74,7 +74,7 @@ Compute once at the start by running `date +%y%m%d` and substituting into the pa **Actions**: 1. Use the JSON summary output from Step 1 to build agent prompts. Do NOT read the job JSON files into the main conversation — the prepare script already printed all job details (artifacts_dir, build_id, job name) and agents receive artifacts_dir directly in their prompt. -2. For **every** failed job across all releases and PRs, launch a separate **Agent** (using the `Agent` tool, NOT the `Skill` tool). For PR jobs, only analyze jobs whose status is `failure` or `FAILURE` (exclude `SUCCESS` and `PENDING`). +2. For **every** failed job across all releases and PRs, launch a separate **Agent** (using the `Agent` tool, NOT the `Skill` tool). For PR jobs, only launch agents for jobs with FAILURE status. **For release jobs:** @@ -121,7 +121,7 @@ Compute once at the start by running `date +%y%m%d` and substituting into the pa ``` 2. The script deterministically: - - Runs `aggregate.py` for each release → `release--summary.json`, and for PRs → `prs-summary.json` + - Runs `aggregate.py` for each release and for PRs → `summary.json` files - Runs `create-report.py` → `report-lvm-operator-ci-doctor.html` 3. Report the script's output to the user