diff --git a/.changeset/fix-false-positive-ready-to-merge-1480.md b/.changeset/fix-false-positive-ready-to-merge-1480.md new file mode 100644 index 000000000..241baea74 --- /dev/null +++ b/.changeset/fix-false-positive-ready-to-merge-1480.md @@ -0,0 +1,5 @@ +--- +'@link-assistant/hive-mind': patch +--- + +fix false positive 'Ready to merge' by adding workflow run grace period (Issue #1480) diff --git a/docs/case-studies/issue-1480/README.md b/docs/case-studies/issue-1480/README.md new file mode 100644 index 000000000..d44f127a5 --- /dev/null +++ b/docs/case-studies/issue-1480/README.md @@ -0,0 +1,158 @@ +# Case Study: Issue #1480 - `Ready to merge` posted as false positive + +## Summary + +The auto-restart-until-mergeable monitor posted a "Ready to merge" comment on PR #1479 +approximately 19 seconds after the last commit was pushed, while CI workflows had not yet +been registered in the GitHub Actions API. CI later started and failed (lint, +check-file-line-limits). + +## Timeline + +| Time (UTC) | Event | +| ---------- | ----------------------------------------------------------------- | +| 09:53:33 | Last commit `9ed29b7` pushed (Revert "Initial commit") | +| ~09:53:50 | Auto-merge monitor runs `getMergeBlockers()` | +| ~09:53:50 | `getDetailedCIStatus()` returns `no_checks` | +| ~09:53:50 | `checkPRMergeable()` returns `mergeable: true` | +| ~09:53:50 | `getActiveRepoWorkflows()` returns `hasWorkflows: true` | +| ~09:53:50 | `getWorkflowRunsForSha()` returns **0 runs** (not yet registered) | +| ~09:53:50 | Code concludes: "CI was definitively NOT triggered" | +| 09:53:52 | "Ready to merge" comment posted (FALSE POSITIVE) | +| 09:55:18 | CI workflow actually starts running | +| 09:56:11 | CI finishes — `lint` and `check-file-line-limits` FAIL | + +## Root Cause + +The fix for issue #1442 added `getWorkflowRunsForSha()` to distinguish between: + +- Workflow runs triggered but check-runs not yet registered (race condition) +- CI not triggered at all (fork PR, `paths-ignore`, etc.) + +The assumption was: if `workflow_runs` API returns 0 runs, CI was "definitively NOT +triggered." But this assumption is flawed — **GitHub Actions workflow runs also take time +to appear in the API after a push** (typically 30-120 seconds). + +The race condition timeline: + +``` +Push → (0-30s) → Workflow runs appear in API → (0-30s) → Check-runs appear in API +``` + +The code only protected against the second race (workflow runs exist but check-runs don't) +but not the first race (workflow runs themselves don't exist yet). + +## GitHub API Data Sources for CI/CD Status + +The fix collects data from all available GitHub API endpoints: + +### 1. Check Runs API (`GET /repos/{owner}/{repo}/commits/{sha}/check-runs`) + +Returns check-runs created by GitHub Actions jobs. These appear **after** a workflow run +starts executing a job. Fields: `status` (queued/in_progress/completed), `conclusion` +(success/failure/cancelled/etc.). + +### 2. Commit Statuses API (`GET /repos/{owner}/{repo}/commits/{sha}/status`) + +Returns legacy commit statuses (used by third-party CI systems like Jenkins, CircleCI). +Fields: `state` (pending/success/failure/error), `context` (name). + +### 3. Workflow Runs API (`GET /repos/{owner}/{repo}/actions/runs?head_sha={sha}`) + +Returns GitHub Actions workflow runs triggered for a specific commit SHA. Appears **before** +check-runs but **after** the GitHub Actions scheduler processes the event (30-120s delay). + +### 4. Active Workflows API (`GET /repos/{owner}/{repo}/actions/workflows`) + +Lists all workflows configured in the repo with their state (active/disabled). Filtered +to exclude GitHub Pages deployment workflows which only run on default branch. + +### 5. Repository Content API (`GET /repos/{owner}/{repo}/contents/.github/workflows`) + +Parses actual workflow YAML files to detect PR-related triggers (`on: pull_request`, +`on: push`, `on: pull_request_target`). Provides ground-truth about what CI **should** run. + +### 6. PR Commits API (`GET /repos/{owner}/{repo}/pulls/{number}/commits`) + +Lists all commits in a PR. Used to check if previous commits had CI workflow runs, +which is a signal that the HEAD commit should also have CI. + +### 7. Commit API (`GET /repos/{owner}/{repo}/commits/{sha}`) + +Returns commit metadata including `commit.committer.date`, used to calculate commit age +for the grace period check. + +## Fix: Multi-Layer Defense + +Instead of immediately concluding "CI not triggered" when `getWorkflowRunsForSha()` returns +0 runs, the code now uses a multi-layer defense: + +### Layer 1: Empty Workflows Folder Detection + +If `.github/workflows/` doesn't exist or has no `.yml`/`.yaml` files, immediately conclude +"no CI configured at file level" — no grace period needed. + +### Layer 2: Grace Period (120 seconds) + +Check the age of the HEAD commit. If pushed within the last 120 seconds, treat as a +potential race condition and return `ci_pending` blocker. Additionally report workflow +file PR triggers as context. + +### Layer 3: Previous Commit CI History + +After grace period elapses, check if earlier commits in the same PR had workflow runs. +If previous commits had CI AND workflow files have PR triggers, it's likely a GitHub API +delay — wait one more cycle as safety measure. + +### Layer 4: Definitive Conclusion + +Only conclude "CI not triggered" when ALL conditions are met: + +- Grace period (120s) has elapsed +- No workflow runs for HEAD SHA +- Previous PR commits did NOT have CI, OR workflow files don't have PR triggers + +## Code Path (after fix) + +``` +getMergeBlockers() → src/solve.auto-merge.lib.mjs:190 +├── getDetailedCIStatus() → returns no_checks +├── checkPRMergeable() → returns mergeable: true +├── getActiveRepoWorkflows() → returns hasWorkflows: true +├── getWorkflowRunsForSha() → returns [] (0 runs for HEAD SHA) +│ ├── checkWorkflowsHavePRTriggers() → parse .github/workflows/*.yml +│ │ ├── hasWorkflowFiles: false → immediate "no CI" conclusion +│ │ └── hasWorkflowFiles: true → +│ ├── getCommitDate() → check commit age +│ │ ├── < 120s → ci_pending blocker (Layer 2: grace period) +│ │ └── >= 120s → +│ │ ├── checkPreviousPRCommitsHadCI() → check earlier commits +│ │ │ ├── had CI + has PR triggers → ci_pending (Layer 3: safety) +│ │ │ └── no previous CI or no triggers → noCiTriggered: true +│ │ └── (Layer 4: definitive conclusion) +``` + +## Related Issues + +- Issue #1442: No CI timeout handling (introduced the flawed `getWorkflowRunsForSha` check) +- Issue #1466: Auto-restart stuck with `action_required` workflows +- Issue #1363: False positive "ready to merge" for repos with no required checks +- Issue #1345: Differentiate "no CI configured" vs "CI not triggered" +- Issue #1425: False positive "failed CI" detection + +## Evidence + +- False positive comment: https://github.com/link-assistant/hive-mind/pull/1479#issuecomment-4125186525 +- CI run showing failures: PR #1479 statusCheckRollup shows lint FAILURE at 09:56:11 +- Solution draft log: https://github.com/link-assistant/hive-mind/pull/1479#issuecomment-4125185439 +- PR #1479 details with full CI timeline: `data/pr-1479-details.json` +- PR #1479 comments: `data/pr-1479-all-comments.txt` + +## GitHub API Documentation References + +- [Check Runs API](https://docs.github.com/en/rest/checks/runs) +- [Commit Statuses API](https://docs.github.com/en/rest/commits/statuses) +- [Workflow Runs API](https://docs.github.com/en/rest/actions/workflow-runs) +- [List Repository Workflows](https://docs.github.com/en/rest/actions/workflows) +- [Get Repository Content](https://docs.github.com/en/rest/repos/contents) +- [List PR Commits](https://docs.github.com/en/rest/pulls/pulls#list-commits-on-a-pull-request) diff --git a/docs/case-studies/issue-1480/data/pr-1479-all-comments.txt b/docs/case-studies/issue-1480/data/pr-1479-all-comments.txt new file mode 100644 index 000000000..43a8a9612 --- /dev/null +++ b/docs/case-studies/issue-1480/data/pr-1479-all-comments.txt @@ -0,0 +1,41 @@ +author: konard +association: member +edited: false +status: none +-- +## 🤖 Solution Draft Log +This log file contains the complete execution trace of the AI solution draft process. + +### 💰 **Cost estimation:** +- Public pricing estimate: $3.537043 +- Calculated by Anthropic: $2.980201 USD +- Difference: $-0.556842 (-15.74%) + +### 🤖 **Models used:** +- Tool: Anthropic Claude Code +- Requested: `opus` +- **Main model: Claude Opus 4.6** (`claude-opus-4-6`) +- **Additional models:** + * **Claude Haiku 4.5** (`claude-haiku-4-5-20251001`) + +### 📎 **Log file uploaded as Gist** (1592KB) +- [View complete solution draft log](https://gist.githubusercontent.com/konard/952725c5f9962321f19afeb714c83d0e/raw/93577632c068860a7bae2eeae4fea85f39bd61dd/solution-draft-log-pr-1774432416684.txt) + +--- +*Now working session is ended, feel free to review and add any feedback on the solution draft.* +-- +author: konard +association: member +edited: false +status: none +-- +## ✅ Ready to merge + +This pull request is now ready to be merged: +- CI workflows exist but were not triggered for this commit +- No merge conflicts +- No pending changes + +--- +*Monitored by hive-mind with --auto-restart-until-mergeable flag* +-- diff --git a/docs/case-studies/issue-1480/data/pr-1479-ci-runs.json b/docs/case-studies/issue-1480/data/pr-1479-ci-runs.json new file mode 100644 index 000000000..fe51488c7 --- /dev/null +++ b/docs/case-studies/issue-1480/data/pr-1479-ci-runs.json @@ -0,0 +1 @@ +[] diff --git a/docs/case-studies/issue-1480/data/pr-1479-comments.json b/docs/case-studies/issue-1480/data/pr-1479-comments.json new file mode 100644 index 000000000..7a18b41c3 --- /dev/null +++ b/docs/case-studies/issue-1480/data/pr-1479-comments.json @@ -0,0 +1,30 @@ +[ + { + "url": "https://api.github.com/repos/link-assistant/hive-mind/issues/comments/4125185439", + "html_url": "https://github.com/link-assistant/hive-mind/pull/1479#issuecomment-4125185439", + "issue_url": "https://api.github.com/repos/link-assistant/hive-mind/issues/1479", + "id": 4125185439, + "node_id": "IC_kwDOPUU0qc714VWf", + "user": { "login": "konard", "id": 1431904, "node_id": "MDQ6VXNlcjE0MzE5MDQ=", "avatar_url": "https://avatars.githubusercontent.com/u/1431904?v=4", "gravatar_id": "", "url": "https://api.github.com/users/konard", "html_url": "https://github.com/konard", "followers_url": "https://api.github.com/users/konard/followers", "following_url": "https://api.github.com/users/konard/following{/other_user}", "gists_url": "https://api.github.com/users/konard/gists{/gist_id}", "starred_url": "https://api.github.com/users/konard/starred{/owner}{/repo}", "subscriptions_url": "https://api.github.com/users/konard/subscriptions", "organizations_url": "https://api.github.com/users/konard/orgs", "repos_url": "https://api.github.com/users/konard/repos", "events_url": "https://api.github.com/users/konard/events{/privacy}", "received_events_url": "https://api.github.com/users/konard/received_events", "type": "User", "user_view_type": "public", "site_admin": false }, + "created_at": "2026-03-25T09:53:41Z", + "updated_at": "2026-03-25T09:53:41Z", + "body": "## 🤖 Solution Draft Log\nThis log file contains the complete execution trace of the AI solution draft process.\n\n### 💰 **Cost estimation:**\n- Public pricing estimate: $3.537043\n- Calculated by Anthropic: $2.980201 USD\n- Difference: $-0.556842 (-15.74%)\n\n### 🤖 **Models used:**\n- Tool: Anthropic Claude Code\n- Requested: `opus`\n- **Main model: Claude Opus 4.6** (`claude-opus-4-6`)\n- **Additional models:**\n * **Claude Haiku 4.5** (`claude-haiku-4-5-20251001`)\n\n### 📎 **Log file uploaded as Gist** (1592KB)\n- [View complete solution draft log](https://gist.githubusercontent.com/konard/952725c5f9962321f19afeb714c83d0e/raw/93577632c068860a7bae2eeae4fea85f39bd61dd/solution-draft-log-pr-1774432416684.txt)\n\n---\n*Now working session is ended, feel free to review and add any feedback on the solution draft.*", + "author_association": "MEMBER", + "reactions": { "url": "https://api.github.com/repos/link-assistant/hive-mind/issues/comments/4125185439/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 }, + "performed_via_github_app": null + }, + { + "url": "https://api.github.com/repos/link-assistant/hive-mind/issues/comments/4125186525", + "html_url": "https://github.com/link-assistant/hive-mind/pull/1479#issuecomment-4125186525", + "issue_url": "https://api.github.com/repos/link-assistant/hive-mind/issues/1479", + "id": 4125186525, + "node_id": "IC_kwDOPUU0qc714Vnd", + "user": { "login": "konard", "id": 1431904, "node_id": "MDQ6VXNlcjE0MzE5MDQ=", "avatar_url": "https://avatars.githubusercontent.com/u/1431904?v=4", "gravatar_id": "", "url": "https://api.github.com/users/konard", "html_url": "https://github.com/konard", "followers_url": "https://api.github.com/users/konard/followers", "following_url": "https://api.github.com/users/konard/following{/other_user}", "gists_url": "https://api.github.com/users/konard/gists{/gist_id}", "starred_url": "https://api.github.com/users/konard/starred{/owner}{/repo}", "subscriptions_url": "https://api.github.com/users/konard/subscriptions", "organizations_url": "https://api.github.com/users/konard/orgs", "repos_url": "https://api.github.com/users/konard/repos", "events_url": "https://api.github.com/users/konard/events{/privacy}", "received_events_url": "https://api.github.com/users/konard/received_events", "type": "User", "user_view_type": "public", "site_admin": false }, + "created_at": "2026-03-25T09:53:52Z", + "updated_at": "2026-03-25T09:53:52Z", + "body": "## ✅ Ready to merge\n\nThis pull request is now ready to be merged:\n- CI workflows exist but were not triggered for this commit\n- No merge conflicts\n- No pending changes\n\n---\n*Monitored by hive-mind with --auto-restart-until-mergeable flag*", + "author_association": "MEMBER", + "reactions": { "url": "https://api.github.com/repos/link-assistant/hive-mind/issues/comments/4125186525/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 }, + "performed_via_github_app": null + } +] diff --git a/docs/case-studies/issue-1480/data/pr-1479-details.json b/docs/case-studies/issue-1480/data/pr-1479-details.json new file mode 100644 index 000000000..8a17b411d --- /dev/null +++ b/docs/case-studies/issue-1480/data/pr-1479-details.json @@ -0,0 +1,67 @@ +{ + "body": "## Summary\n\nFixes #1478\n\nThe `gh pr create` command in `solve.auto-pr.lib.mjs` could fail with transient GitHub server errors (e.g., `\"Something went wrong while executing your query\"`) during service disruptions, causing the entire solve session to abort without any retry attempt.\n\n### Root Cause\n\nOn 2026-03-24, a confirmed GitHub service disruption caused the `gh pr create` GraphQL mutation to return a 500-class error. The Cyrillic characters in the PR title (\"update враг снайпер\") were initially suspected but ruled out — GitHub's API fully supports UTF-8/Cyrillic in PR titles.\n\nThe code already had retry logic for:\n- Compare API eventual consistency (lines 571-624)\n- PR verification eventual consistency (lines 1206-1259)\n\nBut the `gh pr create` command itself had **no retry logic** for transient server errors.\n\n### Changes\n\n- **`src/solve.auto-pr.lib.mjs`**: Added retry loop with exponential backoff (3 attempts, 5s/10s/15s delays) for transient GitHub API errors during PR creation. Non-transient errors (auth, validation, \"No commits between\") still fail immediately.\n- **`tests/test-pr-creation-retry-1478.mjs`**: 24 tests covering transient error detection, retry behavior, backoff timing, and the original bug scenario.\n- **`docs/case-studies/issue-1478/`**: Full root cause analysis with timeline, evidence, and references to GitHub status page and cli/cli issues.\n- **`.changeset/pr-creation-retry-1478.md`**: Changeset for patch release.\n\n## Test plan\n\n- [x] All 24 new tests pass (`node tests/test-pr-creation-retry-1478.mjs`)\n- [x] Transient errors (GraphQL 500, 502, 503, 504, network errors) are correctly detected\n- [x] Non-transient errors (auth, validation) are NOT retried\n- [x] Retry backoff follows correct timing (5s, 10s, 15s)\n- [x] Original bug scenario (exact error message from issue) triggers retry and succeeds\n\n🤖 Generated with [Claude Code](https://claude.com/claude-code)", + "commits": [ + { "authoredDate": "2026-03-25T09:45:13Z", "authors": [{ "email": "drakonard@gmail.com", "id": "MDQ6VXNlcjE0MzE5MDQ=", "login": "konard", "name": "konard" }], "committedDate": "2026-03-25T09:45:13Z", "messageBody": "Adding .gitkeep for PR creation (default mode).\nThis file will be removed when the task is complete.\n\nIssue: https://github.com/link-assistant/hive-mind/issues/1478", "messageHeadline": "Initial commit with task details", "oid": "f95316688c37939863a5d4b115d90c2936084b64" }, + { + "authoredDate": "2026-03-25T09:50:44Z", + "authors": [ + { "email": "drakonard@gmail.com", "id": "MDQ6VXNlcjE0MzE5MDQ=", "login": "konard", "name": "konard" }, + { "email": "noreply@anthropic.com", "id": "MDQ6VXNlcjgxODQ3", "login": "claude", "name": "Claude Opus 4.6" } + ], + "committedDate": "2026-03-25T09:50:44Z", + "messageBody": "…ssue #1478)\n\nThe `gh pr create` command could fail with transient GitHub server errors\n(e.g., \"Something went wrong while executing your query\") during GitHub\nservice disruptions. The code had no retry mechanism for these errors,\ncausing immediate failure. This adds exponential backoff retry (up to 3\nattempts with 5s/10s/15s delays) for transient server errors, following\nthe same pattern used for compare API and PR verification retries.\nAlso adds case study documentation with root cause analysis.\n\nCo-Authored-By: Claude Opus 4.6 ", + "messageHeadline": "add retry logic for transient GitHub API errors during PR creation (I…", + "oid": "017c171a86148b23cd1d7dae8a824570180bef10" + }, + { + "authoredDate": "2026-03-25T09:51:53Z", + "authors": [ + { "email": "drakonard@gmail.com", "id": "MDQ6VXNlcjE0MzE5MDQ=", "login": "konard", "name": "konard" }, + { "email": "noreply@anthropic.com", "id": "MDQ6VXNlcjgxODQ3", "login": "claude", "name": "Claude Opus 4.6" } + ], + "committedDate": "2026-03-25T09:51:53Z", + "messageBody": "24 tests covering:\n- Transient error detection (GraphQL errors, 502/503/504, network errors)\n- Non-transient error exclusion (auth, validation, permissions)\n- Retry logic with exponential backoff (5s, 10s, 15s)\n- Original bug scenario reproduction\n\nCo-Authored-By: Claude Opus 4.6 ", + "messageHeadline": "add tests for PR creation transient error retry logic (Issue #1478)", + "oid": "c6aeef77a86a140cd9ca79973c105e8dccffb425" + }, + { + "authoredDate": "2026-03-25T09:52:37Z", + "authors": [ + { "email": "drakonard@gmail.com", "id": "MDQ6VXNlcjE0MzE5MDQ=", "login": "konard", "name": "konard" }, + { "email": "noreply@anthropic.com", "id": "MDQ6VXNlcjgxODQ3", "login": "claude", "name": "Claude Opus 4.6" } + ], + "committedDate": "2026-03-25T09:52:37Z", + "messageBody": "Co-Authored-By: Claude Opus 4.6 ", + "messageHeadline": "add changeset for PR creation retry fix (Issue #1478)", + "oid": "185a6a0136c368167f5228c7ff51f15e97d5b354" + }, + { "authoredDate": "2026-03-25T09:53:33Z", "authors": [{ "email": "drakonard@gmail.com", "id": "MDQ6VXNlcjE0MzE5MDQ=", "login": "konard", "name": "konard" }], "committedDate": "2026-03-25T09:53:33Z", "messageBody": "This reverts commit f95316688c37939863a5d4b115d90c2936084b64.", "messageHeadline": "Revert \"Initial commit with task details\"", "oid": "9ed29b704cda40ed1680ffaf662946b82842b647" } + ], + "headRefOid": "9ed29b704cda40ed1680ffaf662946b82842b647", + "mergedAt": null, + "state": "OPEN", + "statusCheckRollup": [ + { "__typename": "CheckRun", "completedAt": "2026-03-25T09:55:32Z", "conclusion": "SUCCESS", "detailsUrl": "https://github.com/link-assistant/hive-mind/actions/runs/23535079153/job/68508114680", "name": "detect-changes", "startedAt": "2026-03-25T09:55:20Z", "status": "COMPLETED", "workflowName": "Checks and release" }, + { "__typename": "CheckRun", "completedAt": "2026-03-25T09:55:18Z", "conclusion": "SKIPPED", "detailsUrl": "https://github.com/link-assistant/hive-mind/actions/runs/23535079153/job/68508115035", "name": "Instant Release", "startedAt": "2026-03-25T09:55:18Z", "status": "COMPLETED", "workflowName": "Checks and release" }, + { "__typename": "CheckRun", "completedAt": "2026-03-25T09:55:18Z", "conclusion": "SKIPPED", "detailsUrl": "https://github.com/link-assistant/hive-mind/actions/runs/23535079153/job/68508115109", "name": "Create Changeset PR", "startedAt": "2026-03-25T09:55:18Z", "status": "COMPLETED", "workflowName": "Checks and release" }, + { "__typename": "CheckRun", "completedAt": "2026-03-25T09:55:45Z", "conclusion": "SUCCESS", "detailsUrl": "https://github.com/link-assistant/hive-mind/actions/runs/23535079153/job/68508146469", "name": "Check for Manual Version Changes", "startedAt": "2026-03-25T09:55:34Z", "status": "COMPLETED", "workflowName": "Checks and release" }, + { "__typename": "CheckRun", "completedAt": "2026-03-25T09:55:53Z", "conclusion": "SUCCESS", "detailsUrl": "https://github.com/link-assistant/hive-mind/actions/runs/23535079153/job/68508146459", "name": "Check for Changesets", "startedAt": "2026-03-25T09:55:35Z", "status": "COMPLETED", "workflowName": "Checks and release" }, + { "__typename": "CheckRun", "completedAt": "2026-03-25T09:56:11Z", "conclusion": "FAILURE", "detailsUrl": "https://github.com/link-assistant/hive-mind/actions/runs/23535079153/job/68508146435", "name": "lint", "startedAt": "2026-03-25T09:55:35Z", "status": "COMPLETED", "workflowName": "Checks and release" }, + { "__typename": "CheckRun", "completedAt": "2026-03-25T09:55:18Z", "conclusion": "SKIPPED", "detailsUrl": "https://github.com/link-assistant/hive-mind/actions/runs/23535079153/job/68508115276", "name": "Docker Publish Instant (${{ matrix.platform }})", "startedAt": "2026-03-25T09:55:18Z", "status": "COMPLETED", "workflowName": "Checks and release" }, + { "__typename": "CheckRun", "completedAt": "2026-03-25T09:56:08Z", "conclusion": "SUCCESS", "detailsUrl": "https://github.com/link-assistant/hive-mind/actions/runs/23535079153/job/68508193015", "name": "test-compilation", "startedAt": "2026-03-25T09:55:55Z", "status": "COMPLETED", "workflowName": "Checks and release" }, + { "__typename": "CheckRun", "completedAt": "2026-03-25T09:56:04Z", "conclusion": "FAILURE", "detailsUrl": "https://github.com/link-assistant/hive-mind/actions/runs/23535079153/job/68508192960", "name": "check-file-line-limits", "startedAt": "2026-03-25T09:55:55Z", "status": "COMPLETED", "workflowName": "Checks and release" }, + { "__typename": "CheckRun", "completedAt": "2026-03-25T09:56:09Z", "conclusion": "SUCCESS", "detailsUrl": "https://github.com/link-assistant/hive-mind/actions/runs/23535079153/job/68508192967", "name": "validate-docs", "startedAt": "2026-03-25T09:55:55Z", "status": "COMPLETED", "workflowName": "Checks and release" }, + { "__typename": "CheckRun", "completedAt": "2026-03-25T09:55:53Z", "conclusion": "SKIPPED", "detailsUrl": "https://github.com/link-assistant/hive-mind/actions/runs/23535079153/job/68508193469", "name": "helm-pr-check", "startedAt": "2026-03-25T09:55:53Z", "status": "COMPLETED", "workflowName": "Checks and release" }, + { "__typename": "CheckRun", "completedAt": "2026-03-25T09:55:18Z", "conclusion": "SKIPPED", "detailsUrl": "https://github.com/link-assistant/hive-mind/actions/runs/23535079153/job/68508115160", "name": "Docker Publish Instant (Merge)", "startedAt": "2026-03-25T09:55:18Z", "status": "COMPLETED", "workflowName": "Checks and release" }, + { "__typename": "CheckRun", "completedAt": "2026-03-25T09:56:11Z", "conclusion": "SKIPPED", "detailsUrl": "https://github.com/link-assistant/hive-mind/actions/runs/23535079153/job/68508231890", "name": "test-suites", "startedAt": "2026-03-25T09:56:11Z", "status": "COMPLETED", "workflowName": "Checks and release" }, + { "__typename": "CheckRun", "completedAt": "2026-03-25T09:56:11Z", "conclusion": "SKIPPED", "detailsUrl": "https://github.com/link-assistant/hive-mind/actions/runs/23535079153/job/68508231889", "name": "test-execution", "startedAt": "2026-03-25T09:56:11Z", "status": "COMPLETED", "workflowName": "Checks and release" }, + { "__typename": "CheckRun", "completedAt": "2026-03-25T09:56:11Z", "conclusion": "SKIPPED", "detailsUrl": "https://github.com/link-assistant/hive-mind/actions/runs/23535079153/job/68508231763", "name": "memory-check-linux", "startedAt": "2026-03-25T09:56:11Z", "status": "COMPLETED", "workflowName": "Checks and release" }, + { "__typename": "CheckRun", "completedAt": "2026-03-25T09:56:11Z", "conclusion": "SKIPPED", "detailsUrl": "https://github.com/link-assistant/hive-mind/actions/runs/23535079153/job/68508231606", "name": "docker-pr-check", "startedAt": "2026-03-25T09:56:11Z", "status": "COMPLETED", "workflowName": "Checks and release" }, + { "__typename": "CheckRun", "completedAt": "2026-03-25T09:55:18Z", "conclusion": "SKIPPED", "detailsUrl": "https://github.com/link-assistant/hive-mind/actions/runs/23535079153/job/68508115173", "name": "Helm Release (Instant)", "startedAt": "2026-03-25T09:55:18Z", "status": "COMPLETED", "workflowName": "Checks and release" }, + { "__typename": "CheckRun", "completedAt": "2026-03-25T09:56:11Z", "conclusion": "SKIPPED", "detailsUrl": "https://github.com/link-assistant/hive-mind/actions/runs/23535079153/job/68508232034", "name": "Release", "startedAt": "2026-03-25T09:56:11Z", "status": "COMPLETED", "workflowName": "Checks and release" }, + { "__typename": "CheckRun", "completedAt": "2026-03-25T09:56:11Z", "conclusion": "SKIPPED", "detailsUrl": "https://github.com/link-assistant/hive-mind/actions/runs/23535079153/job/68508231930", "name": "Docker Publish (${{ matrix.platform }})", "startedAt": "2026-03-25T09:56:11Z", "status": "COMPLETED", "workflowName": "Checks and release" }, + { "__typename": "CheckRun", "completedAt": "2026-03-25T09:56:11Z", "conclusion": "SKIPPED", "detailsUrl": "https://github.com/link-assistant/hive-mind/actions/runs/23535079153/job/68508232323", "name": "Docker Publish (Merge)", "startedAt": "2026-03-25T09:56:11Z", "status": "COMPLETED", "workflowName": "Checks and release" }, + { "__typename": "CheckRun", "completedAt": "2026-03-25T09:56:11Z", "conclusion": "SKIPPED", "detailsUrl": "https://github.com/link-assistant/hive-mind/actions/runs/23535079153/job/68508232566", "name": "Helm Release", "startedAt": "2026-03-25T09:56:11Z", "status": "COMPLETED", "workflowName": "Checks and release" } + ], + "title": "add retry logic for transient GitHub API errors during PR creation (Issue #1478)" +} diff --git a/src/github-merge.lib.mjs b/src/github-merge.lib.mjs index 04e738a40..9212a3202 100644 --- a/src/github-merge.lib.mjs +++ b/src/github-merge.lib.mjs @@ -1286,6 +1286,170 @@ export async function getActiveRepoWorkflows(owner, repo, verbose = false) { } } +/** + * Get the committed date of a specific commit from GitHub API + * Issue #1480: Used to determine how recently a commit was pushed, to distinguish between + * "CI not yet registered in API" (race condition) and "CI definitively not triggered" + * @param {string} owner - Repository owner + * @param {string} repo - Repository name + * @param {string} sha - Commit SHA + * @param {boolean} verbose - Whether to log verbose output + * @returns {Promise<{date: Date|null, ageSeconds: number|null}>} + */ +export async function getCommitDate(owner, repo, sha, verbose = false) { + try { + const { stdout } = await exec(`gh api repos/${owner}/${repo}/commits/${sha} --jq '.commit.committer.date'`); + const dateStr = stdout.trim(); + if (!dateStr) { + return { date: null, ageSeconds: null }; + } + const commitDate = new Date(dateStr); + const ageSeconds = Math.floor((Date.now() - commitDate.getTime()) / 1000); + if (verbose) { + console.log(`[VERBOSE] /merge: Commit ${sha.substring(0, 7)} date: ${dateStr} (${ageSeconds}s ago)`); + } + return { date: commitDate, ageSeconds }; + } catch (error) { + if (verbose) { + console.log(`[VERBOSE] /merge: Error fetching commit date for ${sha}: ${error.message}`); + } + return { date: null, ageSeconds: null }; + } +} + +/** + * Check if any previous commits in a PR had workflow runs triggered. + * Issue #1480: If earlier commits in the same PR triggered CI, we should expect CI + * for the HEAD commit too (unless conditions changed). This provides an additional + * signal that CI should be expected and avoids false "CI not triggered" conclusions. + * @param {string} owner - Repository owner + * @param {string} repo - Repository name + * @param {number} prNumber - Pull request number + * @param {string} headSha - Current HEAD SHA (to exclude from check) + * @param {boolean} verbose - Whether to log verbose output + * @returns {Promise<{hadPreviousCI: boolean, previousCommitsWithCI: number, totalPreviousCommits: number}>} + */ +export async function checkPreviousPRCommitsHadCI(owner, repo, prNumber, headSha, verbose = false) { + try { + // Get all commits in the PR + const { stdout: commitsJson } = await exec(`gh api "repos/${owner}/${repo}/pulls/${prNumber}/commits?per_page=100" --jq '[.[].sha]'`); + const allShas = JSON.parse(commitsJson.trim() || '[]'); + + // Exclude the current HEAD SHA + const previousShas = allShas.filter(sha => sha !== headSha); + + if (previousShas.length === 0) { + if (verbose) { + console.log(`[VERBOSE] /merge: PR #${prNumber} has no previous commits to check for CI history`); + } + return { hadPreviousCI: false, previousCommitsWithCI: 0, totalPreviousCommits: 0 }; + } + + // Check the most recent previous commits (limit to last 3 to avoid excessive API calls) + const commitsToCheck = previousShas.slice(-3); + let commitsWithCI = 0; + + for (const sha of commitsToCheck) { + try { + const { stdout } = await exec(`gh api "repos/${owner}/${repo}/actions/runs?head_sha=${sha}&per_page=1" --jq '.total_count'`); + const count = parseInt(stdout.trim(), 10); + if (count > 0) { + commitsWithCI++; + } + } catch { + // Skip errors for individual commits + } + } + + const hadPreviousCI = commitsWithCI > 0; + + if (verbose) { + console.log(`[VERBOSE] /merge: PR #${prNumber} previous CI history: ${commitsWithCI}/${commitsToCheck.length} checked commits had workflow runs (total PR commits: ${allShas.length})`); + } + + return { hadPreviousCI, previousCommitsWithCI: commitsWithCI, totalPreviousCommits: previousShas.length }; + } catch (error) { + if (verbose) { + console.log(`[VERBOSE] /merge: Error checking previous PR commits CI history: ${error.message}`); + } + return { hadPreviousCI: false, previousCommitsWithCI: 0, totalPreviousCommits: 0 }; + } +} + +/** + * Check if any workflow files in the repository have PR-related triggers + * Issue #1480: Used as additional signal to determine if CI should run on PRs. + * Parses .github/workflows/*.yml files from the repository content API. + * @param {string} owner - Repository owner + * @param {string} repo - Repository name + * @param {boolean} verbose - Whether to log verbose output + * @returns {Promise<{hasPRTriggers: boolean, hasWorkflowFiles: boolean, workflows: Array<{name: string, triggers: string[]}>}>} + */ +export async function checkWorkflowsHavePRTriggers(owner, repo, verbose = false) { + try { + // List workflow files in .github/workflows/ + const { stdout: listJson } = await exec(`gh api "repos/${owner}/${repo}/contents/.github/workflows" --jq '[.[] | select(.name | test("\\\\.(yml|yaml)$")) | {name: .name, download_url: .download_url, path: .path}]' 2>/dev/null`); + const files = JSON.parse(listJson.trim() || '[]'); + + if (files.length === 0) { + if (verbose) { + console.log(`[VERBOSE] /merge: No workflow files found in ${owner}/${repo}/.github/workflows/ — no CI/CD will execute`); + } + // Issue #1480: hasWorkflowFiles=false is a strong signal that no CI/CD is configured at the file level + return { hasPRTriggers: false, hasWorkflowFiles: false, workflows: [] }; + } + + const prTriggerPatterns = [/\bon:\s*\n\s+pull_request/m, /\bon:\s*\[.*pull_request.*\]/m, /\bon:\s*pull_request\b/m, /\bpull_request_target\b/m]; + + // Also check for push triggers (push to PR branches triggers CI) + const pushTriggerPatterns = [/\bon:\s*\n\s+push/m, /\bon:\s*\[.*push.*\]/m, /\bon:\s*push\b/m]; + + const results = []; + + for (const file of files) { + try { + // Fetch file content (use raw content from the API) + const { stdout: contentJson } = await exec(`gh api "repos/${owner}/${repo}/contents/${file.path}" --jq '.content'`); + const content = Buffer.from(contentJson.trim().replace(/"/g, ''), 'base64').toString('utf-8'); + + const triggers = []; + if (prTriggerPatterns.some(p => p.test(content))) { + triggers.push('pull_request'); + } + if (pushTriggerPatterns.some(p => p.test(content))) { + triggers.push('push'); + } + + if (triggers.length > 0) { + results.push({ name: file.name, triggers }); + } + + if (verbose) { + console.log(`[VERBOSE] /merge: Workflow ${file.name}: triggers=[${triggers.join(', ')}]`); + } + } catch (fileError) { + if (verbose) { + console.log(`[VERBOSE] /merge: Error reading workflow file ${file.name}: ${fileError.message}`); + } + } + } + + const hasPRTriggers = results.length > 0; + + if (verbose) { + console.log(`[VERBOSE] /merge: ${results.length}/${files.length} workflow files have PR/push triggers`); + } + + return { hasPRTriggers, hasWorkflowFiles: true, workflows: results }; + } catch (error) { + if (verbose) { + console.log(`[VERBOSE] /merge: Error checking workflow PR triggers: ${error.message}`); + } + // On error, assume workflows might have PR triggers (safer: avoids false positives) + return { hasPRTriggers: true, hasWorkflowFiles: true, workflows: [] }; + } +} + // Issue #1341: Re-export post-merge CI functions from separate module import { waitForCommitCI, checkBranchCIHealth, getMergeCommitSha } from './github-merge-ci.lib.mjs'; export { waitForCommitCI, checkBranchCIHealth, getMergeCommitSha }; @@ -1323,6 +1487,10 @@ export default { checkBranchCIHealth, getMergeCommitSha, getActiveRepoWorkflows, + // Issue #1480: Commit date, workflow PR triggers, and previous commit CI history for race condition detection + getCommitDate, + checkPreviousPRCommitsHadCI, + checkWorkflowsHavePRTriggers, // Issue #1413: Use issue timeline to find genuinely linked PRs (avoids false positives from text search) getLinkedPRsFromTimeline, }; diff --git a/src/solve.auto-merge.lib.mjs b/src/solve.auto-merge.lib.mjs index b4c59967e..05850d924 100644 --- a/src/solve.auto-merge.lib.mjs +++ b/src/solve.auto-merge.lib.mjs @@ -33,7 +33,7 @@ const { reportError } = sentryLib; // Import GitHub merge functions const githubMergeLib = await import('./github-merge.lib.mjs'); -const { checkPRMergeable, checkMergePermissions, mergePullRequest, waitForCI, checkForBillingLimitError, getRepoVisibility, BILLING_LIMIT_ERROR_PATTERN, getDetailedCIStatus, rerunWorkflowRun, getWorkflowRunsForSha, getActiveRepoWorkflows } = githubMergeLib; +const { checkPRMergeable, checkMergePermissions, mergePullRequest, waitForCI, checkForBillingLimitError, getRepoVisibility, BILLING_LIMIT_ERROR_PATTERN, getDetailedCIStatus, rerunWorkflowRun, getWorkflowRunsForSha, getActiveRepoWorkflows, getCommitDate, checkPreviousPRCommitsHadCI, checkWorkflowsHavePRTriggers } = githubMergeLib; // Import GitHub functions for log attachment const githubLib = await import('./github.lib.mjs'); @@ -255,14 +255,86 @@ const getMergeBlockers = async (owner, repo, prNumber, verbose = false) => { details: workflowRuns.map(r => r.name), }); } else { - // No workflow runs for this SHA — CI was definitively NOT triggered - // Issue #1442: This is the root cause of the infinite loop. Fork PRs needing - // maintainer approval, paths-ignore filtering, workflow conditions not matching, - // etc. all result in zero workflow runs. No need for timeout — exit immediately. - if (verbose) { - console.log(`[VERBOSE] /merge: PR #${prNumber} has no CI checks and no workflow runs for SHA ${ciStatus.sha.substring(0, 7)} — CI was not triggered (fork PR, paths-ignore, workflow conditions, etc.)`); + // No workflow runs for this SHA — but this could be a race condition! + // Issue #1480: GitHub Actions workflow runs take 30-120 seconds to appear in the + // API after a push. The previous fix (issue #1442) assumed 0 workflow runs meant + // "CI definitively NOT triggered", but this caused false positive "Ready to merge" + // when checked too soon after a push. + // + // Multi-layer defense (Issue #1480 enhanced): + // Layer 1: Grace period — check commit age + // Layer 2: Workflow file parsing — check .github/workflows for PR triggers + // Layer 3: Previous commit CI history — check if earlier PR commits had CI runs + const WORKFLOW_RUN_GRACE_PERIOD_SECONDS = 120; // 2 minutes — generous to cover slow GitHub API registration + const commitInfo = await getCommitDate(owner, repo, ciStatus.sha, verbose); + + // Issue #1480: Parse workflow files for PR triggers (used in both grace period and post-grace checks) + const prTriggers = await checkWorkflowsHavePRTriggers(owner, repo, verbose); + + // Issue #1480: If .github/workflows folder doesn't exist or has no workflow files, + // that's a definitive signal — no CI/CD will execute, skip grace period entirely + if (!prTriggers.hasWorkflowFiles) { + if (verbose) { + console.log(`[VERBOSE] /merge: PR #${prNumber} repo has no workflow files in .github/workflows/ — CI definitively not configured at file level`); + } + return { blockers, ciStatus, noCiConfigured: false, noCiTriggered: true }; + } + + if (commitInfo.ageSeconds !== null && commitInfo.ageSeconds < WORKFLOW_RUN_GRACE_PERIOD_SECONDS) { + // Commit is recent — workflow runs may not have appeared in the API yet + if (verbose) { + console.log(`[VERBOSE] /merge: PR #${prNumber} has no workflow runs for SHA ${ciStatus.sha.substring(0, 7)}, but commit is only ${commitInfo.ageSeconds}s old (grace period: ${WORKFLOW_RUN_GRACE_PERIOD_SECONDS}s) — treating as potential race condition`); + } + + if (prTriggers.hasPRTriggers) { + // Workflows have PR/push triggers AND commit is recent — almost certainly a race condition + if (verbose) { + console.log(`[VERBOSE] /merge: Workflow files confirm PR/push triggers exist (${prTriggers.workflows.map(w => w.name).join(', ')}) — waiting for workflow runs to appear`); + } + blockers.push({ + type: 'ci_pending', + message: `CI/CD workflow runs have not appeared yet — commit is ${commitInfo.ageSeconds}s old, waiting for GitHub to register workflow runs (grace period: ${WORKFLOW_RUN_GRACE_PERIOD_SECONDS}s)`, + details: prTriggers.workflows.map(w => w.name), + }); + } else { + // No PR triggers found in workflow files — but commit is still recent, be safe and wait + if (verbose) { + console.log(`[VERBOSE] /merge: No PR/push triggers found in workflow files, but commit is only ${commitInfo.ageSeconds}s old — waiting to be safe`); + } + blockers.push({ + type: 'ci_pending', + message: `CI/CD workflow runs have not appeared yet — commit is ${commitInfo.ageSeconds}s old, waiting for GitHub to register workflow runs (grace period: ${WORKFLOW_RUN_GRACE_PERIOD_SECONDS}s)`, + details: [], + }); + } + } else { + // Commit is old enough (grace period elapsed) — but check additional signals before concluding + // Issue #1480: Layer 3 — Check if previous commits in this PR had CI runs. + // If earlier commits had CI, the HEAD commit should also have CI unless conditions changed. + const previousCI = await checkPreviousPRCommitsHadCI(owner, repo, prNumber, ciStatus.sha, verbose); + + if (previousCI.hadPreviousCI && prTriggers.hasPRTriggers) { + // Previous commits had CI AND workflow files have PR triggers — something is wrong, + // this could be a GitHub API glitch or delayed registration beyond the grace period. + // Wait one more cycle to be safe. + if (verbose) { + console.log(`[VERBOSE] /merge: PR #${prNumber} previous commits had CI (${previousCI.previousCommitsWithCI}/${previousCI.totalPreviousCommits}) and workflows have PR triggers, but HEAD has no runs — waiting as safety measure`); + } + blockers.push({ + type: 'ci_pending', + message: `CI/CD workflow runs missing for HEAD — previous PR commits had CI (${previousCI.previousCommitsWithCI} of ${previousCI.totalPreviousCommits}), workflows have PR triggers, possible API delay`, + details: prTriggers.workflows.map(w => w.name), + }); + } else { + // CI was definitively NOT triggered + // Issue #1442: Fork PRs needing maintainer approval, paths-ignore filtering, + // workflow conditions not matching, etc. all result in zero workflow runs. + if (verbose) { + console.log(`[VERBOSE] /merge: PR #${prNumber} has no CI checks and no workflow runs for SHA ${ciStatus.sha.substring(0, 7)} (commit age: ${commitInfo.ageSeconds ?? 'unknown'}s, grace period: ${WORKFLOW_RUN_GRACE_PERIOD_SECONDS}s elapsed, previous CI: ${previousCI.hadPreviousCI}, PR triggers: ${prTriggers.hasPRTriggers}) — CI was not triggered`); + } + return { blockers, ciStatus, noCiConfigured: false, noCiTriggered: true }; + } } - return { blockers, ciStatus, noCiConfigured: false, noCiTriggered: true }; } } else { // Repo has NO workflows — this is truly "no CI configured" diff --git a/tests/test-false-positive-workflow-run-race-1480.mjs b/tests/test-false-positive-workflow-run-race-1480.mjs new file mode 100644 index 000000000..2d8dff85b --- /dev/null +++ b/tests/test-false-positive-workflow-run-race-1480.mjs @@ -0,0 +1,604 @@ +#!/usr/bin/env node + +/** + * Unit Tests: Issue #1480 - `Ready to merge` posted as false positive + * + * Tests verify that: + * 1. When workflow runs have not appeared in the API yet (0 runs) but the commit is recent + * (within grace period), the system waits instead of concluding "CI not triggered" + * 2. When the commit is old enough (past grace period) and still no workflow runs, + * the system correctly concludes "CI not triggered" + * 3. Workflow file PR trigger parsing correctly identifies PR/push triggers + * 4. The grace period + workflow file parsing work together for defense in depth + * 5. Edge cases: null commit date, exactly at grace period boundary + * + * Root cause: The fix for issue #1442 assumed that if getWorkflowRunsForSha() returns + * 0 runs, CI was "definitively NOT triggered". But GitHub Actions workflow runs take + * 30-120 seconds to appear in the API after a push, causing false positive + * "Ready to merge" comments when checked within seconds of a push. + * + * Run with: node tests/test-false-positive-workflow-run-race-1480.mjs + * + * @see https://github.com/link-assistant/hive-mind/issues/1480 + * @see https://github.com/link-assistant/hive-mind/issues/1442 (introduced the flawed check) + * @see https://github.com/link-assistant/hive-mind/issues/1363 (related false positive) + */ + +// ANSI color codes for terminal output +const GREEN = '\x1b[32m'; +const RED = '\x1b[31m'; +const RESET = '\x1b[0m'; + +let passed = 0; +let failed = 0; + +const test = (description, fn) => { + try { + fn(); + console.log(` ${GREEN}✅ PASS:${RESET} ${description}`); + passed++; + } catch (e) { + console.log(` ${RED}❌ FAIL:${RESET} ${description}`); + console.log(` Error: ${e.message}`); + failed++; + } +}; + +const assert = (condition, message) => { + if (!condition) { + throw new Error(message); + } +}; + +console.log('================================================================================'); +console.log('Unit Tests: Issue #1480 - `Ready to merge` posted as false positive'); +console.log('================================================================================\n'); + +// ===== Constants matching the actual implementation ===== +const WORKFLOW_RUN_GRACE_PERIOD_SECONDS = 120; + +// ===== Simulate the FIXED getMergeBlockers logic for the no_checks + has workflows + 0 runs path ===== + +/** + * Simulates the fixed getMergeBlockers logic specifically for the path: + * no_checks → mergeable → has workflows → 0 workflow runs → multi-layer defense + * + * This mirrors the actual logic in src/solve.auto-merge.lib.mjs after the Issue #1480 fix. + * + * @param {Object} params + * @param {Array} params.workflowRuns - Workflow runs returned by getWorkflowRunsForSha + * @param {{ageSeconds: number|null}} params.commitInfo - Commit age info from getCommitDate + * @param {{hasPRTriggers: boolean, hasWorkflowFiles: boolean, workflows: Array}} params.prTriggers - PR trigger info from checkWorkflowsHavePRTriggers + * @param {{hadPreviousCI: boolean, previousCommitsWithCI: number, totalPreviousCommits: number}} [params.previousCI] - Previous commit CI history + */ +function simulateFixedWorkflowRunCheck({ workflowRuns, commitInfo, prTriggers, previousCI }) { + const blockers = []; + + // Default previousCI for backward compatibility with existing tests + if (!previousCI) { + previousCI = { hadPreviousCI: false, previousCommitsWithCI: 0, totalPreviousCommits: 0 }; + } + + if (workflowRuns.length > 0) { + // Issue #1466: Check if ALL workflow runs completed without producing check-runs + const allRunsCompleted = workflowRuns.every(r => r.status === 'completed'); + const allRunsNonExecuting = allRunsCompleted && workflowRuns.every(r => r.conclusion === 'action_required' || r.conclusion === 'cancelled' || r.conclusion === 'stale' || r.conclusion === 'skipped'); + + if (allRunsNonExecuting) { + const conclusions = [...new Set(workflowRuns.map(r => r.conclusion))].join(', '); + return { blockers, noCiTriggered: true, workflowRunConclusions: conclusions, raceCondition: false }; + } + + // Some workflow runs are still in progress — genuine race condition + blockers.push({ + type: 'ci_pending', + message: `CI/CD checks have not started yet (${workflowRuns.length} workflow run(s) triggered, waiting for check-runs to appear)`, + details: workflowRuns.map(r => r.name), + }); + return { blockers, noCiTriggered: false, workflowRunConclusions: undefined, raceCondition: true }; + } + + // No workflow runs for this SHA — Issue #1480 multi-layer defense + + // Layer 1: If no workflow files exist at all, no CI will execute + if (!prTriggers.hasWorkflowFiles) { + return { blockers, noCiTriggered: true, workflowRunConclusions: undefined, raceCondition: false }; + } + + // Layer 2: Grace period check + if (commitInfo.ageSeconds !== null && commitInfo.ageSeconds < WORKFLOW_RUN_GRACE_PERIOD_SECONDS) { + // Commit is recent — workflow runs may not have appeared in the API yet + if (prTriggers.hasPRTriggers) { + blockers.push({ + type: 'ci_pending', + message: `CI/CD workflow runs have not appeared yet — commit is ${commitInfo.ageSeconds}s old, waiting for GitHub to register workflow runs (grace period: ${WORKFLOW_RUN_GRACE_PERIOD_SECONDS}s)`, + details: prTriggers.workflows.map(w => w.name), + }); + } else { + blockers.push({ + type: 'ci_pending', + message: `CI/CD workflow runs have not appeared yet — commit is ${commitInfo.ageSeconds}s old, waiting for GitHub to register workflow runs (grace period: ${WORKFLOW_RUN_GRACE_PERIOD_SECONDS}s)`, + details: [], + }); + } + return { blockers, noCiTriggered: false, workflowRunConclusions: undefined, raceCondition: true }; + } + + // Layer 3: Previous commit CI history check (after grace period) + if (previousCI.hadPreviousCI && prTriggers.hasPRTriggers) { + blockers.push({ + type: 'ci_pending', + message: `CI/CD workflow runs missing for HEAD — previous PR commits had CI (${previousCI.previousCommitsWithCI} of ${previousCI.totalPreviousCommits}), workflows have PR triggers, possible API delay`, + details: prTriggers.workflows.map(w => w.name), + }); + return { blockers, noCiTriggered: false, workflowRunConclusions: undefined, raceCondition: true }; + } + + // Layer 4: Definitive conclusion — CI was NOT triggered + return { blockers, noCiTriggered: true, workflowRunConclusions: undefined, raceCondition: false }; +} + +// ===== Test Suite 1: The exact false positive scenario from PR #1479 ===== +console.log('📋 Test Suite 1: Exact false positive scenario from PR #1479\n'); + +test('PR #1479 scenario: 0 workflow runs + commit only 17s old + has PR triggers → wait (NOT "CI not triggered")', () => { + // This is the exact scenario that caused the false positive: + // - Commit 9ed29b7 pushed at 09:53:33Z + // - getMergeBlockers called at ~09:53:50Z (17 seconds later) + // - getWorkflowRunsForSha returned [] (not yet registered) + // - CI actually started at 09:55:18Z and FAILED + const result = simulateFixedWorkflowRunCheck({ + workflowRuns: [], + commitInfo: { ageSeconds: 17 }, + prTriggers: { + hasPRTriggers: true, + hasWorkflowFiles: true, + workflows: [{ name: 'release.yml', triggers: ['push'] }], + }, + }); + + assert(result.noCiTriggered === false, 'Should NOT conclude CI is not triggered — commit is too recent'); + assert(result.raceCondition === true, 'Should treat as race condition'); + assert(result.blockers.length === 1, 'Should add ci_pending blocker'); + assert(result.blockers[0].type === 'ci_pending', 'Blocker type should be ci_pending'); + assert(result.blockers[0].message.includes('17s old'), 'Message should include commit age'); + assert(result.blockers[0].message.includes('grace period'), 'Message should mention grace period'); + assert(result.blockers[0].details.length === 1, 'Details should include workflow file name'); + assert(result.blockers[0].details[0] === 'release.yml', 'Details should include release.yml'); +}); + +// ===== Test Suite 2: Grace period boundary tests ===== +console.log('\n📋 Test Suite 2: Grace period boundary conditions\n'); + +test('Commit is 0 seconds old → wait (within grace period)', () => { + const result = simulateFixedWorkflowRunCheck({ + workflowRuns: [], + commitInfo: { ageSeconds: 0 }, + prTriggers: { hasPRTriggers: true, hasWorkflowFiles: true, workflows: [{ name: 'ci.yml', triggers: ['pull_request'] }] }, + }); + + assert(result.noCiTriggered === false, 'Should NOT conclude CI is not triggered'); + assert(result.blockers.length === 1, 'Should add ci_pending blocker'); +}); + +test('Commit is 119 seconds old → wait (still within grace period)', () => { + const result = simulateFixedWorkflowRunCheck({ + workflowRuns: [], + commitInfo: { ageSeconds: 119 }, + prTriggers: { hasPRTriggers: true, hasWorkflowFiles: true, workflows: [{ name: 'ci.yml', triggers: ['pull_request'] }] }, + }); + + assert(result.noCiTriggered === false, 'Should NOT conclude CI is not triggered'); + assert(result.blockers.length === 1, 'Should add ci_pending blocker'); +}); + +test('Commit is exactly 120 seconds old → conclude CI not triggered (grace period elapsed)', () => { + const result = simulateFixedWorkflowRunCheck({ + workflowRuns: [], + commitInfo: { ageSeconds: 120 }, + prTriggers: { hasPRTriggers: true, hasWorkflowFiles: true, workflows: [{ name: 'ci.yml', triggers: ['pull_request'] }] }, + }); + + assert(result.noCiTriggered === true, 'Should conclude CI is not triggered — grace period elapsed'); + assert(result.blockers.length === 0, 'Should NOT add blockers'); +}); + +test('Commit is 300 seconds old → conclude CI not triggered (well past grace period)', () => { + const result = simulateFixedWorkflowRunCheck({ + workflowRuns: [], + commitInfo: { ageSeconds: 300 }, + prTriggers: { hasPRTriggers: true, hasWorkflowFiles: true, workflows: [] }, + }); + + assert(result.noCiTriggered === true, 'Should conclude CI is not triggered'); + assert(result.blockers.length === 0, 'Should NOT add blockers'); +}); + +// ===== Test Suite 3: Commit date unavailable (null) ===== +console.log('\n📋 Test Suite 3: Commit date unavailable edge cases\n'); + +test('Commit date unknown (ageSeconds=null) → conclude CI not triggered (fail-safe for old behavior)', () => { + // If we can't determine commit age, fall through to the old behavior + // (conclude CI not triggered) to avoid infinite waiting + const result = simulateFixedWorkflowRunCheck({ + workflowRuns: [], + commitInfo: { ageSeconds: null }, + prTriggers: { hasPRTriggers: true, hasWorkflowFiles: true, workflows: [{ name: 'ci.yml', triggers: ['pull_request'] }] }, + }); + + assert(result.noCiTriggered === true, 'Should conclude CI is not triggered when date is unknown'); + assert(result.blockers.length === 0, 'Should NOT add blockers'); +}); + +// ===== Test Suite 4: Workflow file PR trigger analysis ===== +console.log('\n📋 Test Suite 4: Workflow file PR trigger analysis impact\n'); + +test('Recent commit + no PR triggers in workflow files → still wait (be safe)', () => { + // Even if workflow files don't seem to have PR triggers, if the commit is recent + // we should still wait because our parsing might not catch all trigger patterns + const result = simulateFixedWorkflowRunCheck({ + workflowRuns: [], + commitInfo: { ageSeconds: 30 }, + prTriggers: { hasPRTriggers: false, hasWorkflowFiles: true, workflows: [] }, + }); + + assert(result.noCiTriggered === false, 'Should NOT conclude CI is not triggered — commit is recent'); + assert(result.blockers.length === 1, 'Should add ci_pending blocker'); + assert(result.blockers[0].details.length === 0, 'Details should be empty (no PR trigger workflows found)'); +}); + +test('Recent commit + has PR triggers → wait with workflow details', () => { + const result = simulateFixedWorkflowRunCheck({ + workflowRuns: [], + commitInfo: { ageSeconds: 45 }, + prTriggers: { + hasPRTriggers: true, + hasWorkflowFiles: true, + workflows: [ + { name: 'ci.yml', triggers: ['pull_request'] }, + { name: 'tests.yml', triggers: ['push', 'pull_request'] }, + ], + }, + }); + + assert(result.noCiTriggered === false, 'Should NOT conclude CI is not triggered'); + assert(result.blockers.length === 1, 'Should add ci_pending blocker'); + assert(result.blockers[0].details.length === 2, 'Details should have 2 workflow names'); + assert(result.blockers[0].details.includes('ci.yml'), 'Should include ci.yml'); + assert(result.blockers[0].details.includes('tests.yml'), 'Should include tests.yml'); +}); + +// ===== Test Suite 5: Backward compatibility with existing behaviors ===== +console.log('\n📋 Test Suite 5: Backward compatibility with existing behaviors\n'); + +test('Workflow runs exist + action_required → still treat as CI not triggered (issue #1466)', () => { + // The issue #1466 fix must still work correctly + const result = simulateFixedWorkflowRunCheck({ + workflowRuns: [{ id: 1, name: 'CI', status: 'completed', conclusion: 'action_required' }], + commitInfo: { ageSeconds: 10 }, + prTriggers: { hasPRTriggers: true, hasWorkflowFiles: true, workflows: [] }, + }); + + assert(result.noCiTriggered === true, 'Should treat as CI not triggered (action_required)'); + assert(result.workflowRunConclusions === 'action_required', 'Should report conclusion'); +}); + +test('Workflow runs exist + in_progress → genuine race condition (not affected by grace period)', () => { + // When workflow runs exist, the existing behavior should be preserved regardless of commit age + const result = simulateFixedWorkflowRunCheck({ + workflowRuns: [{ id: 1, name: 'CI', status: 'in_progress', conclusion: null }], + commitInfo: { ageSeconds: 5 }, + prTriggers: { hasPRTriggers: true, hasWorkflowFiles: true, workflows: [] }, + }); + + assert(result.noCiTriggered === false, 'Should NOT conclude CI is not triggered'); + assert(result.raceCondition === true, 'Should treat as genuine race condition'); + assert(result.blockers.length === 1, 'Should have ci_pending blocker'); +}); + +test('Workflow runs exist + completed success → no change needed (this path has check-runs)', () => { + // When workflow runs have success conclusion, check-runs should exist too + // This path actually wouldn't reach our code (getDetailedCIStatus would return success/failure) + // But test it for completeness + const result = simulateFixedWorkflowRunCheck({ + workflowRuns: [{ id: 1, name: 'CI', status: 'completed', conclusion: 'success' }], + commitInfo: { ageSeconds: 60 }, + prTriggers: { hasPRTriggers: true, hasWorkflowFiles: true, workflows: [] }, + }); + + // Completed with success is NOT a non-executing conclusion, so it's a genuine race condition + // (check-runs should exist but haven't been registered yet) + assert(result.noCiTriggered === false, 'Success run should be treated as race condition'); + assert(result.raceCondition === true, 'Should be a genuine race condition'); +}); + +// ===== Test Suite 6: Workflow file PR trigger parsing logic ===== +console.log('\n📋 Test Suite 6: Workflow file PR trigger parsing patterns\n'); + +/** + * Simulates checkWorkflowsHavePRTriggers parsing logic for a single workflow content + */ +function checkContentForPRTriggers(content) { + const prTriggerPatterns = [/\bon:\s*\n\s+pull_request/m, /\bon:\s*\[.*pull_request.*\]/m, /\bon:\s*pull_request\b/m, /\bpull_request_target\b/m]; + const pushTriggerPatterns = [/\bon:\s*\n\s+push/m, /\bon:\s*\[.*push.*\]/m, /\bon:\s*push\b/m]; + + const triggers = []; + if (prTriggerPatterns.some(p => p.test(content))) triggers.push('pull_request'); + if (pushTriggerPatterns.some(p => p.test(content))) triggers.push('push'); + return triggers; +} + +test('Detects standard multi-line pull_request trigger', () => { + const content = `name: CI +on: + pull_request: + branches: [main] +jobs: + test: + runs-on: ubuntu-latest`; + const triggers = checkContentForPRTriggers(content); + assert(triggers.includes('pull_request'), `Should detect pull_request trigger, got: [${triggers}]`); +}); + +test('Detects inline array pull_request trigger', () => { + const content = `name: CI +on: [push, pull_request] +jobs: + test: + runs-on: ubuntu-latest`; + const triggers = checkContentForPRTriggers(content); + assert(triggers.includes('pull_request'), 'Should detect inline pull_request'); + assert(triggers.includes('push'), 'Should also detect push'); +}); + +test('Detects simple on: push trigger', () => { + const content = `name: CI +on: push +jobs: + test: + runs-on: ubuntu-latest`; + const triggers = checkContentForPRTriggers(content); + assert(triggers.includes('push'), 'Should detect simple push trigger'); + assert(!triggers.includes('pull_request'), 'Should NOT detect pull_request'); +}); + +test('Detects pull_request_target trigger', () => { + const content = `name: CI +on: + pull_request_target: + types: [opened, synchronize] +jobs: + test: + runs-on: ubuntu-latest`; + const triggers = checkContentForPRTriggers(content); + assert(triggers.includes('pull_request'), 'Should detect pull_request_target as a PR trigger'); +}); + +test('Detects multi-line push trigger', () => { + const content = `name: Checks and release +on: + push: + branches: ['**'] + workflow_dispatch: +jobs: + detect-changes: + runs-on: ubuntu-latest`; + const triggers = checkContentForPRTriggers(content); + assert(triggers.includes('push'), 'Should detect multi-line push trigger'); +}); + +test('Does NOT detect workflow_dispatch-only workflow as PR trigger', () => { + const content = `name: Manual Deploy +on: + workflow_dispatch: + inputs: + environment: + description: Target environment +jobs: + deploy: + runs-on: ubuntu-latest`; + const triggers = checkContentForPRTriggers(content); + assert(triggers.length === 0, `Should not detect any PR/push triggers, got: [${triggers}]`); +}); + +test('Does NOT detect schedule-only workflow as PR trigger', () => { + const content = `name: Nightly Build +on: + schedule: + - cron: '0 2 * * *' +jobs: + build: + runs-on: ubuntu-latest`; + const triggers = checkContentForPRTriggers(content); + assert(triggers.length === 0, `Should not detect any PR/push triggers, got: [${triggers}]`); +}); + +// ===== Test Suite 7: Combined end-to-end scenario tests ===== +console.log('\n📋 Test Suite 7: End-to-end scenario tests\n'); + +test('Scenario: Repo with only schedule/dispatch workflows + recent commit → wait to be safe', () => { + // Even without PR triggers, a recent commit should still wait + const result = simulateFixedWorkflowRunCheck({ + workflowRuns: [], + commitInfo: { ageSeconds: 15 }, + prTriggers: { hasPRTriggers: false, hasWorkflowFiles: true, workflows: [] }, + }); + + assert(result.noCiTriggered === false, 'Should wait even without PR triggers — commit is recent'); + assert(result.blockers.length === 1, 'Should have blocker'); +}); + +test('Scenario: Repo with only schedule/dispatch workflows + old commit → CI not triggered', () => { + // Old commit + no PR triggers → definitely no CI expected + const result = simulateFixedWorkflowRunCheck({ + workflowRuns: [], + commitInfo: { ageSeconds: 300 }, + prTriggers: { hasPRTriggers: false, hasWorkflowFiles: true, workflows: [] }, + }); + + assert(result.noCiTriggered === true, 'Should conclude CI not triggered'); + assert(result.blockers.length === 0, 'No blockers'); +}); + +test('Scenario: Fork PR with paths-ignore + old commit → CI not triggered', () => { + // This is the legitimate "CI not triggered" case from issue #1442 + // paths-ignore filtering means no workflow runs, even after grace period + const result = simulateFixedWorkflowRunCheck({ + workflowRuns: [], + commitInfo: { ageSeconds: 200 }, + prTriggers: { hasPRTriggers: true, hasWorkflowFiles: true, workflows: [{ name: 'ci.yml', triggers: ['pull_request'] }] }, + }); + + assert(result.noCiTriggered === true, 'Should conclude CI not triggered — grace period passed'); + assert(result.blockers.length === 0, 'No blockers'); +}); + +// ===== Test Suite 8: Empty workflows folder detection (Layer 1) ===== +console.log('\n📋 Test Suite 8: Empty workflows folder detection (no .github/workflows files)\n'); + +test('No workflow files at all + recent commit → immediately conclude no CI (skip grace period)', () => { + const result = simulateFixedWorkflowRunCheck({ + workflowRuns: [], + commitInfo: { ageSeconds: 5 }, // Very recent commit + prTriggers: { hasPRTriggers: false, hasWorkflowFiles: false, workflows: [] }, + }); + + assert(result.noCiTriggered === true, 'Should immediately conclude CI not triggered — no workflow files'); + assert(result.blockers.length === 0, 'Should NOT add blockers — no files means no CI'); +}); + +test('No workflow files + old commit → conclude no CI', () => { + const result = simulateFixedWorkflowRunCheck({ + workflowRuns: [], + commitInfo: { ageSeconds: 300 }, + prTriggers: { hasPRTriggers: false, hasWorkflowFiles: false, workflows: [] }, + }); + + assert(result.noCiTriggered === true, 'Should conclude CI not triggered'); + assert(result.blockers.length === 0, 'No blockers'); +}); + +test('No workflow files + null commit date → conclude no CI', () => { + const result = simulateFixedWorkflowRunCheck({ + workflowRuns: [], + commitInfo: { ageSeconds: null }, + prTriggers: { hasPRTriggers: false, hasWorkflowFiles: false, workflows: [] }, + }); + + assert(result.noCiTriggered === true, 'Should conclude CI not triggered — no files regardless of date'); + assert(result.blockers.length === 0, 'No blockers'); +}); + +// ===== Test Suite 9: Previous commit CI history (Layer 3) ===== +console.log('\n📋 Test Suite 9: Previous commit CI history detection\n'); + +test('Grace period elapsed + previous commits had CI + PR triggers → wait (safety measure)', () => { + const result = simulateFixedWorkflowRunCheck({ + workflowRuns: [], + commitInfo: { ageSeconds: 150 }, // Past grace period + prTriggers: { hasPRTriggers: true, hasWorkflowFiles: true, workflows: [{ name: 'ci.yml', triggers: ['push'] }] }, + previousCI: { hadPreviousCI: true, previousCommitsWithCI: 2, totalPreviousCommits: 3 }, + }); + + assert(result.noCiTriggered === false, 'Should NOT conclude CI not triggered — previous commits had CI'); + assert(result.blockers.length === 1, 'Should add ci_pending blocker'); + assert(result.blockers[0].message.includes('previous PR commits had CI'), 'Message should mention previous CI'); + assert(result.blockers[0].message.includes('2 of 3'), 'Message should include commit counts'); +}); + +test('Grace period elapsed + previous commits had CI + NO PR triggers → conclude no CI', () => { + // Previous commits had CI but workflow files don't have PR triggers anymore + // (maybe triggers were changed) — conclude no CI + const result = simulateFixedWorkflowRunCheck({ + workflowRuns: [], + commitInfo: { ageSeconds: 150 }, + prTriggers: { hasPRTriggers: false, hasWorkflowFiles: true, workflows: [] }, + previousCI: { hadPreviousCI: true, previousCommitsWithCI: 1, totalPreviousCommits: 2 }, + }); + + assert(result.noCiTriggered === true, 'Should conclude CI not triggered — no PR triggers in current files'); + assert(result.blockers.length === 0, 'No blockers'); +}); + +test('Grace period elapsed + no previous CI + PR triggers → conclude no CI', () => { + // First commit in PR, no previous CI history, grace period passed + const result = simulateFixedWorkflowRunCheck({ + workflowRuns: [], + commitInfo: { ageSeconds: 200 }, + prTriggers: { hasPRTriggers: true, hasWorkflowFiles: true, workflows: [{ name: 'ci.yml', triggers: ['pull_request'] }] }, + previousCI: { hadPreviousCI: false, previousCommitsWithCI: 0, totalPreviousCommits: 0 }, + }); + + assert(result.noCiTriggered === true, 'Should conclude CI not triggered — no previous CI evidence'); + assert(result.blockers.length === 0, 'No blockers'); +}); + +test('Grace period elapsed + no previous CI + no PR triggers → conclude no CI', () => { + const result = simulateFixedWorkflowRunCheck({ + workflowRuns: [], + commitInfo: { ageSeconds: 300 }, + prTriggers: { hasPRTriggers: false, hasWorkflowFiles: true, workflows: [] }, + previousCI: { hadPreviousCI: false, previousCommitsWithCI: 0, totalPreviousCommits: 0 }, + }); + + assert(result.noCiTriggered === true, 'Should conclude CI not triggered'); + assert(result.blockers.length === 0, 'No blockers'); +}); + +// ===== Test Suite 10: Multi-layer defense interaction tests ===== +console.log('\n📋 Test Suite 10: Multi-layer defense interaction tests\n'); + +test('Layer priority: no workflow files overrides recent commit (Layer 1 > Layer 2)', () => { + // Even though commit is very recent (would normally wait), no workflow files = no CI + const result = simulateFixedWorkflowRunCheck({ + workflowRuns: [], + commitInfo: { ageSeconds: 1 }, + prTriggers: { hasPRTriggers: false, hasWorkflowFiles: false, workflows: [] }, + previousCI: { hadPreviousCI: true, previousCommitsWithCI: 1, totalPreviousCommits: 1 }, + }); + + assert(result.noCiTriggered === true, 'Layer 1 (no files) should take priority over everything'); + assert(result.blockers.length === 0, 'No blockers'); +}); + +test('Layer priority: grace period overrides previous CI (Layer 2 > Layer 3)', () => { + // During grace period, we wait regardless of previous CI history + const result = simulateFixedWorkflowRunCheck({ + workflowRuns: [], + commitInfo: { ageSeconds: 30 }, + prTriggers: { hasPRTriggers: true, hasWorkflowFiles: true, workflows: [{ name: 'ci.yml', triggers: ['push'] }] }, + previousCI: { hadPreviousCI: false, previousCommitsWithCI: 0, totalPreviousCommits: 5 }, + }); + + assert(result.noCiTriggered === false, 'Should wait during grace period regardless of previous CI'); + assert(result.blockers.length === 1, 'Should have ci_pending blocker'); + assert(result.blockers[0].message.includes('30s old'), 'Should mention commit age from grace period'); +}); + +test('Full pipeline: PR #1479 exact scenario with all layers (enhanced)', () => { + // Exact reproduction: commit 17s old, has workflows with push triggers, previous commits had CI + const result = simulateFixedWorkflowRunCheck({ + workflowRuns: [], + commitInfo: { ageSeconds: 17 }, + prTriggers: { + hasPRTriggers: true, + hasWorkflowFiles: true, + workflows: [{ name: 'release.yml', triggers: ['push'] }], + }, + previousCI: { hadPreviousCI: true, previousCommitsWithCI: 4, totalPreviousCommits: 4 }, + }); + + assert(result.noCiTriggered === false, 'Must NOT conclude CI not triggered'); + assert(result.raceCondition === true, 'Should detect race condition'); + assert(result.blockers.length === 1, 'Should have exactly 1 blocker'); + assert(result.blockers[0].type === 'ci_pending', 'Blocker type should be ci_pending'); +}); + +// ===== Summary ===== +console.log('\n================================================================================'); +console.log(`Results: ${passed} passed, ${failed} failed out of ${passed + failed} total`); +console.log('================================================================================'); + +if (failed > 0) { + process.exit(1); +}