fix: prevent AI solver from ignoring changed/expanded requirements#1566
Open
konard wants to merge 4 commits into
Open
fix: prevent AI solver from ignoring changed/expanded requirements#1566konard wants to merge 4 commits into
konard wants to merge 4 commits into
Conversation
Adding .gitkeep for PR creation (default mode). This file will be removed when the task is complete. Issue: #1565
…1565) Root cause: When users expand scope via PR comments (e.g., "sync ALL features, not just temperature"), the AI solver would acknowledge the requirement but then silently narrow scope back to the original issue title. This happened 3 times in agent#244 before compliance. Two fixes: 1. Include latest reviewer comment verbatim in the continue-mode prompt (solve.feedback.lib.mjs) so requirements are impossible to miss 2. Add "Scope management" instructions to all prompt variants that explicitly forbid silent scope narrowing Case study with full timeline and root cause analysis included. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
konard
changed the title
konard
marked this pull request as ready for review
April 10, 2026 21:43
Contributor
Author
🤖 Solution Draft LogThis log file contains the complete execution trace of the AI solution draft process. 💰 Cost: $3.641992📊 Context and tokens usage:
Total: (101.8K + 5.0M cached) input tokens, 19.4K output tokens, $3.641992 cost 🤖 Models used:
📎 Log file uploaded as Gist (2177KB)Now working session is ended, feel free to review and add any feedback on the solution draft. |
Contributor
Author
🔄 Auto-restart triggered (iteration 1)Reason: CI failures detected Starting new session to address the issues. Auto-restart-until-mergeable mode is active. Will continue until PR becomes mergeable. |
Revert manual version change in package.json and manual CHANGELOG entry. Add proper changeset file so the release workflow manages versioning automatically. Fixes failing "Check for Changesets" and "Check for Manual Version Changes" CI checks. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Contributor
Author
🔄 Auto-restart-until-mergeable Log (iteration 1)This log file contains the complete execution trace of the AI solution draft process. 💰 Cost: $0.683837📊 Context and tokens usage:
Total: (33.8K + 744.6K cached) input tokens, 4.0K output tokens, $0.683837 cost 🤖 Models used:
📎 Log file uploaded as Gist (3928KB)Now working session is ended, feel free to review and add any feedback on the solution draft. |
Contributor
Author
✅ Ready to mergeThis pull request is now ready to be merged:
Monitored by hive-mind with --auto-restart-until-mergeable flag |
This reverts commit 8e4cd80.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #1565
solve.feedback.lib.mjs) so requirements are embedded directly in context, not discovered through tool useRoot Cause
The AI model has a built-in tendency toward minimal changes that conflicts with explicit user instructions that expand scope. Key evidence from logs:
The AI correctly understood 5 requirements from the user's comment, then decided to only implement 1 of them.
Changes
src/solve.feedback.lib.mjssrc/claude.prompts.lib.mjssrc/opencode.prompts.lib.mjssrc/codex.prompts.lib.mjssrc/agent.prompts.lib.mjsdocs/case-studies/issue-1565/README.mddocs/case-studies/issue-1565/improvements.mdCHANGELOG.mdpackage.jsonTest plan
eslintpasses on all modified source filesHow to verify the fix works
After this change, when a reviewer posts a comment on a PR and the AI solver restarts:
🤖 Generated with Claude Code