feat: add opusplan runtime verification and upstream bug case study (Issue #1223)

[konard]

Summary

• Root cause analysis: Investigated why opusplan mode doesn't actually use Opus for planning. Confirmed it's an upstream bug in Claude Code CLI (not in hive-mind). Claude Code accepts --model opusplan but resolves it to claude-sonnet-4-6 without enabling the plan-mode model switching.
• Runtime verification: Added detection in executeClaudeCommand() that checks the model reported in Claude CLI's init event and warns when opusplan silently falls back to sonnet.
• Improved model display: When opusplan is active, the command details now show Claude OPUSPLAN (plan: claude-opus-4-6, exec: claude-sonnet-4-6) instead of just Claude SONNET.
• Case study with full evidence: Updated docs/case-studies/issue-1223/ with timeline, log analysis, and references to 6 upstream issues confirming this bug.
• Upstream comment: Added our evidence to anthropics/claude-code#16982 with full API call logs proving zero Opus usage.

Evidence

From the full session log:

What	Expected	Actual
CLI command	claude --model opusplan	Correct
Env vars	ANTHROPIC_DEFAULT_OPUS_MODEL=claude-opus-4-6	Correct
Claude CLI init model	claude-opus-4-6	claude-sonnet-4-6 (bug)
API calls to Opus	>0	0

Upstream Issues

• anthropics/claude-code#16982 — opusplan doesn't switch to Opus (7+ confirmations, v2.1.2-v2.1.86)
• anthropics/claude-code#35650 — /model opusplan maps to sonnet for both phases
• anthropics/claude-code#33401 — system prompt reports wrong model in plan mode

Test plan

• All 46 opusplan tests pass (node tests/test-opusplan-support.mjs)
• ESLint passes on src/claude.lib.mjs
• Runtime verification logic correctly detects sonnet/haiku as unexpected for opusplan
• Runtime verification correctly accepts opus models as expected

Fixes #1223

---

This PR was created automatically by the AI issue solver

[konard]

🤖 Solution Draft Log

This log file contains the complete execution trace of the AI solution draft process.

💰 Cost estimation:

• Public pricing estimate: $13.901058
• Calculated by Anthropic: $8.973825 USD
• Difference: $-4.927233 (-35.45%)

🤖 Models used:

• Tool: Anthropic Claude Code
• Requested: opus
• Main model: Claude Opus 4.6 (claude-opus-4-6)
• Additional models:
  • Claude Haiku 4.5 (claude-haiku-4-5-20251001)

📎 Log file uploaded as Gist (3281KB)

• View complete solution draft log

---

Now working session is ended, feel free to review and add any feedback on the solution draft.

[konard]

🔄 Auto-restart 1/3

Detected uncommitted changes from previous run. Starting new session to review and commit or discard them.

Uncommitted files:

?? ci-logs/opusplan-log.txt

---

Auto-restart will stop after changes are committed or discarded, or after 2 more iterations. Please wait until working session will end and give your feedback.

[konard]

🔄 Auto-restart 1/3 Log

This log file contains the complete execution trace of the AI solution draft process.

💰 Cost estimation:

• Public pricing estimate: $1.092313
• Calculated by Anthropic: $0.574404 USD
• Difference: $-0.517908 (-47.41%)

🤖 Models used:

• Tool: Anthropic Claude Code
• Requested: opus
• Model: Claude Opus 4.6 (claude-opus-4-6)

📎 Log file uploaded as Gist (3634KB)

• View complete solution draft log

---

Now working session is ended, feel free to review and add any feedback on the solution draft.

[konard]

✅ Ready to merge

This pull request is now ready to be merged:

• All CI checks have passed
• No merge conflicts
• No pending changes

---

Monitored by hive-mind with --auto-restart-until-mergeable flag

[konard]

hive@a7dd803b44df:~$ claude --output-format stream-json --verbose --dangerously-skip-permissions --model opusplan -p "hi"
{"type":"system","subtype":"init","cwd":"/home/hive","session_id":"a86f51f0-e61d-4591-8e2c-2cf2a34a7806","tools":["Task","AskUserQuestion","Bash","CronCreate","CronDelete","CronList","Edit","EnterPlanMode","EnterWorktree","ExitPlanMode","ExitWorktree","Glob","Grep","NotebookEdit","Read","RemoteTrigger","Skill","TaskOutput","TaskStop","TodoWrite","ToolSearch","WebFetch","WebSearch","Write"],"mcp_servers":[{"name":"claude.ai Google Calendar","status":"needs-auth"},{"name":"claude.ai Gmail","status":"needs-auth"}],"model":"claude-sonnet-4-6","permissionMode":"bypassPermissions","slash_commands":["update-config","debug","simplify","batch","loop","schedule","claude-api","compact","context","cost","heapdump","init","pr-comments","release-notes","review","security-review","extra-usage","insights"],"apiKeySource":"none","claude_code_version":"2.1.86","output_style":"default","agents":["general-purpose","statusline-setup","Explore","Plan"],"skills":["update-config","debug","simplify","batch","loop","schedule","claude-api"],"plugins":[],"uuid":"503d86d7-ee27-410a-979d-00abc828c4f1","fast_mode_state":"off"}
{"type":"assistant","message":{"model":"claude-sonnet-4-6","id":"msg_016KAPu4m6aViGXjSnvi4knf","type":"message","role":"assistant","content":[{"type":"text","text":"Hi! How can I help you today?"}],"stop_reason":null,"stop_sequence":null,"usage":{"input_tokens":3,"cache_creation_input_tokens":4083,"cache_read_input_tokens":11054,"cache_creation":{"ephemeral_5m_input_tokens":0,"ephemeral_1h_input_tokens":4083},"output_tokens":1,"service_tier":"standard","inference_geo":"not_available"},"context_management":null},"parent_tool_use_id":null,"session_id":"a86f51f0-e61d-4591-8e2c-2cf2a34a7806","uuid":"53b001fb-e367-45a1-8ade-66e0594567a4"}
{"type":"rate_limit_event","rate_limit_info":{"status":"allowed","resetsAt":1774782000,"rateLimitType":"five_hour","overageStatus":"rejected","overageDisabledReason":"org_level_disabled","isUsingOverage":false},"uuid":"e61377b3-a098-451c-b0a6-c40a48bbb2ac","session_id":"a86f51f0-e61d-4591-8e2c-2cf2a34a7806"}
{"type":"result","subtype":"success","is_error":false,"duration_ms":2819,"duration_api_ms":2750,"num_turns":1,"result":"Hi! How can I help you today?","stop_reason":"end_turn","session_id":"a86f51f0-e61d-4591-8e2c-2cf2a34a7806","total_cost_usd":0.01881645,"usage":{"input_tokens":3,"cache_creation_input_tokens":4083,"cache_read_input_tokens":11054,"output_tokens":12,"server_tool_use":{"web_search_requests":0,"web_fetch_requests":0},"service_tier":"standard","cache_creation":{"ephemeral_1h_input_tokens":4083,"ephemeral_5m_input_tokens":0},"inference_geo":"","iterations":[],"speed":"standard"},"modelUsage":{"claude-sonnet-4-6":{"inputTokens":3,"outputTokens":12,"cacheReadInputTokens":11054,"cacheCreationInputTokens":4083,"webSearchRequests":0,"costUSD":0.01881645,"contextWindow":200000,"maxOutputTokens":32000}},"permission_denials":[],"fast_mode_state":"off","uuid":"01eeda43-ba6f-4344-9814-c3232ed5a98a"}

hive@a7dd803b44df:~$ export ANTHROPIC_DEFAULT_OPUS_MODEL=claude-opus-4-6; claude --output-format stream-json --verbose --dangerously-skip-permissions --model opusplan -p "hi"
{"type":"system","subtype":"init","cwd":"/home/hive","session_id":"48dafb01-8c42-437b-a835-b89ee1fd56fa","tools":["Task","AskUserQuestion","Bash","CronCreate","CronDelete","CronList","Edit","EnterPlanMode","EnterWorktree","ExitPlanMode","ExitWorktree","Glob","Grep","NotebookEdit","Read","RemoteTrigger","Skill","TaskOutput","TaskStop","TodoWrite","ToolSearch","WebFetch","WebSearch","Write"],"mcp_servers":[{"name":"claude.ai Google Calendar","status":"needs-auth"},{"name":"claude.ai Gmail","status":"needs-auth"}],"model":"claude-sonnet-4-6","permissionMode":"bypassPermissions","slash_commands":["update-config","debug","simplify","batch","loop","schedule","claude-api","compact","context","cost","heapdump","init","pr-comments","release-notes","review","security-review","extra-usage","insights"],"apiKeySource":"none","claude_code_version":"2.1.86","output_style":"default","agents":["general-purpose","statusline-setup","Explore","Plan"],"skills":["update-config","debug","simplify","batch","loop","schedule","claude-api"],"plugins":[],"uuid":"72726c12-8f97-4a65-b339-55087e2e7217","fast_mode_state":"off"}
{"type":"assistant","message":{"model":"claude-sonnet-4-6","id":"msg_01R5gPq8rJYafhPmgohZgBj9","type":"message","role":"assistant","content":[{"type":"text","text":"Hi! How can I help you?"}],"stop_reason":null,"stop_sequence":null,"usage":{"input_tokens":3,"cache_creation_input_tokens":0,"cache_read_input_tokens":15137,"cache_creation":{"ephemeral_5m_input_tokens":0,"ephemeral_1h_input_tokens":0},"output_tokens":1,"service_tier":"standard","inference_geo":"not_available"},"context_management":null},"parent_tool_use_id":null,"session_id":"48dafb01-8c42-437b-a835-b89ee1fd56fa","uuid":"79fa4194-e3e2-4065-903f-6f1464f61212"}
{"type":"rate_limit_event","rate_limit_info":{"status":"allowed","resetsAt":1774782000,"rateLimitType":"five_hour","overageStatus":"rejected","overageDisabledReason":"org_level_disabled","isUsingOverage":false},"uuid":"0d510603-1839-4afb-a5b7-3fecf3646248","session_id":"48dafb01-8c42-437b-a835-b89ee1fd56fa"}
{"type":"result","subtype":"success","is_error":false,"duration_ms":1555,"duration_api_ms":1507,"num_turns":1,"result":"Hi! How can I help you?","stop_reason":"end_turn","session_id":"48dafb01-8c42-437b-a835-b89ee1fd56fa","total_cost_usd":0.004715099999999999,"usage":{"input_tokens":3,"cache_creation_input_tokens":0,"cache_read_input_tokens":15137,"output_tokens":11,"server_tool_use":{"web_search_requests":0,"web_fetch_requests":0},"service_tier":"standard","cache_creation":{"ephemeral_1h_input_tokens":0,"ephemeral_5m_input_tokens":0},"inference_geo":"","iterations":[],"speed":"standard"},"modelUsage":{"claude-sonnet-4-6":{"inputTokens":3,"outputTokens":11,"cacheReadInputTokens":15137,"cacheCreationInputTokens":0,"webSearchRequests":0,"costUSD":0.004715099999999999,"contextWindow":200000,"maxOutputTokens":32000}},"permission_denials":[],"fast_mode_state":"off","uuid":"f07e7adb-b022-4214-91ce-c484dd2f41ae"}

hive@a7dd803b44df:~$ export ANTHROPIC_DEFAULT_OPUS_MODEL=claude-opus-4-6; export ANTHROPIC_MODEL=opusplan; claude --output-format stream-json --verbose --dangerously-skip-permissions -p "hi"
{"type":"system","subtype":"init","cwd":"/home/hive","session_id":"61faa829-c79b-4423-93a7-63011de2e9a6","tools":["Task","AskUserQuestion","Bash","CronCreate","CronDelete","CronList","Edit","EnterPlanMode","EnterWorktree","ExitPlanMode","ExitWorktree","Glob","Grep","NotebookEdit","Read","RemoteTrigger","Skill","TaskOutput","TaskStop","TodoWrite","ToolSearch","WebFetch","WebSearch","Write"],"mcp_servers":[{"name":"claude.ai Google Calendar","status":"needs-auth"},{"name":"claude.ai Gmail","status":"needs-auth"}],"model":"claude-sonnet-4-6","permissionMode":"bypassPermissions","slash_commands":["update-config","debug","simplify","batch","loop","schedule","claude-api","compact","context","cost","heapdump","init","pr-comments","release-notes","review","security-review","extra-usage","insights"],"apiKeySource":"none","claude_code_version":"2.1.86","output_style":"default","agents":["general-purpose","statusline-setup","Explore","Plan"],"skills":["update-config","debug","simplify","batch","loop","schedule","claude-api"],"plugins":[],"uuid":"14a59cd7-86f7-47b9-b9eb-6d83ca2d583e","fast_mode_state":"off"}
{"type":"assistant","message":{"model":"claude-sonnet-4-6","id":"msg_019LrQ8GTD7e5xtg4HJFKncG","type":"message","role":"assistant","content":[{"type":"text","text":"Hi! How can I help you?"}],"stop_reason":null,"stop_sequence":null,"usage":{"input_tokens":3,"cache_creation_input_tokens":0,"cache_read_input_tokens":15137,"cache_creation":{"ephemeral_5m_input_tokens":0,"ephemeral_1h_input_tokens":0},"output_tokens":1,"service_tier":"standard","inference_geo":"not_available"},"context_management":null},"parent_tool_use_id":null,"session_id":"61faa829-c79b-4423-93a7-63011de2e9a6","uuid":"a343a9cd-55d4-42f7-bffe-a371fd3d97ae"}
{"type":"rate_limit_event","rate_limit_info":{"status":"allowed","resetsAt":1774782000,"rateLimitType":"five_hour","overageStatus":"rejected","overageDisabledReason":"org_level_disabled","isUsingOverage":false},"uuid":"b197801c-9447-402c-8b1d-eb640cf50d65","session_id":"61faa829-c79b-4423-93a7-63011de2e9a6"}
{"type":"result","subtype":"success","is_error":false,"duration_ms":2859,"duration_api_ms":2810,"num_turns":1,"result":"Hi! How can I help you?","stop_reason":"end_turn","session_id":"61faa829-c79b-4423-93a7-63011de2e9a6","total_cost_usd":0.004715099999999999,"usage":{"input_tokens":3,"cache_creation_input_tokens":0,"cache_read_input_tokens":15137,"output_tokens":11,"server_tool_use":{"web_search_requests":0,"web_fetch_requests":0},"service_tier":"standard","cache_creation":{"ephemeral_1h_input_tokens":0,"ephemeral_5m_input_tokens":0},"inference_geo":"","iterations":[],"speed":"standard"},"modelUsage":{"claude-sonnet-4-6":{"inputTokens":3,"outputTokens":11,"cacheReadInputTokens":15137,"cacheCreationInputTokens":0,"webSearchRequests":0,"costUSD":0.004715099999999999,"contextWindow":200000,"maxOutputTokens":32000}},"permission_denials":[],"fast_mode_state":"off","uuid":"c36a452a-e4a8-4d47-8240-6dd332f9852a"}

konard@MacBook-Pro-Konstantin ~ % claude --output-format stream-json --model opusplan -p "hi"
{"type":"system","subtype":"init","cwd":"/Users/konard","session_id":"7e0dcb1f-c65f-4528-8a5d-5c21e5c03860","tools":["Task","AskUserQuestion","Bash","CronCreate","CronDelete","CronList","Edit","EnterPlanMode","EnterWorktree","ExitPlanMode","ExitWorktree","Glob","Grep","LSP","NotebookEdit","Read","RemoteTrigger","Skill","TaskOutput","TaskStop","TodoWrite","ToolSearch","WebFetch","WebSearch","Write","mcp__playwright__browser_click","mcp__playwright__browser_close","mcp__playwright__browser_console_messages","mcp__playwright__browser_drag","mcp__playwright__browser_evaluate","mcp__playwright__browser_file_upload","mcp__playwright__browser_fill_form","mcp__playwright__browser_handle_dialog","mcp__playwright__browser_hover","mcp__playwright__browser_install","mcp__playwright__browser_navigate","mcp__playwright__browser_navigate_back","mcp__playwright__browser_network_requests","mcp__playwright__browser_press_key","mcp__playwright__browser_resize","mcp__playwright__browser_run_code","mcp__playwright__browser_select_option","mcp__playwright__browser_snapshot","mcp__playwright__browser_tabs","mcp__playwright__browser_take_screenshot","mcp__playwright__browser_type","mcp__playwright__browser_wait_for"],"mcp_servers":[{"name":"playwright","status":"connected"},{"name":"claude.ai Google Calendar","status":"needs-auth"},{"name":"claude.ai Gmail","status":"needs-auth"}],"model":"claude-sonnet-4-6","permissionMode":"acceptEdits","slash_commands":["update-config","debug","simplify","batch","loop","schedule","claude-api","playwright-cli","compact","context","cost","heapdump","init","pr-comments","release-notes","review","security-review","extra-usage","insights"],"apiKeySource":"none","claude_code_version":"2.1.86","output_style":"default","agents":["general-purpose","statusline-setup","Explore","Plan"],"skills":["update-config","debug","simplify","batch","loop","schedule","claude-api","playwright-cli"],"plugins":[{"name":"rust-analyzer-lsp","path":"/Users/konard/.claude/plugins/marketplaces/claude-plugins-official/plugins/rust-analyzer-lsp","source":"rust-analyzer-lsp@claude-plugins-official"}],"uuid":"8024dfc9-ab03-42c4-acfd-44179e3e9e12","fast_mode_state":"off"}
{"type":"assistant","message":{"model":"claude-sonnet-4-6","id":"msg_01ENWAmThEJQ9zs1F4m11gzW","type":"message","role":"assistant","content":[{"type":"text","text":"Hi! How can I help you?"}],"stop_reason":null,"stop_sequence":null,"usage":{"input_tokens":3,"cache_creation_input_tokens":15411,"cache_read_input_tokens":0,"cache_creation":{"ephemeral_5m_input_tokens":0,"ephemeral_1h_input_tokens":15411},"output_tokens":1,"service_tier":"standard","inference_geo":"not_available"},"context_management":null},"parent_tool_use_id":null,"session_id":"7e0dcb1f-c65f-4528-8a5d-5c21e5c03860","uuid":"b0d07d44-a439-4056-a1f6-202f297fb495"}
{"type":"rate_limit_event","rate_limit_info":{"status":"allowed","resetsAt":1774782000,"rateLimitType":"five_hour","overageStatus":"rejected","overageDisabledReason":"org_level_disabled","isUsingOverage":false},"uuid":"e89ee632-b560-47a5-91a1-5140ab7fd937","session_id":"7e0dcb1f-c65f-4528-8a5d-5c21e5c03860"}
{"type":"result","subtype":"success","is_error":false,"duration_ms":2570,"duration_api_ms":2551,"num_turns":1,"result":"Hi! How can I help you?","stop_reason":"end_turn","session_id":"7e0dcb1f-c65f-4528-8a5d-5c21e5c03860","total_cost_usd":0.057965249999999996,"usage":{"input_tokens":3,"cache_creation_input_tokens":15411,"cache_read_input_tokens":0,"output_tokens":11,"server_tool_use":{"web_search_requests":0,"web_fetch_requests":0},"service_tier":"standard","cache_creation":{"ephemeral_1h_input_tokens":15411,"ephemeral_5m_input_tokens":0},"inference_geo":"","iterations":[],"speed":"standard"},"modelUsage":{"claude-sonnet-4-6":{"inputTokens":3,"outputTokens":11,"cacheReadInputTokens":0,"cacheCreationInputTokens":15411,"webSearchRequests":0,"costUSD":0.057965249999999996,"contextWindow":200000,"maxOutputTokens":32000}},"permission_denials":[],"fast_mode_state":"off","uuid":"1b29982f-e1b1-4ed3-b168-5d8e7ef529a2"}

konard@MacBook-Pro-Konstantin ~ % claude
▗ ▗   ▖ ▖  Claude Code v2.1.86                                                                                                                                 
           Sonnet 4.6 with low effort · Claude Max                                                                                                             
  ▘▘ ▝▝    /Users/konard                                                                                                                                       
                                                                                                                                                               
❯ /model                                                                                                                                                       
  ⎿  Kept model as Opus 4.6 (1M context) (default)                                                                                                             
                                                                                                                                                               
❯ /model opusplan                                                                                                                                              
  ⎿  Set model to Opus 4.6 in plan mode, else Sonnet 4.6                                                                                                       
                                                                                                                                                               
❯ hi?                                                                                                                                                          
                                                                                                                                                               
⏺ Hi! How can I help you?                                                                                                                                      
                                                                                                                                                               
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
❯                                                                                                                                                              
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
  ⏵⏵ accept edits on (shift+tab to cycle)                                                                                                        15866 tokens                                                                                                                                                            

Looks like it does not work at all, may be we should add these to Claude Code issue we have created, so there that is really easy to reproduce.

We can also try to make this mode work ourselves, as alternative mode like --plan-mode, that will first use:

--permission-mode plan

with opus, after that we need to find a way to switch to sonnet for execution. May be claude command does allow to use not only streaming output, but also streaming input.

Looks like it is only about permissions: https://gist.github.com/konard/1b7b4bedb285a69cc1c54169364d73ef

So in addition to --permission-mode plan, we might need some prompting, capturing of the final message with plan from opus, and using it as input for sonnet.

So we can do alternative experimental --plan-mode, that will coexist with --plan, as Claude Code eventually can fix the problem on their side, and may be their implementation of opusplan will be better. But --plan-mode can reuse the same options for configuration of models like --plan-model and --worker-model.

Also in --plan-mode we should have special plan mode system message, that will contain all relevant ideas from main system prompt for worker, but also will ask opus to not ask interactive questions or any questions, instead prepare detailed plan and that will be given to execution immediately. So in --plan-mode opus model will be asked to just produce the plan for execution with no questions asked. Yet the plan itself can contain asking questions as plan items, so sonnet will be able to communicate with user with GitHub comments if needed, but opus should prepare all sub plans or all instructions what will cover all possible answers to questions.

So in short opus should do only planing, including planing for questions, but it should present the entire plan, that will also plan for different answer possibilities. And sonnet will do the execution, including question asking if needed.