openai-chat streaming: emit ToolCallDelta so the stall watchdog sees tool-call progress (mu-b82rr) - #579
Conversation
|
ci-aipr board outcome: PANEL ESCALATE (no consensus after 4 rounds) — adjudicated APPROVE per the live-seats rule. Final-round positions: glm-5.2 approve, kimi-k3 approve; claude-opus-4-8 and claude-opus-5 seats structurally unparsed (claude-oauth cannot run nested under a claude-code session — known issue), gpt-5.5 timed out rounds 2-4. Every seat that produced a parseable vote in the final round approved. The one substantive needs-changes (kimi-k3, round 1) claimed the regression test's second SSE fragment is malformed (misplaced brace putting Log: scratchpad phase2/ci-aipr-b82rr.log (session 442e5925). Consensus artifacts: /tmp/ai-review-consensus.Nyxw7a. |
…tool-call progress (mu-b82rr) The openai-chat provider accumulated tool-call fragments silently and surfaced them only in Done — a v1 shortcut from before the loop's stall watchdog (mu-197pd) learned to credit ToolCallDelta bytes. A tool call whose arguments stream longer than STREAM_STALL_SECS with no text or reasoning deltas (a large file written via one write call on a local lane) was misread as a dead connection and killed at exactly 300s. Emit a ToolCallDelta per fragment chunk (one-event-per-chunk slot preserved; Done-side assembly unchanged) and update the streaming tests to the new event counts, plus a regression test shaped like the failing capture.
44533c8 to
d3658b2
Compare
1 participant
The openai-chat provider accumulated tool-call fragments silently and surfaced them only in Done. That was a v1 shortcut from before the stall watchdog in the agent loop (mu-197pd) learned to credit ToolCallDelta bytes as liveness. The two halves never met: the anthropic provider emits tool-call deltas and the loop counts them, but the openai-chat path emits nothing between the last text delta and Done.
The consequence is that any single tool call whose arguments stream longer than STREAM_STALL_SECS (300s, const) with no interleaved text or reasoning is misread as a dead connection and the session is killed. At local-model speeds around 60 tok/s that is any file of roughly 50KB or more written through one write call. Found live during the mu-316wl phase-2 harness experiment: qwen3.8-27b on a vllm openai-chat endpoint streamed 51,834 characters of write-tool arguments at steady engine throughput (60-68 tok/s in the serve log the whole window) and mu killed the run at exactly 300s with "provider stream stalled: no bytes for 300s (streaming phase)". Full wire capture and engine-side throughput trace are on beads mu-b82rr and mu-316wl.
The change emits a ProviderEvent::ToolCallDelta per tool-call fragment chunk, preserving the existing one-event-per-chunk slot and leaving Done-side assembly untouched. Continuation fragments may carry an empty id on some backends; the builder's id is used, and the loop only uses the id for status display. The two streaming tests that encoded the old behavior (b8, b9) now expect the deltas, and a regression test shaped like the failing capture (multi-chunk write call, then Done) is added.
Verified: cargo test -p mu-ai --lib all green (239), fmt and clippy clean. A patched binary re-run of the failing experiment cell is in flight and its outcome will be posted on mu-b82rr.
🤖 Generated with Claude Code
https://claude.ai/code/session_01Mgi1TvVjFvaSi5otptH68o