fix(qwen3): size decode page indices for prefix-shared views

[n-WN]

Problem

BatchDecodeBuffers::page_indices_d is sized total_blocks + max_bs, assuming the concatenated per-request page lists of a decode step are bounded by the physical pool. Prefix caching breaks that assumption: matched blocks are shared ImmutableBlocks, so N views holding the same cached prefix each list those page ids again — the concatenated list is counted by reference, not by physical block. Admission cannot catch it either: request_lifetime_blocks charges physical blocks, and sharing makes physical usage far smaller than the by-reference count.

Once the sum crosses the buffer size, sync_paged_meta's memcpy_htod trips cudarc's dst.len() >= src.len() assert, killing qwen3-tp-rank-0; every later step fails with tensor-parallel worker step channel closed while /health stays 200 — the permanent wedge of #403 (fault 1; fault 2, the blast radius, is #420's territory). Backtrace from a live repro on main @ 1275262:

3: cudarc::driver::safe::core::CudaStream::memcpy_htod::<i32, ...>
4: openinfer_qwen3_4b::batch_decode_buffers::BatchDecodeBuffers::sync_paged_meta
5: openinfer_qwen3_4b::weights::Qwen3Model::batch_decode
6: openinfer_qwen3_4b::executor::execute_step_on_lane

This also explains the two puzzles in the issue report. A one-shot 160-request burst cannot reproduce: blocks only become matchable once sealed, and simultaneous prefills do not match each other — sharing needs sustained arrivals over an already-sealed prefix. And the 5070 Ti repro is intermittent because scripts/bench_http_serving.py's 24-word vocabulary yields only 24 distinct prompts, so the shared double-count hovers right at the + max_bs slack the old sizing left.

With a shared prefix the failure is deterministic, not intermittent: warm one long prompt, then fire N concurrent requests with the same prompt (repro script in #403). Common prefixes are the serving norm — a system prompt is exactly this shape.

Change

• Size page_indices_d for the reachable worst case — every decode row presenting a full-context view: max_batch × ceil(max_context_tokens / page_size). The constructor takes page_size instead of max_total_pages; the physical pool size no longer appears in the bound. For Qwen3-4B (bucket 256 × 2560 pages) this is 2.6 MiB of i32 vs ~98 KiB before — noise next to the pool it indexes, and the memory profiler builds the same buffers, so profiled KV sizing absorbs it automatically.
• Fail loud in sync_paged_meta if the concatenated list ever exceeds the buffer (reachable only if a view exceeds the context limit the buffers were sized for): a step Err instead of a dead worker thread.
• Regression test shared_prefix_views_are_counted_by_reference: N views listing the same physical pages sync fine; oversized views return the error instead of tripping the assert. GPU-only, no weights needed.
• docs/models/qwen3/prefix-cache.md: recorded the by-reference page accounting rule so the next concatenated-page-list consumer doesn't re-derive it.

Verification (H100 80GB, sm_90, CUDA 12.4, Qwen3-8B serving / Qwen3-4B gates)

Repro A/B — warm a 3000-word prompt, then 256 concurrent identical requests (max_tokens 64):

	main @ 1275262	this branch
requests OK	193/256 (63 × HTTP 500)	256/256
worker panic	cudarc assert (backtrace above)	none
engine after	wedged: trivial completion 500, /health 200	healthy: completion 200

Gates:

cargo test --release -p openinfer-qwen3-4b -p openinfer-core -p openinfer-kv-cache -p openinfer-sample --lib   # 76 passed
OPENINFER_TEST_MODEL_PATH=<Qwen3-4B> cargo test --release -p openinfer-qwen3-4b --test hf_golden_gate           # pass, 31.5s
OPENINFER_TEST_MODEL_PATH=<Qwen3-4B> cargo test --release -p openinfer-qwen3-4b --test prefix_cache             # pass, 8.3s

(--workspace --lib is not runnable on this box — openinfer-cupti needs CUPTI headers the CUDA 12.4 install lacks; the per-crate runs above cover every touched crate.)

Serving A/B, same box/base, vllm bench serve random 256-out conc=64, N=200 (allocation-only change — expected noise, observed noise):

workload	metric	main	this branch
128-in	output tok/s	5421.6	5514.2
128-in	TPOT mean	9.24 ms	9.03 ms
2048-in	output tok/s	2030.7	2030.5
2048-in	TPOT mean	25.96 ms	26.11 ms

Notes for review

• qwen35's decode_buffers.rs:116 has the same physical-pool sizing. It is unreachable today (no prefix cache on that line yet), so it is flagged for the qwen35 prefix-cache design (qwen35: prefix caching needs a design for recurrent state (discussion) #257) rather than changed here.
• The new ensure! is defense in depth: with the honest bound it should never fire; if it does, it composes with fix(scheduler): contain worker fatal errors #420's containment (step error, not dead worker).

🤖 Generated with Claude Code