feat(dflash): K=γ verify model orchestration + EAGLE ctx-accumulator contiguity fix

[Sujimoshi]

Summary

Next cluster from the #218 split ("DFlash K=γ correctness + throughput optimizations"). Wires the SSM-layer K=γ verify kernels landed in #290 into the model's verify/decode/append orchestration layer.

Stacked on #290 (which stacks on #258) — merge order: #258, #290, then this.

Contents (crates/spark-model/src/model/trait_impl/):

• verify_d.rs, verify_d_kgamma_layers.rs (new), verify_d_kgamma_attn_meta.rs (new) — K=γ verify step implementation using the WY16 GDN kernel and batched phase-1 conv from feat(dflash): SSM-layer K=γ verify kernels — WY16 GDN, batched phase-1 conv, opt-in F32 GDN output #290.
• dflash_append.rs (new) — EAGLE ctx-slot append dispatchers: dflash_accept_append, dflash_eagle_accept_append, dflash_eagle_kgamma_append, plus dflash_decode_ctx_append_dispatch.
• decode_a.rs — calls dflash_decode_ctx_append_dispatch unconditionally at the end of every decode() step (both CUDA-graph-replay and non-graph paths), fixing a real bug: the EAGLE ctx accumulator (ctx_hidden_acc) previously only advanced from the DFlash verify/propose path, so when DFlash gating turned off for a stretch (e.g. inside a <think> block) and resumed, the drafter's first post-resume propose() conditioned on stale/non-contiguous history.
• verify_a.rs, verify_b.rs, prefill_b/batch_kernel.rs, prefill_c.rs, mod.rs — supporting wiring (new trait methods, batch-kernel/prefill hookups).
• crates/spark-model/src/layers/dflash_head.rs / propose.rs — minimal addition: a skip_next_decode_append one-shot flag on DflashProposerState, set by the new dflash_append.rs dispatchers when they've already appended the relevant ctx row(s) before propose() runs, and consumed by propose_drafts to avoid double-appending. Scoped narrowly — the rest of dflash_head/propose.rs (the "dflash_head core" cluster: last_num_accepted, forward_block changes, etc.) is left for a later PR in the stack.

Deviations from #218 as-authored

• NVTX instrumentation (nvtx_diag::NvtxRange spans in verify_d.rs / verify_d_kgamma_layers.rs) stripped — that module doesn't exist yet on this stack (belongs to perf(spark-runtime): opt-in NVTX kernel-attribution + phase-timing instrumentation #255, not yet merged). Substantive comments explaining sync/D2H safety were kept, just de-framed from "NVTX stage" language.
• GdnPrefillBuffers struct-literal duplicate output_f32/output_f32_written fields (added independently by both feat(dflash): SSM-layer K=γ verify kernels — WY16 GDN, batched phase-1 conv, opt-in F32 GDN output #290 and dflash: DFlash K=γ correctness + throughput optimizations (verify 585→269ms, tok/s 2.9→22) #218's original diff at the same call sites) de-duplicated in prefill_b/batch_kernel.rs and prefill_c.rs, keeping the real wired value in prefill_c.rs.

Test plan

• cargo check -p spark-model / cargo check --workspace
• cargo fmt --all -- --check
• cargo clippy --workspace --tests (crate-wide #![deny(clippy::all)])
• file-size-cap (≤500 LoC/file)
• GPU validation on GB10 (Qwen3.6-27B-NVFP4 + DFlash draft, T=0, greedy vs dflash):
  • Default K=2 path: divergence at byte 899 vs greedy baseline (no regression vs feat(dflash): SSM-layer K=γ verify kernels — WY16 GDN, batched phase-1 conv, opt-in F32 GDN output #290's byte 634 — normal run-to-run variance), no crashes/NaN.
  • Forced K=γ path (ATLAS_DFLASH_DRAFT_CAP=16): CUDA graph captures correctly (K=17 slot), but accept rate remains 0/16 (0%) across all verify calls — identical to feat(dflash): SSM-layer K=γ verify kernels — WY16 GDN, batched phase-1 conv, opt-in F32 GDN output #290's pre-existing finding, unchanged by this PR's ctx-accumulator fix. Confirms the K=γ 0%-accept collapse is NOT caused by ctx-accumulator discontiguity; root cause remains the deeper SSM state-management mismatch for K∉{2,3,4} already documented in propose.rs's safety-cap comment. Tracked as a separate, still-open issue — out of scope for this PR.

🤖 Generated with Claude Code

[SeedSource]

[BETA] Automated PR review — SeedSource AI pipeline

Advisory only, not a merge gate. AI-generated (internal risk-triage, Atlas serve gate, GLM-5.2 review), attributed per Atlas convention.

Triage

• Risk: med - serve-test-required (triage advisory): True - path-heuristic (authority): True
• Rationale: NVFP4 lm_head vocab sizing fix and GDN verify buffers added for DFlash drafter.
• Serving-path files: crates/spark-model/src/factory/build.rs, crates/spark-model/src/layer.rs, crates/spark-model/src/layer/transformer_layer.rs, crates/spark-model/src/layers/dflash_head.rs, crates/spark-model/src/layers/dflash_head/forward_block.rs, crates/spark-model/src/layers/dflash_head/from_weights.rs

Serve gate

• Build: 17s - Qwen3.6-35B-A3B-FP8 on GB10 node1
• Top-k KL vs main: 0.0008 (no shift vs main) - decode delta -0.3%
• Coherence probe: coherence=pass ping=10/10 (note: our test_coherence.py is flaky here; KL is the reliable serve signal)

GLM-5.2 code review

The diff is truncated mid-file (layers/dflash_head.rs), so the actual implementation of prefill_phase1_verify and prefill_gdn_wy16 overrides is not visible — cannot fully assess correctness of the K=γ verify path. The trait additions look structurally sound with safe Ok(false) defaults.

Concerns:

1. target_lm_head_nvfp4_vocab_for_dflash: The comment claims config.vocab_size is the capped value, but this is asserted without evidence in the diff. If cap_vocab_size_to_tokenizer mutates config.vocab_size in place, fine; if it stores the cap elsewhere, this passes the uncapped value — the exact OOB the comment warns about. Needs verification.

2. output_f32_written: Cell<bool>: Cell interior mutability in a Send + Sync trait object context is a smell. If GdnPrefillBuffers is shared across threads (batched multi-stream), this is a data race waiting to happen. Confirm buffers are per-stream/per-call.

3. Doc comment removals: The # Arguments blocks were collapsed into prose. Information loss is minor but the per-argument semantics (e.g., seq_len = "starting length before these tokens") are now harder to scan.

Risk: Medium — the vocab-size fix addresses a real OOB, but the truncation hides whether the consumer side uses it correctly. Request full diff for dflash_head.rs and the SSM layer override.

_{Pipeline: build+serve on GB10, GLM-5.2 via NVIDIA NIM, internal risk-triage classifier. External diff treated as untrusted input. Public repo.}