Skip to content

fix(dsv4 sparse-MLA): three cr=4 correctness bugs (fwd NaN, bwd dQ race, bwd dkv misdispatch)#417

Open
wenxie-amd wants to merge 1 commit into
dev/kyle/flydsl_attn_deepseekv4from
dev/wenx/dsv4-flydsl-cr4-correctness-fixes
Open

fix(dsv4 sparse-MLA): three cr=4 correctness bugs (fwd NaN, bwd dQ race, bwd dkv misdispatch)#417
wenxie-amd wants to merge 1 commit into
dev/kyle/flydsl_attn_deepseekv4from
dev/wenx/dsv4-flydsl-cr4-correctness-fixes

Conversation

@wenxie-amd

Copy link
Copy Markdown
Collaborator

Summary

The native-FlyDSL DeepSeek-V4 sparse-MLA v2 attention (primus_turbo/flydsl/attention/kernels/sparse_mla_v2) produced wrong results for the CSA (cr=4) layer. Three independent, cr=4-only bugs, found by comparing fwd+bwd against an fp32 eager reference (and the triton_v2 / gluon_v2 bf16 anchors):

1. Forward fast_path NaN (dsa_fwd.py). The first-pair fixed-max softmax bound _mb[0] = crossgrp_max(...) is -inf when the first key-pair is fully masked (query token t<96 → SWA slots [t-127..t-96] all -1-inf), so exp2(score-(-inf)) = +infNaN on the first 96 query tokens (all seq lengths, flash+pro). Fix: floor the bound to finite, maxnumf(crossgrp_max, 0.0). The common case (fp_max>0) is unchanged and bf16 relative precision is magnitude-independent, so the alpha≡1 fold is preserved → speed-neutral.

2. Backward dQ nondeterministic cross-wave race (dsa_bwd.py::_get_dq). build_bwd_dq (the topk<512 single-tile + inline-delta dQ kernel) corrupts dq d-tile 0 on scattered query tokens at TOPK=384 (cr=4 @ S=1024); the NaN count varies run-to-run. build_bwd_dq_pvk32 (K=32 pair-batch, explicit per-pair WAR barrier) is race-free. Fix: lower the pvk32 dispatch gate 512 → 384 (coupled with the wrapper's dq_folds_delta threshold so the standalone-delta / no-O args stay consistent). The 512 gate was perf-only; at TOPK=384 the rerouted pvk32 bwd is actually slightly faster. Production S=4096 (topk≥512) already used pvk32 → unaffected.

3. Backward dkv small-seq misdispatch (dsa_bwd.py). is_cr128 = (num_kv>total_tokens) and (topk<=256) also catches cr=4 (CSA, random pool) at small seq (S≤512topk≤256), routing its random pool through the cr=128 (HCA) closed-form causal-pool gather → wrong dkv/dpool (e.g. S=256 relL2=0.83 vs eager/triton/gluon). Fix: guard is_cr128 with the same deterministic-pool condition the forward uses (T % npool == 0 and T // npool >= 64) so cr=4 (pool_cr=4) falls through to the CSR gather. Production S=4096 (topk>256) already used CSR → unaffected.

Test plan

  • fp32-eager accuracy suite: flash+pro × cr{0,4,128}, fwd (o, lse) + bwd (dq, dkv, d_sink) at S ∈ {256, 512, 1024, 4096} → goes from SOME FAILED → ALL PASS; cr=4 bwd dq clean over 6/6 repeated runs (race gone).
  • Cross-checked vs triton_v2 / gluon_v2 bf16 anchors.
  • All three fixes speed-neutral (bug 2 slightly faster at the rerouted small-seq shape); S=4096 benchmark unchanged.

Note: dense/HCA banded path is still single-sequence (B=1); multi-batch (mbs>1) dense/HCA is a separate known limitation (the closed-form SWA window crosses batch boundaries) and is out of scope for this correctness PR.

…ce, bwd dkv misdispatch)

The native-FlyDSL DeepSeek-V4 sparse-MLA v2 attention
(primus_turbo/flydsl/attention/kernels/sparse_mla_v2) produced wrong results
for the CSA (cr=4) layer. Three independent bugs, all cr=4-only:

1. fwd fast_path NaN (dsa_fwd.py). The first-pair fixed-max softmax bound
   `_mb[0] = crossgrp_max(...)` is -inf when the first key-pair is fully masked
   (query token t<96 -> SWA slots [t-127..t-96] all -1 -> -inf), so
   exp2(score-(-inf))=+inf -> NaN on the first 96 query tokens (all seq lengths,
   flash+pro). Fix: floor the bound to finite, maxnumf(crossgrp_max, 0.0). The
   common case (fp_max>0) is unchanged and bf16 relative precision is
   magnitude-independent, so it keeps the alpha=1 fold -> speed-neutral.

2. bwd dQ nondeterministic cross-wave race (dsa_bwd.py `_get_dq`). build_bwd_dq
   (the topk<512 single-tile + inline-delta dQ kernel) corrupts dq d-tile 0 on
   ~scattered query tokens at TOPK=384 (cr=4 @ S=1024); the NaN count varies
   run-to-run. build_bwd_dq_pvk32 (K=32 pair-batch, explicit per-pair WAR barrier)
   is race-free. Fix: lower the pvk32 dispatch gate 512 -> 384 (coupled with the
   wrapper's dq_folds_delta threshold so the standalone delta / no-O args stay
   consistent). The 512 gate was a perf choice, not correctness; at TOPK=384 the
   rerouted pvk32 bwd is actually slightly faster. Production S=4096 (topk>=512)
   already used pvk32 and was unaffected.

3. bwd dkv small-seq misdispatch (dsa_bwd.py). `is_cr128 = (num_kv>total_tokens)
   and (topk<=256)` also catches cr=4 (CSA, RANDOM pool) at small seq (S<=512 ->
   topk<=256), routing its random pool through the cr=128 (HCA) closed-form causal
   pool gather -> wrong dkv/dpool (e.g. S=256 relL2=0.83 vs eager/triton/gluon).
   Fix: guard is_cr128 with the same deterministic-pool condition the forward uses
   (T % npool == 0 and T // npool >= 64) so cr=4 (pool_cr=4) falls through to the
   CSR gather. Production S=4096 (topk>256) already used CSR and was unaffected.

Validated vs an fp32 eager reference (and the triton_v2 / gluon_v2 bf16 anchors):
after the fixes, flash+pro x cr{0,4,128} fwd (o, lse) and bwd (dq, dkv, d_sink)
all pass at S in {256, 512, 1024, 4096}; the standalone accuracy suite goes from
"SOME FAILED" to "ALL PASS". All three fixes are speed-neutral (bug 2 slightly
faster at the rerouted small-seq shape); the S=4096 benchmark is unchanged.

Co-authored-by: Cursor <cursoragent@cursor.com>
kyle-256 added a commit that referenced this pull request Jul 14, 2026
Third cr=4 correctness bug (companion to the fwd NaN + bwd dQ race fixed in
10966dd). The cr=128 (HCA) closed-form pool gather assumes a deterministic
causal pool (pool_cr = T/npool). The bare `topk<=256` gate also catches cr=4
(CSA, random pool) at small seq (S<=512 -> topk<=256), routing its random pool
through the closed-form gather -> wrong dkv/dpool (seen: S=512 cr=4 bwd dkv
1.0 dB). Guard is_cr128 with the deterministic-pool condition the forward uses
(T % npool == 0 and T // npool >= 64) so cr=4 (pool_cr=4) falls through to the
CSR gather. Production S=4096 (topk>256) was already CSR -> unchanged.

Verified on MI355X (chi2798) vs fp32 eager: S=512 cr=4 bwd dkv 1.0 -> 50.8 dB;
seq=512 and seq=4096 all 6 shapes ALL PASS (real cr=128 still closed-form).
Matches #417 (dev/wenx/dsv4-flydsl-cr4-correctness-fixes).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants