fix(dsv4 sparse-MLA): three cr=4 correctness bugs (fwd NaN, bwd dQ race, bwd dkv misdispatch)#417
Open
wenxie-amd wants to merge 1 commit into
Conversation
wenxie-amd
commented
Jul 14, 2026
Collaborator
…ce, bwd dkv misdispatch)
The native-FlyDSL DeepSeek-V4 sparse-MLA v2 attention
(primus_turbo/flydsl/attention/kernels/sparse_mla_v2) produced wrong results
for the CSA (cr=4) layer. Three independent bugs, all cr=4-only:
1. fwd fast_path NaN (dsa_fwd.py). The first-pair fixed-max softmax bound
`_mb[0] = crossgrp_max(...)` is -inf when the first key-pair is fully masked
(query token t<96 -> SWA slots [t-127..t-96] all -1 -> -inf), so
exp2(score-(-inf))=+inf -> NaN on the first 96 query tokens (all seq lengths,
flash+pro). Fix: floor the bound to finite, maxnumf(crossgrp_max, 0.0). The
common case (fp_max>0) is unchanged and bf16 relative precision is
magnitude-independent, so it keeps the alpha=1 fold -> speed-neutral.
2. bwd dQ nondeterministic cross-wave race (dsa_bwd.py `_get_dq`). build_bwd_dq
(the topk<512 single-tile + inline-delta dQ kernel) corrupts dq d-tile 0 on
~scattered query tokens at TOPK=384 (cr=4 @ S=1024); the NaN count varies
run-to-run. build_bwd_dq_pvk32 (K=32 pair-batch, explicit per-pair WAR barrier)
is race-free. Fix: lower the pvk32 dispatch gate 512 -> 384 (coupled with the
wrapper's dq_folds_delta threshold so the standalone delta / no-O args stay
consistent). The 512 gate was a perf choice, not correctness; at TOPK=384 the
rerouted pvk32 bwd is actually slightly faster. Production S=4096 (topk>=512)
already used pvk32 and was unaffected.
3. bwd dkv small-seq misdispatch (dsa_bwd.py). `is_cr128 = (num_kv>total_tokens)
and (topk<=256)` also catches cr=4 (CSA, RANDOM pool) at small seq (S<=512 ->
topk<=256), routing its random pool through the cr=128 (HCA) closed-form causal
pool gather -> wrong dkv/dpool (e.g. S=256 relL2=0.83 vs eager/triton/gluon).
Fix: guard is_cr128 with the same deterministic-pool condition the forward uses
(T % npool == 0 and T // npool >= 64) so cr=4 (pool_cr=4) falls through to the
CSR gather. Production S=4096 (topk>256) already used CSR and was unaffected.
Validated vs an fp32 eager reference (and the triton_v2 / gluon_v2 bf16 anchors):
after the fixes, flash+pro x cr{0,4,128} fwd (o, lse) and bwd (dq, dkv, d_sink)
all pass at S in {256, 512, 1024, 4096}; the standalone accuracy suite goes from
"SOME FAILED" to "ALL PASS". All three fixes are speed-neutral (bug 2 slightly
faster at the rerouted small-seq shape); the S=4096 benchmark is unchanged.
Co-authored-by: Cursor <cursoragent@cursor.com>
kyle-256
added a commit
that referenced
this pull request
Jul 14, 2026
Third cr=4 correctness bug (companion to the fwd NaN + bwd dQ race fixed in 10966dd). The cr=128 (HCA) closed-form pool gather assumes a deterministic causal pool (pool_cr = T/npool). The bare `topk<=256` gate also catches cr=4 (CSA, random pool) at small seq (S<=512 -> topk<=256), routing its random pool through the closed-form gather -> wrong dkv/dpool (seen: S=512 cr=4 bwd dkv 1.0 dB). Guard is_cr128 with the deterministic-pool condition the forward uses (T % npool == 0 and T // npool >= 64) so cr=4 (pool_cr=4) falls through to the CSR gather. Production S=4096 (topk>256) was already CSR -> unchanged. Verified on MI355X (chi2798) vs fp32 eager: S=512 cr=4 bwd dkv 1.0 -> 50.8 dB; seq=512 and seq=4096 all 6 shapes ALL PASS (real cr=128 still closed-form). Matches #417 (dev/wenx/dsv4-flydsl-cr4-correctness-fixes).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.