rms_norm: fuse a residual add into RmsNorm

[czoli1976]

Draft — and after further measurement I recommend closing this rather than merging it. Leaving it up for the record and for the harness; the analysis below is the useful part.

Ported from vLLM #48757 ("Fuse Transformers Residual Add + RMSNorm"). AddRmsNorm evaluates a residual add and an RmsNorm over the trailing axis together, exposing the sum as a second output because a transformer reads it again in the next block; detect_add_rms_norm on RmsNorm's declutter collapses the pair and rewires both consumer sets. It is correct — fuses_and_keeps_both_values compares output bytes against the undecluttered graph for f32 and f16 — and it is a wash:

shape	unfused	fused
f16 512x4096	1.9441 ms	1.9526 ms
f32 512x4096	1.1886 ms	1.2118 ms

Why, in two layers

The implementation does not actually fuse anything at the memory level. It adds whole-tensor, clones, then normalises whole-tensor: read a, read b, write sum, read sum, write normed — exactly the unfused sequence, just inside one node. The read-back it was supposed to remove is still there. That is a flaw in this PR, not in the idea.

But the idea does not pay here either. Row-blocking it properly would remove one of eight tensor passes, 12.5% of traffic. That only converts if the pair is bandwidth-bound, and it is not:

	traffic	achieved	share of ~200 GB/s
f32 512x4096	67 MB in 1.189 ms	56 GB/s	28%
f16 512x4096	34 MB in 1.944 ms	17 GB/s	8.5%

At 8-28% of peak the bottleneck is compute and dependency latency, not traffic. For f16 in particular the cost is the whole-tensor widening inside RmsNorm, which #2580 removes — a much larger effect than anything this fusion could reach.

And the reason it cannot be made to pay the way vLLM's does: their kernel updates the residual in place in a preallocated activation buffer, so the sum genuinely costs nothing. tract's graph has no in-place contract, so the sum must be materialised, and the traffic is what it is.

If someone wants to revisit it, the only interesting target is a machine where this pair is bandwidth-bound — much lower bandwidth per unit of compute than an M1 Pro. I have not found one to test on, and I would not merge this on speculation.

Two traps worth recording

The first implementation used a hand-written scalar *s = *s + *r for the add and was 20% slower than unfused, because tract's Add lowers to a vectorised OptBinUnicast; it needs tract_linalg::bin_unicast(dt, BinOp::Add). Separately, a criterion baseline compared across processes reported this change as -14% f16 / -6% f32; interleaving the arms showed the wash above. Cross-run criterion baselines are not reliable on this machine.

Tests

tract-core 270 green. cargo fmt --all clean; cargo clippy clean (the two remaining warnings are pre-existing on main).

🍍