linalg: AVX (no FMA) f32 kernels for pre-Haswell x86 - #2598
Open
czoli1976 wants to merge 2 commits into
Open
Conversation
family) fell back to the generic f32 kernels: the plug tree is gated on avx2 and every f32 mmm kernel needs fma. Add an avx mul+add tier (16x6, 8x8 and 64x1 mmm kernels with a gather-free add_unicast), fix the avx2-only reg-source vbroadcastss encoding in the otherwise avx-clean mul_by_scalar/max/min kernels, and plug the tier whenever the fma one can't load. 1.9-2.6x f32 matmul over generic on an i7-3770. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
Author
|
@kali been doing last PRs on a Linux Intel® Core™ i7-3770 and noticed this gap .... will not move your bench :-) 16x5/24x4/32x3/40x2 and AVX sigmoid/tanh to follow |
matmuls badly tiled and sigmoid/tanh on the generic fallback. Add mul+add ports of the remaining f32 tile geometries (16x5, 24x4, 32x3, 40x2 -- the 12-accumulator tiles have no spare register, so the column broadcast doubles as mul scratch, re-broadcast per row) and of the fma sigmoid/tanh kernels, with the same shape-fitted n dispatch as the fma tier. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CPUs with AVX but without AVX2+FMA (Sandy Bridge / Ivy Bridge, AMD Bulldozer family) currently run the generic f32 kernels, since the x86_64 plug tree is gated on avx2 and all f32 mmm kernels need fma. This adds an AVX mul+add kernel tier covering the full fma-tier f32 tile set (16x6 / 16x5 / 24x4 / 32x3 / 40x2 / 8x8 / 64x1, with gather-free add_unicast and the same shape-fitted n dispatch), mul+add ports of the sigmoid/tanh kernels, and the already-AVX-clean mul_by_scalar/max/min element-wise kernels. Measured on an i7-3770: 1.9-2.6x f32 matmul (up to 3.5x at narrow N) and ~1.8x sigmoid/tanh over the generic kernels.
🤖 Generated with Claude Code
🍍