Skip to content

riscv64: 4 of 4: RVV element-wise and reduction kernels - #2602

Open
czoli1976 wants to merge 6 commits into
sonos:mainfrom
czoli1976:riscv64-rvv-elementwise
Open

riscv64: 4 of 4: RVV element-wise and reduction kernels#2602
czoli1976 wants to merge 6 commits into
sonos:mainfrom
czoli1976:riscv64-rvv-elementwise

Conversation

@czoli1976

Copy link
Copy Markdown
Contributor

Stacked on #2599, #2600 and #2601. GitHub cannot base a pull request on a fork
branch, so this targets main and its diff carries those PRs' commits too. The only
new commit here is 3bc03363 — the rest are reviewed in the three PRs above.

Stacked on the matmul tiers. Adds fifteen kernels: by-scalar and unicast
mul, add, sub, subf, min, max over f32, and the max, min and
sum reductions — filling the register_all_by_scalar / register_all_unicast
registries and the mul_by_scalar_f32, max_f32, min_f32, sum_f32 slots.

Two ways in which these differ from the matmul kernels

No VLEN predicate. They strip-mine on vsetvli instead of pinning vl to
a tile height, so vl is whatever the hart grants and the tail falls out of the
loop condition. That makes them vector-length agnostic outright — nothing to
gate beyond has_rvv(). nr is only the frame's chunking granularity here, not
a tile width.

No external assembler. They are Rust asm! blocks, as the corresponding
arm64 kernels are, so .option arch, +v goes through rustc's own LLVM and no
binutils version can refuse them. They therefore sit outside the tract_rvv
cfg, which records only whether an external assembler could handle the .S
matmul kernels. A toolchain too old for those still gets these.

The reductions keep the running result in element 0 of v1 and feed it back as
the reduction's scalar operand each round, so strip-mining needs no separate
accumulator vector and no final horizontal step — reduction operands are LMUL=1
whatever vtype says. vfredusum is the unordered sum, reassociating as the
arm64 kernels also do.

Testing

The frames' own proptests, with has_rvv() as the condition so they stay
meaningful on a hart without V. 2709 tract-linalg tests and tract-core's 270
pass at VLEN 128, 256 and 512, on stock RVA23, and with V absent, under qemu
10.2.1 and 11.0.3.

No CI change needed. Same caveat as the earlier PRs: correctness under
emulation, no hardware numbers.

🍍

ckristian added 6 commits August 7, 2026 09:26
tract had no RISC-V backend, so rv64gc ran the generic Rust kernels for every
matmul. Add an RVV 1.0 f32 mmm tier, detecting V from the AT_HWCAP bit that
Linux never sets for the incompatible 0.7.1 draft and VLEN from the vlenb CSR.
Because VLEN is a runtime property while MR must be a const generic, each
kernel fixes (MR, NR, LMUL) and pins vl to MR, which is correct wherever
VLMAX >= MR and short below it, so dispatch is gated on the hart's VLMAX
reaching MR and the kernel re-checks the granted vl before running. An
assembler probe keeps toolchains predating RVV 1.0 on the generic fallback.
hwbench's load_a_slice has hand-written versions for x86_64, aarch64 and arm
only, so tract-linalg fails to compile with the hwbench feature on any other
target -- which includes riscv64, and so blocks tract-cli entirely, since it
always enables that feature. Add a portable version reading one volatile word
per cache line, which moves the same traffic; with no wide loads there are
fewer misses in flight, so an out-of-order core may not saturate and the figure
reads low rather than wrong.
The RVV kernels are gated on the hart's vector length, so a single emulated
width would leave half the kernel set untested. Add riscv64gc to the qemu
cross-test platforms twice, at VLEN 256 and 128, which select disjoint halves.
-cpu max rather than a profile model because the generic rv64 model cannot run
Debian's riscv64 glibc at all.
The riscv64 tier covered f32 only, so f16 matmul fell back to the generic
kernels even on parts with native half-precision vectors, which includes the
SpacemiT X60 in the K1. Add an f16 mmm tier from the same template at SEW=16,
where VLMAX doubles and so does every tile height for a given LMUL and VLEN.
Zvfh is read from the /proc/cpuinfo isa line, since RVA23 mandates only
Zvfhmin and that cannot hold an f16 accumulator, and a second assembler probe
keeps toolchains predating Zvfh on the f32 tier alone.
The riscv64 tier had no integer kernels, so quantised models fell back to the
generic ones for every matmul. Add an i32 accumulator tier handling both
packings the frame offers: i32 x i32 at e32, and i8 x i8 through a loop at e16
where vle8.v picks EEW=8 off the instruction and vwmacc.vx widens straight into
the e32 accumulators, which keeps VLMAX identical between the two so one vl
serves both. QScale, RoundingShiftRight and ShiftLeft follow the sign-magnitude
reference in generic/rounding.rs, widening to e64 for the multiply; the
negative-value mask taken before the magnitude overwrites it also serves the
MinusInf and PlusInf nudges, which differ only where the result is zero anyway.
Every element-wise binary op and every f32 reduction still ran the generic
kernels on riscv64. Add RVV versions of by-scalar and unicast mul, add, sub,
subf, min and max, plus the max, min and sum reductions. These strip-mine on
vsetvli rather than fixing a tile, so unlike the matmul kernels they are
vector-length agnostic and need no VLEN predicate, and being Rust asm! blocks
rather than .S files they need only rustc's own assembler, so they sit outside
the tract_rvv cfg that records whether an external one could encode RVV.
@czoli1976

czoli1976 commented Aug 7, 2026

Copy link
Copy Markdown
Contributor Author

@kali these PRs all use rafitied and stable ISA, no vendor specific extensions of any kind (it is quite fragmented with competing proposals and implementations for various extensions) so these PRs add Foundational Risc-V to extend the reach of TRACT to this platform and possibily invite other contributions from the community as the ISA standardisation evolves and solidifies. from a CI point of view QEMU is plenty to verify correctness, I have used both 10.X and 11.X). the fact it runs InceptionV3 executing end-to-end is quite remarkable.

@czoli1976 czoli1976 changed the title riscv64: RVV element-wise and reduction kernels riscv64: 4 of 4: RVV element-wise and reduction kernels Aug 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant