Skip to content

WASM PR 2: move the WASM microbenches out of src and into the bench target - #2564

Open
czoli1976 wants to merge 1 commit into
sonos:mainfrom
czoli1976:refactor/wasm-benches
Open

WASM PR 2: move the WASM microbenches out of src and into the bench target#2564
czoli1976 wants to merge 1 commit into
sonos:mainfrom
czoli1976:refactor/wasm-benches

Conversation

@czoli1976

Copy link
Copy Markdown
Contributor

Stacked on #2563 — its two commits are the first two here.

The wasm backend kept three timing modules inside src/ behind #[cfg(test)] + #[ignore], which no other backend does. benches/wasm.rs already supersedes two of them, and says so itself: its header records that the in-src version looped all four GEMV kernels back-to-back and so biased whichever ran last. Those two go; the activation bench moves across to join it.

numerical_consistency_16x1_vs_32x1 stays in src, in wasm/dispatch_tests.rs — it is an assertion rather than a timer, and the only numerical cross-check between two kernels.

The four copies of the kernel-lookup helper in benches/wasm.rs collapse to one. I left the timing loops alone: they use 50 versus 200 warmup calls and only two of them do min-of-N reporting, and those differences are deliberate, so a shared helper would have quietly changed what the numbers mean.

Net: 335 lines out of src, and wasm.rs is down to 78 lines.

Nothing here ships — test and bench code only, and the kernel disassembly is unchanged. The bench runs: on an M1 Pro the relocated activation bench reports sigmoid 1.49–1.54x and tanh 1.40–1.62x over the scalar path, matching what the module documented.

🍍

… target

Three timing modules lived in src behind #[cfg(test)] + #[ignore], which no
other backend does. benches/wasm.rs already supersedes two of them — its own
header records that looping four kernels back-to-back biased the in-src
version — so those are dropped, and the activation bench moves across. The
numerical-consistency check between 16x1 and 32x1 is an assertion rather than
a timer, so it stays in src alongside the dispatch tests. The four copies of
the kernel-lookup helper in the bench collapse to one; the timing loops keep
their own warmup and repetition counts, which differ on purpose.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@kali

kali commented Aug 3, 2026

Copy link
Copy Markdown
Collaborator

⚠️⚠️⚠️ Just rebased! ⚠️⚠️⚠️

@kali
kali force-pushed the refactor/wasm-benches branch from fc934ff to 34e0454 Compare August 3, 2026 19:25
@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown

🔴 Bench vs main — 3 speed regression(s) · ⚠️ 7 secondary

Reference: 2026-08-03 morning nightly run (0d old) · full report → run

Speed — evaltime · prefill · decode

Δ metric device main → PR
🔴 +15.8% hey_snips_v4_model17
evaltime · 2sec
cortex-a9 38.7 ms → 44.8 ms
🔴 +10.7% inceptionv3
evaltime · pass_mt
cortex-a53 1.12 s → 1.24 s
🔴 +8.6% hey_snips_v4_model17
evaltime · 2sec
cortex-a7 43.6 ms → 47.3 ms

Improvements

Δ metric device main → PR
🟢 -22.0% parakeet_tdt_600m_v3_f32f32_decoder_pass
evaltime · cpu
i9-11900kb_rtx-4060 0.838 ms → 0.654 ms
🟢 -11.1% hey_snips_v4_model17
evaltime · 2sec
apple-m1-max 0.979 ms → 0.87 ms
🟢 -8.0% hey_snips_v4_model17
evaltime · 2sec
i9-11900kb_rtx-4060 0.972 ms → 0.894 ms
🟢 -6.8% inceptionv3
evaltime · pass
apple-m1-max 63.5 ms → 59.2 ms
🟢 -6.7% hey_snips_v1
evaltime · 400ms
apple-m1-max 0.13 ms → 0.121 ms
+2 more improvement(s)
Δ metric device main → PR
🟢 -6.3% hey_snips_v4_model17
evaltime · pulse8
apple-m1-max 0.117 ms → 0.11 ms
🟢 -4.6% openelm_270M_q40ef16_541
decode · cpu
apple-m1-max 12.7 ms/tok
79 tok/s → 12.1 ms/tok
82.82 tok/s
⚠️ 7 secondary regression(s)
Δ metric device main → PR
⚠️ +15.5% arm_ml_kws_cnn_m
load · pass
cortex-a9 84 ms → 97 ms
⚠️ +15.2% hey_snips_v4_model17
load+optimize · 2sec
cortex-a9 4.39 s → 5.06 s
⚠️ +14.8% hey_snips_v4_model17
load+optimize · 2sec
cortex-a7 5.07 s → 5.82 s
⚠️ +14.1% hey_snips_v4_model17
load · 2sec
cortex-a9 3.81 s → 4.34 s
⚠️ +13.4% hey_snips_v4_model17
load · 2sec
cortex-a7 4.34 s → 4.92 s
⚠️ +9.2% arm_ml_kws_cnn_m
load+optimize · pass
cortex-a9 131 ms → 143 ms
⚠️ +5.7% llama_3_2_3B_instruct_q40ef16_541
load+optimize · cuda
jetson-orin-nx 4.14 s → 4.38 s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants