perf(ferret): ~5.7x faster COT extension (consistency check) - #410
Closed
heeckhau wants to merge 3 commits into
Closed
perf(ferret): ~5.7x faster COT extension (consistency check)#410heeckhau wants to merge 3 commits into
heeckhau wants to merge 3 commits into
Conversation
Collaborator
|
Is this speedup for the uniform variant? We're going to remove that, we only use regular |
- inn_prdt_no_red: 8 independent accumulators to break the clmul latency chain; inn_prdt_red gains a rayon-parallel chunked path (identical result). - prg: add random_blocks_par (parallel, byte-identical counter-mode fill), chi_inner_product (regenerates chi on the fly inside the inner product), and blocks_at (seekable single-block gather), with equivalence tests. - ggm: GgmTree takes a caller-provided scratch buffer for its internal nodes, so it can be reused across trees instead of reallocated per call.
The consistency check was the dominant cost. Its chi vector is now regenerated on the fly inside a parallel inner product (no 294 MB materialization; the receiver keeps a 16-byte seed and uses blocks_at for the alpha-indices), and the extend loops reuse a per-thread GGM scratch buffer.
Drives the real phase functions at production LPN params and reports per-phase timings; compares Uniform vs Regular. Run with `cargo test -p mpz-ot-core --release --features rayon ferret::profile -- --ignored --nocapture`.
heeckhau
force-pushed
the
perf/ferret-extension
branch
from
June 8, 2026 09:49
a596b6a to
96e76cc
Compare
Contributor
Author
|
I have removed all improvements for "uniform" and only kept those that target "regular" |
Collaborator
|
Superceded by #425 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Speeds up the Ferret COT extension by attacking its real bottleneck — the
SPCOT consistency check (~90% of per-side cost in the Regular variant). All
changes are variant-agnostic (they live in the SPCOT check + core
primitives), so they benefit both Regular and Uniform.
Regular LPN — per-side compute (profiler,
rayon)(The end-to-end
ferret_sendercriterion bench — which runs the default Uniformconfig — went 1.024 s → 193 ms, 5.3×; the optimized phase is the same one
that dominates Regular.)
The bottleneck
Profiling at production params showed the consistency check, not the LPN encode,
was the cost (the encode is ~1%). The check was a sequential, latency-bound
clmulinner product over the GGM-leaf vector plus a single-threadedchiPRGfill — neither parallelized.
Changes (variant-agnostic)
core/block:inn_prdt_no_reduses 8 independent accumulators (breaks theclmullatency chain);inn_prdt_redgains arayon-parallel chunked path.Result is identical (the reduction is GF(2)-linear over the XOR-combine).
core/prg:random_blocks_par(parallel, byte-identical counter-modefill),
chi_inner_product(regenerateschion the fly inside the innerproduct, so the large
chivector is never materialized), andblocks_at(seekable single-block gather). The SPCOT check on both parties uses these —
the receiver stores a 16-byte seed instead of the full
chivector.core/ggm:GgmTreetakes a caller-provided scratch buffer for itsinternal nodes, reused per-thread in the SPCOT extend loops.
ferret/spcot: wires the above into the consistency check and theGGM generation.
Correctness
All existing protocol tests pass in both
rayonand non-rayonconfigs(
ferret::tests::test_ferretfor Regular + Uniform, plus SPCOT/MPCOT/cuckoo).New unit tests assert the new primitives match the explicit references
(
random_blocks_par/blocks_atvs the sequential PRG stream;chi_inner_productvs
inn_prdt_red(&explicit_chi, b)). The protocol's own check(
hashed_v == hashed_w) would fail on any sender/receiver divergence.Notes
core-primitive work. Earlier revisions also optimized the Uniform-only cuckoo
path (batched AES,
u32indices, parallel combine) — dropped, since you'reremoving Uniform and those would be dead code.
rayonon or off (sequential fallbacks arebyte-identical).
ferret::profile, an#[ignore]d test) is included forreproducibility; happy to drop it.
helps even more on x86 (PCLMULQDQ latency).
Possible follow-ups (not here)
further memory win, ~1.4× more on the Uniform path; on Regular the vector is
already small so the benefit is mostly memory.
clmulfor the inner product on x86 servers.