Skip to content

feat: jagged pcs#31

Open
kunxian-xia wants to merge 4 commits intomainfrom
feat/jagged_commit_adaptor
Open

feat: jagged pcs#31
kunxian-xia wants to merge 4 commits intomainfrom
feat/jagged_commit_adaptor

Conversation

@kunxian-xia
Copy link
Copy Markdown
Collaborator

@kunxian-xia kunxian-xia commented Apr 7, 2026

Summary

This PR includes works in several separate PRs.

PR Note
compute q(b) from the input row major matrices
#32 run the jagged sumcheck $v = \sum_b q(b) * f(b)$ in a memory constrained device (note that both q and f have around 30 ~ 32 variables)
#39 evaluator for indicator function $g(z_1, z_2, z_3, z_4)$
verifier for $f(z) = \sum_c \textrm{eq}(z_c, c) * g(z_r, z, t_c, t_{c+1})$
jagged assist sumcheck
stacked pcs (reshape q(b) as several smaller multilinear polynomials)

@hero78119
Copy link
Copy Markdown
Collaborator

@copilot please review this PR

Copilot AI review requested due to automatic review settings April 14, 2026 07:53
@kunxian-xia kunxian-xia review requested due to automatic review settings April 14, 2026 07:53
Copilot AI requested a review from hero78119 April 14, 2026 07:54
@hero78119 hero78119 force-pushed the feat/jagged_commit_adaptor branch from 0f7f15e to 80d5537 Compare April 14, 2026 08:12
@hero78119
Copy link
Copy Markdown
Collaborator

@copilot review this PR

Copy link
Copy Markdown

Copilot AI commented Apr 14, 2026

@copilot review this PR

Reviewed and fixed in commit b02d8c4:

  1. Removed unused importsuse std::iter::once and IndexedParallelIterator were unused (the former only served the confusing scan trick). Replaced with FieldAlgebra which is needed for the safe initialization below.

  2. Replaced unsafe { set_len } – changed to vec![E::BaseField::ZERO; total_size]. The parallel transpose loop unconditionally overwrites every slot, so this is zero-cost correctness: no behavioral change, no unsafe block.

  3. Removed unreachable poly_heights.is_empty() guard – after the preceding validation (rmms non-empty, each matrix has non-zero columns), poly_heights is always non-empty at that point.

  4. Simplified cumulative heights computation – the chain(once(&0)).scan(...) trick (appending a dummy 0 to coerce scan into emitting one extra value) is fragile and non-obvious. Replaced with a straightforward indexed loop:

    let mut cumulative_heights = vec![0usize; poly_heights.len() + 1];
    for (i, &h) in poly_heights.iter().enumerate() {
        cumulative_heights[i + 1] = cumulative_heights[i] + h;
    }
  5. Added test_bitrev_transpose_layout – verifies the actual data layout after bit-reversal and column transposition against a known 4×2 input (col0=[1,5,3,7], col1=[2,6,4,8]). All 6 tests pass.

@hero78119
Copy link
Copy Markdown
Collaborator

@copilot rollback to 80d5537, then review with ".github/copilot-instructions.md" in base branch. just review, no edit/push. Author will fix it

@hero78119
Copy link
Copy Markdown
Collaborator

@copilot review this PR using .github/copilot-instructions.md in another branch feat/ci_ai_review

* claude code impl plan

* implement the jagged_sumcheck using time-space tradeoff sumcheck prover algorithm

* remove debugging codes

* ref to the original paper

* add jagged sumcheck bench

* #32 parallelize (#34)

* par wip

* check f(z) * g(z) matches expected evaluation

* fix clippy

* fix clippy: add #[cfg(test)] to test-only method and fix unused import/variable warnings

Agent-Logs-Url: https://github.com/scroll-tech/gkr-backend/sessions/4a61316e-cf28-47b8-a43e-fb6ab432701e

Co-authored-by: hero78119 <3962077+hero78119@users.noreply.github.com>

* refactor test

* apply functional programming style

* avoid unnecessary BaseField-to-ExtensionField conversion in q_evals access

Use E * BaseField multiplication directly instead of converting q_evals
elements to extension field with .into() first.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* replace col_row binary search with incremental ColRowIter

Add ColRowIter that does one binary search at construction and O(1)
per step, replacing per-element binary searches in build_m_table,
bind_and_materialize, compute_claimed_sum, and final_evaluations_slow.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: hero78119 <3962077+hero78119@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* extend jagged sumcheck benchmark to cover n=25..31

* switch jagged sumcheck benchmark to BabyBearExt4

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* remove jagged sumcheck plan doc

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: hero78119 <3962077+hero78119@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
@scroll-tech scroll-tech deleted a comment from Copilot AI Apr 14, 2026
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an initial Jagged PCS adaptor to mpcs, including a commit-time packing of many column polynomials into a single “giga” MLE and a streaming jagged sumcheck prover intended for memory-constrained environments (per PR description and linked #32 context).

Changes:

  • Expose a new mpcs::jagged module and re-export jagged commitment/sumcheck APIs from mpcs::lib.
  • Implement jagged_commit (bit-reverse rows + transpose + concatenate) and jagged_sumcheck_prove (epoch-based M-table streaming + fallback to standard sumcheck).
  • Add a Criterion benchmark for jagged sumcheck and introduce tracing-forest as a dev-dependency.

Reviewed changes

Copilot reviewed 4 out of 5 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
crates/mpcs/src/lib.rs Exports the new jagged module and its public API types/functions.
crates/mpcs/src/jagged.rs Implements jagged commit packing and streaming jagged sumcheck prover, plus unit tests.
crates/mpcs/benches/jagged_sumcheck.rs Adds a Criterion benchmark for the jagged sumcheck prover across n=25..31.
crates/mpcs/Cargo.toml Registers the new benchmark and adds tracing-forest dev-dependency.
Cargo.lock Locks tracing-forest and related transitive dependencies.

Comment thread crates/mpcs/src/jagged.rs
Comment thread crates/mpcs/src/jagged.rs
Comment on lines +41 to +42
//! The cumulative heights allow the verifier to succinctly evaluate the indicator function
//! `g(z_r, z_b, t[i-1], t[i])` needed for the jagged sumcheck.
Copy link

Copilot AI Apr 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The module docs describe using an indicator function g(z_r, z_b, t[i-1], t[i]) to handle jagged segment boundaries, but the current implementation of f(b) throughout the prover is eq_row[row] * eq_col[col] (no dependence on t beyond (col,row) mapping). Either update the docs to match the implemented (uniform-height) assumption, or implement the g(...) weighting so variable-height segments are handled as described.

Suggested change
//! The cumulative heights allow the verifier to succinctly evaluate the indicator function
//! `g(z_r, z_b, t[i-1], t[i])` needed for the jagged sumcheck.
//! In this adaptor, `t` is used for segment layout and for recovering the `(column, row)`
//! coordinates of a packed evaluation. The prover logic then uses the standard product-form
//! equality weights induced by those coordinates (effectively `eq_col[column] * eq_row[row]`).
//! This implementation does **not** currently implement the more general boundary-weighting
//! indicator `g(z_r, z_b, t[i-1], t[i])` described in the full jagged-sumcheck construction;
//! accordingly, the documentation here is restricted to the uniform-height packing behavior
//! implemented by this module.

Copilot uses AI. Check for mistakes.
Comment thread crates/mpcs/src/jagged.rs
Comment on lines +523 to +527
|(q_acc, f_acc), ((&eq_r_a, &q), (col, row))| {
(
q_acc + eq_r_a * q,
f_acc + eq_r_a * (input.eq_row[row] * input.eq_col[col]),
)
Copy link

Copilot AI Apr 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

row/col from col_row_iter are used to index input.eq_row[row] and input.eq_col[col] without any validation. If eq_row.len() is smaller than the max polynomial height or eq_col.len() is smaller than num_polys, this will panic. Add an upfront check in jagged_sumcheck_prove (or a constructor for JaggedSumcheckInput) that eq_row.len() >= max_height and eq_col.len() >= num_polys, or change the API to return a Result with a clear error.

Copilot uses AI. Check for mistakes.
Comment thread crates/mpcs/src/jagged.rs
Comment on lines +657 to +662
|(q_acc, f_acc), ((&eq_r_a, &q), (col, row))| {
(
q_acc + eq_r_a * q,
f_acc + eq_r_a * (input.eq_row[row] * input.eq_col[col]),
)
},
Copy link

Copilot AI Apr 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same issue as in build_m_table: input.eq_row[row] and input.eq_col[col] are indexed without checking that the precomputed tables are large enough for the (col,row) pairs implied by cumulative_heights. Without an upfront validation, this can panic for malformed inputs or when polynomials have larger heights than the provided eq_row table.

Copilot uses AI. Check for mistakes.
Comment thread crates/mpcs/src/jagged.rs
Comment on lines +891 to +907
fn test_jagged_sumcheck_n25() {
// n=25: 2^25 = 33M evaluations. Exercises all epochs + 10 rounds of standard sumcheck.
use ff_ext::FromUniformBytes;

tracing_forest::init();

let mut rng = thread_rng();

let num_polys = 1 << 10; // 1024 polynomials
let poly_height = 1 << 15; // 32768 each, s=15
let s = 15;
let total_evals = num_polys * poly_height; // 2^25 = 33554432
let num_giga_vars = 25;

let q_evals: Vec<F> = (0..total_evals)
.map(|i| F::from_canonical_u64((i as u64 * 13 + 7) % (1 << 30)))
.collect();
Copy link

Copilot AI Apr 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test_jagged_sumcheck_n25 allocates and processes 2^25 (= 33,554,432) field elements (and additionally builds f_evals in final_evaluations_slow), which is likely to exceed typical CI time/memory budgets and make cargo test flaky/slow. Consider marking this test #[ignore], gating it behind a feature/env var, or moving it to a Criterion bench (keeping unit tests at small sizes that still cover the same code paths).

Copilot uses AI. Check for mistakes.
Comment thread crates/mpcs/src/jagged.rs
Comment on lines +895 to +896
tracing_forest::init();

Copy link

Copilot AI Apr 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Calling tracing_forest::init() inside a unit test performs global tracing subscriber initialization; this can conflict with other tests that also initialize tracing and can fail/panic depending on the tracing-forest behavior. Prefer a fallible/once-only init (e.g., via a Once guard or a try_init-style API) or remove tracing initialization from the test.

Suggested change
tracing_forest::init();

Copilot uses AI. Check for mistakes.
…rove (#38)

* claude code impl plan

* implement the jagged_sumcheck using time-space tradeoff sumcheck prover algorithm

* remove debugging codes

* ref to the original paper

* add jagged sumcheck bench

* #32 parallelize (#34)

* par wip

* check f(z) * g(z) matches expected evaluation

* fix clippy

* fix clippy: add #[cfg(test)] to test-only method and fix unused import/variable warnings

Agent-Logs-Url: https://github.com/scroll-tech/gkr-backend/sessions/4a61316e-cf28-47b8-a43e-fb6ab432701e

Co-authored-by: hero78119 <3962077+hero78119@users.noreply.github.com>

* refactor test

* apply functional programming style

* avoid unnecessary BaseField-to-ExtensionField conversion in q_evals access

Use E * BaseField multiplication directly instead of converting q_evals
elements to extension field with .into() first.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* replace col_row binary search with incremental ColRowIter

Add ColRowIter that does one binary search at construction and O(1)
per step, replacing per-element binary searches in build_m_table,
bind_and_materialize, compute_claimed_sum, and final_evaluations_slow.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: hero78119 <3962077+hero78119@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* extend jagged sumcheck benchmark to cover n=25..31

* switch jagged sumcheck benchmark to BabyBearExt4

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* remove jagged sumcheck plan doc

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Initial plan

* Make EPOCH_SIZES configurable in jagged_sumcheck_prove via optional parameter

Agent-Logs-Url: https://github.com/scroll-tech/gkr-backend/sessions/b18805c7-6e15-44c0-ab03-a1905068d964

Co-authored-by: hero78119 <3962077+hero78119@users.noreply.github.com>

* Fix doc spacing and add debug_assert for epoch_sizes validation

Agent-Logs-Url: https://github.com/scroll-tech/gkr-backend/sessions/b18805c7-6e15-44c0-ab03-a1905068d964

Co-authored-by: hero78119 <3962077+hero78119@users.noreply.github.com>

---------

Co-authored-by: kunxian xia <xiakunxian130@gmail.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: hero78119 <3962077+hero78119@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants