[Do not merge until Git dependencies removed] Uncompressed Encoding for Affine Points

[Hanting Zhang (winston-h-zhang)]

This PR implements a feature (uncompressed_serde) that optimizes the speed of serializing public parameters by using an uncompressed encoding that removes the overhead of computing the second coordinate of points. Note: this PR depends on PRs in pasta_curves and neptune, here: zcash/pasta_curves#73 and lurk-lab/neptune#192. For now, we point at branches on lurk-lab maintained forks.

I compared the performance on the minroot public parameters. Based on some eyeballing benchmarks run on my local machine, deserialization time reduces from 2.7s to 100ms. Downstream in lurk-rs, where the circuit is much bigger -- and serde has become a significant time factor in initializing proving times -- deserialization time on the sha256 example reduces from 120s to 7s. Note, the uncompressed encoding for pasta points are [u8; 64], so this does have the downside of eating twice as much memory compared to the normal/compressed encoding.

The significant improvement in deserialization time makes caching public parameters viable. This would short circuit the need to regenerate costly public parameters each run, improving the develop/iterate experience. This infrastructure currently exists in lurk-rs, and this PR mainly wants to improve its efficiency. Although hopefully, this clarifies a growing need for systematic approach to deal with public parameters in Nova, and leaves future work open to discussion.

As for the feature itself, it is implemented via the UncompressedEncoding trait in group. We only apply the uncompressed serde to CommitmentKey, since the structure stores 90% of the data.

[François Garillot (huitseeker)]

[on MinRoot] deserialization time reduces from 2.7s to 100ms. Downstream in lurk-rs, where the circuit is much bigger [...] deserialization time on the sha256 benchmark reduces from 120s to 7s.

Nice work!

[Hanting Zhang (winston-h-zhang)]

@microsoft-github-policy-service agree company="lurk-lab"

[Hanting Zhang (winston-h-zhang)]

@microsoft-github-policy-service agree company="Lurk Lab"

[CPerezz (CPerezz)]

This will probably be superset by #198

Also, if we plan to move to halo2curves we already provide this functionallity in our SerdeObject trait. See: https://github.com/privacy-scaling-explorations/halo2curves/blob/main/src/serde.rs

[Srinath Setty (srinathsetty)]

Hi Hanting Zhang (@winston-h-zhang) is this PR still being worked on? I've heard interest in this from several others.

[François Garillot (huitseeker)]

This is blocked on zcash/pasta_curves#73

[Srinath Setty (srinathsetty)]

Hi CPerezz (@CPerezz)

This will probably be superset by #198

Also, if we plan to move to halo2curves we already provide this functionallity in our SerdeObject trait. See: https://github.com/privacy-scaling-explorations/halo2curves/blob/main/src/serde.rs

What is the best way to use SerdeObject trait to get fast deserialization of field elements? Are there examples of this somewhere to compose this automatically with serde's methods?

[François Garillot (huitseeker)]

Srinath Setty (@srinathsetty) some answers to these questions might reside in privacy-ethereum/halo2curves#39

[CPerezz (CPerezz)]

Srinath Setty (@srinathsetty) this is a trait we have for raw/unchecked(and therefore fast) serde. It's not integrated with the regular serde.
The fastest way to serialize/deserialize in terms of CPU time is the raw_checked. As it doesn't imply any tests of correctness of well formation as well as implies that there are no extra ops to perform (no derivation of y coordinate from x or stuff like this.

[Srinath Setty (srinathsetty)]

It seems like this may be a stale PR. Please reopen this if we want to pursue it.