Skip to content

bench: add deflate_many_small_files bench for #633 - #964

Open
dhimasardinata wants to merge 3 commits into
zip-rs:masterfrom
dhimasardinata:perf/bench-633
Open

bench: add deflate_many_small_files bench for #633#964
dhimasardinata wants to merge 3 commits into
zip-rs:masterfrom
dhimasardinata:perf/bench-633

Conversation

@dhimasardinata

Copy link
Copy Markdown

Related to #633, #723

Reproduces allocation blowup reported in #633:

Workload Total Allocated Peak
500x1KB DEFLATE 207 MB 827 KB
500x1KB TAR 326 KB 152 KB

91.7% from zlib_rs::deflate::init via Compress::new (~380KB per entry) + 7.9% flate2::zio::Writer.

This bench compares deflate vs stored for 10/100/500x1KB files to provide baseline for fixing #633 by reusing Compress::reset instead of re-allocating per ZipWriter::start_file.

Run: cargo bench --bench deflate_alloc

Follow-up fix will reuse compressor (see #633 suggested Compress::reset). This bench gives tracking required by #723.

Reproduces allocation blowup from zip-rs#633: 500x1KB DEFLATE 207MB vs TAR 326KB.
Bench compares deflate vs stored for 10/100/500 files to track fix for
reusing Compress::reset instead of re-allocating 380KB per entry.

Run: cargo bench --bench deflate_alloc
Related: zip-rs#633, zip-rs#723
Copilot AI lite review requested due to automatic review settings August 31, 2026 14:22

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new micro-benchmark intended to quantify and track heap allocation overhead when writing ZIP archives containing many small files, specifically comparing DEFLATE vs STORED to establish a baseline for addressing #633 and enabling regression tracking for #723.

Changes:

  • Register a new deflate_alloc benchmark target in Cargo.toml.
  • Add a new benchmark (benches/deflate_alloc.rs) that writes 10/100/500 1KB files using ZipWriter with DEFLATE and STORED compression.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
Cargo.toml Registers the new deflate_alloc benchmark target.
benches/deflate_alloc.rs Implements the benchmark comparing DEFLATE vs STORED for many small files.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread benches/deflate_alloc.rs
Comment on lines +15 to +17
let mut zip = ZipWriter::new(buf);
let opts = SimpleFileOptions::default().compression_method(CompressionMethod::Deflated);
for i in 0..n {
Comment thread benches/deflate_alloc.rs
@@ -0,0 +1,41 @@
use criterion::{criterion_group, criterion_main, Criterion, BenchmarkId, Throughput};
…terion

- Use CompressionMethod::DEFLATE constant (always available) instead of
  CompressionMethod::Deflated (cfg _deflate-any) to compile with --no-default-features
- Add criterion 0.5 to dev-dependencies (bench uses criterion, not bencher)
  per review: cargo bench --bench deflate_alloc failed without criterion
Copilot AI review requested due to automatic review settings September 1, 2026 00:38

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Suppressed comments (2)

Previously missed (1) — in code that hasn't changed since the last review.

benches/deflate_alloc.rs:22

  • format!(...) creates a String, and ZipWriter::start_file immediately calls to_string() internally, which clones/allocates again. That introduces avoidable per-entry allocations in the benchmark and can mask the effect you’re trying to measure. Also, without a black_box on the result, the compiler may be able to optimize away parts of the work.

This issue also appears on line 29 of the same file.

                for i in 0..n {
                    zip.start_file(format!("file_{:04}.txt", i), opts).unwrap();
                    zip.write_all(&data).unwrap();
                }
                zip.finish().unwrap()
            })

benches/deflate_alloc.rs:34

  • Same benchmarking concern as the deflate case: format!(...) + start_file’s internal to_string() causes an extra allocation per entry, and returning the finished buffer without black_box risks dead-code elimination affecting timings.
                for i in 0..n {
                    zip.start_file(format!("file_{:04}.txt", i), opts).unwrap();
                    zip.write_all(&data).unwrap();
                }
                zip.finish().unwrap()
            })

Copilot AI review requested due to automatic review settings September 2, 2026 17:33

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review is ineligible. To be eligible to request a review, you need a paid Copilot license, or your organization must enable Copilot code review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants