Skip to content

feat: add corelib benchmarks and refactor to shared bench_all_phases helper#9859

Merged
orizi merged 1 commit intomainfrom
orizi/04-19-feat_add_corelib_benchmarks_and_refactor_to_shared_bench_all_phases_helper
Apr 23, 2026
Merged

feat: add corelib benchmarks and refactor to shared bench_all_phases helper#9859
orizi merged 1 commit intomainfrom
orizi/04-19-feat_add_corelib_benchmarks_and_refactor_to_shared_bench_all_phases_helper

Conversation

@orizi
Copy link
Copy Markdown
Collaborator

@orizi orizi commented Apr 20, 2026

TL;DR

Added a new benchmark that measures compilation performance of the corelib itself, alongside the existing fib.cairo benchmark.

What changed?

The benchmark infrastructure was refactored to extract reusable build_db and build_test_db helper functions, eliminating repeated RootDatabase builder boilerplate. The benchmark phases (cairo-to-sierra, cairo-to-diagnostics, cairo-to-testing, cairo-to-cache, cache-to-sierra, cache-to-testing) were extracted into a generic bench_all_phases function that accepts a path, group name, label, and a detect_core flag.

A new bench_compile_corelib benchmark was added that runs all the same phases against the corelib directory (with detect_core: false), and both bench_compile and bench_compile_corelib are now registered with criterion_group!.

How to test?

Run the benchmarks with:

cargo bench --bench compile

Both the compile and compile_corelib benchmark groups should appear in the output.

Why make this change?

Benchmarking only a small example file like fib.cairo does not capture the performance characteristics of compiling a large, real-world codebase. Adding corelib as a benchmark target provides a more representative and meaningful measure of compiler performance at scale.

Copy link
Copy Markdown
Collaborator Author

orizi commented Apr 20, 2026

@reviewable-StarkWare
Copy link
Copy Markdown

This change is Reviewable

@orizi orizi force-pushed the orizi/04-19-feat_add_corelib_benchmarks_and_refactor_to_shared_bench_all_phases_helper branch 2 times, most recently from 5d18c01 to aa33bf1 Compare April 20, 2026 11:52
@orizi orizi force-pushed the orizi/04-19-feat_add_cache-to-testing_compile_benchmark branch from 76b8508 to 4432551 Compare April 20, 2026 11:52
@orizi orizi force-pushed the orizi/04-19-feat_add_corelib_benchmarks_and_refactor_to_shared_bench_all_phases_helper branch from aa33bf1 to 45feb8b Compare April 20, 2026 13:55
@orizi orizi force-pushed the orizi/04-19-feat_add_cache-to-testing_compile_benchmark branch from 4432551 to 3135903 Compare April 20, 2026 13:55
@orizi orizi marked this pull request as ready for review April 20, 2026 13:57
@orizi orizi changed the base branch from orizi/04-19-feat_add_cache-to-testing_compile_benchmark to graphite-base/9859 April 20, 2026 14:05
@orizi orizi force-pushed the orizi/04-19-feat_add_corelib_benchmarks_and_refactor_to_shared_bench_all_phases_helper branch from 45feb8b to 5c26e30 Compare April 20, 2026 14:06
@orizi orizi force-pushed the graphite-base/9859 branch from 3135903 to fe48470 Compare April 20, 2026 14:06
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 45feb8b392

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread tests/benches/compile.rs Outdated
@orizi orizi changed the base branch from graphite-base/9859 to main April 20, 2026 14:06
@orizi orizi enabled auto-merge April 20, 2026 14:07
@orizi orizi force-pushed the orizi/04-19-feat_add_corelib_benchmarks_and_refactor_to_shared_bench_all_phases_helper branch from 5c26e30 to 7e37cd6 Compare April 20, 2026 14:07
@orizi orizi disabled auto-merge April 20, 2026 15:12
Copy link
Copy Markdown
Contributor

@eytan-starkware eytan-starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@eytan-starkware reviewed 2 files and all commit messages, and made 2 comments.
Reviewable status: 2 of 3 files reviewed, 2 unresolved discussions (waiting on orizi and TomerStarkware).


tests/benches/compile.rs line 52 at r2 (raw file):

    // Phase: source → Sierra (full IR generation).
    group.bench_function("cairo-to-sierra", |b| {

Dont we need separate function names for differrent benchmarks?


tests/benches/compile.rs line 189 at r2 (raw file):

}

criterion_group!(benches, bench_compile);

Wont this give

Copy link
Copy Markdown
Collaborator Author

@orizi orizi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@orizi made 2 comments.
Reviewable status: 2 of 3 files reviewed, 2 unresolved discussions (waiting on eytan-starkware and TomerStarkware).


tests/benches/compile.rs line 52 at r2 (raw file):

Previously, eytan-starkware wrote…

Dont we need separate function names for differrent benchmarks?

the groups seems to be enough.


tests/benches/compile.rs line 189 at r2 (raw file):

Previously, eytan-starkware wrote…

Wont this give

?

Copy link
Copy Markdown
Contributor

@eytan-starkware eytan-starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@eytan-starkware made 1 comment and resolved 1 discussion.
Reviewable status: 2 of 3 files reviewed, 1 unresolved discussion (waiting on orizi and TomerStarkware).


tests/benches/compile.rs line 52 at r2 (raw file):

Previously, orizi wrote…

the groups seems to be enough.

I am thinking it will be better to see for each phase how each benchmark behaved, currently it will be hard to compare a slowdown in phase x, as we will need to compare 3 different tables

@orizi orizi force-pushed the orizi/04-19-feat_add_corelib_benchmarks_and_refactor_to_shared_bench_all_phases_helper branch from 7e37cd6 to 9d7fce7 Compare April 20, 2026 21:01
@orizi orizi force-pushed the orizi/04-19-feat_add_corelib_benchmarks_and_refactor_to_shared_bench_all_phases_helper branch from 9d7fce7 to 1157d46 Compare April 21, 2026 07:45
Copy link
Copy Markdown
Collaborator Author

@orizi orizi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@orizi made 1 comment.
Reviewable status: 0 of 3 files reviewed, 1 unresolved discussion (waiting on eytan-starkware and TomerStarkware).


tests/benches/compile.rs line 52 at r2 (raw file):

Previously, eytan-starkware wrote…

I am thinking it will be better to see for each phase how each benchmark behaved, currently it will be hard to compare a slowdown in phase x, as we will need to compare 3 different tables

i flipped it now.

@orizi orizi enabled auto-merge April 21, 2026 08:39
Copy link
Copy Markdown
Contributor

@eytan-starkware eytan-starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

@eytan-starkware reviewed 3 files and all commit messages, made 1 comment, and resolved 1 discussion.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on TomerStarkware).

@orizi orizi added this pull request to the merge queue Apr 23, 2026
Merged via the queue into main with commit fd6e910 Apr 23, 2026
53 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants