Skip to content

perf(gnovm): parallelize test suites and add gno test -jobs#5811

Draft
thehowl wants to merge 2 commits into
masterfrom
dev/morgan/gnovm-test-parallel
Draft

perf(gnovm): parallelize test suites and add gno test -jobs#5811
thehowl wants to merge 2 commits into
masterfrom
dev/morgan/gnovm-test-parallel

Conversation

@thehowl

@thehowl thehowl commented Jun 11, 2026

Copy link
Copy Markdown
Member

Note

Part 1 of 6 of the gnovm performance stack (split from #5800), to be merged in order:

  1. perf(gnovm): parallelize test suites and add gno test -jobs #5811 — perf(gnovm): parallelize test suites and add gno test -jobs
  2. perf(gnovm): avoid heap-boxed byte access in copy, range and index reads #5812 — perf(gnovm): avoid heap-boxed byte access in copy, range and index reads
  3. perf(gnovm): recycle runtime blocks through a per-machine pool #5813 — perf(gnovm): recycle runtime blocks through a per-machine pool
  4. perf(gnovm): share interface-held values when copying arrays #5814 — perf(gnovm): share interface-held values when copying arrays (gas-visible)
  5. ci(gnovm): skip print-only coverage instrumentation #5815 — ci(gnovm): skip print-only coverage instrumentation
  6. perf(gnovm): reduce per-call and per-op allocations #5816 — perf(gnovm): reduce per-call and per-op allocations

Each PR is based on the previous one's branch; this one diffs against master. Together: ci / gnovm ~14m → 6m18s; pkg/gnolang test time −64%; VM heap allocations −84% on the heaviest suite.

Summary

ci / gnovm took a stable ~14 minutes, with pkg/gnolang alone at ~708–722s of main / test and the stdlibs / test job fully sequential at 8m33s. This PR restructures the test suites for parallelism — no test content changes — and gives gno test package-level parallelism.

  • TestFiles: the 2339 non-long filetests ran sequentially on one goroutine; they now run as parallel subtests drawing a TestOptions (store) from a GOMAXPROCS-sized pool, so stores and their loaded stdlib packages are still reused, just split N ways. -update-golden-tests keeps the fully sequential single-store behavior (deterministic walk-order writes).
  • TestStdlibs: the sequential shared-store walk ran in the parent test body, which also delayed the already-parallel heavy suites (Go only starts parallel subtests after the parent body returns — math/overflow alone blocked them for ~60s). Every stdlib package now runs as a parallel subtest with its own store, removing the hardcoded special-case lists.
  • gnoBuiltinsCache was a package-level map lazily populated from TypeCheckMemPackage — a latent data race that becomes load-bearing with concurrent type-checks; it is now built eagerly at init and read-only afterwards.
  • gno test -jobs N (default 1 = behavior unchanged): tests up to N packages in parallel, each worker owning a store/typecheck cache reused across the packages it runs; per-package output is buffered and printed in package order as results complete (out/err kept on their original streams). -failfast stops scheduling new packages; in-flight ones finish. Incompatible with the interactive -debug mode. The CI gno test step (gnovm stdlibs and examples jobs) passes -jobs 4 to match the 4-vCPU runners.

Measurements

before after
CI stdlibs / test job 8m33s ~5m30s
CI examples gno-checks / test job 3m44s ~3m10s
local gno test gnovm/stdlibs/..., 4 pinned cores 455s 270s (-jobs 4)
local go test ./pkg/gnolang/ long mode, 16 cores 283.7s 245.0s

main / test itself barely moves from parallelization alone (on a 4-vCPU runner it is bound by total CPU work — go test -p already packed the cores); the rest of the stack attacks the work itself. A measured caveat that shaped the design: N concurrent VMs in one process slow each other down well beyond core-sharing (GC assist/memory pressure; per-package wall inflates 2–3x under -jobs 4), which is why -jobs defaults to 1 and why parallelism only pays where the baseline wasted cores.

Verification: filetest suite pass/fail set unchanged, gno test output equivalence checked for -jobs 1 vs -jobs 4 (same result lines, same order), examples (220 packages) and cmd/gno suites pass, txtar suite passes.

thehowl added 2 commits June 11, 2026 18:31
TestFiles ran its 2339 non-long filetests sequentially on a single
goroutine; they now run as parallel subtests, drawing a TestOptions
(with its store) from a GOMAXPROCS-sized pool so loaded packages are
still reused across tests. -update-golden-tests keeps the fully
sequential single-store behavior.

TestStdlibs ran most stdlib suites sequentially on a shared store in
the parent test body, which also delayed the parallel heavy suites
(bytes, strconv, ...) until the walk finished, since parallel subtests
only start once the parent body returns. Every stdlib package now runs
as a parallel subtest with its own store.

gnoBuiltinsCache was lazily populated from TypeCheckMemPackage without
synchronization; with type-checks now running concurrently from the
start, that latent race becomes load-bearing. The cache is now built
eagerly at package init and read-only afterwards.
gno test runs packages sequentially on a single-threaded VM; on the
gnovm stdlibs CI job this is ~455s of test time on one core of a
4-vCPU runner (bytes 174s, strconv 86s, math/overflow 60s, ...).

-jobs N (default 1: behavior unchanged) tests up to N packages in
parallel. Each worker owns a TestOptions/store, reused across the
packages it runs; per-package output is buffered and printed in
package order as results complete, so runs remain readable and
deterministic. Incompatible with the interactive -debug mode.

The CI gno-test step (gnovm stdlibs and examples jobs) now passes
-jobs 4 to match the runner's 4 vCPUs.
@Gno2D2

Gno2D2 commented Jun 11, 2026

Copy link
Copy Markdown
Collaborator

🛠 PR Checks Summary

All Automated Checks passed. ✅

Manual Checks (for Reviewers):
  • IGNORE the bot requirements for this PR (force green CI check)
Read More

🤖 This bot helps streamline PR reviews by verifying automated checks and providing guidance for contributors and reviewers.

✅ Automated Checks (for Contributors):

No automated checks match this pull request.

☑️ Contributor Actions:
  1. Fix any issues flagged by automated checks.
  2. Follow the Contributor Checklist to ensure your PR is ready for review.
    • Add new tests, or document why they are unnecessary.
    • Provide clear examples/screenshots, if necessary.
    • Update documentation, if required.
    • Ensure no breaking changes, or include BREAKING CHANGE notes.
    • Link related issues/PRs, where applicable.
☑️ Reviewer Actions:
  1. Complete manual checks for the PR, including the guidelines and additional checks if applicable.
📚 Resources:
Debug
Manual Checks
**IGNORE** the bot requirements for this PR (force green CI check)

If

🟢 Condition met
└── 🟢 On every pull request

Can be checked by

  • Any user with comment edit permission

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

🚀 ci 📦 🤖 gnovm Issues or PRs gnovm related

Projects

Development

Successfully merging this pull request may close these issues.

2 participants