Skip to content

[Feature] Return proving/verifying keys from execution methods#1227

Closed
marshacb wants to merge 1 commit into
mainnetfrom
feat/key-caching-execution
Closed

[Feature] Return proving/verifying keys from execution methods#1227
marshacb wants to merge 1 commit into
mainnetfrom
feat/key-caching-execution

Conversation

@marshacb

@marshacb marshacb commented Mar 9, 2026

Copy link
Copy Markdown
Contributor

Motivation

SnarkVM assumes proving/verifying keys persist in long-running processes, but SDK users typically run ephemeral processes where synthesized keys are discarded after each execution. Since key synthesis is the most expensive part of execution (30s+ for complex functions), this meant users were paying that cost repeatedly with no way to retrieve or reuse keys.

This PR adds cacheKeys: true to ExecuteOptions, enabling:

  • Key extraction: buildExecutionTransaction() and run() return the synthesized proving/verifying keys alongside execution results
  • Automatic persistence: Keys are auto-persisted to the configured KeyStore (e.g., LocalFileKeyStore from [Feature] Add KeyStore interface for persistent Proving and Verifying Key storage #1207)
  • Transparent reuse: Subsequent executions automatically check memory cache → KeyStore → network/synthesis, so keys are reused with zero user intervention

See docs/design/key-caching.md for the full design spec.

What changed

  • SDK (program-manager.ts): Added cacheKeys option to ExecuteOptions, ExecutionTransactionResult return type with overloaded buildExecutionTransaction(), memory + KeyStore caching in both buildExecutionTransaction() and run()
  • SDK (keys/provider/memory.ts): AleoKeyProvider now accepts an optional KeyStore via constructor, functionKeys() falls back to KeyStore when memory cache misses
  • WASM (execute.rs, response.rs): New ExecutionTransactionResponse type and buildExecutionTransactionWithKeys method that extracts proving/verifying keys from the process before it's dropped

Follow-up

  • run() positional parameter fragility — should migrate to an options object (existing tech debt)

Test plan

  • 7 unit tests in key-provider.test.ts: KeyStore constructor wiring, setKeyStore(), memory → KeyStore fallback chain, memory cache priority, error paths — all passing
  • 1 integration test in program-manager.test.ts: run() with cacheKeys: true end-to-end — synthesizes keys via WASM, persists to LocalFileKeyStore, verifies keys on disk
  • 1 skipped integration test: buildExecutionTransaction with cacheKeys: true — requires a funded account and network access, to be enabled when test infrastructure supports it
  • Full regression: 41/41 key-provider tests passing, 0 new failures in program-manager suite
How to run
# Full build (WASM must be built first for Rust changes)
yarn build:all

# Run tests
cd sdk && yarn test

@iamalwaysuncomfortable iamalwaysuncomfortable left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some changes requested, and likely some unification needed with PR #1208. We can use the ProgramImports abstraction you built there to collect the imports and then store them in typescript.

Comment on lines +366 to +377
let cached_keys = if cache {
let function_name = IdentifierNative::from_str(function).map_err(|err| err.to_string())?;
let pk = process
.get_proving_key(program_native.id(), &function_name)
.map_err(|_| format!("Could not find proving key for {}/{}", program_native.id(), function))?;
let vk = process
.get_verifying_key(program_native.id(), &function_name)
.map_err(|_| format!("Could not find verifying key for {}/{}", program_native.id(), function))?;
(Some(pk), Some(vk))
} else {
(None, None)
};

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple of notes here:

  1. In the case of imports, sometimes those are synthesized and added to the process, we want to extract those as well.
  2. Perhaps instead of a new transaction type, we might want to pass an object into an existing method that collects the keys like the one you came up with in which can then be extracted later?

Comment on lines +97 to +98
setKeyStore(keyStore: KeyStore | undefined) {
this._keyStore = keyStore;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's this implementation of adding the KeyStore to the key provider waiting in a PR: #1221 - let's see if we can merge the approaches tomorrow.


// Build an execution transaction
if (options.cacheKeys) {
const response: ExecutionTransactionResponse = await WasmProgramManager.buildExecutionTransactionWithKeys(

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this outer scope, one thing we might consider instead of a different wasm method is passing in the wasm object you defined and at the end, get the pks/vks out of it and then destroy/free it.

Comment on lines +117 to +118
provingKey: ProvingKey;
verifyingKey: VerifyingKey;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since wasm is limited to 4gb total memory space, we probably want is to take these out of wasm and destroy them so as not to OOM the process. We also want to return the checksums so that people have a future way of verifying the integrity of the keys.

Suggested change
provingKey: ProvingKey;
verifyingKey: VerifyingKey;
provingKey: Uint8Array;
provingKeyChecksum: string;
verifyingKey: Uint8Array;
verifyingKeyChecksum: string;

async buildExecutionTransaction(
options: ExecuteOptions,
): Promise<Transaction> {
): Promise<Transaction | ExecutionTransactionResult> {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I originally though this signature might be good, but I think it might break a lot of downstream typescript implementors who will have to account for both types in further type lints.

I think instead we want to keep the original signature the same and allow callers to pass in an imports object which gets mutated and callers can extract the imports from there. OR if they have the keyprovider configured, they can extract it from there as well.

programName: "my_program.aleo",
functionName: "my_function",
inputs: ["1u32", "2u32"],
cacheKeys: true,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We might consider changing this to a KeyCache which gets populated with anything not currently existing.

Comment on lines +154 to +159
#[wasm_bindgen]
pub struct ExecutionTransactionResponse {
transaction: TransactionNative,
proving_key: Option<ProvingKeyNative>,
verifying_key: Option<VerifyingKeyNative>,
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think here we want to go in favor of a typescript object instead that gets populated outside the wasm scope. But what we probably want is the ProgramImports object you created in PR #1208 and use that to collect the keys. Then when the execution terminates, we extract the keys out of it into either the KeyProvider or a separate typescript object.

@marshacb

Copy link
Copy Markdown
Contributor Author

covered in #1208

@marshacb marshacb closed this Mar 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants