Skip to content

feat: add proof_getProofWithMeta JSON-RPC method#11498

Open
AnkushinDaniil wants to merge 5 commits intomasterfrom
feature/eth-get-proof-meta
Open

feat: add proof_getProofWithMeta JSON-RPC method#11498
AnkushinDaniil wants to merge 5 commits intomasterfrom
feature/eth-get-proof-meta

Conversation

@AnkushinDaniil
Copy link
Copy Markdown
Contributor

Fixes NethermindEth/state-benchmarks#5

Changes

  • New JSON-RPC method proof_getProofWithMeta(account, storageKeys, blockParameter) on the existing proof_* namespace, returning AccountProofWithMeta { Proof, Meta { NodeLookups, CacheHits, MaxDepth } }. Mirrors eth_getProof and adds per-call diagnostics: nodeLookups, cacheHits, maxDepth (in nibbles).
  • New Nethermind.Trie.ProofDiagnostics carrier and MeteredTrieNodeResolver (a thin ITrieNodeResolver decorator) — observes FindCachedOrUnknown / LoadRlp and uses TreePath.Length for depth.
  • New additive PatriciaTree.AcceptMetered overload; new IStateReader.RunTreeVisitorMetered and IBlockchainBridge.RunTreeVisitorMetered.
  • eth_getProof, AccountProofCollector, CachedTrieStore, the existing Accept / RunTreeVisitor overloads, and ITrieStore DI registration are unchanged. Diagnostics are reachable only when proof_getProofWithMeta is invoked.

Types of changes

What types of changes does your code introduce?

  • Bugfix (a non-breaking change that fixes an issue)
  • New feature (a non-breaking change that adds functionality)
  • Breaking change (a change that causes existing functionality not to work as expected)
  • Optimization
  • Refactoring
  • Documentation update
  • Build-related changes
  • Other: Description

Testing

Requires testing

  • Yes
  • No

If yes, did you write tests?

  • Yes
  • No

Notes on testing

7 new unit tests added: ProofRpcModuleMetaTests (4) and MeteredTrieNodeResolverTests (3). Full Nethermind.Trie.Test (436), Nethermind.State.Test (777), Nethermind.Facade.Test (102), and existing ProofRpcModuleTests (88) all pass with zero failures. Full Nethermind.slnx Release build is clean (0 warnings, 0 errors).

Documentation

Requires documentation update

  • Yes
  • No

Requires explanation in Release Notes

  • Yes
  • No

proof_getProofWithMeta — new JSON-RPC method that mirrors eth_getProof and returns the same payload plus per-call diagnostics (nodeLookups, cacheHits, maxDepth). Useful as a client-agnostic proxy for measuring the work an EL does to serve a proof. eth_getProof and other existing endpoints are byte-identical to upstream.

Returns the standard EIP-1186 AccountProof plus per-call diagnostics
(nodeLookups, cacheHits, maxDepth) for the same query. Provides a
client-agnostic proxy for measuring how much work the EL does to serve
a proof, useful for state-bloating benchmarks.

The implementation is strictly additive: eth_getProof, AccountProofCollector,
CachedTrieStore, the existing PatriciaTree.Accept and RunTreeVisitor
overloads remain byte-identical. Diagnostics are reachable only when
proof_getProofWithMeta is invoked, via a dedicated MeteredTrieNodeResolver
decorator and a new AcceptMetered overload that wraps the resolver
internally for the duration of one RPC call.
@claude
Copy link
Copy Markdown
Contributor

claude Bot commented May 5, 2026

Claude finished @AnkushinDaniil's task in 5m 27s —— View job


PR Review: proof_getProofWithMeta JSON-RPC method

  • Load repo rules and context
  • Read changed files
  • Analyze correctness, edge cases, security, performance
  • Post findings

Summary

The design is sound — a thin ITrieNodeResolver decorator (MeteredTrieNodeResolver) wraps the existing resolver, counts lookups and cache misses, and feeds them into a new ProofDiagnostics carrier. The new AcceptMetered overload, IStateReader.RunTreeVisitorMetered, IBlockchainBridge.RunTreeVisitorMetered, and the RPC method itself are all additive and leave the existing eth_getProof/Accept path byte-identical. Tests are meaningful and cover the key behaviours. The 1000-key guard is a good safeguard against abuse.

Three Medium and two Low findings need attention before merge.


Medium findings

1. AcceptMetered duplicates ~100 lines from Accept (DRY violation)
PatriciaTree.AcceptMetered is a near-verbatim copy of Accept — the only meaningful differences are injecting MeteredTrieNodeResolver after the resolver is built and dropping the RootHash == rootHash ? RootRef fast-path. Any future change to Accept (flag logic, batched-path, storageAddr decoding) must be manually mirrored to AcceptMetered. The ideal fix is to factor out the resolver-building into a shared helper and have both methods delegate to a single traversal body. See inline comment on PatriciaTree.cs:1239.

2. ProofDiagnostics counters are not thread-safe
NodeLookups++, CacheMisses++, and MaxDepth = level are plain (non-atomic) field mutations. AcceptMetered transparently forwards visitingOptions.MaxDegreeOfParallelism, so if it is ever called with DOP > 1 (the BatchedTrieVisitor full-scan path being the obvious case), the metered resolver will be invoked from multiple threads and the counters will race. The proof path today uses VisitingOptions.Default (DOP = 1) so this doesn't manifest, but the API makes no such guarantee. Fix: use Interlocked.Increment for the long counters and a CAS loop for MaxDepth. See inline comment on ProofDiagnostics.cs:12.

3. Missing XML documentation on all new public APIs
Coding style requires <summary>/<param>/<returns> for every public API. None of the following have it: ProofDiagnostics, AccountProofWithMeta, ProofMeta, IStateReader.RunTreeVisitorMetered. See inline comment on ProofDiagnostics.cs:7.


Low findings

4. CacheHits defensive clamp signals a fragile counting invariant
NodeLookups - CacheMisses < 0 ? 0 : NodeLookups - CacheMisses — the clamp exists because LoadRlp/TryLoadRlp can theoretically be called without a preceding FindCachedOrUnknown. For the proof path the invariant holds, but it should be documented. Simplify to Math.Max(0, NodeLookups - CacheMisses) and add a <remarks> or Debug.Assert. See inline comment on MeteredTrieNodeResolver.cs:26.

5. HashSet<UInt256> as RPC parameter silently deduplicates storage keys
eth_getProof takes bytes32[] (array, duplicates preserved). Using HashSet<UInt256> means a client sending ["0x1","0x1"] gets one proof entry without any error, which deviates from standard semantics. Consider accepting an array/enumerable and deduplicating internally, or document the deduplication. The method also lacks [JsonRpcParameter] attributes and an ExampleResponse. See inline comment on IProofRpcModule.cs:38.


Positive notes

  • The decorator pattern for MeteredTrieNodeResolver is clean and non-invasive.
  • GetStorageTrieNodeResolver correctly propagates the same diagnostics object so storage trie lookups are also counted.
  • The 1000-key limit on proof_getProofWithMeta is a good safeguard with a clear error code.
  • HasStateForBlock guard before RunTreeVisitorMetered prevents the InvalidOperationException in FlatStateReader from ever surfacing to the caller.
  • Seven new unit tests are well-scoped and semantically meaningful.

Comment thread src/Nethermind/Nethermind.Trie/PatriciaTree.cs
Comment thread src/Nethermind/Nethermind.Trie/ProofDiagnostics.cs Outdated
Comment thread src/Nethermind/Nethermind.Trie/MeteredTrieNodeResolver.cs
Comment thread src/Nethermind/Nethermind.Trie/ProofDiagnostics.cs
Comment thread src/Nethermind/Nethermind.JsonRpc/Modules/Proof/IProofRpcModule.cs Outdated
- Refactor PatriciaTree.AcceptMetered to delegate to a private AcceptCore
  shared with Accept; eliminates the ~100 lines of duplication.
- Make ProofDiagnostics counters thread-safe (Interlocked.Increment for
  long counters, CAS loop for MaxDepth) so the same instance can be
  shared across threads when MaxDegreeOfParallelism > 1.
- Add XML doc comments on ProofDiagnostics, ProofMeta, AccountProofWithMeta,
  and IStateReader.RunTreeVisitorMetered.
- Simplify ProofDiagnostics.CacheHits clamp to Math.Max with a remarks
  block explaining the invariant.
- Add JsonRpcParameter attributes and ExampleResponse on
  proof_getProofWithMeta; document that storageKeys are deduplicated
  matching eth_getProof's existing HashSet<UInt256> shape.
- Drop unused usings flagged by IDE0005 (Nethermind.Core in trie test,
  Nethermind.Evm in jsonrpc test) so Check code lint passes.
@AnkushinDaniil
Copy link
Copy Markdown
Contributor Author

@claude re-review

@claude
Copy link
Copy Markdown
Contributor

claude Bot commented May 5, 2026

Claude finished @AnkushinDaniil's task in 4m 20s —— View job


PR Re-review: proof_getProofWithMeta JSON-RPC method

  • Load repo rules and context
  • Read changed files in current state
  • Check if previous review findings were addressed
  • Analyze for new issues introduced by refactor
  • Post findings

Summary

All three Medium and both Low findings from the first review have been addressed in commit 2d5ce84. The refactor is clean and correct. No new Medium or High issues were introduced.


Previous findings — status

# Severity Finding Status
1 Medium AcceptMetered duplicated ~100 lines from Accept ✅ Resolved — AcceptCore private method introduced; both Accept and AcceptMetered delegate to it
2 Medium ProofDiagnostics counters not thread-safe ✅ Resolved — Interlocked.Increment, Interlocked.Read, Volatile.Read, and CAS loop in ObserveDepth
3 Medium Missing XML docs on all new public APIs ✅ Resolved — ProofDiagnostics, AccountProofWithMeta, ProofMeta, and IStateReader.RunTreeVisitorMetered all have complete <summary> / <remarks> / <param>
4 Low CacheHits clamp code smell ✅ Addressed — Math.Max(0, ...) + <remarks> documenting the invariant
5 Low HashSet<UInt256> silently deduplicates ✅ Addressed — method description and parameter description now document the deduplication

Remaining Low finding

IBlockchainBridge.RunTreeVisitorMetered lacks XML documentation (IBlockchainBridge.cs:55). The coding style requires <summary> for all public API members. The existing RunTreeVisitor on the same interface also has no docs, so the omission is consistent with the file's current convention — this is a pre-existing pattern, not introduced by this PR. Consider adding a one-line <inheritdoc cref="IStateReader.RunTreeVisitorMetered{TCtx}"/> to both methods as a low-effort fix.


Positive notes

  • The AcceptCore refactor is tight — the only difference between Accept and AcceptMetered code paths is the null-check on diagnostics and the conditional MeteredTrieNodeResolver wrap. No logic is duplicated.
  • Thread-safety in ProofDiagnostics is now formally correct and the XML <remarks> explain the DOP > 1 scenario clearly.
  • ObserveDepth CAS loop is idiomatic and correct.
  • The <remarks> on CacheHits explaining why CacheMisses ≤ NodeLookups holds is exactly the right place for that documentation.
  • All three IStateReader implementations (StateReader, FlatStateReader, OverridableStateReader) correctly implement RunTreeVisitorMetered.

@claude
Copy link
Copy Markdown
Contributor

claude Bot commented May 5, 2026

Claude finished @AnkushinDaniil's task in 2m 40s —— View job


PR Re-review: proof_getProofWithMeta JSON-RPC method

  • Load repo rules and context
  • Read changed files in current state
  • Check if previous review findings were addressed
  • Analyze for new issues
  • Post findings

All five findings from the previous review have been fully resolved. No new issues found.

Previous findings — resolution status

# Severity Finding Status
1 Medium AcceptMetered was a near-verbatim copy of Accept (DRY violation) ✅ Fixed — both methods now delegate to a shared AcceptCore private method
2 Medium ProofDiagnostics counters were not thread-safe ✅ Fixed — Interlocked.Increment for _nodeLookups/_cacheMisses, CAS loop in ObserveDepth, Volatile.Read on MaxDepth
3 Medium Missing XML documentation on all new public APIs ✅ Fixed — ProofDiagnostics, ProofMeta, AccountProofWithMeta, and IStateReader.RunTreeVisitorMetered all have full <summary>/<param>/<returns>/<remarks> docs
4 Low CacheHits defensive clamp signaled a fragile counting invariant ✅ Fixed — uses Math.Max(0, NodeLookups - CacheMisses) with a <remarks> documenting the invariant
5 Low HashSet<UInt256> silently deduplicates; missing [JsonRpcParameter] attributes and ExampleResponse ✅ Addressed — deduplication is now explicitly documented in both the [JsonRpcMethod] description and the storageKeys parameter [JsonRpcParameter(Description = "...")]; all three params have ExampleValue; ExampleResponse is present

Positive notes on the refactor

  • The AcceptCore factoring is clean: Accept becomes a one-liner wrapper and AcceptMetered adds a null-guard then delegates — future changes to the traversal body only need to be made in one place.
  • ObserveDepth's CAS loop correctly handles the concurrent-update case; the logic is idiomatic and correct.
  • The documentation in ProofDiagnostics now explains both the thread-safety guarantee and the CacheMisses ≤ NodeLookups invariant, which makes the class self-describing.

The PR is ready to merge.

Comment thread src/Nethermind/Nethermind.Facade/IBlockchainBridge.cs Outdated
@AnkushinDaniil AnkushinDaniil changed the title feat: add proof_getProofWithMeta JSON-RPC method feat: add proof_getProofWithMeta JSON-RPC method May 5, 2026
Comment thread src/Nethermind/Nethermind.Trie/PatriciaTree.cs Outdated
Per @LukaszRozmej's review, drop the separate AcceptMetered overload and
the private AcceptCore helper. The single public Accept takes an optional
ProofDiagnostics? as its last parameter; when non-null, the resolver is
wrapped with MeteredTrieNodeResolver and per-call counters are populated
during the traversal.

Existing callers that pass only the first 2-5 parameters are unchanged
(the new param is optional with a null default). The three metered
call sites in StateReader, FlatStateReader, and FlatOverridableWorldScope
now pass `diagnostics:` as a named argument.
Comment thread src/Nethermind/Nethermind.Facade/IBlockchainBridge.cs Outdated
Per @LukaszRozmej's review, apply the same collapse pattern at the
IStateReader and IBlockchainBridge layers. Both interfaces now expose a
single RunTreeVisitor with an optional `ProofDiagnostics? diagnostics`
last parameter; when non-null, the underlying PatriciaTree.Accept wraps
the resolver with metering and populates the counters.

ProofRpcModule.proof_getProofWithMeta constructs a fresh
ProofDiagnostics, passes it via the named argument, and reads the
counters back after the visitor returns.

Removed: IStateReader.RunTreeVisitorMetered, StateReader.*,
FlatStateReader.*, FlatOverridableWorldScope.OverridableStateReader.*,
IBlockchainBridge.RunTreeVisitorMetered, BlockchainBridge.*.
/// (e.g. the <c>BatchedTrieVisitor</c> path). For the default proof-RPC code path the
/// traversal is single-threaded and the atomic ops are uncontended.
/// </remarks>
public sealed class ProofDiagnostics
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe rename to VisitingStats?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants