feat(eth): EVM block-gas for EIP-1559 fee projection (takeover of #1585)#1586
Conversation
EthereumTypeGetEip1559Fees called setLatestBlockGas on every request,
which issued an extra synchronous eth_getBlockByNumber("latest") RPC to
populate Eip1559Fees.BlockGasUsed/BlockGasLimit. This had three problems:
- Extra overhead: a third blocking RPC was added to every websocket
estimateFee call (on top of eth_maxPriorityFeePerGas + eth_feeHistory),
and it also fired on the alternative-fee-provider fast path that was
deliberately RPC-free (served from an in-memory cache). On fast L2s with
many subscribers polling fees this scales linearly with load.
- Data race: the alternative provider returns the same cached *Eip1559Fees
pointer to every caller; setLatestBlockGas then mutated BlockGasUsed/
BlockGasLimit on that shared struct with no lock, racing concurrent
estimateFee goroutines (and the field reads in server/websocket.go).
- Incoherent data: baseFeePerGas came from eth_feeHistory (newest=pending)
or an off-chain oracle, while gasUsed/gasLimit came from a separate
"latest" block, so the three values described different blocks and could
not be combined into a correct EIP-1559 next-base-fee projection.
Remove getLatestBlockGas/setLatestBlockGas and the now-unpopulated
BlockGasUsed/BlockGasLimit fields from the pull-side Eip1559Fees (bchain,
api, blockbook-api.ts) and the websocket mapping. The push path
(subscribeNewBlock evm_data) is unaffected: it still carries the block's
baseFeePerGas/gasUsed/gasLimit from a single header, with no extra RPC.
A follow-up commit exposes the next-block base fee from the eth_feeHistory
response already fetched, so the pull path keeps a (consistent, free)
projection input.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The new-block notification payload used a snake_case json tag (evm_data), the only such tag among the websocket/JSON wire types, which otherwise use lowerCamelCase (height, hash, baseFeePerGas, feePerUnit, ...). Rename it to evmData for consistency before the field ships, since changing a wire-level key after release would be a breaking change. Updates the Go json tag, the JSON shape test, blockbook-api.ts and a code comment. The field is documented in openapi.yaml in the next commit. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The subscribeNewBlock notification grew an evmData field (WsNewBlock / EthereumGasData) that was undocumented in openapi.yaml. Because these types are not in the parity allow-list they did not break CI, but the new websocket payload was unspecified. Add EthereumGasData and WsNewBlock schemas (evmData modeled as nullable via the 3.1 oneOf/"null" pattern, matching the Go *EthereumGasData json tag without omitempty), and add both to the parity check so the generated blockbook-api.ts and openapi.yaml stay in sync. Parity typecheck passes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The push path (subscribeNewBlock -> evmData) is the consistent, RPC-free delivery mechanism for the EIP-1559 projection inputs, but only its JSON wire shape was tested, not the block -> payload mapping. Extract that mapping out of onNewBlockAsync into a pure newBlockNotification helper (behavior unchanged) and unit-test it: non-EVM blocks and EVM pre-London blocks leave evmData nil, while EVM post-London blocks carry baseFeePerGas/gasUsed/gasLimit taken from the connected block header. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
WsNewBlock's height, hash and evmData have no omitempty in Go, so they are always present on the wire (evmData serializes as null for non-EVM/pre-London blocks). Mark them required in the schema so the spec reflects the actual always-present (evmData nullable) shape. Parity typecheck still passes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…e fee Records the conclusion of the fee-history investigation inline: eth_feeHistory's extra projected element has backend-dependent semantics (dropped by no-pending nodes like Erigon so baseFeePerGas is already the next-block fee; kept but index-shifted to N+1 or a noisy N+2 by some L2s), so a single "next block" field cannot be defined consistently. Clients needing an exact next-block base fee use the subscribeNewBlock evmData push instead. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The on-chain estimate (used when no alternative fee provider is configured — e.g. ethereum non-archive and the ethereum testnets) set each tier's maxFeePerGas to the priority fee alone, omitting the base fee entirely. That value is below the block's base fee and therefore not mineable. Trezor Suite consumes maxFeePerGas directly (connect EthereumFeeLevels: it only clamps to a floor, it does not add the base fee), so the on-chain path produced unusable EIP-1559 fees. Compute each tier correctly from the eth_feeHistory response: - maxPriorityFeePerGas = the tier's reward percentile (20/70/90/99), i.e. the actual tip distribution, instead of the node's single suggestion doubled. - maxFeePerGas = eip1559BaseFeeMultiplier*baseFee + tip, with the multiplier (2) documented as the EIP-1559-standard buffer (survives ~6 full blocks of +12.5%/block base-fee growth). Tunable via the named constant. This also drops the now-redundant eth_maxPriorityFeePerGas call (Suite floors the tip with coinInfo.minPriorityFee), so the on-chain path makes one RPC (eth_feeHistory) instead of two. Adds a unit test asserting the per-tier values and the maxFeePerGas > baseFee invariant. Note: this does not touch the alternative-fee (Infura) path, where the "fees too high" reports originate — that is a fee-policy decision pending coordination with the Suite team. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
9aadb76 to
461f9f0
Compare
Document, with mermaid diagrams, how EIP-1559 fees are produced across Blockbook and Trezor Suite and which component owns which decision: blockbook provides ground-truth inputs (chain base fee, priority-fee tiers, congestion/trend, block gas), while the wallet owns the fee policy (base-fee source, 2x head-room buffer, maxFeePerGas composition, per-coin clamps). Covers the pull path (EthereumTypeGetEip1559Fees), the push path (subscribeNewBlock evmData), Suite's EthereumFeeLevels, the end-to-end flow, and the maxFeePerGas formula. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
461f9f0 to
9c93cf2
Compare
Move the two existing fee panels (estimated fee rate, alternative fee provider requests) out of General into a new top-level "Fees" section, so fee observability has one home as EVM EIP-1559 fee metrics are added. Pure relocation: panels, queries and the metrics they read are unchanged; only their x-panel-keys move general.* -> fees.* and a row.fees section is introduced. grafana.json is the git-ignored artifact rendered from these two sources by contrib/scripts/render_grafana.py. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
EVM EIP-1559 fees were invisible in metrics (blockbook_estimated_fee is
UTXO-only). Add two gauges populated on the successful estimateFee /
EthereumTypeGetEip1559Fees paths (provider cache hit and on-chain estimate):
- blockbook_eth_eip1559_fee{tier,kind}: per-tier maxFeePerGas / priority fee
- blockbook_eth_eip1559_base_fee: the next-block base fee underlying them
Values are raw wei (base units, like estimated_fee); Grafana divides by 1e9
to show Gwei. Emitted only on the two successful returns so error/disabled
paths never write zeros, and nil tiers are skipped so no empty series appear.
Dashboard (Fees section): maxFeePerGas by tier, priority tip by tier (own
axis - tips are ~1000x smaller), and the maxFee/baseFee buffer ratio with a
dashed 2x reference, which separates a congested chain from an estimator
regression - the "fees too high" diagnostic.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add blockbook_eth_eip1559_fee_source_total{source} incremented on each served
estimate at the serve boundary: provider (alternative provider cache hit),
onchain_fallback (provider configured but cache stale/unready, so eth_feeHistory
was used) or onchain (no provider configured).
This is the one signal neither alternative_fee_provider_requests (background
fetch outcome) nor the provider cache-age gauge captures: how often the wallet-
facing estimate actually bypassed the provider. Emitted only on the two
successful returns, alongside the fee gauges.
Dashboard: a non-stacked "EIP-1559 fee source" panel so a provider->fallback
transition shows as a visible crossover.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add blockbook_eth..._provider_last_sync_timestamp_seconds{provider}, the unix
time of the last successful refresh of the cached EVM provider (infura/1inch)
fees, set via a shared observeSync() called from both providers' processData
(replacing the bare p.lastSync = time.Now()) so lastSync and the metric always
share one instant.
Cache age is plotted as time() - metric. A timestamp gauge, not a computed
*_age_seconds one (the repo's usual form): it is written only on a successful
refresh, so the plotted age keeps climbing when a provider wedges and survives
restarts. This is the leading indicator that explains the onchain_fallback
counted by eth_eip1559_fee_source_total - once age crosses the stale window
(30x the poll period) the pull path falls back to on-chain.
Dashboard: a cache-age panel with a dashed default-cutoff (30m) reference.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add blockbook_eth_block_gas_used_ratio, gasUsed/gasLimit of the most recently connected EVM block (0..1) - the congestion signal that drives the next base fee (>0.5 it rises up to +12.5%/block, <0.5 it falls). This is the push path's own view of why fees move, independent of any provider. Set synchronously in OnNewBlock before the async broadcast: OnNewBlock is invoked in monotonic height order by the single writeBlockWorker, whereas the per-block broadcast goroutines can reorder and let an older block clobber a newer gauge value. Last-value semantics, nil-guarded for non-EVM/pre-London. Dashboard: a "Latest block gas-used ratio" panel (percentunit) with a dashed 50% balance-point reference. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…verlay Add blockbook_eth_block_base_fee, the realized base fee of the most recently connected block (push path), in raw wei. Set in the same OnNewBlock helper as the gas-used ratio, nil-guarded. Paired with eth_eip1559_base_fee (the pull path's next-block projection) on one overlay panel - solid mined vs dashed projected, both in Gwei. The two should track within a block's +/-12.5% step; a persistent gap means the feeHistory projection (or a provider's estimatedBaseFee) is drifting from the chain - the base-fee half of a "fees too high" investigation. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Documentation-only follow-up from an adversarial review of the new Fees section (no functional change, no Go change): - order the section pull-estimate -> provider-health -> push -> legacy, and label the relocated UTXO "Estimated fee rate" so an EVM $coin selection reads as expected-empty rather than an outage - buffer-ratio: note Infura's high tier sits ~2.5x by design, so the dashed 2x line is the on-chain estimator's target only - base-fee overlay & cache-age: scope the "tracks within +/-12.5%" and "30x poll period" expectations to the path/provider they actually hold for (provider coins serve a cached projection up to the stale window old) - eth_eip1559_base_fee help: drop the unconditional "next-block projection" claim (true only on Erigon-like backends) and Infura's estimatedBaseFee field name (1inch feeds the same gauge from baseFee) - priority-tip: "separate panel", not "own axis" Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Review notesWent through the diff carefully. The PR is in good shape — most of the things I initially suspected turned out to be non-issues (see bottom). Two points worth a look, one actionable. 1. On-chain priority fee now comes solely from
|
Bounds-check the per-tier column index so a non-compliant backend returning a reward row shorter than the requested percentile count is skipped instead of panicking EthereumTypeGetEip1559Fees. Divide the tip average by the rows that actually contributed, so a skipped short row does not deflate the tier. Document the accepted zero-tip case on idle chains, and add a regression test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Fixed in 17c494d — bounds-guarded the reward column index (short rows skipped, divisor counts only contributing rows) and documented the zero-tip-on-idle-chains accept. Added a regression test. |
The 1inch alternative fee provider never set staleSyncDuration, so it defaulted to the zero value (0s), making the cache immediately stale after every write. GetEip1559Fees always fell back to on-chain estimation via eth_feeHistory, defeating the purpose of having a provider cache. Mirror the same pattern used by the Infura provider: introduce oneInchFeeStalePeriods=30 and oneInchFeeStaleDuration, and set p.staleSyncDuration in the constructor so cached fees are reused within the 30×pollPeriod window (e.g. 30min for a 60s poll).
Replace the Infura gas API with 1inch Gas Price API for Ethereum mainnet archive. The 1inch API returns raw wei integers and uses Bearer token auth (ONE_INCH_API_KEY). The staleSyncDuration bug that made 1inch cache always immediately stale was fixed in the preceding commit.
Grafana: - alt_fee_provider_requests tooltip mode: single -> multi (inconsistent with other fee panels) - eip1559_max_fee description: generalize Infura-only language to mention both providers and clarify instant tier behavior differs per provider - eip1559_buffer_ratio description: generalize Infura-only padding reference - alt_fee_provider_cache_age description: generalize "Infura (the EVM default)" to mention both providers use the same 30x stale window docs/fees.md: - Replace Infura-only references with provider-agnostic language throughout - Keep accurate historical context (Infura as original source of fees-too-high reports) - Update mermaid flowcharts from "Infura fees" to "provider fees" - Document both Infura and 1inch behavior in the provider section
1inch returns 4 tiers (low/medium/high/instant) with tighter per-tier values than Infura (3 tiers). The previous 1:1 mapping (suite low/medium/high ← 1inch low/medium/high) produced fees that were too conservative compared to Infura, causing ~2x fee differences between providers. Remap so the suite's low/medium/high tiers are fed from comparable aggressiveness: 1inch medium → suite low 1inch high → suite medium 1inch instant → suite high 1inch's low tier is discarded (too conservative to be useful). The fee formula (maxFeePerGas = 2×baseFee + tip) is already correct on the suite side (fix/evm-fee branch) — this change ensures the input tier values are comparable regardless of which provider serves the response.
This builds on and takes over #1585 (@53gur0's
feat/evm-1559-fields). The four original commits are preserved as-is; everything else here is review fixes and the decisions described below.Built on the existing PR
A takeover of #1585. The original push-path work (block-level gas data on the
subscribeNewBlocknotification, plus its tests) is kept unchanged. The added commits address issues found in review, correct the on-chain fee tiers, and document the result.What it delivers
EVM block-level gas data so wallets can project EIP-1559 base fees, via two paths:
subscribeNewBlocknow carriesevmData { baseFeePerGas, blockGasUsed, blockGasLimit }taken from the just-connected block header. All three values come from one block, with no extra RPC; the wallet can apply the consensus base-fee formula to get the next block deterministically.estimateFeekeeps returningeip1559.baseFeePerGas, which (viaeth_feeHistory) is already the projected base fee of the next block, and now returns corrected on-chain fee tiers (see below).Companion Trezor Suite change
The user-facing "fees too high" fix lives in the wallet, on the companion branch:
https://github.com/trezor/trezor-suite/tree/fix/evm-fee
We went with Option B — the wallet owns the fee-buffer policy:
maxFeePerGas = 2 × baseFee + tipitself, from the chain's base fee, instead of passing a provider's pre-paddedsuggestedMaxFeePerGasstraight through. Infura pads the high tier to ~2.5× the base fee and Suite usedmaxFeePerGasverbatim — that was the "too high".EthereumFeeLevelsthe block-based estimate is preferred and falls back to the backend tiers when block data is unavailable, so nothing regresses.getBlock('latest'); blockbook-backed networks fall back to the backend (Infura) tiers until thesubscribeNewBlockevmDatafrom this PR is wired in as their block-data source.docs/fees.md(added here) documents the full cross-repo flow.Why
setLatestBlockGaswas removedThe original approach added a
setLatestBlockGasstep toEthereumTypeGetEip1559Fees, issuing an extraeth_getBlockByNumber("latest")to populateBlockGasUsed/BlockGasLimit. Two real problems and one minor one:*Eip1559Feespointer to every caller;setLatestBlockGasthen mutatedBlockGasUsed/BlockGasLimiton it without a lock, racing concurrentestimateFeegoroutines (reproduced with-race).baseFeePerGas(frometh_feeHistory, newest =pending) andgasUsed/gasLimit(from a separatelatestblock) described different blocks, so the trio could not form a correct single-block EIP-1559 projection.eth_getBlockByNumber("latest", false)perestimateFee. That call is cheap on its own (header only, usually served from the node's memory), so it is not a significant cost. The fair point is architectural rather than raw overhead: it also fired on the alternative-fee-provider path, which is deliberately served from an in-memory cache with no node RPC, so it quietly undid that optimization.The deciding factor is that the data was not needed in the pull path at all:
baseFeePerGasis already the next block's projected fee, and the pushevmDatacarries per-block gas consistently. Fix: dropsetLatestBlockGas— nothing is lost.Corrected the on-chain fee tiers
On the on-chain estimate (coins with no alternative provider —
ethereumnon-archive and the ETH testnets), each tier'smaxFeePerGaswas set to the priority fee alone, omitting the base fee — i.e. below the block's base fee, so not mineable. Since Suite usesmaxFeePerGasdirectly, these coins produced unusable EIP-1559 fees.Now each tier is built from the
eth_feeHistoryresponse:maxPriorityFeePerGas= the tier's reward percentile (the real tip), andmaxFeePerGas = eip1559BaseFeeMultiplier(2) × baseFee + tip— the EIP-1559-standard 2× buffer (survives ~6 full blocks of +12.5%/block growth), in a documented, tunable constant. This also drops the now-redundanteth_maxPriorityFeePerGascall, so the on-chain path makes one RPC (eth_feeHistory) instead of two. The Infura path is untouched.Decision: no separate "next-block base fee" field in the pull path
We considered exposing the next-block base fee from the
eth_feeHistoryresponse already fetched (no extra RPC). Investigation (go-ethereumeth/gasprice/feehistory.go+ live probes against the deployed backends) showed the extra projected elementeth_feeHistoryreturns has backend-dependent semantics:baseFeePerGasis already the next block's projected fee.baseFeePerGasbecomes the latest mined block.No single field can describe all of these consistently, so we deliberately did not add one. The reasoning is captured inline in
ethrpc.go. Clients needing an exact next-block base fee should use the pushevmData(the real previous-block header). The existing pull-sidebaseFeePerGasis left as-is; its own backend-dependent meaning is a pre-existing property, out of scope here.Could it be richer?
The one genuinely-sound enrichment would be a server-computed next-block base fee on the push side — safe there because we have the exact header (deterministic), unlike the pull path:
But I'd lean against it:
Fee observability (Grafana + metrics)
The dashboard had only two fee panels (UTXO
estimated_feeandalternative_fee_provider_requests), both buried in General, and EVM EIP-1559 fees were invisible in metrics entirely. This adds a dedicated Fees section and the metrics behind it, aimed at troubleshooting the "fees too high" reports and at making both the pull and push paths observable.Six new metrics (declared in
configs/metrics.yaml, the single source of truth; the dashboard is generated fromtemplate.json+panels.yamlbyrender_grafana.py):eth_eip1559_fee{tier,kind}(the per-tiermaxFeePerGas/ priority fee actually served) andeth_eip1559_base_fee(the base fee they are built on), in raw wei; pluseth_eip1559_fee_source_total{source}=provider/onchain_fallback/onchain, i.e. how often the served estimate bypassed the provider.alternative_fee_provider_last_sync_timestamp_seconds{provider}: cache age viatime() - metric(a timestamp, not a computed age, so it keeps rising when a provider wedges). The leading indicator that explains a fallback.eth_block_gas_used_ratio(latest connected block's congestion, the driver of base-fee moves) andeth_block_base_fee(realized base fee), the latter overlaid against the pull-path projection.All values are emitted in base units (wei / ratio), nil-guarded, and only on the relevant success paths; nil tiers create no empty series. The push gauges are set synchronously in
OnNewBlock(invoked in monotonic height order) rather than in the per-block broadcast goroutines, so an older block can't clobber a newer value.The Fees section orders the panels pull → provider-health → push → legacy and includes a buffer-ratio panel (
maxFeePerGas / baseFee, with a dashed 2× reference) that separates a congested chain from estimator/provider over-buffering — the direct read on "fees too high". Every panel carries a self-contained description.The design and the final diff were each run through an adversarial multi-lens review; the only changes it produced were wording-precision tweaks (already applied). It also surfaced one pre-existing, out-of-scope issue: the 1inch provider never sets
staleSyncDuration, so its cache is treated as stale immediately — its series is dormant in current deployments, and the newfee_source/ cache-age panels would now make it visible.Other fixes
evm_data→evmData(it was the only snake_case key among the websocket types) before it ships, since changing a wire key after release is breaking.subscribeNewBlockpayload (WsNewBlock/EthereumGasData, withevmDatamodeled as nullable in 3.1) and added them to the parity check. The parity typecheck (a CI gate) had been failing because the newEip1559Feesfields were not mirrored inopenapi.yaml; it passes again now.evmDatamapping (non-EVM, pre-London, post-London) and for the corrected on-chain tier computation (assertingmaxFeePerGas > baseFee).docs/fees.mdwith diagrams of the full cross-repo fee flow (pull path, push path, Suite policy, end-to-end).Testing
common/eth/server, including new coverage for every fee-metric emit helper);go vetclean (the only vet warnings are pre-existing inserver/public_test.go, unrelated).ethereum=main/rpc(incl.EstimateFee— which now exercises the corrected on-chain tiers — andGetBlock) andbsc=main/rpc/EstimateFeepass.lint:spec,generate, and paritytypecheckall pass.render_grafana.py --checkconsistent (102 panels, 85 metrics);go.mod/go.sumuntouched.grafana.jsonis the git-ignored rendered artifact.Deployment / compatibility
This change is backward-compatible and can be deployed independently of, and before, the Suite release:
subscribeNewBlockgainsevmDataadditively ({height, hash}→{height, hash, evmData}); existing clients ignore the new field, and the deployed Suite reads onlyheight/hashfrom the notification.estimateFee/Eip1559Feesresponse shape is unchanged vsmaster.The only behavioral change for current clients is on the no-provider on-chain coins, where
maxFeePerGasgoes from unmineable to correct (a bugfix). The user-facing "too high" improvement lands only once the Suite branch ships; deploying blockbook first is safe and makesevmDataavailable for it.1inch provider tier remapping
1inch returns 4 fee tiers (low/medium/high/instant) with systematically tighter values per tier than Infura (3 tiers). The previous 1:1 mapping (suite low/medium/high ← 1inch low/medium/high) produced fees that were too conservative compared to Infura, causing ~2× fee differences between providers that made reliable cross-provider fee estimation impossible.
Fix: remap the tiers in the 1inch provider so the suite's low/medium/high are fed from comparable aggressiveness regardless of which provider serves the response:
lowlowmediummediummediumhighhighhighinstant1inch's
lowtier is discarded (too conservative to be useful). Theinstantfield (not consumed by the suite) is left nil, consistent with Infura's behavior. The on-chain fallback path is unaffected — its percentile-based tiers (p20/p70/p90/p99) already produce correct per-tier values. The fee formula (maxFeePerGas = 2×baseFee + tip) is already handled on the suite side; this change ensures the input tier values are comparable.Verified live against dev blockbooks via WebSocket:
1inch response (commit
66d7affdeployed — tier remapping active):lowmediumnormalhighhighinstantInfura response for comparison:
lowlownormalmediumhighhighAbsolute values differ (different chains/networks), but the remapping ensures each suite tier receives a meaningful priority fee — without it, the suite's
lowwould have been populated from 1inch'slow(~100K wei priority fee, too conservative to be reliably mineable).