Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
78 changes: 78 additions & 0 deletions .github/workflows/test-express-execution.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
name: Test Express Execution

# Observe-only monitor for Axelar express-execution reimbursement. For each
# requested express-supported chain it queries the Axelarscan GMP API and
# reports, per recent express transfer, the two phases:
# Phase 1 — express executed (a relayer fronted funds to the recipient)
# Phase 2 — executor reimbursed (canonical ITS.execute landed, firing
# ExpressExecutionFulfilled atomically)
#
# This is a MONITOR, not an executor: axe never originates an express transfer
# and never relays. The command reads the public GMP API only — no wallet keys,
# RPC overrides, or chains-config are needed, so this workflow defines no
# secrets and no per-chain RPC env block.

on:
workflow_dispatch:
inputs:
chains:
description: Express-supported chains to monitor (space- or comma-separated axelar ids, e.g. "avalanche linea")
type: string
required: true
network:
description: Axelar network (testnet + mainnet have express data)
type: choice
required: true
default: testnet
options:
- testnet
- mainnet

jobs:
express-monitor:
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- uses: actions/checkout@v4

# Build-or-download strategy mirrors test-amplifier-routes.yml so a code
# change automatically invalidates a stale prebuilt artifact.
- name: Download pre-built axe (${{ github.sha }})
id: download_axe
uses: dawidd6/action-download-artifact@v6
with:
workflow: build-axe.yml
name: axe-${{ github.sha }}
commit: ${{ github.sha }}
path: target/release
if_no_artifact_found: warn

- name: Mark pre-built axe executable
if: steps.download_axe.outputs.found_artifact == 'true'
shell: bash
run: chmod +x target/release/axe

- name: Set up Rust toolchain (fallback)
if: steps.download_axe.outputs.found_artifact != 'true'
uses: dtolnay/rust-toolchain@stable

- name: Cargo cache (fallback)
if: steps.download_axe.outputs.found_artifact != 'true'
uses: Swatinem/rust-cache@v2

- name: Build axe (fallback)
if: steps.download_axe.outputs.found_artifact != 'true'
shell: bash
run: cargo build --release --locked

- name: Monitor express execution
shell: bash
env:
CHAINS: ${{ inputs.chains }}
NETWORK: ${{ inputs.network }}
run: |
set -euo pipefail
# Inputs are space- or comma-separated; normalise commas to spaces and
# forward each chain as a positional arg.
read -ra CHAIN_ARGS <<< "${CHAINS//,/ }"
./target/release/axe test express-execution "${CHAIN_ARGS[@]}" --network "$NETWORK"
99 changes: 99 additions & 0 deletions AXE_STATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,55 @@ GMP is the cross-chain **delivery** primitive; ITS rides the identical
verify→route→approve→execute path and additionally needs its token registered on
each endpoint. Validating GMP validates the delivery path for both.

### `test express-execution` — express-reimbursement monitor (observe-only)

`axe test express-execution <chains…> [--source-tx <hash>] [--network …]
[--recent N] [--timeout-secs N]` monitors Axelar **express execution
reimbursement** via the Axelarscan GMP API (`/gmp/searchGMP`; testnet base for
testnet/stagenet/devnet, mainnet base for mainnet). Express = a relayer fronts
tokens to the recipient on the destination ITS edge (`expressExecute`) *before*
the canonical GMP proof lands, then is **reimbursed** when the canonical
`ITS.execute` lands (`ExpressExecutionFulfilled` fires atomically inside that
execute tx). The command reports two phases per transfer: **Phase 1** —
express executed (executor EOA / contract + express tx), and **Phase 2** —
executor reimbursed (canonical execute tx), or PENDING/timeout if the execute
hasn't landed. On reimbursement it also runs an **amount check** (MOU-27): it
decodes the executor EOA's outbound ERC-20 `Transfer`s in the express tx
(fronted) and its inbound `Transfer`s in the execute tx (reimbursed), both from
the GMP-API receipt logs, and asserts the two are equal. A mismatch or a missing
inbound transfer is surfaced as an error and **fails** the single-tx watch
(non-zero exit); the fronted/reimbursed base-unit amounts are printed either way.
Two modes: a chains scan (newest `--recent` express transfers per chain) and a
single-tx watch (`--source-tx`, polled every 10 s up to `--timeout-secs`, default
1800). It needs no wallet keys, RPCs, or chains-config — GMP-API reads only. CI:
`.github/workflows/test-express-execution.yml`.

**v1 is monitor-only.** axe does **not** yet originate a qualifying express
transfer — producing one requires routing through an express-enabled project
(e.g. the Squid router), which is an open board decision tracked as a follow-up.
v1 only observes reimbursement on transfers initiated elsewhere.

**On-chain reimbursement verified (MOU-26, mainnet, 2026-06-25).** The monitor's
API-derived "reimbursed" flag was cross-checked against the destination-chain
receipts for three real express transfers spanning the finality spectrum. In
every case the express executor EOA `0xe743…cea84` pays out token amount X in the
express tx and receives the **exact same X** back (mint → executor) inside the
canonical execute tx — full, exact-amount reimbursement, not just "execute
landed":

| source (finality) | route | express→execute gap | amount fronted = reimbursed |
| --- | --- | --- | --- |
| avalanche (~instant) | → moonbeam | 78 s | `0xbe4488` |
| base (moderate) | → avalanche | 1527 s | `0x119db3af5` |
| ethereum (~13–16 min) | → polygon | 1079 s | `0x49292` |

Two ethereum-source transfers were also caught mid-flight (`express_executed`,
execute not yet landed) — the source-finality wait the monitor reports as Phase 2
PENDING. The amount-equality invariant above is now asserted **by the tool**
(MOU-27): Phase 2 decodes the executor EOA's outbound `Transfer` in the express
tx and inbound `Transfer` in the execute tx from the GMP-API receipt logs and
fails on mismatch or missing inbound — no longer a manual check.

---

## 3. Validated on-chain — executed end-to-end
Expand Down Expand Up @@ -132,6 +181,56 @@ does not).
| hedera → monad-3 | EVM → amplifier-EVM | ITS ✅ |
| linea-sepolia → immutable | amplifier-EVM → EVM | GMP ✅ |

### 3d. Mainnet — MOU-29 15-route validation batch (2026-06-23 – 2026-06-26)

Deliberate end-to-end validation covering amplifier non-EVM ↔ non-EVM, amplifier
EVM ↔ non-EVM, legacy ↔ amplifier non-EVM, and XRPL paths. A route is ✅ only
when the destination message reached `executed` (GMP-API `executed` state +
destination execute tx confirmed by axe verifier or GMP-API `recovered_via_api`
backstop).

| # | Route | Proto | Class | Source Tx | Result |
|---|---|---|---|---|---|
| 01 | solana → sui | ITS | amplifier non-EVM ↔ non-EVM | `51TcpvU19G9UD97TPFjg8AeQkkLHcRNuKHbqa7W83W` | ✅ executed |
| 02 | sui → solana | ITS | amplifier non-EVM ↔ non-EVM | `wDMmNf5AJsRCAmqQEmoUgHKoJVPa945PameRgAmSd2` | ✅ executed |
| 03 | solana → sui | GMP | amplifier non-EVM ↔ non-EVM | `3UArKocnjH1tbhnhK3Sx8GsW95xsB89m4nVz72hMJ8` | ✅ executed |
| 04 | sui → hyperliquid | GMP | amplifier non-EVM → amplifier EVM | `9xnicQn9UccGRU8V4vgLvotpRGuienfQayoMrqtEfM` | ✅ executed |
| 05 | stellar → solana | GMP | amplifier non-EVM ↔ non-EVM | `0xd8a924923a8868879318ad9eb912da89f8fbc00e` | ✅ executed |
| 06 | xrpl → xrpl-evm | ITS | XRPL canonical XRP | `0x5037290b3a9ec3eccff49fe714803f381c44cd35` | ✅ executed |
| 07 | xrpl-evm → xrpl | ITS | XRPL canonical XRP | `0xbbab736ab1cbc9911b7e22bed8aa6fb37d1bcb55` | ✅ executed |
| 08 | hyperliquid → stellar | ITS | amplifier EVM → non-EVM | `0x1c6c898da42d3edf6a1b991dc27802e7ead44a9c` | ✅ executed (52.5 s) |
| 09 | avalanche → solana | GMP | legacy EVM → amplifier non-EVM | `0x6519283767a8504868a06a82eae632130fcf1c0e` | ✅ executed |
| 10 | avalanche → stellar | GMP | legacy EVM → amplifier non-EVM | `0x6488a701a7ebd008dcfb6d10be5ffd4b850c1cbb` | ✅ executed |
| 11 | avalanche → sui | GMP | legacy EVM → amplifier non-EVM | `0x48abdda2c0b882813c78da72d8719b7b82d127b7` | ✅ executed |
| 12 | stellar → arbitrum | GMP | amplifier non-EVM → legacy EVM | `0x9824e77f21919c58ff6d16792b95680ac4c75020` | ✅ executed |
| 13 | sui → avalanche | GMP | amplifier non-EVM → legacy EVM | `FnJCBmjKRvwAVuYR729NPPSq6usz2H86E4Ph4R3RLu` | ✅ executed |
| 14 | avalanche → base | GMP | legacy EVM ↔ legacy EVM | `0xae0d52ca1de181624df9d75c0fb5b901afa41822` | ✅ executed |
| 15 | kava → moonbeam | GMP | legacy EVM ↔ legacy EVM | `0xbedaf89d3a9be09fa77a7c1425e6af6079b6fe80` | ✅ executed |

**Result: 15/15 ✅** All routes reached `executed` on destination.

**Part-B reliability findings (MOU-29 audit):**
- Fast non-EVM routes (stellar, sui, solana) typically execute in < 60 s; axe's
5 s poll interval + live verifier may miss execution before a short per-run cap,
but the `recovered_via_api` GMP-API backstop correctly catches them.
- EVM destination view-call retry fixed (`8e1f972`) — transient RPC errors on
`isMessageApproved` / `isMessageExecuted` / `isCommandExecuted` no longer fail
the verifier loop prematurely.
- **Stellar destination view-call retry fixed (`55e3dbe`)** — same bug class as
the EVM fix, surfaced empirically by this batch: the hyperliquid→stellar route
(08) **executed on-chain at T+13–31 s**, but axe's verifier crashed (exit 1) on
a transient Stellar RPC *connection reset* at `stellar/rpc.rs:746`
(`simulate_view` → `simulate_transaction_envelope`, no retry). Wrapped the
read-only simulation in `retry_all`, mirroring the EVM + XRPL patterns.
- `INACTIVITY_TIMEOUT=7200 s` and `POLL_INTERVAL=5 s` are correctly sized for the
observed latency distribution (max seen: 3226 s for mantle→kava).
- Re-run notes: route 10 (avalanche→stellar) was a source-side setup cap on the
first pass (SenderReceiver deploy still confirming when the batch cap fired);
it passes cleanly with a longer cap. Route 14 (avalanche→base) executed on-chain
at T+2 s; an axe-side verify error only appeared when a token-gated public Base
RPC rejected the archive `getLogs` scan — an RPC-selection issue, not an axe
bug (use a private/full Base RPC for the legacy `ContractCallApproved` scan).

---

## 4. Open — routes still needing a run
Expand Down
27 changes: 27 additions & 0 deletions src/cli.rs
Original file line number Diff line number Diff line change
Expand Up @@ -231,6 +231,33 @@ pub enum TestCommands {
fresh_token: bool,
},

/// Monitor express-execution reimbursement (observe-only): for each chain
/// report recent express transfers' two phases (express-executed →
/// executor reimbursed), or watch a single source tx through both phases.
ExpressExecution {
/// Express-supported chains to monitor (axelar IDs / config keys).
/// Ignored when `--source-tx` is given.
chains: Vec<String>,

/// Monitor exactly this source tx through both phases (overrides the
/// chains scan).
#[arg(long)]
source_tx: Option<String>,

/// Path to chains config JSON (reserved; chain ids are passed directly).
#[arg(long, env = "CHAINS_CONFIG")]
config: Option<PathBuf>,

/// How many recent express transfers per chain to report in scan mode.
#[arg(long, default_value = "5")]
recent: usize,

/// Seconds to wait for the canonical execute (reimbursement) in
/// single-tx mode before reporting PENDING/timeout.
#[arg(long, default_value = "1800")]
timeout_secs: u64,
},

/// Cross-chain load test (auto-detects chains, RPCs, and test type from config)
LoadTest {
/// Path to chains config JSON (e.g. devnet-amplifier.json,
Expand Down
7 changes: 7 additions & 0 deletions src/commands/load_test/gmp.rs
Original file line number Diff line number Diff line change
Expand Up @@ -192,12 +192,14 @@ pub(super) async fn run_sol_to_evm(args: LoadTestArgs, _run_start: Instant) -> R
let vdest_rpc = args.destination_rpc.clone();
let vdone = std::sync::Arc::clone(&send_done);
let vgw = gateway_addr;
let vnetwork = args.network;
let verify_handle = tokio::spawn(async move {
let spinner = spinner_rx.await.expect("spinner channel dropped");
verify::verify_onchain_evm_streaming(
&vconfig,
&vsource,
&vdest,
vnetwork,
&vdest_addr,
vgw,
&vdest_rpc,
Expand Down Expand Up @@ -592,13 +594,15 @@ pub(super) async fn run_evm_to_evm(args: LoadTestArgs, _run_start: Instant) -> R
let vdone = std::sync::Arc::clone(&send_done);
let vgw = dest_gateway_addr;
let vlegacy = legacy_route;
let vnetwork = args.network;
let verify_handle = tokio::spawn(async move {
let spinner = spinner_rx.await.expect("spinner channel dropped");
if vlegacy {
verify::verify_onchain_evm_legacy_streaming(
&vconfig,
&vsource,
&vdest,
vnetwork,
&vdest_addr,
vgw,
&vdest_rpc,
Expand All @@ -612,6 +616,7 @@ pub(super) async fn run_evm_to_evm(args: LoadTestArgs, _run_start: Instant) -> R
&vconfig,
&vsource,
&vdest,
vnetwork,
&vdest_addr,
vgw,
&vdest_rpc,
Expand Down Expand Up @@ -952,12 +957,14 @@ pub(super) async fn run_stellar_to_evm(args: LoadTestArgs, _run_start: Instant)
let vdest_rpc = evm_rpc_url.clone();
let vdone = std::sync::Arc::clone(&send_done);
let vgw = gateway_addr;
let vnetwork = args.network;
let verify_handle = tokio::spawn(async move {
let spinner = spinner_rx.await.expect("spinner channel dropped");
verify::verify_onchain_evm_streaming(
&vconfig,
&vsource,
&vdest,
vnetwork,
&vdest_addr,
vgw,
&vdest_rpc,
Expand Down
11 changes: 11 additions & 0 deletions src/commands/load_test/helpers.rs
Original file line number Diff line number Diff line change
Expand Up @@ -630,6 +630,16 @@ pub(crate) fn print_final_report(report: &LoadTestReport) {
prev = Some(*total);
}

// Recovered via the final GMP-API check (executed on-chain despite a
// polling-loop timeout — counted as successful, surfaced distinctly).
if v.recovered_via_api > 0 {
println!();
println!(
" recovered via GMP-API final check: {}",
v.recovered_via_api
);
}

// Stuck
if v.stuck > 0 {
let stuck_detail: Vec<String> = v
Expand Down Expand Up @@ -1192,6 +1202,7 @@ pub(crate) async fn finalize_sui_dest_run_its(
&args.config,
&args.source_axelar_id,
&args.destination_axelar_id,
args.network,
sui_rpc,
&mut report.transactions,
)
Expand Down
3 changes: 3 additions & 0 deletions src/commands/load_test/its_evm_to_evm.rs
Original file line number Diff line number Diff line change
Expand Up @@ -483,12 +483,14 @@ async fn run_sustained_pipeline(
let vdest = args.destination_axelar_id.clone();
let vdest_rpc = dest_rpc_url.to_string();
let vdone = std::sync::Arc::clone(&send_done);
let vnetwork = args.network;
let verify_handle = tokio::spawn(async move {
let spinner = spinner_rx.await.expect("spinner channel dropped");
super::verify::verify_onchain_evm_its_streaming(
&vconfig,
&vsource,
&vdest,
vnetwork,
dest_gateway_addr,
&vdest_rpc,
verify_rx,
Expand Down Expand Up @@ -742,6 +744,7 @@ async fn run_burst_pipeline(
&args.config,
&args.source_axelar_id,
&args.destination_axelar_id,
args.network,
&format!("{}", targets.its_proxy_addr),
dest_gateway_addr,
dest_rpc_url,
Expand Down
3 changes: 3 additions & 0 deletions src/commands/load_test/its_evm_to_stellar.rs
Original file line number Diff line number Diff line change
Expand Up @@ -640,12 +640,14 @@ async fn run_sustained_pipeline(
let vstellar_gw = stellar.gateway_addr.clone();
let vsigner_pk = stellar.signer_pk;
let vdone = Arc::clone(&send_done);
let vnetwork = args.network;
let verify_handle = tokio::spawn(async move {
let spinner = spinner_rx.await.expect("spinner channel dropped");
super::verify::verify_onchain_stellar_its_streaming(
&vconfig,
&vsource,
&vdest,
vnetwork,
&vstellar_rpc,
&vstellar_net,
&vstellar_gw,
Expand Down Expand Up @@ -877,6 +879,7 @@ async fn run_burst_pipeline(
&args.config,
&args.source_axelar_id,
&args.destination_axelar_id,
args.network,
stellar_recipient_addr,
&stellar.rpc,
&stellar.network_type,
Expand Down
3 changes: 3 additions & 0 deletions src/commands/load_test/its_evm_to_xrpl.rs
Original file line number Diff line number Diff line change
Expand Up @@ -582,12 +582,14 @@ async fn run_sustained_pipeline(
let vxrpl_rpc = xrpl.xrpl_rpc.clone();
let vrecipient = xrpl.recipient_addr.clone();
let vdone = std::sync::Arc::clone(&send_done);
let vnetwork = args.network;
let verify_handle = tokio::spawn(async move {
let spinner = spinner_rx.await.expect("spinner channel dropped");
super::verify::verify_onchain_xrpl_its_streaming(
&vconfig,
&vsource,
&vdest,
vnetwork,
&vxrpl_rpc,
&vrecipient,
verify_rx,
Expand Down Expand Up @@ -814,6 +816,7 @@ async fn run_burst_pipeline(
&args.config,
&args.source_axelar_id,
&args.destination_axelar_id,
args.network,
&xrpl.xrpl_rpc,
&xrpl.recipient_addr,
&mut report.transactions,
Expand Down
4 changes: 3 additions & 1 deletion src/commands/load_test/its_sol_to_evm.rs
Original file line number Diff line number Diff line change
Expand Up @@ -377,10 +377,11 @@ async fn run_sustained_pipeline(
let vdest_rpc = evm_rpc_url.clone();
let vdone = Arc::clone(&send_done);
let vgw = evm.evm_gateway_addr;
let vnetwork = args.network;
let verify_handle = tokio::spawn(async move {
let spinner = spinner_rx.await.expect("spinner channel dropped");
super::verify::verify_onchain_evm_its_streaming(
&vconfig, &vsource, &vdest, vgw, &vdest_rpc, verify_rx, vdone, spinner,
&vconfig, &vsource, &vdest, vnetwork, vgw, &vdest_rpc, verify_rx, vdone, spinner,
)
.await
});
Expand Down Expand Up @@ -717,6 +718,7 @@ async fn run_burst_pipeline(
&args.config,
&args.source_axelar_id,
&args.destination_axelar_id,
args.network,
&format!("{}", evm.its_proxy_addr),
evm.evm_gateway_addr,
evm_rpc_url,
Expand Down
Loading
Loading