PE-9132: Release v2.86.0 — file sharing redesign, authenticated downloads, single-gateway sync - #2173
Open
vilenarios wants to merge 47 commits into
Open
PE-9132: Release v2.86.0 — file sharing redesign, authenticated downloads, single-gateway sync#2173vilenarios wants to merge 47 commits into
vilenarios wants to merge 47 commits into
Conversation
Two optimizations for snapshot loading during sync:
1. Prefetch: start downloading the next snapshot body while the
current one is being parsed and written to DB. Only 1 ahead to
avoid gateway 429s. Overlaps network I/O with CPU work.
2. Streaming parse: instead of jsonDecode on the entire snapshot
(which builds a massive Map with all entries in memory), scan
the JSON string for individual txSnapshot object boundaries
using brace counting and parse each one separately. This:
- Avoids building the full nested Map (lower peak memory)
- Starts yielding transactions immediately
- Still needs the full string downloaded, but parsing is
incremental
For a drive with 3 snapshots of ~15MB, ~10MB, ~5MB:
Before: download1 + parse1 + download2 + parse2 + download3 + parse3
After: download1 + [parse1 || download2] + [parse2 || download3] + parse3
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This reverts commit 7833178.
…2142) * perf: bulk-load revision lookups instead of per-entity DB queries PE-9103 During sync, for each file/folder entity, the code issued individual SELECT queries to find the latest and oldest revisions. For a drive with 19k files, that was 38k+ individual DB queries (19k latest + 19k oldest for files, plus similar for folders). Changes: - Add 4 bulk SQL queries to drive_queries.drift (latest/oldest × files/folders) using GROUP BY with MAX/MIN dateCreated - Add 4 helper methods to DriveDao returning Map<entityId, Revision> - Pre-load all revision maps at the start of each transaction chunk - Pass maps through to _addNewFileEntityRevisions, _addNewFolderEntityRevisions, _computeRefreshedFileEntriesFromRevisions, _computeRefreshedFolderEntriesFromRevisions - Update latestRevisionsCache as new revisions are inserted so subsequent sub-batches see fresh data - First-sync shortcut: skip all bulk queries when drive.lastBlockHeight is 0/null (every entity is new, no previous revisions exist) Performance: 38,000+ individual SELECTs → 4 bulk SELECTs per chunk. First sync: 0 queries (all skipped). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * perf: add 5s request timeout + reduce retries on data gateway PE-9103 Entity metadata fetches had NO timeout — a slow gateway could hang indefinitely. With 2 retries × 5 gateways = 10 attempts with no timeout, a single failed entity could take minutes. Changes: - Add 5-second timeout per HTTP request (metadata is tiny JSON) - Reduce retries per gateway from 2 to 1 (move on, don't retry slow) - Reduce GAR fallbacks from 3 to 2 Worst case per entity: 4 attempts × 5s = 20s max (was: 10 attempts, unlimited time) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
A single missing/broken tx could block sync for minutes as the fallback chain tried every gateway with CORS timeouts and 504s. Added a 15-second total timeout wrapping the entire fallback chain (primary → GAR gateways → arweave.net). Combined with the existing 5s per-request timeout, worst case for any single tx is now 15s max. This prevents the sync from appearing "stuck" — even if metadata fetches fail, the sync completes within a bounded time and the periodic sync can trigger again. Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Two optimizations for snapshot loading during sync:
1. Prefetch: start downloading the next snapshot body while the
current one is being parsed and written to DB. Only 1 ahead to
avoid gateway 429s. Overlaps network I/O with CPU work.
2. Streaming parse: instead of jsonDecode on the entire snapshot
(which builds a massive Map with all entries in memory), scan
the JSON string for individual txSnapshot object boundaries
using brace counting and parse each one separately. This:
- Avoids building the full nested Map (lower peak memory)
- Starts yielding transactions immediately
- Still needs the full string downloaded, but parsing is
incremental
For a drive with 3 snapshots of ~15MB, ~10MB, ~5MB:
Before: download1 + parse1 + download2 + parse2 + download3 + parse3
After: download1 + [parse1 || download2] + [parse2 || download3] + parse3
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…PE-9103 - getPrice now retries 3 times with backoff instead of failing on a single 502 from the gateway - AR cost calculation in the upload modal falls back to zero estimate on failure so the modal still opens with Turbo available instead of crashing entirely Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…c PE-9103 toEntryCompanion() on file/folder revision companions produces entry companions with dateCreated = Value.absent() (schema has DEFAULT). On first sync, oldestRevisionsCache is empty, so the fallback .dateCreated.value returns null → JSNull crash on web. Fixed by keeping a separate map of revision dateCreated values and using those as the fallback instead of the entry companion's field. Same bug pattern as the drive revision fix (line 1887), but for files (line 1838) and folders (line 1871). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add an optional owners filter to the by-ID GraphQL queries that the gateway struggles with (large ClickHouse row scans). Scoping by owner lets the gateway prune its search space. The owner variable is nullable, so when an owner can't be determined the query behaves exactly as before. - TransactionStatuses, LicenseComposed, LicenseAssertions: add $owners - getTransactionConfirmations / getLicenseComposed / getLicenseAssertions: accept an optional owner and pass it through - tx status (per-drive): scope to the drive's ownerAddress - tx status (global): scope to the logged-in wallet's address - licenses: scope to the best-known tx owner (revision/file owner) Edge case: data txs of files pinned from other authors are owned by those authors and won't match the owner filter; treated as best-effort. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…E-9126 Owner-scoped getTransactionConfirmations leaves cross-owner txs (e.g. the data tx of a file pinned from another author's upload) at -1, which the sync status logic would turn into 'failed' after the pending timeout. Add a fallback pass: after the selective owner-scoped query, re-query any ids still unresolved (-1) without the owner filter to determine their true status. The residual set is normally tiny, so the selectivity win of the first pass is preserved while confirmed cross-owner txs are no longer misclassified as failed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…y PE-9126 The previous fallback re-queried unresolved (-1) txs without an owner filter, which can re-trigger the gateway's expensive row scan that the owner scoping was meant to avoid — and, because the call is wrapped in a 5s timeout, a slow/erroring fallback discards the whole batch's results and makes no progress, repeating every sync. Replace it with a selective approach: only re-query unresolved ids whose real on-chain owner is known locally. Currently that's pinned files, whose data tx is owned by the original uploader (pinnedDataOwnerAddress). These are re-queried scoped to that owner, so every query stays selective. Genuinely missing txs have no override and are left unresolved, handled by the caller's existing pending/failed logic — no unscoped scan, no retry storm on the common "not found yet" case. - add pinnedFileRevisions drift query - getTransactionConfirmations: ownerOverrides param replaces unscoped pass - sync repo: build pin owner overrides and pass them through Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The second (pinned-owner) pass merges into the already-populated first-pass map, so its failure must never discard first-pass progress. Wrap it in a try/catch and bound it with its own 3s timeout: even if the pin re-queries error or stall, the confirmations resolved by the owner-scoped first pass are still returned. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ink PE-9126 getTransactionConfirmations is wrapped in a 5s timeout by the sync status update; on expiry it previously returned an empty map, discarding every confirmation already resolved that cycle (the whole 5000-tx page). Add an optional caller-owned verifiedSink that the method populates with each resolved confirmation (>= 0 only) as queries complete. The callers pass one in and, on timeout, fall back to it instead of an empty map — so work done before the deadline (including a fully-completed first pass when only the pinned-owner pass is slow) is applied rather than thrown away. Because the sink holds only positive verifications and never the -1 "not found" placeholders, applying it after a partial/timed-out run only upgrades txs to confirmed and never marks anything failed off incomplete data. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Cap the per-chunk GraphQL fan-out in getTransactionConfirmations to maxConcurrentDataFetches instead of launching every chunk at once, so a large pending-tx page can't burst into a concurrent-retry storm against the gateway (matches the throttling on the other gateway-heavy paths). - Replace `sublist(...) as List<String>?` with `.whereType<String>().toList()`. The cast threw a TypeError for the pinned-owner pass (whose ids list is typed List<String?>), which the best-effort catch swallowed — silently disabling pin recovery. The filter yields a real List<String> regardless of input type. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ql-queries PE-9126: Add drive owner to tx status and license GraphQL queries
* fix: empty explorer after drive attach PE-9103 Two bugs caused the explorer to show a permanent spinner after attaching a drive (data was in DB but UI didn't update): 1. buildWhen guard in drive_detail_page.dart blocked BlocBuilder rebuilds when SyncCubit was SyncInProgress. If DriveDetailCubit emitted DriveDetailLoadSuccess during sync, the emission was permanently skipped with no replay mechanism. Removed the sync check — DriveDetailCubit already gates emissions via waitCurrentSync() in the Rx.combineLatest3 callback. 2. startSyncForDrive silently aborted when a sync was in progress. The .then(selectDrive) still fired, selecting a drive whose content was never synced. Changed to await waitCurrentSync() so the single-drive sync runs after the current sync finishes. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * perf: faster snapshot validation — skip fallback on 404, reduce timeouts PE-9103 Snapshot validation was slow on localhost (and anywhere Solana RPC is unavailable): 2 HEAD retries × 10s timeout + GAR fallback via Solana = 25+ seconds per snapshot. With 16 drives having multiple snapshots, sync appeared stuck. Changes: - 1 HEAD attempt instead of 2 (fail fast, fall back to GQL) - HEAD timeout 10s → 5s - GAR list timeout 5s → 3s - Skip GAR fallback entirely when primary returned 404 (snapshot doesn't exist, no point trying other gateways via Solana RPC) - Only try GAR fallback for transient errors (timeout, 5xx) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: guard startSyncForDrive race after waitCurrentSync PE-9103 Two callers could both pass the SyncInProgress guard after waitCurrentSync() returned, causing concurrent single-drive syncs. Re-check state after wait — if another sync started, bail out. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Pulls in the TransactionData null-field fix so downloads no longer crash on web with "type 'JSNull' is not a subtype of type 'String'" when a gateway returns null for optional tx fields (e.g. anchor on turbo-gateway.com). Updates both the dependency and the override. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
PE-9128: Bump arweave-dart to v4.0.2 to fix file download crash
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* ui: modernize share file and download all modals PE-9103 Both modals used the old ArDriveStandardModal with deprecated typography and color tokens. Updated to match the current design system used by drive attach, upload, and other modern modals. Share File modal: - ArDriveStandardModal → ArDriveStandardModalNew (adds red header bar) - Old typography (buttonNormalBold, buttonLargeRegular) → semantic ArDriveTypographyNew (paragraphSmall, paragraphNormal) - Old colors (themeFgDefault, themeWarningEmphasis) → colorTokens (textHigh, textMid, textLow, strokeRed) - ArDriveTextField → ArDriveTextFieldNew - Warning banner wrapped in styled container with containerL1 bg Download All Files modal: - All 3 ArDriveStandardModal instances → ArDriveStandardModalNew - File list items wrapped in styled containers (containerL1 bg, rounded corners, file/folder icons, proper spacing) - Old typography (smallBold, smallRegular) → semantic typography - Old colors (themeFgSubtle, themeFgMuted) → colorTokens - Error state text styled with new typography Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address CodeRabbit review issues in share/download modals PE-9103 - add explicit FileShareLoadedPendingFile state branch with localized body text - add TODO comment for hardcoded pending warning string (needs ARB extraction) - add close action to fallback modal in multiple file download Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…t PE-9126 The global tx-status update assumed the logged-in wallet owns every pending tx and scoped the whole batch by walletAddress. That breaks for ATTACHED drives owned by other wallets: their pending data txs were queried under the wrong owner, and when no wallet is present (browsing a public/attached drive) walletAddress is null, so TransactionStatuses went out unscoped — hitting the gateway's expensive full-scan path and timing out. Resolve each pending tx's owner from the drive it actually belongs to: - add pendingDataFileRevisions drift query (read-only; no schema change) - _buildPendingTxDriveOwners: map pending data tx id -> its drive's ownerAddress - getTransactionConfirmations: new ownersByTxId map; first pass groups txs by their resolved per-tx owner (map wins, else the single owner fallback) and queries each owner once, so a batch spanning multiple drives stays selective. A tx with no resolvable owner is left unresolved rather than queried unscoped. - global _updateTransactionStatuses passes the per-tx owner map; walletAddress remains only as a fallback for unmapped txs. The per-drive path is unchanged (single drive => single owner). Pinned-owner recovery (pass 2) still works: pins map to their drive owner in pass 1 (miss) and are recovered under pinnedDataOwnerAddress in pass 2. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…E-9126 A pending tx with no resolvable owner (e.g. global path with no wallet and a tx not mappable to a drive) is never queried, but was left at its pre-seeded -1 in the result map. The caller reads -1 as "not found" and can age an old pending tx into failed. Remove such txs from the returned map so they're absent rather than -1: the caller skips absent txs (no status change), leaving them pending to be retried once their owner is known — the same "unknown => skip, never fail off incomplete data" rule used by the verified-sink path. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…apping PE-9126: Scope tx-status query per-tx by drive owner (fix attached-drive sync)
The confirmation query intermittently takes ~10s when turbo-gateway's indexer-core circuit breaker is open (observed 9.87s with UPSTREAM_CIRCUIT_OPEN / "timeout of 9500ms exceeded" warnings, returning valid data). The client's 5s per-batch timeout fired first and discarded the whole batch, leaving long-confirmed txs stuck as pending even though the gateway returned their confirmations. Move both timeouts above that ~9.5s ceiling: - per-batch getTransactionConfirmations: 5s -> 15s - overall per-drive/global status update: 10s -> 30s Extracted as named constants with the rationale. Typical responses are ~0.5s, so this only lengthens waits while the gateway is degraded (when we want to wait, not drop the batch); the verified sink still preserves partial progress. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…-9126 Reword the timeout rationale in terms of general gateway slowness rather than a specific gateway's internal index/circuit-breaker/warning codes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…lerance PE-9126: Raise tx-status timeouts above gateway upstream ceiling
* perf: sync pipeline performance and resilience optimizations PE-9103 - skip unchanged drives via bulk GQL probe (1 query per owner replaces ~40 queries per idle drive) - add database indexes on file_revisions.dataTxId and network_transactions.status - bulk pre-load dateCreated for tx status updates (eliminates N+1 per-tx DB queries) - bulk pre-load folders and drives in ghost creation (eliminates N+1 per-folder DB queries) - batch transaction status writes using insertNewNetworkTransactions (replaces individual writes) - replace full file_revisions table load with filtered query for snapshot tx matching - fix DataGatewayFallback timeout budget (15s→25s) so arweave.net is reachable - fix 4 GraphQL queries bypassing GraphQLRetry (missing retry + fallback) - remove 200ms artificial delay between tx status batches - bump database schema version 28→29 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: don't skip post-sync ops when all drives are unchanged PE-9103 The early return on numberOfDrivesToSync == 0 skipped transaction status updates, ghost folder creation, and ARNS record updates. Pending transactions from recent uploads need confirmation checks even when no drives have new entities. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * perf: skip GraphQL call for public file downloads PE-9103 Public file downloads were calling getTransactionDetails() via GraphQL before starting the download — entirely unnecessary since the txId is already known locally. This GQL call was also one of the four that bypassed GraphQLRetry, making it the likely cause of downloads failing to start when the gateway returns 429/5xx. Now only private/encrypted files call GraphQL (to fetch cipher/IV tags from the data transaction). Public files start downloading immediately with no network round-trip. Refactored ArDriveDownloader interface: replaced TransactionCommonMixin dataTx parameter with String txId + bool verifyDownload, since only those two values were ever used from the full transaction object. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: hedged gateway requests, download fallback, retry UX, drive attach dedup PE-9103 hedged gateway requests: - replace serial waterfall with staggered parallel requests in DataGatewayFallback - fire primary immediately, launch fallbacks every 1.5s if no response - first 200 response wins, rest ignored — worst case ~5s instead of ~20s - applies to all metadata fetches automatically (no caller changes) download resilience: - add downloadWithFallback() for file downloads with same hedged pattern - add fetchManifestWithFallback() for manifest downloads (had zero fallback) - add stall detection: throws DownloadStalledException if no chunk for 60s - typed exceptions: DownloadFileNotFoundException, DownloadNetworkException, DownloadRateLimitException, DownloadStalledException download UX: - differentiated error dialogs: network error, file not found, rate limited - retry button on retryable failures (network, rate limit, unknown) - file not found shows OK only (retry won't help) - error classification in both personal and shared download cubits drive attach dedup: - getDrivePrivacyForId() now returns DrivePrivacyResult with owner + tx node - drivePrivacyLoader() passes owner to getLatestDriveEntityWithId(), skipping redundant owner lookup — 4 GQL queries reduced to 2 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: revert metadata fetches to serial fallback, keep hedged for downloads only PE-9103 Hedged (staggered parallel) requests fire extra gateway requests when the primary is slow but succeeds. During sync with hundreds of metadata fetches, this wastes bandwidth and could trigger rate limits on GAR gateways. Now: - metadata fetches (fetchData): serial waterfall (primary → GAR → arweave.net) - file downloads (downloadWithFallback): hedged staggered (latency-sensitive, single request) - manifest downloads (fetchManifestWithFallback): serial waterfall Also fixes stall detection for empty files — timer only starts after the first chunk arrives, so 0-byte files don't trigger DownloadStalledException. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * perf: batch snapshot queries and share gateway cache PE-9103 batch snapshot queries: - SnapshotEntityHistory.graphql now accepts $driveIds array instead of single $driveId — fetches snapshots for all drives in one paginated query - syncAllDrives() prefetches snapshots for all drives per owner before the per-drive sync loop, passes results to _syncDrive() - reduces N snapshot GQL queries (one per drive) to 1 per unique owner - also fixes Entity-Type tag syntax: values: "snapshot" → values: ["snapshot"] share gateway cache: - DataGatewayFallback.cachedGateways is now public so SnapshotValidationService can reuse the same gateway list - syncAllDrives() passes the cache before sync starts - eliminates 1 duplicate Solana RPC call per sync cycle Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address CodeRabbit review — stall detection + mobile GCM path PE-9103 - forward verifyDownload param to mobile AES-GCM download path - wrap mobile GCM stream with _withStallDetection (was bypassed) - cancel upstream subscription when stall timer fires - guard against adding to closed StreamController in stall detection Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * perf: eliminate redundant GraphQL calls in sync pipeline PE-9103 HAR analysis of a real sync session (16 drives, no changes) revealed 130 GraphQL calls taking 376s. This commit reduces that to ~3 calls and <1s for incremental syncs with no changes. Changes: - fix DriveActivityProbe: partition drives into never-synced and previously-synced before probing. Never-synced drives (lastBlockHeight=0) were poisoning the probe's minBlockHeight to 0, causing it to query from genesis, overflow the page limit, and fall back to syncing ALL drives. Now only previously-synced drives are probed. - cache UserDriveEntityTxs in ArweaveService with event-based invalidation. Auth flow (isExistingUser, _validateUser) and sync (updateUserDrives) all call getUniqueUserDriveEntityTxs for the same wallet within seconds. Cache is cleared after sync completion. - cache updateUserDrives in SyncRepository with event-based flag. Multiple entry points (syncMetadataOnly, startSync, startSyncForDrive) call it redundantly. Flag cleared only when sync processes drives. - skip genesis block [0,0] range in GQLDriveHistory. Snapshot gaps at block 0 produced phantom ranges causing 3 wasted queries per drive. - add local DB pre-check for PendingDriveEntities. Only query gateway when pendingTransactionsForDrive returns local entries. Skip entirely for non-owned (read-only) drives. - stop Solana RPC retry spam in DataGatewayFallback and SnapshotValidationService. Cache empty gateway list on first failure instead of retrying every fetchData/validation call. - fix zero-drives-to-sync: when probe skips all drives, return emptySyncCompleted instead of falling through to "all failed". Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address CodeRabbit review — race condition, cache sharing, probe gaps PE-9103 - fix updateUserDrives race condition: replace boolean flag with Future so concurrent callers await the in-flight request instead of both firing - fix snapshot prefetch minBlock: exclude never-synced drives from the min calculation so they don't drag the batched snapshot query to block 0 - fix gateway cache sharing: pass DataGatewayFallback reference to SnapshotValidationService instead of copying the list, so both services share one cache and writes propagate bidirectionally - add cancellation check in probe loop before each owner group Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * perf: cache entity data, drive signatures, and skip redundant balance refresh PE-9103 - cache raw entity data bytes from getUniqueUserDriveEntities so getLatestDriveEntityWithId (called during password validation) can re-parse without re-downloading from the gateway - cache drive signatures permanently (immutable on-chain) to avoid redundant GQL + data fetch on every login for v1-signed private drives - skip refreshBalance after no-op sync (drivesSynced == 0) to avoid redundant PendingTxFees query when nothing changed - skip redundant getLatestDriveEntityWithId in drive attach flow when drivePrivacyLoader already cached the entity Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * perf: use unfiltered GQL strategy for drive history (48 calls → 16) PE-9103 Switch GQLDriveHistory from GetSegmentedTransactionFromDrive- FilteringByEntityTypeStrategy (3 queries per drive: drive, folder, file) to the unfiltered strategy (1 query per drive returning all entity types). The downstream parsing pipeline already separates entities by type via whereType<DriveEntity/FolderEntity/FileEntity>, so the per-type filtering at the query level was redundant round trips. For 16 drives on first sync: 48 → 16 DriveEntityHistory calls. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: clear cached updateUserDrives future on error to allow retry PE-9103 If the updateUserDrives future completes with an error (transient network issue), subsequent callers would receive the same cached error without retrying. Now the cached future is cleared on error so the next caller gets a fresh attempt. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address CodeRabbit review — infinite loop guard, dead code cleanup PE-9103 - add empty-edges guard in getAllSnapshotsForDrives pagination loop to prevent infinite loop when gateway returns hasNextPage=true with 0 edges - remove unreachable duplicate cachedDriveEntity check in driveNameLoader - fix misleading comment in drivePrivacyLoader Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* ux: sync modal improvements — retry, drive names, elapsed time, probe status PE-9103
- show "Checking for changes..." during drive activity probe phase
instead of misleading "0 of 16 Drives Synced"
- include drive names in sync error messages (e.g., "My Drive: Gateway
timeout (504)") so users know which drive failed
- add "Retry Failed" button in sync error modal to retry only the
drives that failed without re-syncing everything
- show elapsed time (e.g., "45s elapsed") after 5 seconds during sync
so users know the sync is progressing on longer first syncs
- expose syncStartTime getter on SyncCubit for elapsed time widget
- add localization keys for all new strings in 6 locales
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: retry only failed drives, not all drives PE-9103
CodeRabbit correctly identified that retryFailedDrives was calling
startSync(deepSync: true) which resyncs ALL drives. Now iterates
over failed drive IDs and syncs each individually.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: route retry through syncAllDrives with driveIdsToRetry filter PE-9103
Instead of looping startSyncForDrive (which flashes the modal per
drive), add driveIdsToRetry parameter to syncAllDrives that filters
the drives list. Retry runs as a single sync session with one modal,
proper ghost creation, and transaction status updates.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: address CodeRabbit review — safe getter, consistent error names PE-9103
- make _initSync non-late (initialized to DateTime.now()) to prevent
LateInitializationError if syncStartTime is read before sync starts
- prefix drive name in syncSingleDrive error messages to match
syncAllDrives format ("Drive Name: error message")
- skip localization finding: statusMessage strings are hardcoded English
as a pre-existing pattern — repository has no BuildContext access
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* perf: sync pipeline performance and resilience optimizations PE-9103 - skip unchanged drives via bulk GQL probe (1 query per owner replaces ~40 queries per idle drive) - add database indexes on file_revisions.dataTxId and network_transactions.status - bulk pre-load dateCreated for tx status updates (eliminates N+1 per-tx DB queries) - bulk pre-load folders and drives in ghost creation (eliminates N+1 per-folder DB queries) - batch transaction status writes using insertNewNetworkTransactions (replaces individual writes) - replace full file_revisions table load with filtered query for snapshot tx matching - fix DataGatewayFallback timeout budget (15s→25s) so arweave.net is reachable - fix 4 GraphQL queries bypassing GraphQLRetry (missing retry + fallback) - remove 200ms artificial delay between tx status batches - bump database schema version 28→29 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: don't skip post-sync ops when all drives are unchanged PE-9103 The early return on numberOfDrivesToSync == 0 skipped transaction status updates, ghost folder creation, and ARNS record updates. Pending transactions from recent uploads need confirmation checks even when no drives have new entities. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * perf: skip GraphQL call for public file downloads PE-9103 Public file downloads were calling getTransactionDetails() via GraphQL before starting the download — entirely unnecessary since the txId is already known locally. This GQL call was also one of the four that bypassed GraphQLRetry, making it the likely cause of downloads failing to start when the gateway returns 429/5xx. Now only private/encrypted files call GraphQL (to fetch cipher/IV tags from the data transaction). Public files start downloading immediately with no network round-trip. Refactored ArDriveDownloader interface: replaced TransactionCommonMixin dataTx parameter with String txId + bool verifyDownload, since only those two values were ever used from the full transaction object. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: hedged gateway requests, download fallback, retry UX, drive attach dedup PE-9103 hedged gateway requests: - replace serial waterfall with staggered parallel requests in DataGatewayFallback - fire primary immediately, launch fallbacks every 1.5s if no response - first 200 response wins, rest ignored — worst case ~5s instead of ~20s - applies to all metadata fetches automatically (no caller changes) download resilience: - add downloadWithFallback() for file downloads with same hedged pattern - add fetchManifestWithFallback() for manifest downloads (had zero fallback) - add stall detection: throws DownloadStalledException if no chunk for 60s - typed exceptions: DownloadFileNotFoundException, DownloadNetworkException, DownloadRateLimitException, DownloadStalledException download UX: - differentiated error dialogs: network error, file not found, rate limited - retry button on retryable failures (network, rate limit, unknown) - file not found shows OK only (retry won't help) - error classification in both personal and shared download cubits drive attach dedup: - getDrivePrivacyForId() now returns DrivePrivacyResult with owner + tx node - drivePrivacyLoader() passes owner to getLatestDriveEntityWithId(), skipping redundant owner lookup — 4 GQL queries reduced to 2 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: revert metadata fetches to serial fallback, keep hedged for downloads only PE-9103 Hedged (staggered parallel) requests fire extra gateway requests when the primary is slow but succeeds. During sync with hundreds of metadata fetches, this wastes bandwidth and could trigger rate limits on GAR gateways. Now: - metadata fetches (fetchData): serial waterfall (primary → GAR → arweave.net) - file downloads (downloadWithFallback): hedged staggered (latency-sensitive, single request) - manifest downloads (fetchManifestWithFallback): serial waterfall Also fixes stall detection for empty files — timer only starts after the first chunk arrives, so 0-byte files don't trigger DownloadStalledException. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * perf: batch snapshot queries and share gateway cache PE-9103 batch snapshot queries: - SnapshotEntityHistory.graphql now accepts $driveIds array instead of single $driveId — fetches snapshots for all drives in one paginated query - syncAllDrives() prefetches snapshots for all drives per owner before the per-drive sync loop, passes results to _syncDrive() - reduces N snapshot GQL queries (one per drive) to 1 per unique owner - also fixes Entity-Type tag syntax: values: "snapshot" → values: ["snapshot"] share gateway cache: - DataGatewayFallback.cachedGateways is now public so SnapshotValidationService can reuse the same gateway list - syncAllDrives() passes the cache before sync starts - eliminates 1 duplicate Solana RPC call per sync cycle Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address CodeRabbit review — stall detection + mobile GCM path PE-9103 - forward verifyDownload param to mobile AES-GCM download path - wrap mobile GCM stream with _withStallDetection (was bypassed) - cancel upstream subscription when stall timer fires - guard against adding to closed StreamController in stall detection Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * perf: eliminate redundant GraphQL calls in sync pipeline PE-9103 HAR analysis of a real sync session (16 drives, no changes) revealed 130 GraphQL calls taking 376s. This commit reduces that to ~3 calls and <1s for incremental syncs with no changes. Changes: - fix DriveActivityProbe: partition drives into never-synced and previously-synced before probing. Never-synced drives (lastBlockHeight=0) were poisoning the probe's minBlockHeight to 0, causing it to query from genesis, overflow the page limit, and fall back to syncing ALL drives. Now only previously-synced drives are probed. - cache UserDriveEntityTxs in ArweaveService with event-based invalidation. Auth flow (isExistingUser, _validateUser) and sync (updateUserDrives) all call getUniqueUserDriveEntityTxs for the same wallet within seconds. Cache is cleared after sync completion. - cache updateUserDrives in SyncRepository with event-based flag. Multiple entry points (syncMetadataOnly, startSync, startSyncForDrive) call it redundantly. Flag cleared only when sync processes drives. - skip genesis block [0,0] range in GQLDriveHistory. Snapshot gaps at block 0 produced phantom ranges causing 3 wasted queries per drive. - add local DB pre-check for PendingDriveEntities. Only query gateway when pendingTransactionsForDrive returns local entries. Skip entirely for non-owned (read-only) drives. - stop Solana RPC retry spam in DataGatewayFallback and SnapshotValidationService. Cache empty gateway list on first failure instead of retrying every fetchData/validation call. - fix zero-drives-to-sync: when probe skips all drives, return emptySyncCompleted instead of falling through to "all failed". Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address CodeRabbit review — race condition, cache sharing, probe gaps PE-9103 - fix updateUserDrives race condition: replace boolean flag with Future so concurrent callers await the in-flight request instead of both firing - fix snapshot prefetch minBlock: exclude never-synced drives from the min calculation so they don't drag the batched snapshot query to block 0 - fix gateway cache sharing: pass DataGatewayFallback reference to SnapshotValidationService instead of copying the list, so both services share one cache and writes propagate bidirectionally - add cancellation check in probe loop before each owner group Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * perf: cache entity data, drive signatures, and skip redundant balance refresh PE-9103 - cache raw entity data bytes from getUniqueUserDriveEntities so getLatestDriveEntityWithId (called during password validation) can re-parse without re-downloading from the gateway - cache drive signatures permanently (immutable on-chain) to avoid redundant GQL + data fetch on every login for v1-signed private drives - skip refreshBalance after no-op sync (drivesSynced == 0) to avoid redundant PendingTxFees query when nothing changed - skip redundant getLatestDriveEntityWithId in drive attach flow when drivePrivacyLoader already cached the entity Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * perf: use unfiltered GQL strategy for drive history (48 calls → 16) PE-9103 Switch GQLDriveHistory from GetSegmentedTransactionFromDrive- FilteringByEntityTypeStrategy (3 queries per drive: drive, folder, file) to the unfiltered strategy (1 query per drive returning all entity types). The downstream parsing pipeline already separates entities by type via whereType<DriveEntity/FolderEntity/FileEntity>, so the per-type filtering at the query level was redundant round trips. For 16 drives on first sync: 48 → 16 DriveEntityHistory calls. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: clear cached updateUserDrives future on error to allow retry PE-9103 If the updateUserDrives future completes with an error (transient network issue), subsequent callers would receive the same cached error without retrying. Now the cached future is cleared on error so the next caller gets a fresh attempt. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address CodeRabbit review — infinite loop guard, dead code cleanup PE-9103 - add empty-edges guard in getAllSnapshotsForDrives pagination loop to prevent infinite loop when gateway returns hasNextPage=true with 0 edges - remove unreachable duplicate cachedDriveEntity check in driveNameLoader - fix misleading comment in drivePrivacyLoader Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * perf: snapshot creation progress reporting, caching, and retry PE-9103 progress reporting: - ComputingSnapshotData state now includes processedTransactions and totalTransactions (optional, defaults to 0 for backward compat) - SnapshotItemToBeCreated accepts onProgress callback, fires after each batch of 100 transactions - dialog shows "Processing X of Y transactions..." instead of just "This may take a while" performance: - cache drive privacy check once in _reset() instead of querying driveDao.driveById() per transaction (eliminates N+1 DB queries) - cache MetadataCache instance once instead of re-creating per transaction retry without recompute: - cache computed snapshot data after _getSnapshotData() completes - on upload failure, "Try Again" reuses cached data (skips 30-120s recomputation) - cache cleared on success or drive/range change Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: snapshot creation cache staleness + missing Drive-Id logging PE-9103 - always refresh MetadataCache on _reset() instead of ??= to avoid stale cache references from prior sessions - clear _cachedSnapshotData on cancellation to prevent reusing partially computed data on retry - log warning when snapshot transaction has no Drive-Id tag during batched prefetch (silent skip was hiding malformed snapshots) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: guard progress emit against disposed cubit PE-9103 If the user dismisses the snapshot dialog while computation is running, the progress callback would call emit() on a closed cubit, crashing with StateError. Now checks isClosed before emitting. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: download success dialog shows filename instead of duplicate title PE-9103 - success dialog was showing "Download Finished" as both title and description — now shows the filename as description - check saveResult in onDone handler so cancelled browser save dialogs don't incorrectly show the success modal Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: turbo payment failure no longer blocks snapshot creation PE-9103 If the Turbo payment service is unreachable (e.g. payment.ardrive.dev returns 404/500), the entire snapshot flow failed with ComputeSnapshotDataFailure — even though AR payment would have worked. Now wraps Turbo cost calculation in try-catch: on failure, Turbo is marked unavailable and the confirmation dialog shows with AR as the only payment option. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: lazy-init MetadataCache to fix CI test failures PE-9103 Moving newSharedPreferencesCacheStore() into _reset() broke all 7 create_snapshot_cubit tests in CI — the shared_preferences plugin isn't available in the test environment (no platform channel). Now lazily initialized on first use in _jsonMetadataOfTxId() instead of eagerly in _reset(). Cache is still cleared on reset (set to null) so stale references are avoided. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * ci: trigger CI run --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* perf: prefetch next snapshot + streaming JSON parse PE-9103
Two optimizations for snapshot loading during sync:
1. Prefetch: start downloading the next snapshot body while the
current one is being parsed and written to DB. Only 1 ahead to
avoid gateway 429s. Overlaps network I/O with CPU work.
2. Streaming parse: instead of jsonDecode on the entire snapshot
(which builds a massive Map with all entries in memory), scan
the JSON string for individual txSnapshot object boundaries
using brace counting and parse each one separately. This:
- Avoids building the full nested Map (lower peak memory)
- Starts yielding transactions immediately
- Still needs the full string downloaded, but parsing is
incremental
For a drive with 3 snapshots of ~15MB, ~10MB, ~5MB:
Before: download1 + parse1 + download2 + parse2 + download3 + parse3
After: download1 + [parse1 || download2] + [parse2 || download3] + parse3
* Revert "perf: prefetch next snapshot + streaming JSON parse PE-9103"
This reverts commit 7833178.
* PE-9103: Bulk-load revision lookups instead of per-entity DB queries (#2142)
* perf: bulk-load revision lookups instead of per-entity DB queries PE-9103
During sync, for each file/folder entity, the code issued individual
SELECT queries to find the latest and oldest revisions. For a drive
with 19k files, that was 38k+ individual DB queries (19k latest +
19k oldest for files, plus similar for folders).
Changes:
- Add 4 bulk SQL queries to drive_queries.drift (latest/oldest ×
files/folders) using GROUP BY with MAX/MIN dateCreated
- Add 4 helper methods to DriveDao returning Map<entityId, Revision>
- Pre-load all revision maps at the start of each transaction chunk
- Pass maps through to _addNewFileEntityRevisions,
_addNewFolderEntityRevisions, _computeRefreshedFileEntriesFromRevisions,
_computeRefreshedFolderEntriesFromRevisions
- Update latestRevisionsCache as new revisions are inserted so
subsequent sub-batches see fresh data
- First-sync shortcut: skip all bulk queries when drive.lastBlockHeight
is 0/null (every entity is new, no previous revisions exist)
Performance: 38,000+ individual SELECTs → 4 bulk SELECTs per chunk.
First sync: 0 queries (all skipped).
* perf: add 5s request timeout + reduce retries on data gateway PE-9103
Entity metadata fetches had NO timeout — a slow gateway could hang
indefinitely. With 2 retries × 5 gateways = 10 attempts with no
timeout, a single failed entity could take minutes.
Changes:
- Add 5-second timeout per HTTP request (metadata is tiny JSON)
- Reduce retries per gateway from 2 to 1 (move on, don't retry slow)
- Reduce GAR fallbacks from 3 to 2
Worst case per entity: 4 attempts × 5s = 20s max (was: 10 attempts,
unlimited time)
---------
* fix: add 15s total timeout on data fetch fallback chain PE-9103 (#2143)
A single missing/broken tx could block sync for minutes as the
fallback chain tried every gateway with CORS timeouts and 504s.
Added a 15-second total timeout wrapping the entire fallback chain
(primary → GAR gateways → arweave.net). Combined with the existing
5s per-request timeout, worst case for any single tx is now 15s max.
This prevents the sync from appearing "stuck" — even if metadata
fetches fail, the sync completes within a bounded time and the
periodic sync can trigger again.
* perf: prefetch next snapshot + streaming JSON parse PE-9103 (#2141)
Two optimizations for snapshot loading during sync:
1. Prefetch: start downloading the next snapshot body while the
current one is being parsed and written to DB. Only 1 ahead to
avoid gateway 429s. Overlaps network I/O with CPU work.
2. Streaming parse: instead of jsonDecode on the entire snapshot
(which builds a massive Map with all entries in memory), scan
the JSON string for individual txSnapshot object boundaries
using brace counting and parse each one separately. This:
- Avoids building the full nested Map (lower peak memory)
- Starts yielding transactions immediately
- Still needs the full string downloaded, but parsing is
incremental
For a drive with 3 snapshots of ~15MB, ~10MB, ~5MB:
Before: download1 + parse1 + download2 + parse2 + download3 + parse3
After: download1 + [parse1 || download2] + [parse2 || download3] + parse3
* fix: retry gateway price fetch and handle AR cost failure gracefully PE-9103
- getPrice now retries 3 times with backoff instead of failing on a
single 502 from the gateway
- AR cost calculation in the upload modal falls back to zero estimate
on failure so the modal still opens with Turbo available instead of
crashing entirely
* fix: dateCreated null crash in file/folder entry refresh on first sync PE-9103
toEntryCompanion() on file/folder revision companions produces entry
companions with dateCreated = Value.absent() (schema has DEFAULT).
On first sync, oldestRevisionsCache is empty, so the fallback
.dateCreated.value returns null → JSNull crash on web.
Fixed by keeping a separate map of revision dateCreated values and
using those as the fallback instead of the entry companion's field.
Same bug pattern as the drive revision fix (line 1887), but for
files (line 1838) and folders (line 1871).
* perf: add drive owner to tx status and license gql queries PE-9126
Add an optional owners filter to the by-ID GraphQL queries that the
gateway struggles with (large ClickHouse row scans). Scoping by owner
lets the gateway prune its search space. The owner variable is nullable,
so when an owner can't be determined the query behaves exactly as before.
- TransactionStatuses, LicenseComposed, LicenseAssertions: add $owners
- getTransactionConfirmations / getLicenseComposed / getLicenseAssertions:
accept an optional owner and pass it through
- tx status (per-drive): scope to the drive's ownerAddress
- tx status (global): scope to the logged-in wallet's address
- licenses: scope to the best-known tx owner (revision/file owner)
Edge case: data txs of files pinned from other authors are owned by
those authors and won't match the owner filter; treated as best-effort.
* fix: re-query owner-mismatched txs unscoped to avoid false failures PE-9126
Owner-scoped getTransactionConfirmations leaves cross-owner txs (e.g. the
data tx of a file pinned from another author's upload) at -1, which the
sync status logic would turn into 'failed' after the pending timeout.
Add a fallback pass: after the selective owner-scoped query, re-query any
ids still unresolved (-1) without the owner filter to determine their true
status. The residual set is normally tiny, so the selectivity win of the
first pass is preserved while confirmed cross-owner txs are no longer
misclassified as failed.
* fix: resolve owner-mismatched txs via pin owner, not unscoped re-query PE-9126
The previous fallback re-queried unresolved (-1) txs without an owner
filter, which can re-trigger the gateway's expensive row scan that the
owner scoping was meant to avoid — and, because the call is wrapped in a
5s timeout, a slow/erroring fallback discards the whole batch's results
and makes no progress, repeating every sync.
Replace it with a selective approach: only re-query unresolved ids whose
real on-chain owner is known locally. Currently that's pinned files,
whose data tx is owned by the original uploader (pinnedDataOwnerAddress).
These are re-queried scoped to that owner, so every query stays selective.
Genuinely missing txs have no override and are left unresolved, handled by
the caller's existing pending/failed logic — no unscoped scan, no retry
storm on the common "not found yet" case.
- add pinnedFileRevisions drift query
- getTransactionConfirmations: ownerOverrides param replaces unscoped pass
- sync repo: build pin owner overrides and pass them through
* fix: make pinned-owner confirmation recovery best-effort PE-9126
The second (pinned-owner) pass merges into the already-populated first-pass
map, so its failure must never discard first-pass progress. Wrap it in a
try/catch and bound it with its own 3s timeout: even if the pin re-queries
error or stall, the confirmations resolved by the owner-scoped first pass
are still returned.
* perf: preserve resolved confirmations across a timeout via verified sink PE-9126
getTransactionConfirmations is wrapped in a 5s timeout by the sync status
update; on expiry it previously returned an empty map, discarding every
confirmation already resolved that cycle (the whole 5000-tx page).
Add an optional caller-owned verifiedSink that the method populates with
each resolved confirmation (>= 0 only) as queries complete. The callers pass
one in and, on timeout, fall back to it instead of an empty map — so work
done before the deadline (including a fully-completed first pass when only
the pinned-owner pass is slow) is applied rather than thrown away.
Because the sink holds only positive verifications and never the -1
"not found" placeholders, applying it after a partial/timed-out run only
upgrades txs to confirmed and never marks anything failed off incomplete data.
* fix: bound confirmation fan-out and use type-safe id filtering PE-9126
- Cap the per-chunk GraphQL fan-out in getTransactionConfirmations to
maxConcurrentDataFetches instead of launching every chunk at once, so a
large pending-tx page can't burst into a concurrent-retry storm against
the gateway (matches the throttling on the other gateway-heavy paths).
- Replace `sublist(...) as List<String>?` with `.whereType<String>().toList()`.
The cast threw a TypeError for the pinned-owner pass (whose ids list is
typed List<String?>), which the best-effort catch swallowed — silently
disabling pin recovery. The filter yields a real List<String> regardless
of input type.
* PE-9103: Fix empty explorer after drive attach (#2145)
* fix: empty explorer after drive attach PE-9103
Two bugs caused the explorer to show a permanent spinner after
attaching a drive (data was in DB but UI didn't update):
1. buildWhen guard in drive_detail_page.dart blocked BlocBuilder
rebuilds when SyncCubit was SyncInProgress. If DriveDetailCubit
emitted DriveDetailLoadSuccess during sync, the emission was
permanently skipped with no replay mechanism. Removed the sync
check — DriveDetailCubit already gates emissions via
waitCurrentSync() in the Rx.combineLatest3 callback.
2. startSyncForDrive silently aborted when a sync was in progress.
The .then(selectDrive) still fired, selecting a drive whose
content was never synced. Changed to await waitCurrentSync()
so the single-drive sync runs after the current sync finishes.
* perf: faster snapshot validation — skip fallback on 404, reduce timeouts PE-9103
Snapshot validation was slow on localhost (and anywhere Solana RPC is
unavailable): 2 HEAD retries × 10s timeout + GAR fallback via Solana
= 25+ seconds per snapshot. With 16 drives having multiple snapshots,
sync appeared stuck.
Changes:
- 1 HEAD attempt instead of 2 (fail fast, fall back to GQL)
- HEAD timeout 10s → 5s
- GAR list timeout 5s → 3s
- Skip GAR fallback entirely when primary returned 404 (snapshot
doesn't exist, no point trying other gateways via Solana RPC)
- Only try GAR fallback for transient errors (timeout, 5xx)
* fix: guard startSyncForDrive race after waitCurrentSync PE-9103
Two callers could both pass the SyncInProgress guard after
waitCurrentSync() returned, causing concurrent single-drive syncs.
Re-check state after wait — if another sync started, bail out.
---------
* fix: bump arweave-dart to v4.0.2 to fix file download crash
Pulls in the TransactionData null-field fix so downloads no longer crash
on web with "type 'JSNull' is not a subtype of type 'String'" when a
gateway returns null for optional tx fields (e.g. anchor on
turbo-gateway.com). Updates both the dependency and the override.
* chore(version): bump version to 2.84.0
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Ariel Melendez <ariel@ardrive.io>
Co-authored-by: arielmelendez <ariel.l.melendez@gmail.com>
* fix: refresh AR balance after Turbo topup from profile dropdown PE-9103 The profile card's topup button was not passing an onSuccess callback to showTurboTopupModal, so no balance refresh happened after a successful topup. Turbo credits update automatically on next dropdown open (fresh TurboBalanceCubit), but the AR balance was stale if the user paid with AR. Now passes onSuccess to call profileCubit.refreshBalance() after successful topup, matching the pattern already used in payment_method_selector_widget.dart. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: replace deprecated EquatableMixin with Equatable extends PE-9103 EquatableMixin is deprecated in favor of using Equatable directly. Changed IndexedItem from `with EquatableMixin` to `extends Equatable` with const constructor. This fixes the CI analyze failure in the ardrive_ui package. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Updates all four service endpoints in the dev (development) flavor to the new AR.IO testnet: - GraphQL gateway: ardrive.net -> ar-io.dev - data gateway: turbo-gateway.com -> ar-io.dev (label "AR.IO Testnet") - turbo upload: upload.ardrive.dev -> upload.services.ar-io.dev - turbo payment: payment.ardrive.dev -> payment.services.ar-io.dev All four verified reachable (HTTP 200, graphql answers a query). Staging and prod are unchanged; only local `--dart-define=environment=development` runs hit the testnet. Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
…2168) The typeform survey (pds-inc.typeform.com/ardrive) is no longer monitored. The post-share prompt was already disabled at the cubit level, leaving the sidebar link as the only live entry point. - remove Resources.surveyFeedbackFormUrl and openFeedbackSurveyUrl() - delete FeedbackSurveyCubit/State, FeedbackSurveyModal and their test - unwire openRemindMe() from the drive and file share dialogs, the router delegate listeners, and the main.dart provider - point Resources.helpCenterLink at https://ardrive.io/help (it was an unused constant aimed at /contact) and surface it as a "Help Center" link at the top of the support modal's Resources list, replacing the "Leave Feedback" row - drop the six now-unused l10n keys from all locales; weWontRemindYou is kept since prompt_to_snapshot_dialog still uses it Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
…d payment failures (#2166) * docs: implementation plan for turbo free-tier restriction PE-9132 10 MiB free pool per wallet, 105 KiB per-item eligibility, paid-only after exhaustion, credits never replenish free. Inventories the 15 silent-free posting paths, the current failure behavior on payment rejection, and phases the work: failure honesty (unblocked now), pool-aware eligibility (needs turbo API contract), surfacing UX. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat: phase 1 of turbo free-tier readiness - typed payment failures PE-9132 Prepares the client for the restricted free tier (10 MiB pool, 105 KiB per-item, paid-only after exhaustion). Policy-independent hardening: - decode HTTP 402 into TurboPaymentRequiredException and 429 into TurboRateLimitException in the app-side TurboUploadService (pure turboExceptionForStatusCode mapping, unit tested); the uploader package maps 402 to its existing UnderFundException and excludes 402/429 from its 8-attempt retry loops (retrying payment rejections multiplies load and metered usage) - rename (file/folder) failures now dismiss the progress dialog and show an honest error - payment-specific copy when the rejection was 402 (previously: spinner forever) - move gains a failure state and dialog handling, no longer emits Success after an error (removes the TODO admitting it), and is reordered to post-then-commit: data items are prepared and posted BEFORE the local database transaction, so a rejected move can no longer leave local state claiming a move the chain never saw - TurboUploadService fetches maxItemBytes from GET /v1/info once at construction (maxFreeItemSizeBytes, config fallback) - the server-driven per-item free threshold per the descoped plan; wiring it into UploadPaymentEvaluator is the next commit - implementation plan doc updated with the decision: no pool tracking, no balance-endpoint dependency, static free-tier messaging Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: analyzer errors in phase 1 (entities import, stub private member) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat: payment-aware failure UX across all metadata ops + server threshold PE-9132 Completes the free-tier client readiness: every operation that posts to Turbo now recognizes a payment rejection and shows one consistent, actionable message instead of a generic error or (previously) a hang. - new shared TurboPaymentRequired dialog (ArDriveStandardModalNew with a "Buy Credits" action into the existing top-up flow) as the single source of truth for the free-allowance-used-up UX; localized strings freeAllowanceUsedUpTitle/Description added to all six ARB files - new isTurboPaymentError() classifier; each op's failure state carries an isPaymentError flag set from the caught exception: rename, move, drive rename, folder create, drive create, hide/unhide, pin, license, ghost fixer - every corresponding form/dialog branches to the shared payment dialog on payment errors and keeps its existing generic error otherwise; folder-create, drive-rename and ghost-fixer gained the failure handling they previously lacked - UploadPaymentEvaluator now resolves the free per-item threshold from the server (TurboUploadService.maxFreeItemSizeBytes via /v1/info), falling back to allowedDataItemSizeForTurbo; wired through DI for the main upload flow (metadata/manifest paths keep the config fallback) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: const DriveRenameFailure constructor, drop unused hide import Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: address CodeRabbit review on free-tier UX PE-9132 - include isPaymentError in Equatable props on every failure state (drive/folder create, drive/folder/file rename, hide, ghost fixer, license) so a payment failure emitted after a generic one is not treated as an equal state and actually re-triggers the listener - license: preserve the original exception via logger.e before addError - license failure card: on a payment error the action becomes "Buy Credits" (opens the shared payment dialog) instead of retrying the same rejected operation - localize the generic metadata-op failure message via a shared actionFailedTryAgain key across all locales, replacing hardcoded English descriptions Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat: extend payment-failure UX to file uploads and all remaining post paths PE-9132 Audit found the metadata ops were covered but the highest-visibility upload surfaces were not. Closes those gaps. Cross-cutting fix — the two upload services throw different payment exceptions (app-side TurboPaymentRequiredException vs ardrive_uploader package UnderFundException, sometimes wrapped in UploadStrategyException). isTurboPaymentError() now recognizes all of them; ardrive_uploader exports its exceptions so the app can classify them. Newly covered: - main file upload and folder upload: UploadCubit classifies a payment rejection from the failed-task list into UploadErrors.turboPaymentRequired (UploadFailure gains isPaymentError-carrying props); the failure widget shows the Buy-Credits dialog instead of a "Re-Upload" that would 402 again - post-upload ArNS name assignment: wrapped in try/catch so a rejected name data item can no longer hang an already-successful upload (the name can be reassigned later) - snapshot creation, manifest creation (unwrapping the task-list error through ManifestCreationException), standalone ArNS assignment, bulk import (unwrapping FileMetadataUploadException.originalError), and single/multi thumbnail creation all classify payment errors and show the shared dialog Deferred (documented): private-drive migration and login verification posts — tiny, effectively always-free signature items where a payment dialog mid-flow would be worse UX than the near-impossible failure. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: resolve analyzer issues from upload-surface coverage - hide the ardrive_uploader package's TurboUploadTimeoutException / TurboRateLimitException in upload_cubit (they collide with the app-side classes of the same name now that the package exports its exceptions); the app-side types are the ones _emitError intends - add missing app_localizations imports to assign_name and thumbnail creation modals - drop the unused shared-dialog import in upload_form (its failure widget builds the payment modal inline) - const the thumbnail error state constructors - remove now-redundant direct exceptions.dart imports in three ardrive_uploader files (the barrel provides them) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: close the two remaining payment-UX holes found in verification PE-9132 Verification audit found my earlier thumbnail/bulk-import fixes were in the wrong place — the failure never reached the bloc catch I'd wired. - thumbnail_repository: the upload controller's onError only logged and never completed the completer, so a 402 (which fires onError, not onDone) hung single AND multi thumbnail creation in Loading forever. onError now errors the completer, unwrapping the task list via anyTaskIsTurboPaymentError (previously dead) into the typed exception so the blocs classify it. The onDone body (which posts the thumbnail metadata data item) is also guarded so a payment rejection there errors the completer instead of hanging. - bulk_import_bloc: the actual import-execution catch swallowed the error and emitted a const BulkImportError (isPaymentError always false), so a 402 during real bulk import showed a generic dialog. The error is now captured and classified via _isBulkImportPaymentError (unwrapping FileMetadataUploadException.originalError). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: surface payment errors swallowed by the shared WorkerPool PE-9132 Final verification found multi-thumbnail and bulk import still swallowed 402s: the ardrive_utils WorkerPool caught task exceptions in Worker._execute and passed only the task (not the exception) to onWorkerError, so the pool completed normally and the awaiting bloc never saw the failure. - WorkerPool.onWorkerError now receives the exception (Function(T, Object)) - multi thumbnail: onWorkerError captures a payment rejection into a flag and, after onAllTasksCompleted, emits MultiThumbnailCreationError( isPaymentError: true) instead of reporting completion - bulk import: FileImportFailure now preserves originalError; the worker records failures into BulkImportResult (was: only logged); the bloc captures the result and classifies the terminal error from the failures' originalError as well as the outer catch - fixes a scope bug: importResult is declared before the try so it is visible in the post-catch classification This closes the last two surfaces; single-thumbnail and the other seven were already verified. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * nit: const MultiThumbnailCreationError emit Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat: config-based direct-to-network fallback for private-drive migration PE-9132 Private-drive migration was the one ArFS metadata op with no L1 fallback — it posted the drive-signature data item only via Turbo. It now uses the same config-based branch every other op has: post via Turbo when useTurboUpload is enabled, otherwise wrap the already-signed data item in a DataBundle and post directly to the network via ArweaveService.postTx (pays AR from the wallet). Closes the deferred migration item from the free-tier work. Note this is a config switch (useTurboUpload=false), not an automatic on-402 fallback; migration signature items are ~1 KB and effectively always free-eligible, so pool exhaustion here is negligible. Login wallet-creation verification posts are intentionally NOT given this branch: the wallet is brand-new with no AR during creation, and the ETH path involves cross-chain signing — an L1 fallback there would fail, not help. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: trigger payment dialog from BlocConsumer listener, not builder PE-9132 CodeRabbit (Critical/Major) caught the payment dialog being triggered from inside the builder via addPostFrameCallback in five modals. A builder can run many times for the same state (repaints, resizes, ancestor rebuilds), each queuing another dialog → duplicate/stacked modals. Moved every trigger to the BlocConsumer listener, which fires once per state transition: snapshot, single + multi thumbnail, standalone ArNS assign, and manifest (the last two CodeRabbit didn't flag but had the identical bug). Builders now render only the static fallback content. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat: only promise a free upload when the wallet allowance covers it PE-9132 Turbo's new GET /v1/account/free?address=<wallet> reports how many free bytes a wallet has left. Until now isFreeThanksToTurbo was derived purely from item size, so a user whose free pool was used up was shown "this transaction is free thanks to Turbo", had the payment method selector hidden, and then hit a 402 mid-upload. The 402 handling recovered gracefully but the promise should never have been made. - add TurboFreeAllowance, modelling unlimited / limited / disabled / unknown, plus covers() and isExhaustedFor() - add PaymentService.getFreeAllowance and a non-throwing TurboBalanceRetriever.getFreeAllowance wrapper - require both size eligibility and allowance coverage before treating an upload as free, in the entity, bundle and snapshot paths - explain the switch to a payment selector when the allowance ran out, instead of silently swapping the UI - fetch the allowance per preparation, unlike the static item-size limit The value is advisory: Turbo's response stays the authority on whether an upload was free, and an unreachable endpoint falls back to the previous size-only behaviour rather than telling a user with allowance left to pay. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PsV2WFHZMGD9DUfX65cxuW * fix: close two free-allowance gaps in the manifest upload paths PE-9132 An audit of every surface that promises a free upload found two the first pass missed, both in the manifest flow: - UploadManifestModel.freeThanksToTurbo was still derived from item size alone, in both prepareManifestUpload and prepareUploadPlanAndCostEstimates. This is worse than a cosmetic false promise: when every manifest is marked free, UploadCubit skips the payment method selection entirely, so an exhausted wallet went straight to an upload that 402s. - create_manifest_form showed the payment options with no explanation when the allowance ran out, unlike the upload and snapshot dialogs. Also stores arDriveUploadManager, until now a required UploadCubit constructor parameter that was never assigned to a field, so the cubit can reach the allowance without a new dependency. Adds a regression test for the manifest path, verified to fail without the fix, plus the missing allowance stubs for the shared setUpAll mocks. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PsV2WFHZMGD9DUfX65cxuW * refactor: model free-tier state as one value and render it in one widget PE-9132 The free-tier UX was carried by two parallel booleans (isFreeThanksToTurbo and isFreeAllowanceExhausted) threaded through four state classes, where "free" and "allowance used up" could both be true, and by six hand-rolled conditionals across three dialogs. That duplication is how the manifest form ended up promising "free" without ever explaining what happened when it stopped being free. - add FreeUploadStatus (free / allowanceUsedUp / notEligible) and a freeUploadStatusFor helper holding the two rules in one place - store that single value in UploadPaymentInfo, UploadPaymentMethodInfo, ConfirmingSnapshotCreation and CreateManifestUploadReview, keeping the existing booleans as derived getters so no consumer changes - add TurboFreeStatusMessage, the one widget that renders the one status line, collapsing entirely when there is nothing to say - tighten the used-up copy to lead with the fact Behaviour is unchanged: same 720 tests pass, including the ~35 existing isFreeUploadPossibleUsingTurbo assertions untouched, and the manifest regression test still fails when the underlying fix is reverted. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PsV2WFHZMGD9DUfX65cxuW * fix: address CodeRabbit review on the free-tier work PE-9132 Three of the five findings were valid: - manifest free-eligibility used the static config item-size limit while the shared upload path prefers Turbo's /v1/info value. Adds ArDriveUploadPreparationManager.getMaxFreeItemBytes(), alongside the existing getFreeAllowance(), and uses it at both manifest sites. This also removes UploadCubit's last read of the global configService from main.dart. - getUploadPaymentInfoForEntities passed the item size as its own size limit, so the free-tier per-item cap was vacuously satisfied (x <= x). Passes the actual limit. No practical change for metadata items, which are far below it, but the cap is now real. - the snapshot dialog is shown with barrierDismissible: false and did not pop itself before opening the payment dialog, leaving an invisible undismissable barrier over the app once that dialog was closed. It now pops first, like create_manifest_form already did. Declined, with reasons: - gating snapshot free-status on appConfig.useTurboUpload: free uploads deliberately bypass that flag ("Even if this feature flag is off, it will be possible to upload using turbo for free files"), and gating it would restore the false "free" promise this PR exists to remove. - catching getFreeAllowance exceptions in the snapshot cubit: the retriever wrapper already catches everything and returns unknown, and auth.currentUser is read earlier in the same try by _computeBalanceEstimate. - popping the route in multi_thumbnail_creation_modal: it is an OverlayEntry, not a route, so Navigator.pop would dismiss the drive page underneath it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PsV2WFHZMGD9DUfX65cxuW * fix: check free allowance regardless of turbo flag; dismiss thumbnail overlay PE-9132 Resolves the two CodeRabbit findings I had previously declined, correctly: - The free allowance was fetched only when the useTurboUpload flag was on, but free uploads deliberately bypass that flag. So with the flag off a small item still uploaded via Turbo yet was promised "free" without ever checking the allowance — the exact bug this PR removes, hidden behind a flag — and the snapshot path (which checks unconditionally) disagreed. CodeRabbit proposed making snapshot honor the flag; that is the wrong direction, as it would send free-eligible items down the paid path against the documented intent. Instead getFreeAllowance is now unconditional, so free-ness is always verified. The paid-turbo gate on _getTurboBalance is unchanged. No runtime effect today (useTurboUpload is true in all flavors). - The multi-thumbnail modal is an OverlayEntry, not a route, so Navigator.pop would have dismissed the drive page underneath — which is why popping was declined. But the overlay was still left behind the payment dialog. It now dismisses through its own CloseMultiThumbnailCreation event, the mechanism the modal already uses for closing. Adds an assertion that the allowance is consulted even with the flag off. Endpoint host confirmed empirically: GET payment.ardrive.io/v1/account/free returns 200 {"bytesRemaining":10485760} (10 MiB), and upload.ardrive.io 404s, so turboPaymentUri is the correct host. An unknown address returns the full allowance rather than 404, so new wallets correctly read as free. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PsV2WFHZMGD9DUfX65cxuW * feat: distinguish an upload exceeding the free allowance from it being used up PE-9132 A bulk/folder upload of small files whose total is larger than the wallet's remaining free pool was labelled "Free allowance used up" — wrong when the user still has most of their allowance and the upload simply exceeds it. Adds FreeUploadStatus.exceedsAllowance, distinct from allowanceUsedUp, chosen when the wallet still has a positive allowance but the upload is bigger than it. TurboFreeStatusMessage now shows an honest note for that case: "This upload exceeds your free allowance and will need Credits or AR." Deliberately robust rather than precise. The message states the fact (upload > remaining) and the outcome (needs payment) without predicting how many bytes end up free, because the client cannot know that: Turbo applies the free tier server-side, does not expose whether it bills per-item or per-bundle, and enforces a second per-IP pool that /v1/account/free does not report. So it is true whether Turbo frees part of the upload or none of it, and the 402 remains the authority on what is actually charged. This also corrects a stale comment that asserted all-or-nothing billing as fact. Behaviour is otherwise unchanged: exceedsAllowance is not free, so the payment selector still shows and upload-method selection is untouched. Single item / snapshot / manifest paths are unaffected in practice. The one existing assertion that expected "used up" for a partial multi-item upload is updated to expect the new, more accurate status; the derived getters and the widget's now-exhaustive switch keep every other consumer compiling unchanged. Adds unit coverage for freeUploadStatusFor (including boundaries and fail-open) and a widget test asserting each status renders the right message, both verified to fail under a collapsing mutation. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PsV2WFHZMGD9DUfX65cxuW --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: vilenarios <philip.mataras@gmail.com>
…oad prep PE-9132 (#2169) getFreeAllowance() (added in #2166) was awaited late in each upload-prep method, after the balance and cost round-trips that already run serially. The allowance call is independent — it only needs the wallet — so it was adding an extra sequential Turbo round-trip to every upload-modal open, and doubling the worst-case wait when payment.ardrive.io is unavailable (two 8s timeouts back to back instead of one) before it fails open. Start the future up front in both getUploadPaymentInfoForEntities and getUploadPaymentInfoForUploadPlans and await it where the free status is computed, so it overlaps the balance, size and cost work. Timing only — no value changes; getFreeAllowance is a non-throwing wrapper, so the in-flight future cannot become an unhandled rejection. 733 tests unchanged. Claude-Session: https://claude.ai/code/session_01PsV2WFHZMGD9DUfX65cxuW Co-authored-by: vilenarios <philip.mataras@gmail.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
…2170) * fix: keep modal actions reachable on short/mobile screens; congestion popup PE-9150 User report: the Arweave network-congestion popup covers/hides its buttons on mobile. Root cause is vertical, not width: ArDriveStandardModalNew constrains maxWidth to the viewport but sets no maxHeight and its body does not scroll, so a modal taller than the screen (the congestion warning wraps to many lines on narrow devices) pushes its action buttons off the bottom. barrierDismissible: false then leaves the user trapped with no reachable button. - add an opt-in `scrollableContent` flag to ArDriveStandardModalNew: when set, the body is bounded to 80% of the viewport height and scrolls, so actions can never be clipped off-screen. Default false — the ~40 existing modals are unchanged, which matters because several pass content with an Expanded/ ListView that would break under an unconditional scroll view. - opt the congestion warning into scrollableContent, and make it barrierDismissible (tap-outside == "Try Later", so it is never a trap). Adds widget tests: the action is reachable (ensureVisible + tap) on a 360x480 screen with tall content, and default (non-scrollable) behaviour is unchanged. ardrive_ui suite (52) and app suite (733) both green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PsV2WFHZMGD9DUfX65cxuW * test: scope modal scroll-view assertions to the modal subtree PE-9150 Addresses CodeRabbit: find.byType(SingleChildScrollView).findsWidgets could pass on an unrelated ancestor scroll view. Scope both assertions to a descendant of ArDriveStandardModalNew — present on the scrollableContent branch, absent by default — so the two tests genuinely distinguish the branches (and prove the default path adds no scroll view for the other modals). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PsV2WFHZMGD9DUfX65cxuW --------- Co-authored-by: vilenarios <philip.mataras@gmail.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: vilenarios <philip.mataras@gmail.com>
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
* perf: parallelize upload preparation — conflict detection, cost estimation, plans PE-9103 - parallelize conflict detection: check all files concurrently via Future.wait instead of sequential DB query per file - pre-compute all file lengths in parallel at start of preparation, cache in _fileLengthCache to avoid redundant sequential reads - parallelize cost estimation: turbo balance + AR sizes fetched in parallel, then AR and Turbo cost calculations run concurrently - parallelize upload plan creation: AR and Turbo plans created simultaneously via Future.wait instead of sequentially - remove duplicate wallet mismatch check (called twice in prepareUploadPlanAndCostEstimates) - skip ANT records re-fetch if already populated from initial fire-and-forget fetch - reduce 100ms artificial UI delay to Duration.zero (just yields to event loop for state update) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: wrap FutureOr<int> in async closure for Future.wait PE-9103 IOFile.length returns FutureOr<int>, not Future<int>. Future.wait requires Future<T>, so wrap in async closure. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: check _ants.isEmpty instead of null (non-nullable list) PE-9103 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use try/catch instead of catchError for sync-throw safety PE-9103 Mocktail's thenThrow throws synchronously before returning a Future, so .catchError never gets attached. Wrapping in async IIFE with try/catch handles both sync and async throws correctly. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: vilenarios <philip.mataras@gmail.com>
#2174) * fix: read Turbo free-item cap from /v1/info freeTier, not stale config PE-9132 refreshMaxItemBytes read a top-level `maxItemBytes` key that /v1/info does not return (the per-item cap is nested under `freeTier.maxItemBytes`, with a top-level `freeUploadLimitBytes` mirror). So the server value was never applied and the app silently fell back to the config value — enforcing a free cap of 100000 bytes (97.66 KiB) instead of Turbo's actual 107520 (105 KiB). Files between ~97.7 KiB and 105 KiB were charged when Turbo would have made them free. Now reads freeTier.maxItemBytes, then top-level freeUploadLimitBytes, then maxItemBytes (defensive), accepting an int or num and staying on the config fallback when absent or on error. Adds unit tests covering the real payload shape, the fallbacks, and fail-safe behaviour. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PsV2WFHZMGD9DUfX65cxuW * fix: validate each /v1/info cap candidate before choosing one PE-9132 From CodeRabbit's review. Two real defects, one recommendation declined. - the candidates were chosen with `??` and validated afterwards, so any non-null preferred value won even when it was unusable. A payload with `freeTier.maxItemBytes: "oops"` and a perfectly good `freeUploadLimitBytes` beside it fell back to the stale config value this fix exists to stop using. Each candidate is now validated first; the first *valid* one wins, in the same priority order. - a fractional value passed `> 0` and was then truncated: `0.5` became a cap of **0 bytes**, which is not the same as no cap - `maxFreeItemSizeBytes` returns it in preference to the configured fallback, so nothing at all would have qualified for a free upload. Fractional values are now rejected rather than truncated, and infinity and NaN are rejected before `toInt()`, which throws on them. Declined: resetting `_serverMaxItemBytes` to null on an error or an unusable payload. That would make a transient /v1/info failure silently reintroduce the stale cap this PR removes, and a cap the server actually reported is better information than the config value. Two tests pin the choice - a later failure, and a later capless payload, both keep the last good cap. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: vilenarios <philip.mataras@gmail.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
#2177) Everything here was verified against the code rather than inferred. - config is resolved in layers, and editing assets/config/*.json does nothing for anyone who has already run the app unless configVersion is also bumped. This is the most common "my config change did nothing" trap. - identity is not Arweave-only: Ethereum and Solana providers exist alongside JWK and ArConnect, so auth and signing paths cannot assume a wallet type. - not every feature lives under blocs/; several are top-level directories bundling their own bloc, repository and UI. lib/l11n/ is not localization. - upload behavior mostly lives in packages/ardrive_uploader, not lib/blocs/upload, and the cipher is size-dependent at 100 MiB. - sync lives in lib/sync with its own repository, snapshot validation and a deliberate failure simulator; ghost folders are a normal state. - share link shapes are permanent public API, and the hash URL strategy is what keeps the key out of anything a server sees. - drift migration fixtures stop at v19 while schemaVersion is 29, so a bump is not automatically covered by migration tests. Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* chore: vendor pdf.js and add the pdfx dependency PE-9200 Split out of PE-9200 so the feature PR fits under CodeRabbit's 100 file limit. This carries only third-party assets and the dependency change; the code that uses them is in the feature PR, which is based on this branch. - `pdfx: 2.6.0`, pinned exactly. 2.7.0+ require `meta ^1.15.0` against the `meta 1.11.0` Flutter 3.19.6 pins exactly, and 2.9.x additionally require Flutter >= 3.24.0, so this is the newest version that resolves here. The vendored pdf.js is the exact build it targets - the two move together. - pdf.js 2.12.313 vendored under `web/js/pdfjs/`, never a CDN: this build ships to Arweave, where it is permanent and could never be patched, and a CDN script also breaks under the strict CSP an AR.IO gateway serves. - the CJK cmaps are deliberately not included - 1.3 MB for a narrow class of older PDFs. See the note in `pdfjs_config.js`. - `isEvalSupported` is patched to default false in the vendored build. CVE-2024-4367: this version compiles Type1 glyph paths with `new Function` from a FontMatrix it does not type-check, and the PDFs reaching it are attacker-chosen. It is patched in the file rather than passed as an option because pdfx merges its option map with `Map.addAll` over a raw JS object, which throws under dart2js. Re-apply on any rebuild, or move to pdf.js >= 4.2.67 once a Flutter upgrade lets pdfx target it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test: guard the vendored pdf.js CVE patch, and keep our own config reviewable PE-9200 Both from CodeRabbit on #2176. - the `!web/js/pdfjs/**` filter also excluded `pdfjs_config.js`, which is our file living in a vendored directory - it sets the worker path and documents the CVE mitigation. Now only the upstream bundles beside it are filtered. - nothing stopped the vendored bundle drifting back to an unpatched `isEvalSupported=!0` on a future re-vendor. The note in `pdfjs_config.js` asked the next person to remember; this asks CI instead. Verified by reverting the patch and watching the test fail with the message that tells you how to fix it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
… and a recipient page built for strangers (#2175) * chore: vendor pdf.js and add the pdfx dependency PE-9200 Split out of PE-9200 so the feature PR fits under CodeRabbit's 100 file limit. This carries only third-party assets and the dependency change; the code that uses them is in the feature PR, which is based on this branch. - `pdfx: 2.6.0`, pinned exactly. 2.7.0+ require `meta ^1.15.0` against the `meta 1.11.0` Flutter 3.19.6 pins exactly, and 2.9.x additionally require Flutter >= 3.24.0, so this is the newest version that resolves here. The vendored pdf.js is the exact build it targets - the two move together. - pdf.js 2.12.313 vendored under `web/js/pdfjs/`, never a CDN: this build ships to Arweave, where it is permanent and could never be patched, and a CDN script also breaks under the strict CSP an AR.IO gateway serves. - the CJK cmaps are deliberately not included - 1.3 MB for a narrow class of older PDFs. See the note in `pdfjs_config.js`. - `isEvalSupported` is patched to default false in the vendored build. CVE-2024-4367: this version compiles Type1 glyph paths with `new Function` from a FontMatrix it does not type-check, and the PDFs reaching it are attacker-chosen. It is patched in the file rather than passed as an option because pdfx merges its option map with `Map.addAll` over a raw JS object, which throws under dart2js. Re-apply on any rebuild, or move to pdf.js >= 4.2.67 once a Flutter upgrade lets pdfx target it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix: harden the shared file view and download paths PE-9200 - download the newest revision on the shared file panel, not the oldest. The revision list is newest-first, so both download buttons' `.last` served the original bytes while the header showed the current name. Now behind a documented `latestRevision` getter with a regression test. - add `SharedFileLoadFailure` with a working Retry, so a network or gateway error no longer leaves an infinite spinner - distinguish a wrong key from a missing file; a rejected key now keeps the key field on screen instead of reading "file does not exist" - carry a damaged-key signal from the route parser through to the cubit, and guard the base64 decode on both the file and drive routes - guard `Uri.parse`/`queryParameters` so a bad percent-escape cannot throw during route parsing - stop over-limit images from fetching (a missing `return`) and cap public images, which were previously uncapped - route preview fetches through the gateway fallback waterfall - render an explicit "too large to preview" state, and stop the preview widget's `default` branch from blind-casting every unhandled state to `FsEntryPreviewVideo` - translate download, preview and shared-file strings into es/hi/ja/zh/ zh-HK (machine translations, pending native-speaker review) Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat: self-describing v2 share links and a recipient page built around them PE-9200 A share link now carries what the recipient's page needs - data and metadata transaction ids, owner, name, size, type, cipher and IV - so a complete link paints with zero blocking GraphQL. Every field is independently optional and degrades on its own; a field the link never carried, or that arrived mangled and was dropped, falls back to exactly the lookup a v1 link would have done. v1 links resolve exactly as they always have. The key is no longer in the link by default. It travels out of band as a separate artifact, and the share dialog hands over link and key separately with copy that pushes them down different channels. Putting the key in the link is an opt-in, off by default, stated in plain language. - new `shared_file_link.dart`: the payload model, the one place a parameter name is spelled, key-source precedence, and shape validation before any crypto is attempted - the payload is an optimization, never a source of truth: everything it asserts is checked against the file's own record in the background, and a link whose claims disagree with the record loses - the resolved revision becomes the target, for display and for download - recipient page rebuilt as the state machine of the design doc: locked, resolving, ready, not found, network trouble, damaged link - each with copy written for someone who has never heard of ArDrive, and no transaction ids until a drawer is opened - a not-yet-propagated upload retries instead of claiming the file is missing - an access key can be remembered for the tab in sessionStorage, opt-in, never localStorage - freshness offers a newer version and never swaps bytes underneath a download - version history loads when it is opened, not during first paint - `DetailsPanel` loses the `isSharePage` flag and the shared-file inputs that only ever arrived with it; the recipient page is its own widget tree now - 51 new strings localized into all six locales (machine translations, pending native-speaker review) Three bugs the compiler and the suite found once this ran together: - a fragment-placed key on the hash route was percent-encoded into the last query parameter's value, putting key material in the query. A URL has one fragment and the hash route has already spent it, so the generator now refuses; fragment placement becomes real with Phase 3's path routes. - `ArDriveButton.isDisabled` styles a button without stopping it from reporting a tap, so a double press moved the download target twice. The guard now lives in the handler. - the freshness notice put an `Expanded` message beside a button sized to its own sentence-long label, overflowing on a 400px card. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat: resumable-decrypt foundation, chain-based integrity, and preview coverage PE-9200 Two independent pieces of the download and preview work. Neither changes the download path yet - that is the next step, which consumes both. Crypto foundation (packages/ardrive_crypto): - AES-CTR streamed decryption takes a start offset, so a download can resume mid-file. The plan said offsets must align to the 256 KiB chunk; they do not. 16 bytes is the only real constraint - the counter is a pure function of the block index, so the chunk grid restarts coherently from any block boundary. Resume now wastes at most 15 bytes instead of 256 KiB. Tested byte-identical against decrypt-then-slice at nine offsets, deliberately including ones that are not chunk-aligned. - the counter field is 4 bytes, so streamed decryption is capped at 64 GiB; beyond that is now an ArgumentError rather than a failure inside hex.decode - GCM's MAC trim became offset-relative, which the plan did not mention and which would otherwise truncate a resumed stream by exactly the offset - new StreamedDataItemVerifier: computes the ANS-104 deep hash incrementally over the ciphertext as it streams and checks it against the item's signature, so integrity costs no extra round trip. Three verdicts, and the distinction matters: `notVerified` is not `failed`. A resumed download, a missing field, or an Ethereum/Solana-signed item reports "could not verify" - only bytes that contradict the signature report failure. - deleted the mothballed authenticate.dart, superseded by the above - unauthenticated GCM streaming is now reached through an explicitly named entry point with a kill switch, ready to be turned off once the download path stops using it Integrity inputs (GraphQL): - SingleTransaction gains signature, anchor, recipient and owner's key. There is no `target` field - the schema calls it `recipient`. `owner { key }` had to be aliased: TransactionCommon already selects `owner { address }`, and Artemis puts fragment fields on a mixin without deduplicating against the class, so selecting it twice makes the *generated* file uncompilable. - an empty `anchor` is ambiguous across gateways, so a failed check must read as "could not verify", never "tampered" Preview coverage: - recipient thumbnails render, through the gateway fallback rather than one hardcoded gateway. Private thumbnails decrypt: ArFS encrypts them with the file key, which is exactly the access key a recipient holds. - private video and audio preview on web, decrypted in memory to an object URL that only ever reaches <audio>/<video> - never a frame - and revoked when the preview closes - PDFs preview at last. The dead `case 'pdf'` never fired because a PDF's MIME type starts with `application`. They open on the gateway's own origin in a new tab, so no untrusted bytes become script-capable on ours. - a pinned video in a private drive is no longer permanently unavailable Also: the WebCrypto capability probe imported a key and called it a day, but importRawKey never touches BoringSSL, so the guarded tests ran anyway and failed. It now performs a real encryption, so they skip cleanly where the native library is unavailable - which includes CI. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat: authenticate private downloads, resume large ones, render PDFs inline PE-9200 Closes F24. On web, no private download was cryptographically authenticated at any size: both ciphers were routed through a streaming decryptor whose GCM branch trims the MAC and never checks it. Only mobile GCM took the buffered, verified path. Download path, split by cipher: - GCM (every private file under 100 MiB) now buffers, verifies the MAC, and only then writes. On every platform, not just mobile. A tampered ciphertext produces a typed DownloadIntegrityException and zero bytes on disk. - AesGcmStream is gone from the download path - the only reference left in the app is the line that disarms it - and unauthenticated GCM decryption is switched off, so any accidental reuse throws. - CTR (every private file at or above 100 MiB) streams, and resumes from the 16-byte-aligned plaintext offset after a mid-stream failure, producing byte-identical output to an uninterrupted download. - integrity for CTR and public files now comes from the chain: the ANS-104 deep hash is computed over the ciphertext as it streams and checked against the data item's signature. The verdict is advisory and never delays the save. A resumed download reports "could not verify" rather than guessing, and an Ethereum- or Solana-signed item does too - only bytes that contradict the signature report failure. Range support is not universal, which the resume path has to survive: turbo-gateway.com answers 206, arweave.net ignores the header and returns the whole body. The client branches on the response, never on having sent the header - a 200 body starts at byte zero, and writing it at the resume offset would silently corrupt the file - and a gateway serving a different range than asked for is caught by preferring content-range over the request. Also: - new DownloadPolicy: one home for the size gates, and the single call site of the Safari constant. Safari has two different ceilings depending on the flow (1 GiB single file, 500 MiB bundle); both are preserved and the discrepancy is written down rather than silently reconciled. - PDFs render inline, rasterized to images by pdfx and painted as Flutter widgets - never a frame, never a blob URL, scripting off - so a PDF's embedded JavaScript cannot run and private PDFs preview for the first time. pdf.js is vendored under web/js/pdfjs rather than fetched from a CDN: this app deploys to Arweave, where a build is permanent, and a CDN script tag would bake an external dependency into something that can never be patched. - GraphQL retries drop from 8 attempts to 3. Exponential backoff turned 8 into roughly 25 seconds spent on an endpoint that is usually not coming back, before the fallback endpoint was tried at all. Retrying a gateway that is down does not recover a query; switching endpoints does. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat: path-route capability, legacy boot shim, and a viewer for any transaction PE-9200 Path routing is built and switched OFF. Flipping it needs SPA rewrites that do not exist yet - without them every direct navigation and every refresh 404s - so `kAppUrlStrategy` defaults to hash routing and today's behavior is byte-identical. Enabling it is two lines, and a test fails if only one moves. - `/share/{fileId}` and `/view/{txId}` parse under both strategies, reusing the v2 payload and key precedence rather than duplicating them. Under path routing the key finally lives in a real `#k=` fragment; the guard that refuses fragment placement is now conditional on the strategy instead of unconditional. - `PathUrlStrategy` is constructed with `includeHash: true`. Flutter's `usePathUrlStrategy()` helper defaults it to false, which would have made `#k=` and every legacy `#/file/...` link invisible to the router. - a boot shim rewrites legacy hash links to `/share/{id}#k=...` via `replaceState`, which issues no request, so the key never leaves the browser during migration. It is gated on the same flag: rewriting a working link into a path that 404s would be worse than leaving it alone. - legacy links keep resolving forever under both strategies. The generalized viewer answers "why send an app link instead of a gateway link": any Arweave transaction gets a filename, a preview and a download. Its rule is that only bytes unlock an inline renderer, and a declared content type can only ever restrict. HTML dressed as image/png is still sandboxed; SVG labelled text/plain is too. Script-capable content is never rendered here - it is offered on the gateway's own per-transaction base32 subdomain, a different origin, inside a frame sandboxed without allow-same-origin. That matters because a recipient's access key can live in sessionStorage. `SandboxedTransactionView` takes a URL and never bytes, which makes srcdoc, blob-HTML and innerHtml structurally impossible rather than merely absent. Also: `openUrl` now accepts only http and https. Two existing callers pass URLs that are not constants - the remote-config announcement banner and the localStorage-overridable data gateway - and are now protected. The sandbox subdomain computation is confirmed against a live gateway: the base32 this derives for a real transaction is byte-for-byte the subdomain arweave.net redirects that transaction to. Note for whoever enables path routing: `PathUrlStrategy` throws without a `<base>` element, and adding `base href="/"` breaks serving from a sub-path, which is how gateway `/{manifestTxId}/...` URLs work. Path routing is only viable from an origin root. The full infrastructure list is in the plan. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix: remediate the adversarial review of this branch PE-9200 Three reviews - security, regression, test quality - over the whole branch. Everything below was confirmed against the code before it was changed. Security: - the vendored pdf.js 2.12.313 is vulnerable to CVE-2024-4367: it compiles Type1 glyph paths with `new Function(...)` from a FontMatrix it does not type-check, so a crafted PDF ran arbitrary JavaScript on the app origin - the origin holding a recipient's access key. Reachable from /view, from a shared link, and from the ordinary explorer preview. `isEvalSupported` now defaults to false in the vendored build. It is patched there rather than passed as a param because pdfx merges that param map into getDocument with `Map.addAll` over a raw JS object, which throws under dart2js - the tidier fix would have broken PDF rendering instead of securing it. - plausible.js sent `location.href`, fragment included, to plausible.io on load. On a key-in-link share that string *is* the file key, so a recipient handed the decryption key to a third party before Flutter booted - while the UI told them the key never leaves their device. The tag is gone; the Dart tracker already POSTs a URL synthesised from an enum. - the "Verified" badge was forgeable: the link's `mtx` was fetched by id with no owner constraint, so anyone could post metadata claiming a real file id and get a green check over their own bytes. The metadata's author must now match the file's first writer. No pre-paint round trip was added - the owner comes from the owner-scoped lookup freshness already made. - a link asserting nothing no longer verifies vacuously. - `n` accepted Unicode bidi overrides, and `n` becomes the saved filename: `?n=Q3-Report<U+202E>fdp.exe` displayed as `Q3-Reportexe.pdf` in the save dialog. Such a name is dropped now, in both the share and /view paths. - seven sites logged a `FormatException` whose `source` could be a key, into a log store the user can export by email. They log the reason now. - added a strict-origin referrer policy, which path routing will need. Regressions this branch introduced, and one it inherited: - mobile downloads never completed. The mobile saver blocks until the downloader says whether to keep what it wrote, and the downloader only answered on cancel - so progress sat at 100% forever, and pressing Cancel deleted the finished file. This branch deleted the one mobile path that worked; on dev the same hang covered every private CTR download, every private manifest, and every shared-link download. The downloader now answers when the source drains, so all of them complete. - AES-GCM files over 100 MiB were hard-rejected. "Bounded by construction" only holds for files this uploader wrote after the GCM/CTR split; legacy and ardrive-cli files are GCM at any size. They stream again, with the integrity caveat stated rather than assumed. - gateway failures reported as "Download cancelled", after prompting for a save location. The first response is opened eagerly again, so a 404 or a rate limit throws before the picker opens. - the resume loop could retry forever against a gateway dribbling one byte per connection. Total ceiling plus backoff. - GraphQL retries: 8 attempts is ~51s of backoff, not the ~25s claimed in the previous commit, and 3 was too sharp - the fallback endpoint only arms for 429 and 5xx, so a dropped socket or CORS failure has no second endpoint and that budget is its only resilience. Five. Tests that passed without proving anything: - nothing asserted which URL the sandbox frame is pointed at, so pointing every transaction at one shared origin kept all seven files green - "never localStorage" was asserted nowhere: every test injected an in-memory fake, so swapping the default kept ten tests green - the splice guard's source half had no coverage - the fake computed the 200-to-a-Range behavior itself, so neutering the real code still passed - nothing checked the integrity verifier is fed ciphertext, and ciphertext and plaintext are the same length - the media object-URL fake discarded its bytes, so feeding a `<video>` ciphertext instead of plaintext was invisible - a test greped for an ASCII apostrophe against a U+2019 string, and no test anywhere pumped the mismatch state - `pin`, `hid`, `in` and `thn` appeared in no assertion: renaming one would have silently downgraded every pinned link in the wild - one test pinned unsafe behavior, asserting a drive key belongs in a query Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs: record that PDFs render inline, and why that keeps the §4.3 invariant PE-9200 The rasteriser turns pages into images rendered as Flutter widgets, so no untrusted byte becomes script-capable content on the app origin. Leaving the doc forbidding what the code now does would have been a landmine. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix: open the save picker before downloading, not after PE-9200 Chrome only allows `window.showSaveFilePicker` while the click that started the download still counts as transient user activation - a few seconds. On `dev` the buffered AES-GCM branch was gated `if (AppPlatform.isMobile)`, so web always streamed and the picker always opened one gateway-open after the click. This branch removed that gate to authenticate GCM everywhere, which moved the picker to the *end* of a full download: on Chrome, every private file under 100 MiB downloaded completely, verified its MAC, and then failed - reported as "Download cancelled", because a rejected picker surfaces as `saveResult: false` with no source error. On the web the saver is now handed a file whose read stream performs the download and the MAC check, so the picker opens on the first turn and the bytes arrive later. Off the web there is no picker and no gesture to race, so nothing is opened until the MAC has passed and the stronger guarantee stands: a unified ordering would have left a 0-byte file in Downloads on every failed GCM download, network failures included, for no benefit. The cost, stated rather than discovered later: on the web an integrity failure leaves a 0-byte file at the path the user chose, because the picker creates that entry when the user confirms, before any of our code runs. No plaintext is ever written unauthenticated, and a file the user chose to overwrite is untouched - `createWritable` writes to a swap file that is never closed. `finalize(false)` cannot clean it up: the error propagates into web_io's `on Exception`, which returns before the `handle.remove()` branch, and ending the stream normally instead would report a cancellation rather than the integrity failure. The new test asserts the ordering itself - save target opened, then download opened, then bytes written - which is the property nothing checked before and which fails on the old ordering. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore: drop the vendored CJK character maps PE-9200 1.3 MB across 168 .bcmap files, on a build that is permanent on Arweave. They are only consulted by PDFs using a *predefined* CJK character encoding - typically older documents. Anything with embedded fonts, which is the modern norm, never touches them, so this is not "PDFs stop working for Chinese and Japanese users": it is a narrower class of older documents rendering with missing glyphs instead of correct ones. `cMapUrl` is removed with them. Left pointing at a directory that is not there, pdf.js would fetch a 404 per glyph rather than skip them. What inline PDF still costs, stated where the next person will look: 1.3 MB vendored, of which the 238 KB pdf.min.js loads on every page - login included - because the pdfx web plugin asserts `pdfjsLib` at plugin registration. Only the 1 MB worker is deferred to when a PDF is actually opened. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix: address CodeRabbit's review of this PR PE-9200 Nine findings, all verified against the code first. Seven fixed outright, two partially rejected with reasons. - a file that failed its MAC check said "unknown error" and offered Try Again. It now says the file could not be verified, that nothing was saved, and that retrying will not help - with the new string localized into all six locales. - an unhandled error in the shared-file size gate left the dialog stuck on "starting" forever, because the check runs from the constructor and its future was discarded. It now proceeds, matching the personal cubit. - the sandboxed iframe derived its view type from `DateTime.now()`, which on the web has millisecond resolution - so `microsecondsSinceEpoch` is always a multiple of 1000 and two frames registered in the same millisecond collided, one replacing the other's factory. A monotonic counter now guarantees what the clock only appeared to. - opening the first gateway response eagerly (so a dead gateway is reported before the save picker opens, rather than as "Download cancelled" after) leaked that response and the integrity verifier if nothing ever listened to the stream - a dismissed save picker, for instance. The eager open stays; the release is now threaded to every settle path. - turning off "remember while this tab is open" forgot the stored key but left the pending one set, so a successful unlock wrote it back anyway, against the choice the recipient had just made. - a remembered key read from session storage could land after the recipient had typed their own, replacing their input and submitting the stored one - or submit after the page had already unlocked. - an oversized `thn` in a crafted link was buffered whole before the size check could reject it, and the timeouts did not bound it: the per-gateway timeout does not cancel the buffered fetch beneath it. The fetch is now bounded by a Range request, rejects on a declared oversize without reading a byte, and cuts the body off at the cap otherwise. - `openUrl` logged a refused URL and then threw into ~30 fire-and-forget call sites, where the exception became an unhandled async error and the user saw a dead link. It returns false now, and the pre-existing launch failure is logged rather than thrown too. - the boot shim passed a caught error to `console.warn`, and the URL it is parsing ends in `#k=<key>`. Rejected, with reasons: `StreamedDataItemVerifier` does not need a `dispose` separate from `markUnverifiable` - that method already cancels the feed and settles the verdict, and a second name for one operation invites abandoning a check twice. `downloadToMemory` does not need the release path either: it listens on the statement after the stream is built, with nothing between that can throw, and a safeguard that can never run is worse than none. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs: record why markUnverifiable is also the verifier's disposal PE-9200 Missed from 9f5ac21 - the argument for rejecting CodeRabbit's suggested separate `dispose`, written where the next person will look for it rather than only in a PR comment. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
…dead code removal (#2178) * fix: give every gateway fetch a fallback, and clear code the refactor stranded PE-9202 Follow-ups found during the file-sharing work and deferred as out of scope. - `fetchManifestWithFallback` wrapped a manifest body in `http.Response(String, int)`, which encodes as **latin1**. A manifest containing any non-Latin-1 character - a CJK filename, an emoji - threw at construction, was swallowed by the waterfall as a gateway failure, and told the user every gateway had failed while every gateway was returning the same perfectly good manifest. Live today for manifest downloads and previews. It now keeps the bytes. - `DownloadService` fetched from a single gateway while everything around it had a waterfall. All four callers want one: bulk import and pin resolution fetch manifests, password login fetches a signature transaction, and the zip download fetches arbitrary file bytes. - that last one needed a new entry point rather than the existing one. The bounded `fetchData` caps the whole waterfall at 25 seconds, which is fine for a thumbnail and hopeless for a 500 MB zip. `fetchFileData` uses idle timeouts - time to headers, and the gap between chunks - so a slow but healthy download is never killed for being large. - deleted what the refactor stranded: `owner_field.dart` (no importers), `FsEntryInfoCubit`'s shared-file parameters (its only caller stopped passing them, leaving `?? maybeRevisions!.first` a force-unwrap that could only throw), `FsEntryFileInfoSuccess.ownerAddress` (written, never read), and an ignored `drivePrivacy` parameter - which cascaded into `DetailsPanel`, since removing its last reader left a required public field nothing consumed. `FsEntryPreviewMemory` also went, though that one was already dead on `dev`. Video and audio previews still stream from a single gateway, and that is left deliberately. The players take one URL and build from it in `initState`, so surviving a mid-stream failure means re-initialising the controller across four widgets with position restore - and `just_audio` and `video_player` need different retry shapes. Pre-resolving a gateway instead would put a probe on the critical path of every preview and still not survive the first bad byte. It belongs with player work. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore: vendor pica and ethers, the last two CDN scripts PE-9202 Same argument that carried for pdf.js: this build is permanent on Arweave and could never be patched, a CDN script breaks under the strict CSP an AR.IO gateway serves, and these two had full run of the origin where a recipient's access key can sit in sessionStorage - with no SRI and no CSP. Both are byte-identical to what jsdelivr serves - downloaded and `cmp`'d - and both npm tarballs match the registry's published integrity hashes, so this is a runtime no-op. The `window.define` juggling around the ethers injection is preserved verbatim; it exists to dodge a require.js conflict in debug mode. The cost is worth stating: 524 KB added to a permanent deploy, and 487 KB of that is ethers, loaded eagerly on every page including login, by every user including those who never touch an Ethereum wallet. It is a strong candidate for `LazyLoader` - larger than `pst.min.js` and `ario_sdk.min.js`, which are already lazy, and `loadArweaveWallet()` already performs the identical define/require juggling. That needs `await`s at four Dart call sites, so it is a separate change rather than a quiet addition to this one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * ui: give the recipient page a desktop layout, and make it accessible PE-9202 The redesign replaced the old desktop/mobile fork with one 400px column at every width. On `dev` the locked and error cards were 400 too - correctly, a key-entry card should not be 1200px - but a *loaded* file filled the viewport through DetailsPanel, and that is what was lost. It matters more now than it did then, because the ready state is where the preview lives. Ready now opens into two panes on desktop: actions and identity on the left, capped at the width of the download button, and the file itself on the right. Locked, resolving, not found and damaged-link stay narrow at every width. The pane keeps its height whether or not the preview is open, so pressing Preview does not reflow the page, and the phone column is untouched - a 700px window and a landscape phone both still get it. The accessibility pass found more than the layout did: - the "a newer version exists" notice was **invisible in dark mode**: white text and a white icon on a pale info panel, 1.23:1. Both tokens are theme-invariant, so the pairing can never be right in both themes. - `themeFgSubtle` is grey.500 in both themes - about 2.7:1 on the card, where 4.5:1 is required. It was carrying the trust strip, the drawer headers, the details labels, the version rows and every caption on the locked gate. - the "Verified" badge sat at 2.2:1, and the status icons under the 3:1 that graphics need. - the copy control in the details drawer was a 16px gesture detector: not focusable, not keyboard-operable, unlabelled, a quarter of the target size. - the remember-key checkbox was invisible to both keyboard and screen reader. - key-field focus was dropped while a key was being checked and never came back after a rejection - exactly when the recipient needs to retype. Reported and deliberately not fixed, because they are `ardrive_ui` and would land on every screen in the app: the text field's obfuscation toggle is an unlabelled 24px gesture detector, `ArDriveClickArea` does not create the hit area its name implies, `ArDriveButton` has no visible focus ring in any variant, and `themeInfoSubtle` paired with `themeFgDefault` is unreadable in dark mode wherever else it appears. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat: pack the share link payload into one parameter PE-9202 A v2 link was 344 characters. Chat clients truncate them, mail clients wrap them, and they are hostile to a QR code. The payload is now one `d` parameter holding a packed byte string: a version byte, a flags byte, a field bitmap, the fields the bitmap names, then extensible records. public 268 -> 232 private, keyless 301 -> 248 private, key 347 -> 294 pinned + hidden 264 -> 222 Modest, and worth understanding why before taking it: a 43-character Arweave id already *is* base64url of 32 bytes, so re-encoding three of them inside a larger blob saves nothing. Packing recovers parameter names, separators, the `AES256-GCM` spelling and long MIME types - the content-type table turns the Word type from 77 characters into one. Of a 232-character public link, 71 are the origin and route and 128 are the three ids. It is free to do now because the v2 format has never shipped; v1 links are untouched and still parse exactly as they always have. The container is the other half of the value: a new field is a new record tag, and old clients skip what they do not recognise by its length rather than failing. `k` is deliberately not inside the payload - it has to stay independently placeable in a fragment. The next 43 characters are `own`, worth another 28-32%. It is never a query input and forgery detection does not rest on it - authorship is established by the first-writer probe - so its only unique job is painting "Shared by" for the few hundred milliseconds before the metadata lands. That is a product call about first paint, so the numbers are written into the design doc rather than acted on. Also fixes the non-canonical example ids that have now cost two rounds of test failures: they are 43 base64url characters whose final character carries bits a 32-byte value cannot have, so they never decoded. There is one shared `isCanonical32ByteId` check now. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat: tell the recipient what happened to their download PE-9202 Phase 2 built resume, mid-stream gateway failover and chain-based integrity verification, and none of it reached the user. The verdict was computed on every download and dropped on the floor: a corrupted public or AES-CTR file still said "Saved". A download that lost its connection and resumed showed a frozen progress bar. Nothing distinguished a retry that resumes from one that has to start over. Built into the existing download modal rather than as the inline-on-button progress the design doc asked for. That document was written before anyone checked it against the app, and the modal - ArDriveProgressBar, a `4.8 MB/12 MB` line, a percentage - is the house pattern. A second progress pattern beside it would have been the wrong kind of new. - a verifying state, so the modal no longer jumps from 100% to done while a check is still running - the verdict, in three honest registers. `notVerified` is *not* evidence of a problem and does not read as one: it happens whenever a download resumed, or the file was signed by an Ethereum or Solana wallet, which cannot be deep-hash verified at all. `failed` means the bytes contradict the signature, and that one takes over the modal, names the file, and says not to open it. - reconnecting, driven by the resume events the downloader already emitted - "Start Over" rather than "Try Again" when a gateway ignored Range, because resume is impossible for that download and the button should not imply otherwise The verdict is awaited after the progress stream completes, never during - saving must not wait on verification - with a ceiling that falls back to "couldn't check" rather than trapping the user in a modal over a file already on disk. The mobile public path reports nothing at all, deliberately: it hands the transfer to the platform and never sees the bytes, so naming an absence would report a check nobody attempted. Ten strings, six locales. The five translations are machine-generated and need native review. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * ui: fix what the rendered screens showed PE-9202 Rendered every state to PNG at desktop and phone sizes in both themes, with real fonts, and looked at them. These are the defects that found. - **the desktop preview pane was an empty box.** A 580x430 rectangle holding one 40px icon, more than half the card, on arrival. It is now a designed empty state: a tinted tile, a line saying what the pane is for, and the Preview action anchored inside it - which also removes the second red button that used to compete with Download. - **the access key field was a white slab in dark mode.** `themeInputBackground` is `white` in both themes, so the most prominent element on a near black card was pure white. This page is where a first time recipient meets it. - **an em dash and an ellipsis rendered as tofu.** Wavehaus has neither glyph, so `originally uploaded - something changed it` showed a replacement box in the middle of a sentence telling someone not to open a file. Removed from recipient facing copy in all six locales. - **the integrity failure modal looked exactly like success.** Same red strip, same neutral body - and every ArDrive modal wears that strip, so red carried no meaning. It now has an alert icon and lifts the body into a bordered notice, so the most severe outcome in the download path stops reading as routine. - **a two line field error lost its second line.** `AnimatedTextFieldLabel` reserved a fixed 22px inside a `ClipRect`, so on a 390px phone the wrong key message read `...and try` with no ellipsis. A minimum height instead of a fixed one keeps the no jump behaviour and lets the message grow. This one is in `ardrive_ui` and affects every text field in the app - deliberately, as clipping the one string a reader must read is not a behaviour worth keeping anywhere. - an orphaned separator dot trailed every meta line: `4.60 MiB . PDF .` Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * ui: stop setting the file name as heavy as the download button PE-9202 `headline.headline5Bold` is 22px at w800 - the heaviest weight in the system - and is what ardrive_ui gives button labels and modal titles. The file name was set exactly as heavy as the Download button beneath it. `heading5` at w700 from the newer scale still reads as the page's one heading without competing with the primary action, and it is the scale details_panel.dart uses - the surface this page replaced. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
… block four (#2179) * perf: sync reads one gateway, and stops letting one slow item block four PE-9203 Sync fetched every metadata item through the multi-gateway waterfall: the configured gateway, up to two GAR gateways, then arweave.net, serially, five seconds each. A flaky gateway turned each item into twenty seconds, and a sync reads hundreds of them. Building that list also cost a Solana RPC call. Sync now reads the configured gateway only - two attempts, five seconds each, then the item is skipped. A 404 is not retried, because the same host will not change its mind. Twenty seconds becomes ten, and the Solana RPC is gone from the per-item path entirely. The waterfall is untouched everywhere else. Downloads, previews, manifests, shared links and drive attach keep it, and a recipient with one dead gateway still gets their file. The larger win is scheduling. The concurrency limiter was a barrier, not a pool: for (start = 0; start < items; start += 5) await Future.wait(...five items...) Each batch waited for its slowest member, so one ten-second failure left four workers idle for it. A sliding window keeps five in flight at all times. Measured over 300 items with 10% failing: 321s to 83s, 3.9x. Concurrency itself is unchanged at five - the pool is the structural fix, and raising the number is a separate knob best turned with evidence, not at the same moment we removed the fallback. Snapshot validation loses its GAR branch too; it already fast-failed on 404, and the remaining path existed only for transient errors at the cost of that same RPC. What this PR found is worth more than what it changes: **sync already drops files silently, today.** A failed metadata read returns empty bytes rather than an error (`arweave_service.dart`), those bytes fail to parse, the parse exception is logged as a warning, and the entity never reaches the drive - while the watermark advances regardless. The block-height rewind is a two hour recency window, not a retry mechanism, so anything older is gone until a manual deep sync. Skipped transaction ids and their drive ids are now carried out on the sync result instead of vanishing, and `docs/SYNC_SKIPPED_ENTITY_PERSISTENCE.md` sets out the fix: a table, a block-height range union, and a backoff. No UI for skipped items in this PR. Per-file state cannot be coherent while the list lives only in memory - rows would appear and vanish across restarts - so it lands with the persistence that makes it stable. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test: make the snapshot validation test actually validate something PE-9203 From CodeRabbit. `validateSnapshotItems([])` returns before the loop, so the test asserted nothing about the path it was named for - no HEAD request, no rejection. It now passes a snapshot item, so the unreachable configured gateway is really contacted and really rejected, which is the point where `dev` would have reached for the GAR list. The `verifyZeroInteractions` on the SDK stays, but labelled for what it is: the service is never handed that object, so the compile-time signature above is what proves the branch is gone. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix: drive discovery was still fanning out to fallback gateways PE-9203 `getUniqueUserDriveEntities` is sync's drive-discovery phase, and it called the waterfall once per drive transaction, in parallel. A user with a dozen drives opened a dozen fan-outs to GAR gateways on every sync - the exact cost this change exists to remove, sitting in the one sync path that was left alone. It was left alone on the belief that the call is shared with login. It is not: `getUniqueUserDriveEntities` has exactly one caller in the codebase, `_SyncRepository.updateUserDrives`. The login path is `getDriveSignatureForDrive` (`ardrive_auth.dart`), a different method, which keeps the waterfall and should. Audited the rest rather than trusting the earlier list: no other waterfall method - getLatestDriveEntityWithId, getAllFileEntitiesWithId, getLatestFileEntityWithId, dataFromTxId - is reachable from lib/sync at all. Drive discovery was the last one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * perf: pool the last two fan-outs in the sync path PE-9203 Auditing the rest of sync after the drive-discovery miss turned up two more, both reachable from every sync. `getTransactionConfirmations` had the identical chunked barrier the metadata reads had: chunks of GraphQL confirmation queries dispatched `maxConcurrent` at a time with an `await Future.wait` between batches, so each batch waited for its slowest query while the other workers sat idle. It runs on every sync through `_updateTransactionStatuses`. Same pool, same bound. Drive discovery was unbounded - a `Future.wait` over every drive transaction at once. That was survivable while it fanned out across several gateways; now that it reads one, an unbounded burst is aimed entirely at that single host, and a user with many drives would open every connection simultaneously. Bounded to the same limit the metadata reads use. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix: the drive signature read put the waterfall back into sync PE-9203 From CodeRabbit, and correct. Drive discovery fetches a drive signature for every private drive whose key is not already in memory, and it did so through `getDriveSignatureForDrive` - the login path, which uses the waterfall. So the fan-out was back in the sync path by the side door, one private drive at a time. Both readers now share one implementation and differ only in how they fetch: `getDriveSignatureForDrive` keeps the waterfall for login, where one unreachable gateway must not cost someone their session, and `getDriveSignatureForDriveOnSync` reads the configured gateway only. Worth recording why this survived two audits: both greps that looked for waterfall callers excluded `arweave_service.dart` itself, and this call is inside it. An internal caller is invisible to a search that assumes the caller lives somewhere else. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix: retry the snapshot HEAD once, like every other sync read PE-9203 Sync metadata reads get two attempts, because one blip should not cost a file. Snapshot validation got one - and failing there is the more expensive outcome: a rejected snapshot discards its whole block range, which sync then re-queries over GraphQL. The cheap failure had the retry and the expensive one did not. Two attempts now, same budget and the same 300ms gap. Statuses the host will simply repeat - 400, 401, 402, 403 - still reject on the first answer rather than spending time to be told twice. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Brings in everything that landed after the branch was cut: #2161 parallelize upload preparation #2174 read the Turbo free-item cap from /v1/info #2175 self-describing share links, authenticated downloads, recipient page #2176 vendor pdf.js and add pdfx #2177 CLAUDE.md #2178 gateway fallback for every fetch, vendored CDN scripts, dead code #2179 sync reads a single gateway, pooled instead of chunked No conflicts. Verified on the merged tree: analyze clean, 1319 app tests, 43 crypto tests. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
vilenarios
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release v2.86.0
🔒 Security
went through a streaming decryptor whose AES-GCM branch trims the authentication
tag and never checks it. GCM now buffers, verifies, and writes nothing on
failure; AES-CTR and public files get chain-based integrity — the ANS-104 deep
hash computed over the ciphertext as it streams, checked against the data item's
signature.
arbitrary JavaScript on the app origin, where a recipient's access key can sit
in
sessionStorage. Mitigated in the vendored build with a CI guard so itcannot silently revert. The real fix is pdf.js ≥ 4.2.67, which needs a newer
Flutter than the pinned 3.19.6.
plausible.jssentlocation.href, fragment included, to plausible.io. Ona key-in-link share that string is the file key. The tag is removed.
the link's metadata was fetched with no owner constraint. Authorship is now
checked, without adding a round trip before first paint.
🔗 File sharing
The access key no longer rides in the link by default. It is a separate artifact
sent out of band, and the share dialog hands over both with copy that pushes them
down different channels. Key-in-link stays as an explicit opt-in.
A link now carries what the recipient's page needs, so a complete one paints with
zero blocking GraphQL. Every field degrades independently, and v1 links
resolve exactly as they always have.
The recipient page is a real state machine — locked, resolving, ready, not found,
network trouble, damaged link. No eternal spinners, a wrong key says so rather
than "file does not exist", a recently-uploaded file retries instead of claiming
it is missing, and transaction ids live behind a drawer. PDFs render inline,
private video and audio preview, and recipients see real thumbnails.
⚡ Sync
Sync read every metadata item through a multi-gateway waterfall — configured
gateway, two GAR gateways, then arweave.net, serially at 5s each — and building
that list cost a Solana RPC. It now reads the configured gateway with two
attempts, and the RPC is gone from the sync path.
The larger win was scheduling: the concurrency limiter was a barrier rather than
a pool, so one slow item idled four workers. Over 300 items at a 10% failure
rate, 321s → 83s.
Known and documented: sync already drops a file silently when its metadata
cannot be read — empty bytes, a warning, and the watermark advances anyway. The
block-height rewind is a ~2 hour recency window, not a retry. Skipped transaction
ids now ride out on the sync result instead of vanishing, and
docs/SYNC_SKIPPED_ENTITY_PERSISTENCE.mdsets out the fix.💳 Turbo free-tier support
The app now makes an honest free-vs-paid promise on every upload:
used up — you're told clearly ("exceeds your free allowance" / "free allowance
used up") and shown how to pay, instead of a surprise failure.
(files/folders, metadata ops, snapshots, manifests, thumbnails, bulk import,
ArNS name assignment): failures no longer hang — you get a clear dialog with a
Buy Credits action.
and rename no longer hangs on a payment rejection.
/v1/inforather than a stale config value.📱 Mobile fix
The Arweave network-congestion popup no longer covers its own buttons on small
screens (height-bounded, scrollable, dismissible).
🧹 Cleanup
preparation, so the upload modal opens a touch faster.
picaandethersare vendored — no runtime CDN dependency ships in a buildthat is permanent on Arweave.
Before tagging
ja, zh, zh-HK — including the copy telling a recipient to send the access
key through a different channel, and the copy telling someone not to open a
file that failed its integrity check. Structurally verified; never read by
a speaker.
100 MiB. The picker is a native dialog no automation can drive.
three real defects surfaced in it after review had passed.
build-iosandbuild-androidareif: falsein all three workflows, so nothingmobile is validated by this PR. Unchanged from prior releases.
Verification
flutter analyzeclean · 1,319 app tests · 43ardrive_cryptotests · mergedfrom
devwith no conflicts.Includes #2161, #2174, #2175, #2176, #2177, #2178, #2179.
Full changelog:
v2.85.0...v2.86.0