Decouple and complete the resumable Last.fm import pipeline

[rianjs]

Outcome

Decouple and complete the resumable Last.fm import pipeline as a follow-up to #39 and #40. Keep the core deterministic, keep Spotify traffic behind the shared client/request gate, and make source download independently resumable before any Spotify-derived work.

Acceptance criteria

• Keep user.getRecentTracks at the documented maximum of 200 rows per page. Capture one fixed historyTo as the recorded pre-scrobbling cutoff.
• Add an exportable account-bound lastfmScrobblingProfile {username, startedAt}. The first successful connection/enable records it; toggling the same username preserves it; a different username replaces it; legacy records missing it are generically backfilled on read/write. Later live proof on this workstation seeds Unix 1786804381; this ticket must not mutate app data during implementation or review.
• Replace importer session V1 with V2 and quarantine V1. Download requires only Last.fm; defer Spotify account binding until the first page match. One metadata probe discovers the total, then download fetches the oldest page toward page 1.
• Atomically cache parsed raw pages under a snapshot-specific machine cache, write the manifest only after each page write, and enforce 100 MiB safety ceilings. Do not aggregate until every page is present; then read, sort timestamp-ascending, aggregate once, atomically enter review, and best-effort delete the raw cache.
• Retry transient Last.fm failures with the existing capped backoff. Relaunching the importer resumes the exact missing page.
• Remove automatic global Spotify matching and the normal matching phase. Add an aggregating phase plus downloadedPages/totalPages.
• When a visible review batch is opened, lazily match it through the shared Spotify client/request gate, serialize duplicate requests, cache the result, and make revisits issue no API call. Defer Spotify identity resolution/binding to the first match; a later mismatch suspends Spotify-derived work. Do not add adjacent prefetch.
• Make Accept All the explicit bulk exception: sequentially match/cache all remaining batches, show exact entity counts, and apply only after confirmation.
• Update UI states and the owning architecture documentation. Do not add a dependency or generic importer abstraction.
• Quarantine/restart any incomplete V1 cache because the changed cutoff changes page boundaries.
• Add regression coverage for cutoff/order, complete-before-aggregate, interruption/reload/orphan/oversize/corruption, zero Spotify during source work, one-batch lazy matching/cache/revisit, account binding/mismatch, settings export/restore, UI states, and Accept All. Run the full documented checks.
• Later live proof: back up app data, seed 1786804381, run a real import, verify resume and that no Spotify request occurs before a visible batch, and do not apply/import any content.
• Delivery uses TDD, one independent Terra-high coverage review, and a live CR review at <pr-url> with inline replies, up to three passes while feedback remains high-value. CI is green; merge and clean up the branch/worktree.
• Put durable architecture decisions in the architecture docs and delete the temporary implementation plan after merge.

Scope guard

No product implementation, documentation edits, tests, commits, pushes, PR, native app launch, or connected Spotify/Last.fm mutations are part of the planning gate.

[rianjs]

Plan: Decouple and complete the resumable Last.fm import pipeline

WORK_KEY=41
Issue: #41
Base: main at ecc435944e559dac41e0e2a223e9a738b0877a3f
Branch: feature/41-decouple-lastfm-import

Context and scope

Issue #39 and merged PR #40 delivered the first importer as
LastFmImportSessionV1. The current implementation binds Spotify before
source work, aggregates rows while downloading, and globally matches every
batch before review. This follow-up keeps the existing deterministic library
mutation and Spotify save/match helpers, but changes the import boundary to a
Last.fm-only resumable snapshot followed by visible-batch matching.

No implementation, product/docs/test edit, commit, push, PR, native launch, or
connected-service mutation is part of this planning gate.

Decisions

1. Account-bound scrobbling profile and cutoff

• Add LastFmScrobblingProfile { username, startedAt } to the persisted
  Settings shape using the existing camelCase serde/TypeScript conventions,
  and include the same non-secret field in ExportSettings so library
  export/restore carries it.
• Centralize profile reconciliation at the existing settings/Last.fm lifecycle
  boundary. A successful finish_lastfm and an enable transition while a
  Last.fm account is connected call the same helper:
  • missing profile: create it with the current Last.fm username and one
    captured unix_now() value;
  • same username: preserve startedAt, including disable/re-enable;
  • different username: replace both fields with the new username and now;
  • legacy settings with no profile: generic backfill on the first available
    connected/enable/import boundary, never a username-specific migration.
• The import's fixed historyTo is the profile's recorded startedAt cutoff.
  Capture it once into V2 at session creation and never recompute it on resume.
  The later live proof seeds this workstation's profile/cutoff to Unix
  1786804381; this gate does not mutate app data.

2. V2 session and machine-local raw-page snapshot

• Replace LastFmImportSessionV1 with LastFmImportSessionV2 and set the
  session version to 2. Keep the existing compact source rows, review
  decisions, options, match cache, count modes, and search-term preference,
  but make spotifyAccountId nullable until the first Spotify match and store
  the immutable historyTo, snapshot cache ID, page cursor, totals, and
  downloadedPages.
• Keep the existing user.getRecentTracks limit constant at 200. A single
  metadata probe uses the same username, historyTo, page-size, and Last.fm
  retry policy to discover totalPages/total scrobbles. It is not a completed
  source page. Download then requests pages in descending order
  totalPages ... 1, so the oldest history is acquired first and the next
  missing page is unambiguous after interruption.
• Store parsed page checkpoints, not full HTTP responses, under a
  snapshot-specific machine cache keyed by the V2 snapshot ID. Each page file
  is written atomically first; only after that succeeds, atomically replace a
  manifest containing expected page numbers, byte totals, historyTo, and
  totalPages. The session JSON records the cache ID and cursor but does not
  aggregate partial rows.
• Enforce the existing 100 MiB serialized-session ceiling and a matching
  100 MiB total raw-cache/manifest ceiling before committing a page. The
  manifest is authoritative: a page file written before its manifest update is
  a harmless orphan, so retry may ignore or overwrite it (with best-effort
  cleanup). A manifest-acknowledged page that is missing, corrupt, oversized,
  or metadata-mismatched invalidates/quarantines the entire snapshot and
  starts a fresh V2 session; it is never treated as merely the next missing
  page.
• Do not aggregate while pages are arriving. Once the manifest covers every
  page, enter Aggregating, read all pages, validate their metadata, sort
  scrobbles by timestamp ascending with deterministic text/stable-ID
  tie-breakers, call the existing aggregation helper exactly once, construct
  review batches, and atomically persist the complete rows plus Review phase.
  Best-effort deletion of the raw snapshot happens only after that durable
  transition; failure to delete is disposable-cache noise, not an import
  failure.
• V1, unknown-version, corrupt, incomplete, or cutoff-incompatible session
  files/caches are quarantined and restarted. V1 cannot be resumed because the
  fixed cutoff changes page boundaries.

3. Source-only runner and retry/resume behavior

• Change start_import/resume to require only the connected Last.fm username.
  It must not resolve Spotify identity, acquire the Spotify membership gate, or
  construct a Spotify provider during downloading or aggregation.
• Reuse lastfm::Service::import_recent_tracks_page and its existing capped
  transient retry/backoff. Apply the same behavior to the metadata probe. If
  that internal capped retry is exhausted and the call returns a retryable
  error, persist the retry state, wait one capped backoff interval, and retry
  the same probe/page in-process while the app remains running. Stop only for a
  non-retryable or account failure, or when the app exits. A failure never
  advances the cursor; a successful page advances only after both the page
  file and manifest are durable.
• Replace Matching with Aggregating in the backend and frontend state
  machine. Expose downloadedPages and totalPages; remove the automatic
  global matching progress/counts. Source completion alone enters review with
  no Spotify match requests having occurred.
• Keep Last.fm account-change suspension and the existing serialized
  session/persistence ownership guarantees. A Spotify account is irrelevant
  while spotifyAccountId is null; after binding, any account mismatch
  suspends Spotify-derived work and redacts/blocks unsafe review data until an
  owner-valid resume.

4. Lazy matching and account binding

• Reuse the existing album_candidates, track_search_term,
  match_result_for, candidate classification, provider adapters, shared
  SpotifyClient, request gate, and ownership checks. Delete run_matching
  and its normal Matching phase.
• Make the existing visible-page path (lastfm_import_page) load or match the
  requested artist/album batch. If every source row in that batch already has
  a cached result for this snapshot, return it without another Spotify search
  or album-track request. Otherwise, serialize lazy batch matching with one
  importer-wide async mutex, re-read the session after acquiring it, and let a
  second caller consume the first caller's persisted result. This is the
  deliberate ponytail: global lock; use per-batch locks only if measured
  throughput later requires them.
• Resolve /me/the exact current Spotify account only for the first match (or
  a later ownership check required for an already-bound session), and persist
  spotifyAccountId together with the batch match results in one atomic V2
  mutation. Do not bind Spotify during source work. Revalidate Last.fm and
  Spotify ownership after the search and before persistence; an account change
  writes no stale result and suspends the session.
• Match one visible batch only, with no adjacent prefetch. Manual album/track
  rematches use the same ownership-validated cache/mutation boundary. Cached
  revisits use the persisted match results and make no matching/provider call.
• Keep review/apply semantics from feat(desktop): add resumable Last.fm importer #40: explicit content actions reuse the
  existing membership gate and save helpers; local history/metadata is
  materialized only after the upstream operation succeeds and the final
  account check passes.

5. Accept All as the only bulk exception

• Replace the current implicit-count Accept All flow with a two-step flow. The
  toolbar action first invokes a preparation command that sequentially loads
  every remaining batch through the same lazy matcher, caches all results, and
  returns exact unique Spotify album and track entity counts after current
  selections, exclusions, intents, and whole-album/track mode are applied.
• Show those counts in the existing confirmation dialog. Cancel leaves only
  the match cache/account binding; confirm applies remaining batches
  sequentially through the existing per-page apply boundary. A failed apply
  stops at that batch and leaves later batches reviewable; it never pretends a
  cross-Spotify rollback exists.
• Accept All is the only path allowed to perform adjacent batch matching.

6. UI, persistence, and architecture documentation

• Update apps/desktop/src/LastFmImporter.tsx and
  apps/desktop/src/lastfmImportState.ts for Aggregating, page download
  progress, no global matching pane, visible-batch matching/busy status,
  cached revisit behavior, and the two-step Accept All confirmation. Preserve
  existing accessible controls, queue sorting, review decisions, and theme
  behavior.
• Update apps/desktop/src/types.ts and appState.ts for the V2 state shape
  and exportable profile. Keep legacy frontend defaults compatible.
• Update store.rs/lib.rs for profile migration and export/restore without
  exporting Last.fm credentials or importer/cache data. Keep all persistent
  writes atomic; raw snapshot data remains machine-local and excluded from
  backup/restore.
• Update ARCHITECTURE.md, docs/architecture/library.md,
  docs/architecture/persistence.md, and docs/architecture/spotify.md to
  document V2, the Last.fm-only source boundary, cache/quarantine rules,
  aggregating phase, nullable/deferred Spotify binding, lazy matching, and
  Accept All. No new dependency or generic importer abstraction.

TDD and regression evidence

Add the smallest focused red/green tests in the existing Rust/frontend suites:

• Last.fm contract: limit 200, fixed cutoff propagation, descending oldest-to-
  newest page order, retry behavior, and no Spotify/provider/gate activity
  during source work.
• V2 persistence: complete-before-aggregate, atomic page/manifest ordering,
  orphan ignore/overwrite on retry, invalidation/quarantine and fresh-session
  restart for manifest-acknowledged missing/corrupt/oversized/mismatched
  pages, oversize rejection, V1 quarantine/restart, and atomic
  aggregate-to-review transition.
• Profile/settings: first connection/enable, same-user toggle preservation,
  different-user replacement, generic legacy backfill, and export/restore
  round-trip of lastfmScrobblingProfile without credentials.
• Lazy review: one visible batch causes only its needed match calls; duplicate
  concurrent opens share one result; revisiting is cached/no match call;
  first-match Spotify binding is persisted; mismatch suspends; no adjacent
  prefetch occurs.
• Accept All/UI: sequential bulk preparation, exact deduped entity counts,
  confirmation-before-apply, cancellation, partial failure, aggregating and
  download progress states, suspended state, and existing review controls.
• Run all documented checks from docs/DEVELOPMENT.md: docs check, Rust fmt,
  workspace clippy with warnings denied, workspace tests, frontend install/
  tests/lint/typecheck/build, and diff checks.

Later live proof and delivery gate

After implementation is reviewed locally, back up app data, seed this
workstation's profile/cutoff to Unix 1786804381, run a real Last.fm import,
interrupt/relaunch to prove exact page resume, verify no Spotify request occurs
before a visible review batch, and do not apply/import any content or play
counts. On successful proof, retain the newly completed V2 session and the
approved local scrobbling profile so review can continue. Keep the backup for
recovery and restore it only if validation fails or rollback is required.

Delivery is TDD, one independent Terra-high coverage review, then a live CR
review at the eventual <pr-url> with inline replies for up to three passes
while feedback remains high-value. CI must be green before merge. After merge,
clean the branch/worktree and delete this temporary plan; durable decisions
remain in the architecture docs.