feat(dashboard): make the 30s auto-refresh ingest new usage (revives #59)

[Lyazid21]

Problem

With the dashboard open, fresh Claude Code usage never appears. The 30-second auto-refresh only re-fetches /api/data, which is a pure read of usage.db — nothing on the server re-scans the transcript JSONL after the single startup background scan. Users end up clicking ⟳ Rescan every time, or (more commonly, if they don't realize the button is the ingestion path) Ctrl+C-ing and relaunching the server. This is the limitation noted in the #138 discussion: "Auto-refresh only re-reads /api/data and never scans, so the button is the only in-session way to ingest new turns."

Fix

• The 30s timer now fires autoRefreshTick(), which POSTs /api/rescan — the same incremental, non-destructive scan the button runs — and then re-reads /api/data. An in-flight guard keeps ticks from stacking behind a slow scan, and a guarded tick still re-reads, so a long scan renders progressively (preserving the serve-first/scan-behind cold-start behavior of cmd_dashboard).
• A module-level RESCAN_LOCK serializes scans server-side: a rescan arriving while another runs (second tab, tick racing the manual button) gets an immediate {"busy": true} instead of a concurrent scan, and triggerRescan shows "Rescan (already scanning)" for that case. cmd_dashboard's startup scan holds the same lock, so a tick during a cold start skips instead of double-scanning the backlog.
• SQLite lock waits raised 5s → 30s on both sides (scanner.get_db connect timeout, the dashboard's read busy_timeout): scans now routinely run beside reads, and the end-of-scan sessions recompute can hold the write lock longer than 5s on a big DB.
• Tests: /api/rescan busy path, plus template guards pinning the timer to autoRefreshTick (never bare loadData) and the scan-before-read order inside the tick.

Verification

• Full suite passes on top of v1.5.5 (149 tests), including the three new ones.
• Live sandboxed run (fake HOME + CLAUDE_USAGE_DB, real server): turns appended to a session JSONL after startup were ingested by the tick's POST /api/rescan and reflected in /api/data with exact token totals; the busy path returns {"busy": true} while the lock is held and resumes normally after.
• Both embedded <script> blocks of the served page pass node --check.

History

#59 proposed exactly this (auto-refresh running an incremental scan) back in April and was closed — understandably, since /api/rescan was a destructive full rebuild at the time, and auto-firing it would have been dangerous. #138 → 97e1d91 (v1.2.6) made the endpoint incremental and append-only, which is what makes an auto-firing rescan safe now. This PR revives #59's intent on that foundation.

Notes

• CHANGELOG.md deliberately untouched (version headings accumulate on DEV); suggested bullet for the next section:

  Made the 30s auto-refresh actually ingest new usage: each tick now runs the same incremental scan as the Rescan button before re-reading the DB, so fresh turns appear without clicking Rescan or restarting the server. Concurrent scans are serialized behind a rescan lock.

• One possible follow-up (not in this PR to keep it minimal): the end-of-scan sessions recompute is unscoped, so on very large DBs (hundreds of thousands of turns) an active tick rewrites every sessions row (~0.8s measured at 500k turns). Scoping it to sessions touched by the scan would make active ticks O(delta).

🤖 Generated with Claude Code