Skip to content

feat: replace polling with SSE for real-time scan status #4

Description

@monkey-mode

What problem does this solve?

The frontend currently polls GET /scan/{job_id} every 2 seconds to check scan progress. With up to 10 parallel uploads this means ~5 requests/second during active scanning. More importantly, the user waits up to 2 seconds after a scan completes before the UI updates.

Proposed solution

Replace the polling loop in ScanStatus with a Server-Sent Events stream. The client opens one persistent connection per job and receives status transitions the moment they happen.

Frontend (components/ScanStatus.tsx / components/SlipUploader.tsx)

  • Replace setInterval polling with new EventSource("/scan/{job_id}/stream")
  • Handle message events to update phase, close on done or failed
  • Keep the 30-attempt timeout as a fallback via setTimeout

Backend (backend/sync/slip/routers/scan.py)

  • Add GET /scan/{job_id}/stream returning StreamingResponse with text/event-stream
  • slip-api already has Redis access — subscribe to the scan:{job_id} channel to receive the completion event from slip-worker, then yield it to the client
  • Fall back to polling Postgres internally if the Redis message is missed (e.g. client connects after scan completes)
Frontend ──GET /scan/{job_id}/stream──► slip-api (keeps connection open)
         ◄── data: {"status":"resizing"} ──
         ◄── data: {"status":"processing"} ──
         ◄── data: {"status":"done","result":{...}} ──
         (server closes stream)

Why SSE over WebSocket

WebSocket is bidirectional — the client never needs to send anything after the upload, so the extra complexity is unjustified. SSE is HTTP-native, works through all proxies without config, and auto-reconnects in the browser.

Which area does this touch?

  • Frontend (UI / UX)
  • slip-api (port 8000)

How important is this to you?

Nice to have

Alternatives considered

  • Keep polling — current behaviour, acceptable for single uploads, gets noisier with 10 parallel jobs
  • WebSocket — bidirectional, more complex, proxy config required, no benefit here since the client never sends mid-stream

Additional context

slip-api already subscribes to nothing today — it only publishes to Redis (resize:{job_id}). The SSE endpoint would be the first place slip-api subscribes to Redis (specifically the scan:{job_id} completion channel). The slip-worker already publishes there after writing the expense row, so no worker changes are needed.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions