Skip to content

fix(compute): show queued jobs on the notebook bar and drain them on restart - #371

Draft
wen2zhou wants to merge 4 commits into
aipoch:mainfrom
wen2zhou:fix/compute-queued-jobs
Draft

fix(compute): show queued jobs on the notebook bar and drain them on restart#371
wen2zhou wants to merge 4 commits into
aipoch:mainfrom
wen2zhou:fix/compute-queued-jobs

Conversation

@wen2zhou

Copy link
Copy Markdown
Contributor

Problem

When compute jobs exceed a concurrency limit (session limit or provider ceiling) they enter a
queued status while waiting for a slot. Two gaps made queued jobs invisible and fragile:

  1. Invisible on the notebook bar. submitJob only broadcast job-updated for dispatched
    (submitted) jobs; a queued job was written to the DB but never broadcast, so the renderer store
    — which hydrates once then lives entirely off broadcasts — never learned about it. On top of
    that, RemoteJobBadge only counted running/submitted jobs and subscribed to a stable
    selector reference, so it did not re-render on status changes anyway.
  2. Orphaned on restart. Jobs queued when the process exited were never re-dispatched on the next
    start, so they sat in queued forever even after capacity freed up.

Proposed change

  • Broadcast on create. submitJob now broadcasts job-updated immediately after creating any
    job (queued or submitted), so the renderer sees it instantly. This also closes a brief invisible
    window for submitted jobs before the async dispatcher emits its first transition.
  • Show queued jobs. RemoteJobBadge now displays N running · M queued while jobs are queued,
    stays amber (not gray idle), lists queued rows in the tooltip with a literal "queued" label
    (instead of a misleading elapsed time), and carries an accurate aria-label.
  • Scoped subscription. The badge subscribes to only its session's job slice via useShallow, so
    a job update in another session no longer re-renders it.
  • Drain on startup. New drainQueuedJobs() / drainOnStartup() dispatch queued jobs that can
    now fill free capacity after a restart; wired in ipc.ts right after the poller starts.

Scope and non-goals

  • Scope: queued-job visibility on the notebook bar + restart drain, within the existing concurrency
    model (session limit + provider ceiling). No schema changes.
  • Non-goals: changing the admission/concurrency logic itself, the global queue limit, or other job
    surfaces (e.g. a full job-list page).

Acceptance criteria and validation

  • A queued job appears on the notebook bar immediately (amber, "N queued"), drops from the queued
    segment once dispatched to running, and the badge returns to gray "N jobs" once every job is
    terminal.
  • After a restart with free capacity, previously-queued jobs are dispatched.
  • npm run typecheck, npm run lint, npm run test all pass (full suite: 5795 passed, 0 failed).
  • Tests: 16 RemoteJobBadge render tests, a queued-broadcast integration test, a drainOnStartup
    unit test + a multi-job "drain up to ceiling" unit test, and a restart-drain integration test
    (the latter now polls instead of a fixed delay to avoid CI flake).

Review focus

  • compute-service.ts — broadcast-on-create placement and the drainQueuedJobs facade.
  • RemoteJobBadge.tsxuseShallow session scoping and the segments / aria-label logic.
  • concurrency-manager.tsdrainOnStartup (delegates to tryDispatchNext, which already
    iterates the whole queue under the shared admit lock).

wen2zhou and others added 4 commits July 23, 2026 08:25
When jobs exceed concurrency limits and enter queued status, they were
invisible on the notebook bar badge. Root cause had two layers:

1. Main process: submitJob only broadcast job-updated for submitted jobs
   (via dispatchJob), while queued jobs were written to DB but never
   broadcast. The renderer store relies purely on broadcasts after initial
   hydrate, so queued jobs never reached the UI.

2. Renderer: RemoteJobBadge only counted running+submitted, ignoring queued.

Fixes:
- Broadcast job-updated immediately after creating any job (queued or
  submitted) so the renderer store sees it instantly
- Badge now displays 'N running · M queued' when queued jobs exist, stays
  amber (not gray idle), and tooltip shows queued rows with 'queued' label
- Subscribe to jobsById map (not stable fn ref) so badge re-renders on
  any job status change, not just the 1s elapsed-time tick

Tests: 16 badge render tests + 1 integration test for queued broadcast.
All compute-service/concurrency suites pass (138 tests, no regressions).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… queued tasks

- Add drainOnStartup() to ConcurrencyManager (delegates to tryDispatchNext)
- Add drainQueuedJobs() facade on ComputeService
- Wire void computeService.drainQueuedJobs() in ipc.ts after jobPoller.start()
- Tests: drainOnStartup unit test + integration test for restart scenario
Subscribe to only the active session's job slice via useShallow instead of the whole jobsById map.
applyUpdate swaps the map on every broadcast, so a job update in another session no longer re-renders
this badge — only this session's own add/status changes do. The 1s elapsed-time tick still drives
fresh durations independently.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Replace the fixed 200ms setTimeout wait in the drainOnStartup integration test with a polling
  waitFor helper so it does not flake under CI load.
- Add a multi-job drainOnStartup unit test asserting it drains every queued job up to the provider
  ceiling and leaves the remainder queued, making the drain contract explicit.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@github-actions github-actions Bot added the bug Something isn't working label Jul 23, 2026
@wen2zhou
wen2zhou marked this pull request as draft July 27, 2026 10:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant