Skip to content

pending-store queue follow-ups from PR #693 review: fsync durability + O(n) rescan #695

Description

@EtanHey

Two non-blocking findings from the codex review of #693 (round 6, head 8dcfd2d), split out so the receipt-truth fix ships:

  1. Crash-durability of queued writes (P1-as-filed): queue_io.py:50-52 writes+renames without fsyncing file or parent dir; the legacy append in store_handler.py:505-508 syncs the file but not the directory on creation; Swift appendPending similar. A host/kernel crash right after the DEFERRED receipt can lose the entry. Note: the main DB runs WAL with synchronous=NORMAL, so this is about aligning the queue with whatever crash-durability standard we actually claim — decide the standard first, then fix all three sites together.
  2. O(n) rescan under prolonged outage (P2): with the 100-entry trim removed (it silently deleted acknowledged stores), every store during an outage reads/splits the whole fallback file under the exclusive lock — aggregate quadratic. Fix candidates: track depth via file size/line counter instead of full read, or move the depth warning off the write path.

Context: #693 made the DEFERRED receipt read as success and enforced its auto-persist promise across six layers (injection drains, unbounded replay, restart re-arm, no-trim, non-blocking startup snapshot, identity-less flush).

Filed by brainlayerClaude lead (Fable 5).

🤖 Generated with Claude Code

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions