[rescue] rescue/2026-08-09-bl-599 — restore transcript-window backfill with safety hardening

[EtanHey]

Rescue context

This preserves the full transcript-window backfill implementation and its later safety hardening: idempotent replay, watcher/denylist integration, CLI coverage, progress integrity, quarantine, and fail-loud behavior.

The work came from the retained bl-599 worktree. Raw ahead and verified-unlanded counts are both 21 against main; original PR #599 was CLOSED without merge, and the source/test diff is substantive rather than generated noise.

It was rescued instead of merged because the original feature PR closed without landing and the branch later accumulated additional safety fixes that need a deliberate shipping decision.

— maintenanceCodex (worker) · codex/gpt-5.6-sol

Note

Restore watch-backfill CLI with time-windowed, idempotent backfill and safety hardening

• Adds --since/--until window flags and --legacy-excluded-only scope to the watch-backfill CLI command in cli/init.py, enabling time-bounded, repeatable backfills over previously blanket-denied paths
• Introduces WindowedFlush in backfill.py to filter watcher entries to a UTC half-open interval, advancing offsets for non-matching lines to ensure idempotency
• Adds backfill_run_lock (fcntl file lock) to serialize concurrent backfill runs per registry; exits with code 2 on contention and code 1 when work remains incomplete
• Adds is_legacy_backfill_denylisted in ingest_denylist.py that applies the current denylist minus retired blanket patterns (cursor agent transcripts, claude subagents, codex/gemini sessions), still blocking brain-worker subagents
• Extends JSONLTailer.read_new_lines in watcher.py with per-call byte caps and _ProgressMarker emission for blank/malformed lines so offsets advance without enqueuing content
• Risk: normalize_provider_entry now synthesizes a UTC timestamp and sets _timestamp_synthesized=True when the source timestamp is missing or unparseable, changing downstream entry shape

📊 Macroscope summarized 90ac44b. 5 files reviewed, 0 issues evaluated, 0 issues filtered, 0 comments posted

🗂️ Filtered Issues

No issues evaluated.

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 8afafa9a-3c82-4af7-8593-04daee03b4cf

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[macroscopeapp]

🟠 High brainlayer/backfill.py:55

backfill_run_lock derives the lock filename from registry_path without resolving symlinks, so two processes that pass the same registry through different symlink aliases open different .backfill.lock files and both acquire the lock. This means scan, enqueue, and offset persistence are not serialized for that registry. Consider resolving the registry path (and its parent) before deriving the lock path.

-    registry = Path(registry_path).expanduser()
+    registry = Path(registry_path).expanduser().resolve()

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @src/brainlayer/backfill.py around line 55:

`backfill_run_lock` derives the lock filename from `registry_path` without resolving symlinks, so two processes that pass the same registry through different symlink aliases open different `.backfill.lock` files and both acquire the lock. This means scan, enqueue, and offset persistence are not serialized for that registry. Consider resolving the registry path (and its parent) before deriving the lock path.

[macroscopeapp]

🟡 Medium scripts/run_tests.sh:145

When src/brainlayer/cli/__init__.py changes, only test_cli*.py and test_watch_backfill_cli.py are selected and mapped is set to 1, so the pre-push run skips every other test that exercises the CLI. Tests like test_agent_profiles.py, test_doctor.py, test_git_learning.py, test_reembed_backfill.py, and test_status_truthfulness.py are all silently excluded when the CLI entrypoint changes, letting regressions they cover pass the check. Consider not marking this case as fully mapped — leave changed_source_unmapped=1 (or map to all relevant tests) so the full suite runs as a fallback.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @scripts/run_tests.sh around line 145:

When `src/brainlayer/cli/__init__.py` changes, only `test_cli*.py` and `test_watch_backfill_cli.py` are selected and `mapped` is set to `1`, so the pre-push run skips every other test that exercises the CLI. Tests like `test_agent_profiles.py`, `test_doctor.py`, `test_git_learning.py`, `test_reembed_backfill.py`, and `test_status_truthfulness.py` are all silently excluded when the CLI entrypoint changes, letting regressions they cover pass the check. Consider not marking this case as fully mapped — leave `changed_source_unmapped=1` (or map to all relevant tests) so the full suite runs as a fallback.

[EtanHey]

Lead disposition (Etan delegated): holding open as REFERENCE for Wave 4 backfill lanes rather than merging — the base is stale vs current main, but the transcript-window backfill logic here is directly relevant to Wave 4a-4c. Will close when the Wave 4 implementation supersedes it. — brainlayerClaude lead (Fable 5)

🤖 Generated with Claude Code