⚙️ FEATURE: Regression detection — capture lint/test baseline before and after each session

[FernandoCelmer]

Problem

pycodeloop's JSONL tracer records tool calls and token usage, but does not capture the state of tests or lint before/after each session. If the agent introduces a regression (breaks a previously-green test), it goes undetected unless the human runs the test suite manually after the session ends.

"Code as Agent Harness" (arXiv 2605.18747) identifies regression prevention as an open challenge in long-horizon agentic tasks.

Expected behavior

When a check_cmd is configured, CodeLoop automatically:

1. Runs the command before the session and records the result in the JSONL trace as {"type": "baseline_check", "ok": true, "output": "..."}
2. Runs the command after the session and records {"type": "post_check", "ok": false, "output": "..."}
3. If baseline was ok and post is not → emits {"type": "regression_detected"} and surfaces a warning in CLI output

Suggested implementation

codeloop = CodeLoop(config=Config(
    check_cmd="pytest tests/ -q",
))

• check_cmd runs at session start and session end
• Results appended to existing JSONL trace
• cli output shows: ⚠️ Regression detected: 3 tests broke if applicable

References

• "Code as Agent Harness" — arXiv 2605.18747, regression prevention challenge