Problem
pycodeloop agents write code directly from a prompt with no enforced test-first discipline. The verification loop (#37) and regression detection (#38) can only catch failures after code is written. Research papers (TestART, RLTF, Klear-CodeTest) demonstrate that generating failing tests before implementation turn forces specification clarity and provides a tight feedback signal that guides the agent toward correct behavior faster.
Expected behavior
codeloop = CodeLoop(config=Config(
tdd=True,
check_cmd="pytest tests/ -q",
))
Turn sequence when tdd=True:
Turn 1 (forced): "Write failing tests for: <original prompt>"
→ agent writes test file
→ check_cmd runs → tests fail (expected)
Turn 2+: "<original prompt> — make the tests pass"
→ agent writes implementation
→ check_cmd runs → tests pass → done
Suggested implementation
References
- "Code as Agent Harness" — arXiv 2605.18747, Section 3.2 (Execution-based verification)
- TestART: Improving LLM-based Unit Test via Iterative Clarification
- RLTF: Reinforcement Learning from Unit Test Feedback
Problem
pycodeloop agents write code directly from a prompt with no enforced test-first discipline. The verification loop (#37) and regression detection (#38) can only catch failures after code is written. Research papers (TestART, RLTF, Klear-CodeTest) demonstrate that generating failing tests before implementation turn forces specification clarity and provides a tight feedback signal that guides the agent toward correct behavior faster.
Expected behavior
Turn sequence when
tdd=True:Suggested implementation
tdd: bool = FalsetoConfigAgent.run(), whentdd=True, prepend a forced system turn before the user's prompt:"Before writing any implementation, first write failing unit tests that specify the expected behavior. Do not write implementation code in this turn."check_cmdfrom ⚙️ FEATURE: Execution-based verification loop (write → execute → verify → re-plan) #37/⚙️ FEATURE: Regression detection — capture lint/test baseline before and after each session #38 — tests written in TDD turn become the verification signalReferences