Skip to content

⚙️ FEATURE: TDD mode — auto-generate failing tests before code synthesis #52

Description

@FernandoCelmer

Problem

pycodeloop agents write code directly from a prompt with no enforced test-first discipline. The verification loop (#37) and regression detection (#38) can only catch failures after code is written. Research papers (TestART, RLTF, Klear-CodeTest) demonstrate that generating failing tests before implementation turn forces specification clarity and provides a tight feedback signal that guides the agent toward correct behavior faster.

Expected behavior

codeloop = CodeLoop(config=Config(
    tdd=True,
    check_cmd="pytest tests/ -q",
))

Turn sequence when tdd=True:

Turn 1 (forced): "Write failing tests for: <original prompt>"
  → agent writes test file
  → check_cmd runs → tests fail (expected)

Turn 2+: "<original prompt> — make the tests pass"
  → agent writes implementation
  → check_cmd runs → tests pass → done

Suggested implementation

References

  • "Code as Agent Harness" — arXiv 2605.18747, Section 3.2 (Execution-based verification)
  • TestART: Improving LLM-based Unit Test via Iterative Clarification
  • RLTF: Reinforcement Learning from Unit Test Feedback

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions