Skip to content

Add root and child outcome summaries to Halter trace files #172

Description

@pbdeuchler

Summary

Halter trace files contain enough raw JSONL to reconstruct a run, but they do not include a compact root/child outcome summary. Debugging recent software-factory failures required manually separating root sessions from appended child/model-judge sessions before the real failure mode was visible.

What & why

TraceRecorder writes one file per root session and appends subagent headers/events into that file. This is useful for full replay, but the first diagnostic question is usually simpler: did the root session complete, which child sessions failed, which failures were cancelled/retryable, and what was the last committed/pending state?

Without a summary, raw TurnFailed counts are misleading. In the June 19 software-factory traces, several TurnFailed events were child/model-judge panel failures while the root session completed successfully. A small summary would make traces immediately useful for humans and for automated run reports.

References

  • crates/halter-runtime/src/trace_recorder.rs:15 - trace files are currently per-root JSONL streams with pending and committed events.
  • crates/halter-runtime/src/trace_recorder.rs:19 - subagent events are appended to the root trace rather than written to separate files.
  • crates/halter-runtime/src/trace_recorder.rs:64 - subagent sessions are marked with subagent_header lines.
  • crates/halter-runtime/src/trace_recorder.rs:152 - pending events are recorded before commit.
  • crates/halter-runtime/src/trace_recorder.rs:196 - committed events are appended as raw SessionEvent lines.
  • crates/halter/src/builder.rs:207 - configured runtime.traces_dir wires the recorder into the runtime.
  • Trace evidence: June 19 software-factory traces under ~/.halter/traces/ required root/child separation to distinguish root code-review failures from child model-judge/provider failures.

Nuances & considerations

  • The existing event stream should remain replayable and append-only. A summary can be an additional JSONL record such as trace_summary emitted on root shutdown, or a separate sidecar file generated by a trace summarizer.
  • Runtime can summarize session/turn/tool outcomes, but application-specific parse outcomes are outside the runtime unless embedders can emit stage metadata or annotations.
  • Pending events and committed events should be summarized separately enough to identify interrupted traces.
  • The summary should preserve child lineage: root session id, child session ids, parent ids, terminal state, last event kind, failure message, cancelled, retryable, model/provider metadata when available, and usage totals when available.

Open questions

  • Should this be written eagerly as the trace closes, generated lazily by a CLI command, or both?
  • Should embedders be able to attach a stage label or application-specific metadata to the trace summary?
  • Does adding a trace_summary JSONL record require bumping TRACE_FORMAT_VERSION, or can readers ignore the new kind compatibly?

Definition of done

  • A human can inspect a trace and immediately see root terminal state and child terminal states without writing a custom parser.
  • Child/model-judge failures are clearly distinguished from root-session failures.
  • Interrupted traces expose the last pending and last committed event for each session.
  • Tests cover root-only traces, root plus subagent traces, child failure with root success, root failure, and interrupted/no-terminal traces.
  • Documentation describes the summary record or CLI summarizer and its compatibility behavior.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions