Skip to content

Compaction v2 (2/3): ModelSummary and ProviderDefault strategies, delete signal pruning - #198

Open
pbdeuchler wants to merge 4 commits into
masterfrom
compaction-v2-strategies
Open

Compaction v2 (2/3): ModelSummary and ProviderDefault strategies, delete signal pruning#198
pbdeuchler wants to merge 4 commits into
masterfrom
compaction-v2-strategies

Conversation

@pbdeuchler

@pbdeuchler pbdeuchler commented Sep 4, 2026

Copy link
Copy Markdown
Owner

Closes #195. Builds on merged PR #197 / #194, and incorporates the boundary requirements discovered while designing #196.

Cohesive issue lineage

Issue Pull request Responsibility
#194 #197 Token ledger, Halter-owned triggers, hard context cap, and the first-class strategy seam
#195 #198 ModelSummary and ProviderDefault, config selection, and deletion of signal pruning
#196 follow-up CleanWindow; its per-session reminders and forced rollover drove the revised boundary API in this PR

Summary

This supplies what happens when #197's ledger says it is time to rewrite context.

ModelSummary (new default)

  • When the task tool is registered, first nudges the model to persist concise remaining work. Only task calls execute; text and foreign calls are excluded from the next request while the successful exchange remains in the event log.
  • Sends the built-in checkpoint instructions plus manual custom instructions.
  • Starts a fresh window with one user-role checkpoint message after the byte-stable static prefix. No old tail is retained; a todo reminder is appended only when task calls ran.

ProviderDefault

  • Delegates to Provider::compact only when Halter's ledger requests it. Provider-side automatic triggers remain disabled.
  • Uses provider capability metadata to select a dedicated or inline transport.
  • Dedicated compaction rewrites everything before the latest assistant block. Inline compaction rewrites the older prefix before the latest user message and preserves the user-led suffix, so the provider-native summary remains chronologically before retained messages.
  • Empty provider output is an error and cannot erase the session.

Config and deletion

  • context.compaction = "model_summary" | "provider_default"; model_summary is the default.
  • Selecting provider_default for an unsupported default provider fails during HalterBuilder::build.
  • Removes signal scoring and pruning, preserved-tail summary types, context.pre_compaction_target, context.prune_signal_threshold, and Provider::compaction_window.

Cohesive-review remediation

The follow-up commit 8fae7e3 remediates the findings from reviewing #197 and #198 together:

  • Complete context accounting: the ledger now includes prompt/skill/hook segments, tool declarations, per-message overhead, and media-only messages. It tracks request-base deltas without double-counting provider anchors.
  • Safe persistence and replay: versioned legacy/unknown ledgers rebuild from the transcript before use. ContextProjectionUpdated makes request-base changes replayable; compaction usage and logical-window advancement fold identically in live and replayed state.
  • Correct boundaries and caps: automatic compaction triggers exactly at the configured threshold; the hidden 100-token buffer is gone. Stop-hook and PostCompact hook context, plus request-only model-judge guidance, participate in cap checks.
  • Transactional strategies: Err and Ok(None) restore staged transcript, usage, events, and fired-hook state. Empty ModelSummary and ProviderDefault output cannot destroy context. Automatic no-op passes still complete the PreCompact/PostCompact hook pair.
  • Usage telemetry: ModelSummary inference and provider-native compaction usage reach SessionState::usage_so_far, TurnCompleted, compaction events, and replay.
  • CleanWindow-ready boundary API: threshold_notifications is replaced by context_boundary(CompactionBoundary), with session/window identity, previous/current counts, persisted per-window notification IDs, and Continue / Compact / Rollover directives. Successful rewrites advance the logical window and re-arm notifications.

Commit 96cb77a makes the boundary tests derive their thresholds from the session's actual prompt and compiled tool specs, keeping their intended ordering stable under both default and all-features builds.

Breaking changes

  • The default strategy changes from provider-native compaction to ModelSummary; select provider_default to retain the prior behavior.
  • Removed pruning config/types and Provider::compaction_window as described above.
  • CompactionStrategy::threshold_notifications is replaced by context_boundary; CompactionTrigger gains Rollover.
  • CompactionContext fields are private; use its accessors and action methods.
  • CompactionEffects and CompactionEventEffects gain usage.
  • SessionState gains context_window and compaction_notifications.
  • SessionEventPayload gains ContextProjectionUpdated; exhaustive matches need a new arm.
  • TokenLedger gains request-base/accounting-version fields; struct literals should use ..TokenLedger::default().
  • RuntimeServices gains context/strategy fields, ContextSettings gains max_tokens, and resolve_response_chain takes four arguments.

The full migration list is recorded in CHANGELOG.md.

Verification

  • PLAYWRIGHT_DRIVER_VERSION=1.59.1 cargo test --workspace — 937 unit/integration tests plus 5 doctests passed.
  • PLAYWRIGHT_DRIVER_VERSION=1.59.1 cargo test --workspace --all-features — 968 unit/integration tests plus 5 doctests passed.
  • PLAYWRIGHT_DRIVER_VERSION=1.59.1 cargo clippy --workspace --all-targets --all-features -- -D warnings — clean.
  • cargo fmt --all -- --check and git diff --check — clean.
  • Mutation checks deliberately broke six crucial assumptions: the exact-threshold comparison, PostCompact request-base refresh, logical-window replay comparison, compiled-tool contribution to test thresholds, the pre-tool one-token boundary gap, and ModelSummary headroom. Every predicted regression test failed, and every mutation was restored.

pbdeuchler and others added 4 commits September 4, 2026 12:49
Compaction v2, part 2 (#195). The runtime has known *when* to compact since
#194; this supplies *what happens*, makes it selectable from config, and
deletes the signal-pruning machinery it replaces. Net diff is negative.

- `ModelSummary` is the new default. When the `task` tool is registered it
  first nudges the model to persist its remaining work; only that reply's
  `task` calls run and reach the next request (text and other calls are
  dropped, though the event log keeps the whole reply, so no dangling call is
  ever sent back to the provider). It then sends the built-in checkpoint
  instructions and starts the next window from the reply alone, as one
  user-role message after the static prefix, with a todo reminder when the
  nudge ran calls.
- `ProviderDefault` delegates the rewrite to `Provider::compact` on Halter's
  trigger only and derives the window it may replace from the provider's
  `compaction_strategy`. Anthropic keeps the inline summarization request:
  the shipped `compact_20260112` context edit fires only on the server's own
  token trigger (50,000 minimum) and cannot be invoked on demand, which
  would hand the trigger to the server.
- `context.compaction = "model_summary" | "provider_default"`. Selecting
  `provider_default` for a default model whose provider cannot compact fails
  `HalterBuilder::build`, never the first compaction.
- `CompactionContext` is now a runtime-backed engine: `append`, `record`,
  `infer`, and `execute_tool_calls` run through the turn's own planner,
  prompt cache, hooks, policy, and event log. The turn loop shares
  `plan_and_assemble` and `run_provider_request` with it.
- Removed: `Provider::compaction_window` and every adapter impl, signal
  scoring and pruning, `MessageSignal`, `PruneSignalThreshold`,
  `CompactionWindow`, `SummarySlice`, `SessionState::summaries`,
  `ContextPlan::carried_summaries`, `context.pre_compaction_target`,
  `context.prune_signal_threshold`, and the interim `ProviderCompaction`.
- `default_compaction_prompt()` is now the context-checkpoint request; it
  ends by asking for plain text and no tool calls.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f6squKQimqeu4FLRvHPhT
Make context accounting complete and replayable, make strategy passes transactional, preserve provider chronology and usage, and replace stateless threshold notifications with per-window boundary directives. Add regressions for cap enforcement, rollback, replay, usage, exact thresholds, and CleanWindow rollover prerequisites.

Assisted-By: devx/fb87890b-8124-4c12-88e2-65b8e1dbaeca
Assisted-By: devx/fb87890b-8124-4c12-88e2-65b8e1dbaeca
…ger to the runtime

Review follow-ups on the compaction v2 remediation commits.

- A pass that appends and then fails or finds nothing to compact no longer
  truncates the event log or rolls back usage. Everything the pass did stays
  in the log and the session totals; the runtime records a `ContextRestored`
  event that puts the transcript window and ledger back, applied through the
  same fold transition replay uses. The manual `compact()` path commits the
  pass's events before surfacing its error.
- `context_boundary` returns notifications only. The runtime compacts on
  `ContextSettings::compaction_due`; `CompactionDirective`,
  `CompactionBoundaryResult`, and the `Rollover` trigger are gone.
- The ledger is prepared before the turn's boundary baseline is taken, so a
  fresh session's request base does not read as a crossing.
- Stale inline-window docs in the protocol crate and the fold's field list
  are corrected; README, runtime README, and CHANGELOG follow.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f6squKQimqeu4FLRvHPhT
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Compaction v2 (2/3): ModelSummary and ProviderDefault strategies, delete signal pruning (on strategy API #194)

1 participant