Skip to content

fix(agent-server): base_state.json as single source of truth for the agent (end meta.json duplication) - #26

Open
enyst wants to merge 3 commits into
mainfrom
fix/single-source-of-truth-agent-state-2026-08-09
Open

enyst wants to merge 3 commits into
mainfrom
fix/single-source-of-truth-agent-state-2026-08-09

Conversation

@enyst

@enyst enyst commented Aug 9, 2026

Copy link
Copy Markdown
Owner

What & why

The agent-server persisted a conversation's agent (LLM + condenser + tools) in two files:

  • base_state.json (ConversationState), and
  • meta.json (StoredConversation, which extended StartConversationRequest).

On resume the agent was rebuilt from meta.json and assigned over the state loaded from base_state.json — so meta.json silently won. A model switch written to one file but not the other was reverted on an idle-eviction reload. This is the class of bug Neal reported in Slack (#general), and matches the earlier "two sources of truth" analysis.

Design writeup (a/b/c alternatives + the resume/reattach edge): https://enyst.github.io/arch/meta-vs-base-state-alternatives.html

Approach (option c): remove the duplication at the root

  • SDK: extract ConversationConfig (everything except the agent) as a shared base. StartConversationRequest adds the agent; StoredConversation now extends the agent-less ConversationConfig, so the agent cannot appear in meta.json by construction.
  • SDK: ConversationState.create() and LocalConversation accept agent=None; on resume the persisted base_state.json agent is kept. Passing an explicit agent keeps the legacy verify-and-override behavior (back-compat).
  • agent-server: EventService takes the new-conversation agent separately and, on resume, loads it from base_state.json. switch_acp_model no longer mirrors the model into meta.json (the SDK already persists it to base_state); the credential scrub and codex detection read the agent from base_state / the live conversation; telemetry reads the live agent.

base_state.json is now the single source of truth for the agent.

Compatibility

Old meta.json files that still contain an agent key load fine — Pydantic ignores unknown keys — so no migration is needed.

Tests

  • New SDK regression tests: base_state-authoritative resume (agent=None keeps the persisted agent; a durable switch survives reload), explicit-agent override still works, new conversation still requires an agent.
  • New end-to-end agent-server test: meta.json has no agent, and a fresh ConversationService (restart) reloads the agent from base_state.json.
  • Full tests/agent_server suite updated to the new API and green (1980 passing locally). Ruff + pyright clean.

Reviewer note ⚠️

The ACP / Codex-subscription persistence paths (switch_acp_model, credential scrub, codex detection) are re-homed and covered at unit level only — I could not exercise them against a live ACP/Codex session. Please give those a live check before merge.

Co-authored-by: smolpaws engel@enyst.org

Summary by CodeRabbit

  • Improvements
    • Conversation agents are reliably preserved when conversations are paused, resumed, forked, or restarted.
    • Resumed conversations can restore their saved agent or accept a compatible replacement.
    • Credentials and encrypted secrets continue to resolve and scrub correctly.
    • Agent and model changes remain consistent across ACP model switching.
    • Streaming callbacks are enabled only when supported by the resumed agent.
  • Bug Fixes
    • Improved conversation startup validation when an agent is missing or incompatible.
    • Fixed agent restoration after service restarts.

…r the agent

The agent-server persisted a conversation's agent (LLM + condenser + tools)
in TWO files: base_state.json (ConversationState) and meta.json
(StoredConversation, which extended StartConversationRequest). On resume the
agent was rebuilt from meta.json and overwrote base_state.json, so meta.json
silently won. A model switch written to one file but not the other was
reverted on an idle-eviction reload.

This removes the duplication at its root:

- SDK: extract ConversationConfig (everything except the agent) as the shared
  base. StartConversationRequest adds the agent; StoredConversation now extends
  the agent-less ConversationConfig, so the agent cannot appear in meta.json by
  construction.
- SDK: ConversationState.create() and LocalConversation accept agent=None; on
  resume the persisted base_state.json agent is kept (a durable switch_llm/
  switch_acp_model survives reload). Passing an explicit agent keeps the legacy
  verify-and-override behavior for back-compat.
- agent-server: EventService takes the new-conversation agent separately and,
  on resume, loads it from base_state.json. switch_acp_model no longer mirrors
  the model into meta.json (the SDK persists it to base_state); the credential
  scrub and codex detection read the agent from base_state / the live
  conversation; telemetry reads the live agent.

Old meta.json files with an 'agent' key still load (unknown keys are ignored),
so no migration is needed.

Adds regression coverage: base_state-authoritative resume at the SDK level, and
an end-to-end check that meta.json has no agent and a fresh service reloads the
agent from base_state.json.

Note: the ACP/Codex-subscription persistence paths are covered at unit level
only; they were not exercised against a live ACP/Codex session.

Co-authored-by: smolpaws <engel@enyst.org>
@coderabbitai

coderabbitai Bot commented Aug 9, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: fcdc49c5-03d6-4bdc-8e62-cf7e6e728bb9

📥 Commits

Reviewing files that changed from the base of the PR and between 0c8dbb6 and e718eda.

⛔ Files ignored due to path filters (1)
  • uv.lock is excluded by !**/*.lock
📒 Files selected for processing (4)
  • openhands-agent-server/pyproject.toml
  • openhands-sdk/pyproject.toml
  • openhands-tools/pyproject.toml
  • openhands-workspace/pyproject.toml

📝 Walkthrough

Walkthrough

Agents are no longer stored in meta.json. Conversation state in base_state.json is authoritative for resumed conversations. New conversations receive agents explicitly, and related credential, fork, telemetry, streaming, ACP, and test paths were updated.

Changes

Agent state migration

Layer / File(s) Summary
Shared configuration and resume contracts
openhands-sdk/.../conversation/request.py, openhands-sdk/.../conversation/state.py, openhands-sdk/.../conversation/impl/local_conversation.py, openhands-agent-server/.../models.py, tests/sdk/conversation/test_base_state_single_source.py
Introduces agent-less ConversationConfig. StoredConversation uses it. Resume operations preserve persisted agents or apply compatible replacements.
Server startup and persistence wiring
openhands-agent-server/.../conversation_service.py, tests/agent_server/test_conversation_service.py, tests/agent_server/test_agent_*.py, tests/agent_server/test_conversation_service_plugin.py, tests/agent_server/test_conversation_tags.py
New conversations pass agents explicitly. Resumed conversations load agents from base_state.json. Forks, credential resolution, profile startup, worktrees, and telemetry use the updated flow.
Runtime capabilities and state updates
openhands-agent-server/.../event_service.py, tests/agent_server/test_event_service.py, tests/agent_server/test_credential_binding.py, tests/agent_server/test_event_streaming.py, tests/agent_server/test_webhook_subscriber.py, tests/agent_server/test_goal_loop.py, tests/agent_server/telemetry/test_telemetry_subscriber.py
Event service startup validates agent sources. Credential scrubbing, token streaming detection, ACP model persistence, and telemetry use live or persisted conversation agents. Tests remove agent mirrors from meta.json.
Package version updates
openhands-agent-server/pyproject.toml, openhands-sdk/pyproject.toml, openhands-tools/pyproject.toml, openhands-workspace/pyproject.toml
Package versions change from 1.41.0 to 1.42.0.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Client
  participant ConversationService
  participant EventService
  participant LocalConversation
  participant ConversationState
  participant base_state.json

  Client->>ConversationService: start or resume conversation
  ConversationService->>EventService: pass new agent or no agent
  EventService->>LocalConversation: create conversation
  LocalConversation->>ConversationState: restore or apply agent
  ConversationState->>base_state.json: load persisted agent
  ConversationState-->>EventService: active conversation agent
  EventService-->>Client: started conversation
Loading

Suggested reviewers: all-hands-bot

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 60.55% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly identifies the agent-server change to make base_state.json the single source of truth and remove meta.json duplication.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/single-source-of-truth-agent-state-2026-08-09

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Aug 9, 2026

Copy link
Copy Markdown

REST API breakage checks (OpenAPI) — ✅ PASSED

Result:PASSED

Action log

@github-actions

github-actions Bot commented Aug 9, 2026

Copy link
Copy Markdown

Python API breakage checks — ❌ FAILED

Result:FAILED

⚠️ Breaking API changes or policy violations detected.

Log excerpt (first 1000 characters)

============================================================
Checking openhands-sdk (openhands.sdk)
============================================================
Comparing openhands-sdk 1.41.0 against 1.41.0
::warning file=openhands-sdk/openhands/sdk/conversation/impl/local_conversation.py,line=370,title=LocalConversation.agent::Attribute value was changed: `agent` -> `self._state.agent`
::error title=SemVer::Breaking changes detected (1); require at least minor version bump from 1.41.x, but new is 1.41.0

============================================================
Checking openhands-workspace (openhands.workspace)
============================================================
Comparing openhands-workspace 1.41.0 against 1.41.0
No breaking changes detected

============================================================
Checking openhands-tools (openhands.tools)
============================================================
Comparing openhands-tools 1.41.0 against 1.41.0
No breaking changes 

Action log

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
openhands-sdk/openhands/sdk/conversation/impl/local_conversation.py (1)

321-343: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Make resume reject or handle explicit client_tools safely.

When LocalConversation resumes with agent=None, the new client-tool handling is skipped and the caller-supplied non-empty client_tools are not registered or copied into the resumed agent. Document this gap and add a test; or run the tool registration/injection against self._state.agent after state creation so client_tools apply even on resume.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@openhands-sdk/openhands/sdk/conversation/impl/local_conversation.py` around
lines 321 - 343, Update LocalConversation resume handling so explicit non-empty
client_tools are registered and injected into the resumed self._state.agent,
rather than being silently skipped when agent is None. Reuse the existing
register_client_tools and duplicate-name filtering behavior, and add a test
covering resume with caller-supplied client_tools.
🧹 Nitpick comments (2)
tests/agent_server/test_conversation_service.py (1)

87-119: 🗄️ Data Integrity & Integration | 🔵 Trivial | ⚡ Quick win

Add a companion test for legacy meta.json (with an agent key) compatibility.

This test proves a newly started conversation writes an agent-free meta.json. It does not prove the PR's other explicit claim: that a pre-existing meta.json file which still contains an agent key (written by older server versions) loads without error and without resurrecting the agent on StoredConversation.

Add a test that writes a meta.json file by hand with an agent key present (plus a matching base_state.json), then constructs a ConversationService and calls get_conversation()/search_conversations() to confirm it loads successfully and the agent comes from base_state.json, not from the stale agent key in meta.json.

As per coding guidelines, "Write just enough tests to cover edge cases" — this is exactly the kind of edge case (backward-compatible on-disk format) the guideline calls out for coverage.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/agent_server/test_conversation_service.py` around lines 87 - 119, Add a
companion async test near test_meta_json_has_no_agent_and_reload_uses_base_state
that manually creates legacy meta.json containing a stale agent key alongside
matching base_state.json, then initializes ConversationService and exercises
get_conversation and/or search_conversations. Assert loading succeeds and
StoredConversation uses the agent from base_state.json rather than the stale
meta.json agent.

Source: Coding guidelines

openhands-sdk/openhands/sdk/conversation/impl/local_conversation.py (1)

201-201: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Update the docstring for the new agent=None semantics.

The Args: entry for agent still reads "The agent to use for the conversation." It does not mention that agent=None now means "resume the persisted agent from base_state.json," and that a fresh conversation still requires a non-None agent (enforced by ConversationState.create). Document this so SDK consumers understand the new contract without reading the implementation.

📝 Suggested docstring update
         Args:
-            agent: The agent to use for the conversation.
+            agent: The agent to use for the conversation. Pass ``None`` to
+                resume with the agent already persisted in ``base_state.json``
+                (the single source of truth on resume). Required (non-``None``)
+                when creating a brand-new conversation.
             workspace: Working directory for agent operations and tool execution.

Also applies to: 237-237

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@openhands-sdk/openhands/sdk/conversation/impl/local_conversation.py` at line
201, Update the Args documentation for the agent parameter in the local
conversation constructor to state that None resumes the persisted agent from
base_state.json, while fresh conversations require a non-None agent as enforced
by ConversationState.create.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@openhands-agent-server/openhands/agent_server/models.py`:
- Around line 77-87: Update the StoredConversation compatibility behavior:
either configure StoredConversation (or its ConversationConfig base) to ignore
unknown fields so legacy meta.json files containing agent validate successfully,
or remove/replace the docstring claim that legacy compatibility is supported. Do
not restore an agent field.

In `@openhands-sdk/openhands/sdk/conversation/state.py`:
- Line 449: Update the create method docstring around the agent parameter to
document both behaviors: new conversations require an agent, while resume with
agent=None preserves the persisted agent and an explicitly provided agent
restores with that agent. Revise the “New conversation,” “Restored
conversation,” “Args,” and “Raises” sections accordingly, including the
applicable validation conditions.

In `@tests/agent_server/test_agent_profile_conv_start.py`:
- Around line 725-727: Rename the unused first unpacked value from stored to
_stored in both _start_from_profile calls around the affected test cases,
including the second occurrence, while leaving the agent assertions and other
behavior unchanged.

---

Outside diff comments:
In `@openhands-sdk/openhands/sdk/conversation/impl/local_conversation.py`:
- Around line 321-343: Update LocalConversation resume handling so explicit
non-empty client_tools are registered and injected into the resumed
self._state.agent, rather than being silently skipped when agent is None. Reuse
the existing register_client_tools and duplicate-name filtering behavior, and
add a test covering resume with caller-supplied client_tools.

---

Nitpick comments:
In `@openhands-sdk/openhands/sdk/conversation/impl/local_conversation.py`:
- Line 201: Update the Args documentation for the agent parameter in the local
conversation constructor to state that None resumes the persisted agent from
base_state.json, while fresh conversations require a non-None agent as enforced
by ConversationState.create.

In `@tests/agent_server/test_conversation_service.py`:
- Around line 87-119: Add a companion async test near
test_meta_json_has_no_agent_and_reload_uses_base_state that manually creates
legacy meta.json containing a stale agent key alongside matching
base_state.json, then initializes ConversationService and exercises
get_conversation and/or search_conversations. Assert loading succeeds and
StoredConversation uses the agent from base_state.json rather than the stale
meta.json agent.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 62d04be1-67b0-454b-9a10-c6ac2e2c55a4

📥 Commits

Reviewing files that changed from the base of the PR and between be6cd3b and 0c8dbb6.

📒 Files selected for processing (21)
  • openhands-agent-server/openhands/agent_server/conversation_service.py
  • openhands-agent-server/openhands/agent_server/event_service.py
  • openhands-agent-server/openhands/agent_server/models.py
  • openhands-sdk/openhands/sdk/conversation/impl/local_conversation.py
  • openhands-sdk/openhands/sdk/conversation/request.py
  • openhands-sdk/openhands/sdk/conversation/state.py
  • tests/agent_server/telemetry/test_telemetry_disabled_by_default.py
  • tests/agent_server/telemetry/test_telemetry_subscriber.py
  • tests/agent_server/test_agent_launch_additions.py
  • tests/agent_server/test_agent_profile_conv_start.py
  • tests/agent_server/test_auto_title_span_metadata.py
  • tests/agent_server/test_conversation_info_model.py
  • tests/agent_server/test_conversation_service.py
  • tests/agent_server/test_conversation_service_plugin.py
  • tests/agent_server/test_conversation_tags.py
  • tests/agent_server/test_credential_binding.py
  • tests/agent_server/test_event_service.py
  • tests/agent_server/test_event_streaming.py
  • tests/agent_server/test_goal_loop.py
  • tests/agent_server/test_webhook_subscriber.py
  • tests/sdk/conversation/test_base_state_single_source.py
💤 Files with no reviewable changes (3)
  • tests/agent_server/test_auto_title_span_metadata.py
  • tests/agent_server/test_conversation_info_model.py
  • tests/agent_server/telemetry/test_telemetry_disabled_by_default.py

Comment on lines +77 to 87
class StoredConversation(ConversationConfig):
"""Stored details about a conversation.

Extends StartConversationRequest with server-assigned fields.
Extends :class:`ConversationConfig` (the agent-less shared config) with
server-assigned fields. It deliberately does NOT carry the ``agent``: the
single source of truth for the agent / runtime state is
``ConversationState`` persisted to ``base_state.json``. Because
``StoredConversation`` is not a ``StartConversationRequest``, the agent
cannot silently re-appear in ``meta.json``.
"""

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "--- model_config on ConversationConfig / StoredConversation ---"
rg -n "model_config" openhands-sdk/openhands/sdk/conversation/request.py openhands-agent-server/openhands/agent_server/models.py

echo "--- any extra= override in the SDK/agent-server that could apply to these classes via inheritance ---"
rg -n "extra\s*=\s*['\"](forbid|allow|ignore)['\"]" openhands-sdk openhands-agent-server -g '*.py'

echo "--- class hierarchy for ConversationConfig / OpenHandsModel base ---"
rg -n "class ConversationConfig|class OpenHandsModel" -A 5 openhands-sdk -g '*.py'

echo "--- existing tests referencing a legacy meta.json with an agent key ---"
rg -n "meta.json" tests/agent_server/test_conversation_service.py | rg -n "agent"

Repository: enyst/agent-sdk

Length of output: 6191


Reject legacy meta.json compatibility or restore an ignored agent field.

StoredConversation inherits ModelConfig(extra="forbid") from ConversationConfig, so StoredConversation.model_validate_json() will reject an old meta.json that still contains an agent key. If the compatibility objective remains, add extra="ignore" on StoredConversation or ConversationConfig; otherwise remove/replace that compatibility claim.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@openhands-agent-server/openhands/agent_server/models.py` around lines 77 -
87, Update the StoredConversation compatibility behavior: either configure
StoredConversation (or its ConversationConfig base) to ignore unknown fields so
legacy meta.json files containing agent validate successfully, or remove/replace
the docstring claim that legacy compatibility is supported. Do not restore an
agent field.

cls: type["ConversationState"],
id: ConversationID,
agent: AgentBase,
agent: AgentBase | None,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Update the create API documentation for agent=None.

agent=None is now valid during resume and retains the persisted agent. The docstring still describes agent as required and always used for restore. Update the “New conversation,” “Restored conversation,” Args, and Raises sections to state the two behaviors.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@openhands-sdk/openhands/sdk/conversation/state.py` at line 449, Update the
create method docstring around the agent parameter to document both behaviors:
new conversations require an agent, while resume with agent=None preserves the
persisted agent and an explicitly provided agent restores with that agent.
Revise the “New conversation,” “Restored conversation,” “Args,” and “Raises”
sections accordingly, including the applicable validation conditions.

Comment on lines +725 to 727
stored, agent = await _start_from_profile(
tmp_path, profile, resolved_settings, persisted
)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Mark the unused stored conversation value as intentionally unused.

stored is not read after either unpacking operation. Rename it to _stored to satisfy Ruff RUF059 and show that these tests validate only the resolved agent.

Proposed fix
-        stored, agent = await _start_from_profile(
+        _stored, agent = await _start_from_profile(
             tmp_path, profile, resolved_settings, persisted
         )

Apply the same change at Line 756.

Also applies to: 756-758

🧰 Tools
🪛 Ruff (0.16.1)

[warning] 725-725: Unpacked variable stored is never used

Prefix it with an underscore or any other dummy variable pattern

(RUF059)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/agent_server/test_agent_profile_conv_start.py` around lines 725 - 727,
Rename the unused first unpacked value from stored to _stored in both
_start_from_profile calls around the affected test cases, including the second
occurrence, while leaving the agent assertions and other behavior unchanged.

Source: Linters/SAST tools

enyst and others added 2 commits August 9, 2026 23:26
…ange

Removing the agent field from StoredConversation (it no longer extends
StartConversationRequest) is a breaking API change, which the api-breakage
check requires a minor version bump for.

Co-authored-by: smolpaws <engel@enyst.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant