Skip to content

fix(sync): enforce agent preset version immutability on replay - #3130

Open
daryllimyt wants to merge 9 commits into
mainfrom
daryl/ws-sync-preset-version-immutability
Open

fix(sync): enforce agent preset version immutability on replay#3130
daryllimyt wants to merge 9 commits into
mainfrom
daryl/ws-sync-preset-version-immutability

Conversation

@daryllimyt

@daryllimyt daryllimyt commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

Stacked on #3117 (daryl/ws-sync-custom-catalog).

Problem

_upsert_agent_preset_version unconditionally overwrote an existing exact-number AgentPresetVersion row's attributes — including catalog_id and base_url — and its caller then unconditionally replaced the version's skill bindings. Preset versions are pinned by consumers, and #3117 introduces per-pull catalog mapping choices, so replaying the same Git commit with a different mapping choice silently rewrote a historical version's provider routing for everything pinned to it.

Fix

  • Get-or-create-and-compare: an existing exact-number version is never written to. The desired state (version attributes normalized through _version_attrs_from_spec, resolved subagent config, and the skill-binding set) is compared against the stored state via a shared _version_changed_fields helper (_version_matches_preset now delegates to it, same contract).
    • Exact replay → zero writes; the existing row and bindings are returned untouched.
    • Different content → the pull fails with TracecatValidationError carrying detail.code = "immutable_agent_preset_version_conflict" plus the preset slug, version number, and the list of changed field names. The transaction rolls back, so the stored version keeps its original content.
    • New version numbers create-and-bind exactly as before.
  • Structured surfacing: the import transaction handler now merges TracecatException.detail into the failure diagnostic's details, so the conflict code reaches the pull result machine-readably instead of only as stringified text.

To intentionally change a version's provider routing, publish a new preset version; rebinding an existing pinned version would be a separate explicit feature.

Review guide

The invariant to check is in _upsert_agent_preset_version (tracecat/workspace_sync/adapters/agent_preset.py): the existing-row branch performs no attribute assignment, no flush, and no binding replacement. The comparison-normalization asymmetries (actions or None, default-filled model fields, resolved agents) are the false-positive risk; the regression tests exercise the real pull path twice specifically to catch those.

Tests

Four new regressions in tests/unit/test_workspace_sync_acceptance_contract.py, all through the real sync pull path:

  • test_agent_preset_exact_version_replay_is_no_op
  • test_agent_preset_conflicting_exact_version_replay_rolls_back
  • test_agent_preset_higher_version_replay_preserves_existing_version
  • test_agent_preset_defaulted_exact_version_replay_is_no_op (normalization/defaults round-trip)

Suite: 109 passed. Ruff, ruff format, and basedpyright (--warnings, including the test file) are clean.


Summary by cubic

Prevents accidental mutation of existing exact-number AgentPresetVersion on sync replay by enforcing immutability and surfacing structured conflicts. Exact replays are no-ops; conflicting replays fail and roll back safely.

  • Bug Fixes
    • Do not mutate existing exact-number versions: compare normalized version attributes and skill bindings; create only for new numbers.
    • On content mismatch, raise TracecatValidationError with code immutable_agent_preset_version_conflict including preset slug, version, and changed fields; transaction rolls back.
    • Exact replay performs zero writes and preserves existing bindings.
    • Pull diagnostics now include TracecatException.detail for machine-readable error details.

Written for commit dfc6b53. Summary will update on new commits.

Review in cubic

- Compute dry-run pull diffs from the correlated snapshot so an
  already-applied cross-environment preset no longer previews as a
  spurious modification.
- Clear the deployment-local base_url when remapping a catalog UUID so
  the target's provider credentials are never routed to the source
  deployment's endpoint (openai/anthropic honor preset base_url at the
  gateway; custom providers re-resolve it from local provider config).
Add set-based AgentCatalogService.enabled_catalog_ids and
resolve_catalog_ids_by_models with identical visibility, enablement,
org-over-platform precedence, and id tie-break semantics as the
single-item resolvers, and rewire agent preset correlation to resolve
all incoming catalog UUIDs and model tuples in at most three round
trips (override scope check, enabled-id set, model resolution) instead
of up to four queries per distinct key.
Replace the bare tuples in agent preset catalog correlation with named
types, matching the PreparedSnapshot pattern:

- CorrelatedAgentPresets for the (presets, diagnostics) return of
  AgentPresetAdapter.correlate_catalog_ids
- ModelKey for the (model_provider, model_name) pair used to resolve
  deployment-local catalog ids

Both are NamedTuples, so existing tuple unpacking and dict lookups with
plain tuples are unaffected.
Quality-only cleanup of the agent preset catalog correlation path; no
behavior change.

- Drop the unused resolve_catalog_ids_by_models batch resolver, whose
  query duplicated catalog_candidates_by_models.
- Collapse CatalogMatchCandidate into CatalogMappingCandidate so the
  catalog service and sync vocabulary share one type.
- Add _failed_pull_result to replace three near-identical failure
  results, and drop the always-empty diagnostics splat in the dry run.
- Reduce the preset rewrite loop to two branches and skip rebuilding
  the spec when no catalog id is remapped.
- Precompute reference paths and drop the redundant catalog id field.
- Memoize pull tab selection state and make candidate label
  disambiguation a single pass.
- Extract catalog-enabling and YAML patch helpers in the sync tests.
@daryllimyt daryllimyt added fix Bug fix agents LLM agents labels Jul 25, 2026
@zeropath-ai

zeropath-ai Bot commented Jul 25, 2026

Copy link
Copy Markdown

No security or compliance issues detected. Reviewed everything up to dfc6b53.

Security Overview
Detected Code Changes
Change Type Relevant files
Refactor ► tracecat/workspace_sync/adapters/agent_preset.py
     Modify import_specs flow to adjust version upsert logic and bindings
► tracecat/workspace_sync/service.py
     Add enhanced exception detail handling in _import_snapshot
Bug Fix / Enhancement ► tracecat/workspace_sync/adapters/agent_preset.py
     Introduce _version_changed_fields and adjust _version_matches_preset to use it
► tracecat/workspace_sync/adapters/agent_preset.py
     Update _upsert_agent_preset_version to enforce immutability with detailed conflict reporting and integrate skill_target handling
► tracecat/workspace_sync/adapters/agent_preset.py
     Adjust _replace_version_skill_bindings usage timing and ensure bindings align with upserted version
► tracecat/workspace_sync/service.py
     Enhance error handling in _import_snapshot to propagate detailed exception info

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: dfc6b53f43

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

changed_fields = [
key
for key, desired_value in desired_attrs.items()
if getattr(version, key) != desired_value

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Normalize unordered fields before comparing versions

When a preset version was initially imported with noncanonical ordering (for example actions: ["z", "a"], similarly namespaces, mcp_integrations, or subagents), this direct list equality reports different content after a normal export/import round trip: _version_specs_for_preset and _subagent_refs sort these collections during projection, while _version_attrs_from_spec and _resolved_subagents_config preserve the incoming order when storing them. Pulling the projected commit therefore raises immutable_agent_preset_version_conflict and rolls back even though the version is semantically unchanged; normalize these unordered collections before insertion/comparison.

Useful? React with 👍 / 👎.

@jordan-umusu
jordan-umusu force-pushed the daryl/ws-sync-custom-catalog branch 2 times, most recently from 80dbcc7 to 89321fa Compare July 27, 2026 22:44
Base automatically changed from daryl/ws-sync-custom-catalog to main July 28, 2026 16:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents LLM agents fix Bug fix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant