Skip to content

perf(registry): deduplicate manifests in batch queries - #3169

Open
daryllimyt wants to merge 2 commits into
mainfrom
daryl/eng-1575-registry-manifest-dedup
Open

perf(registry): deduplicate manifests in batch queries#3169
daryllimyt wants to merge 2 commits into
mainfrom
daryl/eng-1575-registry-manifest-dedup

Conversation

@daryllimyt

@daryllimyt daryllimyt commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

Summary

  • split batched registry action lookup into a metadata query followed by a distinct-version manifest query
  • apply origin and entitlement filtering before loading large manifest payloads
  • validate and reuse one manifest object for every selected action from the same registry version

Root cause

The batched action query joined each index row to the complete registry-version manifest. A representative request therefore transferred the same multi-megabyte JSON document once per action even though PostgreSQL execution itself was fast.

Impact

Manifest transfer and validation now scale with the number of distinct selected registry versions instead of the number of requested actions. Organization precedence, custom-registry entitlement filtering, and result behavior are preserved.

Validation

  • uv run pytest tests/unit/test_registry_actions_custom_entitlement.py -q — 6 passed
  • uv run ruff check and uv run ruff format --check on changed files
  • targeted uv run basedpyright — 0 diagnostics
  • full signed-commit hooks, including OpenAPI client generation and repository Python typechecking

LOC breakdown

Category + -
Logic 97 22
Tests 20 0

Tracking

ENG-1575


Summary by cubic

Deduplicates registry manifests in batched action queries and adds a safe retry when versions change mid-read. This reduces transfer and validation cost while preserving org precedence and entitlements; addresses ENG-1575.

  • Refactors
    • Split batch lookup into a lightweight metadata query, then fetch manifests by distinct registry version.
    • Filter by origin and required entitlements before loading manifest payloads.
    • Reuse one validated manifest per (source, version) and retry manifest lookup if a selected version is replaced between queries.
    • Added tests for manifest reuse and version-replacement retry.

Written for commit 87b0e98. Summary will update on new commits.

Review in cubic

@daryllimyt daryllimyt added api Improvements or additions to the backend API performance Changes that improve performance labels Jul 31, 2026 — with ChatGPT Codex Connector
@daryllimyt
daryllimyt marked this pull request as ready for review July 31, 2026 23:05
@zeropath-ai

zeropath-ai Bot commented Jul 31, 2026

Copy link
Copy Markdown

No security or compliance issues detected. Reviewed everything up to 87b0e98.

Security Overview
Detected Code Changes
Change Type Relevant files
Enhancement ► tracecat/registry/actions/service.py
    Update _ActionIndexRow docstring and introduce new _ActionMetadataRow, _VersionManifestRow data structures
► tracecat/registry/actions/service.py
    Modify get_actions_from_index to use new metadata/manifest loading flow and add retry logic
► tracecat/registry/actions/service.py
    Add _load_action_manifests method to batch-load manifests for actions
► tests/unit/test_registry_actions_custom_entitlement.py
    Extend tests to cover manifest reuse across actions and version replacement handling
► tests/unit/test_registry_actions_custom_entitlement.py
    Add imports and test scaffolding for new manifest loading behavior

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0f9604d6b3

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread tracecat/registry/actions/service.py
@daryllimyt
daryllimyt requested a review from jordan-umusu August 3, 2026 22:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api Improvements or additions to the backend API performance Changes that improve performance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants