test(registry): drop third-party contract tests - #3205
Conversation
topher-lo
added
integrations
|
✅ No security or compliance issues detected. Reviewed everything up to 14a1f36. Security Overview
Detected Code ChangesThe diff is too large to display a summary of code changes. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ac3121097a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 34c84abf64
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 14a1f36e36
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| with pytest.raises(SecretNotFoundError, match="one auth source"): | ||
| okta_sdk._build_okta_config() |
There was a problem hiding this comment.
When an Okta action has a base URL but no Tracecat-managed authentication source, this is the only regression test ensuring _build_okta_config() raises before constructing the third-party client, where SDK configuration could otherwise resolve credentials outside Tracecat's secret boundary. A repo-wide search found no replacement coverage, so retain this narrow missing-auth test rather than removing it with the provider-contract assertions.
Useful? React with 👍 / 👎.
1 participant
Summary
packages/tracecat-registry/AGENTS.mdpackages/tracecat-registry/CLAUDE.mdas a symlink toAGENTS.mdThis PR makes no runtime integration or public API changes. Existing integration contracts remain authoritative until an explicit migration is planned.
Third-party integration policy
The registry guidance now requires endpoint-specific research against official documentation, OpenAPI specifications when available, and official SDK or MCP schemas. It prefers direct YAML REST templates, requires a planning choice before adopting an official Python SDK, and keeps integrations close to pass-through wrappers.
The policy also covers API-native inputs, secrets and OAuth, base-URL precedence, encoded path segments, raw HTTP response envelopes, bounded pagination, negative polling predicates, narrow permitted boundary handling, and the prohibition on provider-specific white-box tests. Live or sandbox coverage remains an explicit planning choice when a reliable environment exists.
The test exception is limited to Tracecat-owned platform security or protocol boundaries—for example credential isolation, preventing host filesystem or subprocess access, blocking ambient credential discovery, network-target restrictions, and shared protocol machinery. Provider-local dispatch, validation, pagination, and serialization do not qualify merely because Tracecat implements them.
Current integration-conformance audit
Note
This snapshot is advisory, non-exhaustive, and intentionally out of scope for remediation in this PR. Counts describe the current catalog; they do not imply that every historical implementation is defective. Existing public inputs and outputs must not be changed without an explicitly planned migration.
.result.dataand 160 return the full result envelope. The remaining templates use other return shapes.core.script.run_python, and 17 usecore.transform.*. These need case-by-case review because small mechanical request assembly remains permitted.status_code != 202predicate illustrates the new negative/transient-state rule.base_url, while 312 use the input-to-workspace-variable fallback.found/not_foundaggregation; Cloudflare validates dispatch names and flattens SDK page objects. Some of these behaviors may be justified as credentials/security, SDK serialization, or bounded pagination, but each should be assessed against the new thin-wrapper rules when materially changed.Verification
PG_PORT=5832 uv run pytest -q -r fE tests/registry/test_templates.py tests/registry/test_kubernetes_sdk.py tests/unit/test_aws_assume_role.py -n auto— 706 passedPG_PORT=5832 uv run pytest -q tests/unit/test_aws_assume_role.py tests/unit/test_dsl_common.py -n auto— 16 passeduv run basedpyright tests/unit/test_dsl_common.py— 0 errors, 0 warningsuv run ruff check .uv run ruff format --check .git diff --checkreadlink packages/tracecat-registry/CLAUDE.md—AGENTS.mdLOC breakdown