Skip to content

feat(audit): audit human approval decisions - #3162

Draft
topher-lo wants to merge 1 commit into
mainfrom
agent/approval-audit-events
Draft

feat(audit): audit human approval decisions#3162
topher-lo wants to merge 1 commit into
mainfrom
agent/approval-audit-events

Conversation

@topher-lo

@topher-lo topher-lo commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Summary

  • emit one enriched agent_approval audit webhook event per accepted or rejected tool decision
  • attribute authenticated Tracecat UI/API decisions with actor, request context, decision time, source, tool identifiers, and optional sanitized denial reason
  • keep audit configuration, enrichment, deduplication, and webhook I/O outside the Temporal approval response path
  • deduplicate concurrent/retried submissions with an opaque ID derived from the stable Temporal update identity
  • deliver each batch through one HTTP client with four-request concurrency and a ten-second total deadline

Security and performance boundaries

  • never include tool arguments, override values, prompts, credentials, or tool outputs
  • omit Slack/service-originated decisions in this first PR; external actor attribution will follow separately
  • preserve best-effort audit delivery semantics and shed work at bounded in-process queue limits
  • expand the existing pending-approval query projection for audit-safe identifiers without adding a synchronous database query

Testing

  • uv run ruff check ...
  • uv run ruff format --check ...
  • uv run basedpyright --warnings --threads 4
  • uv run pytest tests/unit/test_agent_approval_audit.py -q --confcutdir=tests/unit (9 passed)
  • focused batch delivery tests in tests/unit/test_audit_service.py (3 passed)
  • signed commit hooks, including OpenAPI client generation and Python type checking

The normal database-backed suite could not run locally because Docker/PostgreSQL was unavailable; the DB-independent regression tests above passed.


Summary by cubic

Adds audit logging for human approval decisions in the agent and introduces batched webhook delivery to keep the approval path fast and reliable.

  • New Features

    • Emit one agent_approval webhook event per accepted or rejected tool call from the UI/API; Slack/service-originated decisions are excluded for now.
    • Event data includes actor label, IP/UA, decision time, source, tool identifiers, and sanitized optional denial reason; never includes tool args, override values, prompts, credentials, or outputs.
    • Deduplicate with a stable per-update dedupe_id and Redis (24h TTL); fall back to best-effort if Redis is unavailable.
    • Run enrichment and delivery off the request path with a bounded in-process queue; batch delivery uses one httpx client, up to 4 concurrent posts, and a 10s batch deadline.
  • Refactors

    • AuditService now supports batched enrichment via AuditEventInput and create_events, resolving webhook/actor/headers once and streaming through spawn_audit_batch.
    • Introduced tracecat/audit/batch.py for grouped delivery (AuditBatchDelivery, deliver_audit_batch) with pending limits.
    • Expanded the pending-approval query to include audit-safe identifiers (approval ID, tool name) and updated sanitization to allow safe keys (e.g., arguments_overridden, decision) while stripping secrets from denial_reason.

Written for commit 23f9281. Summary will update on new commits.

Review in cubic

@topher-lo topher-lo added agents LLM agents logging security Security related issue labels Jul 30, 2026
topher-lo added a commit that referenced this pull request Jul 31, 2026
Add detail pages for platform, organization agent, and workspace MCP
access logs, and expand the organization audit page for detection
engineers with attribution, scope, approval, and privacy detail.

Rewrite the security architecture page to lead each control with the
concrete threat it answers, and correct the Mermaid diagram: MCP access
activity records inbound external MCP client calls, not outbound proxy
calls. Section order is unchanged.

Agent OTel and agent_approval content follows the contracts in #2638,
#2649, #2650, and #3162, which are not yet merged.
@topher-lo topher-lo changed the title feat(agent): audit human approval decisions feat(audit): audit human approval decisions Aug 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents LLM agents logging security Security related issue

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant