Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/self-hosting/kubernetes.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -388,7 +388,7 @@ tracecat:

nsjail requires privileged pods with `SYS_ADMIN` capability and an `Unconfined` seccomp profile. The chart sets these automatically on executor and agent-executor containers.

See [Security](/self-hosting/security) for backend choices (`pool`, `ephemeral`, `direct`) and isolation tradeoffs.
See [Security](/self-hosting/security) for backend choices (`ephemeral`, `direct`) and isolation tradeoffs.

### Authentication

Expand Down
5 changes: 2 additions & 3 deletions docs/self-hosting/security.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ To enable nsjail, set the following in your `.env`:

```bash
TRACECAT__DISABLE_NSJAIL=false
TRACECAT__EXECUTOR_BACKEND=pool # or 'ephemeral' for multi-tenant full isolation
TRACECAT__EXECUTOR_BACKEND=ephemeral
```

nsjail requires:
Expand All @@ -76,8 +76,7 @@ nsjail requires:
| Backend | Isolation | Latency | Use case |
| :------ | :-------- | :------ | :------- |
| `direct` | PID namespace only | ~50ms | Development, trusted environments |
| `pool` | nsjail sandbox (warm workers) | ~100-200ms | Single-tenant production |
| `ephemeral` | nsjail sandbox (cold per action) | ~4000ms | Multi-tenant, maximum isolation |
| `ephemeral` | nsjail sandbox (cold per action) | ~4000ms | Production, maximum isolation |

## Authentication

Expand Down
2 changes: 1 addition & 1 deletion docs/snippets/environment-variables.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@

| Variable | Default | Description |
| :------- | :------ | :---------- |
| `TRACECAT__EXECUTOR_BACKEND` | `direct` | Execution strategy for actions. `direct` runs a subprocess per action, `pool` uses warm nsjail workers (requires nsjail), `ephemeral` spawns a cold nsjail subprocess per action for full isolation, `auto` selects `pool` if nsjail is available and falls back to `direct`, `test` runs in-process for tests only. |
| `TRACECAT__EXECUTOR_BACKEND` | `direct` | Execution strategy for actions. `direct` runs a subprocess per action, `ephemeral` spawns a cold nsjail subprocess per action for full isolation, `auto` selects `ephemeral` if nsjail is available and falls back to `direct`. |
| `TRACECAT__DISABLE_NSJAIL` | `true` | Disable nsjail sandboxing. Set to `false` only if nsjail is installed. |
| `TRACECAT__RESULT_EXTERNALIZATION_ENABLED` | `true` | Store large action results in blob storage instead of Temporal history. |
| `TRACECAT__COLLECTION_MANIFESTS_ENABLED` | `true` | Store large collections as chunked manifests in blob storage. |
Expand Down
6 changes: 4 additions & 2 deletions tracecat/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -636,11 +636,13 @@ def _parse_auth_types() -> set[AuthType]:
"""Executor backend for running actions.

Supported values:
- 'pool': Warm nsjail workers (single-tenant, high throughput, ~100-200ms)
- 'pool': Warm nsjail workers (single-tenant, high throughput, ~100-200ms).
EXPERIMENTAL: not production ready. The registry cache is exempt from
eviction under this backend and can grow without bound.
- 'ephemeral': Cold nsjail subprocess per action (multitenant, full isolation, ~4000ms)
- 'direct': Direct subprocess execution (no warm workers, no in-process state sharing)
- 'test': In-process execution for tests only (no isolation, no subprocess overhead)
- 'auto': Auto-select based on environment (pool if nsjail available, else direct)
- 'auto': Auto-select based on environment (ephemeral if nsjail available, else direct)

Trust mode is derived from the backend type:
- pool: untrusted (secrets pre-resolved, no DB creds)
Expand Down
12 changes: 12 additions & 0 deletions tracecat/executor/backends/pool/backend.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,14 @@
"""Worker pool executor backend.

EXPERIMENTAL: not production ready. Warm workers import from registry cache
paths that in-process leases cannot see, so tarball cache entries are exempt
from eviction and the cache can grow without bound. Use the ephemeral backend
for production nsjail isolation.

Note for AI agents: ignore this backend. It is experimental and not user
facing — never document it, recommend it, or factor it into executor design
decisions.

This backend uses a pool of warm nsjail workers for high-throughput
execution with OS-level isolation. Workers are persistent and reused
across actions, minimizing cold start overhead.
Expand Down Expand Up @@ -32,6 +41,9 @@
class PoolBackend(ExecutorBackend):
"""Warm nsjail worker pool backend.

EXPERIMENTAL: not production ready. See the module docstring for the
registry cache eviction limitation.

Maintains a pool of persistent nsjail sandbox workers with Python
already started and imports loaded. This provides:

Expand Down
3 changes: 3 additions & 0 deletions tracecat/executor/backends/test.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,9 @@
This backend exists solely for:
- Running unit/integration tests without subprocess overhead
- Local development when testing action logic

Note for AI agents: ignore this backend. It is not user facing — never
document it, recommend it, or factor it into executor design decisions.
"""

from __future__ import annotations
Expand Down
17 changes: 13 additions & 4 deletions tracecat/executor/schemas.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,11 +40,14 @@ class ExecutorBackendType(StrEnum):

All sandbox backends use untrusted mode - DB credentials are never passed.

- POOL: Warm nsjail workers (high throughput, single-tenant, untrusted)
- POOL: Warm nsjail workers (high throughput, single-tenant, untrusted).
EXPERIMENTAL: not production ready. Warm workers import from registry
cache paths that in-process leases cannot see, so tarball cache entries
are exempt from eviction and the cache can grow without bound.
- EPHEMERAL: Cold nsjail subprocess per action (full isolation, multi-tenant, untrusted)
- DIRECT: Direct subprocess execution (no warm workers)
- TEST: In-process execution for tests only
- AUTO: Auto-select based on environment
- AUTO: Auto-select based on environment (never selects experimental backends)
"""

POOL = "pool"
Expand Down Expand Up @@ -85,14 +88,20 @@ def resolve_backend_type() -> ExecutorBackendType:
backend_type = ExecutorBackendType.DIRECT
elif _is_nsjail_available():
logger.info(
"Auto-selecting 'pool' backend (nsjail available)",
"Auto-selecting 'ephemeral' backend (nsjail available)",
)
backend_type = ExecutorBackendType.POOL
backend_type = ExecutorBackendType.EPHEMERAL
Comment thread
daryllimyt marked this conversation as resolved.
else:
logger.warning(
"Auto-selecting 'direct' backend (nsjail not available)",
)
backend_type = ExecutorBackendType.DIRECT
elif backend_type == ExecutorBackendType.POOL:
logger.warning(
"The 'pool' executor backend is experimental and not production "
"ready: its registry cache is exempt from eviction and can grow "
"without bound. Use 'ephemeral' for production nsjail isolation.",
)

return backend_type

Expand Down
Loading