Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 10 additions & 3 deletions docs/self-hosting/security.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ To enable nsjail, set the following in your `.env`:

```bash
TRACECAT__DISABLE_NSJAIL=false
TRACECAT__EXECUTOR_BACKEND=pool # or 'ephemeral' for multi-tenant full isolation
TRACECAT__EXECUTOR_BACKEND=ephemeral
```

nsjail requires:
Expand All @@ -76,8 +76,15 @@ nsjail requires:
| Backend | Isolation | Latency | Use case |
| :------ | :-------- | :------ | :------- |
| `direct` | PID namespace only | ~50ms | Development, trusted environments |
| `pool` | nsjail sandbox (warm workers) | ~100-200ms | Single-tenant production |
| `ephemeral` | nsjail sandbox (cold per action) | ~4000ms | Multi-tenant, maximum isolation |
| `ephemeral` | nsjail sandbox (cold per action) | ~4000ms | Production, maximum isolation |
| `pool` | nsjail sandbox (warm workers) | ~100-200ms | Experimental — not production ready |

<Warning>
The `pool` backend is experimental and not production ready. Its warm workers
import from registry cache paths that eviction cannot reclaim, so the
executor's registry cache can grow without bound. Use `ephemeral` for
production deployments.
</Warning>

## Authentication

Expand Down
2 changes: 1 addition & 1 deletion docs/snippets/environment-variables.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@

| Variable | Default | Description |
| :------- | :------ | :---------- |
| `TRACECAT__EXECUTOR_BACKEND` | `direct` | Execution strategy for actions. `direct` runs a subprocess per action, `pool` uses warm nsjail workers (requires nsjail), `ephemeral` spawns a cold nsjail subprocess per action for full isolation, `auto` selects `pool` if nsjail is available and falls back to `direct`, `test` runs in-process for tests only. |
| `TRACECAT__EXECUTOR_BACKEND` | `direct` | Execution strategy for actions. `direct` runs a subprocess per action, `ephemeral` spawns a cold nsjail subprocess per action for full isolation, `auto` selects `ephemeral` if nsjail is available and falls back to `direct`, `pool` uses warm nsjail workers (experimental, not production ready), `test` runs in-process for tests only. |
| `TRACECAT__DISABLE_NSJAIL` | `true` | Disable nsjail sandboxing. Set to `false` only if nsjail is installed. |
| `TRACECAT__RESULT_EXTERNALIZATION_ENABLED` | `true` | Store large action results in blob storage instead of Temporal history. |
| `TRACECAT__COLLECTION_MANIFESTS_ENABLED` | `true` | Store large collections as chunked manifests in blob storage. |
Expand Down
6 changes: 4 additions & 2 deletions tracecat/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -636,11 +636,13 @@ def _parse_auth_types() -> set[AuthType]:
"""Executor backend for running actions.

Supported values:
- 'pool': Warm nsjail workers (single-tenant, high throughput, ~100-200ms)
- 'pool': Warm nsjail workers (single-tenant, high throughput, ~100-200ms).
EXPERIMENTAL: not production ready. The registry cache is exempt from
eviction under this backend and can grow without bound.
- 'ephemeral': Cold nsjail subprocess per action (multitenant, full isolation, ~4000ms)
- 'direct': Direct subprocess execution (no warm workers, no in-process state sharing)
- 'test': In-process execution for tests only (no isolation, no subprocess overhead)
- 'auto': Auto-select based on environment (pool if nsjail available, else direct)
- 'auto': Auto-select based on environment (ephemeral if nsjail available, else direct)

Trust mode is derived from the backend type:
- pool: untrusted (secrets pre-resolved, no DB creds)
Expand Down
8 changes: 8 additions & 0 deletions tracecat/executor/backends/pool/backend.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
"""Worker pool executor backend.

EXPERIMENTAL: not production ready. Warm workers import from registry cache
paths that in-process leases cannot see, so tarball cache entries are exempt
from eviction and the cache can grow without bound. Use the ephemeral backend
for production nsjail isolation.

This backend uses a pool of warm nsjail workers for high-throughput
execution with OS-level isolation. Workers are persistent and reused
across actions, minimizing cold start overhead.
Expand Down Expand Up @@ -32,6 +37,9 @@
class PoolBackend(ExecutorBackend):
"""Warm nsjail worker pool backend.

EXPERIMENTAL: not production ready. See the module docstring for the
registry cache eviction limitation.

Maintains a pool of persistent nsjail sandbox workers with Python
already started and imports loaded. This provides:

Expand Down
17 changes: 13 additions & 4 deletions tracecat/executor/schemas.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,11 +40,14 @@ class ExecutorBackendType(StrEnum):

All sandbox backends use untrusted mode - DB credentials are never passed.

- POOL: Warm nsjail workers (high throughput, single-tenant, untrusted)
- POOL: Warm nsjail workers (high throughput, single-tenant, untrusted).
EXPERIMENTAL: not production ready. Warm workers import from registry
cache paths that in-process leases cannot see, so tarball cache entries
are exempt from eviction and the cache can grow without bound.
- EPHEMERAL: Cold nsjail subprocess per action (full isolation, multi-tenant, untrusted)
- DIRECT: Direct subprocess execution (no warm workers)
- TEST: In-process execution for tests only
- AUTO: Auto-select based on environment
- AUTO: Auto-select based on environment (never selects experimental backends)
"""

POOL = "pool"
Expand Down Expand Up @@ -85,14 +88,20 @@ def resolve_backend_type() -> ExecutorBackendType:
backend_type = ExecutorBackendType.DIRECT
elif _is_nsjail_available():
logger.info(
"Auto-selecting 'pool' backend (nsjail available)",
"Auto-selecting 'ephemeral' backend (nsjail available)",
)
backend_type = ExecutorBackendType.POOL
backend_type = ExecutorBackendType.EPHEMERAL
Comment thread
daryllimyt marked this conversation as resolved.
else:
logger.warning(
"Auto-selecting 'direct' backend (nsjail not available)",
)
backend_type = ExecutorBackendType.DIRECT
elif backend_type == ExecutorBackendType.POOL:
logger.warning(
"The 'pool' executor backend is experimental and not production "
"ready: its registry cache is exempt from eviction and can grow "
"without bound. Use 'ephemeral' for production nsjail isolation.",
)

return backend_type

Expand Down
Loading