Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,8 @@ function scaleQuotas(limit, weights) {
const order = Object.keys(weights);
const quotas = Object.fromEntries(order.map((key) => [key, 0]));
if (slots < order.length) {
for (const key of order) quotas[key] = 1;
const priority = [...order].sort((a, b) => weights[b] - weights[a]);
for (const key of priority.slice(0, slots)) quotas[key] = 1;
return quotas;
}

Expand All @@ -61,10 +62,12 @@ function scaleQuotas(limit, weights) {
}

function legacyMemoryQuotas(limit) {
return {
...scaleQuotas(limit, { events: 10, entities: 10, preferences: 3 }),
experiences: 0,
};
return scaleQuotas(limit, {
events: 10,
entities: 10,
experiences: 3,
preferences: 3,
});
}

function codingQuotas(limit) {
Expand Down
58 changes: 48 additions & 10 deletions examples/codex-memory-plugin/DESIGN.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ events imply "context for a particular codex `session_id` is gone".
memory extractor) at session-end-equivalent moments. `/messages`
auto-creates the OV session, so the plugin does not call session create.
- **State file** — `~/.openviking/codex-plugin-state/<safe-codex-session-id>.json`,
shape `{ codexSessionId, ovSessionId, capturedTurnCount, createdAt, lastUpdatedAt }`.
shape `{ codexSessionId, ovSessionId, capturedTurnCount, revision, createdAt, lastUpdatedAt }`.
- **Active window** — state files whose `lastUpdatedAt` is within
`ACTIVE_WINDOW_MS` (default 2 min) of "now". Used to detect "the codex
session that just ended".
Expand Down Expand Up @@ -228,6 +228,7 @@ OV session id, while commits create additional archives under that session.
"codexSessionId": "0193af...", // codex thread id
"ovSessionId": "cx-0193af...-or-null", // null means "committed, awaiting next Stop"
"capturedTurnCount": 7, // turns from transcript already appended
"revision": 4, // monotonic across save, clear, and recreation
"createdAt": 1715000000000,
"lastUpdatedAt": 1715000300000
}
Expand All @@ -238,7 +239,37 @@ Legacy state files from earlier plugin versions may still contain a UUID
next resolve. The migration window for preserving old UUID sessions has
closed.

State files are atomic-write (tmpfile + rename) to survive crash mid-write.
State files use a unique tmpfile + rename and recover the newest complete,
session-matching tmp after a crash. A monotonic revision counter lives beside
the permanent per-session lock baton; `clear` advances that tombstone before
removing the final file, so state recreation cannot reset the generation or
make a stale SessionStart snapshot look current.

Stop, PreCompact, and SessionStart run each same-session state/remote-I/O
lifecycle under that cross-process baton. Acquire atomically renames
`available` to a unique `owner-<token>` path; release and dead-owner recovery
move only that unique source, avoiding pathname ABA. Automatic abandoned-owner
recovery is intentionally limited to the same host and Linux PID namespace,
where machine id, PID-namespace identity, boot id, namespace-local PID, and
`/proc` start time can prove that the exact owner process is gone. Legacy claims
without namespace identity and claims from another host or container namespace
are never stolen on elapsed time alone because OpenViking does not provide a
fencing token for the remote session operations; operators sharing one state
directory across hosts or PID namespaces must recover an abandoned baton
explicitly. Automatic dead-owner recovery is Linux-only; other platforms do not
have this full identity tuple and therefore fail safe to manual recovery.
Ownerless contender claim metadata is pruned on a later acquisition only when
the same Linux identity proof confirms that its process is dead; otherwise it is
left for manual cleanup under the same cross-host/container safety rule.

The ordinary state-lock wait is 60 seconds so an in-flight append or commit is
not mistaken for abandonment. The default Stop hook launches its writer in a
detached process, so Codex's 30-second Stop deadline does not bound that
writer. If an explicitly synchronous Stop writer is killed, the durable
revision/tmp recovery above lets a later hook resume from the last accepted
batch. PreCompact instead waits at most 20 seconds by default and reports a
deferred capture on contention; this preserves most of its 60-second hook
deadline for catch-up and commit work without discarding recoverable progress.

## Configuration

Expand All @@ -247,14 +278,16 @@ Env var overrides for tuning without rebuilding:
| Var | Default | Purpose |
|---|---|---|
| `OPENVIKING_CODEX_STATE_DIR` | `~/.openviking/codex-plugin-state` | state file dir |
| `OPENVIKING_CODEX_STATE_LOCK_TIMEOUT_MS` | `60000` | maximum wait for the same-session cross-process baton |
| `OPENVIKING_PRECOMPACT_STATE_LOCK_TIMEOUT_MS` | `20000` | PreCompact-specific baton wait; timeout reports deferral and preserves state |
| `OPENVIKING_CODEX_ACTIVE_WINDOW_MS` | `120000` (2 min) | rule-3 active window |
| `OPENVIKING_CODEX_IDLE_TTL_MS` | `1800000` (30 min) | idle sweep TTL |
| `OPENVIKING_RECALL_TIMEOUT_MS` | `120000` (2 min) | whole UserPromptSubmit auto-recall deadline |
| `OPENVIKING_RECALL_COMPRESS` | `1` | set `0` / `off` to skip `codex exec` compression |
| `OPENVIKING_RECALL_COMPRESS_MODEL` | unset | custom first-choice compressor model; `off` disables compression |
| `OPENVIKING_RECALL_COMPRESS_THINKING` | unset | custom `model_reasoning_effort`; `default` means omit override; alias `OPENVIKING_RECALL_COMPRESS_REASONING_EFFORT` |
| `OPENVIKING_RECALL_COMPRESS_DETECT_ON_STARTUP` | `1` | recreate/cache compressor profile during every `SessionStart` |
| `OPENVIKING_RECALL_COMPRESS_DETECT_TIMEOUT_MS` | `15000` | per-candidate compressor probe timeout |
| `OPENVIKING_RECALL_COMPRESS_DETECT_TIMEOUT_MS` | `15000` | compatibility setting for older installers; current detection does not launch a startup probe |
| `OPENVIKING_RECALL_COMPRESS_DETECT_TTL_MS` | `604800000` (7 days) | cache TTL used by `UserPromptSubmit` reads |
| `OPENVIKING_RESUME_ARCHIVE_INJECT` | `1` | inject latest archive summary on `source=resume` when no live OV session is open |
| `OPENVIKING_RESUME_ARCHIVE_TOKEN_BUDGET` | `32000` | token budget for `/sessions/{id}/context` on resume |
Expand Down Expand Up @@ -288,21 +321,26 @@ codex -m <model> -c 'model_reasoning_effort="low"' exec ...
`thinking=default` omits the `model_reasoning_effort` override. This is
important for model families whose default effort is tuned by Codex.

Model availability is re-probed at every `SessionStart`, not in every
`UserPromptSubmit`. Recreating the profile on each session start catches
cross-session env/config changes. The detector writes
Model availability is resolved from Codex's model catalogue when the cached
profile is missing, expired, or marked runtime-failed; it is not probed with a
child process on every `SessionStart` or `UserPromptSubmit`. The detector writes
`recall-compressor-profile.json` under `OPENVIKING_CODEX_STATE_DIR` and
auto-recall reads that cache. Cache misses in auto-recall use the first
candidate directly and fall back to deterministic digest if `codex exec`
fails.
auto-recall reads that cache. A prompt tries at most two distinct candidates
inside one shared timeout. If every attempt fails, it injects nothing and
records those failed models; it does not turn unverified candidates into a
deterministic digest. Explicitly configured `off` still uses deterministic
formatting without `codex exec`.

Fallback order:

1. configured model/thinking (`OPENVIKING_RECALL_COMPRESS_MODEL` +
`OPENVIKING_RECALL_COMPRESS_THINKING`)
2. `gpt-5.3-codex-spark`, thinking `default`
3. `gpt-5.6-luna`, thinking `low`
4. off (deterministic digest, no child `codex exec`)

When a configured model is distinct from both defaults, only it and the
primary default fit in the first prompt's two-attempt budget; an untried
fallback remains eligible on a later prompt.

Configured `off` (`OPENVIKING_RECALL_COMPRESS=0`, model `off`, or thinking
`off`) skips all probing and writes a disabled profile.
Expand Down
57 changes: 50 additions & 7 deletions examples/codex-memory-plugin/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -204,23 +204,31 @@ On `resume`, the script skips commit/sweep. It still injects the profile block.

Codex injects `additionalContext` into the model turn, so memories arrive without an extra tool call. By default the hook runs a Codex compression pass over recalled candidates before injection, dropping weakly-related memories and preserving only a short digest. If the compressor returns `NO_RELEVANT_MEMORY`, empty text, or non-digest chatter, the hook emits `{}` and injects nothing. The whole hook has its own `OPENVIKING_RECALL_TIMEOUT_MS` deadline (default 120s); the bundled `hooks.json` gives Codex 130s so the script can return `{}` before Codex kills it. Digests may keep `viking://` source URIs and point the model at the OpenViking MCP `read`/`search` tools for details when the inline bullet is intentionally short. The outer `<openviking-context ...>` wrapper is deterministic, not compressor-generated; capture strips it to distinguish recalled context from the user's prompt. Set `OPENVIKING_RECALL_COMPRESS=0` to fall back to deterministic short formatting.

The compressor profile is recreated on every `SessionStart` and cached under `OPENVIKING_CODEX_STATE_DIR` so cross-session config changes are picked up but each `UserPromptSubmit` does not probe models. Default fallback order:
The compressor profile is resolved on `SessionStart` when the cache is missing,
expired, or marked runtime-failed, then cached under
`OPENVIKING_CODEX_STATE_DIR`. A prompt tries at most two distinct model
candidates within one shared timeout budget. If every runtime attempt fails,
recall fails closed and injects nothing; a weak deterministic digest is never
substituted merely because the relevance check failed. Explicitly disabling
compression still uses the deterministic formatter. Candidate order is:

1. configured `OPENVIKING_RECALL_COMPRESS_MODEL` + `OPENVIKING_RECALL_COMPRESS_THINKING`
2. `gpt-5.3-codex-spark` with thinking `default`
3. `gpt-5.6-luna` with thinking `low`
4. off (deterministic digest, no `codex exec` compression)

If a configured model occupies the first slot, the remaining candidate can be
retried on a later prompt after failed models are recorded.

Config knobs:

| Env var | Default | Meaning |
|---|---|---|
| `OPENVIKING_RECALL_LIMIT` | `10` | Legacy quota-scaling input; explicit values are converted to six coding quotas, not enforced as a final result cap. |
| `OPENVIKING_RECALL_LIMIT` | `10` | Legacy quota-scaling input; an explicit value is distributed across coding categories and the resulting quotas sum to that value. |
| `OPENVIKING_RECALL_COMPRESS` | `1` | Set `0` / `off` to disable `codex exec` compression. |
| `OPENVIKING_RECALL_COMPRESS_MODEL` | unset | Custom first-choice compressor model. Set `off` to disable compression. |
| `OPENVIKING_RECALL_COMPRESS_THINKING` | unset | Custom `model_reasoning_effort`; `default` omits the Codex config override. Alias: `OPENVIKING_RECALL_COMPRESS_REASONING_EFFORT`. |
| `OPENVIKING_RECALL_COMPRESS_DETECT_ON_STARTUP` | `1` | Recreate/cache compressor profile in `SessionStart`. |
| `OPENVIKING_RECALL_COMPRESS_DETECT_TIMEOUT_MS` | `15000` | Per-candidate startup probe timeout. |
| `OPENVIKING_RECALL_COMPRESS_DETECT_TIMEOUT_MS` | `15000` | Compatibility setting retained for older installers; current detection reads the local model catalogue and does not launch a startup probe. |
| `OPENVIKING_RECALL_COMPRESS_DETECT_TTL_MS` | `604800000` | Cache TTL used by `UserPromptSubmit` when reading the latest profile. |
| `OPENVIKING_RECALL_MAX_TOKENS` | `1600` | Token budget the server assembles the context block within, independent of the local compressor input limit. |
| `OPENVIKING_RECALL_DEDUP_TURNS` | `5` | Cross-turn cooldown: URIs served in the last N turns are skipped. |
Expand All @@ -233,9 +241,9 @@ that endpoint fall back to `/api/v1/search/recall`, and that outcome is cached s
only the first turn pays for the probe. Server-owned Context defaults are omitted
unless explicitly configured, so the plugin follows the server instead of copying
values such as `limit=10` or `max_tokens=1600`. An explicit legacy `recallLimit`
is converted to per-category coding quotas, not a final result cap. Values
from 1 through 5 therefore produce an effective total quota of 6, one retrieval
slot for each coding domain. Local `codex exec` compression is
is converted to per-category coding quotas whose total equals the configured
value; when the value is smaller than the number of categories, only the
highest-priority categories receive a slot. Local `codex exec` compression is
unchanged and still runs on top of whichever path answered.

Client-side knobs can also live in `~/.openviking/ovcli.conf` under
Expand All @@ -249,6 +257,15 @@ defaults.

After a successful append, Stop reads the session meta and commits when `pending_tokens >= OPENVIKING_COMMIT_TOKEN_THRESHOLD` (default `20000`). Threshold commits pass `keep_recent_count=OPENVIKING_COMMIT_KEEP_RECENT_COUNT` (default `10`) so the newest turns remain live for continuity while older context is archived and extracted. `PreCompact` still commits everything before compaction.

Same-session state transitions are serialized across hook processes and every
accepted append batch advances durable state before the next batch starts. The
default Stop path launches its writer asynchronously, so Codex's 30-second
hook deadline does not cut short a writer waiting on the 60-second state-lock
budget. If asynchronous writing is disabled and Codex terminates a synchronous
writer, the next hook recovers a complete temporary state file, resumes from
the last recorded turn count, and safely reclaims a confirmed-dead owner in the
same host and Linux PID namespace.

### PreCompact (deterministic commit)

`pre-compact-capture.mjs`:
Expand All @@ -257,6 +274,32 @@ After a successful append, Stop reads the session meta and commits when `pending
2. Commit the long-lived OV session so the extractor runs against the full pre-compact transcript
3. Reset `ovSessionId` to `null` so the next `Stop` re-derives the same `cx-<safe-session-id>` and appends the post-compact half under that deterministic OV session id

PreCompact uses a shorter state-lock wait (20 seconds by default; override
with `OPENVIKING_PRECOMPACT_STATE_LOCK_TIMEOUT_MS`) so lock contention cannot
silently consume its entire 60-second hook deadline. On timeout it emits a
`systemMessage`, leaves durable capture progress untouched, and lets a later
hook catch up.

If multiple hosts share `OPENVIKING_CODEX_STATE_DIR`, the plugin never steals
a different host's owner baton based only on elapsed time: without a server
fencing token that could overlap remote writes. After a host failure, an
operator must first verify that the remote owner process is gone and then
recover the abandoned baton manually. Prefer a host-local state directory
unless that operational coordination is available.

The same fail-safe rule applies when multiple containers share that directory.
Linux automatic recovery requires matching machine-id and PID namespace
identity; legacy claims or claims from another PID namespace are left for
manual recovery rather than treating an invisible container process as dead.
Automatic dead-owner recovery is Linux-only; other platforms also require
manual recovery because hostname and PID alone cannot prove host/process
identity safely.

The same rule applies to a contender that crashes before acquiring the baton:
a later acquisition removes its ownerless claim metadata only when the same
Linux identity tuple proves that process is dead. Unverifiable remote, legacy,
container, or non-Linux claims remain available for manual cleanup.

### Known gap: SIGTERM / Ctrl+C / `/exit` are silent

Codex fires no hook on process exit. `/compact` is the only fully-deterministic "context disappearing" signal. If you `/exit` without `/compact`, the OV session for that codex session_id stays open. Two fallbacks recover the orphan:
Expand Down
Loading