Skip to content
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 65 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
name: CI

on:
pull_request:
branches: [master]
push:
branches: [master]
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Run CI on the repository’s active branch

These branch filters only trigger on master, so if development happens on main (as in this repo), none of the lint/test jobs run for normal pushes or PRs. That leaves the new checks effectively disabled for day-to-day work and allows regressions to merge without CI coverage.

Useful? React with 👍 / 👎.


jobs:
lint-and-typecheck:
name: Lint & Typecheck
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 22
cache: npm
cache-dependency-path: package-lock.json
- run: npm ci
- run: npm run check

unit-tests:
name: Unit Tests
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 22
cache: npm
cache-dependency-path: package-lock.json
- run: npm ci
- name: Pure unit tests
run: |
npm run test:chunker
npm run test:vectorstore
npm run test:dedup
npm run test:smart-compile
npm run test:skill-sources
npm run test:mcp-hosts
npm run test:modes
npm run test:crypto
npm run test:security
npm run test:validation
npm run test:backup
npm run test:ranking
npm run test:mutex

integration-tests:
name: Integration Tests
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 22
cache: npm
cache-dependency-path: package-lock.json
- run: npm ci
- name: Server + HTTP tests
run: |
npm run smoke
npm run test:handoffs
npm run test:projects
4 changes: 2 additions & 2 deletions bench/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ For each task in `tasks.json`, three numbers (apples-to-apples on full skill bod
- **Smart** — tokens after CE's `/api/compile/smart` picks the relevant skills for this specific task. Same content type as Raw all, just a subset.
- **Search** — tokens an MCP host (Claude Desktop, Codex) actually pulls when it calls `context_engine_search` — chunks, not full skills.

Reference column: `CONTEXT.md` size — CE's pre-compressed system-prompt summary, a *different* path entirely. Reported alongside so both CE delivery paths are visible without inflating the savings number by mixing content types.
Reference column: `CONTEXT.md` size — CE's pre-compressed system-prompt summary, a _different_ path entirely. Reported alongside so both CE delivery paths are visible without inflating the savings number by mixing content types.

Optional quality column (`--grade`): for each task, actually run it through Claude in both smart and search modes, capture the response, and have a judge model score it 1-10. The summary adds tokens-per-quality-point efficiency — the real "are we saving tokens AND maintaining answer quality?" question.

Expand Down Expand Up @@ -64,7 +64,7 @@ Cost expectation at defaults (Haiku for both task + grader, 15 tasks): ~$0.10 pe

- **Tokenizer**: tiktoken `cl100k_base` (the BPE GPT-3.5/4 uses). It's within ~5% of Anthropic's own tokenizer on prose. Same encoding is used for baseline + smart + search counts, so the percentages are internally consistent.
- **Baseline definition**: only **active** skills (per `data/skill-states.json`) — that's the realistic ceiling. Including inactive skills would inflate the saving artificially.
- **Search realism**: simulates a host calling `context_engine_search` *once* per task and pulling N chunks. Real host apps may call it multiple times or fall back to `context_engine_get_skill` for full bodies, so this is a lower-bound on what they actually consume.
- **Search realism**: simulates a host calling `context_engine_search` _once_ per task and pulling N chunks. Real host apps may call it multiple times or fall back to `context_engine_get_skill` for full bodies, so this is a lower-bound on what they actually consume.
- **Smart-compile token budget**: defaults to 16k. Run with `--max-tokens 4000` to see how aggressive selection becomes for small-context models, or `--max-tokens 64000` to see what large-context users get.
- **Cross-check**: the summary prints CE's own estimator's ratio vs tiktoken's count. If those diverge a lot, CE's internal budget UI is over- or under-stating.

Expand Down
2 changes: 1 addition & 1 deletion cli/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -434,4 +434,4 @@ function help() {
context-engine add ./my-skills/react-rules
context-engine status
`);
}
}
4 changes: 4 additions & 0 deletions data/projects.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{
"version": "1.0",
"projects": []
}
42 changes: 21 additions & 21 deletions docs/specs/onboarding-redesign.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,12 +28,12 @@ The sibling product `model-db` ships a polished onboarding wizard. Its pattern

Four steps, each focused on one decision.

| # | Step | Concern | Replaces |
| - | ---------- | ----------------------------------------------------------------------------- | --------------------------------------- |
| 1 | **Connect**| Pick which MCP hosts (Claude Desktop, Codex CLI, ChatGPT) to wire CE into. | Most of current "Discover" + "Connect". |
| 2 | **Context**| Review skills/memory/index. Optional inline "Build index" if Ollama present. | "Available context" panel + part of "Health". |
| 3 | **IDE** | Show detected fallback targets. Note that file output remains available. | "IDE and file-output surfaces" panel. |
| 4 | **Health** | Quick verification: hosts connected, skills active, index ready. Finish. | Current "Health" step. |
| # | Step | Concern | Replaces |
| --- | ----------- | ---------------------------------------------------------------------------- | --------------------------------------------- |
| 1 | **Connect** | Pick which MCP hosts (Claude Desktop, Codex CLI, ChatGPT) to wire CE into. | Most of current "Discover" + "Connect". |
| 2 | **Context** | Review skills/memory/index. Optional inline "Build index" if Ollama present. | "Available context" panel + part of "Health". |
| 3 | **IDE** | Show detected fallback targets. Note that file output remains available. | "IDE and file-output surfaces" panel. |
| 4 | **Health** | Quick verification: hosts connected, skills active, index ready. Finish. | Current "Health" step. |

Each step has Back / Next (or Skip for now / Finish on the last step). Steps 1 and 3 are skippable — the user can connect later via the Connections tab.

Expand Down Expand Up @@ -67,11 +67,11 @@ No `.onboarding-mark` text monogram. No typeset "Context Engine" string paired w

Horizontal row of numbered circles connected by short lines. State per step:

| State | Circle background | Circle text | Connector line |
| -------- | -------------------- | --------------- | --------------- |
| Done | `--accent` | white | `--accent` |
| Current | `--accent` | white | `--dram-line-subtle` (line *after* current) |
| Upcoming | `--dram-bg-recessed` | `--text-4` | `--dram-line-subtle` |
| State | Circle background | Circle text | Connector line |
| -------- | -------------------- | ----------- | ------------------------------------------- |
| Done | `--accent` | white | `--accent` |
| Current | `--accent` | white | `--dram-line-subtle` (line _after_ current) |
| Upcoming | `--dram-bg-recessed` | `--text-4` | `--dram-line-subtle` |

Connector: `height: 1px; width: 32px;`. Circle: `28×28; border-radius: 999px`. Step label sits under each circle in `--text-3`, `11px`, uppercase, `letter-spacing: 0.07em` (matches card-name typography from DRAM).

Expand All @@ -97,16 +97,16 @@ Common pattern inside each step:

All cards in the onboarding body use the global card pattern from [dram-design-map.md → Cards](../dram-design-map.md#cards):

| Property | Token |
| ------------------ | --------------------------- |
| Background | `--dram-card-bg` |
| Border | `--dram-card-border` |
| Hover background | `--dram-card-hover` |
| Hover border | `--dram-card-border-hover` |
| Selected background| `--dram-card-active` |
| Selected border | `--dram-card-border-active` |
| Radius | `--r-3` |
| Disabled opacity | `--dram-card-muted-opacity` |
| Property | Token |
| ------------------- | --------------------------- |
| Background | `--dram-card-bg` |
| Border | `--dram-card-border` |
| Hover background | `--dram-card-hover` |
| Hover border | `--dram-card-border-hover` |
| Selected background | `--dram-card-active` |
| Selected border | `--dram-card-border-active` |
| Radius | `--r-3` |
| Disabled opacity | `--dram-card-muted-opacity` |

Three card variants used:

Expand Down
48 changes: 26 additions & 22 deletions docs/specs/skill-sources.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,13 +61,13 @@ A registered external directory CE reads `SKILL.md` files from. Stored in `data/
- `removeSource(id)` → deletes by id. Refuses if id is `internal`.
- `scanHostSkillPaths()` → probes known locations, returns `{ path, label, exists, skillCount }[]`. Probed paths:

| Path | Label |
| ------------------------------------- | --------------------------- |
| `~/.claude/skills` | Claude Code (global) |
| `<CWD>/.claude/skills` | Claude Code (current project)|
| `<CWD>/.clinerules` | Cline / Roo rules |
| `<CWD>/.continue/rules` | Continue.dev rules |
| `~/.opencode/skills` | OpenCode (global) |
| Path | Label |
| ----------------------- | ----------------------------- |
| `~/.claude/skills` | Claude Code (global) |
| `<CWD>/.claude/skills` | Claude Code (current project) |
| `<CWD>/.clinerules` | Cline / Roo rules |
| `<CWD>/.continue/rules` | Continue.dev rules |
| `~/.opencode/skills` | OpenCode (global) |

Each entry returns the resolved absolute path, a friendly label, `exists` boolean, and skill count if any.

Expand Down Expand Up @@ -99,13 +99,13 @@ The denylist is reused but **inverted** for this case: we're reading, not writin

### New endpoints

| Method | Path | Description |
| ------ | ----------------------------- | ------------------------------------------------------------------------------ |
| GET | `/api/skill-sources` | List registered sources + implicit internal. Returns skill counts per source. |
| POST | `/api/skill-sources` | Add an external source. Body: `{ path, label? }`. Validates + denylist-checks. |
| DELETE | `/api/skill-sources/:id` | Remove a source. Refuses on `internal`. |
| GET | `/api/skill-sources/scan` | Probe known host-skill paths. Returns the candidates with counts. |
| POST | `/api/skill-sources/import` | Phase 2 — copy a source's contents into `<CE_ROOT>/skills/imported/<id>/`. |
| Method | Path | Description |
| ------ | --------------------------- | ------------------------------------------------------------------------------ |
| GET | `/api/skill-sources` | List registered sources + implicit internal. Returns skill counts per source. |
| POST | `/api/skill-sources` | Add an external source. Body: `{ path, label? }`. Validates + denylist-checks. |
| DELETE | `/api/skill-sources/:id` | Remove a source. Refuses on `internal`. |
| GET | `/api/skill-sources/scan` | Probe known host-skill paths. Returns the candidates with counts. |
| POST | `/api/skill-sources/import` | Phase 2 — copy a source's contents into `<CE_ROOT>/skills/imported/<id>/`. |

## UI

Expand Down Expand Up @@ -167,6 +167,7 @@ A new "Sources" affordance in the Skills tab header — a small select/expander
## Phase 2 detailed design (2026-05-11)

Locked decisions:

- **Import action lives on the linked row.** Link is the cheap commitment, Import is the heavier one; forcing Link-then-Import is the right progression. No Import button on candidate rows.
- **Manual Sync only.** No filesystem watching; a Sync button on imported rows triggers the diff. Keeps the user in control of when CE walks their dirs.

Expand All @@ -175,6 +176,7 @@ Locked decisions:
Three states for a registered source: `external` (linked, read from original), `imported` (linked + a copy/hard-link tree has been written into `<CE_ROOT>/skills/imported/<sourceId>/`), `internal` (implicit, never stored).

Transitions:

```
external --[POST /api/skill-sources/:id/import]--> imported
imported --[POST /api/skill-sources/:id/sync/apply]--> imported (manifest rewritten)
Expand All @@ -187,6 +189,7 @@ Unlinking an imported source intentionally **keeps** the imported tree. The user
### Import strategy: hard-link + copy fallback

File-level `fs.linkSync` is the primary strategy. Fall back to `fs.copyFileSync` per-file on:

- `EXDEV` — cross-volume on Windows. NTFS hard-links are intra-volume only.
- `EPERM` / `EACCES` — source permissions.
- Non-link-capable filesystems (FAT/exFAT).
Expand All @@ -205,9 +208,7 @@ One file per imported source at `data/skill-imports/<sourceId>.json`:
"importedAt": "2026-05-11T15:00:00Z",
"lastSyncedAt": "2026-05-11T15:00:00Z",
"aggregateStrategy": "link",
"files": [
{ "rel": "react/SKILL.md", "size": 2341, "mtimeMs": 1715000000000, "strategy": "link" }
]
"files": [{ "rel": "react/SKILL.md", "size": 2341, "mtimeMs": 1715000000000, "strategy": "link" }]
}
```

Expand All @@ -226,23 +227,25 @@ One file per imported source at `data/skill-imports/<sourceId>.json`:
```

Definitions:

- **Added**: in source, absent from manifest.
- **Removed**: in manifest, absent from source.
- **Modified**: in both, but `size` or `mtimeMs` differs **and** the per-file strategy is `copy`. Hard-linked files can't drift in content (shared inode), so we don't list them as modified even if mtime moved on the source.

`POST /api/skill-sources/:id/sync/apply` body `{ mode: 'append' | 'overwrite' }`:

- `append`: adds new files only. Removed and modified are left.
- `overwrite`: applies all three categories — add new, delete removed, re-link/re-copy modified.

Manifest is rewritten after successful apply.

### Endpoints (Phase 2)

| Method | Path | Description |
| ------ | ----------------------------------------------- | -------------------------------------------------------------------------- |
| POST | `/api/skill-sources/:id/import` | Walk source + write imported tree + manifest. Idempotent: refuses if already imported. |
| GET | `/api/skill-sources/:id/sync` | Compute diff without applying. |
| POST | `/api/skill-sources/:id/sync/apply` | Body `{ mode }`. Apply diff + rewrite manifest. |
| Method | Path | Description |
| ------ | ----------------------------------- | -------------------------------------------------------------------------------------- |
| POST | `/api/skill-sources/:id/import` | Walk source + write imported tree + manifest. Idempotent: refuses if already imported. |
| GET | `/api/skill-sources/:id/sync` | Compute diff without applying. |
| POST | `/api/skill-sources/:id/sync/apply` | Body `{ mode }`. Apply diff + rewrite manifest. |

### UI placement (Phase 2)

Expand Down Expand Up @@ -306,6 +309,7 @@ Verification (direct node script): linking a fixture with both an `app-launcher`
- `conflicts`: dest AND source both diverged. Overwrite discards the local edit.

Apply behaviour:

- `append` ignores both.
- `overwrite` re-places `modified` + `localEdits` + `conflicts` (excluding conflicts whose source file no longer exists; those flow through `removed`).

Expand Down
4 changes: 2 additions & 2 deletions electron/main.cjs
Original file line number Diff line number Diff line change
Expand Up @@ -345,9 +345,9 @@ ipcMain.on('window:close', () => {
ipcMain.handle('dialog:select-folder', async (_event, options) => {
const dialogOptions = {
properties: ['openDirectory'],
title: (options && typeof options.title === 'string' ? options.title : 'Pick a folder'),
title: options && typeof options.title === 'string' ? options.title : 'Pick a folder',
};
const result = await dialog.showOpenDialog(mainWindow || undefined, dialogOptions);
if (result.canceled || !result.filePaths?.length) return null;
return result.filePaths[0];
});
});
2 changes: 1 addition & 1 deletion electron/preload.cjs
Original file line number Diff line number Diff line change
Expand Up @@ -31,4 +31,4 @@ contextBridge.exposeInMainWorld('contextEngineDesktop', {

window.addEventListener('DOMContentLoaded', () => {
document.documentElement.dataset.runtime = 'electron';
});
});
2 changes: 1 addition & 1 deletion electron/updater.cjs
Original file line number Diff line number Diff line change
Expand Up @@ -88,4 +88,4 @@ function startAutoUpdate(window, options = {}) {
}, intervalMs);
}

module.exports = { startAutoUpdate };
module.exports = { startAutoUpdate };
2 changes: 1 addition & 1 deletion mcp-http-server.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -175,4 +175,4 @@ server.listen(PORT, HOST, () => {
`context-engine HTTP MCP server listening on http://${HOST}:${PORT}/mcp (auth=${auth}, CE_BASE=${CE_BASE})\n`,
);
if (oauth) process.stderr.write('OAuth consent passphrase is set with MCP_OAUTH_PASSWORD.\n');
});
});
2 changes: 1 addition & 1 deletion mcp-oauth.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -299,4 +299,4 @@ async function collect(req) {
const chunks = [];
for await (const chunk of req) chunks.push(chunk);
return chunks;
}
}
2 changes: 1 addition & 1 deletion mcp-server.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -14,4 +14,4 @@ const transport = new StdioServerTransport();
await server.connect(transport);

// stderr is the only safe channel under stdio MCP; stdout is protocol traffic.
process.stderr.write(`context-engine MCP server connected (CE_BASE=${CE_BASE})\n`);
process.stderr.write(`context-engine MCP server connected (CE_BASE=${CE_BASE})\n`);
4 changes: 2 additions & 2 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

10 changes: 9 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "context-engine",
"version": "0.3.1",
"version": "0.4.0",
"description": "Local-first continuity layer for AI work across tools, providers, and fresh sessions.",
"main": "electron/main.cjs",
"bin": {
Expand Down Expand Up @@ -40,6 +40,14 @@
"test:skill-sources": "node scripts/skill-sources-smoke.js",
"migrate:handoffs": "node scripts/migrate-legacy-handoff.js",
"test:mcp-hosts": "node scripts/mcp-host-config-smoke.js",
"test:modes": "node scripts/mode-apply-smoke.js",
"test:projects": "node scripts/projects-smoke.js",
"test:crypto": "node scripts/crypto-smoke.js",
"test:security": "node scripts/security-smoke.js",
"test:validation": "node scripts/validation-smoke.js",
"test:backup": "node scripts/backup-smoke.js",
"test:ranking": "node scripts/ranking-smoke.js",
"test:mutex": "node scripts/mutex-smoke.js",
"mcp": "node mcp-server.mjs",
"mcp:http": "node mcp-http-server.mjs",
"mcpb:pack": "powershell -NoProfile -ExecutionPolicy Bypass -File scripts/pack-mcpb.ps1",
Expand Down
Loading
Loading