feat: core/drift.ts + core/differ.ts — three-way merge engine (#11)

.github/workflows/ci.yml

@@ -8,9 +8,19 @@ on:
 
 jobs:
   check:
-    runs-on: ubuntu-latest
+    # Full OS × Node matrix: shardmind installs as a global CLI and must
+    # run on whatever the user's vault lives on. Windows coverage caught
+    # the CRLF merge regression Copilot flagged on PR #48; keep it first-class.
+    runs-on: ${{ matrix.os }}
+    defaults:
+      run:
+        # Use bash on all three runners so the same step commands work
+        # (Windows runners have git-bash pre-installed).
+        shell: bash
     strategy:
+      fail-fast: false
       matrix:
+        os: [ubuntu-latest, windows-latest, macos-latest]
         # Node 22+ matches obsidian-mind's requirement (hook TS strip
         # via --experimental-strip-types, stable in 22.6+). Node 18 hit
         # EOL April 2025; Node 20 EOL April 2026. Keep the matrix
@@ -25,7 +35,7 @@ jobs:
         with:
           node-version: ${{ matrix.node-version }}
 
-      - run: rm package-lock.json && npm install
+      - run: rm -f package-lock.json && npm install
       - run: npm run typecheck
       - run: npm test
       - run: npm run build

.gitignore

@@ -58,6 +58,7 @@ yarn.lock
 # (commands, agents, scripts, skills, settings.json) that ShardMind installs.
 # Only ignore the auto-memory subdirectories that agents generate at runtime.
 .claude/memory/
+.claude/scheduled_tasks.lock
 .claude/projects/
 .agent/
 memory/

CHANGELOG.md

@@ -1,10 +1,36 @@
 # Changelog
 
-Populated on each release. Between releases:
-
-- **What's merged but not released:** `git log`
-- **What's planned:** [`ROADMAP.md`](ROADMAP.md)
-
 The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+Between releases: see `git log` for merged work and [`ROADMAP.md`](ROADMAP.md) for planned work.
+
 <!-- First release: v0.1.0, target end of Milestone 6. -->
+
+## [Unreleased]
+
+### Added
+
+- **Three-way merge engine** (`source/core/drift.ts`, `source/core/differ.ts`). Solves "propagate template updates to existing vaults" — the same problem Backstage (Spotify) has had open for 3+ years and that cruft/create-react-app/Yeoman never solved. Uses `node-diff3`'s Khanna–Myers algorithm via `diff3MergeRegions` for accurate stats.
+  - `detectDrift()` classifies every tracked file into `managed / modified / volatile / missing / orphaned` by sha256-comparing disk content against `state.json.rendered_hash`. Orphan detection is non-recursive: parent directories of tracked files become tracked dirs; files in a tracked dir not in `state.files` are orphans. Engine scaffolding (`shard-values.yaml`, `.shardmind/`) and third-party metadata (`.git/`, `.obsidian/`) are excluded.
+  - `computeMergeAction()` returns `skip | overwrite | auto_merge | conflict` — skip when `base === ours`, overwrite for managed files, three-way merge for modified files. Conflict markers use git vocabulary: `<<<<<<< yours` / `=======` / `>>>>>>> shard update`.
+  - `threeWayMerge()` is a pure primitive — line-based, CRLF-tolerant on input, LF on output.
+- **`renderString()` in `source/core/renderer.ts`** — frontmatter-aware render helper for in-memory templates, used by the merge engine to render base/ours without touching disk.
+- **Typed `ErrorCode` union** in `source/runtime/errors.ts` — 39 codes grouped by domain; `ShardMindError.code` is now strictly typed. Exported from `shardmind/runtime` for hook consumers.
+- **Cross-OS CI matrix** — `{ubuntu, windows, macos} × {node 22, node 24}`, `fail-fast: false`. Windows coverage caught a CRLF regression on first run.
+- **20 fixture scenarios** for the merge engine (`tests/fixtures/merge/`) — 17 spec-defined plus 3 edge cases (empty file, UTF-8 non-ASCII, frontmatter-on-modified-ownership merge).
+- **Direct unit tests** for merge primitives: `tests/unit/three-way-merge.test.ts` pins down stats accounting; `tests/unit/differ-line-endings.test.ts` covers CRLF robustness; `tests/unit/drift-classification.test.ts` covers every `DriftReport` bucket including orphans.
+
+### Changed
+
+- `ShardMindError.code` typed as `ErrorCode` instead of free-form `string` (compile-time check of every call site).
+- `detectDrift` now runs per-file reads in parallel via `Promise.all`, and runs the orphan scan in parallel with the classification.
+- `source/runtime/errno.ts` centralizes `errnoCode` / `isEnoent`; 8 copies of the `err instanceof Error && 'code' in err ? ...` pattern collapsed.
+
+### Fixed
+
+- CRLF on Windows-saved user files no longer produces spurious conflicts against LF-rendered base/ours.
+
+### Docs
+
+- `docs/ARCHITECTURE.md` §17 updated: scenario table now shows 20 rows, `§17.4` code sample shows the actual `diff3MergeRegions` algorithm, `§17.5` corrects the frontmatter-merge decision to match implementation (line-merge of rendered YAML, not YAML object deep-merge).
+- `docs/IMPLEMENTATION.md` §4.8 documents orphan detection semantics; §4.9 documents the `diff3MergeRegions` variant and CRLF split.

ROADMAP.md

@@ -33,9 +33,9 @@ Ship the core: install, update, status. Prove that vault template upgrades work
 ### Milestone 3: Merge Engine (Day 3)
 
 - [x] Write all 17 merge fixture directories — fixtures before code ([#10](https://github.com/breferrari/shardmind/issues/10))
-- [ ] `core/drift.ts` + `core/differ.ts` — three-way merge engine ([#11](https://github.com/breferrari/shardmind/issues/11))
-- [ ] Iterate until all 17 scenarios pass
-- [ ] Add edge case fixtures: frontmatter merge, empty file, binary-identical, encoding
+- [x] `core/drift.ts` + `core/differ.ts` — three-way merge engine ([#11](https://github.com/breferrari/shardmind/issues/11))
+- [x] Iterate until all 17 scenarios pass
+- [x] Add edge case fixtures: empty file (18), UTF-8 non-ASCII (19), frontmatter merge on modified ownership (20). Hash-identical behavior already covered by scenarios 01 and 05 (both hit the `sha256(base) === sha256(ours)` shortcut)
 
 ### Milestone 4: Update Command + Status (Day 4)
 

docs/ARCHITECTURE.md

@@ -1199,7 +1199,14 @@ tests/fixtures/
 └── migration/
 ```
 
-### 17.3 The 16 Merge Scenarios
+### 17.3 The 20 Merge Scenarios
+
+Seventeen spec scenarios plus three edge cases. Each is a directory under
+`tests/fixtures/merge/` with `scenario.yaml` + templates + values + actual
+file + expected output. The fixture runner dispatches by scenario flag:
+standard scenarios go through `computeMergeAction`; orchestration-level
+scenarios (new file / removed / volatile / module change) dispatch to
+rendering or drift classification directly.
 
 | # | Scenario | Ownership | Template? | Values? | User edited? | Expected |
 |---|----------|-----------|-----------|---------|--------------|----------|
@@ -1210,54 +1217,79 @@ tests/fixtures/
 | 05 | User edited, nothing upstream | modified | no | no | yes | Skip. Preserve user's. |
 | 06 | Template changed, user edited different section | modified | yes | no | yes | Auto-merge |
 | 07 | Template changed, user edited SAME section | modified | yes | no | yes | Conflict → TUI |
-| 08 | User deleted section | modified | yes | no | deletion | Show diff |
+| 08 | User deleted section | modified | yes | no | deletion | Auto-merge (honor deletion, apply non-conflicting template edit) |
 | 09 | User added below template | modified | yes | no | addition | Auto-merge, preserve additions |
 | 10 | New file from template | — | — | — | — | Render, add to state |
 | 11 | File removed from template | managed | — | — | no | Ask: delete or keep? |
-| 12 | Frontmatter only change | managed | yes (fm) | no | no | Re-render with FM-aware merge |
+| 12 | Frontmatter only change | managed | yes (fm) | no | no | Re-render (managed overwrite) |
 | 13 | Module newly included on update | — | — | — | — | Render as fresh install for that module |
 | 14 | Module excluded on update | included | — | — | — | Ask: delete files or keep as user-owned? |
 | 15 | _each item added | managed | no | yes | no | Render new file |
 | 16 | CLAUDE.md partial updated | managed | yes | no | no | Re-render full CLAUDE.md |
+| 17 | Volatile file, template changed | volatile | yes | no | yes | Skip (never re-render volatile) |
+| 18 | Empty file → shard adds content | managed | yes | no | no | Overwrite |
+| 19 | UTF-8 non-ASCII round-trip | modified | yes | no | yes | Auto-merge, bytes preserved |
+| 20 | Frontmatter edited + body edited (modified) | modified | yes | no | yes | Auto-merge (user tag + shard body change) |
 
 ### 17.4 Three-Way Merge Implementation
 
-Uses `node-diff3` (Khanna-Myers algorithm):
+Uses `node-diff3`'s `diff3MergeRegions` (Khanna–Myers algorithm) — **not**
+the flat `diff3Merge`. The regions variant exposes `buffer: 'a' | 'o' | 'b'`
+on stable regions and separate `aContent / oContent / bContent` on unstable
+ones, which is the only way to distinguish stable-unchanged lines from
+stable-auto-merged lines and produce accurate merge stats.
 
 ```typescript
-import { diff3Merge } from 'node-diff3';
+import { diff3MergeRegions } from 'node-diff3';
 
-export function threeWayMerge(input: MergeInput): MergeResult {
-  const merged = diff3Merge(
-    input.theirs.split('\n'),  // user's file
-    input.base.split('\n'),    // old rendered (cached)
-    input.ours.split('\n'),    // new rendered
+export function threeWayMerge(base, theirs, ours): ThreeWayMergeResult {
+  const regions = diff3MergeRegions(
+    theirs.split(/\r?\n/),  // a — user on disk
+    base.split(/\r?\n/),    // o — old rendered (cached)
+    ours.split(/\r?\n/),    // b — new rendered
   );
-  // Process regions: ok → keep, conflict → mark
-  // Return content + hasConflicts + conflict regions + stats
+
+  for (const r of regions) {
+    if (r.stable) {
+      emit(r.bufferContent);
+      if (r.buffer === 'o') stats.linesUnchanged   += r.bufferContent.length;
+      else                  stats.linesAutoMerged  += r.bufferContent.length;
+      continue;
+    }
+    // Unstable: classify as auto-merge or true conflict.
+    if (arraysEqual(r.aContent, r.oContent))       emit(r.bContent);  // ours
+    else if (arraysEqual(r.bContent, r.oContent))  emit(r.aContent);  // theirs
+    else if (arraysEqual(r.aContent, r.bContent))  emit(r.aContent);  // false conflict
+    else                                           emitConflictMarkers(...);
+  }
 }
 ```
 
+The `/\r?\n/` split tolerates CRLF on Windows-saved user files while base and
+ours are renderer output (always LF). Merged output is always LF — callers
+convert at the write boundary if they need platform-native line endings.
+
 ### 17.5 Edge Cases
 
 | Edge case | Decision |
 |-----------|----------|
-| **Frontmatter merge** | Parse both as YAML objects, deep-merge, don't line-merge |
-| **Empty file on disk** | Treat as modified. Show diff. |
-| **Hash-identical after re-render** | Short-circuit. No diff needed. |
-| **Wikilink targets moved** | Warn. Suggest find-and-replace. |
-| **_each item renamed** | Rename file, don't delete+recreate (preserves backlinks) |
-| **Encoding** | Test with Unicode, emoji, CJK in values |
+| **Frontmatter merge** | Render the whole file; renderer parses frontmatter and emits normalized YAML (parse → stringify via `yaml`). diff3 then line-merges the whole document. User tag additions and shard field additions merge cleanly because disjoint YAML keys become non-adjacent lines. Fixture 20 proves this. |
+| **Empty file** | Render pipeline produces an empty string; `sha256(base) === sha256(ours)` short-circuits the no-change path. Fixture 18 tests the variant where base is empty but ours has content → overwrite. |
+| **Hash-identical after re-render** | Short-circuit via `sha256(base) === sha256(ours)`. No diff3 pass. Covered by scenarios 01, 05. |
+| **CRLF user file** | `split(/\r?\n/)` normalizes on the way in; merged output is LF. Covered by `tests/unit/differ-line-endings.test.ts`. |
+| **UTF-8 non-ASCII** | diff3 is byte/line-agnostic for string content. Fixture 19 exercises emoji, accented Latin, and CJK. |
+| **Wikilink targets moved** | Warn. Suggest find-and-replace. *(v0.2 — out of scope for the engine.)* |
+| **_each item renamed** | Rename file, don't delete+recreate (preserves backlinks). *(v0.2 — requires iterator-aware diffing beyond the single-file merge primitive.)* |
 
 ### 17.6 Test-First Build Order
 
-1. Write all 16 fixture files (templates, values, actual, expected)
-2. Write test runner that auto-discovers fixtures
-3. Run → all fail
-4. Implement `computeMergeAction()` → ownership tests pass
-5. Implement `threeWayMerge()` → merge output tests pass
-6. Add edge case fixtures → find bugs → fix
-7. Wire into `commands/update.tsx`
+1. ✅ Write 17 spec-defined fixture files (templates, values, actual, expected) in PR #10.
+2. ✅ Write test runner (`tests/unit/drift.test.ts`) that auto-discovers fixtures and dispatches by scenario kind.
+3. ✅ `it.fails` all 17 before the engine exists.
+4. ✅ Implement `computeMergeAction()` → skip / overwrite ownership tests pass.
+5. ✅ Implement `threeWayMerge()` → auto_merge / conflict tests pass.
+6. ✅ Add edge-case fixtures (empty, UTF-8, frontmatter-modified-merge) plus direct unit tests for stats invariants (`tests/unit/three-way-merge.test.ts`) and CRLF robustness (`tests/unit/differ-line-endings.test.ts`).
+7. Wire into `commands/update.tsx` — *Milestone 4 / issue #12, out of scope for the merge-engine PR.*
 
 ---
 

docs/IMPLEMENTATION.md

@@ -487,14 +487,15 @@ interface DriftEntry {
 ```
 
 **Algorithm**:
-1. For each file in `state.files`:
-   a. If `ownership === 'volatile'` → add to `volatile`
-   b. Read file from disk. If not found → add to `missing`
-   c. Compute `sha256(file content)`
-   d. Compare against `state.files[path].rendered_hash`
-   e. If equal → `managed`
-   f. If different → `modified`
-2. Return classified report
+1. For each file in `state.files` (in parallel via `Promise.all`):
+   a. If `FileState.ownership === 'user'` (volatile at install time) → `DriftEntry` with `ownership: 'volatile'` → add to `volatile`. Never hashed; content may diverge by design.
+   b. Read file from disk. If ENOENT → add to `missing` (propagate state ownership).
+   c. Compute `sha256(file content)`.
+   d. Compare against `state.files[path].rendered_hash`. Equal → `managed`. Different → `modified`.
+2. Orphan scan (runs in parallel with the classification): union of parent directories of every tracked path is the set of tracked directories. For each tracked directory, `readdir` non-recursively and report files not in `state.files` as orphans. Excludes engine-reserved files (`VALUES_FILE`) and never-scanned directories (`.shardmind`, `.git`, `.obsidian`). Subdirectories of a tracked directory are not auto-scanned — they only count if they themselves contain a tracked file.
+3. Return classified report.
+
+**Rationale for non-recursive orphan scan**: the shard only claims to manage what it tracks. A user's `brain/daily/2026-04-19.md` under an untracked subdirectory is their territory, not an orphan. But a `skills/my-extra.md` sibling of a tracked `skills/leadership.md` is an orphan because `skills/` is territory the shard already claims.
 
 **Dependencies**: `node:fs`, `node:crypto`.
 
@@ -536,17 +537,23 @@ interface MergeStats {
 4. If `sha256(base) === sha256(ours)` → no upstream change → `{ type: 'skip' }`
 5. If ownership is `managed` (base === theirs) → `{ type: 'overwrite', content: ours }`
 6. If ownership is `modified`:
-   a. Run `diff3Merge(theirs.split('\n'), base.split('\n'), ours.split('\n'))`
-   b. If no conflicts → `{ type: 'auto_merge', content: merged }`
-   c. If conflicts → `{ type: 'conflict', result: { content, hasConflicts, conflicts, stats } }`
+   a. Run `diff3MergeRegions(theirs.split(/\r?\n/), base.split(/\r?\n/), ours.split(/\r?\n/))` — not the flat `diff3Merge`; the regions variant exposes `buffer: 'a' | 'o' | 'b'` on stable regions and `aContent / oContent / bContent` on unstable ones, which is the only way to distinguish stable-unchanged (`buffer === 'o'`) from stable-auto-merged (`buffer === 'a' | 'b'`) lines. The `/\r?\n/` split tolerates CRLF on Windows-saved files; merged output is always LF.
+   b. For each stable region: emit `bufferContent`. For each unstable region: if `aContent === oContent` take `bContent`; if `bContent === oContent` take `aContent`; if `aContent === bContent` take either (false conflict); else emit git-style conflict markers and record a `ConflictRegion`.
+   c. No conflicts → `{ type: 'auto_merge', content, stats }`. Conflicts → `{ type: 'conflict', result: { content, hasConflicts: true, conflicts, stats } }`.
 
 **`MergeResult`** (for conflicts):
 ```typescript
+interface MergeStatsWithConflicts {
+  linesUnchanged: number;
+  linesAutoMerged: number;
+  linesConflicted: number;
+}
+
 interface MergeResult {
   content: string;              // Merged content with conflict markers
   hasConflicts: boolean;
   conflicts: ConflictRegion[];
-  stats: { linesUnchanged: number; linesAutoMerged: number; linesConflicted: number; };
+  stats: MergeStatsWithConflicts;
 }
 
 interface ConflictRegion {

package.json

@@ -48,6 +48,7 @@
     "@types/react": "^19.2.14",
     "@types/semver": "^7.5.0",
     "@types/tar": "^6.1.0",
+    "fast-check": "4.7.0",
     "ink-testing-library": "^4.0.0",
     "tsup": "^8.5.1",
     "typescript": "^6.0.3",