perf(sync): pace CRDT write-back by what the last pass cost

[h4yfans]

Problem

crdt-writeback.ts schedules a full yDocToMarkdown → CriticMarkup serialize → safeRead → parseNote → serializeParsedNote → atomicWrite → syncNoteToCache (FTS re-index) pass on a 500 ms debounce, re-armed on every Y.Doc update.

The issue's stated trigger is not quite right, so I measured before touching anything.

Measured cycle counts, 30 s of simulated typing on origin/main (fake timers, counting mocked pipeline stages):

pattern	full cycles / 30 s
unbroken typing @200 ms per keystroke (150 updates)	0 during, 1 after the trailing pause
unbroken typing @100 ms per keystroke (300 updates)	0 during, 1 after the trailing pause
prose: 5-char bursts @150 ms + 700 ms word pause	20
uniform 500 ms gaps (60 updates)	60

So the debounce is trailing-only with no max-wait: continuous typing produces zero passes, not 60. What does produce a pass per update is any rhythm whose gaps sit at or above 500 ms — exactly the word-and-sentence pauses of normal prose typing. That reaches 20–60 passes per 30 s.

Measured cost of one pass (real yDocToMarkdown + real parseNote/serializeParsedNote, 20 iterations each):

note	yDocToMarkdown	parse + re-serialize	total
2.4 KB / 10 sections	35.8 ms	0.18 ms	36 ms
12 KB / 50 sections	134.3 ms	0.05 ms	134 ms
49 KB / 200 sections	427.3 ms	0.15 ms	427 ms

The pipeline is ~99% yDocToMarkdown, and it scales with note size. At the 500 ms rhythm a 12 KB note spends ~27% of a core in write-back and a 49 KB note ~85%. That is the real defect, and it is worse than the issue's framing on large notes and absent on the "continuous typing" case it names.

Root cause

The debounce delay is a constant. It bounds how soon a pass runs but not how often, and it is unrelated to what a pass costs — so the same 500 ms applies to a note whose pass takes 36 ms and one whose pass takes 430 ms.

Fix

Keep the 500 ms debounce; add a cost-proportional cooldown measured from the end of the previous pass:

const cooldownMs = Math.min(last.durationMs * WRITEBACK_COOLDOWN_FACTOR, WRITEBACK_MAX_COOLDOWN_MS)
return Math.max(WRITEBACK_DEBOUNCE_MS, last.finishedAt + cooldownMs - Date.now())

FACTOR = 9, ceiling 5 s. Write-back therefore consumes at most ~1/10 of wall clock per note. A 36 ms pass earns a 324 ms cooldown, below the 500 ms debounce floor — small notes behave exactly as before. Only notes whose pass exceeds ~55 ms are paced at all.

The suggested "cheap dirty check (doc update counter / state-vector compare) before serializing" was deliberately not implemented: scheduleWriteback is only ever called from onDocUpdate, so the doc has always advanced since the last pass and such a check can never short-circuit. It would be dead code.

Test evidence

New tests in crdt-writeback.test.ts pin exact operation counts and the exact bytes handed to atomicWrite.

RED on origin/main:

× throttles the write-back pipeline in proportion to what the last pass cost
  → expected "vi.fn()" to be called 16 times, but got 60 times
× caps the cooldown so a pathologically slow pass cannot strand the file
  → expected "vi.fn()" to be called 7 times, but got 60 times
× flushPendingWritebacks still lands a write-back that is inside its cooldown
  → expected "vi.fn()" to be called 1 times, but got 2 times
Tests  3 failed | 13 passed (16)

GREEN after: Tests 16 passed (16). Measured before/after for the 200 ms-pass note: 60 → 16 cycles per 30 s (serialize, read, parse, write and re-index all asserted individually); for a 3 s pass, 60 → 7.

Mutation verification — two single-line reverts, each targeted by line number:

1. line 238 }, writebackDelayMs(noteId)) → }, WRITEBACK_DEBOUNCE_MS) → 3 tests red (16→60, 7→60, 1→2).
2. line 193 WRITEBACK_MAX_COOLDOWN_MS → Number.POSITIVE_INFINITY → the ceiling test red (7→2).

No survivors.

Byte fidelity. A non-trivial note (frontmatter with a custom key, checked/unchecked tasks, wiki link, inline tag, nested lists, ordered list, blockquote, fenced ts code block, table, ![[Embed.png]]) was pushed through the real converter and real frontmatter serializer, capturing the atomicWrite payload on origin/main and on this branch:

efae9e0eb8ca4ff518513b83c5ec96e4a258e4844e7755299c669c7879576844  before.md
efae9e0eb8ca4ff518513b83c5ec96e4a258e4844e7755299c669c7879576844  after.md

Identical. The change is timing-only and touches no serialization path.

Risk and backward compatibility

Durability of a pending write-back

• App quit — flushed. before-quit → stopSyncRuntime() → CrdtProvider.destroy() → flushPendingWritebacks(), which clears the timers and runs every deferred pass immediately (a test covers the in-cooldown case). This is the same flushPendingWritebacks path as today; the app:request-flush / app:flush-done handshake scoped by fix(shutdown): scope the window flush handshake per window and request #1150 runs earlier in that chain and is untouched by this PR.
• Window close — unaffected. The Y.Doc and the timer live in main; closing a window neither cancels nor delays the pass.
• Vault switch — flushed. closeVault() → stopSyncRuntime() → the same destroy() → flushPendingWritebacks().
• Crash — the .md file loses whatever the deferred pass would have written (today up to 500 ms of edits, now up to 5 s on an expensive note). The typing itself is not lost: onDocUpdate persists every update to y-leveldb before it schedules the write-back, and seedFromMarkdown only seeds a doc whose fragment is empty, so the recovered Y.Doc wins over the stale file and the next edit or sync writes it out. The note editor's own 1 s autosave (updateNote) is a second, independent writer of the same file and is not throttled here.

Search index. syncNoteToCache runs inside the pass, so the index catches up exactly when the file does — ≤5 s after a keystroke on a large note instead of ≤500 ms. A user who searches inside that window sees the note's previous body. In exchange, a large note no longer re-indexes twice a second while being typed into.

Markdown-as-truth readers. hasPendingWriteback() stays true for the longer window, so reconcileTaskCheckboxesFromMarkdown stands down for longer. That is the safe direction — it is what stops a stale file from reverting a checkbox the user just ticked.

Compatibility. No vault file format, sync protocol, CRDT payload, IPC contract, DB schema or settings shape is touched. sourceWindowId tagging and main-owned Y.Docs are untouched. Two number fields per note in one module-level Map, cleared by cancelPendingWritebacks().

Relation to open PRs. #1118 makes buffered network CRDT updates survive shutdown; #1116 coalesces full-document snapshot uploads. Both pace the sync side of a doc update. This PR paces the third fan-out of the same onDocUpdate — the vault-file write — and shares no code or state with either. Their scheduler/queue changes and this cooldown compose without interacting.

Verification

pnpm typecheck                          16 successful, 16 total
pnpm lint                               ✖ 15 problems (0 errors, 15 warnings)   # all pre-existing, untouched files
pnpm --filter @memry/desktop test:main   Test Files 477 passed | 1 skipped (478)
                                         Tests 5444 passed | 1 expected fail | 4 skipped (5449)
git diff --check                         clean
pnpm docs:impact --base origin/main --strict   covered
pnpm docs:build                          build complete

Docs: apps/docs/src/architecture/crdt.md gains a "Markdown Write-Back" section — the gate reported missing-docs and the write-back timing genuinely was undocumented, so it was written rather than skipped.

Closes #1030

[github-actions]

React Doctor found no new issues. 🎉

_{Reviewed by React Doctor for commit 59ab41a.}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!