feat(space): maintain sessions.visible_message_count counter [#2336] - #2358
feat(space): maintain sessions.visible_message_count counter [#2336]#2358lsm wants to merge 6 commits into
Conversation
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
There was a problem hiding this comment.
lsm has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.
3afafb3 to
721c5cf
Compare
There was a problem hiding this comment.
lsm has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.
lsm
left a comment
There was a problem hiding this comment.
🤖 Review by glm-5.1[1m] (GLM / Zhipu)
Model: glm-5.1[1m] | Client: NeoKai | Provider: GLM (Zhipu)
Recommendation: REQUEST_CHANGES (one P3 nit; everything else is solid).
What I verified
- The perf fix lands correctly.
spaceSessions.bySpacenow readss.visible_message_countdirectly — the correlated per-sessionCOUNT(*)is gone. The new "messageCount is decoupled from sdk_messages" test proves adding message rows can no longer move the badge; only an explicit update to the maintained column does. - Predicate parity is exact across all three sites — INSERT (
isVisibleBadgeRow), RECOMPUTE (recomputeVisibleMessageCount), and the migration-170 backfill. I checked every dimension:parent_tool_use_id IS NULL, user rows gated onsend_status IN (consumed, failed)with NULL→consumed, subtype not inHIDDEN_SYSTEM_SUBTYPES + thinking_tokens, and non-user rows always visible. NULL/empty-string handling formessage_subtypeandsend_statusis identical in all three.messages.bySessionmatches on every badge-relevant dimension (its two extra filters are transcript-rendering only, intentionally excluded from the badge). - Every runtime
sdk_messagesmutation is covered. All writes funnel throughSDKMessageRepository; the 6 structural methods all maintain the counter (3 inserts bump, 2 rewinds + pending-delete + status-flip recompute). Session archive preserves messages (counter stays correct); hard-delete FK-cascades and the session row — and the counter — go together, so no drift. No clear/reset/import/fork bulk-insert path exists. - Transactions on the hot paths.
saveSDKMessage/saveUserMessagewrap insert + bump inthis.db.transaction(...)— atomic. ✓ - Migration is robust: idempotent (recompute-on-rerun), missing-table guards, correct backfill. Tested.
- No-op guard (
supportsVisibleMessageCount) makes maintenance a safe no-op for partial-schema test harnesses — good design. - Tests are comprehensive: visible/invisible inserts, status flips, action messages, rewind recompute, an anti-drift cross-check (counter == fresh
COUNT(*)) across a mixed sequence, plus migration backfill/idempotency/guards. Relocating the visibility-predicate coverage from the LQH test to the repo test is the right call.
Verification run locally
check:db-schema-parity ✅ · 4-space-storage shard 1893/1893 ✅ · live-query-handlers test 108/108 ✅ · new repo + migration-170 tests 128/128 ✅.
Findings
- P3 —
saveHyperNeoActionMessagecounter bump is outside a transaction (see anchored comment onsdk-message-repository.ts:1786). The other two save paths wrap insert+bump in a transaction; this one doesn't. Trivial one-line fix for consistency. The only thing keeping this from an approve.
Optional observation (not a finding, no change requested)
BADGE_HIDDEN_SUBTYPES (Set, insert path) and EXCLUDED_FROM_PAGINATION_SQL_LIST (SQL, recompute path) both encode HIDDEN_SYSTEM_SUBTYPES + 'thinking_tokens' ~20 lines apart. They're consistent today and the anti-drift test guards against divergence; this also matches the file's existing multi-list style, so I'm not asking for a change — just noting the two representations must stay in sync.
Summary
Clean, well-scoped implementation of exactly what the issue asked for, with thorough tests and correct predicate parity. Only the one P3 transaction-consistency nit on the action-message path — fix that and this is good to go.
…ction Match saveSDKMessage / saveUserMessage: wrap the INSERT and the visible_message_count bump in a single transaction so a failure between the two autocommit statements can't leave the counter under-counted. Addresses review P3 on #2358.
There was a problem hiding this comment.
lsm has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.
lsm
left a comment
There was a problem hiding this comment.
🤖 Review by glm-5.1[1m] (GLM / Zhipu)
Model: glm-5.1[1m] | Client: NeoKai | Provider: GLM (Zhipu)
Recommendation: APPROVE — zero findings remain.
The P3 from the previous round is addressed correctly in dfa38a0: saveHyperNeoActionMessage now wraps the INSERT + bumpVisibleMessageCount in this.db.transaction(...), with the prepared statement hoisted and upsertMessageSearchRow left outside — exactly matching saveSDKMessage / saveUserMessage. Surgical change, no scope creep.
Re-verified this round:
- Fix is well-formed and consistent with the other two save paths.
- Repo suite 121/121 (incl. the action-message increment test) ✅.
4-space-storageshard 1893/1893 ✅.- PR is OPEN / MERGEABLE; the single review thread is resolved.
Everything from the prior review stands: the perf fix lands (spaceSessions.bySpace reads the maintained column, correlated per-session COUNT(*) removed), predicate parity is exact across insert/recompute/backfill (incl. NULL handling), every runtime sdk_messages mutation maintains the counter, and tests are comprehensive. Approving.
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: dfa38a031f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
lsm
left a comment
There was a problem hiding this comment.
🤖 Review by glm-5.1[1m] (GLM / Zhipu)
Model: glm-5.1[1m] | Client: NeoKai | Provider: GLM (Zhipu)
Recommendation: REQUEST_CHANGES — supersedes my earlier APPROVE. A reactivity regression surfaced (raised by @chatgpt-codex-connector and confirmed on re-investigation) that breaks the live unread badge — the PR's central purpose. The earlier P3 (transaction wrap) is fixed and fine; this is a separate, more serious issue I missed in both prior rounds.
P1 — badge query lost its sdk_messages dependency, and the counter write emits no sessions event
How the badge stayed live before. LiveQueryEngine derives each query's table dependencies from extractTables(sql) and re-evaluates only on a change to one of those tables (live-query.ts:306, 402). The old spaceSessions.bySpace SQL had a correlated FROM sdk_messages subquery, so its deps were ["sdk_messages","sessions","spaces"]. Every saveSDKMessage emits a sdk_messages change (reactive-database.ts METHOD_TABLE_MAP), which re-evaluated the badge — that was the live-update path.
What changed. I ran extractTables on both SQL strings:
- OLD deps:
["sdk_messages","sessions","spaces"] - NEW deps:
["sessions","spaces"]←sdk_messagesis gone.
So a sdk_messages change no longer re-evaluates spaceSessions.bySpace. And the maintained counter is updated by raw UPDATE sessions SET visible_message_count=… inside SDKMessageRepository.bumpVisibleMessageCount / recomputeVisibleMessageCount. The reactive proxy emits table events from the method name (METHOD_TABLE_MAP), not from the SQL — prepare().run() binds to the raw target and emits nothing. So that sessions write emits no sessions change either.
Net effect. A counter-changing event no longer triggers badge re-evaluation. The badge now refreshes only incidentally when an unrelated reactive updateSession (e.g. a processing_state transition around a turn) happens to fire. That is unreliable and demonstrably misses cases the old code handled:
saveHyperNeoActionMessage— action-message inserts with no coincident session-row write.updateMessageStatusflips (deferred→consumed) after a turn ends.- the tail of a turn once
processing_statehas gone idle.
This violates the codebase's own reactivity pattern. Every other repository that writes raw SQL calls reactiveDb.notifyChange('<table>') afterwards (task, skill, provider, mcp-enablement, app-mcp-server, space-task, goal, …). state-projection-service.ts:285/308 even calls notifyChange('sessions') for its raw UPDATE sessions SET last_error. The counter maintenance is the one raw sessions write that skips it.
Suggested fix (preserves the perf win)
Thread reactiveDb into SDKMessageRepository (as task/skill/provider repos already do) and call this.reactiveDb.notifyChange('sessions') after bumpVisibleMessageCount / recomputeVisibleMessageCount. This restores the live re-eval without re-introducing the correlated COUNT(*): the query stays a cheap column read, and re-evaluation frequency returns to what it was (still 150ms-debounced via DEBOUNCE_SPACE_SESSIONS_MS).
A test for this should assert that a saveSDKMessage on a space session causes spaceSessions.bySpace to re-evaluate (e.g. via the LiveQueryEngine change event / a subscriber delta), not just that the column is updated — the repo-level counter tests pass today but don't cover the reactivity gap.
Standing
Everything else from prior rounds holds (predicate parity across insert/recompute/backfill; all mutation paths covered; migration idempotent; the P3 transaction wrap is now in). Only this P1 blocks.
|
To use Codex here, create an environment for this repo. |
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: dfa38a031f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
lsm
left a comment
There was a problem hiding this comment.
🤖 Review by glm-5.1[1m] (GLM / Zhipu)
Model: glm-5.1[1m] | Client: NeoKai | Provider: GLM (Zhipu)
Both @chatgpt-codex-connector P2s are legitimate — I verified the mechanisms. Consolidating my take and how they relate to the in-flight P1 fix.
P2 (endorse, recommend fixing now) — wrap status/delete mutations + counter in one transaction
upsertMessageSearchRow (sdk-message-repository.ts:245) only guards JSON.parse; its FTS DELETE/INSERT writes (313–341) are unguarded. So in the four structural paths the ordering is: mutation commits → FTS loop → recomputeVisibleMessageCount:
updateMessageStatus(1324 run → 1325 FTS loop → 1326 recompute)deleteMessagesAfter/deleteMessagesAtAndAfter(DELETE → 1426/1450 FTS loop → 1427/1451 recompute)deletePendingUserMessage(DELETE → 1386 FTS → 1389 recompute)
If an FTS write throws, the counter recompute is skipped and the status/delete is already committed → permanent drift (until an incidental recompute). A crash in that window drifts too, but self-heals; the FTS-throw case does not, on its own. This is the same class of issue as the insert-path P3 already fixed in dfa38a031, just unfixed on these four methods.
Recommend doing this together with the P1 notifyChange work — the coder is already in these methods. Wrap each as this.db.transaction(() => { mutation; recompute; notifyChange('sessions'); }) and keep the FTS loop outside (best-effort, as today). Cheap, consistent, and closes the gap.
P2 (real, but cost tradeoff — conscious decision) — rollback reconciliation
The mechanism checks out: M170 is marked (runs once), the column persists across a rollback to a pre-M170 binary (which writes sdk_messages without maintaining it), and re-upgrade skips M170 → stale counters until an incidental recompute. There's clear precedent for an unmarked startup reconciliation here — reconcileSdkMessageReplacementProjection (migrations.ts:752-756, "intentionally runs outside the one-shot marker") and the artifact_type reconciliation (776-777) exist for exactly this rollback-drift reason.
The caveat is cost: the backfill is UPDATE sessions SET visible_message_count = (SELECT COUNT(*) …) across all sessions — O(sessions × messages) on every boot, even when nothing rolled back. Worth doing, but I'd shape it as a cheap incremental pass or gate it rather than a blind full backfill each startup. Reasonable to land as a follow-up with eyes open; not a blocker alongside the P1.
Summary
- P1 (reactivity, prior review) — blocks.
- P2 transaction-wrap — endorse; fold into the in-flight P1 fix since the methods overlap.
- P2 rollback reconciliation — real, has precedent, but carries a per-boot cost; fine as a deliberate follow-up.
|
To use Codex here, create an environment for this repo. |
…updates
P1 (reactivity): removing the correlated COUNT(*) dropped sdk_messages from
spaceSessions.bySpace's table-deps, and the raw counter UPDATE emitted no
reactive 'sessions' event — so the live badge never re-evaluated on message
saves. Thread reactiveDb into SDKMessageRepository and call
notifyChange('sessions') after counter changes (wired in the facade and
space-runtime write paths; read-only repo instances unaffected). notifyChange
fires after the mutation transaction commits so re-eval sees committed state.
P2 (transaction-wrap): wrap mutation + counter recompute in one transaction for
updateMessageStatus / deleteMessagesAfter / deleteMessagesAtAndAfter /
deletePendingUserMessage (FTS loop stays outside, best-effort), matching the
insert-path fix from dfa38a0 — an FTS throw can no longer leave the counter
stale.
Adds sdk-message-repository-live-query.test.ts asserting a visible save (and a
status flip into visibility) triggers a spaceSessions.bySpace re-eval with the
new count, while an invisible save does not.
Addresses review rounds 2-3 on #2358.
There was a problem hiding this comment.
lsm has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1cac717408
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Replace the correlated COUNT(*) over sdk_messages that spaceSessions.bySpace ran for every session on every 150ms-debounced re-evaluation (~92ms warm for dev-neokai) with a maintained sessions.visible_message_count column read directly. - schema: add visible_message_count INTEGER NOT NULL DEFAULT 0 to sessions - migration 170: ALTER + one-time backfill using the badge predicate (renumbered from 169 because dev shipped 169 for message_subtype_norm in #2346) - SDKMessageRepository: increment on visible inserts (saveSDKMessage, saveUserMessage, saveHyperNeoActionMessage); recompute affected sessions on send_status transitions, rewind deletes, and pending-message removal - spaceSessions.bySpace: select s.visible_message_count instead of the per-session subquery Counter maintenance no-ops gracefully when the column/table is absent (unit test harnesses), keeping the existing suite green. The visibility predicate (top-level rows, non-deferred user rows, non-hidden subtypes) is centralized in isVisibleBadgeRow so the maintained counter cannot drift from the predicate it replaces; a cross-check test asserts it equals a fresh COUNT(*) across a mixed sequence of inserts, status flips, and rewinds.
…ction Match saveSDKMessage / saveUserMessage: wrap the INSERT and the visible_message_count bump in a single transaction so a failure between the two autocommit statements can't leave the counter under-counted. Addresses review P3 on #2358.
…updates
P1 (reactivity): removing the correlated COUNT(*) dropped sdk_messages from
spaceSessions.bySpace's table-deps, and the raw counter UPDATE emitted no
reactive 'sessions' event — so the live badge never re-evaluated on message
saves. Thread reactiveDb into SDKMessageRepository and call
notifyChange('sessions') after counter changes (wired in the facade and
space-runtime write paths; read-only repo instances unaffected). notifyChange
fires after the mutation transaction commits so re-eval sees committed state.
P2 (transaction-wrap): wrap mutation + counter recompute in one transaction for
updateMessageStatus / deleteMessagesAfter / deleteMessagesAtAndAfter /
deletePendingUserMessage (FTS loop stays outside, best-effort), matching the
insert-path fix from dfa38a0 — an FTS throw can no longer leave the counter
stale.
Adds sdk-message-repository-live-query.test.ts asserting a visible save (and a
status flip into visibility) triggers a spaceSessions.bySpace re-eval with the
new count, while an invisible save does not.
Addresses review rounds 2-3 on #2358.
…import Move notifySessionsChanged() to immediately after the counter transaction commits and BEFORE the best-effort search-index work (upsert/deleteMessageSearchRow) across all six message-mutation paths, so an FTS throw can no longer strand the live badge with a committed-but-unnotified counter. Addresses the codex P2 on #2358. Also adopt dev's sqlite-compat DB import in the new reactivity test (bun:sqlite is being phased out under the Vitest + node:sqlite migration).
1cac717 to
8fb1403
Compare
lsm
left a comment
There was a problem hiding this comment.
🤖 Review by GLM-5.1 (HyperNeo)
Model: GLM-5.1 | Client: HyperNeo | Provider: HyperNeo
Recommendation: REQUEST_CHANGES — CI is red on head 8fb140339, and the reactivity fix in 441b2676d/8fb140339 introduces a real reactive-transaction scope-poisoning bug.
🔴 P1 (blocking): unscoped notifyChange('sessions') poisons reactive-transaction flush scope → CI red
CI on this head:
Daemon Unit Tests (4-space-storage)— ❌ failureAll Tests Pass— ❌ failure
This contradicts the last handoff ("239 affected + 1510 space-runtime tests pass under Vitest") — the storage shard wasn't covered by that verification.
The single failing test (scoped-invalidation.test.ts, not changed by this branch — last touched by dev's Vitest migration):
ReactiveDatabase — scope extraction > transaction flush preserves compatible scope
expect(txEvent.scope).toEqual({ sessionId: 'sess-5' })
Expected: { sessionId: 'sess-5' }
Received: undefined
Local repro: cd packages/daemon && bun test tests/unit/4-space-storage/storage/scoped-invalidation.test.ts → 15 pass, 1 fail.
Root cause. SDKMessageRepository.notifySessionsChanged() calls reactiveDb.notifyChange('sessions') with no scope (sdk-message-repository.ts:1176-1177). notifyChange is the manual escape hatch and takes no scope param, so incrementAndEmit('sessions') runs with scope = undefined. Inside a reactive transaction:
addPendingScope('sessions', undefined)— the guard!scope?.sessionId && !scope?.taskIdis true → marks thesessionspending scopenull(reactive-database.ts:289-293).- On
commitTransaction,flushPendingTablesseeshasUnscopedTable(any pending table whose scope isnull) and emits the entire batch witheventScope = undefined(reactive-database.ts:333-337).
In the failing test, saveSDKMessage('sess-5') runs inside beginTransaction()/commitTransaction(). Its unscoped notifyChange('sessions') pollutes the batch, so the properly-scoped sdk_messages write for sess-5 is emitted with scope: undefined instead of { sessionId: 'sess-5' }.
Introduced by this PR, not pre-existing. origin/dev's sdk-message-repository.ts contains zero notifyChange / notifySessionsChanged / reactiveDb references — they were added in 441b2676d ("restore live badge reactivity"). I verified with git show origin/dev:...sdk-message-repository.ts | grep notifyChange → NONE on origin/dev.
Production impact (beyond the test). During batched/transactional message writes, the unscoped sessions notify defeats LiveQuery scope filtering: every sessions-dependent query re-evaluates on each flush regardless of scope (the scopeFilter is skipped when scope is absent — live-query.ts:417). That's over-broad re-evaluation — the opposite of this PR's perf goal — and can drive cross-space spaceSessions.bySpace re-evaluation during streaming turns.
Fix (backward-compatible)
1. reactive-database.ts — let the escape hatch carry scope:
// ReactiveDatabase interface + impl
notifyChange(table: string, scope?: TableChangeScope): void {
incrementAndEmit(table, scope);
}Adding an optional param is non-breaking — the ~40 existing notifyChange(table) callers across other repositories (task/skill/space-task/mcp-enablement/provider/…) are unaffected. (reactive-database.ts isn't in this PR's diff, so it'll need to be added in the same PR.)
2. sdk-message-repository.ts — thread the session id:
private notifySessionsChanged(sessionId: string): void {
this.reactiveDb?.notifyChange('sessions', { sessionId });
}{ sessionId } alone is sufficient for spaceSessions.bySpace: its scope filter does a live membership lookup (members.has(scope.sessionId), live-query-handlers.ts:3543) and doesn't require spaceId on the scope. The single-session call sites (saveSDKMessage / saveUserMessage / saveHyperNeoActionMessage / deleteMessagesAfter / deleteMessagesAtAndAfter) just pass their sessionId.
updateMessageStatus can touch multiple sessions — affectedSessions is DISTINCT session_id over messageIds (line 1341). Collect the changed set and notify per session so scopes don't collapse to undefined:
const changed = new Set<string>();
this.db.transaction(() => {
stmt.run(newStatus, ...messageIds);
for (const { sid } of affectedSessions) {
if (this.recomputeVisibleMessageCount(sid)) changed.add(sid);
}
})();
for (const sid of changed) this.notifySessionsChanged(sid);After the fix, please run the full 4-space-storage shard (./scripts/test-daemon.sh 4-space-storage), not just the affected files, and confirm CI is green before re-requesting review. The prior rounds' predicate-parity and transaction-wrap work is solid; this is purely the reactivity-notification scope.
… scope
notifySessionsChanged() called reactiveDb.notifyChange('sessions') with no
scope. Inside a reactive transaction that batches a scoped sdk_messages write,
addPendingScope('sessions', undefined) marks the sessions scope null, and
flushPendingTables then emits the whole batch with undefined scope — poisoning
the properly-scoped sdk_messages write and defeating scope filtering. This made
the "transaction flush preserves compatible scope" test fail (4-space-storage
shard red) and caused over-broad spaceSessions.bySpace re-eval in production.
Fix:
- reactive-database.ts: notifyChange(table, scope?) forwards the scope to
incrementAndEmit (optional param, non-breaking for the other callers).
- sdk-message-repository.ts: notifySessionsChanged(sessionId) passes
{ sessionId }, compatible with the sdk_messages scope in the same batch.
- updateMessageStatus (multi-session): collect the changed sessions and notify
per session instead of a single unscoped notify.
Verified the full 4-space-storage shard (1896/1896) under Vitest. Addresses the
P1 on #2358.
lsm
left a comment
There was a problem hiding this comment.
🤖 Review by GLM-5.1 (HyperNeo)
Model: GLM-5.1 | Client: HyperNeo | Provider: HyperNeo
Recommendation: APPROVE
The P1 reactive-transaction scope-poisoning bug is fixed correctly in 3a119076b, and CI is green. All prior-round findings are addressed.
Verified this round (fresh):
- Fix correctness.
notifyChange(table, scope?)now forwardsscopetoincrementAndEmit(reactive-database.ts) — backward-compatible: the ~40 other single-arg callers receivescope = undefined, identical to prior behavior.notifySessionsChanged(sessionId)passes{ sessionId }. All six call sites updated (saveSDKMessage,saveUserMessage,saveHyperNeoActionMessage,deleteMessagesAfter,deleteMessagesAtAndAfter,updateMessageStatus).updateMessageStatuscorrectly collects the changed-session set and notifies per session (it can span multiple sessions via theDISTINCT session_idlookup), so multi-session batches don't collapse toundefined. { sessionId }is sufficient.spaceSessions.bySpace's scope filter does a live membership lookup (members.has(scope.sessionId)), so it doesn't needspaceIdon the scope — the session's own space's badge feed re-evaluates, other spaces skip. Scope filtering is preserved end-to-end.- Interface safety. The one zero-arg mock (
{ notifyChange: () => {} } as unknown as ReactiveDatabase) is cast-bypassed and ignores extra args; single-argnotifyChange('tasks')callers are unchanged. No implementer breaks; typecheck is additive. - Empirical. Full
4-space-storageshard: 1896/1896 pass under Vitest (incl. the previously-failing"transaction flush preserves compatible scope"), reproduced locally. - CI.
All Tests Pass✅ green; zero failing checks on3a119076b(this aggregator was red last round due to 4-space-storage — now green). - Conversations. All 7 review threads resolved (0 unresolved).
Thanks for running the full shard this time rather than just the affected files — that's what caught the regression path. The predicate-parity, transaction-wrap, and now scoped-notification work are all solid.
P0: 0 · P1: 0 · P2: 0 · P3: 0
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3a119076bd
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
lsm
left a comment
There was a problem hiding this comment.
🤖 Review by GLM-5.1 (HyperNeo)
Model: GLM-5.1 | Client: HyperNeo | Provider: HyperNeo
Recommendation: REQUEST_CHANGES (supersedes my earlier approve/submit — a valid P2 surfaced after I submitted).
To be clear: the core change in 3a119076b is sound — the P1 scope-poisoning fix is correct and backward-compatible, CI All Tests Pass is green, the full 4-space-storage shard is 1896/1896, and all prior threads are resolved. I'm re-opening solely on a new finding (raised by @chatgpt-codex-connector) that I verified.
🟡 P2 (should-fix): scripts/recover-messages.ts bypasses the counter → badge drift after message recovery
I confirmed the bypass by reading the script:
- Step 4 (
recover-messages.ts:156-182): recovers messages into existing sessions via directINSERT OR IGNORE INTO sdk_messages (...), never going throughSDKMessageRepositoryand never touchingsessions.visible_message_count. The existing session's counter is now stale (too low) for any recovered visible message. - Step 5 (
recover-messages.ts:199-206): creates placeholder sessions withINSERT INTO sessions (...)that omitsvisible_message_count→ it defaults to0, then inserts messages into them. Those sessions show0visible messages despite containing recovered ones.
Why it doesn't self-heal at startup: migration 171 is already marked on such a database, so a restart won't rerun the backfill. The drift persists until a later send_status transition or rewind delete happens to call recomputeVisibleMessageCount for those sessions.
This is a regression introduced by this PR. Under the old correlated COUNT(*), recovery was correct (the badge recomputed live). Switching to a maintained column creates an obligation this script violates — a classic integration gap ("omissions and integration risks"). Severity is P2 (not P1): it's a manual operational tool, the drift is display-only, bounded, and partially self-healing — but the fix is trivial and deterministic, so worth doing now rather than as a silent known-bad.
Fix: after the inserts, recompute visible_message_count for every touched session using the same badge predicate as isVisibleBadgeRow / migration 171 backfill (top-level rows; non-deferred user rows with NULL→consumed; exclude HIDDEN_SYSTEM_SUBTYPES + 'thinking_tokens'). Sketch:
// After Step 4 + Step 5 inserts, for each touched session id:
db.prepare(`
UPDATE sessions SET visible_message_count = (
SELECT COUNT(*) FROM sdk_messages
WHERE session_id = ?
AND parent_tool_use_id IS NULL
AND (message_type != 'user' OR COALESCE(send_status,'consumed') IN ('consumed','failed'))
AND COALESCE(message_subtype,'') NOT IN (${EXCLUDED_FROM_PAGINATION_SQL_LIST})
) WHERE id = ?
`).run(sid, sid);To avoid predicate drift, prefer reusing the shared predicate rather than re-literalizing it here (e.g., expose the backfill SQL or a recomputeVisibleMessageCount(sid) entry point the script can call). If you'd rather track this as a follow-up instead of fixing inline, that's defensible (consistent with the deferred rollback-reconciliation) — but please state it explicitly in the PR description rather than leaving it silent.
Secondary check: grep confirms recover-messages.ts is the only non-repository direct INSERT into sdk_messages; all runtime mutations go through SDKMessageRepository. So this is the one bypass to close.
P0: 0 · P1: 0 · P2: 1 · P3: 0
|
To use Codex here, create an environment for this repo. |
scripts/recover-messages.ts inserted sdk_messages rows directly, bypassing SDKMessageRepository, so the maintained counter drifted after message recovery (a regression vs. the old live COUNT(*)). Exposed recomputeVisibleMessageCount as a public entry point (same shared badge predicate — no re-literalizing) and call it for every session touched by the script's direct inserts (existing sessions that received messages + newly created orphan sessions). No-op on a pre-M171 schema that doesn't carry the column yet (the migration backfill covers that). Added a repo test that recomputes after a simulated bypass insert. Addresses the P2 on #2358.
lsm
left a comment
There was a problem hiding this comment.
🤖 Review by GLM-5.1 (HyperNeo)
Model: GLM-5.1 | Client: HyperNeo | Provider: HyperNeo
Recommendation: APPROVE
The recover-messages P2 is fixed correctly and inline in 179bb2fd9. All findings across rounds are now resolved; the counter-maintenance contract is airtight.
Verified this round (fresh):
- Fix correctness.
recomputeVisibleMessageCountis now public and reuses the exact shared badge predicate — same SQL asisVisibleBadgeRow/ the M171 backfill, so no re-literalizing and no predicate drift (exactly what was asked). The script tracks every touched session: step 4 adds on each successful insert into an existing session, step 5 adds each newly-created orphan session, and step 6 recomputes each after all inserts complete (so it sees the recovered rows).new SDKMessageRepository(db)is a valid construction (reactiveDb optional), and recompute needs onlythis.db+ the schema guard — no live-query notify, which is correct for an offline maintenance tool. - No-op pre-M171.
supportsVisibleMessageCount()guards the recompute, so the migration backfill still owns first-time population. ✓ - Test. The new repo test simulates a raw bypass INSERT (assistant + user, both visible), asserts the counter stays stale at 0, repairs via the public recompute → counter == 2 == fresh
COUNT(*), and confirms a second call is a no-op (returns false). Good coverage of the bypass→repair path. - No regression. Making the method public is safe (idempotent; consumed by script + test, so knip-clean); existing private call sites are unchanged;
mergeablereconfirmed. - Empirical. Full
4-space-storageshard: 1897/1897 pass under Vitest (incl. the new test). CIDaemon Unit Tests (4-space-storage)✅ success with zero failing completed checks; all review threads resolved (0 unresolved); PR open + mergeable.
Good call fixing it inline rather than deferring, and reusing the shared predicate was exactly the right move. Approving.
P0: 0 · P1: 0 · P2: 0 · P3: 0
lsm
left a comment
There was a problem hiding this comment.
🤖 Review by GLM-5.1 (HyperNeo)
Model: GLM-5.1 | Client: HyperNeo | Provider: HyperNeo
Follow-up: CI All Tests Pass red on 179bb2fd9 is a pre-existing flake, not a PR regression. Code approval stands.
The aggregator went red only because Daemon Unit Tests (4-space-migrations-a) failed — specifically migration-45_test.ts > "fresh DB has node column names (not step)" with a test timeout (5000ms), not an assertion failure.
Evidence it's a flake unrelated to this PR:
- Reproduces green locally. The full
4-space-migrations-ashard passes locally: 248/248, 0 fail, 45.3s (CI took 155.6s under coverage — ~3.4× slower). The file alone passes 6/6 in 2.48s. - Unrelated migration. It's Migration 45 (rename
step→nodein workflow tables). This PR adds Migration 171 (visible_message_count). The PR doesn't touchmigration-45_test.ts, migration 45, or the migration runner (git log origin/dev..HEAD -- migration-45_test.ts→ empty). - Full chain runs fine.
migration-45_test.ts:68callsrunMigrations(db)(all migrations, incl. the new 171) then asserts — it passes, so migration 171 neither breaks nor hangs the chain. - Flake signature + origin. Timeout (not assertion); the stack originates in
tests/bun-test-shim.ts:32, introduced by dev's Vitest/node:sqlite migrationfd867582f(#2364) — the same commit that last touched this test file. Not in the flaky-quarantine list.
Recommendation: rerun the failed 4-space-migrations-a job (it'll pass on retry). The durable fix — raising the per-test timeout for full-chain migration tests, or splitting the chain — is a dev-side test-infra matter out of scope for this PR. My code approval (P1 + P2 fixed and verified) stands; I'll submit for approval once CI is green.
lsm
left a comment
There was a problem hiding this comment.
🤖 Review by glm-5.1 (GLM)
Model: glm-5.1 | Client: HyperNeo | Provider: GLM
Recommendation: APPROVE — CI is now green on 179bb2fd9; all findings resolved.
Update on the CI flake: the earlier All Tests Pass red was a pre-existing flake, now confirmed empirically. I reran the single failed job Daemon Unit Tests (4-space-migrations-a) (the migration-45 timeout, unrelated to this PR) and it passed. The full PR check matrix is now green: All Tests Pass: pass, Daemon Unit Tests (4-space-storage): pass (this PR's own shard — includes the new recomputeVisibleMessageCount repair test and the Migration 171 test), Coverage Quality Gate: pass, and all daemon-online/lint/web jobs pass.
Review cycle summary (both findings fixed and verified):
- P1 — scope poisoning (
3a119076b):notifyChange(table, scope?)now accepts an optional scope;notifySessionsChanged(sessionId)passes{ sessionId };updateMessageStatuscollects changed sessions and notifies per-session. Verified: full4-space-storageshard green, plus thescoped-invalidationtests. - P2 — recover-messages bypass (
179bb2fd9):recomputeVisibleMessageCountexposed public; the recovery script tracks every session touched by its direct inserts and recomputes via the shared badge predicate (no drift; no-op pre-M171). Verified by a new repo test simulating a bypass insert → repair.
Merge readiness: OPEN, MERGEABLE (CLEAN), 8/8 review conversations resolved (0 unresolved), CI green. The PR's own tests pass in CI; the only prior red was the unrelated migration-45 timeout flake, now cleared on rerun. Ready to merge into dev (squash, no --delete-branch).
Submitting for human approval (space autonomy requires sign-off).
Replaces the correlated
COUNT(*)oversdk_messagesthatspaceSessions.bySpaceran for every session on every 150ms-debounced re-evaluation (~92ms warm for dev-neokai) with a maintainedsessions.visible_message_countcolumn that the query reads directly.What changed
visible_message_count INTEGER NOT NULL DEFAULT 0onsessions.ALTER TABLE+ one-time backfill using the badge visibility predicate. (Renumbered twice — dev shipped 169 formessage_subtype_norm(perf(daemon): make sdk_messages subtype filters sargable (#2330) #2346) and 170 for preset-agent backfill (fix(space): backfill missing preset agents into existing Spaces [#846] #2370), both since rebased onto.)SDKMessageRepository: increments on visible inserts (saveSDKMessage/saveUserMessage/saveHyperNeoActionMessage); recomputes affected sessions onsend_statustransitions, rewind deletes (deleteMessagesAt/After), and pending-message removal. Each mutation + counter update is wrapped in one transaction (FTS search-index work stays outside, best-effort) so an FTS throw can't strand the counter. The visibility predicate (top-level rows, non-deferred user rows, non-hidden subtypes) is centralized inisVisibleBadgeRow.reactiveDbinto the repo;notifyChange('sessions')fires after the counter transaction commits and before the fallible FTS work, so the live badge re-evaluates on every counter-changing event. Wired through the facade and the space-runtime write path.spaceSessions.bySpace: selectss.visible_message_countinstead of the per-session subquery.Maintenance no-ops gracefully when the column/table is absent, so partial-schema test harnesses are unaffected.
Verification
spaceSessions.bySpaceno longer runs a per-sessionCOUNT(*)(decoupling test); a visible save / status-flip-into-visibility triggers a live re-eval with the new count, while an invisible save does not (reactivity test).COUNT(*)).bun run check(lint/typecheck/knip/db-schema-parity) green; storage/runtime/handlers tests pass under Vitest.Notes
check:test-qualityflags one pre-existing issue inprovider-registry.test.ts(unmodified).