Skip to content

fix: report the typed-fact backlog nothing will drain - #2322

Merged
JBailes merged 3 commits into
testingfrom
aimee/session/facts-backlog
Aug 4, 2026
Merged

fix: report the typed-fact backlog nothing will drain#2322
JBailes merged 3 commits into
testingfrom
aimee/session/facts-backlog

Conversation

@JBailes

@JBailes JBailes commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Two gates that do not agree

memory.store enqueues a memory_facts job whenever typed_facts_enabled is on — the default:

if (config_typed_facts_enabled() && out.id > 0)
   (void)db2_kb_async_enqueue("memory_facts", out.id, "memory");

The only consumer of those jobs is kb_memory_facts_drain, which runs on the curator LLM lane. That lane deliberately does not start without a synthesis endpoint:

/* NO SYNTHESIS PROVIDER => NO LLM LANE. ... running them anyway is not merely idle */

Both decisions are individually correct. Together they mean an install with no synthesis provider — a supported configuration — enqueues one row per stored memory that nothing will ever claim.

Found by exploratory testing, not by a failing check

On the e2e VM:

  • 4 memory_facts jobs pending for 11.5 hours, attempts=0 — never even claimed
  • /v1/healthstatus: ok, warnings: [], blockers: []
  • aimee kb statusQueue: 4 pending, 0 running, 0 failed, with nothing to say that pending here means forever

It grows without bound for the life of the install. Same family as #2319 and #2321: the evidence existed, the summary did not carry it.

Reported as a warning, not a blocker

Running without a synthesis provider is explicitly supported, and memory store and search work perfectly — only typed-fact enrichment is deferred. Degrading the verdict for a supported configuration is exactly the dilution the status derivation added in #2319 exists to avoid. This is the distinction that PR built, being used as intended.

The message also says the backlog is not lost: configuring a provider drains it. That is why the fix is not "stop enqueueing" — the queue is a backfill, and dropping it would silently forfeit facts for every memory stored before a provider was configured.

Counting needs pending, not total

db2_kb_async_count_kind counts every row regardless of status. That cannot distinguish a queue that is draining from one that never will — which is the entire distinction being reported — and would keep warning about work that already completed. Added db2_kb_async_count_kind_pending alongside it.

Verification

Test asserts pending tracks the backlog rather than the row total: two enqueued → 2 pending / 2 total; one marked done → 1 pending / 2 total; an extract_doc row in the shared table is not counted in. Red verified by dropping the status = 'pending' filter — fails on exactly the pending-vs-total assertion.

Lockfile: kb-synthesis source pin resynced, recomputed with the exporter's own digest functions.

Gate: clang-format → make lint (41/41) → check-docs.pycheck_c_repository_lock.py — all green.

JBailes added 3 commits August 4, 2026 09:57
Two gates that do not agree.

memory.store enqueues a "memory_facts" job whenever typed_facts_enabled is
on, which is the DEFAULT. The only consumer of those jobs is
kb_memory_facts_drain, which runs on the curator LLM lane -- and that lane
deliberately does not start without a synthesis endpoint ("NO SYNTHESIS
PROVIDER => NO LLM LANE"). Both decisions are individually correct.
Together they mean an install with no synthesis provider, which is a
supported configuration, enqueues one row per stored memory that nothing
will ever claim.

Found by exploratory testing of :testing, not by a failing check. On the
e2e VM: 4 memory_facts jobs pending for 11.5 hours with attempts=0, while
/v1/health reported status ok with an empty warnings array and `aimee kb
status` printed "4 pending" with nothing to say that pending here means
forever. It grows without bound for the life of the install.

Reported as a WARNING, not a blocker. Running without a synthesis provider
is explicitly supported and memory store and search work perfectly; only
typed-fact enrichment is deferred. Degrading the verdict for a supported
configuration is exactly the dilution the status derivation exists to
avoid. The message says the backlog is not lost: configuring a provider
drains it.

Counting needs PENDING specifically, so add
db2_kb_async_count_kind_pending alongside the existing total. Total
conflates work still waiting with work long finished and cannot tell a
queue that is draining from one that never will -- which is the entire
distinction being reported. Test red-verified by dropping the status
filter.
The kb target compiles kb_service_kb.c with a different include set than
the server target, so config_synth_chat_endpoint_current was an implicit
declaration there and `make ../aimee-kb` failed under -Werror. The
container build is that target, so the image build broke while the
default target still passed.
The backlog warning added in the previous commit reached /v1/health and
stopped there. server_health_add_kb passed blockers through and dropped
the kb's warnings array on the floor, and the status renderer only ever
printed blockers.

Verified live before this commit: the kb reported the 4-job typed-fact
backlog in its warnings array, while `aimee status` printed "aimee-kb: ok"
with four clean detail lines and `aimee kb status` printed a bare "4
pending". Publishing a finding into a field no surface renders is the same
defect as never computing it -- the evidence exists and the summary does
not carry it, which is the whole shape this series is about.

Warnings are advisory by construction: they do not move the verdict, and
that is exactly why they need printing. A kb that is genuinely ok can
still be accumulating work nothing will process, and a status line that
renders only blockers reports that as a clean bill of health.

Rendered as "note:" to keep it visibly distinct from "BLOCKED:", which
means cannot work.

Extracted kb_health_add_lines: two copies of the same newline-splitting
arithmetic is how the warnings array came to be dropped to begin with.
Test asserts warnings survive with the verdict unchanged, and that the
blockers key is absent rather than empty when there are none. Red verified
by removing the pass-through.
@JBailes
JBailes merged commit a810209 into testing Aug 4, 2026
28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant