fix(cache): don't promote truncated LLM chunks to the semantic cache as complete

[TPAteeq]

Problem

When an LLM chunk is truncated (finish_reason="length") and can't be recovered by splitting — or the adaptive-retry recursion hits its depth cap — the extractor keeps the partial node set it managed to parse. That partial result was then checkpointed and written to the content-hash semantic cache and manifest-stamped as a complete extraction. Because the cache is keyed by file content, the incomplete node set is served on every subsequent run forever — until the file's content changes or the user passes --force. Half a document's symbols silently masquerade as the whole, with no persisted signal that anything was missing.

Fix

A truncated give-up result is tagged with an internal _partial marker on each of its items. save_semantic_cache stamps the affected file's cache entry partial: True (detected either from that marker or from an explicit partial_source_files argument), and load_cached treats a partial entry as a cache miss, so the file is re-dispatched and retried instead of replayed. To make that re-dispatch actually happen on the common path, the partial file is also left unstamped in the manifest — mirroring the existing failed-chunk handling (#933) — so detect_incremental re-queues it on the next incremental update/extract, not only on a full/--force/content-change run.

A file split into slices across chunks is handled correctly: the merge_existing checkpoint peeks at a partial previous entry (load_cached(allow_partial=True)) so it accumulates all slices and keeps the entry partial, rather than a later clean slice silently dropping a truncated one and promoting the half-file to complete. The whole thing self-heals — a later complete extraction overwrites the same content-hash key with a non-partial entry — and the internal marker is stripped after the final save so it never leaks into graph.json.

Tests

tests/test_partial_cache.py and additions to tests/test_chunking.py: partial entry is a load miss; explicit partial_source_files; self-heal on complete re-extraction; sliced-file accumulation stays partial without losing a slice; manifest excludes partial files; the marker helpers; and an end-to-end truncated extract_corpus_parallel checkpoint that reloads as a miss. Full test suite green (3297 passed).

[safishamsi]

Landed on v8 (ships in 0.9.18), cherry-picked to preserve your authorship. Thanks @TPAteeq — truncated chunks are no longer cached/stamped as complete. Follow-up added for the empty-parse case: a mid-JSON cut parses to zero items, so the give-up sites now record the chunk own files independently of parsed items (propagated through the merges), the cache stamps a partial file even with no items, and a later clean slice cannot re-promote a partial entry. That closes the sliced-document scenario end to end.