ES/OS _count is refresh-scoped and ignores _shards.failed — corpus-size reads can mislead

[fcostaoliveira]

Follow-up from #271 (#238).

#271 added Engine::corpus_row_count(); for Elasticsearch and OpenSearch it reads GET /<index>/_count. Two caveats remain:

1. Refresh visibility. Both engines set refresh_interval: -1 for the duration of a load and refresh explicitly at the end. An upload killed between the last bulk and the refresh() leaves documents durably indexed but invisible to _count, so a later --skip-upload reads an empty or short corpus and hard-errors over intact data. A _refresh (or _count with the segment stats) before the probe would settle it.
2. Partial shard results. fix(skip-flags): --skip-upload must reuse the corpus, not destroy it (#238) #271 now rejects a reply with _shards.failed > 0 rather than treating a partial count as the corpus size — but the same blind spot exists in other _count/_search reads in these engines, which check the HTTP status only. A degraded cluster returns HTTP 200 with fewer shards answering.

Co-Authored-By: Claude Opus 5 (1M context) noreply@anthropic.com