Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions DEVELOPER_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ For indexing, a nonblank invalid environment value retains the existing warning-

## Build & Test

`audit --all --progress` uses `ConsoleUi.AuditProgress` with a monotonic clock and a synchronized timer independent of the sequential child query. Start/terminal notifications bypass the one-second heartbeat throttle; each ASCII payload is at most 256 characters. Only numeric recipe/query ordinals are published, preventing custom identifiers from leaking private data. The renderer checks stderr's own terminal state, crops redraws to terminal width, flushes captured lines, and stops emission under its gate before disposal. Output serialization completes before the terminal state is selected, including byte-budget failures and partial results accepted with `--allow-partial`. Existing quiet and disabled-animation controls override explicit progress; recipe matching, SQLite cancellation, and audit deadlines are unchanged.

Initial full-index implementation notes / 初回フルインデックスの実装ノート:
[Initial full-index performance](docs/initial-index-performance.md).

Expand Down Expand Up @@ -4313,6 +4315,8 @@ For symmetry, the MCP server no longer echoes raw `Exception.Message` content in
<a id="開発者ガイド"></a>
# 開発者ガイド

`audit --all --progress` は `ConsoleUi.AuditProgress` を使い、単調増加時計と同期された timer により、逐次実行する子 query の処理中も進捗を通知します。開始/終了通知を除く heartbeat は毎秒最大1回で、ASCII payload は最大256文字です。recipe/query は番号だけを出力し、カスタム識別子からの機密情報漏洩を防ぎます。stderr 自身の端末状態を確認し、再描画は端末幅で切り詰め、取得用の行は flush し、破棄前に gate 内で通知を停止します。出力のシリアライズ後に終了状態を決定するため、byte 上限の失敗と `--allow-partial` で許可された不完全な結果も区別します。既存の quiet と animation 無効化指定は明示的な progress より優先され、recipe matching、SQLite cancellation、audit deadline は維持されます。

### 索引のファイルサイズ方針

索引作成時の空白以外の不正な環境変数値は、従来どおり警告して既定値へ戻します。有効な上限の明示指定があれば、そちらを優先します。
Expand Down
6 changes: 6 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,10 @@
agents, MCP clients, and LSP-native editors can run fast full-text, symbol,
dependency, and inspection queries without rescanning the same tree for every query.

## Audit progress

For captured audit liveness, use `cdidx audit --all --summary-only --json --progress`. Progress goes to stderr while stdout retains JSON/NDJSON. `--quiet` and `--no-progress` override it regardless of argument order. See [audit progress](USER_GUIDE.md#audit-progress).

## File-size limits and freshness

For indexing, a nonblank invalid `CDIDX_MAX_FILE_BYTES` value falls back to the 4 MiB default, as the warning indicates, unless an explicit valid limit is supplied.
Expand Down Expand Up @@ -286,6 +290,8 @@ For commercial use, integration, and naming guidance, see

# cdidx(日本語)

audit の出力を取得しながら実行状況を確認するには、`cdidx audit --all --summary-only --json --progress` を使います。進捗は stderr に出力し、stdout の JSON/NDJSON を維持します。`--quiet` と `--no-progress` は引数の順序に関係なく優先されます。[audit の進捗](USER_GUIDE.md#audit-の進捗)を参照してください。

### ファイルサイズ上限と鮮度チェック

索引作成時に空白以外の不正な`CDIDX_MAX_FILE_BYTES`が指定されている場合、有効な上限の明示指定がなければ、警告の案内どおり既定の4 MiBに戻ります。
Expand Down
8 changes: 8 additions & 0 deletions TESTING_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,10 @@ If you change test code, test helpers, test execution flow, or testing conventio

## Quick Start

The #5260 redraw test also pauses the renderer, proves later timer ticks emit nothing during result output, and checks that clearing precedes the result and the terminal notification follows it.

Issue #5260 coverage in `QueryCommandRunnerAuditProgressIssue5260Tests` waits for a real heartbeat while a child query is held, verifies JSON/NDJSON stdout and private-identifier exclusion, and exercises cancellation, query/output failures, the deadline, global quiet/no-progress precedence, flag scope, and deterministic monotonic rate/redraw/disposal behavior on both frameworks. Keep console mutations inside the console-sensitive collection and restore captured writers and test budgets in `finally`.

Issue #5258 coverage in `IndexCommandRunnerIssue5258Tests.cs` checks explicit/environment/default size limits, ordinary root and shared-workspace checks, scoped updates, full refreshes, missing/invalid legacy policy, actual deletion, failed reads, MCP persistence/dry runs, and the retained read ceiling on both frameworks. Keep environment and current-directory changes inside the SQLite pool sensitive collection and restore them in `finally`.

```bash
Expand Down Expand Up @@ -1177,6 +1181,10 @@ Check the following:
<a id="テストガイド"></a>
# テストガイド

#5260 の再描画テストは、renderer の一時停止後の timer tick が結果出力中に何も書かないことと、行消去・結果出力・終了通知の順序も確認します。

Issue #5260 の `QueryCommandRunnerAuditProgressIssue5260Tests` は子 query を待機させて実際の heartbeat を確認し、stdout の JSON/NDJSON とカスタム識別子の非公開、取消、query/出力の失敗、deadline、quiet/no-progress の優先順位、flag の適用範囲、単調増加時計による通知頻度と再描画/破棄を両 framework で検証します。console の変更は console-sensitive collection 内で行い、writer とテスト用 budget を `finally` で復元してください。

`IndexCommandRunnerIssue5258Tests.cs`のIssue #5258検証は、明示指定・環境変数・既定のサイズ上限、通常のルート/共有workspaceチェック、部分更新、全体更新、旧方針の欠落・不正値、実際の削除、読み取り失敗、MCPの保存とdry run、読み取り上限の維持を両フレームワークで確認します。環境変数とカレントディレクトリの変更はSQLite pool sensitiveコレクション内に置き、`finally`で復元してください。

このドキュメントは、`cdidx` のテストスイートがどう構成されているか、どのように安全にテストを追加・更新するか、そして挙動やテスト基盤を変更したときに従うべき規約をまとめたものです。
Expand Down
31 changes: 28 additions & 3 deletions USER_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -1622,15 +1622,27 @@ details, 32 returned errors, a five-minute deadline enforced within each query,
and a 4 MiB JSON response when no explicit `--max-json-bytes` is supplied.
Structured rows are admitted incrementally against that byte budget, and an
active SQLite read is interrupted when its scoped deadline is cancelled.
Interactive terminals show progress on stderr; use `--no-progress` to suppress
it, and machine-readable stdout remains clean. A requested `--total-limit` truncation
Interactive terminals show progress on stderr (see below), and machine-readable stdout remains clean. A requested `--total-limit` truncation
is a successful bounded result. Query failures do not discard successful
sibling recipes: the run continues, records bounded errors, and returns partial
exit code 11; `--allow-partial` accepts that incomplete result with exit code 0.
Cancellation returns exit code 8 with the completed/omitted accounting retained.
Use an individual `cdidx audit <recipe>` for SARIF, issue drafts, child-query
cursoring, or recipe-specific aggregation. `cdidx recipes` and `cdidx batch`
remain the lower-level discovery and explicit orchestration tools.

#### Audit progress

Progress pauses and clears any terminal redraw before result output, then emits the final status after output finishes. Database-open failures have no live redraw to corrupt the error output.

The compact terminal labels `r`, `q`, `done` (recipes/queries), `fail`, and `ms` represent the same fields described below for captured lines.

Use `cdidx audit --all --summary-only --json --progress` to opt in when stderr is captured or redirected. `--progress` is valid only with `audit --all`. Without it, non-terminal stderr stays silent; terminal stderr shows progress automatically. `--quiet` (including its aliases and `CDIDX_QUIET`) and `--no-progress` win regardless of argument order. `CDIDX_DISABLE_PROGRESS` and `PREFERS_REDUCED_MOTION` also suppress it.

Progress keeps stdout in its requested format. Stderr receives an immediate start, at most one heartbeat per second even within a slow child query, and one terminal notification. Each ASCII payload is at most 256 characters, followed by the platform newline for captured output. Terminals redraw one line cropped to their width and finish with a newline. `elapsed_ms` uses a monotonic clock. `recipes_completed` and `queries_completed` show successful/selected counts; `queries_failed` counts failed children. No percentages or estimates of remaining time are used. `active_recipe` is the one-based ordinal in the selected, name-sorted registry; `active_query` is the one-based query ordinal within that recipe. Zero means no active child. These identifiers correspond to the result order; names, paths, query text, and exception details are never included in progress.

The final state is `completed`, `partial` (including result/byte limits, deadline, or child failures), `cancelled`, or `failed` (command/output failure). `--allow-partial` changes the exit policy but does not relabel partial progress as completed. Progress does not change the five-minute audit budget or cancellation behavior.

Built-in recipe queries may also include `risk_evidence`, a short set of
positive and negative evidence facets that explain why a hit is risky or likely
bounded/safe. Recipe run JSON repeats those facets on each matching result so
Expand Down Expand Up @@ -5439,13 +5451,26 @@ accumulation は query ごとに最大 10,000 candidate row、返却する query
32 件、各 query 内でも強制される実行 deadline は5分、明示的な `--max-json-bytes` がない JSON response は
4 MiB に制限されます。structured row はこの byte budget に対して逐次受理され、scope 付き deadline が
cancel されると実行中の SQLite read も interrupt されます。interactive terminal の progress は stderr
へ出力され、`--no-progress` で抑止できます。機械可読な stdout は progress と混在しません。
へ出力されます(下記参照)。機械可読な stdout は progress と混在しません。
指定した `--total-limit` による truncation は正常な上限付き結果です。query failure が起きても
成功済み sibling recipe は破棄せず、実行を継続して上限付き error を記録し、partial exit code 11
を返します。`--allow-partial` を指定すると不完全な状態を維持したまま exit code 0 を許可します。
cancellation は completed / omitted accounting を保持し、exit code 8 を返します。SARIF、issue draft、
child-query cursor、recipe 固有 aggregation が必要な場合は個別の `cdidx audit <recipe>` を使ってください。
`cdidx recipes` と `cdidx batch` は lower-level の discovery / 明示的 orchestration tool として維持されます。

#### Audit の進捗

結果を出力する前に進捗を一時停止して端末の描画行を消去し、出力後に最終状態を通知します。DB を開く段階で失敗した場合は再描画が始まっていないため、エラー出力に混在しません。

端末で使う短いラベル `r`、`q`、`done`(recipe/query)、`fail`、`ms` は、以下の取得用出力と同じ項目を表します。

stderr を取得/リダイレクトする場合は、`cdidx audit --all --summary-only --json --progress` で明示的に有効化します。`--progress` は `audit --all` 専用です。未指定では非端末の stderr に進捗を出さず、端末の stderr では自動表示します。`--quiet`(別名と `CDIDX_QUIET` を含む)および `--no-progress` は引数の順序に関係なく優先されます。`CDIDX_DISABLE_PROGRESS` と `PREFERS_REDUCED_MOTION` も進捗を抑止します。

stdout は指定された形式を維持します。stderr には開始直後の通知、遅い子 query の実行中を含む毎秒最大1回の heartbeat、終了通知を出力します。各 ASCII payload は最大256文字で、取得用の出力には platform の改行が続きます。端末では幅に合わせて1行を再描画し、終了時に改行します。`elapsed_ms` は単調増加時計を使います。`recipes_completed` と `queries_completed` は成功件数/選択件数、`queries_failed` は失敗した子 query の件数です。百分率や残り時間の推測は表示しません。`active_recipe` は名前順に選択した registry の1始まりの番号、`active_query` はその recipe 内の1始まりの query 番号です。0 は実行中の子 query がないことを示します。番号は結果の順序に対応し、名前、パス、query text、例外の詳細は進捗に含めません。

終了状態は `completed`、`partial`(結果/byte 上限、deadline、子 query の失敗を含む)、`cancelled`、`failed`(command/出力の失敗)です。`--allow-partial` は終了コードの扱いだけを変え、不完全な進捗を completed に変更しません。5分の audit budget と cancellation の挙動は維持されます。

組み込み recipe には `risky-code`、`json-parse-apis`、`dotnet-risk-patterns`、
`auth-token-audit`、`string-comparison-semantics`、`dogfood-risk-patterns`、
`sqlite-query-policy-surfaces`、`unsupported-operation-boundaries`、`xml-parser-security`、
Expand Down
16 changes: 16 additions & 0 deletions changelog.d/unreleased/5260.added.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
---
category: added
issues:
- 5260
affected:
- src/CodeIndex/Cli/QueryCommandRunner.AuditAll.cs
- src/CodeIndex/Cli/ConsoleUi.AuditProgress.cs
---

## English

- **Captured audit progress (#5260)** — `audit --all --progress` emits bounded stderr heartbeats during slow child queries while preserving JSON/NDJSON stdout. Progress reports active numeric recipe/query identifiers, completed/selected counts, monotonic elapsed time, and accurate terminal states; quiet/no-progress controls take precedence.

## 日本語

- **取得した audit 出力で進捗を確認 (#5260)** — `audit --all --progress` は遅い子 query の実行中にも上限付き heartbeat を stderr に出力し、stdout の JSON/NDJSON を維持します。実行中の recipe/query 番号、完了/選択件数、単調増加時計による経過時間、終了状態を報告し、quiet/no-progress の指定を優先します。
3 changes: 2 additions & 1 deletion src/CodeIndex/Cli/CliFlagSchema.cs
Original file line number Diff line number Diff line change
Expand Up @@ -707,7 +707,8 @@ private static IReadOnlyList<CliFlag> BuildAll()
new() { Name = "--exclude-comments", Description = "Search/Audit: suppress comment-only matches after origin classification", PrimaryCommands = Set("search", "audit") },
new() { Name = "--exclude-strings", Description = "Search/Audit: suppress string, regex, and help-text matches after origin classification", PrimaryCommands = Set("search", "audit") },
new() { Name = "--exclude-fixtures", Description = "Search/Audit: suppress fixture-only matches in tests after origin classification", PrimaryCommands = Set("search", "audit") },
new() { Name = "--no-progress", Description = "Disable animated progress and spinner output", PrimaryCommands = Set(AllCommands.ToArray()), TopLevel = true },
new() { Name = "--progress", Description = "Audit --all: opt in to bounded stderr progress, including captured JSON output; quiet/no-progress take precedence", PrimaryCommands = Set("audit") },
new() { Name = "--no-progress", Description = "Disable animated progress, spinner output, and explicit audit progress", PrimaryCommands = Set(AllCommands.ToArray()), TopLevel = true },
new() { Name = "--name", ValuePlaceholder = "<name>", Description = "Exact symbol name", PrimaryCommands = Set("symbols") },
new() { Name = "--max-line-width", ValuePlaceholder = "<n>", Description = "Clamp long single-line payloads (0 disables clamping)", PrimaryCommands = Set(MaxLineWidthCommands) },
new() { Name = "--snippet-lines", ValuePlaceholder = "<n>", Description = "Snippet length; graph --body uses it for definition and centered call-site evidence; issue-drafts accept 0 for path/line-only evidence", PrimaryCommands = Set("search", "audit", "find", "references", "callers", "callees", "impact") },
Expand Down
Loading
Loading