Skip to content

fix(core): tune provider generation and drift coverage#170

Open
parse wants to merge 13 commits intomainfrom
parse/update-models
Open

fix(core): tune provider generation and drift coverage#170
parse wants to merge 13 commits intomainfrom
parse/update-models

Conversation

@parse
Copy link
Copy Markdown
Contributor

@parse parse commented May 4, 2026

Summary

Refactors provider generation controls from shared raw token budgets to provider-agnostic generation intent profiles, with provider-local translation for OpenAI, Gemini, and Anthropic. Also updates provider defaults/docs, adds ADR documentation, fixes web lint coverage in CI, improves drift model-update coverage, and keeps the simplified post-commit hook cleanup because the previous ADR automation did not work.

Consumer-facing changes

  • OpenAI defaults changed from gpt-4.1-mini / gpt-4.1 to gpt-5-mini / gpt-5.
  • Anthropic defaults changed from dated Claude snapshots to claude-haiku-4-5 / claude-sonnet-4-6.
  • OpenAI provider calls now use the Responses API with max_output_tokens and reasoning effort instead of Chat Completions max_tokens.
  • Gemini defaults remain gemini-2.5-flash / gemini-2.5-pro; generation controls are model-aware so Gemini 2.5 uses thinkingBudget and Gemini 3-style models use thinkingLevel.
  • Model patching now carries a dynamic output-size hint so larger LikeC4/Structurizr files can receive enough output budget for complete-file patch responses.
  • Drift analysis now receives dependency evidence snippets and must account for every added dependency when proposing model updates, reducing missed relationships in multi-component changes.
  • Users with restricted model access, cost/latency constraints, or provider quota limits may need explicit ERODE_*_FAST_MODEL and ERODE_*_ADVANCED_MODEL overrides.
  • The local post-commit hook no longer tries to run ADR automation; it now only emits a simple success message.

Validation

  • npm run typecheck --workspace=packages/core
  • npm run lint
  • npm run test
  • npm run build
  • npm run check:ci

Summary by CodeRabbit

  • New Features

    • Introduced intent-based generation profiles to control output size and reasoning effort across analysis phases.
    • OpenAI Responses API support with configurable timeout and improved truncation/error handling.
  • Refactor

    • Providers now use generation profiles to determine output limits and reasoning settings.
  • Tests

    • Expanded unit tests covering providers, truncation handling, formatting, and generation profiles.
  • Chores

    • Updated default AI model names and docs; added OpenAI environment variables and timeout.
  • Style

    • Simplified post-commit hook.

@parse parse requested a review from a team as a code owner May 4, 2026 05:20
Copy link
Copy Markdown

@greptile-apps greptile-apps Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your free trial has ended. If you'd like to continue receiving code reviews, you can add a payment method here.

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 4, 2026

Warning

Rate limit exceeded

@parse has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 19 minutes and 26 seconds before requesting another review.

To continue reviewing without waiting, purchase usage credits in the billing tab.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 1c0dbcc0-8e0d-4330-9376-b214a8409012

📥 Commits

Reviewing files that changed from the base of the PR and between 2927b1b and 4271caf.

📒 Files selected for processing (10)
  • packages/core/src/providers/__tests__/generation-profile.test.ts
  • packages/core/src/providers/anthropic/__tests__/provider.test.ts
  • packages/core/src/providers/anthropic/provider.ts
  • packages/core/src/providers/base-provider.ts
  • packages/core/src/providers/gemini/__tests__/provider.test.ts
  • packages/core/src/providers/gemini/provider.ts
  • packages/core/src/providers/generation-profile.ts
  • packages/core/src/providers/openai/__tests__/provider.test.ts
  • packages/core/src/providers/openai/provider.ts
  • packages/web/package.json
📝 Walkthrough

Walkthrough

Refactors provider callsites to use intent-based GenerationProfile objects, implements profile-to-provider translation in Anthropic/OpenAI/Gemini providers, updates default model identifiers (OpenAI → gpt-5 / gpt-5-mini; Anthropic → claude-haiku-4-5 / claude-sonnet-4-6), and updates schemas, tests, docs, and small config/tooling.

Changes

Generation profiles & provider refactor

Layer / File(s) Summary
Data Shape / Types
packages/core/src/providers/generation-profile.ts
Adds GenerationProfile, OutputSize, ReasoningEffort, getGenerationProfileForPhase, and getGenerationProfileForModelPatch.
Core Provider API
packages/core/src/providers/base-provider.ts
Abstract callModel now accepts generationProfile; executeStage, selectComponent, and patchModel updated to derive/use profiles; timing/logging added.
Anthropic Implementation
packages/core/src/providers/anthropic/provider.ts, packages/core/src/providers/anthropic/models.ts
callModel consumes GenerationProfile, computes max_tokens via per-size baselines and optional content-hint; model constants updated to claude-haiku-4-5 / claude-sonnet-4-6.
OpenAI Implementation
packages/core/src/providers/openai/provider.ts, packages/core/src/providers/openai/models.ts
Switches to Responses API, derives max_output_tokens and optional reasoning.effort from profiles, adds incomplete-response handling and text extraction helpers; model constants updated to gpt-5-mini / gpt-5.
Gemini Implementation
packages/core/src/providers/gemini/provider.ts
callModel consumes GenerationProfile, computes maxOutputTokens and thinkingConfig per Gemini family, and throws on truncation finish reasons; helper functions and guards added.
Tests & ADR
packages/core/src/providers/__tests__/*, packages/core/src/providers/*/__tests__/*, packages/core/src/providers/__tests__/generation-profile.test.ts, packages/web/src/content/docs/contributing/adrs/011-*
Adds/tests update expectations for profile-driven behavior, new generation-profile tests, and ADR-011 documenting intent-based profiles.

Model identifier, schema, docs & env

Layer / File(s) Summary
Core Model Constants
packages/core/src/providers/anthropic/models.ts, packages/core/src/providers/openai/models.ts
Update exported model identifiers: Anthropic FAST → claude-haiku-4-5, ADVANCED → claude-sonnet-4-6; OpenAI FAST → gpt-5-mini, ADVANCED → gpt-5.
JSON Schemas
packages/core/schemas/eroderc.schema.json, packages/web/public/schemas/v0/eroderc.schema.json
Schema defaults for Anthropic/OpenAI model fields updated to the new identifiers.
Env Example
.env.example
Adds ERODE_OPENAI_API_KEY, ERODE_OPENAI_TIMEOUT=60000, commented OpenAI model defaults, and anthopic placeholders without dated suffixes.
Documentation
packages/web/src/content/docs/docs/reference/ai-providers.md
Docs updated to reflect new default model names.
Provider Factory Tests
packages/core/src/providers/__tests__/provider-factory.test.ts
Mocks and expectations updated to use new OpenAI model identifiers.

Drift-analysis prompt & evidence formatting

Layer / File(s) Summary
Prompt Template
packages/core/src/analysis/prompts/drift-analysis.md
Adds “Dependency Coverage” section requiring explicit handling/classification of every added dependency.
Formatter Logic
packages/core/src/analysis/section-formatters.ts
formatDependencyChanges now includes conditional Evidence: blocks when dependency code is present.
Tests
packages/core/src/analysis/__tests__/prompt-builder.test.ts, packages/core/src/analysis/__tests__/section-formatters.test.ts
Adds tests asserting evidence inclusion and that the drift template contains the new dependency-coverage instructions.

Tooling, config, and small infra changes

Layer / File(s) Summary
ESLint exports & web config
packages/eslint-config/base.js, packages/eslint-config/index.js, packages/eslint-config/index.d.ts, packages/web/eslint.config.mjs
Adds and re-exports disableTypeChecked and applies it to **/*.astro files in the web ESLint config.
Web CI script
packages/web/package.json
check:ci script now runs npm run lint before npm run typecheck && astro build.
Git hook
.githooks/post-commit
Replaced complex ADR-generation background hook with a simple echo and exit.
Tests (wiring/coverage)
packages/core/src/providers/*/__tests__/*
Extensive test updates across providers to assert new request shapes, truncation handling, and profile-driven budgets.

Sequence Diagram(s)

sequenceDiagram
    participant Orch as Analysis Orchestrator
    participant Base as BaseProvider
    participant Prov as Concrete Provider
    participant SDK as LLM SDK

    Orch->>Base: executeStage(phase, prompt, optional generationProfile?)
    Base->>Base: profile = getGenerationProfileForPhase(phase) (if absent)
    Base->>Prov: callModel(model, prompt, phase, generationProfile)
    Prov->>Prov: compute token/effort (max tokens / reasoning / thinkingConfig) from profile
    Prov->>SDK: send request with computed config
    SDK-->>Prov: response (complete or incomplete/truncated)
    Prov->>Base: return normalized text or throw ErodeError (truncation/safety)
    Base->>Orch: return result / propagate error
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

Poem

🐰 I hopped through profiles in the night,
Mapped tokens, effort, set things right,
Claude and GPT got tidy names,
Tests and docs joined eager games,
A rabbit cheers the new provider light.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The pull request title 'fix(core): tune provider generation and drift coverage' clearly and concisely summarizes the main changes: tuning provider generation controls and improving drift analysis coverage. The title directly reflects the core refactoring of generation profiles and drift model-update improvements detailed in the changeset.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch parse/update-models

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@packages/web/public/schemas/v0/eroderc.schema.json`:
- Around line 212-218: The schema's default model IDs for the "fastModel" and
"advancedModel" properties are invalid; update the defaults in the
eroderc.schema.json to use real OpenAI model identifiers (e.g., replace
"gpt-5.3-mini" with a valid mini model like "gpt-5.4-mini" or "gpt-5-mini/nano"
and replace "gpt-5.3" with a valid full model like "gpt-5.4" or "gpt-5.5") so
API calls won't fail—ensure you only use model names listed in the OpenAI models
docs and update both "fastModel" and "advancedModel" default values accordingly.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: a9556923-6e87-4f4e-842f-dafe0044a3d6

📥 Commits

Reviewing files that changed from the base of the PR and between 18d8a4e and 18267db.

📒 Files selected for processing (9)
  • .env.example
  • packages/core/schemas/eroderc.schema.json
  • packages/core/src/providers/__tests__/provider-factory.test.ts
  • packages/core/src/providers/anthropic/__tests__/provider.test.ts
  • packages/core/src/providers/anthropic/models.ts
  • packages/core/src/providers/openai/__tests__/provider.test.ts
  • packages/core/src/providers/openai/models.ts
  • packages/web/public/schemas/v0/eroderc.schema.json
  • packages/web/src/content/docs/docs/reference/ai-providers.md

Comment thread packages/web/public/schemas/v0/eroderc.schema.json
@parse parse changed the title update models for anthropic and openai chore: update models for anthropic and openai May 4, 2026
@parse parse marked this pull request as draft May 4, 2026 05:55
parse added 6 commits May 4, 2026 12:07
Move output sizing intent into shared generation profiles.

Keep provider-specific token parameters inside each AI provider.

Document the provider boundary change and add web lint coverage to CI.
Disable Gemini thinking for low-reasoning phases.

Raise Gemini output budgets and report max-token truncation clearly.
@parse parse changed the title chore: update models for anthropic and openai fix(core): tune provider generation and drift coverage May 5, 2026
@parse parse marked this pull request as ready for review May 5, 2026 16:50
@greptile-apps
Copy link
Copy Markdown

greptile-apps Bot commented May 5, 2026

Greptile Summary

This PR refactors how AI providers control generation output by replacing raw per-call maxTokens arguments with intent-based GenerationProfile objects (outputSize + reasoningEffort + optional outputContentHint), with each provider translating profiles to its own token/effort APIs. It also migrates the OpenAI provider from Chat Completions to the Responses API, updates default model names for OpenAI and Anthropic, improves drift-analysis prompt coverage for multi-dependency changes, and adds code-fence stripping for patchModel responses.

  • Generation profiles: New generation-profile.ts module maps analysis phases to provider-agnostic output intents; each provider holds its own OutputSize → token table and converts ReasoningEffort to its native API field.
  • OpenAI Responses API: callModel now calls client.responses.create with max_output_tokens and optional reasoning.effort; handleIncompleteResponse handles truncation/safety/unknown incomplete reasons, and extractText has a fallback path that walks response.output message items.
  • Dynamic patch budget: getGenerationProfileForModelPatch estimates output size from file length and emits an outputContentHint that resolveOutputTokenLimit promotes above the base bucket ceiling, ensuring large architecture files get enough headroom.

Confidence Score: 5/5

Safe to merge; all provider paths are covered by tests and the generation-profile abstraction is well-bounded.

The generation-profile refactor is self-contained and well-tested across all three providers. The OpenAI migration to the Responses API is the largest surface area change, but the new handleIncompleteResponse now correctly throws for every known and unknown incomplete reason. The Gemini thinking-config dispatch correctly handles 2.5, 3-dash, and 3-dot point-release model naming via a clean predicate chain verified by the new tests. No token-budget regressions were introduced: CHANGE_ANALYSIS now reaches Anthropic's large bucket (8192 tokens) and MODEL_UPDATE gets a dynamic content hint that promotes the limit well above the base bucket.

No files require special attention; the two findings are non-blocking style improvements.

Important Files Changed

Filename Overview
packages/core/src/providers/generation-profile.ts New module defining OutputSize/ReasoningEffort types, phase-to-profile mapping, dynamic patch profile, and token limit resolution; logic is clean and well-tested.
packages/core/src/providers/openai/provider.ts Migrated to Responses API with generation profiles; handleIncompleteResponse always throws but is typed as void instead of never, creating a minor TypeScript flow-analysis gap.
packages/core/src/providers/gemini/provider.ts Adds thinking-config dispatch for Gemini 2.5 (thinkingBudget) and Gemini 3/3.x (thinkingLevel); helper functions use hoisted declarations after try-catch, which works but is an unusual code layout.
packages/core/src/providers/anthropic/provider.ts Straightforward adoption of generation profiles; CHANGE_ANALYSIS now maps to large (8192 tokens) which is appropriate for drift analysis responses.
packages/core/src/providers/base-provider.ts Removes hardcoded maxTokens from executeStage/patchModel, adds formatDuration debug logging, and moves unwrapModelPatchResponse here as a private helper.
packages/core/src/analysis/section-formatters.ts Adds indented code evidence snippets to formatted dependency changes, improving drift-analysis prompt coverage for multi-component PRs.

Sequence Diagram

sequenceDiagram
    participant BP as BaseProvider
    participant GP as generation-profile.ts
    participant P as Provider (OpenAI/Gemini/Anthropic)
    participant SDK as Provider SDK

    BP->>GP: getGenerationProfileForPhase(phase)
    GP-->>BP: GenerationProfile {outputSize, reasoningEffort, outputContentHint?}

    BP->>P: callModel(model, prompt, phase, generationProfile)
    P->>GP: resolveOutputTokenLimit(profile, sizeTable)
    GP-->>P: maxOutputTokens (max of bucket limit vs hint/4)

    alt OpenAI
        P->>SDK: responses.create({input, max_output_tokens, reasoning?})
        SDK-->>P: Response {status, output_text, output[]}
        P->>P: handleIncompleteResponse() [throws on incomplete]
        P->>P: extractText() [output_text or walk output items]
    else Gemini
        P->>P: getThinkingConfig(model, reasoningEffort)
        P->>SDK: generateContent({config: {maxOutputTokens, thinkingConfig}})
        SDK-->>P: GenerateContentResponse
    else Anthropic
        P->>SDK: messages.create({max_tokens, messages})
        SDK-->>P: Message {stop_reason, content}
    end

    P-->>BP: text string
    BP->>BP: unwrapModelPatchResponse(text) [strip markdown fences]
Loading

Reviews (4): Last reviewed commit: "fix(core): restore drift analysis output..." | Re-trigger Greptile

Comment thread packages/core/src/providers/openai/provider.ts
Comment thread packages/core/src/providers/openai/provider.ts
Comment thread packages/core/src/providers/gemini/provider.ts
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
packages/core/src/providers/anthropic/provider.ts (1)

69-76: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Anthropic truncation userMessage is less actionable than its OpenAI / Gemini counterparts.

The current message 'The AI response was truncated. The output may be partial.' doesn't tell the user what "output budget" means or what to do. The OpenAI and Gemini equivalents both say "used the available output budget before completion. Try a smaller change or tune the provider output budget".

✏️ Align with OpenAI / Gemini wording
-          'The AI response was truncated. The output may be partial.',
+          'The Anthropic response used the available output budget before completion. Try a smaller change or tune the provider output budget.',
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/core/src/providers/anthropic/provider.ts` around lines 69 - 76, The
Anthropic truncation error message is less actionable; update the ErodeError
thrown when response.stop_reason === 'max_tokens' (in the provider.ts block that
constructs ErodeError using model, phase, outputTokenLimit) to use the same
actionable wording as OpenAI/Gemini: explain that the model used the available
output budget before completion and suggest trying a smaller change or tuning
the provider output budget, while keeping the same
ErrorCode.PROVIDER_INVALID_RESPONSE and the existing metadata (model, phase,
outputTokenLimit).
🧹 Nitpick comments (3)
packages/core/src/providers/openai/provider.ts (2)

52-114: ⚡ Quick win

Silent fallthrough for unknown incomplete reasons.

When response.status === 'incomplete' with a reason other than max_output_tokens or content_filter, handleIncompleteResponse returns without throwing, and partially-truncated text from extractText is treated as a successful response. The OpenAI SDK types incomplete_details.reason as an open string, so future or undocumented reasons would be swallowed silently and propagate corrupted data downstream into JSON parsing/validation.

Consider a final throw for unknown reasons so the caller surfaces a clear truncation error instead of a generic schema validation failure:

♻️ Proposed catch-all branch
       if (response.incomplete_details?.reason === 'content_filter') {
         throw new ErodeError(
           'OpenAI safety filters blocked the response',
           ErrorCode.PROVIDER_SAFETY_BLOCK,
           'Content was blocked by the AI provider safety filters. Try simplifying the input.',
           { model: incompleteModel, phase: incompletePhase }
         );
       }
+
+      throw new ErodeError(
+        'OpenAI returned an incomplete response',
+        ErrorCode.PROVIDER_INVALID_RESPONSE,
+        `The AI response was marked incomplete (reason: ${response.incomplete_details?.reason ?? 'unknown'}).`,
+        { model: incompleteModel, phase: incompletePhase }
+      );
     }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/core/src/providers/openai/provider.ts` around lines 52 - 114, handle
the silent fallthrough in handleIncompleteResponse by adding a catch-all branch
for any incomplete_details.reason not equal to 'max_output_tokens' or
'content_filter' that throws a descriptive ErodeError; locate the function
handleIncompleteResponse (used when response.status === 'incomplete') and after
the two existing if blocks inspect response.incomplete_details?.reason and throw
an error indicating an unknown/unsupported incomplete reason (include
model/phase/maxOutputTokens and the raw reason in the error context) so callers
never treat truncated responses from extractText as valid.

82-158: 💤 Low value

Hoist helpers out of callModel for readability and reuse.

The five nested function declarations (handleIncompleteResponse, extractText, supportsReasoningEffort, getOpenAIReasoningEffort, getMaxOutputTokens) sit after the try/catch/return that calls them. They work due to hoisting, but reading the method top-down requires jumping past the early return to find behavior. None of them reference this, so they can be promoted to module-private functions next to MAX_OUTPUT_TOKENS_BY_OUTPUT_SIZE. That also makes them straightforward to unit-test in isolation (e.g., reasoning-effort mapping, hint-to-token conversion) without instantiating the provider.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/core/src/providers/openai/provider.ts` around lines 82 - 158, These
helper functions (handleIncompleteResponse, extractText,
supportsReasoningEffort, getOpenAIReasoningEffort, getMaxOutputTokens) are
declared nested after the callModel return and should be hoisted to module scope
for readability and reuse; move each function out of the callModel block and
place them as module-private functions adjacent to
MAX_OUTPUT_TOKENS_BY_OUTPUT_SIZE, keeping their signatures and internal logic
unchanged, update any local references in callModel to use the relocated
functions, and ensure any used types (OpenAI.Responses.Response, AnalysisPhase,
GenerationProfile, OpenAIReasoningEffort, ReasoningEffort) are available in the
file scope (import or reference) so the functions compile and can be unit-tested
independently.
packages/core/src/providers/base-provider.ts (1)

77-92: 💤 Low value

Timing logs are inconsistent across stages and lost on error.

patchModel (line 158) wraps the timing debugLog in .finally(), so duration is reported even when the call throws. selectComponent (line 92) and executeStage (line 182) instead place the debugLog after await, meaning they only log on success — which is precisely the case where timing is least interesting for triaging. Aligning all three to log via .finally() (or a small time(label, fn) helper) would make the observability behavior uniform and surface latency on the failure path too.

♻️ Example for selectComponent
-    debugLog('selectComponent using model', this.fastModel);
-    const startedAt = process.hrtime.bigint();
-    const responseText = await withRetry(
-      () =>
-        this.callModel(
-          this.fastModel,
-          prompt,
-          AnalysisPhase.COMPONENT_RESOLUTION,
-          getGenerationProfileForPhase(AnalysisPhase.COMPONENT_RESOLUTION)
-        ),
-      {
-        retries: 2,
-        shouldRetry: (error) => this.isRetryableError(error),
-      }
-    );
-    debugLog('selectComponent completed in', formatDuration(startedAt));
+    debugLog('selectComponent using model', this.fastModel);
+    const startedAt = process.hrtime.bigint();
+    const responseText = await withRetry(
+      () =>
+        this.callModel(
+          this.fastModel,
+          prompt,
+          AnalysisPhase.COMPONENT_RESOLUTION,
+          getGenerationProfileForPhase(AnalysisPhase.COMPONENT_RESOLUTION)
+        ),
+      {
+        retries: 2,
+        shouldRetry: (error) => this.isRetryableError(error),
+      }
+    ).finally(() => {
+      debugLog('selectComponent completed in', formatDuration(startedAt));
+    });

Also applies to: 174-182

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/core/src/providers/base-provider.ts` around lines 77 - 92,
selectComponent and executeStage only log duration after the await, so failures
don't report timing; patchModel uses .finally() correctly. Modify
selectComponent (the block that calls withRetry/this.callModel and uses
startedAt = process.hrtime.bigint()) and executeStage (the analogous call) to
attach the debugLog('... completed in', formatDuration(startedAt)) in a
.finally() on the promise returned by withRetry (or wrap the call in a small
time(label, fn) helper) so the duration is logged on both success and error—keep
existing symbols: withRetry, this.callModel, AnalysisPhase.*, debugLog,
formatDuration, startedAt.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/core/schemas/eroderc.schema.json`:
- Around line 167-172: The schema currently uses floating Anthropic aliases;
update the defaults to pinned snapshot identifiers to ensure stable behavior:
replace the default value "claude-haiku-4-5" with the pinned snapshot
"claude-haiku-4-5-20251001" (for the fast/haiku model entry that corresponds to
the ERODE_*_FAST_MODEL env var) and replace "claude-sonnet-4-6" in the
"advancedModel" property with the pinned snapshot "claude-sonnet-4-6-20260218";
edit the default strings in the properties whose default values currently show
those alias names (e.g., the property with default "claude-haiku-4-5" and the
"advancedModel" property) so new installs use the fixed snapshot versions.

In `@packages/core/src/analysis/section-formatters.ts`:
- Around line 60-63: The evidence string in the map callback (where evidence is
built from dep.code.trim()) can contain newlines and currently only the first
line is indented, breaking bullet structure; change the construction of evidence
in that map (the anonymous function mapping dep) to normalize multiline evidence
by splitting dep.code.trim() on newlines and rejoining with a consistent
per-line prefix (e.g., two spaces plus the same "Evidence:" header on the first
line or an indented continuation for subsequent lines) so every evidence line
remains indented under the dependency bullet (ensure you update the variable
used to return the final string in that map).

In `@packages/core/src/providers/openai/provider.ts`:
- Around line 129-133: supportsReasoningEffort currently treats any model
starting with prefixes like "gpt-5-" as reasoning-capable, which incorrectly
includes chat-tuned variants (e.g., "gpt-5-chat-latest"); update the function
(supportsReasoningEffort) to also reject models that contain the 'chat'
identifier (for example check model.includes('chat') or a '-chat' segment) so it
only returns true when the model equals a prefix or starts with `${prefix}-` and
does not identify as a chat variant.

---

Outside diff comments:
In `@packages/core/src/providers/anthropic/provider.ts`:
- Around line 69-76: The Anthropic truncation error message is less actionable;
update the ErodeError thrown when response.stop_reason === 'max_tokens' (in the
provider.ts block that constructs ErodeError using model, phase,
outputTokenLimit) to use the same actionable wording as OpenAI/Gemini: explain
that the model used the available output budget before completion and suggest
trying a smaller change or tuning the provider output budget, while keeping the
same ErrorCode.PROVIDER_INVALID_RESPONSE and the existing metadata (model,
phase, outputTokenLimit).

---

Nitpick comments:
In `@packages/core/src/providers/base-provider.ts`:
- Around line 77-92: selectComponent and executeStage only log duration after
the await, so failures don't report timing; patchModel uses .finally()
correctly. Modify selectComponent (the block that calls withRetry/this.callModel
and uses startedAt = process.hrtime.bigint()) and executeStage (the analogous
call) to attach the debugLog('... completed in', formatDuration(startedAt)) in a
.finally() on the promise returned by withRetry (or wrap the call in a small
time(label, fn) helper) so the duration is logged on both success and error—keep
existing symbols: withRetry, this.callModel, AnalysisPhase.*, debugLog,
formatDuration, startedAt.

In `@packages/core/src/providers/openai/provider.ts`:
- Around line 52-114: handle the silent fallthrough in handleIncompleteResponse
by adding a catch-all branch for any incomplete_details.reason not equal to
'max_output_tokens' or 'content_filter' that throws a descriptive ErodeError;
locate the function handleIncompleteResponse (used when response.status ===
'incomplete') and after the two existing if blocks inspect
response.incomplete_details?.reason and throw an error indicating an
unknown/unsupported incomplete reason (include model/phase/maxOutputTokens and
the raw reason in the error context) so callers never treat truncated responses
from extractText as valid.
- Around line 82-158: These helper functions (handleIncompleteResponse,
extractText, supportsReasoningEffort, getOpenAIReasoningEffort,
getMaxOutputTokens) are declared nested after the callModel return and should be
hoisted to module scope for readability and reuse; move each function out of the
callModel block and place them as module-private functions adjacent to
MAX_OUTPUT_TOKENS_BY_OUTPUT_SIZE, keeping their signatures and internal logic
unchanged, update any local references in callModel to use the relocated
functions, and ensure any used types (OpenAI.Responses.Response, AnalysisPhase,
GenerationProfile, OpenAIReasoningEffort, ReasoningEffort) are available in the
file scope (import or reference) so the functions compile and can be unit-tested
independently.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 147a4c9b-dfa1-4843-83c7-9f299e3b8b0a

📥 Commits

Reviewing files that changed from the base of the PR and between 18267db and 8b9cac8.

⛔ Files ignored due to path filters (1)
  • .claude-adr.log is excluded by !**/*.log
📒 Files selected for processing (26)
  • .githooks/post-commit
  • packages/core/schemas/eroderc.schema.json
  • packages/core/src/analysis/__tests__/prompt-builder.test.ts
  • packages/core/src/analysis/__tests__/section-formatters.test.ts
  • packages/core/src/analysis/prompts/drift-analysis.md
  • packages/core/src/analysis/section-formatters.ts
  • packages/core/src/providers/__tests__/generation-profile.test.ts
  • packages/core/src/providers/__tests__/provider-factory.test.ts
  • packages/core/src/providers/anthropic/__tests__/provider.test.ts
  • packages/core/src/providers/anthropic/provider.ts
  • packages/core/src/providers/base-provider.ts
  • packages/core/src/providers/gemini/__tests__/provider.test.ts
  • packages/core/src/providers/gemini/provider.ts
  • packages/core/src/providers/generation-profile.ts
  • packages/core/src/providers/openai/__tests__/provider.test.ts
  • packages/core/src/providers/openai/models.ts
  • packages/core/src/providers/openai/provider.ts
  • packages/eslint-config/base.js
  • packages/eslint-config/index.d.ts
  • packages/eslint-config/index.js
  • packages/web/eslint.config.mjs
  • packages/web/package.json
  • packages/web/public/schemas/v0/eroderc.schema.json
  • packages/web/src/content/docs/docs/contributing/adrs/011-intent-based-provider-generation-profiles.md
  • packages/web/src/content/docs/docs/contributing/adrs/index.md
  • packages/web/src/content/docs/docs/reference/ai-providers.md
✅ Files skipped from review due to trivial changes (4)
  • packages/web/src/content/docs/docs/contributing/adrs/index.md
  • packages/eslint-config/index.js
  • packages/web/src/content/docs/docs/contributing/adrs/011-intent-based-provider-generation-profiles.md
  • packages/web/src/content/docs/docs/reference/ai-providers.md
🚧 Files skipped from review as they are similar to previous changes (3)
  • packages/core/src/providers/openai/models.ts
  • packages/web/public/schemas/v0/eroderc.schema.json
  • packages/core/src/providers/tests/provider-factory.test.ts

Comment thread packages/core/schemas/eroderc.schema.json
Comment thread packages/core/src/analysis/section-formatters.ts
Comment thread packages/core/src/providers/openai/provider.ts
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
packages/core/src/providers/openai/provider.ts (1)

142-170: 💤 Low value

Map 'low' reasoning effort to OpenAI's 'low' instead of silently downgrading to 'minimal'.

The function maps both 'low' and undefined to 'minimal', but OpenAI's Responses API accepts 'low' as a distinct tier between 'minimal' and 'medium'. Multiple analysis phases explicitly request 'low' (DEPENDENCY_SCAN, COMPONENT_RESOLUTION, CHANGE_ANALYSIS), so these requests are being silently downgraded. Either map 'low''low' to preserve caller intent, or document this intentional downgrade with rationale in generation-profile.ts.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/core/src/providers/openai/provider.ts` around lines 142 - 170, The
getOpenAIReasoningEffort function currently maps both 'low' and undefined to
'minimal', which silently downgrades callers requesting 'low' (e.g.,
DEPENDENCY_SCAN, COMPONENT_RESOLUTION, CHANGE_ANALYSIS); update the switch in
getOpenAIReasoningEffort (and its ReasoningEffort -> OpenAIReasoningEffort
mapping) so case 'low' returns 'low' (preserve undefined -> 'minimal' if
desired), ensuring the function returns OpenAI's distinct 'low' tier rather than
downgrading to 'minimal'.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/core/src/providers/anthropic/provider.ts`:
- Around line 86-93: Extract the duplicated character-to-token logic into a
shared exported helper (e.g., resolveOutputTokenLimit(profile, sizeTable)) in
packages/core/src/providers/generation-profile.ts, implement it to compute
profileLimit from the provided sizeTable and hintedLimit as
Math.ceil(profile.outputContentHint.characters / 4) and return
Math.max(profileLimit, hintedLimit); then replace the body of
getOutputTokenLimit in packages/core/src/providers/anthropic/provider.ts and
getMaxOutputTokens in packages/core/src/providers/openai/provider.ts to call
resolveOutputTokenLimit(profile, MAX_TOKENS_BY_OUTPUT_SIZE) (or the OpenAI size
table) and move these calls to module scope so the helper isn’t re-declared per
call.

In `@packages/core/src/providers/gemini/provider.ts`:
- Around line 134-146: The switch handling reasoningIntent currently forces any
"pro" model detected by isGemini3ProModel(thinkingModel) to ThinkingLevel.HIGH
for the "medium" case; update the logic so Gemini 3.1+ Pro models can remain
MEDIUM while only legacy gemini-3-pro* IDs escalate to HIGH. Modify either
isGemini3ProModel to distinguish legacy IDs (e.g., match exact "gemini-3-pro" or
"gemini-3-pro-*" but not "gemini-3.1-pro*"/"gemini-3.x-pro*"), or add a new
helper (e.g., isLegacyGemini3ProModel) and use that in the medium branch inside
the switch to decide HIGH vs MEDIUM while preserving ThinkingLevel.MEDIUM for
3.1+ Pro models.

---

Nitpick comments:
In `@packages/core/src/providers/openai/provider.ts`:
- Around line 142-170: The getOpenAIReasoningEffort function currently maps both
'low' and undefined to 'minimal', which silently downgrades callers requesting
'low' (e.g., DEPENDENCY_SCAN, COMPONENT_RESOLUTION, CHANGE_ANALYSIS); update the
switch in getOpenAIReasoningEffort (and its ReasoningEffort ->
OpenAIReasoningEffort mapping) so case 'low' returns 'low' (preserve undefined
-> 'minimal' if desired), ensuring the function returns OpenAI's distinct 'low'
tier rather than downgrading to 'minimal'.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: fa0956b5-1ff8-4c0c-b80c-8000222e0273

📥 Commits

Reviewing files that changed from the base of the PR and between 8b9cac8 and 2927b1b.

📒 Files selected for processing (12)
  • .env.example
  • packages/core/src/analysis/__tests__/prompt-builder.test.ts
  • packages/core/src/analysis/__tests__/section-formatters.test.ts
  • packages/core/src/analysis/section-formatters.ts
  • packages/core/src/providers/__tests__/generation-profile.test.ts
  • packages/core/src/providers/anthropic/__tests__/provider.test.ts
  • packages/core/src/providers/anthropic/provider.ts
  • packages/core/src/providers/gemini/__tests__/provider.test.ts
  • packages/core/src/providers/gemini/provider.ts
  • packages/core/src/providers/openai/__tests__/provider.test.ts
  • packages/core/src/providers/openai/provider.ts
  • packages/web/src/content/docs/docs/contributing/adrs/011-intent-based-provider-generation-profiles.md
✅ Files skipped from review due to trivial changes (1)
  • packages/web/src/content/docs/docs/contributing/adrs/011-intent-based-provider-generation-profiles.md
🚧 Files skipped from review as they are similar to previous changes (1)
  • packages/core/src/analysis/section-formatters.ts

Comment thread packages/core/src/providers/anthropic/provider.ts Outdated
Comment thread packages/core/src/providers/gemini/provider.ts
Comment thread packages/core/src/providers/anthropic/provider.ts
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant