Skip to content

fix: raise max_tokens for AI analysis to avoid thinking-model truncation #241

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
AmintaCCCP merged 2 commits into from
Jul 11, 2026
+5 −3
Merged

fix: raise max_tokens for AI analysis to avoid thinking-model truncation #241

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
c26dcd7
fix: raise max_tokens for AI analysis/reranking to avoid thinking-mod…
AmintaCCCP Jul 11, 2026
8006507
refactor: split reranking token limit into RERANKING_MAX_TOKENS
AmintaCCCP Jul 11, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions src/services/aiService.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,8 @@ export class AIService {
private config: AIConfig;
private language: string;
private static readonly ANALYSIS_MAX_ATTEMPTS = 3;
private static readonly ANALYSIS_MAX_TOKENS = 4096;

@gemini-code-assist gemini-code-assist Bot Jul 11, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Sharing a single constant ANALYSIS_MAX_TOKENS for both repository analysis and search reranking reduces maintainability and flexibility. Repository analysis (which generates a detailed summary, tags, and platforms) and search reranking (which only outputs a list of IDs) have very different output sizes and latency requirements.

If you want to adjust the token limit for repository analysis in the future (e.g., increasing it to 8192), it will unintentionally increase the limit for search reranking as well, potentially leading to higher latency and costs for search queries.

Consider defining separate constants for these tasks so they can be tuned independently.

Suggested change
private static readonly ANALYSIS_MAX_TOKENS = 4096;
private static readonly ANALYSIS_MAX_TOKENS = 4096;
private static readonly RERANKING_MAX_TOKENS = 4096;

private static readonly RERANKING_MAX_TOKENS = 4096;

constructor(config: AIConfig, language: string = 'zh') {
this.config = config;
Expand Down Expand Up @@ -514,7 +516,7 @@ ${options.user}` : options.user;
? prompt
: this.createAnalysisRetryPrompt(prompt, lastContent, lastInvalidReason),
temperature: attempt === 1 ? 0.3 : 0.1,
maxTokens: 1000,
maxTokens: AIService.ANALYSIS_MAX_TOKENS,
signal,
});

Expand Down Expand Up @@ -620,7 +622,7 @@ AI Summary: ${gist.ai_summary || 'None'}`;
system,
user: `Query: ${query}\n\nGists:\n${this.sanitizeForPrompt(gistSummaries)}`,
temperature: 0.1,
maxTokens: 1200,
maxTokens: AIService.RERANKING_MAX_TOKENS,
});

try {
Expand Down Expand Up @@ -682,7 +684,7 @@ AI Summary: ${gist.ai_summary || 'None'}`;
system,
user: `Query: ${query}\n\nRepositories:\n${this.sanitizeForPrompt(repoSummaries)}`,
temperature: 0.1,
maxTokens: 800,
maxTokens: AIService.RERANKING_MAX_TOKENS,
signal,
});

Expand Down
Loading