Skip to content

feat(web): customisable AI code review agent configs#1143

Open
fatmcgav wants to merge 6 commits intosourcebot-dev:mainfrom
fatmcgav:feat-customisable-agents
Open

feat(web): customisable AI code review agent configs#1143
fatmcgav wants to merge 6 commits intosourcebot-dev:mainfrom
fatmcgav:feat-customisable-agents

Conversation

@fatmcgav
Copy link
Copy Markdown
Contributor

@fatmcgav fatmcgav commented Apr 21, 2026

Adds a per-org/connection/repo agent config system for the code review agent.

Database:

  • New AgentConfig model with AgentType, AgentScope, and PromptMode enums
  • Junction tables AgentConfigToRepo and AgentConfigToConnection for scope targeting
  • Migration: 20260421203635_add_agent_config

Review agent:

  • Priority resolver: REPO > CONNECTION > ORG > system defaults
  • resolveRules(): APPEND or REPLACE custom prompt against built-in rules
  • parseAgentConfigSettings(): validates per-config JSON settings
  • webhookUtils: isAutoReviewEnabled and getReviewCommand respect per-config overrides
  • invokeDiffReviewLlm: accepts per-config model override
  • fetchContextFile: fetches repo-level instruction files (e.g. AGENTS.md) once per PR and injects as context for every diff hunk; supports comma/space-separated list of paths; missing files silently ignored
  • Webhook route wires resolved config through to all processing functions

API:

  • GET/POST /api/agents — list and create agent configs
  • GET/PATCH/DELETE /api/agents/[agentId] — manage individual configs
  • Full OpenAPI registration with request/response schemas

UI:

  • Agents list page shows all configs with scope/type/enabled badges
  • Create and edit forms with scope picker, repo/connection checkbox lists with filter inputs, custom prompt with APPEND/REPLACE mode, and settings overrides (model dropdown, review command, auto-review, context files)

Tests:

  • 60 unit tests across resolveAgentConfig, app, webhookUtils, and the agents API route

Docs:

  • Updated review-agent.mdx with agent configs, custom prompt, and context files sections
  • Env var table updated to note per-config override support

Builds on changes from #1104

Summary by CodeRabbit

  • New Features

    • Added full agent configuration CRUD with UI for listing, creating, editing, and deleting configs.
    • Per-scope overrides (Org, Connection, Repo) with scope-priority resolution.
    • Custom prompts with Replace/Append modes, model and auto-review overrides, review-trigger command overrides, and support for repo-level context files.
    • Auto-review can be enabled per-config and used by webhook flows.
  • Documentation

    • Documented agent config behavior, scope priorities, prompt modes, context files, and overrideable environment settings.

Adds a per-org/connection/repo agent config system for the code review agent.

Database:
- New `AgentConfig` model with `AgentType`, `AgentScope`, and `PromptMode` enums
- Junction tables `AgentConfigToRepo` and `AgentConfigToConnection` for scope targeting
- Migration: `20260421203635_add_agent_config`

Review agent:
- Priority resolver: REPO > CONNECTION > ORG > system defaults
- `resolveRules()`: APPEND or REPLACE custom prompt against built-in rules
- `parseAgentConfigSettings()`: validates per-config JSON settings
- `webhookUtils`: `isAutoReviewEnabled` and `getReviewCommand` respect per-config overrides
- `invokeDiffReviewLlm`: accepts per-config model override
- `fetchContextFile`: fetches repo-level instruction files (e.g. AGENTS.md) once per PR and injects as context for every diff hunk; supports comma/space-separated list of paths; missing files silently ignored
- Webhook route wires resolved config through to all processing functions

API:
- `GET/POST /api/agents` — list and create agent configs
- `GET/PATCH/DELETE /api/agents/[agentId]` — manage individual configs
- Full OpenAPI registration with request/response schemas

UI:
- Agents list page shows all configs with scope/type/enabled badges
- Create and edit forms with scope picker, repo/connection checkbox lists with filter inputs, custom prompt with APPEND/REPLACE mode, and settings overrides (model dropdown, review command, auto-review, context files)

Tests:
- 60 unit tests across resolveAgentConfig, app, webhookUtils, and the agents API route

Docs:
- Updated review-agent.mdx with agent configs, custom prompt, and context files sections
- Env var table updated to note per-config override support

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Apr 21, 2026

Walkthrough

Adds a complete Agent Config system: database schema, CRUD REST API, web UI, OpenAPI schemas, webhook integration, review-agent runtime support (prompts, model, context files), and tests to enable per-ORG/CONNECTION/REPO overrides for code-review agents.

Changes

Cohort / File(s) Summary
Database Schema & Migration
packages/db/prisma/schema.prisma, packages/db/prisma/migrations/..._add_agent_config/migration.sql
Adds enums (AgentType, AgentScope, PromptMode), AgentConfig model, join tables AgentConfigToRepo / AgentConfigToConnection, constraints/indexes, and relations to Org/Repo/Connection.
API Routes (list/create)
packages/web/src/app/api/(server)/agents/route.ts, docs/api-reference/sourcebot-public.openapi.json
New GET /api/agents and POST /api/agents with Zod validation, org-role gating, name and scope validation, ownership checks, scope-conflict detection, and OpenAPI registration.
API Routes (get/update/delete)
packages/web/src/app/api/(server)/agents/[agentId]/route.ts, docs/api-reference/sourcebot-public.openapi.json
New GET, PATCH, DELETE handlers with validation, uniqueness and scope-conflict enforcement, junction table rebuild on updates, and standardized error responses.
API Tests
packages/web/src/app/api/(server)/agents/route.test.ts, packages/web/src/app/api/(server)/agents/[agentId]/route.test.ts
Comprehensive Vitest coverage for list/create/update/delete flows, schema validation, name collisions, scope-specific ID requirements, and conflict branches.
Web UI Pages
packages/web/src/app/(app)/agents/page.tsx, packages/web/src/app/(app)/agents/configs/new/page.tsx, packages/web/src/app/(app)/agents/configs/[agentId]/page.tsx
Authenticated pages (OWNER) for agents index, new config, and edit config; server queries for org repos/connections and agent config loading with 404 handling.
Agent Config Form Component
packages/web/src/app/(app)/agents/configs/agentConfigForm.tsx
Client component managing form state, model fetching, scope selection (repo/connection), prompt modes, settings overrides (model, review command, auto-review, context files), save (POST/PATCH) and delete flows with toasts/navigation.
OpenAPI Schemas & Docs
packages/web/src/openapi/publicApiSchemas.ts, packages/web/src/openapi/publicApiDocument.ts, docs/docs/features/agents/review-agent.mdx
New public Zod/OpenAPI schemas for agent configs and CRUD bodies/responses; OpenAPI document updated with /api/agents endpoints and documentation content updated for agent-config behavior.
Review-Agent Resolution & Helpers
packages/web/src/features/agents/review-agent/resolveAgentConfig.ts, .../resolveAgentConfig.test.ts, .../webhookUtils.ts, .../webhookUtils.test.ts
Priority resolver REPO → CONNECTION → ORG for enabled configs; helpers isAutoReviewEnabled and getReviewCommand that prefer per-config settings over env; tests for precedence and behavior.
Review-Agent Runtime (rules/settings)
packages/web/src/features/agents/review-agent/app.ts, .../app.test.ts
Adds DEFAULT_RULES, resolveRules, agentConfigSettingsSchema, parseAgentConfigSettings; processGitHubPullRequest and processGitLabMergeRequest accept optional AgentConfig and apply rules/settings.
Context File Support
packages/web/src/features/agents/review-agent/nodes/fetchFileContent.ts, .../fetchFileContent.test.ts
Adds fetchContextFile to fetch repo-level context files by head commit returning repo_instructions contexts or null on errors; tests for fetch and schema handling.
Prompt Generation & LLM Integration
packages/web/src/features/agents/review-agent/nodes/generatePrReview.ts, .../generatePrReview.test.ts, .../invokeDiffReviewLlm.ts
generatePrReviews accepts optional modelOverride and contextFiles, fetches context files once per PR and includes them in prompts; invokeDiffReviewLlm accepts modelOverride with precedence logic.
Types & Context Schema
packages/web/src/features/agents/review-agent/types.ts
Extends sourcebot_context_schema to include repo_instructions.
Error Codes
packages/web/src/lib/errorCodes.ts
Adds AGENT_CONFIG_NOT_FOUND, AGENT_CONFIG_ALREADY_EXISTS, AGENT_CONFIG_SCOPE_CONFLICT.
Changelog
CHANGELOG.md
Adds Unreleased entry documenting the new experimental agent-config feature.

Sequence Diagram(s)

sequenceDiagram
    participant GitHub as GitHub Webhook
    participant Webhook as Webhook Handler
    participant PrismaDB as Prisma/Database
    participant ReviewAgent as Review Agent
    participant LLM as LLM Service

    GitHub->>Webhook: POST /api/webhook (PR event)
    Webhook->>PrismaDB: Query Repo by external ID + host URL
    PrismaDB-->>Webhook: Return Repo
    Webhook->>PrismaDB: resolveAgentConfig(repoId, orgId, 'CODE_REVIEW')
    
    alt REPO-scoped config exists
        PrismaDB-->>Webhook: AgentConfig (REPO scope)
    else CONNECTION-scoped config exists
        PrismaDB->>PrismaDB: Query via Connection relation
        PrismaDB-->>Webhook: AgentConfig (CONNECTION scope)
    else ORG-scoped config exists
        PrismaDB-->>Webhook: AgentConfig (ORG scope)
    else No config found
        PrismaDB-->>Webhook: null
    end

    Webhook->>ReviewAgent: processGitHubPullRequest(..., config)
    ReviewAgent->>ReviewAgent: resolveRules(config)
    
    alt config with REPLACE prompt mode
        ReviewAgent->>ReviewAgent: Use only custom prompt
    else config with APPEND mode or no config
        ReviewAgent->>ReviewAgent: Append custom prompt to defaults
    end
    
    ReviewAgent->>ReviewAgent: parseAgentConfigSettings(config.settings)
    ReviewAgent->>ReviewAgent: Fetch context files (if specified)
    ReviewAgent->>LLM: invokeDiffReviewLlm(..., rules, modelOverride)
    LLM-->>ReviewAgent: Review comments
    ReviewAgent-->>Webhook: Complete
Loading
sequenceDiagram
    participant User as User/UI
    participant Page as Agent Config Page
    participant Form as AgentConfigForm
    participant API as REST API
    participant PrismaDB as Prisma/Database

    User->>Page: Navigate to /agents/configs/new
    Page->>PrismaDB: Fetch org repos & connections
    PrismaDB-->>Page: repos[], connections[]
    Page->>Form: Render with initial repos/connections
    
    User->>Form: Fill form (name, scope, repoIds, settings, etc.)
    User->>Form: Click Save
    
    Form->>Form: Build request payload
    Form->>API: POST /api/agents (for new) or PATCH /api/agents/{id} (for edit)
    
    alt Create (POST)
        API->>PrismaDB: Check name uniqueness (orgId, name)
        alt Name collision exists
            PrismaDB-->>API: Exists
            API-->>Form: 409 Conflict
        else Name available
            PrismaDB-->>API: Not found
            API->>PrismaDB: Create AgentConfig
            API->>PrismaDB: Create AgentConfigToRepo/Connection join records
            PrismaDB-->>API: Success
            API-->>Form: 201 Created + config
        end
    else Update (PATCH)
        API->>PrismaDB: Check scope-specific ID requirements
        API->>PrismaDB: Verify org ownership of IDs
        API->>PrismaDB: Update config fields
        API->>PrismaDB: Rebuild junction tables (delete old, create new)
        PrismaDB-->>API: Updated config
        API-->>Form: 200 OK + config
    end
    
    Form->>Form: Toast success message
    Form->>Page: Navigate to /agents
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~65 minutes

Possibly related PRs

Suggested labels

sourcebot-team

Suggested reviewers

  • brendan-kellam
  • msukkari
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 36.36% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'feat(web): customisable AI code review agent configs' accurately and concisely summarizes the main change—introducing configurable agent settings at multiple scopes for the review system.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@fatmcgav
Copy link
Copy Markdown
Contributor Author

Couple of screenshots showing the new UI functionality:
image
image

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 20

🧹 Nitpick comments (6)
packages/db/prisma/migrations/20260421203635_add_agent_config/migration.sql (1)

29-42: Missing reverse-FK indexes on join tables.

Same as noted on schema.prisma: AgentConfigToRepo(repoId) and AgentConfigToConnection(connectionId) are only indexed as the 2nd column of the composite PK, so cascade deletes from Repo/Connection (and resolveAgentConfig lookups by repo/connection) will seq-scan. Add indexes via the Prisma model, which will regenerate the migration.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/db/prisma/migrations/20260421203635_add_agent_config/migration.sql`
around lines 29 - 42, The join tables AgentConfigToRepo and
AgentConfigToConnection lack single-column indexes on repoId and connectionId,
causing seq-scans for deletes/lookups; update the Prisma models for
AgentConfigToRepo and AgentConfigToConnection to add explicit @@index([...]) or
`@unique/`@index on the repoId and connectionId fields (respectively) so Prisma
regenerates a migration that creates indexes on repoId and connectionId in the
SQL, then run prisma migrate to emit the updated migration SQL.
packages/db/prisma/schema.prisma (1)

641-660: Consider adding indexes on repoId / connectionId for join tables.

AgentConfigToRepo and AgentConfigToConnection only have composite PKs (agentConfigId, repoId/connectionId), which PostgreSQL indexes on the leading column only. Cascade deletes triggered from Repo / Connection and reverse lookups (e.g., from resolveAgentConfig given a repoId) will have to seq-scan the join table. Adding @@index([repoId]) and @@index([connectionId]) respectively mirrors what RepoToConnection does (@@index([repoId, connectionId])) and keeps cascade deletes fast.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/db/prisma/schema.prisma` around lines 641 - 660, The join models
AgentConfigToRepo and AgentConfigToConnection currently only have composite
primary keys (agentConfigId, repoId) and (agentConfigId, connectionId) which
index the leading column only; add explicit secondary indexes by adding
@@index([repoId]) to the AgentConfigToRepo model and @@index([connectionId]) to
the AgentConfigToConnection model so lookups and cascade deletes from
Repo/Connection are efficient (mirror the existing pattern used by
RepoToConnection with @@index([repoId, connectionId])).
packages/web/src/features/agents/review-agent/nodes/fetchFileContent.ts (1)

10-23: Module-level cachedOrg is shared across PRs and never invalidated.

Not new to this PR, but now exercised on every diff hunk via fetchContextFile. If the Org row changes (e.g., rename, metadata updates), the process keeps the stale copy until restart. Consider either fetching per call (the lookup is cheap by PK) or adding a short TTL. Happy to defer if intentional.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/web/src/features/agents/review-agent/nodes/fetchFileContent.ts`
around lines 10 - 23, The module-level cachedOrg used by getOrg (backed by
__unsafePrisma.org.findUnique and SINGLE_TENANT_ORG_ID) is never invalidated and
can serve stale org data; update getOrg to avoid a long-lived module cache:
either remove cachedOrg and always call __unsafePrisma.org.findUnique by PK
inside getOrg, or implement a short TTL-based cache (store timestamp alongside
cachedOrg and refresh when expired) so renames/metadata updates are observed
without requiring a process restart. Ensure error handling (throwing
"Organization not found") remains when lookup returns null.
docs/docs/features/agents/review-agent.mdx (2)

169-171: Use second person for override descriptions.

The repeated “Can be overridden...” fragments are passive. Use direct second-person wording.

Proposed wording cleanup
-| `REVIEW_AGENT_AUTO_REVIEW_ENABLED` | `false` | Automatically review new and updated PRs/MRs. Can be overridden per agent config. |
-| `REVIEW_AGENT_REVIEW_COMMAND` | `review` | Comment command that triggers a manual review (without the `/`). Can be overridden per agent config. |
-| `REVIEW_AGENT_MODEL` | first configured model | `displayName` of the language model to use for reviews. Can be overridden per agent config. |
+| `REVIEW_AGENT_AUTO_REVIEW_ENABLED` | `false` | Automatically review new and updated PRs/MRs. You can override this per agent config. |
+| `REVIEW_AGENT_REVIEW_COMMAND` | `review` | Comment command that triggers a manual review, without the `/`. You can override this per agent config. |
+| `REVIEW_AGENT_MODEL` | first configured model | `displayName` of the language model to use for reviews. You can override this per agent config. |

As per coding guidelines, docs/**/*.mdx files should use second person present tense.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@docs/docs/features/agents/review-agent.mdx` around lines 169 - 171, The docs
use passive phrasing for override notes; update the three rows for
REVIEW_AGENT_AUTO_REVIEW_ENABLED, REVIEW_AGENT_REVIEW_COMMAND, and
REVIEW_AGENT_MODEL so the override note uses second-person present tense (e.g.,
change "Can be overridden per agent config." to "You can override this in the
agent config." or similar) to match the docs guideline for docs/**/*.mdx.

142-163: Replace em dashes and hyphenate the separator phrase.

The new section uses em dashes and “comma or space separated”. Please use commas/colons and comma- or space-separated instead.

Proposed wording cleanup
-Agent configs let you customise how the review agent behaves per repository, connection, or your whole org. You can override the model, review command, auto-review behaviour, custom prompt, and context files — all without changing environment variables.
+Agent configs let you customise how the review agent behaves per repository, connection, or your whole org. You can override the model, review command, auto-review behaviour, custom prompt, and context files without changing environment variables.
...
-- **Repo** — applies to specific repositories (highest priority)
-- **Connection** — applies to all repos in a specific connection
-- **Org** — applies to all repositories in your org (lowest priority, catch-all)
+- **Repo**: Applies to specific repositories. This has the highest priority.
+- **Connection**: Applies to all repos in a specific connection.
+- **Org**: Applies to all repositories in your org. This has the lowest priority.
...
-- **Append** (default) — your instructions are added after the built-in review rules.
-- **Replace** — your instructions entirely replace the built-in rules. Use this when you want full control over what the agent looks for.
+- **Append** (default): Your instructions are added after the built-in review rules.
+- **Replace**: Your instructions replace the built-in rules. Use this when you want full control over what the agent looks for.
...
-You can configure one or more repository files to be fetched at review time and injected as additional context for the model. This is useful for encoding project-specific conventions that the model should be aware of when reviewing diffs — for example, preferred error handling patterns, style rules, or areas of the codebase that need extra scrutiny.
+You can configure one or more repository files to be fetched at review time and injected as additional context for the model. This is useful for project-specific conventions, such as preferred error handling patterns, style rules, or areas of the codebase that need extra scrutiny.
...
-Set **Context files** in the agent config form to a comma or space separated list of paths relative to the repository root (e.g. `AGENTS.md .sourcebot/review.md`). Files that do not exist in the repository are silently ignored. The files are fetched once per PR from the head commit and included in the context for every diff hunk.
+Set **Context files** in the agent config form to a comma- or space-separated list of paths relative to the repository root (e.g. `AGENTS.md .sourcebot/review.md`). Files that do not exist in the repository are silently ignored. The files are fetched once per PR from the head commit and included in the context for every diff hunk.

As per coding guidelines, docs/**/*.mdx files must not use em dashes and should keep sentences short.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@docs/docs/features/agents/review-agent.mdx` around lines 142 - 163, In the
"Agent configs" intro and the "Context files" paragraph (look for the
headings/phrases "Agent configs let you customise...", "custom prompt", and the
sentence beginning "Set **Context files**..."), replace em dashes with commas or
a colon and change "comma or space separated" to the hyphenated phrase "comma-
or space-separated"; ensure the sentence that currently reads "...context files
— all without changing environment variables." becomes something like
"...context files, all without changing environment variables." and the "Set
**Context files**..." line reads with "comma- or space-separated list" instead
of the em dash and unhyphenated phrase.
packages/web/src/features/agents/review-agent/app.test.ts (1)

51-63: Cover contextFiles as a recognized setting.

The parser tests should include the new context-file override so schema regressions are caught alongside model, reviewCommand, and autoReviewEnabled.

🧪 Proposed test update
         const result = parseAgentConfigSettings({
             autoReviewEnabled: false,
             reviewCommand: 'check',
             model: 'claude-sonnet-4-6',
+            contextFiles: 'AGENTS.md .sourcebot/review.md',
         });
 
         expect(result).toEqual({
             autoReviewEnabled: false,
             reviewCommand: 'check',
             model: 'claude-sonnet-4-6',
+            contextFiles: 'AGENTS.md .sourcebot/review.md',
         });
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/web/src/features/agents/review-agent/app.test.ts` around lines 51 -
63, The test for parseAgentConfigSettings is missing the new contextFiles
setting; update the test 'parses all recognised fields' to include a sample
contextFiles value in the input and expected output so the parser is validated
for the context-file override (reference parseAgentConfigSettings and the
contextFiles property).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@packages/web/src/app/`(app)/agents/configs/[agentId]/page.tsx:
- Line 10: The page currently allows any authenticated user to edit agent
configs; update the authenticatedPage call to require org owners by passing the
options object { minRole: OrgRole.OWNER, redirectTo: '/settings' } to
authenticatedPage (i.e., change authenticatedPage(async (...) => ...) to
authenticatedPage({ minRole: OrgRole.OWNER, redirectTo: '/settings' }, async
(...) => ...)); also ensure OrgRole is imported where used (add import for
OrgRole if missing) so non-owners are redirected to /settings.

In `@packages/web/src/app/`(app)/agents/configs/agentConfigForm.tsx:
- Around line 89-94: The form is submitting settings.contextFiles but the
validation schema agentConfigSettingsSchema doesn't include contextFiles, so
those values are dropped; update the agentConfigSettingsSchema to add a
contextFiles property with the correct type/shape (e.g., array of file
identifiers or objects matching what the form sends) and default/optional rules,
and ensure any POST/PATCH validators or types that consume
agentConfigSettingsSchema (and any related request DTOs or parse/validate calls)
are updated to accept and persist settings.contextFiles (look for usages of
agentConfigSettingsSchema and the settings object in the agent save/update
handlers).

In `@packages/web/src/app/`(app)/agents/configs/new/page.tsx:
- Line 5: Update UI and server to require OrgRole.OWNER for agent config
creation/editing: change the page export that currently calls
authenticatedPage(...) to pass the options { minRole: OrgRole.OWNER, redirectTo:
'/settings' } (look for authenticatedPage in the new page and the edit page
under the agents config routes). In the API handler that uses withAuth (the POST
handler wrapped by apiHandler), replace or wrap the authorization check with
withMinimumOrgRole(OrgRole.OWNER, async ({ org, prisma }) => { ...existing POST
logic... }) so only owners can create configs; ensure you keep existing request
handling inside that callback. Also apply the same authenticatedPage gating to
the edit page (agents/configs/[agentId]/page) so UI and API enforce the same
minimum role.

In `@packages/web/src/app/`(app)/agents/page.tsx:
- Line 37: The page is exposing agent configs to all authenticated org members;
update the authenticatedPage call to require owner role by passing options like
{ minRole: OrgRole.OWNER, redirectTo: '/settings' } so only org owners can
access it; modify the existing authenticatedPage(...) invocation (the exported
default function) to include these options and ensure OrgRole is imported where
used.

In `@packages/web/src/app/api/`(server)/agents/[agentId]/route.ts:
- Around line 117-140: Detect when scope changes by first fetching the existing
agent config (e.g., via prisma.agentConfig.findUnique) and, inside the update
for prisma.agentConfig.update, ensure stale junction rows are cleared: always
include repos.deleteMany when new scope is CONNECTION or ORG (or when switching
away from REPO) unless repoIds are explicitly provided, and always include
connections.deleteMany when new scope is REPO or ORG (or when switching away
from CONNECTION) unless connectionIds are explicitly provided; reference the
existing scope, the new scope variable (scope), and the update block that
currently uses repos/create and connections/create to conditionally add
deleteMany entries so stale relations are removed even if the caller did not
supply repoIds/connectionIds.
- Around line 117-140: Before calling prisma.agentConfig.update, validate that
every id in repoIds and connectionIds belongs to the current org by querying the
corresponding tables and comparing counts/returned ids; if any provided id is
not owned by org.id return a 400 error. Specifically, when handling repoIds and
connectionIds in the update block (the code that constructs the nested repos and
connections create/deleteMany objects passed to prisma.agentConfig.update),
first run lookups like fetching repos with where: { id: { in: repoIds }, orgId }
and connections with where: { id: { in: connectionIds }, orgId }, ensure the
result length equals the supplied array length (or compute the set difference to
identify invalid ids) and abort with an appropriate error if mismatches exist;
only then include the repos: { deleteMany: {}, create: [...] } and connections:
{ deleteMany: {}, create: [...] } sections.
- Line 88: The PATCH and DELETE handlers currently call withAuth (e.g., the call
to withAuth at the start of the mutation handlers) which only rejects
unauthenticated/guest users; replace those calls with withMinimumOrgRole to
enforce owner/admin permissions. Specifically, swap uses of withAuth with
withMinimumOrgRole (requiring "owner" or "admin" per policy) in the agent
mutation paths (the invocation around withAuth in the mutation at the position
referenced and the similar call around line 162) so that the functions handling
PATCH and DELETE require the minimum org role instead of just authentication.
- Around line 89-116: Before performing the update, check whether the incoming
name (e.g., the request's name field) is being changed to a value that already
exists for another agent config in the same org and return a 409 Conflict
instead of letting the DB unique-constraint error surface; specifically, in the
PATCH handler in route.ts (after loading existing via
prisma.agentConfig.findFirst and before the try-update block), query
prisma.agentConfig.findFirst where name equals the requested name and orgId
equals org.id and id is not agentId, and if found return a 409 response
(StatusCodes.CONFLICT) with an appropriate errorCode and message indicating the
name collision.
- Around line 15-19: The PATCH validation schema agentConfigSettingsSchema
currently only allows autoReviewEnabled, reviewCommand, and model, so
settings.contextFiles from the form is dropped; update agentConfigSettingsSchema
to include a contextFiles field (e.g., contextFiles:
z.array(z.string()).optional()) so incoming settings.contextFiles are validated
and preserved by the PATCH handler that consumes agentConfigSettingsSchema.

In `@packages/web/src/app/api/`(server)/agents/route.test.ts:
- Around line 91-93: The "name collision" describe block fails because prior
tests leave Prisma mocks called; inside the describe that contains the
name-collision tests add a local beforeEach(() => vi.clearAllMocks()) so
agentConfig.create (and other mocks) have cleared call history before each test;
target the describe block that asserts agentConfig.create is not called and add
this beforeEach to ensure the assertion is reliable.

In `@packages/web/src/app/api/`(server)/agents/route.ts:
- Line 78: The route currently uses withAuth to allow any authenticated member
to create org-level review-agent configs; change the authorization wrapper to
withMinimumOrgRole and require the appropriate elevated role (e.g., "owner" or
"admin") so only org owners/admins can perform this mutation. Locate the call to
withAuth in the agents route handler (the async callback that returns result)
and replace it with withMinimumOrgRole, passing the required role constant/enum
and the same handler; ensure the handler still receives the org and prisma
parameters and that error handling remains unchanged.
- Around line 92-126: Before calling prisma.agentConfig.create, verify that all
repoIds (when scope === AgentScope.REPO) and connectionIds (when scope ===
AgentScope.CONNECTION) belong to the current org (org.id): query the repos table
for repoIds with where: { id: { in: repoIds }, orgId: org.id } and the
connections table similarly, compare the returned count to
repoIds.length/connectionIds.length and return StatusCodes.BAD_REQUEST +
ErrorCode.INVALID_REQUEST_BODY if any IDs are missing/invalid; only then pass
the nested repos: { create: ... } or connections: { create: ... } to
prisma.agentConfig.create so you avoid cross-org associations or raw Prisma
errors.
- Around line 16-20: The agentConfigSettingsSchema currently lacks the
contextFiles field so submitted contextFiles from the UI are dropped; update
agentConfigSettingsSchema to include a contextFiles property (optional) that
matches the shape used by the UI/review agent (e.g., an optional array of
strings or objects as expected by the review flow) so validation preserves and
returns contextFiles to the review agent; modify the schema defined in
agentConfigSettingsSchema to add this contextFiles entry to align with the
review agent's expected input.

In `@packages/web/src/app/api/`(server)/webhook/route.ts:
- Around line 147-160: The code currently resolves agent configs using
SINGLE_TENANT_ORG_ID which forces lookups into the single-tenant org; update the
logic so that after finding the repo record(s) you call resolveAgentConfig with
the matched repo.orgId instead of SINGLE_TENANT_ORG_ID (use the repo variable
returned by __unsafePrisma.repo.findFirst or change to findMany and iterate if
you need to handle multiple matching repo records), and apply the same change to
resolveGitLabAgentConfig so configs are resolved against the actual orgId(s) of
the repo(s).

In `@packages/web/src/features/agents/review-agent/nodes/fetchFileContent.ts`:
- Around line 56-93: The docstring promises content is capped at
CONTEXT_FILE_MAX_BYTES but fetchContextFile currently returns the full
parsed.data.source; update fetchContextFile (the function named fetchContextFile
in this diff) to enforce the cap: measure the UTF-8 byte length of
parsed.data.source (use Buffer.byteLength), and if it exceeds
CONTEXT_FILE_MAX_BYTES, truncate the string to fit that many bytes without
breaking multi-byte UTF-8 characters (iterate/truncate safely or use a helper to
slice by bytes), replace content with the truncated version, and add a debug/log
message indicating truncation occurred; ensure the returned context uses the
possibly-truncated content.

In `@packages/web/src/features/agents/review-agent/nodes/generatePrReview.ts`:
- Around line 16-18: The Promise.all call that builds repoInstructionsContexts
lets a single fetchContextFile throw aborting the whole operation; change to
handle failures per-file by mapping contextFilePaths to per-path try/catch (or
use Promise.allSettled) and call fetchContextFile(pr_payload, p) inside each
handler, skipping or returning null on error, then filter with the existing type
guard to produce repoInstructionsContexts; keep symbols
repoInstructionsContexts, fetchContextFile, contextFilePaths and pr_payload to
locate and update the logic.

In `@packages/web/src/features/agents/review-agent/resolveAgentConfig.ts`:
- Around line 21-70: The findFirst queries (prisma.agentConfig.findFirst) that
populate repoConfig, connectionConfig, and orgConfig are non-deterministic when
multiple enabled configs match; update each query to include an orderBy: {
updatedAt: 'desc' } so the most recently updated config is consistently selected
(or alternatively enforce uniqueness on the combination of
orgId,type,enabled,scope if you want to prevent duplicates); locate the three
queries that assign repoConfig, connectionConfig, and orgConfig and add the
orderBy clause to each.

In `@packages/web/src/features/agents/review-agent/webhookUtils.test.ts`:
- Around line 57-60: The tests currently treat an undefined environment flag as
enabling auto-review; update them to reflect the documented default of disabled
by asserting false. For the tests that call isAutoReviewEnabled with a null/no
config (e.g., the test titled "returns true when no config and env flag is not
set" and the similar cases around the 87-102 block), set
mocks.env.REVIEW_AGENT_AUTO_REVIEW_ENABLED = undefined (or leave it unset) and
change the expectation to expect(isAutoReviewEnabled(null)).toBe(false); apply
the same change to the other affected test cases to ensure the default is
disabled.

In `@packages/web/src/features/agents/review-agent/webhookUtils.ts`:
- Line 17: The current check returns true when
env.REVIEW_AGENT_AUTO_REVIEW_ENABLED is undefined, enabling auto-review by
default; change the condition so auto-review is enabled only when
REVIEW_AGENT_AUTO_REVIEW_ENABLED is explicitly set to "true" (e.g., replace the
loose inequality check of env.REVIEW_AGENT_AUTO_REVIEW_ENABLED with a strict
comparison to "true" or equivalent explicit boolean parsing) so the expression
using env.REVIEW_AGENT_AUTO_REVIEW_ENABLED only returns true when the flag
equals "true".

In `@packages/web/src/openapi/publicApiSchemas.ts`:
- Around line 59-63: The publicAgentConfigSettingsSchema is missing the
contextFiles field used by the review agent; update the z.object for
publicAgentConfigSettingsSchema to include a new optional contextFiles property
typed as an array of strings (e.g., z.array(z.string()).optional()) and add a
descriptive .describe(...) (e.g., "List of file paths or glob patterns to
include as context for reviews; overrides the REVIEW_AGENT_CONTEXT_FILES env
var") so the OpenAPI output and clients can discover and validate this setting.

---

Nitpick comments:
In `@docs/docs/features/agents/review-agent.mdx`:
- Around line 169-171: The docs use passive phrasing for override notes; update
the three rows for REVIEW_AGENT_AUTO_REVIEW_ENABLED,
REVIEW_AGENT_REVIEW_COMMAND, and REVIEW_AGENT_MODEL so the override note uses
second-person present tense (e.g., change "Can be overridden per agent config."
to "You can override this in the agent config." or similar) to match the docs
guideline for docs/**/*.mdx.
- Around line 142-163: In the "Agent configs" intro and the "Context files"
paragraph (look for the headings/phrases "Agent configs let you customise...",
"custom prompt", and the sentence beginning "Set **Context files**..."), replace
em dashes with commas or a colon and change "comma or space separated" to the
hyphenated phrase "comma- or space-separated"; ensure the sentence that
currently reads "...context files — all without changing environment variables."
becomes something like "...context files, all without changing environment
variables." and the "Set **Context files**..." line reads with "comma- or
space-separated list" instead of the em dash and unhyphenated phrase.

In `@packages/db/prisma/migrations/20260421203635_add_agent_config/migration.sql`:
- Around line 29-42: The join tables AgentConfigToRepo and
AgentConfigToConnection lack single-column indexes on repoId and connectionId,
causing seq-scans for deletes/lookups; update the Prisma models for
AgentConfigToRepo and AgentConfigToConnection to add explicit @@index([...]) or
`@unique/`@index on the repoId and connectionId fields (respectively) so Prisma
regenerates a migration that creates indexes on repoId and connectionId in the
SQL, then run prisma migrate to emit the updated migration SQL.

In `@packages/db/prisma/schema.prisma`:
- Around line 641-660: The join models AgentConfigToRepo and
AgentConfigToConnection currently only have composite primary keys
(agentConfigId, repoId) and (agentConfigId, connectionId) which index the
leading column only; add explicit secondary indexes by adding @@index([repoId])
to the AgentConfigToRepo model and @@index([connectionId]) to the
AgentConfigToConnection model so lookups and cascade deletes from
Repo/Connection are efficient (mirror the existing pattern used by
RepoToConnection with @@index([repoId, connectionId])).

In `@packages/web/src/features/agents/review-agent/app.test.ts`:
- Around line 51-63: The test for parseAgentConfigSettings is missing the new
contextFiles setting; update the test 'parses all recognised fields' to include
a sample contextFiles value in the input and expected output so the parser is
validated for the context-file override (reference parseAgentConfigSettings and
the contextFiles property).

In `@packages/web/src/features/agents/review-agent/nodes/fetchFileContent.ts`:
- Around line 10-23: The module-level cachedOrg used by getOrg (backed by
__unsafePrisma.org.findUnique and SINGLE_TENANT_ORG_ID) is never invalidated and
can serve stale org data; update getOrg to avoid a long-lived module cache:
either remove cachedOrg and always call __unsafePrisma.org.findUnique by PK
inside getOrg, or implement a short TTL-based cache (store timestamp alongside
cachedOrg and refresh when expired) so renames/metadata updates are observed
without requiring a process restart. Ensure error handling (throwing
"Organization not found") remains when lookup returns null.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 3f53e2ce-1434-46ad-90f6-ed02e8835240

📥 Commits

Reviewing files that changed from the base of the PR and between d111397 and 18304b0.

📒 Files selected for processing (25)
  • docs/api-reference/sourcebot-public.openapi.json
  • docs/docs/features/agents/review-agent.mdx
  • packages/db/prisma/migrations/20260421203635_add_agent_config/migration.sql
  • packages/db/prisma/schema.prisma
  • packages/web/src/app/(app)/agents/configs/[agentId]/page.tsx
  • packages/web/src/app/(app)/agents/configs/agentConfigForm.tsx
  • packages/web/src/app/(app)/agents/configs/new/page.tsx
  • packages/web/src/app/(app)/agents/page.tsx
  • packages/web/src/app/api/(server)/agents/[agentId]/route.ts
  • packages/web/src/app/api/(server)/agents/route.test.ts
  • packages/web/src/app/api/(server)/agents/route.ts
  • packages/web/src/app/api/(server)/webhook/route.ts
  • packages/web/src/features/agents/review-agent/app.test.ts
  • packages/web/src/features/agents/review-agent/app.ts
  • packages/web/src/features/agents/review-agent/nodes/fetchFileContent.ts
  • packages/web/src/features/agents/review-agent/nodes/generatePrReview.ts
  • packages/web/src/features/agents/review-agent/nodes/invokeDiffReviewLlm.ts
  • packages/web/src/features/agents/review-agent/resolveAgentConfig.test.ts
  • packages/web/src/features/agents/review-agent/resolveAgentConfig.ts
  • packages/web/src/features/agents/review-agent/types.ts
  • packages/web/src/features/agents/review-agent/webhookUtils.test.ts
  • packages/web/src/features/agents/review-agent/webhookUtils.ts
  • packages/web/src/lib/errorCodes.ts
  • packages/web/src/openapi/publicApiDocument.ts
  • packages/web/src/openapi/publicApiSchemas.ts

params: Promise<{ agentId: string }>;
};

export default authenticatedPage(async ({ prisma, org }, { params }: Props) => {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Gate config editing to org owners.

This page exposes the edit form and config details with plain authentication. Add the owner-role option to authenticatedPage so non-owners cannot access agent configuration management.

Proposed fix
+import { OrgRole } from "@sourcebot/db";
...
-export default authenticatedPage(async ({ prisma, org }, { params }: Props) => {
+export default authenticatedPage(async ({ prisma, org }, { params }: Props) => {
     const { agentId } = await params;
...
-});
+}, { minRole: OrgRole.OWNER, redirectTo: "/settings" });

Based on learnings: Use authenticatedPage with { minRole: OrgRole.OWNER, redirectTo: '/settings' } option to gate pages by role in the (app) route group.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
export default authenticatedPage(async ({ prisma, org }, { params }: Props) => {
import { OrgRole } from "@sourcebot/db";
// ... other imports ...
export default authenticatedPage(async ({ prisma, org }, { params }: Props) => {
const { agentId } = await params;
// ... rest of the function body ...
}, { minRole: OrgRole.OWNER, redirectTo: "/settings" });
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/web/src/app/`(app)/agents/configs/[agentId]/page.tsx at line 10, The
page currently allows any authenticated user to edit agent configs; update the
authenticatedPage call to require org owners by passing the options object {
minRole: OrgRole.OWNER, redirectTo: '/settings' } to authenticatedPage (i.e.,
change authenticatedPage(async (...) => ...) to authenticatedPage({ minRole:
OrgRole.OWNER, redirectTo: '/settings' }, async (...) => ...)); also ensure
OrgRole is imported where used (add import for OrgRole if missing) so non-owners
are redirected to /settings.

Comment on lines +89 to +94
settings: {
...(autoReviewEnabled !== undefined && { autoReviewEnabled }),
...(reviewCommand && { reviewCommand }),
...(model && { model }),
...(contextFiles && { contextFiles }),
},
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Confirm whether contextFiles is accepted by the agents API schemas.
rg -n -C4 'contextFiles|agentConfigSettingsSchema|createAgentConfigBodySchema|updateAgentConfigBodySchema' \
  --glob 'packages/web/src/app/api/**/*.ts' \
  --glob 'packages/web/src/app/(app)/agents/**/*.tsx'

Repository: sourcebot-dev/sourcebot

Length of output: 9864


Add contextFiles to the agent config settings schema before shipping this field.

The form submits settings.contextFiles (line 93), but the agents POST/PATCH schemas define agentConfigSettingsSchema with only autoReviewEnabled, reviewCommand, and model. Users can enter context files but the setting will be rejected or lost on save.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/web/src/app/`(app)/agents/configs/agentConfigForm.tsx around lines
89 - 94, The form is submitting settings.contextFiles but the validation schema
agentConfigSettingsSchema doesn't include contextFiles, so those values are
dropped; update the agentConfigSettingsSchema to add a contextFiles property
with the correct type/shape (e.g., array of file identifiers or objects matching
what the form sends) and default/optional rules, and ensure any POST/PATCH
validators or types that consume agentConfigSettingsSchema (and any related
request DTOs or parse/validate calls) are updated to accept and persist
settings.contextFiles (look for usages of agentConfigSettingsSchema and the
settings object in the agent save/update handlers).

import { NavigationMenu } from "@/app/(app)/components/navigationMenu";
import { AgentConfigForm } from "../agentConfigForm";

export default authenticatedPage(async ({ prisma, org }) => {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
rg -nP -C3 'withMinimumOrgRole|OrgRole\.' packages/web/src/app/api/\(server\)/agents
fd -t f 'page.tsx' packages/web/src/app/\(app\)/agents | xargs -I{} sh -c 'echo "=== {} ==="; cat "{}"'

Repository: sourcebot-dev/sourcebot

Length of output: 14689


🏁 Script executed:

cat packages/web/src/app/api/\(server\)/agents/route.ts

Repository: sourcebot-dev/sourcebot

Length of output: 5751


Gate this page and the API with a minimum org role.

Agent config creation is a privileged operation that can change prompts, models, and auto-review behavior org-wide. Using authenticatedPage with { minRole: OrgRole.OWNER, redirectTo: '/settings' } will gate the page, but the API endpoint also needs role enforcement. The POST /api/agents handler currently uses only withAuth without withMinimumOrgRole, allowing any authenticated org member to create configs. Both UI and API must enforce the same role requirement.

Suggested diff for page.tsx
-export default authenticatedPage(async ({ prisma, org }) => {
+export default authenticatedPage(async ({ prisma, org }) => {
     ...
-});
+}, { minRole: OrgRole.OWNER, redirectTo: '/settings' });

Also update packages/web/src/app/api/(server)/agents/route.ts to enforce the same role:

export const POST = apiHandler(async (request: NextRequest) => {
    ...
    const result = await withMinimumOrgRole(OrgRole.OWNER, async ({ org, prisma }) => {
        // existing logic
    });
});

Apply the same gating to the edit page at packages/web/src/app/(app)/agents/configs/[agentId]/page.tsx for consistency.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/web/src/app/`(app)/agents/configs/new/page.tsx at line 5, Update UI
and server to require OrgRole.OWNER for agent config creation/editing: change
the page export that currently calls authenticatedPage(...) to pass the options
{ minRole: OrgRole.OWNER, redirectTo: '/settings' } (look for authenticatedPage
in the new page and the edit page under the agents config routes). In the API
handler that uses withAuth (the POST handler wrapped by apiHandler), replace or
wrap the authorization check with withMinimumOrgRole(OrgRole.OWNER, async ({
org, prisma }) => { ...existing POST logic... }) so only owners can create
configs; ensure you keep existing request handling inside that callback. Also
apply the same authenticatedPage gating to the edit page
(agents/configs/[agentId]/page) so UI and API enforce the same minimum role.

CODE_REVIEW: "Code Review",
};

export default authenticatedPage(async ({ prisma, org }) => {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Restrict the agent configuration list to org owners.

This page now exposes config names, descriptions, scopes, repo/connection associations, and edit links to any authenticated org member. Gate it with the owner role.

Proposed fix
+import { OrgRole } from "@sourcebot/db";
...
-export default authenticatedPage(async ({ prisma, org }) => {
+export default authenticatedPage(async ({ prisma, org }) => {
     const agentConfigs = await prisma.agentConfig.findMany({
...
-});
+}, { minRole: OrgRole.OWNER, redirectTo: "/settings" });

Based on learnings: Use authenticatedPage with { minRole: OrgRole.OWNER, redirectTo: '/settings' } option to gate pages by role in the (app) route group.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
export default authenticatedPage(async ({ prisma, org }) => {
import { OrgRole } from "@sourcebot/db";
export default authenticatedPage(async ({ prisma, org }) => {
const agentConfigs = await prisma.agentConfig.findMany({
// ... rest of the function body
});
// ... return statement
}, { minRole: OrgRole.OWNER, redirectTo: "/settings" });
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/web/src/app/`(app)/agents/page.tsx at line 37, The page is exposing
agent configs to all authenticated org members; update the authenticatedPage
call to require owner role by passing options like { minRole: OrgRole.OWNER,
redirectTo: '/settings' } so only org owners can access it; modify the existing
authenticatedPage(...) invocation (the exported default function) to include
these options and ensure OrgRole is imported where used.

Comment thread packages/web/src/app/api/(server)/agents/[agentId]/route.ts Outdated
Comment thread packages/web/src/features/agents/review-agent/nodes/generatePrReview.ts Outdated
Comment thread packages/web/src/features/agents/review-agent/resolveAgentConfig.ts
Comment on lines +57 to +60
test('returns true when no config and env flag is not set', () => {
mocks.env.REVIEW_AGENT_AUTO_REVIEW_ENABLED = undefined;

expect(isAutoReviewEnabled(null)).toBe(true);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Update the unset-env expectations to default to disabled.

These tests currently lock in undefined env as auto-review enabled. That conflicts with the documented default and would keep the unsafe fallback from being fixed.

Proposed test update
-    test('returns true when no config and env flag is not set', () => {
+    test('returns false when no config and env flag is not set', () => {
         mocks.env.REVIEW_AGENT_AUTO_REVIEW_ENABLED = undefined;
 
-        expect(isAutoReviewEnabled(null)).toBe(true);
+        expect(isAutoReviewEnabled(null)).toBe(false);
     });
...
-    test('per-config false overrides unset env (which would default to true)', () => {
+    test('per-config false is preserved when env is unset', () => {
         mocks.env.REVIEW_AGENT_AUTO_REVIEW_ENABLED = undefined;
 
         expect(isAutoReviewEnabled(makeConfig({ autoReviewEnabled: false }))).toBe(false);
     });
...
-    test('returns true when config is provided but has no settings at all', () => {
+    test('returns false when config is provided but has no settings at all', () => {
         mocks.env.REVIEW_AGENT_AUTO_REVIEW_ENABLED = undefined;
 
-        expect(isAutoReviewEnabled(makeConfig({}))).toBe(true);
+        expect(isAutoReviewEnabled(makeConfig({}))).toBe(false);
     });

Also applies to: 87-102

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/web/src/features/agents/review-agent/webhookUtils.test.ts` around
lines 57 - 60, The tests currently treat an undefined environment flag as
enabling auto-review; update them to reflect the documented default of disabled
by asserting false. For the tests that call isAutoReviewEnabled with a null/no
config (e.g., the test titled "returns true when no config and env flag is not
set" and the similar cases around the 87-102 block), set
mocks.env.REVIEW_AGENT_AUTO_REVIEW_ENABLED = undefined (or leave it unset) and
change the expectation to expect(isAutoReviewEnabled(null)).toBe(false); apply
the same change to the other affected test cases to ensure the default is
disabled.

return settings.autoReviewEnabled;
}
}
return env.REVIEW_AGENT_AUTO_REVIEW_ENABLED !== "false";
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Keep auto-review disabled when the env flag is unset.

Line 17 enables auto-review for undefined, which contradicts the documented default of disabled unless REVIEW_AGENT_AUTO_REVIEW_ENABLED is set to true. This can unexpectedly trigger reviews for every PR/MR when the env value is absent.

Proposed fix
-    return env.REVIEW_AGENT_AUTO_REVIEW_ENABLED !== "false";
+    return env.REVIEW_AGENT_AUTO_REVIEW_ENABLED === "true";
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
return env.REVIEW_AGENT_AUTO_REVIEW_ENABLED !== "false";
return env.REVIEW_AGENT_AUTO_REVIEW_ENABLED === "true";
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/web/src/features/agents/review-agent/webhookUtils.ts` at line 17,
The current check returns true when env.REVIEW_AGENT_AUTO_REVIEW_ENABLED is
undefined, enabling auto-review by default; change the condition so auto-review
is enabled only when REVIEW_AGENT_AUTO_REVIEW_ENABLED is explicitly set to
"true" (e.g., replace the loose inequality check of
env.REVIEW_AGENT_AUTO_REVIEW_ENABLED with a strict comparison to "true" or
equivalent explicit boolean parsing) so the expression using
env.REVIEW_AGENT_AUTO_REVIEW_ENABLED only returns true when the flag equals
"true".

Comment on lines +59 to +63
const publicAgentConfigSettingsSchema = z.object({
autoReviewEnabled: z.boolean().optional().describe('Whether the agent automatically reviews new PRs/MRs. Overrides the REVIEW_AGENT_AUTO_REVIEW_ENABLED env var.'),
reviewCommand: z.string().optional().describe('Comment command that triggers a manual review (without the leading /). Overrides the REVIEW_AGENT_REVIEW_COMMAND env var.'),
model: z.string().optional().describe('Display name of the language model to use for this config. Overrides the REVIEW_AGENT_MODEL env var.'),
}).openapi('PublicAgentConfigSettings');
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Verify that all agent-config settings schemas expose contextFiles consistently.
rg -n -C3 'agentConfigSettingsSchema|publicAgentConfigSettingsSchema|contextFiles' \
  --glob 'packages/web/src/**/*.ts' \
  --glob 'packages/web/src/**/*.tsx'

Repository: sourcebot-dev/sourcebot

Length of output: 13388


Add contextFiles to the public agent settings schema.

contextFiles is actively used by the review agent and sent by the form, but the public schema omits this field. External API clients and generated OpenAPI documentation will be unable to discover or validate this setting.

🔧 Proposed fix
 const publicAgentConfigSettingsSchema = z.object({
     autoReviewEnabled: z.boolean().optional().describe('Whether the agent automatically reviews new PRs/MRs. Overrides the REVIEW_AGENT_AUTO_REVIEW_ENABLED env var.'),
     reviewCommand: z.string().optional().describe('Comment command that triggers a manual review (without the leading /). Overrides the REVIEW_AGENT_REVIEW_COMMAND env var.'),
     model: z.string().optional().describe('Display name of the language model to use for this config. Overrides the REVIEW_AGENT_MODEL env var.'),
+    contextFiles: z.string().optional().describe('Comma- or space-separated repository paths whose contents are injected as review guidance.'),
 }).openapi('PublicAgentConfigSettings');
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
const publicAgentConfigSettingsSchema = z.object({
autoReviewEnabled: z.boolean().optional().describe('Whether the agent automatically reviews new PRs/MRs. Overrides the REVIEW_AGENT_AUTO_REVIEW_ENABLED env var.'),
reviewCommand: z.string().optional().describe('Comment command that triggers a manual review (without the leading /). Overrides the REVIEW_AGENT_REVIEW_COMMAND env var.'),
model: z.string().optional().describe('Display name of the language model to use for this config. Overrides the REVIEW_AGENT_MODEL env var.'),
}).openapi('PublicAgentConfigSettings');
const publicAgentConfigSettingsSchema = z.object({
autoReviewEnabled: z.boolean().optional().describe('Whether the agent automatically reviews new PRs/MRs. Overrides the REVIEW_AGENT_AUTO_REVIEW_ENABLED env var.'),
reviewCommand: z.string().optional().describe('Comment command that triggers a manual review (without the leading /). Overrides the REVIEW_AGENT_REVIEW_COMMAND env var.'),
model: z.string().optional().describe('Display name of the language model to use for this config. Overrides the REVIEW_AGENT_MODEL env var.'),
contextFiles: z.string().optional().describe('Comma- or space-separated repository paths whose contents are injected as review guidance.'),
}).openapi('PublicAgentConfigSettings');
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/web/src/openapi/publicApiSchemas.ts` around lines 59 - 63, The
publicAgentConfigSettingsSchema is missing the contextFiles field used by the
review agent; update the z.object for publicAgentConfigSettingsSchema to include
a new optional contextFiles property typed as an array of strings (e.g.,
z.array(z.string()).optional()) and add a descriptive .describe(...) (e.g.,
"List of file paths or glob patterns to include as context for reviews;
overrides the REVIEW_AGENT_CONTEXT_FILES env var") so the OpenAPI output and
clients can discover and validate this setting.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@packages/db/prisma/schema.prisma`:
- Around line 640-660: Add a tenant-scoped orgId column to both
AgentConfigToRepo and AgentConfigToConnection and make the PK include orgId
(e.g. @@id([orgId, agentConfigId, repoId]) and @@id([orgId, agentConfigId,
connectionId])) so the DB enforces tenant isolation; also add reverse lookup
indexes @@index([repoId, agentConfigId]) and @@index([connectionId,
agentConfigId]) respectively to speed queries. Ensure the new orgId field type
matches your org identifier (e.g. String) and populate/migrate it from
AgentConfig/Repo/Connection so the FK/PK invariants hold.

In `@packages/web/src/app/api/`(server)/agents/[agentId]/route.test.ts:
- Around line 127-166: The test suite's local beforeEach in the "name collision"
describe block doesn't clear prior mock call history causing the negative
assertion on prisma.agentConfig.findUnique (in the test that expects it not to
have been called) to be flaky; update that describe's beforeEach to call
vi.clearAllMocks() (or vi.resetAllMocks()) before setting
prisma.agentConfig.findFirst.mockResolvedValue(...) so that
prisma.agentConfig.findUnique calls from earlier tests are cleared and the
expect(prisma.agentConfig.findUnique).not.toHaveBeenCalled() assertion is valid.

In
`@packages/web/src/features/agents/review-agent/nodes/fetchFileContent.test.ts`:
- Around line 11-20: The test currently replaces fileSourceResponseSchema with
an overly permissive mock; instead restore the real fileSourceResponseSchema and
only mock getFileSourceForRepo (keep mockGetFileSourceForRepo). In
fetchFileContent.test.ts, remove the fake schema from the vi.mock and
import/retain the actual fileSourceResponseSchema used in production, then
update all success fixtures returned by mockGetFileSourceForRepo to include the
required fields (source, language, path, repo, repoCodeHostType, webUrl) so they
conform to the real schema’s validation.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: ca46226b-3494-4ba8-9186-02b4e35ce290

📥 Commits

Reviewing files that changed from the base of the PR and between 18304b0 and 561ee96.

📒 Files selected for processing (14)
  • packages/db/prisma/schema.prisma
  • packages/web/src/app/(app)/agents/configs/[agentId]/page.tsx
  • packages/web/src/app/(app)/agents/configs/new/page.tsx
  • packages/web/src/app/api/(server)/agents/[agentId]/route.test.ts
  • packages/web/src/app/api/(server)/agents/[agentId]/route.ts
  • packages/web/src/app/api/(server)/agents/route.ts
  • packages/web/src/app/api/(server)/webhook/route.ts
  • packages/web/src/features/agents/review-agent/app.ts
  • packages/web/src/features/agents/review-agent/nodes/fetchFileContent.test.ts
  • packages/web/src/features/agents/review-agent/nodes/fetchFileContent.ts
  • packages/web/src/features/agents/review-agent/nodes/generatePrReview.test.ts
  • packages/web/src/features/agents/review-agent/nodes/generatePrReview.ts
  • packages/web/src/features/agents/review-agent/resolveAgentConfig.ts
  • packages/web/src/openapi/publicApiSchemas.ts
✅ Files skipped from review due to trivial changes (1)
  • packages/web/src/app/(app)/agents/configs/new/page.tsx
🚧 Files skipped from review as they are similar to previous changes (9)
  • packages/web/src/features/agents/review-agent/nodes/generatePrReview.ts
  • packages/web/src/features/agents/review-agent/nodes/fetchFileContent.ts
  • packages/web/src/features/agents/review-agent/resolveAgentConfig.ts
  • packages/web/src/openapi/publicApiSchemas.ts
  • packages/web/src/app/(app)/agents/configs/[agentId]/page.tsx
  • packages/web/src/app/api/(server)/agents/[agentId]/route.ts
  • packages/web/src/app/api/(server)/agents/route.ts
  • packages/web/src/features/agents/review-agent/app.ts
  • packages/web/src/app/api/(server)/webhook/route.ts

Comment thread packages/db/prisma/schema.prisma
Comment thread packages/web/src/app/api/(server)/agents/[agentId]/route.test.ts
Comment on lines +11 to +20
vi.mock('@/features/git', () => ({
getFileSourceForRepo: (...args: unknown[]) => mockGetFileSourceForRepo(...args),
fileSourceResponseSchema: {
safeParse: (v: unknown) => {
if (v && typeof v === 'object' && 'source' in v) {
return { success: true, data: v };
}
return { success: false, error: new Error('parse failure') };
},
},
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Use the real fileSourceResponseSchema in this mock.

The mocked schema accepts any object with source, but the real schema also requires fields like language, path, repo, repoCodeHostType, and webUrl. As written, the success tests use responses production validation would reject, so schema-regression coverage is weakened. Prefer preserving the real schema and mocking only getFileSourceForRepo, then update success fixtures to be valid file-source responses.

Suggested direction
-vi.mock('@/features/git', () => ({
-    getFileSourceForRepo: (...args: unknown[]) => mockGetFileSourceForRepo(...args),
-    fileSourceResponseSchema: {
-        safeParse: (v: unknown) => {
-            if (v && typeof v === 'object' && 'source' in v) {
-                return { success: true, data: v };
-            }
-            return { success: false, error: new Error('parse failure') };
-        },
-    },
-}));
+vi.mock('@/features/git', async (importOriginal) => {
+    const actual = await importOriginal<typeof import('@/features/git')>();
+    return {
+        ...actual,
+        getFileSourceForRepo: (...args: unknown[]) => mockGetFileSourceForRepo(...args),
+    };
+});
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/web/src/features/agents/review-agent/nodes/fetchFileContent.test.ts`
around lines 11 - 20, The test currently replaces fileSourceResponseSchema with
an overly permissive mock; instead restore the real fileSourceResponseSchema and
only mock getFileSourceForRepo (keep mockGetFileSourceForRepo). In
fetchFileContent.test.ts, remove the fake schema from the vi.mock and
import/retain the actual fileSourceResponseSchema used in production, then
update all success fixtures returned by mockGetFileSourceForRepo to include the
required fields (source, language, path, repo, repoCodeHostType, webUrl) so they
conform to the real schema’s validation.

@gaeljw
Copy link
Copy Markdown

gaeljw commented Apr 22, 2026

I like the idea of having customizable config per repo, that's what other tools offer as well (like Qodo PR Agent).

However I would have expected to define them in a file in the repo itself (.sourcebot for instance).

The benefits of having in-repo is: not another UI to play with, versioned and included in the regular development lifecycle (versionning, PR, review, IDE, ...).

@fatmcgav
Copy link
Copy Markdown
Contributor Author

I like the idea of having customizable config per repo, that's what other tools offer as well (like Qodo PR Agent).

However I would have expected to define them in a file in the repo itself (.sourcebot for instance).

The benefits of having in-repo is: not another UI to play with, versioned and included in the regular development lifecycle (versionning, PR, review, IDE, ...).

@gaeljw Thanks for taking the time to review and comment :)

I did debate that, but thought there's still benefit in keeping the "core" prompt logic in Sourcebot, as that's where the bulk of the prompt is formed.

However there should be enough configurability when combining instructions, which live in Sourcebot and can either extend the "default" rules or replace them, with context files that exist in the repo itself, such as AGENTS.md or REVIEW.md.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
docs/docs/features/agents/review-agent.mdx (1)

163-163: Minor: hyphenate compound modifier.

"comma or space separated list" reads more cleanly as "comma- or space-separated list" (per LanguageTool).

✏️ Proposed fix
-Set **Context files** in the agent config form to a comma or space separated list of paths relative to the repository root (e.g. `AGENTS.md .sourcebot/review.md`). Files that do not exist in the repository are silently ignored. The files are fetched once per PR from the head commit and included in the context for every diff hunk.
+Set **Context files** in the agent config form to a comma- or space-separated list of paths relative to the repository root (e.g. `AGENTS.md .sourcebot/review.md`). Files that do not exist in the repository are silently ignored. The files are fetched once per PR from the head commit and included in the context for every diff hunk.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@docs/docs/features/agents/review-agent.mdx` at line 163, Update the phrasing
in the review-agent documentation sentence that currently reads "comma or space
separated list" to use a hyphenated compound modifier: change it to "comma- or
space-separated list" in the text inside
docs/docs/features/agents/review-agent.mdx (the sentence describing Context
files in the agent config form).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@docs/docs/features/agents/review-agent.mdx`:
- Around line 146-148: Replace the em dashes in the three bullets ("**Repo** —",
"**Connection** —", "**Org** —") with an allowed separator; update each line to
use a colon (e.g., "**Repo:** applies to specific repositories (highest
priority)"), or a period after the bold term, ensuring the rest of the bullet
text remains unchanged and no em dash characters remain in review-agent.mdx.

---

Nitpick comments:
In `@docs/docs/features/agents/review-agent.mdx`:
- Line 163: Update the phrasing in the review-agent documentation sentence that
currently reads "comma or space separated list" to use a hyphenated compound
modifier: change it to "comma- or space-separated list" in the text inside
docs/docs/features/agents/review-agent.mdx (the sentence describing Context
files in the agent config form).
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 7d80f85a-0985-4ecc-b8b7-f8573eee6ea7

📥 Commits

Reviewing files that changed from the base of the PR and between 561ee96 and 036f626.

📒 Files selected for processing (5)
  • docs/docs/features/agents/review-agent.mdx
  • packages/db/prisma/migrations/20260422071731_add_agent_config/migration.sql
  • packages/db/prisma/schema.prisma
  • packages/web/src/app/api/(server)/agents/route.test.ts
  • packages/web/src/features/agents/review-agent/app.test.ts
✅ Files skipped from review due to trivial changes (1)
  • packages/web/src/features/agents/review-agent/app.test.ts

Comment on lines +146 to +148
- **Repo** — applies to specific repositories (highest priority)
- **Connection** — applies to all repos in a specific connection
- **Org** — applies to all repositories in your org (lowest priority, catch-all)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Replace em dashes in bullet list.

Lines 146-148 use em dashes () which are disallowed by the docs style guide. Use a different separator (e.g., a colon or a period after the bolded term).

✏️ Proposed fix
-- **Repo** — applies to specific repositories (highest priority)
-- **Connection** — applies to all repos in a specific connection
-- **Org** — applies to all repositories in your org (lowest priority, catch-all)
+- **Repo**: applies to specific repositories (highest priority)
+- **Connection**: applies to all repos in a specific connection
+- **Org**: applies to all repositories in your org (lowest priority, catch-all)

As per coding guidelines: "do NOT use em dashes (—)".

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
- **Repo** applies to specific repositories (highest priority)
- **Connection** applies to all repos in a specific connection
- **Org** applies to all repositories in your org (lowest priority, catch-all)
- **Repo**: applies to specific repositories (highest priority)
- **Connection**: applies to all repos in a specific connection
- **Org**: applies to all repositories in your org (lowest priority, catch-all)
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@docs/docs/features/agents/review-agent.mdx` around lines 146 - 148, Replace
the em dashes in the three bullets ("**Repo** —", "**Connection** —", "**Org**
—") with an allowed separator; update each line to use a colon (e.g., "**Repo:**
applies to specific repositories (highest priority)"), or a period after the
bold term, ensuring the rest of the bullet text remains unchanged and no em dash
characters remain in review-agent.mdx.

PATCH handler findFirst include — Added repos: { select: { repoId: true } } and connections: { select: { connectionId: true } } to the PATCH handler's initial
findFirst, so existing.repos and existing.connections are available when computing effective IDs for the scope conflict check.

resolveAgentConfig.ts — Removed orderBy: { updatedAt: 'desc' } from all three findFirst queries (REPO, CONNECTION, ORG). Scope-level uniqueness is now enforced at
the API layer, making tiebreaker ordering unnecessary.

Tests — Added scope conflict coverage to both route test files:
- route.test.ts (POST): 5 new tests — ORG conflict, REPO overlap, CONNECTION overlap, disabled bypass, no create on conflict
- [agentId]/route.test.ts (PATCH): 5 new tests — ORG conflict, REPO overlap, CONNECTION overlap, disabled bypass, no update on conflict
- Fixed beforeEach mock chaining in PATCH name-collision and successful-update suites (was inadvertently passing makeDbConfig back for the conflict findFirst too)
- Updated makeDbConfig overrides type to accept repos/connections relation arrays
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (1)
docs/docs/features/agents/review-agent.mdx (1)

146-148: ⚠️ Potential issue | 🟡 Minor

Replace em dashes with colons.

Lines 146-148 still use em dashes () which violate the docs style guide. Replace them with colons.

✏️ Proposed fix
-- **Repo** — applies to specific repositories (highest priority)
-- **Connection** — applies to all repos in a specific connection
-- **Org** — applies to all repositories in your org (lowest priority, catch-all)
+- **Repo**: applies to specific repositories (highest priority)
+- **Connection**: applies to all repos in a specific connection
+- **Org**: applies to all repositories in your org (lowest priority, catch-all)

As per coding guidelines: "do NOT use em dashes (—)".

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@docs/docs/features/agents/review-agent.mdx` around lines 146 - 148, The three
list items using em dashes ("**Repo** — applies to specific repositories
(highest priority)", "**Connection** — applies to all repos in a specific
connection", "**Org** — applies to all repositories in your org (lowest
priority, catch-all)") should use colons instead; edit the text in
review-agent.mdx to replace each em dash with a colon so they read "**Repo**:
applies to specific repositories (highest priority)", "**Connection**: applies
to all repos in a specific connection", and "**Org**: applies to all
repositories in your org (lowest priority, catch-all)".
🧹 Nitpick comments (2)
packages/web/src/app/api/(server)/agents/[agentId]/route.test.ts (2)

32-42: Avoid letting the mocked settings schema drift from production validation.

This replaces agentConfigSettingsSchema with a hand-rolled permissive schema, so route validation tests can pass even if the production schema rejects or requires different settings fields. Consider moving the schema to a lightweight module that both app.ts and these tests can import, or add focused contract tests against the real schema.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/web/src/app/api/`(server)/agents/[agentId]/route.test.ts around
lines 32 - 42, The test replaces agentConfigSettingsSchema with a permissive
hand-rolled schema, which can drift from production validation; instead extract
the real Zod schema into a small shared module (e.g., export
agentConfigSettingsSchema from a lightweight file) that both app.ts and the
tests import, or update the test mock to import that shared schema so
updateAgentConfigBodySchema.safeParse uses the true validation; additionally add
a focused contract test against agentConfigSettingsSchema to ensure the route
test inputs are validated the same way as production.

246-274: Assert the existing-config fetch requests scope relation IDs.

These tests inject repos/connections into the mocked existing config regardless of the Prisma query shape. If the handler stops including/selecting those relation IDs, real overlap detection can break while these tests still pass.

Test hardening example
             const res = await PATCH(makePatchRequest(AGENT_ID, { prompt: 'updated' }), params);
 
             expect(res.status).toBe(StatusCodes.CONFLICT);
+            expect(prisma.agentConfig.findFirst).toHaveBeenNthCalledWith(
+                1,
+                expect.objectContaining({
+                    include: expect.objectContaining({
+                        repos: expect.anything(),
+                        connections: expect.anything(),
+                    }),
+                }),
+            );
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/web/src/app/api/`(server)/agents/[agentId]/route.test.ts around
lines 246 - 274, The tests currently inject full configs with repos/connections
regardless of what the handler actually selects, masking regressions; update the
two tests that use makeDbConfig + prisma.agentConfig.findFirst to (1) assert
that prisma.agentConfig.findFirst was called with a select/include that requests
the relation ID fields for the given scope (repos for REPO, connections for
CONNECTION) and (2) change the mocks to return objects shaped exactly like the
query would (e.g., use mockImplementationOnce on prisma.agentConfig.findFirst to
inspect the incoming args and return an object with only the requested repos or
connections id arrays), while keeping the rest of the test flow using
PATCH(makePatchRequest(AGENT_ID, { prompt: 'updated' }), params) and existing
expect(res.status).toBe(StatusCodes.CONFLICT).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@docs/docs/features/agents/review-agent.mdx`:
- Line 171: The sentence under the "Context files" description uses incorrect
hyphenation; update the phrase that currently reads "comma or space separated
list of paths" (the sentence starting with "Set **Context files** in the agent
config form...") to "comma- or space-separated list of paths relative to the
repository root (e.g. `AGENTS.md .sourcebot/review.md`)" so the compound
adjectives are properly hyphenated.

---

Duplicate comments:
In `@docs/docs/features/agents/review-agent.mdx`:
- Around line 146-148: The three list items using em dashes ("**Repo** — applies
to specific repositories (highest priority)", "**Connection** — applies to all
repos in a specific connection", "**Org** — applies to all repositories in your
org (lowest priority, catch-all)") should use colons instead; edit the text in
review-agent.mdx to replace each em dash with a colon so they read "**Repo**:
applies to specific repositories (highest priority)", "**Connection**: applies
to all repos in a specific connection", and "**Org**: applies to all
repositories in your org (lowest priority, catch-all)".

---

Nitpick comments:
In `@packages/web/src/app/api/`(server)/agents/[agentId]/route.test.ts:
- Around line 32-42: The test replaces agentConfigSettingsSchema with a
permissive hand-rolled schema, which can drift from production validation;
instead extract the real Zod schema into a small shared module (e.g., export
agentConfigSettingsSchema from a lightweight file) that both app.ts and the
tests import, or update the test mock to import that shared schema so
updateAgentConfigBodySchema.safeParse uses the true validation; additionally add
a focused contract test against agentConfigSettingsSchema to ensure the route
test inputs are validated the same way as production.
- Around line 246-274: The tests currently inject full configs with
repos/connections regardless of what the handler actually selects, masking
regressions; update the two tests that use makeDbConfig +
prisma.agentConfig.findFirst to (1) assert that prisma.agentConfig.findFirst was
called with a select/include that requests the relation ID fields for the given
scope (repos for REPO, connections for CONNECTION) and (2) change the mocks to
return objects shaped exactly like the query would (e.g., use
mockImplementationOnce on prisma.agentConfig.findFirst to inspect the incoming
args and return an object with only the requested repos or connections id
arrays), while keeping the rest of the test flow using
PATCH(makePatchRequest(AGENT_ID, { prompt: 'updated' }), params) and existing
expect(res.status).toBe(StatusCodes.CONFLICT).
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: ede740dd-5474-45a5-83f6-c41b6860a47a

📥 Commits

Reviewing files that changed from the base of the PR and between 036f626 and c6878cf.

📒 Files selected for processing (8)
  • CHANGELOG.md
  • docs/docs/features/agents/review-agent.mdx
  • packages/web/src/app/api/(server)/agents/[agentId]/route.test.ts
  • packages/web/src/app/api/(server)/agents/[agentId]/route.ts
  • packages/web/src/app/api/(server)/agents/route.test.ts
  • packages/web/src/app/api/(server)/agents/route.ts
  • packages/web/src/features/agents/review-agent/resolveAgentConfig.ts
  • packages/web/src/lib/errorCodes.ts
✅ Files skipped from review due to trivial changes (3)
  • packages/web/src/lib/errorCodes.ts
  • CHANGELOG.md
  • packages/web/src/app/api/(server)/agents/[agentId]/route.ts
🚧 Files skipped from review as they are similar to previous changes (3)
  • packages/web/src/features/agents/review-agent/resolveAgentConfig.ts
  • packages/web/src/app/api/(server)/agents/route.test.ts
  • packages/web/src/app/api/(server)/agents/route.ts


You can configure one or more repository files to be fetched at review time and injected as additional context for the model. This is useful for encoding project-specific conventions that the model should be aware of when reviewing diffs (for example, preferred error handling patterns, style rules, or areas of the codebase that need extra scrutiny).

Set **Context files** in the agent config form to a comma or space separated list of paths relative to the repository root (e.g. `AGENTS.md .sourcebot/review.md`). Files that do not exist in the repository are silently ignored. The files are fetched once per PR from the head commit and included in the context for every diff hunk.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Fix hyphenation in compound adjective.

The phrase "comma or space separated list" should be "comma- or space-separated list" per standard grammar rules for compound adjectives.

✏️ Proposed fix
-Set **Context files** in the agent config form to a comma or space separated list of paths relative to the repository root (e.g. `AGENTS.md .sourcebot/review.md`).
+Set **Context files** in the agent config form to a comma- or space-separated list of paths relative to the repository root (e.g. `AGENTS.md .sourcebot/review.md`).
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
Set **Context files** in the agent config form to a comma or space separated list of paths relative to the repository root (e.g. `AGENTS.md .sourcebot/review.md`). Files that do not exist in the repository are silently ignored. The files are fetched once per PR from the head commit and included in the context for every diff hunk.
Set **Context files** in the agent config form to a comma- or space-separated list of paths relative to the repository root (e.g. `AGENTS.md .sourcebot/review.md`). Files that do not exist in the repository are silently ignored. The files are fetched once per PR from the head commit and included in the context for every diff hunk.
🧰 Tools
🪛 LanguageTool

[grammar] ~171-~171: Use a hyphen to join words.
Context: ...he agent config form to a comma or space separated list of paths relative to the ...

(QB_NEW_EN_HYPHEN)

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@docs/docs/features/agents/review-agent.mdx` at line 171, The sentence under
the "Context files" description uses incorrect hyphenation; update the phrase
that currently reads "comma or space separated list of paths" (the sentence
starting with "Set **Context files** in the agent config form...") to "comma- or
space-separated list of paths relative to the repository root (e.g. `AGENTS.md
.sourcebot/review.md`)" so the compound adjectives are properly hyphenated.

@gaeljw
Copy link
Copy Markdown

gaeljw commented Apr 22, 2026

I understand these are 2 different visions and ways of working.

This is just genuine feedback that having to configure repo settings in Sourcebot UI will be a deal breaker for me in the way we are used to work: GitLab (or GitHub) being the central place. The permissions and versioning aspects are also important: I would want to see who changed which settings in the Git history.

EDIT: to give more context, I don't think I would ever customise prompts at the repo level. I would definitely like to enable isAutoReviewEnabled per repo, and directly in my repo without going to Sourcebot UI though. (By the way, a list of excluded branches pattern could be interesting, though it can be set at the webhook level).

@fatmcgav
Copy link
Copy Markdown
Contributor Author

This is just genuine feedback that having to configure repo settings in Sourcebot UI will be a deal breaker for me in the way we are used to work: GitLab (or GitHub) being the central place. The permissions and versioning aspects are also important: I would want to see who changed which settings in the Git history.

Makes sense @gaeljw ... I guess a good natural evolution would be to have a Terraform provider that would allow IaC control of the agents config in this proposed model...

I don't think I would ever customise prompts at the repo level. I would definitely like to enable isAutoReviewEnabled per repo, and directly in my repo without going to Sourcebot UI though.

OK, that feels like an interesting extension. and could be a good use for a .sourcebot/settings.yml file or similar... However I believe you'd still need to do a initial Org agent setup in Sourcebot so that it "knows" to look at the repos in the first place...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants