feat: separate profile scope from agent execution

[neubig]

Why

A saved agent profile chooses credentials and agent configuration. It should not also force every deterministic automation script to create an agent conversation and, in Docker mode, a container. Scheduled scanners need their selected API token, while only the agent work they select needs a conversation runtime.

Summary

Add an explicit execution_mode snapshot alongside agent_profile_id. Raw automation bundles default to script; prompt and plugin presets use agent. Profile-backed scripts remain ordinary automation commands and pass the profile ID through the typed software-agent-sdk command API so the Agent Server exposes only that profile's selected secrets. Existing profile-backed definitions migrate to agent, preserving their current behavior.

Webhook-created runs now snapshot the same profile and execution fields as scheduled runs. CRUD responses and Git sync preserve both fields.

Issue Number

Closes #464.

How to Test

Focused backend, execution, CRUD, preset, Git-sync, queued-run, and SQLite migration tests pass. Formatting, Ruff, pycodestyle, and Pyright pass.

Review order and dependencies

Stack #454: #449 → #453 → #466 → #467 → #468. Review and merge in that order. This PR also depends on software-agent-sdk #5046; replace the temporary immutable source pin with its release before merging.

[github-actions]

Warning

Your comment is too long (maximum is 65536 characters), so the coverage report was not added. See the job log for how to reduce it.

[all-hands-bot]

🤖 OpenHands is reviewing this PR.

Head commit: 435b3154743efeecf317fad8e1c97f870b3eb34d
View the conversation: https://oss-agent-canvas.ngrok.dev/conversations/e09c105f-4557-4fe5-9524-8eb76d4e65f7

This comment was posted by an AI agent (OpenHands).

[all-hands-bot]

This review was created by an AI agent (OpenHands) on behalf of the repository maintainers.

Summary

This PR introduces an explicit execution_mode field (script | agent) alongside agent_profile_id, cleanly separating credential scoping from conversation runtime creation. Profile-backed scripts pass the profile ID through the typed SDK command API so the Agent Server exposes only that profile's selected secrets, without creating a conversation or Docker runtime. Prompt/plugin presets default to agent; raw bundles default to script. Existing profile-backed definitions migrate to agent, preserving current behavior.

Analysis

Data structure: Good taste. Adding execution_mode as an independent dimension eliminates the implicit coupling between profile selection and conversation creation — previously, setting a profile ID implicitly forced conversation-backed execution. The AutomationExecutionMode enum provides a single source of truth for the valid values and defaults.

Migration: Cross-database compatible — generic sa.Column with String(20), server_default="script", and raw op.execute UPDATE statements that work on both PostgreSQL and SQLite. The backfill (SET execution_mode = 'agent' WHERE agent_profile_id IS NOT NULL) correctly preserves the previous behavior for existing automations. Downgrade is clean.

Backend selection: get_backend now gates on run.execution_mode == "agent" and bool(profile_id), which correctly handles all four combinations (script/agent x profile/no-profile). Script runs with a profile fall through to LocalAgentServerBackend but get the profile ID passed to workspace.start_command for secret scoping. Agent runs without a profile also fall through — reasonable since there's nothing to configure the agent with.

Dispatcher: agent_profile_id is only forwarded to execute_in_context for script runs. For agent runs it's None, which is correct — the SDK conversation resolves the profile itself.

Git sync: Export includes execution_mode; import defaults to "script" when absent (backward compat with older YAML). Validation rejects invalid values. Consistent with the existing agent_profile_id handling pattern.

Tests: test_conversation_backend.py parametrizes all three meaningful combinations (script-no-profile, script-with-profile, agent-with-profile). test_execution.py verifies agent_profile_id is forwarded through the full call chain. test_db.py checks the migration column exists on both tables. These exercise real code paths, not just mock assertions.

Non-blocking observation

UpdateAutomationRequest.execution_mode is typed as Literal["script", "agent"] | None = None. If a client explicitly sends "execution_mode": null in a PATCH body, model_dump(exclude_unset=True) includes it, and the setattr loop sets the NOT NULL column to None, causing a 500 at flush time. This is low impact (requires explicit action, result is just an error) but could be avoided by either omitting None from the union or skipping None values in the setattr loop for NOT NULL fields.

Risk Assessment

🟢 LOW — The change is additive: a new nullable-with-default column, a new field on existing schemas, and a new branch in backend selection. The migration preserves existing behavior. No breaking changes to public APIs (the new field defaults to script, and existing automations are backfilled to agent). The SDK pin update is first-party (OpenHands/software-agent-sdk) and acknowledged as temporary in the PR description.

Verdict: ✅ Worth merging

Key insight: Decoupling execution_mode from agent_profile_id is the right abstraction — it lets deterministic scripts use profile-scoped secrets without the overhead of a conversation runtime, and it makes the execution path explicit rather than implied by the presence of a profile ID.

---

Improve this review? If any feedback above seems incorrect or irrelevant to this repository, you can teach the reviewer to do better:

1. Add a .agents/skills/custom-codereview-guide.md file to your branch (or edit it if one already exists) with the /codereview trigger and the context the reviewer is missing (e.g., "Security concerns about X do not apply here because Y"). See the customization docs for the required frontmatter format.
2. Re-request a review - the reviewer reads guidelines from the PR branch, so your changes take effect immediately.
3. When your PR is merged, the guideline file goes through normal code review by repository maintainers.

Resolve with AI? Install the iterate skill in your agent and run /iterate to automatically drive this PR through CI, review, and QA until it's merge-ready.

Was this review helpful? React with 👍 or 👎 to give feedback.

Automated review used the wrong decision (APPROVED instead of COMMENT) and is dismissed. Findings are reposted as a comment.

[all-hands-bot]

Posted an APPROVED review to PR #466 on GitHub (review ID 5204534183, commit 435b315).

Verdict: ✅ Worth merging — 🟢 LOW risk

The PR cleanly separates execution_mode (script|agent) from agent_profile_id, allowing deterministic automation scripts to use profile-scoped secrets without creating a conversation runtime. Key findings:

• Good data structure design: Decoupling execution mode from profile selection eliminates implicit coupling — previously a profile ID implicitly forced conversation-backed execution.
• Migration is correct: Cross-database compatible, backfills existing profile-backed automations to agent mode, preserving current behavior.
• Backend selection logic correctly handles all four combinations of execution_mode × profile.
• Tests exercise real code paths across backend selection, execution forwarding, and migration verification.
• One non-blocking observation: UpdateAutomationRequest.execution_mode allows None, which could cause a 500 if a client explicitly sends null in a PATCH body (the column is NOT NULL). Low impact, included in the review body.

Per the custom codereview guide, submitted as APPROVED (not COMMENTED) since the verdict is "Worth merging" with LOW risk and no blocking issues.

This comment was posted by an AI agent (OpenHands).