feat: add automation draft lifecycle, endpoints, and synthetic event payloads

[malhotra5]

Summary

This is PR 2 of a stacked series splitting PR #417 (automation draft lifecycle dispatch) into smaller, reviewable PRs.

Stack:

• PR 1 (feat: add lifecycle_status enum and trigger_source for automation runs #438): lifecycle_status enum + trigger_source on runs → main
• PR 2 (this PR): draft lifecycle, endpoints, synthetic event payloads → openhands/lifecycle-state
• (Future PR 3+ if needed)

What This PR Adds

Builds on the lifecycle_status / trigger_source foundation from PR #438:

Draft lifecycle (draft_router.py, draft_schemas.py)

• Full CRUD for server-backed automation drafts (POST/GET/PATCH/DELETE /v1/drafts)
• Draft validation against the same models used by the real creation endpoints (/v1, /v1/preset/prompt, /v1/preset/plugin)
• Materialization: a dispatchable draft is turned into a DRAFT-state (disabled) Automation
• POST /v1/drafts/{id}/dispatch — test-run a draft as a manual run

Synthetic event payloads (user requirement: option b)

• POST /v1/drafts/{id}/dispatch accepts an optional DraftDispatchRequest body with event_payload
• An authenticated user can supply a synthetic webhook payload for event-triggered draft automations
• This bypasses webhook signature verification — the caller is already authenticated, so the payload is trusted as test input rather than a real delivery
• create_pending_run() now accepts an optional event_payload parameter

Other changes

• models.py: AutomationDraft model with source_automation_id, materialized_automation_id, last_test_run_id foreign keys
• schemas.py: CreateAutomationDraftRequest, UpdateAutomationDraftRequest, AutomationDraftResponse, AutomationDraftListResponse, DraftDispatchRequest
• capabilities_router.py: draft schemas registry, automationDrafts in static features
• app.py: register draft_router
• Migration 024: automation_drafts table

Test plan

• All 1721 tests pass (7 new in test_draft_router.py, including test_dispatch_event_draft_with_synthetic_payload)
• Existing 1714 tests from PR 1 still pass
• New test verifies synthetic payload is stored on the run for event-triggered drafts

---

This PR was created by an AI agent (OpenHands) on behalf of the user.

@malhotra5 can click here to continue refining the PR

[github-actions]

Warning

Your comment is too long (maximum is 65536 characters), so the coverage report was not added. See the job log for how to reduce it.

[github-actions]

📁 PR Artifacts Notice

This PR contains a .pr/ directory with temporary PR-specific documents. The directory will be automatically removed when the PR is approved.

[malhotra5]

QA pass for the requested draft event lifecycle flow.

I exercised the flow through the FastAPI app with authenticated API requests, a SQLite test DB, and an in-memory file store:

1. Configured custom event source qa-draft-events.
2. Created an incomplete event-based prompt draft; it stayed non-dispatchable with materialized_automation_id: null and a prompt validation error.
3. Patched the draft complete; it became dispatchable.
4. Dispatched a first test run using a synthetic event_payload body and no webhook signature headers.
5. Confirmed the first run was selected by the dispatcher and completed in the QA harness.
6. Re-modified the draft prompt, dispatched a second synthetic event test run, and confirmed the same materialized draft automation was reused/updated.
7. Confirmed the second run was selected by the dispatcher and completed in the QA harness.
8. Activated the materialized automation through the normal automation API; final state was lifecycle_status: ACTIVE, enabled: true.

Notes:

• The test dispatch path accepted synthetic event payloads from authenticated requests without requiring webhook signature verification.
• Event-draft validation still requires the custom event source to be configured before the draft is dispatchable.
• Since this container does not have a live sandbox backend, dispatcher selection was exercised and the async sandbox execution task was replaced with a QA completion stub to avoid external sandbox/network calls.
• Detailed evidence was pushed in .pr/qa-draft-event-lifecycle.md and .pr/qa-draft-event-lifecycle.json.

This comment was created by an AI agent (OpenHands) on behalf of the user.

[malhotra5]

Correction/update on QA: the earlier QA evidence used an in-process ASGI client plus a dispatcher completion stub, not a standalone uvicorn backend with external REST calls.

I re-ran the requested flow the stricter way:

• Started the automation backend with uv run uvicorn openhands.automation.app:app --host 127.0.0.1 --port 8123.
• Configured local mode with the existing runtime agent server (AUTOMATION_AGENT_SERVER_URL=$RUNTIME_URL) and a local API key.
• Used curl REST calls against http://127.0.0.1:8123/api/automation/v1.
• Uploaded a raw custom script tarball through POST /uploads.
• Created an incomplete event draft, patched it complete, dispatched two synthetic event test runs, re-modified the draft between runs, and activated the final materialized automation.
• The two runs completed via the normal callback path from the executed script.
• Both completed runs had sandbox_id: null, confirming local execution without a Cloud sandbox.

Pushed evidence in commit 87b7df0:

• .pr/qa-rest-draft-event-lifecycle.md
• .pr/qa-rest-draft-event-lifecycle.json

This comment was created by an AI agent (OpenHands) on behalf of the user.

[all-hands-bot]

🤖 OpenHands is reviewing this PR.

Head commit: 316158b7fe2b1eb4db208d54701f585f879565bb
View the conversation: https://oss-agent-canvas.ngrok.dev/conversations/5d3ce5a7-fe23-4c0f-ac70-fb614cdc03d5

This comment was posted by an AI agent (OpenHands).

[all-hands-bot]

🤖 OpenHands is reviewing this PR.

Head commit: 316158b7fe2b1eb4db208d54701f585f879565bb
View the conversation: https://oss-agent-canvas.ngrok.dev/conversations/35383b4a-7bd9-49a6-b829-37d3addb096e

This comment was posted by an AI agent (OpenHands).

[all-hands-bot]

This review was created by an AI agent (OpenHands) on behalf of the repository maintainers.

Summary

This PR adds a server-backed automation draft lifecycle (CRUD + materialization + dispatch), synthetic event payloads for test dispatching, and restricts the public creation endpoints from setting state=DRAFT directly.

Taste Rating: Good taste -- the design cleanly separates draft state from automation state, reuses existing validation/materialization paths, and correctly prevents DRAFT automations from being triggered by the scheduler or webhook ingest (both filter on enabled=True AND state=ACTIVE).

Key findings

No material bugs, security problems, or significant design flaws found.

Security -- synthetic event_payload bypass (verified safe): The POST /v1/drafts/{id}/dispatch endpoint accepts an optional event_payload that bypasses webhook signature verification. This is explicitly documented and gated behind _require_manage_automations authentication -- the caller is trusted, so the payload is test input, not a real delivery. The synthetic payload flows correctly through create_pending_run -> stored on run.event_payload -> the dispatcher's _build_event_payload merges it into AUTOMATION_EVENT_PAYLOAD. Sound design.

DRAFT state isolation (verified safe): Materialized draft automations are created with enabled=False, state=DRAFT. The scheduler (_fetch_enabled_automations) and webhook dispatch (get_event_automations) both filter on enabled=True AND state=ACTIVE, so draft automations cannot be triggered automatically. The reject_public_draft_state validator prevents direct creation of DRAFT automations through /v1, /v1/preset/prompt, and /v1/preset/plugin. The normalize_automation_state_enabled function correctly handles the DRAFT case (skips the enabled-state conflict check, sets enabled=False).

Materialization reuse (verified safe): _get_live_materialized_draft_automation correctly returns None if the linked automation was activated (state != DRAFT), deleted, or is in a different org -- causing a new DRAFT automation to be created rather than overwriting an active one. This is well-tested in test_dispatch_draft_does_not_overwrite_linked_non_draft_automation.

Migration (verified compatible): Migration 024 uses generic SQLAlchemy types (sa.Uuid, sa.JSON, sa.Boolean, sa.String, sa.DateTime) and has the _is_sqlite() guard for the PostgreSQL-only COMMENT. Cross-database compatible per the AGENTS.md guidance.

Note (non-blocking): The .pr/ QA artifacts should be removed before merge. The PR Artifacts workflow will handle this for same-repo PRs, but it's worth confirming.

Test coverage

7 new tests in test_draft_router.py exercise the real code paths (no mock-only tests): incomplete draft validation, endpoint schema enforcement, tarball upload validation, dispatch with synthetic payload, re-dispatch with materialization reuse, incomplete-edit edge case, non-draft automation protection, and soft-delete cleanup. The synthetic payload test (test_dispatch_event_draft_with_synthetic_payload) verifies the payload is stored on the run.

[RISK ASSESSMENT]

• [Overall PR] Risk Assessment: LOW
  The change adds a new API surface (drafts) without modifying existing automation dispatch behavior. DRAFT-state automations are correctly isolated from automatic triggers. The synthetic payload bypass is authenticated and documented. The migration is cross-database compatible. Existing tests (1714 from PR 1) still pass.

VERDICT: Worth merging

KEY INSIGHT: The draft lifecycle cleanly separates "editing state" (drafts) from "runtime state" (automations) by materializing drafts into disabled DRAFT-state rows only on explicit dispatch, with proper guards preventing both automatic triggering and direct DRAFT creation through public endpoints.

Automated review used the wrong decision (APPROVED instead of COMMENT) and is dismissed. Findings are reposted as a comment.

[all-hands-bot]

👋 This PR needs a couple of things fixed before OpenHands can review it:

• the PR description's HUMAN: section needs at least 20 characters describing what you tested, not just the template placeholder

Push an update once this is addressed and this check re-runs automatically.

This is an automated check - no AI was used to generate this comment.

[all-hands-bot]

This review was created by an AI agent (OpenHands) on behalf of the repository maintainers.

Review Summary

🟢 Good taste — The draft lifecycle is well-designed: DRAFT state is properly fenced off from the public creation endpoints, materialization correctly reuses/overwrites draft automations, and the synthetic event payload path is a sensible authenticated-user test mechanism.

Key design validations

• DRAFT state isolation: reject_public_draft_state on all public create/update endpoints prevents direct DRAFT creation, while the normalize_automation_state_enabled special case allows draft bodies to carry state=DRAFT without spurious errors. The Pydantic validation ordering (model_validator before → field_validator before) ensures rejection works correctly.

• Materialization lifecycle: _get_live_materialized_draft_automation correctly returns None when the linked automation has been activated (state ≠ DRAFT), causing a fresh automation to be created rather than overwriting a live one. Tested in test_dispatch_draft_does_not_overwrite_linked_non_draft_automation.

• Dispatcher integration: Manual runs (trigger_source=manual) are picked up by the dispatcher regardless of automation enabled/state, so DRAFT-state materialized automations dispatch correctly. The synthetic event_payload flows through the same _build_event_payload path as real webhook events.

• Delete cascade: delete_draft soft-deletes the materialized automation, skips pending runs (including manual), and marks git sync dirty. Does not touch linked non-DRAFT automations.

• Migration: Cross-database compatible — uses generic SQLAlchemy types, guards COMMENT ON TABLE with _is_sqlite(). Chain 023 → 024 is correct.

• Org-scoped access: All draft queries filter by org_id and deleted_at IS NULL. manage_automations permission for mutations, view_automations for reads.

Non-blocking observations

1. Private function coupling (draft_router.py lines 44-52, 37-40): The draft router imports 6 underscore-prefixed functions from preset_router.py and 3 from capabilities_router.py. This creates tight coupling between modules, but is pragmatic given the shared tarball-generation and validation logic. Consider promoting the most reused helpers to a shared utility module in a future refactor.

2. No trigger-type guard on event_payload (draft_router.py line 610): DraftDispatchRequest.event_payload is accepted for any draft type, including cron-triggered ones. For a cron draft, the synthetic payload would still be injected into AUTOMATION_EVENT_PAYLOAD by the dispatcher's _build_event_payload. This is harmless but could surprise a user testing a cron automation. Optionally, consider rejecting event_payload when the materialized automation's trigger type is not event.

Security assessment

The synthetic event payload bypasses webhook signature verification by design — the caller is authenticated with manage_automations permission, so the payload is trusted as test input. This is the correct approach for a test-dispatch feature. The payload is stored on the run and injected into the sandbox environment the same way real webhook payloads are, so the execution path is identical.

.pr/ directory

The .pr/ QA artifacts (JSON evidence + markdown reports) will be auto-removed by the PR Artifacts workflow on approval. No action needed.

[RISK ASSESSMENT]

• Overall PR Risk: 🟢 LOW
  • New table with proper migration, no changes to existing table schemas
  • DRAFT state is additive — existing endpoints reject it, so no regression risk
  • All new endpoints are org-scoped with permission checks
  • Comprehensive test coverage (12 tests covering CRUD, dispatch, materialization, delete, synthetic payloads, edge cases)
  • No dependency changes

VERDICT: ✅ Worth merging

KEY INSIGHT: The draft lifecycle cleanly separates incomplete setup state from dispatchable automations via DRAFT state + materialization, with the synthetic event payload being a well-scoped authenticated-user test mechanism.

---

Improve this review? If any feedback above seems incorrect or irrelevant to this repository, you can teach the reviewer to do better:

1. Add a .agents/skills/custom-codereview-guide.md file to your branch (or edit it if one already exists) with the /codereview trigger and the context the reviewer is missing (e.g., "Security concerns about X do not apply here because Y"). See the customization docs for the required frontmatter format.
2. Re-request a review - the reviewer reads guidelines from the PR branch, so your changes take effect immediately.
3. When your PR is merged, the guideline file goes through normal code review by repository maintainers.

Resolve with AI? Install the iterate skill in your agent and run /iterate to automatically drive this PR through CI, review, and QA until it's merge-ready.

Was this review helpful? React with 👍 or 👎 to give feedback.

[all-hands-bot]

Non-blocking: event_payload is accepted for any draft type, including cron-triggered ones. For a cron draft, the synthetic payload would still be stored on the run and injected into AUTOMATION_EVENT_PAYLOAD by the dispatcher's _build_event_payload. This is harmless but could surprise a user testing a non-event automation. Consider optionally rejecting event_payload when the materialized automation's trigger type is not event, to give clearer feedback.

[all-hands-bot]

Non-blocking: Importing 6 private (_-prefixed) functions from preset_router creates tight coupling between these modules. The shared tarball-generation and validation helpers (_generate_tarball, _generate_plugin_tarball, _bytes_to_async_iter, _safe_truncate, etc.) are good candidates for promotion to a shared utility module in a future refactor, so both routers depend on a public interface rather than each other's internals.