feat(automations): add test-run gate to creation

[XiaoFeiCode]

HUMAN:

I manually verified both the successful and failed test-run flows. When a test run fails, the enable action is no longer shown, the error output remains visible, and the test can be retried. After a successful test run, the automation can be enabled. The overall workflow behaves as expected.

AGENT:

The automation setup flow was exercised end to end against the stateful Mock Local backend: a bundle was created disabled, manually dispatched, polled through completion, linked to its spawned conversation, and enabled only after the successful run. The failure state was also reproduced with preserved service output and no enable action.

---

Why

Creating an automation currently redirects users as soon as the record exists, even though integrations, runtime execution, or the automation itself may still fail. The first scheduled or event-triggered execution can therefore run untested automation code.

Summary

• create direct prompt and bundle automations disabled, then keep setup in a dedicated test-run step
• dispatch and poll a controlled run, showing status, phase, conversation details, and actionable failure output
• expose enable/finalize only after a successful run, while allowing repeat test runs
• make automation MSW handlers stateful enough to exercise bundle creation and run polling in Mock Local
• add complete translations and regression coverage for successful and failed flows

Issue Number

Fixes #16567

How to Test

Automated checks run locally:

• npm test -- --run __tests__/api/automation-handlers.test.ts __tests__/manifests/manifest-actions.test.ts __tests__/components/manifest/manifest-setup-dialog.test.tsx — 36/36 tests passed
• npm run lint — passed with 0 errors; 3 unrelated pre-existing warnings remain
• npm run check-translation-completeness — passed
• npm run build — passed
• git diff --check — passed

Manual reproduction:

1. Run npm run dev:mock and select the Mock Local backend.
2. Open Automate → Templates → Daily news digest.
3. Complete the form and confirm creation. Verify setup stays open and says the automation is saved but disabled.
4. Run the controlled test. Verify live status/phase updates and the spawned-conversation link appear.
5. Verify a failed run preserves its error and offers no enable action.
6. Verify a successful run offers both rerun and Enable and finish; enabling navigates to an active automation detail page.

Video/Screenshots

Successful test run — rerun or enable/finalize:

Failed test run — actionable error preserved and no enable action:

Type

• Bug fix
• Feature
• Refactor
• Breaking change
• Docs / chore

Notes

Raw bundle automations need the companion backend change in OpenHands/automation#387 so the record can be created disabled atomically. Prompt/plugin creation already supports the same field.

[github-actions]

❌ Mock-LLM E2E Tests

63/66 passed · 1 failed · 2 skipped

Commit: 4de67213 · Workflow run · Test artifacts

Details

Status	Test	Duration
✅	automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 1: setup LLM profile and register automation trajectory	6.5s
❌	automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 2: create automation and dispatch run via the UI	39.5s
⏭️	automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 3: verify automation and run on the automations page	0ms
✅	automations/mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › automation card sends the correct slash command to a conversation	16.0s
✅	automations/mock-llm-preset-automation.spec.ts › preset automation → slash command conversation › direct slash command from home page triggers skill activation	13.6s
✅	backends/mock-llm-auth-modes.spec.ts › auth mode: fresh install with runtime-injected key › reaches the onboarding modal without pre-seeded localStorage	1.4s
✅	backends/mock-llm-auth-modes.spec.ts › auth mode: non-public key rotation › recovers when localStorage has a stale session API key	5.4s
✅	backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › shows first-run onboarding before the auth screen when no key is configured	1.3s
✅	backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › rejects an incorrect key with an inline error	1.5s
✅	backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › allows access after pasting the correct key	1.6s
✅	backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › skips auth screen for returning user with valid stored key	1.2s
✅	backends/mock-llm-auth-modes.spec.ts › auth mode: public gate › re-prompts when the server rotates its key (stale localStorage)	1.3s
✅	backends/mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → backend-only › frontend-only connects to a separate backend-only instance	17.4s
✅	backends/mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → multiple backends › connects to two separate backends and switches between them	21.6s
✅	backends/mock-llm-cross-connect.spec.ts › cross-connect: sidebar links pin their backend › cmd-clicking a sidebar conversation opens it on the owning backend	31.2s
✅	backends/mock-llm-partial-stack.spec.ts › partial stack: --frontend-only › serves the frontend but returns 503 for backend routes	7.4s
✅	backends/mock-llm-partial-stack.spec.ts › partial stack: --backend-only › serves backend APIs but returns 503 for the frontend root	15.1s
✅	backends/mock-llm-partial-stack.spec.ts › partial stack: port conflict › fails with a clear error when the ingress port is occupied	111ms
✅	backends/mock-llm-partial-stack.spec.ts › partial stack: port conflict › starts successfully on a free port after a conflict	6.0s
✅	conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 1: create an LLM profile pointing at the mock LLM server	6.3s
✅	conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 2: activate the mock-llm profile and verify settings API	10.7s
✅	conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 3: run a conversation with the mock LLM	7.4s
✅	conversations/mock-llm-conversation.spec.ts › mock-LLM agent-server conversation › step 4: resume conversation from sidebar after navigating away	5.9s
✅	conversations/mock-llm-image-upload.spec.ts › mock-LLM image upload › attaching an image embeds it as base64 in the LLM completion call	13.9s
✅	files/mock-llm-files-and-git.spec.ts › files tab, conversation overview git, and browser tab › step 1: ensure mock LLM profile is configured	6.4s
✅	files/mock-llm-files-and-git.spec.ts › files tab, conversation overview git, and browser tab › step 2: start conversation and attach workspace metadata	12.5s
✅	files/mock-llm-files-and-git.spec.ts › files tab, conversation overview git, and browser tab › step 3: conversation overview shows workspace and git identity	26.1s
✅	files/mock-llm-files-and-git.spec.ts › files tab, conversation overview git, and browser tab › step 4: commits tab opens for attached workspace	5.9s
✅	files/mock-llm-files-and-git.spec.ts › files tab, conversation overview git, and browser tab › step 5: browser tab shows empty state	6.4s
✅	files/mock-llm-files-and-git.spec.ts › files tab, conversation overview git, and browser tab › step 6: files tab defaults to file-tree view without attached workspace	7.9s
✅	home/mock-llm-folder-workspace.spec.ts › mock-LLM folder browser → workspace → conversation › step 1: browse to a folder, add it as a workspace, and launch a conversation with the correct working_dir	8.5s
✅	mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 1: GitHub card is visible on the MCP marketplace page	5.5s
✅	mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 2: clicking GitHub add control opens the install modal with correct fields	5.7s
✅	mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 3: full install flow — fill PAT, submit, verify installed	11.9s
✅	mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › step 4: installed GitHub server can be deleted	5.9s
✅	mcp/mock-llm-mcp-github.spec.ts › MCP GitHub server install flow › regression: sibling create, edit, and delete preserve GitHub credentials with one MCP request each	12.8s
✅	mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: invalid Slack credentials are blocked with a credential-check error	5.8s
✅	mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: a valid token missing only a scope still installs (missing_scope is not a credential failure)	5.9s
✅	mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › install: an older agent server that omits tool_result still installs (compat)	5.8s
✅	mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › edit: Test Connection verifies the stored credentials and surfaces a credential failure	5.7s
✅	mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › edit: Test Connection reports success for valid stored credentials	5.7s
✅	mcp/mock-llm-mcp-slack-credentials.spec.ts › MCP Test Connection credential verification (Slack) › custom (non-catalog) server: Test Connection attaches no verification probe	5.8s
✅	onboarding/mock-llm-onboarding-happy-path.spec.ts › onboarding happy path › completes the full onboarding flow and launches a conversation	5.1s
✅	onboarding/mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › keeps the modal open on backdrop click and Escape	1.6s
✅	onboarding/mock-llm-onboarding-regressions.spec.ts › onboarding recent regressions › defaults the LLM setup step to OpenHands GLM-5.2	1.5s
✅	regressions/mock-llm-ui-regressions.spec.ts › UI regressions › scopes standalone styles to the agent-server-ui shell	1.3s
✅	regressions/mock-llm-ui-regressions.spec.ts › UI regressions › renders critic results on agent messages and finish actions	1.6s
✅	regressions/mock-llm-ui-regressions.spec.ts › UI regressions › loads older events when scrolling up	2.0s
✅	regressions/mock-llm-ui-regressions.spec.ts › UI regressions › selected workspace persists after navigating away and returning	2.7s
✅	regressions/mock-llm-ui-regressions.spec.ts › UI regressions › cleared sessionStorage yields empty workspace selection	1.4s
✅	settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 1: configure ACP agent via Settings → Agent UI	13.3s
✅	settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 2: reload and verify ACP settings are persisted in UI	5.7s
✅	settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 3: start ACP conversation and verify agent reply	8.7s
✅	settings/mock-llm-acp-agent.spec.ts › mock-LLM ACP agent conversation › step 4: resume ACP conversation from sidebar after navigating away	5.9s
✅	settings/mock-llm-acp-auth-banner.spec.ts › mock-LLM ACP credentials-configured banner (#1244) › Claude credential in the store surfaces a 'configured' banner, never 'signed in'	6.7s
✅	settings/mock-llm-cloud-providers-pagination.spec.ts › cloud LLM provider-picker pagination › surfaces providers past page 1 (e.g. xai) in the picker on a cloud backend	6.3s
✅	settings/mock-llm-cloud-providers-pagination.spec.ts › cloud LLM provider-picker pagination › does not regress the local path: 150+ providers render in the picker	5.9s
✅	settings/mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 1: configure LLM, create switch-target profile, register trajectory	7.0s
✅	settings/mock-llm-model-switch.spec.ts › mock-LLM /model slash command › step 2: start conversation, switch profile via /model, verify switch	7.7s
✅	settings/mock-llm-profile-management.spec.ts › active profile deletion + reconciliation › active profile is deletable and reconciliation activates another profile	8.2s
✅	settings/mock-llm-profile-management.spec.ts › same-model profile identity › chat header shows the correct profile when two profiles share the same model	15.6s
✅	settings/mock-llm-profile-management.spec.ts › OpenHands provider hidden base_url preservation › re-saving an OpenHands profile from Basic view preserves hidden base_url	7.5s
✅	settings/mock-llm-provider-connection-selector.spec.ts › provider connection selector › selects a supported provider in Add provider	8.1s
✅	skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › project skill in workspace/.agents/skills/ triggers on matching keyword	13.8s
⏭️	skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › user skill in ~/.openhands/skills/ triggers on matching keyword	0ms
✅	skills/mock-llm-skills.spec.ts › skill loading: project, user, and deletion › deleting a user skill removes it from subsequent conversations	0ms

🔍 Failure details (1)

❌ automations/mock-llm-automation.spec.ts › mock-LLM automation lifecycle › step 2: create automation and dispatch run via the UI

Error: Created automation ID was not reported after 30000ms

_{Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses)}

[all-hands-bot]

🚦 CI is currently failing on this PR's latest commit.

Please fix the failing checks before OpenHands reviews it - this is re-checked automatically once you push a new commit. (A maintainer can also request @all-hands-bot as a reviewer to have it reviewed regardless of CI status.)

This is an automated check - no AI was used to generate this comment.

[all-hands-bot]

🚦 CI is currently failing on this PR's latest commit.

Please fix the failing checks before OpenHands reviews it - this is re-checked automatically once you push a new commit. (A maintainer can also request @all-hands-bot as a reviewer to have it reviewed regardless of CI status.)

This is an automated check - no AI was used to generate this comment.