revert deprecated state validation map

[raylrui]

Resolve #163.

Temporarily only allow SUCCEEDED to transition to DEPRECATED.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[reisingerf]

Looks good to me, thought it leaves a gap in current functionality which we might want to address.
There are a few options here.

• go ahead as is, might get messy and will require manual clean-up bypassing transition rules
• allow transition, but restrict access to it (UI permission limitations)
• create better transition support (see below)

@alexiswl @raylrui happy for feedback!

Currently runs "stuck" in DRAFTs and RUNNING states, which apparently seems happen quite regularly with manual submissions, can be deprecated (which helps with declutter and readability). If we disable this transition, we have to deal with the fall-out.

An option would be to introduce CANCELLED as a new terminal state.
Note: as we can't guarantee that it would actually cancel/abort any ongoing processes, this would - on the WFM side - simply ignore any state updates thereafter. Other services, like the Pipeline Managers, would be free to do their best to abort related processes by any means necessary to avoid unnecessary cost (from the OrcaBus/WFM side those runs would already be terminated anyway).

That would give the following transition rules.
(note: using RUNNING as a catch-all for all transitive states after READY, things like SUBMITTED, RUNNABLE, STARTING, RUNNING, ...)

Normal/regular transitions:

• DRAFT -> READY
• READY -> RUNNING
• RUNNING -> SUCCEEDED, FAILED
• SUCCEEDED -> DEPRECATED
• FAILED -> RESOLVED

Additional new transitions:

• DRAFT, READY, RUNNING -> CANCELLED

Notes on (automatic) data clean-up:
RESOLVED, CANCELLED and DEPRECATED could then result in automatic data deletion with a subtle difference:

• RESOLVED / DEPRECATED states are only assigned to terminal states and after an investigation and approval process
• CANCELLED states are assigned to transitive states ("ongoing" processes) and there is no additional verification / approval step

Again, there would be no two-step approach, no review or "warning" when CANCELLED data is removed. However, I think that's OK as I assume processes would only be cancelled if the reason is already known and there is no interest in any potentially generated data.

Also, note: the "cancel" action would have to remain a human intervention. Any automatic abortion should probably be handled as failure (otherwise we'd loose the human control over data deletion and cleanup).

[raylrui]

An option would be to introduce CANCELLED as a new terminal state.
Note: as we can't guarantee that it would actually cancel/abort any ongoing processes, this would - on the WFM side - simply ignore any state updates thereafter. Other services, like the Pipeline Managers, would be free to do their best to abort related processes by any means necessary to avoid unnecessary cost (from the OrcaBus/WFM side those runs would already be terminated anyway).

RESOLVED, CANCELLED and DEPRECATED could then result in automatic data deletion with a subtle difference:

• RESOLVED / DEPRECATED states are only assigned to terminal states and after an investigation and approval process
• CANCELLED states are assigned to transitive states ("ongoing" processes) and there is no additional verification / approval step

yes @reisingerf , I agree with introducing CANCELLED as a distinct terminal state. It gives us a clean way to terminate stuck DRAFT/READY/transitive runs without overloading DEPRECATED.

There are a few implementation details we should cover:

1. CANCELLED should be human-only; however, we may receive cancellation status from an execution service, define here, this state should be modified or represented as FAILED.

2. The manual action ( RESOLVED / DEPRECATED CANCELLED state transition) should emit a WorkflowRunStateChange event, allowing Pipeline Managers to make a best-effort attempt to abort execution. This will be done in my next PR.

[reisingerf]

Yes, that sounds good!

Also agree with 1. and 2.

[alexiswl]

Before we merge this, can we ensure that CANCELLED is available for DRAFT runs? Trying to get people into the habit of removing drafts that will not go into a running state

[raylrui]

Before we merge this, can we ensure that CANCELLED is available for DRAFT runs? Trying to get people into the habit of removing drafts that will not go into a running state

This PR will merge after the #173 for emit wrsc events for state transition api.

yes, @alexiswl , CANCELLED will be implemented as a distinct, human-only terminal state and release it alongside the DEPRECATED restriction:

• DEPRECATED: only from SUCCEEDED. which means a previously successful result is no longer valid or needed.
• RESOLVED: only from FAILED.
• CANCELLED: from any non-terminal/transitive state, including DRAFT, READY, SUBMITTED, RUNNABLE, STARTING and RUNNING ....

[raylrui]

Merged. For verified permission, we will implement that in our next PR.
Refer issue of Separate Workflow Run Custom State API into Dedicated Endpoints: #175