Skip to content

feat(odin): Execute incoming actions from Odin#538

Merged
vikramlc-cognite merged 18 commits into
masterfrom
EDGE-183-execute-incoming-actions
Jun 11, 2026
Merged

feat(odin): Execute incoming actions from Odin#538
vikramlc-cognite merged 18 commits into
masterfrom
EDGE-183-execute-incoming-actions

Conversation

@vikramlc-cognite

@vikramlc-cognite vikramlc-cognite commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

PR Short Summary

Wires the Extractor as the action dispatcher for CheckinWorker and implements the full action routing pipeline: _handle_actions fans out each incoming action to a dedicated daemon thread, _dispatch_single_action routes by matching the action name against registered available actions, and the three handler methods (_handle_start_task_action, _handle_stop_task_action, _handle_custom_action) execute the action and report lifecycle status updates back through the checkin worker.

Type of change

  • New feature (non-breaking change that adds functionality)

What changed

cognite/extractorutils/unstable/core/base.py

  • Added imports: Action, ActionStatus, ActionUpdate from _dto; ActionContext from actions
  • _handle_actions(actions) — spawns one daemon thread per action named Action-{external_id} so a slow action never blocks the next
  • _dispatch_single_action(action) — builds O(1) sets of known start/stop/custom action names at dispatch time and routes accordingly; any unrecognised name immediately reports failed with a descriptive message (no exception raised)
  • _handle_start_task_action(action) — checks task existence, holds _running_task_tokens_lock during the already-running guard, then reports running
    → _run_task_with_token → succeeded/failed
  • _handle_stop_task_action(action) — snapshots the child token under lock, cancels it, reports canceled; reports failed if the task is not currently running
  • _handle_custom_action(action) — builds ActionContext with call_metadata, invokes custom.target(ctx), reports running → succeeded/failed
  • start() — calls self._checkin_worker.set_action_dispatcher(self._handle_actions) before the checkin thread is spawned so no pending actions can arrive without a dispatcher

tests/test_unstable/test_action_dispatch.py — new file, 17 tests covering all acceptance criteria

Why it changed

Odin dispatches pending actions in the checkin response. Without this wiring, actions are received by CheckinWorker but never executed. This completes the round-trip: the extractor announces capabilities at startup (Task 5) and can now act on them.

What to focus on during review

  • Dispatch key design: type is inferred by matching action_name against the three known sets (start/stop/custom) rather than a type field on Action (which the DTO does not carry). Custom actions named "Start X" where X is not a registered ScheduledTask are legal and route correctly to the custom handler.
  • _handle_start_task_action concurrency: the already-running check holds _running_task_tokens_lock for the read. There is a narrow TOCTOU window between the lock release and _run_task_with_token acquiring its own lock; this matches the existing scheduler behaviour and is intentional - _run_task_with_token's identity-check finally already handles concurrent overwrites correctly.
  • start() ordering: dispatcher is registered before the checkin thread starts — verified by test_dispatcher_registered_before_checkin_thread_starts.

Test evidence

tests/test_unstable/test_action_dispatch.py 17 passed
tests/test_unstable/test_action_registration.py 15 passed
tests/test_unstable/test_actions.py 10 passed
ruff check — All checks passed

Risks and unknowns

  • The "Start {t.name}" / "Stop {t.name}" name convention is shared between _get_startup_request (registration) and _dispatch_single_action (routing). A single source-of-truth constant would eliminate future drift if the convention changes.
  • _run_task_with_token's inner run_task wrapper catches all Exceptions internally, so _handle_start_task_action will almost always report succeeded even when the task fails internally (the failure is reported via the error pipeline). This is consistent with the framework's design but means the action status alone does not indicate task success.

Rollout and rollback

  • Purely additive. Extractors that do not register scheduled tasks or custom actions receive no _handle_actions calls and behave identically to before. The dispatcher is only invoked when the server returns pending_actions.

Checklist

  • Self-reviewed the diff
  • Tests added or updated
  • Docs updated (N/A — internal framework)
  • No secrets, credentials, or PII committed
  • Breaking changes called out above and communicated to affected teams (none)

@vikramlc-cognite vikramlc-cognite self-assigned this Jun 4, 2026
@vikramlc-cognite

Copy link
Copy Markdown
Contributor Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces custom actions and task control actions (Start/Stop) to the extractor framework, enabling external invocation of tasks and custom targets. It includes registration logic, startup reporting of available actions, multi-threaded action dispatching, and comprehensive unit tests. The review feedback identifies a TOCTOU race condition when starting tasks via actions, recommending atomic check-and-set token registration under a lock. Additionally, it suggests validating that newly added scheduled tasks do not generate action names that conflict with already registered custom actions.

Comment thread cognite/extractorutils/unstable/core/base.py
Comment thread cognite/extractorutils/unstable/core/base.py Outdated
Comment thread cognite/extractorutils/unstable/core/base.py Outdated

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request implements custom action registration and dispatching mechanisms for the Extractor class, allowing users to register custom actions and control scheduled tasks (Start/Stop) externally. It also adds comprehensive unit tests to verify action dispatching and registration lifecycles. The review feedback highlights a critical TOCTOU (Time-of-Check to Time-of-Use) race condition between checking if a task is running and registering its token. The reviewer suggested resolving this by creating and registering the child token atomically under a lock, and provided corresponding updates for the test suite to accommodate the signature changes.

Comment thread cognite/extractorutils/unstable/core/base.py Outdated
Comment thread cognite/extractorutils/unstable/core/base.py
Comment thread tests/test_unstable/test_action_dispatch.py Outdated
@codecov

codecov Bot commented Jun 5, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 87.69231% with 8 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.73%. Comparing base (71ed9f5) to head (77cc9ab).
⚠️ Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
cognite/extractorutils/unstable/core/base.py 87.69% 8 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #538      +/-   ##
==========================================
+ Coverage   82.58%   82.73%   +0.14%     
==========================================
  Files          44       44              
  Lines        4370     4430      +60     
==========================================
+ Hits         3609     3665      +56     
- Misses        761      765       +4     
Files with missing lines Coverage Δ
cognite/extractorutils/unstable/core/base.py 84.29% <87.69%> (+1.35%) ⬆️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@vikramlc-cognite
vikramlc-cognite marked this pull request as ready for review June 5, 2026 05:57
@vikramlc-cognite
vikramlc-cognite requested a review from a team as a code owner June 5, 2026 05:57
@vikramlc-cognite
vikramlc-cognite marked this pull request as draft June 5, 2026 05:57
@vikramlc-cognite
vikramlc-cognite marked this pull request as ready for review June 5, 2026 10:47
@thorkildcognite thorkildcognite added the risk-review-exemption-big-pr Approved exemption for large PR (>500 lines) label Jun 5, 2026
Yaseen-A-Khan

This comment was marked as duplicate.

@Yaseen-A-Khan

Copy link
Copy Markdown
Contributor

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces support for registering and dispatching custom actions and scheduled task control actions (Start/Stop) in the extractor framework, along with comprehensive tests. Feedback highlights a critical concurrency issue where a scheduled task triggered by the scheduler can run concurrently with the same task triggered by an action, potentially overwriting the running task token. Additionally, when a task started via an action is cancelled, the action status incorrectly reports as succeeded instead of cancelled.

Comment thread cognite/extractorutils/unstable/core/base.py
Comment thread cognite/extractorutils/unstable/core/base.py Outdated
@vikramlc-cognite
vikramlc-cognite changed the base branch from master to EDGE-181-integrate-available-actions June 8, 2026 02:43
Comment thread cognite/extractorutils/unstable/core/base.py
Toshad
Toshad previously approved these changes Jun 8, 2026

@Yaseen-A-Khan Yaseen-A-Khan left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have some comments regarding the changes

Comment thread tests/test_unstable/test_action_dispatch.py
Comment thread cognite/extractorutils/unstable/core/base.py
Comment thread cognite/extractorutils/unstable/core/base.py
Comment thread cognite/extractorutils/unstable/core/base.py
@andersfylling andersfylling removed the waiting-for-risk-review Waiting for a member of the risk review team to take an action label Jun 8, 2026
@andersfylling andersfylling self-assigned this Jun 8, 2026
@andersfylling

Copy link
Copy Markdown

once team has approved, ping me on slack and I'll do the risk review ASAP =)
Removing waiting on risk review, since a team approval is required.

Yaseen-A-Khan
Yaseen-A-Khan previously approved these changes Jun 9, 2026
@vikramlc-cognite vikramlc-cognite added the waiting-for-risk-review Waiting for a member of the risk review team to take an action label Jun 9, 2026
@andersfylling andersfylling added risk-review-ongoing Risk review is in progress and removed waiting-for-risk-review Waiting for a member of the risk review team to take an action labels Jun 9, 2026
Comment thread cognite/extractorutils/unstable/core/base.py
Comment thread cognite/extractorutils/unstable/core/base.py
@andersfylling andersfylling added the waiting-for-team Waiting for the submitter or reviewer of the PR to take an action label Jun 9, 2026
@vikramlc-cognite vikramlc-cognite added waiting-for-risk-review Waiting for a member of the risk review team to take an action and removed waiting-for-team Waiting for the submitter or reviewer of the PR to take an action labels Jun 9, 2026
andersfylling
andersfylling previously approved these changes Jun 9, 2026

@andersfylling andersfylling left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🦄

@andersfylling andersfylling added waiting-for-team Waiting for the submitter or reviewer of the PR to take an action and removed waiting-for-risk-review Waiting for a member of the risk review team to take an action labels Jun 9, 2026
Base automatically changed from EDGE-181-integrate-available-actions to master June 9, 2026 16:57
@vikramlc-cognite
vikramlc-cognite dismissed stale reviews from andersfylling, Yaseen-A-Khan, and Toshad June 9, 2026 16:57

The base branch was changed.

@andersfylling andersfylling left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🦄

@vikramlc-cognite
vikramlc-cognite merged commit 600b62b into master Jun 11, 2026
7 checks passed
@vikramlc-cognite
vikramlc-cognite deleted the EDGE-183-execute-incoming-actions branch June 11, 2026 04:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk-review-exemption-big-pr Approved exemption for large PR (>500 lines) risk-review-ongoing Risk review is in progress waiting-for-team Waiting for the submitter or reviewer of the PR to take an action

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants