Skip to content

feat: adds online providers warm up before starting discovery#3235

Open
stalniy wants to merge 1 commit into
mainfrom
feat/provider-inventory-warm-up
Open

feat: adds online providers warm up before starting discovery#3235
stalniy wants to merge 1 commit into
mainfrom
feat/provider-inventory-warm-up

Conversation

@stalniy
Copy link
Copy Markdown
Contributor

@stalniy stalniy commented May 28, 2026

Why

ref CON-405

What

Summary by CodeRabbit

  • New Features
    • App startup now runs a warm-up phase before starting, restoring lifecycle streams for previously online providers via batched, paginated streaming.
  • Tests
    • Added extensive tests for online-provider streaming, pagination behavior, and the scheduler's warm-up behavior (including error and empty-stream cases).

Review Change Stack

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 28, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 40c483f7-8424-4344-9a66-480281d642d5

📥 Commits

Reviewing files that changed from the base of the PR and between fd866bd and 892522e.

📒 Files selected for processing (6)
  • apps/provider-inventory/src/providers-sync-app.ts
  • apps/provider-inventory/src/repositories/provider-inventory/provider-inventory.repository.integration.ts
  • apps/provider-inventory/src/repositories/provider-inventory/provider-inventory.repository.spec.ts
  • apps/provider-inventory/src/repositories/provider-inventory/provider-inventory.repository.ts
  • apps/provider-inventory/src/services/discovery-scheduler/discovery-scheduler.service.spec.ts
  • apps/provider-inventory/src/services/discovery-scheduler/discovery-scheduler.service.ts
💤 Files with no reviewable changes (1)
  • apps/provider-inventory/src/repositories/provider-inventory/provider-inventory.repository.spec.ts
🚧 Files skipped from review as they are similar to previous changes (4)
  • apps/provider-inventory/src/providers-sync-app.ts
  • apps/provider-inventory/src/repositories/provider-inventory/provider-inventory.repository.ts
  • apps/provider-inventory/src/services/discovery-scheduler/discovery-scheduler.service.ts
  • apps/provider-inventory/src/services/discovery-scheduler/discovery-scheduler.service.spec.ts

📝 Walkthrough

Walkthrough

The PR adds online-provider warm-up initialization to the provider discovery service. The repository gains a keyset-paginated streaming method for online providers, the scheduler adds a warmUp() lifecycle setup phase before normal discovery, and bootstrap now awaits that warm-up before starting the scheduler.

Changes

Online Provider Warm-up Initialization

Layer / File(s) Summary
Repository streaming contract
apps/provider-inventory/src/repositories/provider-inventory/provider-inventory.repository.ts
Drizzle imports updated to include InferSelectModel. Exports new ProviderInventory type and DEFAULT_ONLINE_STREAM_BATCH_SIZE constant.
Repository streaming implementation & tests
apps/provider-inventory/src/repositories/provider-inventory/provider-inventory.repository.ts, apps/provider-inventory/src/repositories/provider-inventory/provider-inventory.repository.integration.ts, apps/provider-inventory/src/repositories/provider-inventory/provider-inventory.repository.spec.ts
streamOnlineProviders async generator implements keyset pagination filtering by isOnline = true, ordering by owner, and yielding {owner, hostUri} batches. Adds integration and unit tests validating single-page, multi-page pagination, empty-stream termination, upsert/update behaviors, and test helpers/fixtures supporting paginated selects.
Scheduler warmUp method and tests
apps/provider-inventory/src/services/discovery-scheduler/discovery-scheduler.service.ts, apps/provider-inventory/src/services/discovery-scheduler/discovery-scheduler.service.spec.ts
Service now depends on ProviderInventoryRepository (replacing writer). Adds warmUp(signal?) that iterates streamOnlineProviders() and calls lifecycle.start for each owner with empty attribute blobs; updates upsert to use repository API. Tests cover dispatch, empty skip, abort forwarding, error propagation, and non-blocking behavior.
Bootstrap warmUp invocation
apps/provider-inventory/src/providers-sync-app.ts
Bootstrap now awaits scheduler.warmUp() before calling scheduler.start().

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

  • akash-network/console#3174: Refactors DiscoverySchedulerService to stream from the chain-provider poller; related to scheduler changes and startup sequencing.
  • akash-network/console#3164: Refactors lifecycle stream invocation patterns that the new warmUp method interacts with.
  • akash-network/console#3197: Adjusts bootstrap startup ordering for DiscoverySchedulerService similar to this PR.

Suggested labels

size: M

Suggested reviewers

  • baktun14
  • ygrishajev
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/provider-inventory-warm-up

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 ESLint

If the error stems from missing dependencies, add them to the package.json file. For unrecoverable errors (e.g., due to private dependencies), disable the tool in the CodeRabbit configuration.

ESLint install failed: one or more packages not found in the registry.


Comment @coderabbitai help to get the list of available commands and usage tips.

@codecov
Copy link
Copy Markdown

codecov Bot commented May 28, 2026

Codecov Report

❌ Patch coverage is 96.00000% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 66.06%. Comparing base (5dcff08) to head (892522e).
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
apps/provider-inventory/src/providers-sync-app.ts 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3235      +/-   ##
==========================================
- Coverage   66.69%   66.06%   -0.64%     
==========================================
  Files        1065     1024      -41     
  Lines       26123    25110    -1013     
  Branches     6291     6127     -164     
==========================================
- Hits        17423    16588     -835     
+ Misses       7602     7437     -165     
+ Partials     1098     1085      -13     
Flag Coverage Δ *Carryforward flag
api 84.67% <ø> (ø) Carriedforward from 5dcff08
deploy-web 50.54% <ø> (ø) Carriedforward from 5dcff08
log-collector ?
notifications 91.06% <ø> (ø) Carriedforward from 5dcff08
provider-console 81.38% <ø> (ø) Carriedforward from 5dcff08
provider-inventory 80.36% <96.00%> (+0.34%) ⬆️
provider-proxy 86.08% <ø> (ø) Carriedforward from 5dcff08
tx-signer ?

*This pull request uses carry forward flags. Click here to find out more.

Files with missing lines Coverage Δ
...rovider-inventory/provider-inventory.repository.ts 91.66% <100.00%> (+3.66%) ⬆️
...discovery-scheduler/discovery-scheduler.service.ts 100.00% <100.00%> (ø)
apps/provider-inventory/src/providers-sync-app.ts 0.00% <0.00%> (ø)

... and 41 files with indirect coverage changes

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
apps/provider-inventory/src/providers-sync-app.ts (1)

31-35: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Abort warm-up streams on warm-up failure to avoid partial startup leakage.

If warmUp() fails after starting some lifecycle streams, the catch path closes the HTTP server but does not cancel already-started warm-up streams. Add a warm-up AbortController and abort it in the catch block before rethrowing.

Suggested fix
   if (server) {
+    const scheduler = container.resolve(DiscoverySchedulerService);
+    const warmUpController = new AbortController();
     try {
-      await container.resolve(DiscoverySchedulerService).warmUp();
-      container.resolve(DiscoverySchedulerService).start();
+      await scheduler.warmUp(warmUpController.signal);
+      scheduler.start();
     } catch (error) {
+      warmUpController.abort();
       server.close();
       throw error;
     }
   }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/provider-inventory/src/providers-sync-app.ts` around lines 31 - 35, The
warm-up streams started by DiscoverySchedulerService.warmUp() can leak if warmUp
fails because you only close the HTTP server; create an AbortController before
calling warmUp(), pass its signal into DiscoverySchedulerService.warmUp(signal)
(and any downstream warm-up calls), and in the catch block call
abortController.abort() before calling server.close() and rethrowing the error
so any started warm-up streams are cancelled; ensure any invocation of
DiscoverySchedulerService.start() that relies on warm-up uses the same
controller/signal if needed.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@apps/provider-inventory/src/repositories/provider-inventory/provider-inventory.repository.ts`:
- Line 35: Validate that options?.batchSize is a positive integer before using
it in limit(...) by checking Number.isInteger(batchSize) && batchSize > 0; if
validation fails, fall back to DEFAULT_ONLINE_STREAM_BATCH_SIZE (or throw an
explicit error if that fits the function contract). Update the variable
assignment for batchSize (the constant currently set from options?.batchSize ??
DEFAULT_ONLINE_STREAM_BATCH_SIZE) to perform this guard so the value passed into
the query's limit(...) call is always a safe positive integer.

---

Outside diff comments:
In `@apps/provider-inventory/src/providers-sync-app.ts`:
- Around line 31-35: The warm-up streams started by
DiscoverySchedulerService.warmUp() can leak if warmUp fails because you only
close the HTTP server; create an AbortController before calling warmUp(), pass
its signal into DiscoverySchedulerService.warmUp(signal) (and any downstream
warm-up calls), and in the catch block call abortController.abort() before
calling server.close() and rethrowing the error so any started warm-up streams
are cancelled; ensure any invocation of DiscoverySchedulerService.start() that
relies on warm-up uses the same controller/signal if needed.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 23ecf03c-b351-405a-a42b-b1ffb9b9aa34

📥 Commits

Reviewing files that changed from the base of the PR and between 5dcff08 and fd866bd.

📒 Files selected for processing (5)
  • apps/provider-inventory/src/providers-sync-app.ts
  • apps/provider-inventory/src/repositories/provider-inventory/provider-inventory.repository.spec.ts
  • apps/provider-inventory/src/repositories/provider-inventory/provider-inventory.repository.ts
  • apps/provider-inventory/src/services/discovery-scheduler/discovery-scheduler.service.spec.ts
  • apps/provider-inventory/src/services/discovery-scheduler/discovery-scheduler.service.ts

@stalniy stalniy force-pushed the feat/provider-inventory-warm-up branch from fd866bd to 892522e Compare May 28, 2026 10:28
@github-actions github-actions Bot added size: L and removed size: M labels May 28, 2026
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 28, 2026

Actionable comments posted: 0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants