Skip to content

Ondemand sandbox#736

Open
YunchuWang wants to merge 82 commits into
mainfrom
wangbill/serverless-private-preview-op1
Open

Ondemand sandbox#736
YunchuWang wants to merge 82 commits into
mainfrom
wangbill/serverless-private-preview-op1

Conversation

@YunchuWang

@YunchuWang YunchuWang commented Jun 8, 2026

Copy link
Copy Markdown
Member

The scenario

A .NET app using Durable Task Scheduler should be able to keep its normal orchestration worker local, but run selected expensive or isolated activities in a DTS-started sandbox worker container.

The intended flow is:

  1. The local app registers orchestrators and local activities with the existing DTS client/worker APIs.
  2. The local app declares a sandbox worker profile: activity names, container image, managed identities, CPU/memory, environment variables, and max concurrency.
  3. DTS stores that declaration as configuration. It means: when this activity is needed, this is the sandbox worker profile that can run it.
  4. DTS starts a sandbox worker container from the declared image.
  5. The sandbox worker calls UseSandboxWorker(), connects back to DTS, reports the activities it registered, and heartbeats active activity count.
  6. DTS only advertises live capacity after the worker registration matches the declaration, then routes the declared activity work to that sandbox worker.

In the sample, the local app runs the orchestration and LocalHello; the sandbox worker image runs RemoteHello. Both processes connect to the same task hub, but they must not receive the same work.

What's missing

durabletask-dotnet did not have a .NET SDK surface for this pattern.

There was no client package for declaring sandbox worker profiles, no worker package for code running inside the sandbox, and no shared .NET transport for the sandbox activity protobuf service.

A few correctness details also fell out during validation:

  • The local sample worker must call UseWorkItemFilters(). Without it, DTS can dispatch RemoteHello to the local worker, which does not implement that activity, so the orchestration can get stuck retrying the wrong worker.
  • Worker registration metadata must be trustworthy. Sending a whitespace task hub or empty DTS_SANDBOX_ID makes server-side routing and telemetry hard to reason about.
  • The sandbox worker needs activity start/completed notifications to report active activity count, but failures in that notification hook must not break normal activity dispatch.

The change

The SDK surface is split into two opt-in preview packages:

  • Microsoft.DurableTask.Client.AzureManaged.Sandboxes
  • Microsoft.DurableTask.Worker.AzureManaged.Sandboxes

A good review path is:

  1. Start with src/Grpc/sandbox_service.proto and src/Shared/AzureManaged.Sandboxes/. This is the shared contract and transport for declaring sandbox activities, removing declarations, and connecting a live sandbox worker stream.
  2. Read src/Client/AzureManaged.Sandboxes/. This is the local declarer-app side: [SandboxWorkerProfile], SandboxWorkerProfileOptions, declaration discovery, validation, and SandboxActivitiesClient.
  3. Read src/Worker/AzureManaged.Sandboxes/. This is the sandbox-worker side: UseSandboxWorker(), activity-only filters, DTS-injected runtime settings, start/heartbeat messages, and active activity tracking.
  4. Read the small gRPC worker hook changes under src/Worker/Grpc/. These add guarded activity execution notifications used by the sandbox worker heartbeat path.
  5. Read samples/on-demand-sandbox/ last. It shows the intended end-to-end shape: local declarer app, remote worker image, and shared activity names.

The new sandbox packages target net10.0 only because this is new preview functionality. NuGet publish jobs are split per package so an already-published AzureManaged package version does not block publishing the new Sandboxes packages.

YunchuWang added 30 commits May 13, 2026 19:29
ServerlessActivitiesClientAdapter now takes an attachTaskHubMetadata flag (default true). The Azure-managed channel already injects the taskhub header via CallCredentials, so the AddDurableTaskScheduler* path constructs the adapter with attachTaskHubMetadata: false to avoid sending duplicate headers on DeclareServerlessActivities and ConnectServerlessActivityWorker. Added two unit tests with a recording CallInvoker covering both modes.
This reverts commit 18c8e02.
- Updated README.md to clarify remote worker image settings.
- Simplified task hub retrieval in Program.cs.
- Removed unnecessary endpoint configuration in remote worker.
- Added Azure.Identity package reference in csproj.
- Refined serverless worker extensions for environment configuration.
- Updated serverless activity configuration to handle registered activities.
- Modified tests to reflect changes in activity registration and filtering.
Copilot AI review requested due to automatic review settings June 12, 2026 18:20

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 50 out of 50 changed files in this pull request and generated 2 comments.

Comment thread samples/on-demand-sandbox/remote-worker/Containerfile Outdated
Copilot AI review requested due to automatic review settings June 12, 2026 19:02

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 50 out of 50 changed files in this pull request and generated 1 comment.

Comment thread test/Worker/AzureManaged.Tests/SandboxActivitiesTests.cs Outdated
Copilot AI review requested due to automatic review settings June 12, 2026 19:18

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 50 out of 50 changed files in this pull request and generated 3 comments.

Comment thread src/Worker/Grpc/Internal/InternalOptionsExtensions.cs
Comment thread src/Grpc/orchestrator_service.proto
Copilot AI review requested due to automatic review settings June 12, 2026 20:23

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 50 out of 50 changed files in this pull request and generated 1 comment.

Comment thread src/Worker/AzureManaged.Sandboxes/DurableTaskSchedulerSandboxWorkerExtensions.cs Outdated
Replace legacy DTS_SUBSTRATE and DTS_ON_DEMAND_SANDBOX_MAX_ACTIVITIES env vars with DTS_SANDBOX_PROVIDER and DTS_SANDBOX_MAX_ACTIVITIES. Updated worker code (environment parsing and validation), message builder, README, and tests to use the new names and updated validation/error messages. Keeps behavior the same while standardizing sandbox-related environment variable names.
@YunchuWang YunchuWang requested a review from torosent June 12, 2026 21:49
Copilot AI review requested due to automatic review settings June 13, 2026 00:03

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 50 out of 50 changed files in this pull request and generated 2 comments.

Comment thread src/Shared/AzureManaged.Sandboxes/SandboxActivityMetadata.cs
Copilot AI review requested due to automatic review settings June 13, 2026 00:17

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 50 out of 50 changed files in this pull request and generated no new comments.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants