Skip to content

fix(async): preserve optimization after result wait timeout - #3783

Open
Kuang-xianxin wants to merge 2 commits into
milvus-io:masterfrom
Kuang-xianxin:codex/preserve-async-optimization-on-wait-timeout
Open

fix(async): preserve optimization after result wait timeout#3783
Kuang-xianxin wants to merge 2 commits into
milvus-io:masterfrom
Kuang-xianxin:codex/preserve-async-optimization-on-wait-timeout

Conversation

@Kuang-xianxin

@Kuang-xianxin Kuang-xianxin commented Sep 5, 2026

Copy link
Copy Markdown

Problem

AsyncOptimizeTask.result(timeout=...) passes its owned background task directly to asyncio.wait_for. A short wait cancels the optimization, affects other result waiters, and prevents a later retry. For the preview optimize(wait=False) API this can stop the client workflow during compaction before index rebuild and load refresh.

Cancellation also needs to distinguish ownership: a caller holding an AsyncOptimizeTask can wait again or explicitly cancel it, while optimize(wait=True) does not return a handle when it exits early.

Change

  • Shield the background task in both timed and untimed result() waits. A local wait timeout raises the existing timeout error; caller cancellation propagates as CancelledError, allowing standard asyncio.wait_for and asyncio.timeout wrappers to work without cancelling the shared optimization.
  • Keep explicit task.cancel() behavior and the optimization's own timeout forwarding.
  • In optimize(wait=True), cancel and await unfinished owned work before returning a wait timeout or propagating caller cancellation. This prevents detaching an inaccessible task and retrieves its cancellation exception.
  • Document the ownership distinction. Server-side compaction cancellation behavior is unchanged.

Regression coverage includes later success/failure after a short wait, concurrent waiters, explicit task cancellation, external timeout wrappers, and zero/positive timeouts or cancellation of optimize(wait=True). A public optimize(wait=False) workflow test uses mocked RPC responses and verifies load refresh after a timed-out result wait.

Validation

Python 3.13.14 on Windows:

  • Original source: 5 initial regression cases failed.
  • Before the review revision: 9 additional cancellation/ownership regressions failed on d22d7af.
  • Latest revision: 148 passed with:
uv run --no-sync pytest tests/unit/test_async_optimize_task.py tests/unit/test_async_milvus_client_ops.py tests/unit/test_optimize_task.py -q -o log_cli=false --tb=short
  • Ruff and Black checks pass on all four changed files.
  • The asyncio.timeout context test is skipped on Python versions before 3.11; wait_for cases cover older versions.
  • No live Milvus server or performance benchmark was used; this is a client task-lifecycle fix.

Found through source inspection. Searches did not find a matching optimization/result-timeout fix; the existing async refresh-load PRs concern separate RPC options.

@sre-ci-robot

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: Kuang-xianxin
To complete the pull request process, please assign tedxu after the PR has been reviewed.
You can assign the PR to them by writing /assign @tedxu in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@sre-ci-robot

Copy link
Copy Markdown

Welcome @Kuang-xianxin! It looks like this is your first PR to milvus-io/pymilvus 🎉

Assisted-by: Codex
Signed-off-by: Kuang-xianxin <243476082+Kuang-xianxin@users.noreply.github.com>
@Kuang-xianxin
Kuang-xianxin force-pushed the codex/preserve-async-optimization-on-wait-timeout branch from df639b2 to d22d7af Compare September 6, 2026 02:59
@mergify mergify Bot added dco-passed and removed needs-dco labels Sep 6, 2026
@mergify

mergify Bot commented Sep 6, 2026

Copy link
Copy Markdown

Tick the box to add this pull request to the merge queue (same as @mergifyio queue).

  • Queue this pull request

Comment thread pymilvus/milvus_client/async_optimize_task.py
Comment thread pymilvus/milvus_client/async_optimize_task.py Outdated
@codecov

codecov Bot commented Sep 8, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.24%. Comparing base (a4b3b38) to head (d22d7af).

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #3783      +/-   ##
==========================================
+ Coverage   94.22%   94.24%   +0.01%     
==========================================
  Files          77       77              
  Lines       16204    16204              
==========================================
+ Hits        15268    15271       +3     
+ Misses        936      933       -3     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@mergify mergify Bot added the ci-passed label Sep 8, 2026
Preserve background tasks when result waiters are cancelled, propagate caller cancellation, and cancel and retrieve the owned task when optimize(wait=True) exits early. Add nine regressions for external timeout wrappers and owned-task cleanup.

Assisted-by: Codex
Signed-off-by: Kuang-xianxin <243476082+Kuang-xianxin@users.noreply.github.com>
@mergify mergify Bot removed the ci-passed label Sep 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants