Skip to content

enhance: add compaction lifecycle observability APIs - #3791

Draft
MrPresent-Han wants to merge 2 commits into
milvus-io:masterfrom
MrPresent-Han:codex/compaction-e2e-observability-pymilvus
Draft

enhance: add compaction lifecycle observability APIs#3791
MrPresent-Han wants to merge 2 commits into
milvus-io:masterfrom
MrPresent-Han:codex/compaction-e2e-observability-pymilvus

Conversation

@MrPresent-Han

@MrPresent-Han MrPresent-Han commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

issue: #3790

depends on: milvus-io/milvus-proto#668 and milvus-io/milvus#53308

What changed

  • add lifecycle-state filtering and lineage metadata to persistent segment listing
  • add list_segments with the full retained lifecycle-state default
  • add list_serving_segments as the explicit serving-set API
  • add collection-scoped list_compaction_tasks while preserving job-scoped plan lookup
  • expose complete compaction plan and task metadata through client result types
  • support database context in segment and collection task requests
  • preserve cleaned task failure reasons in Plan.failure_reason

Retention semantics

Dropped segments and terminal compaction tasks are retained server metadata rather than a permanent audit log, so E2E callers must poll and cache observations within the retention window.

Verification

  • 799 related unit tests passed across gRPC handling, request preparation, client types, sync client, and async client
  • all 46 gRPC utility tests passed, including cleaned task failure-reason parsing
  • local standalone E2E passed after three insert and flush batches: four retained tasks were observable, the active lineage target exactly matched the serving segment set, and all 300 PK and scalar rows were retrieved intact
  • this PR remains draft until both protocol and server dependencies are consumable

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
@sre-ci-robot

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: MrPresent-Han
To complete the pull request process, please assign longjiquan after the PR has been reviewed.
You can assign the PR to them by writing /assign @longjiquan in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@codecov

codecov Bot commented Sep 9, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 98.38710% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 94.23%. Comparing base (a4b3b38) to head (b549a54).

Files with missing lines Patch % Lines
pymilvus/client/prepare.py 88.88% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #3791      +/-   ##
==========================================
+ Coverage   94.22%   94.23%   +0.01%     
==========================================
  Files          77       77              
  Lines       16204    16250      +46     
==========================================
+ Hits        15268    15313      +45     
- Misses        936      937       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants