Skip to content

fix(doris): support IVF on disk index properties#768

Open
zhiqiang-hhhh wants to merge 2 commits intozilliztech:mainfrom
zhiqiang-hhhh:codex/doris-ivf-on-disk-nlist
Open

fix(doris): support IVF on disk index properties#768
zhiqiang-hhhh wants to merge 2 commits intozilliztech:mainfrom
zhiqiang-hhhh:codex/doris-ivf-on-disk-nlist

Conversation

@zhiqiang-hhhh
Copy link
Copy Markdown
Contributor

Summary

  • Allow Doris ANN index properties to be passed through via --index-prop, including future Doris index types and index-specific options.
  • Fix ivf_on_disk index creation by ensuring nlist is preserved and forwarded through Doris SDK ANN properties.
  • Update Doris CLI docs to describe generic index property passthrough and remove HNSW-specific convenience CLI options.

Motivation

Doris supports more ANN index types than HNSW, and new index properties may be added in Doris without requiring VectorDBBench changes. Previously the Doris client validated index types locally and only handled HNSW/IVF paths, which made ivf_on_disk and future index properties fragile.
This PR makes Doris index configuration more forward-compatible by treating unknown index types and unmapped index options as passthrough properties.

Changes

  • Normalize Doris index_type and default it to hnsw when omitted.
  • Merge index_properties before applying known convenience fields.
  • Require nlist for known IVF index types: ivf and ivf_on_disk.
  • Pass unknown Doris index types through instead of rejecting them locally.
  • Attach unmapped index parameters to IndexOptions.ann_properties or IndexOptions.properties.
  • Preserve runtime index options from both ann_properties and properties.
  • Remove Doris CLI inheritance from HNSW-specific options.
  • Document --index-prop index_type=... usage with an ivf_on_disk example.

Testing

  • Ran python -m compileall -q vectordb_bench/backend/clients/doris
  • Ran git diff --check

@sre-ci-robot
Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: zhiqiang-hhhh
To complete the pull request process, please assign xuanyang-cn after the PR has been reviewed.
You can assign the PR to them by writing /assign @xuanyang-cn in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants