Support for Microsoft.Extensions.VectorData

[verdie-g]

.NET now provides abstractions for vector stores (see Introducing Microsoft.Extensions.VectorData Preview). Is there an opportunity for Milvus to implement it so it can be easily integrated in libraries/applications.

[roji]

There's an issue tracking this here. Note that ideally the SDK would be brought up to date first, as it hasn't been worked on for a while.

[verdie-g]

Note that ideally the SDK would be brought up to date first

Understood. Do you have a list of tasks in mind?

[roji]

I'd start by updating the .NET SDK to use the latest gRPC schema from Milvus and mapping out what newer features are currently missing; for example, #96 points out to some ways in which the SDK is behind. I'd do a quick pass over other open issues as well.

Basically the SDK is currently missing an owner, i.e. someone to invest in it from time to time, keep up with Milvus developments, etc. It shouldn't take a lot of time, but it will needs to be done.

If the SDK is a bit more actively maintained, I may be able to find a bit of time to write at least the initial version of the Microsoft.Extensions.VectorData provider over it; though as with this SDK, I'm not going to be able to own it long-term.

[xiaofan-luan]

Hi @roji
I'm the maintainer of Milvus core team.
First of all, thanks you and the microsoft team for the active maintenance of the csharp client.

Would you mind have a meeting this week, here is a couple topic I want to cover in the meeting:

1. Short overview about Milvus 2.6 and all the SDK requirement needed.
2. Align with a plan where we can put C# client into official recommended API list and offer C# document in milvus.io
3. long term plan to keep is repo active and catch up

feel free to reach me at james.luan@zilliz.com. Anyone interested in joining this discussion is welcomed

[verdie-g]

@roji would you have some time to define tasks to improve this SDK. You mentioned #96 earlier so I suppose one of the task would be to support https://milvus.io/docs/full-text-search.md.

My company is committed to use Milvus so I'll probably have to contribute to some extent.

[roji]

@verdie-g that's great news! I'd be happy to help with guidance (especially around public API design) and PR reviews.

I'd first of all start with bringing the low-level .NET SDK (this repo) up to date with the latest Milvus gRPC schema (that's #96), and implementing basic features introduced after Milvus 2.2, which is when this was originally introduced. IMHO once we have an up-to-date SDK, covering a lot of the features, at that point we can also implement a Microsoft.Extensions.VectorData wrapper over it (that's what this issue tracks). I can help there as well.

I'd also recommend identifying the new Milvus features which were introduced since 2.2 and which are currently lacking here; we should have an issue tracking each one. @xiaofan-luan maybe you (or someone from Milvus) can help with that?

[xiaofan-luan]

@yhmo can you help a generate on list.
Yhmo is maintainer of cpp and java SDK so he can definitely help

[verdie-g]

at that point we can also implement a Microsoft.Extensions.VectorData wrapper over it

Btw I did try to look into that but it's extremely hard to implement because of the use of Expression :( Anyway, the MS agent framework does not rely on this for RAG so I'm not super interested into that feature anymore. I just want to avoid developers in the company being blocked because a feature is bugged or unavailable.

[roji]

Btw I did try to look into that but it's extremely hard to implement because of the use of Expression :(

I get that working with Expressions can seem complicated... I recommend looking at the other provider implementations (code) and using them as a baseline for work. Most functionality in an MEVD provider also doesn't depend on Expression - only the Filter functionality, so there's ample work there before ever having to touch Expression.

Finally, if there's technical difficulties around Expression specifically, I might be able to clear up a bit of time and do that work at some point.

[yhmo]

The following list is the key features that each Milvus SDK needs to be implemented on the client-side(Maybe I missed something). You can take a look at the interfaces of Milvus C++ SDK: https://github.com/milvus-io/milvus-sdk-cpp/blob/2.5/src/include/milvus/MilvusClient.h. Most of the v2.5 features have been implemented in the C++ SDK(almost one hundred interfaces).

v2.3

Features

• SCANN index
• GPU_IVF_FLAT/GPU_IVF_PQ index
• Support QueryIterator/SearchIterator
• Cosine metric
• Upsert()
• DescribeAlias()/ListAliases()
• Range Search
• Array field
• Dynamic field
• Delete by filter expression

Other changes

• Remove index types: IVF_HNSW/RHNSW_FLAT/RHNSW_SQ/RHNSW_PQ/ANNOY
• Remove metrics: TANIMOTO/Superstructure/Substructure
• Deprecated Time Travel

v2.4

Features

• Add new index types: DISKANN/AUTOINDEX/GPU_BRUTE_FORCE/GPU_CAGRA/TRIE/STL_SORT/INVERTED/SPARSE_INVERTED_INDEX/SPARSE_WAND
• Add new metric types: COSINE
• SparseVector field
• Multi-Embedding fields in one collection
• Float16Vector/BFloat16Vector field
• HybridSearch()
• AlterCollectionField()
• AlterIndexProperties()/DropIndexProperties()
• Partial load
• Allow upsert to auto-id collection
• RBAC enhancements, new interfaces

Improvement

• Cache collection schema to reduce DescribeCollection call
• Support search with grouping by field
• Retry machinery for RPC call
• UseDatabase()
• Row-based insert/upsert
• QuertResults/SearchResults to return row-based results

Other changes

• Remove CalcDistance interface, not supported by milvus 2.4
• Remove Flush call in GetCollectionStats/GetPartitionStats/CreateIndex, milvus 2.4 doesn't allow calling Flush constantly
• CreateIndex() internally calls DescribeIndex() to check index state instead of GetIndexState()
• Remove client-side checks for index parameters and search parameters, let milvus server validate

v2.5

Features

• Add new index types: Bitmap
• HybridSearch supports group by
• Clustering compaction
• Nullable field
• Default value of field
• Text match
• Full text search(BM25 function)
• RunAnalyzer()
• DropCollectionProperties()/DropCollectionFieldProperties()

v2.6

Features

• Add new index types: MINHASH_LSH/IVF_RABITQ/NGRAM
• Int8Vector
• AddCollectionField()
• Rerank functions for Search/HybridSearch
• Struct field
• Geometry field
• Timestamptz field

Optional Tools

• BulkWriter
• BulkImport by RESTful API
• Client pool

[roji]

Thanks @yhmo! Reposting this in #96 (comment), as this issue is about the Microsoft.Extensions.VectorData, which is a separate thing.