[Bug]: High client-side peak memory in MilvusClient.insert for float vectors

[pymilvus-bot]

Is there an existing issue for this?

• I have searched the existing issues

Describe the bug

MilvusClient.insert() has high client-side peak memory when packing a large batch of float vectors.

This issue intentionally covers only this path:

MilvusClient.insert() -> GrpcHandler.insert_rows() -> Prepare.row_insert_param() -> unary gRPC serialization

Collection.insert() and the ORM/column-insert path are out of scope.

On PyMilvus master at cf0bc76c, inserting 10,000 768-dimensional float32 vectors gives:

• Raw NumPy vector payload: 29.3 MiB
• Full MilvusClient.insert() incremental process peak: about 157 MiB, or 5.4x the raw payload
• Isolated MilvusClient request packing plus serialization: about 155 MiB, or 5.3x the raw payload
• Isolated request packing time: about 0.59 seconds

The current path has several contributors:

• _pack_fp32_vector_value() converts every NumPy row with np.asarray(..., dtype=np.float32).tolist().
• Field data is built separately and copied into InsertRequest by request.fields_data.extend(...).
• The protobuf repeated-float representation and unary gRPC serialization both need full-batch storage.

Some extra memory is unavoidable, but the current peak scales to roughly five times the vector payload. A prototype that removed tolist() and filled the request field in place only reduced the measured peak from about 5.3x to 5.2x. This means the benchmark and fix must cover the complete request-packing and serialization lifecycle, rather than only one Python conversion.

Expected Behavior

• Add a client-only benchmark for MilvusClient.insert() packing latency and peak memory across representative batch sizes.
• Reduce avoidable allocations while preserving the public API and protobuf wire output.
• If a single unary request cannot provide a materially lower bound, provide a bounded-memory batching strategy or clear API guidance.
• Keep Collection.insert() and ORM serialization changes out of this issue.

Steps/Code To Reproduce behavior

import resource
import sys
import tempfile
import time

import numpy as np

from pymilvus import DataType, MilvusClient


def peak_mib():
    value = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss
    return value / (1024 * 1024) if sys.platform == "darwin" else value / 1024


with tempfile.TemporaryDirectory(dir="/tmp", prefix="mc-insert-") as directory:
    client = MilvusClient(uri=f"{directory}/bench.db")

    schema = MilvusClient.create_schema(auto_id=True)
    schema.add_field("id", DataType.INT64, is_primary=True, auto_id=True)
    schema.add_field("vector", DataType.FLOAT_VECTOR, dim=768)
    client.create_collection("insert_memory_bench", schema=schema)

    baseline = peak_mib()
    vectors = np.random.default_rng(0).random((10_000, 768), dtype=np.float32)
    rows = [{"vector": vector} for vector in vectors]

    started = time.perf_counter()
    result = client.insert("insert_memory_bench", rows)
    elapsed = time.perf_counter() - started
    peak = peak_mib()

    raw_mib = vectors.nbytes / (1024 * 1024)
    print(
        {
            "insert_count": result["insert_count"],
            "raw_vector_mib": raw_mib,
            "incremental_peak_mib": peak - baseline,
            "peak_to_raw_ratio": (peak - baseline) / raw_mib,
            "elapsed_seconds": elapsed,
        }
    )

    client.close()

Environment details

• PyMilvus: master cf0bc76c
• Python: 3.13.11
• Platform: macOS arm64
• NumPy: 2.4.2
• protobuf: 5.29.6
• grpcio: 1.66.2
• Milvus Lite: 2.5.1

Anything else?

Suggested benchmark location: tests/benchmark/test_insert_bench.py, with the gRPC call mocked so the benchmark measures only MilvusClient request construction and serialization.

Related usage report: #988.

[lancy69]

I'd like to work on this. My understanding: on the MilvusClient.insert() → GrpcHandler.insert_rows() → Prepare.row_insert_param() → unary gRPC serialization path, packing a large float32 batch peaks around 5.4x the raw NumPy payload due to redundant full-buffer copies during request packing/serialization.

Plan: profile that path to pin down the redundant copies, reduce them (avoid materialising an extra copy of the vector data before it is written into the request, and reuse buffers where the proto packing allows), and verify with a peak-memory measurement on a large batch alongside the existing insert tests. Is it available for me to take?