Skip to content

Fix undercounting of RAM used by vectors buffered in in-memory segments#15982

Open
iprithv wants to merge 2 commits intoapache:mainfrom
iprithv:fix/vector-ram-accounting-undercount
Open

Fix undercounting of RAM used by vectors buffered in in-memory segments#15982
iprithv wants to merge 2 commits intoapache:mainfrom
iprithv:fix/vector-ram-accounting-undercount

Conversation

@iprithv
Copy link
Copy Markdown
Contributor

@iprithv iprithv commented Apr 24, 2026

Description

Vector RAM accounting in ramBytesUsed() had three bugs causing IndexWriter to undercount memory usage for buffered vectors, leading to delayed flush decisions and higher than expected memory consumption.

Bugs Fixed

Fixes #15901

1. BufferingKnnVectorsWriter hardcoded Float.BYTES for all encodings

Byte vectors (VectorEncoding.BYTE) were reported as 4x their actual size because ramBytesUsed() always multiplied by Float.BYTES (4) instead of Byte.BYTES (1). This is technically an overcount for byte vectors, but it's wrong in the opposite direction, it masks the undercounting elsewhere and produces incorrect flush thresholds.

2. Quantized writers never counted rawVectorDelegate RAM

Lucene104ScalarQuantizedVectorsWriter, Lucene99ScalarQuantizedVectorsWriter, and Lucene102BinaryQuantizedVectorsWriter all wrap a rawVectorDelegate (Lucene99FlatVectorsWriter). For FLOAT32 fields, the delegate's field-level data was counted indirectly through FieldWriter.flatFieldVectorsWriter.ramBytesUsed(). But for BYTE fields, which bypass the quantized FieldWriter entirely, the delegate was never queried, making byte vector RAM completely invisible (48 bytes reported for hundreds of KB of actual data).

Refactored all three writers to call rawVectorDelegate.ramBytesUsed() at the writer level for all flat vector data, and quantizationOverheadBytesUsed() for quantization-specific state (magnitudes, dimensionSums) to avoid double-counting.

3. dimensionSums array not counted

The float[dimension] array used for centroid calculation during flush was not included in ramBytesUsed() for Lucene104ScalarQuantizedVectorsWriter and Lucene102BinaryQuantizedVectorsWriter.

@iprithv iprithv force-pushed the fix/vector-ram-accounting-undercount branch from f97d7cf to 918fa44 Compare April 24, 2026 21:11
@iprithv
Copy link
Copy Markdown
Contributor Author

iprithv commented Apr 28, 2026

@mikemccand could you please take a look at this when you get a chance? Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Are we undercounting RAM used by vectors buffered in an in-memory segment?

1 participant