Skip to content

feat(index): add IVF+PQ compression flag for the FAISS build - #155

Open
dex0shubham wants to merge 1 commit into
StarTrail-org:mainfrom
dex0shubham:feat/faiss-ivfpq
Open

feat(index): add IVF+PQ compression flag for the FAISS build#155
dex0shubham wants to merge 1 commit into
StarTrail-org:mainfrom
dex0shubham:feat/faiss-ivfpq

Conversation

@dex0shubham

Copy link
Copy Markdown
Contributor

Closes #153.

Motivation

The FAISS backend uses IndexIVFFlat, storing every vector at full float32
precision — ~245 GB of RAM for a 30 M-vector corpus at dim 2048. That's out of
reach for air-gapped or Docker-free deployments that can't run a Qdrant sidecar,
even though Qdrant users can already compress via --qdrant-quantization-config.
This adds the equivalent knob on the FAISS side.

Change

Opt-in --pq-m / --pq-nbits flags select IndexIVFPQ (Product Quantization)
instead of IndexIVFFlat. At pq_m=64, nbits=8 each vector shrinks from 8192
to 64 bytes (~128x, ~1.9 GB for 30 M vectors) for a typical small recall cost.

  • embed/src/pixelrag_embed/index.pybuild_ivf takes pq_m (default 0 =
    unchanged IVFFlat) and pq_nbits (4 or 8). When pq_m > 0 it builds
    IndexIVFPQ; the existing .train() call trains the PQ codebooks too, so no
    new training path. summary.json records index_type / pq_m / pq_nbits
    so the served index is self-describing.
  • index/src/pixelrag_index/pipelines.py — forwards index.pq_m /
    index.pq_nbits from pixelrag.yaml to the build command.
index:
  backend: faiss
  pq_m: 64      # sub-quantizers; must divide the embedding dim evenly
  pq_nbits: 8   # 4 or 8 (default 8)

One fix vs. the proposal

The issue's sketch was faiss.IndexIVFPQ(quantizer, dim, nlist, pq_m, pq_nbits),
which defaults to METRIC_L2 — wrong for the IP-normalized embeddings this
index uses, and a silent ranking regression. This PR passes metric_type
explicitly and a test locks it in.

Validation: pq_m must divide the embedding dim evenly, else a clear
ValueError (FAISS would otherwise assert deep in C++).

Tests

tests/test_ivfpq_index.py (plain pytest, faiss via importorskip, no model
— builds a real index from a synthetic shard):

  • IVFPQ index is built, trained, has ntotal == n, is searchable, and its
    metric_type is inner product (guards the L2-default bug).
  • summary.json records index_type="ivfpq" and the PQ params.
  • pq_m not dividing the dim raises ValueError.
  • pq_m=0 (default) still produces an IndexIVFFlat — no behavior change.

All pass; ruff check / ruff format --check clean.

@vercel

vercel Bot commented Sep 5, 2026

Copy link
Copy Markdown

@dex0shubham is attempting to deploy a commit to the andylizf's projects Team on Vercel.

A member of the Team first needs to authorize it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[feat] Add IVF+PQ FAISS flag for compressed-memory deployments (air-gapped / Docker-free parity with Qdrant)

1 participant