Skip to content

perf: add hive_notifs(dst_id, post_id, type_id) index (v27→v28)#371

Open
ety001 wants to merge 1 commit into
masterfrom
feat/add-hive-notifs-dst-post-type-index
Open

perf: add hive_notifs(dst_id, post_id, type_id) index (v27→v28)#371
ety001 wants to merge 1 commit into
masterfrom
feat/add-hive-notifs-dst-post-type-index

Conversation

@ety001

@ety001 ety001 commented May 30, 2026

Copy link
Copy Markdown
Member

Problem

The hivemind indexer's notify-account-votes query was performing a full table scan on hive_notifs (~97M rows), causing each query to take ~2 seconds:

SELECT 1 FROM hive_notifs
WHERE dst_id = :dst_id AND post_id = :post_id AND type_id = 16

This single query consumed 99%+ of sync time (stats showed 30x/60s at ~2050ms each), making sync stall completely for days.

Root Cause

hive_notifs only had the primary key index (id) plus 6 partial indexes — none with a leading (dst_id, post_id, type_id) prefix that this query could use.

The closest existing index hive_notifs_ix5 (post_id, type_id, dst_id, src_id) WHERE post_id IS NOT NULL AND type_id IN (16,17) cannot help because:

  • Leading column is post_id, but the query filters on dst_id first
  • It is a partial index with a narrower WHERE clause

Fix

Add a composite index (dst_id, post_id, type_id) via db migration v27→v28.

Query time: ~2000ms → ~30ms (65x improvement)

Safety

  • CREATE INDEX CONCURRENTLY — no read/write blocking
  • IF NOT EXISTS — idempotent; safe if the index was already manually added (e.g. on dzire)
  • Follows the same migration pattern as v22, v23, v26, v27

Index Overlap Check

Existing hive_notifs indexes do NOT cover this query pattern:

Index Leading Columns Covers (dst_id, post_id, type_id)?
hive_notifs_ix1 dst_id, id ❌ missing post_id, type_id
hive_notifs_ix5 post_id, type_id, dst_id ❌ wrong column order
hive_notifs_ix6 dst_id, created_at, score, id ❌ missing post_id, type_id

Testing

Verified on dzire production instance — index created, sync resumed immediately after days of being stalled.

The indexer's notify-account-votes query:
  SELECT 1 FROM hive_notifs
  WHERE dst_id = :dst_id AND post_id = :post_id AND type_id = 16
was doing a full table scan (~2s per query on 97M rows), causing sync
to stall completely. This composite index reduces query time to <1ms.

Uses CREATE INDEX CONCURRENTLY IF NOT EXISTS so it's safe for:
- Already-running instances (no lock contention)
- Instances where the index was manually added (idempotent)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant