perf: add hive_notifs(dst_id, post_id, type_id) index (v27→v28)#371
Open
ety001 wants to merge 1 commit into
Open
perf: add hive_notifs(dst_id, post_id, type_id) index (v27→v28)#371ety001 wants to merge 1 commit into
ety001 wants to merge 1 commit into
Conversation
The indexer's notify-account-votes query: SELECT 1 FROM hive_notifs WHERE dst_id = :dst_id AND post_id = :post_id AND type_id = 16 was doing a full table scan (~2s per query on 97M rows), causing sync to stall completely. This composite index reduces query time to <1ms. Uses CREATE INDEX CONCURRENTLY IF NOT EXISTS so it's safe for: - Already-running instances (no lock contention) - Instances where the index was manually added (idempotent)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The hivemind indexer's
notify-account-votesquery was performing a full table scan onhive_notifs(~97M rows), causing each query to take ~2 seconds:This single query consumed 99%+ of sync time (stats showed 30x/60s at ~2050ms each), making sync stall completely for days.
Root Cause
hive_notifsonly had the primary key index(id)plus 6 partial indexes — none with a leading(dst_id, post_id, type_id)prefix that this query could use.The closest existing index
hive_notifs_ix5 (post_id, type_id, dst_id, src_id) WHERE post_id IS NOT NULL AND type_id IN (16,17)cannot help because:post_id, but the query filters ondst_idfirstFix
Add a composite index
(dst_id, post_id, type_id)via db migration v27→v28.Query time: ~2000ms → ~30ms (65x improvement)
Safety
CREATE INDEX CONCURRENTLY— no read/write blockingIF NOT EXISTS— idempotent; safe if the index was already manually added (e.g. on dzire)Index Overlap Check
Existing
hive_notifsindexes do NOT cover this query pattern:(dst_id, post_id, type_id)?hive_notifs_ix1dst_id, idhive_notifs_ix5post_id, type_id, dst_idhive_notifs_ix6dst_id, created_at, score, idTesting
Verified on dzire production instance — index created, sync resumed immediately after days of being stalled.