perf: consolidate the Frames indexes, and drop unused and duplicate indexes elsewhere

[connortechnology]

Two migrations, from an audit of every table in zm_create.sql.in against the queries that actually read it. version.txt goes to 1.39.28.

• db/zm_update-1.39.27.sql — replace the three Frames secondary indexes with one composite.
• db/zm_update-1.39.28.sql — drop seven indexes elsewhere that are never read or duplicate another index.

---

1.39.27 — Frames

Frames carries three secondary indexes: EventId_idx (EventId), Type (Type) and TimeStamp (TimeStamp). These become a single EventId_FrameId_idx (EventId, FrameId).

Access patterns

Every query against Frames is anchored on EventId, and nearly all of them order or range by FrameId within the event:

Site	Query
src/zm_eventstream.cpp:281, src/zma.cpp:418	WHERE EventId=? ORDER BY FrameId ASC
web/includes/Event.php:770,774, web/ajax/status.php:513,515	WHERE EventId=? AND FrameId </> ? ORDER BY FrameId LIMIT 1
web/includes/Event.php:368, web/includes/functions.php:841	WHERE EventId=? AND Score=? ORDER BY FrameId LIMIT 1
scripts/zmfilter.pl.in	WHERE EventId=? AND Type=? ORDER BY FrameId
ZoneMinder/Event.pm:1062, scripts/zmaudit.pl.in:775	max(TimeStamp), max(FrameId), sum(Score) WHERE EventId=?
src/zm_eventstream.cpp:438, web/ajax/status.php:187-190	min/max(Delta), min/max(FrameId) per event
web/api (/frames/index/EventId:N, montage review)	WHERE EventId IN (...)

The plain EventId index left all of those to a filesort. For the LIMIT 1 prev/next-frame lookups on the playback path, that meant reading and sorting an entire event's frames to return one row.

Nothing filters or sorts on Type or TimeStamp without EventId, so neither index was ever used for reading. Type is a three-value enum and cannot be selective in any case. Both only cost insert time on the highest-insert-rate table in the schema, plus space, plus work on every DELETE ... WHERE EventId.

Measured

200 events x 500 frames, same data in a table with the old indexes and one with the new, MySQL 8.4:

Query	Before	After
WHERE EventId=? AND FrameId < ? ORDER BY FrameId DESC LIMIT 1	ref EventId_idx, rows 500, Using filesort	range, Using index condition; Backward index scan
WHERE EventId=? ORDER BY FrameId	Using filesort	no filesort
max(FrameId) WHERE EventId=?	rows 500	Select tables optimized away

Ordering

The composite index is added before EventId_idx is dropped, so that its leftmost prefix covers EventId for any install still carrying the foreign key on Frames.EventId that 1.35.11 added and 1.37.31 drops only when it happens to be named Frames_ibfk_1. Dropping first would fail on those. Tested against a schema with that foreign key present: the drop succeeds and the constraint survives.

The index is deliberately not UNIQUE. FrameId is unique within an event — it was the primary key before 1.28.107 — but a UNIQUE constraint would turn a duplicate-id bug into frames dropped mid-recording rather than a log line.

---

1.39.28 — the rest of the schema

Index	Why
Logs.TimeKey	Exact duplicate of Logs_TimeKey_idx, created three lines below it in zm_create.sql.in and added to upgraded installs by 1.31.11. Both present everywhere, and every log INSERT maintains both. Logs is the second highest insert-rate table in the schema.
Stats.MonitorId, Stats.ZoneId	Never read. Every query on Stats is anchored on EventId — the per-frame zone stats view, the ZoneId filter term in FilterTerm.php and Filter.pm, the deletes in Event.php, Event.pm and zmaudit.pl — and EventId_ZoneId serves them all. A row is written per frame per zone when ZM_RECORD_EVENT_STATS is on, so this is the same insert-rate argument as the Frames indexes above.
EncoderTemplates.Encoder	Leftmost prefix of Encoder_Name.
Role_Groups_Permissions_RoleId_idx, Role_Monitors_Permissions_RoleId_idx	Leftmost prefix of the UNIQUE index beside them. Small, rarely written tables — tidying rather than a saving.
Monitor_Status_UpdatedOn_idx	Dropped by zm_update-1.37.76.sql but left in zm_create.sql.in, so fresh installs have carried it and upgraded installs have not. Removed from zm_create.sql.in and repeated in the migration so the two agree from here on.

Stats.MonitorId and Stats.ZoneId are the only two with no other index covering their column, so their drop is guarded on there being no foreign key on it. 1.37.31 drops the Stats foreign keys only when they are named Stats_ibfk_1..4; where one survived under another name the drop would fail with errno 150 and abort the upgrade. Built that case and confirmed it skips with a message instead.

---

Testing

Against MySQL 8.4:

• master's zm_create.sql.in, with 1.39.27 and 1.39.28 applied, is index for index identical to a fresh install from the updated zm_create.sql.in — 123 indexes across 55 tables, all 29 foreign keys intact.
• Both migrations verified idempotent by running them twice.
• The Frames foreign-key case and the Stats foreign-key case each built explicitly and confirmed.

Notes

On a Frames table with hundreds of millions of rows the ADD INDEX is an InnoDB in-place build: it does not block writes, but it will take a while. The migration prints a warning before starting. Every drop is effectively instant.

The per-event aggregates (sum(Score), max(TimeStamp), min/max(Delta)) still read every row of the event. Covering them would need all six columns in the index, i.e. a second copy of the table, so they are left alone.

Checked and left as they are: Events (all four indexes map to real query shapes), Events_Tags (the ON E.Id = ET.EventId join in the events list is covered by the Events_Tags_ibfk_2 foreign key index), Server_Stats (both indexes used), and the small tables. AI_Detections carries three single-column indexes and has no consumer anywhere in the tree yet, so there is nothing to judge them against — left for whenever that feature lands.

🤖 Generated with Claude Code

[connortechnology]

Added a second commit (11bade086) extending this to the rest of the schema: an audit of every table in zm_create.sql.in against the queries that read it.

db/zm_update-1.39.28.sql drops seven indexes:

Index	Why
Logs.TimeKey	Exact duplicate of Logs_TimeKey_idx, created three lines below it in zm_create.sql.in and added to upgraded installs by 1.31.11. Both present everywhere; every log INSERT maintains both.
Stats.MonitorId, Stats.ZoneId	Never read. Every query on Stats is anchored on EventId and EventId_ZoneId serves them. A row per frame per zone when ZM_RECORD_EVENT_STATS is on.
EncoderTemplates.Encoder	Leftmost prefix of Encoder_Name.
Role_Groups_Permissions_RoleId_idx, Role_Monitors_Permissions_RoleId_idx	Leftmost prefix of the UNIQUE index beside them.
Monitor_Status_UpdatedOn_idx	Dropped by 1.37.76 but left in zm_create.sql.in, so fresh and upgraded installs disagreed.

The Stats drops are guarded on there being no foreign key on the column: 1.37.31 drops the Stats foreign keys only when they are named Stats_ibfk_1..4, and where one survived under another name the drop would fail with errno 150 and abort the upgrade. Built that case and confirmed it skips with a message rather than aborting.

End-to-end test: master's zm_create.sql.in + 1.39.27 + 1.39.28 is index for index identical to a fresh install from the updated zm_create.sql.in — 123 indexes across 55 tables, all 29 foreign keys intact. Both migrations verified idempotent.

Checked and left alone: Events (all four indexes map to real query shapes), Events_Tags (the ON E.Id = ET.EventId join in the events list is covered by the Events_Tags_ibfk_2 foreign key index), Server_Stats (both indexes used), and the small tables. AI_Detections has three single-column indexes and no consumer anywhere in the tree yet, so there is nothing to judge them against — left for whenever that feature lands.

[Copilot]

🟡 Changes recommended

The PR metadata (title/description) does not match the actual version/migration scope (includes 1.39.28 and additional index drops), which should be reconciled before approval.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR updates the ZoneMinder database schema to reduce write overhead and improve query performance by consolidating and removing redundant secondary indexes (most notably on the high-write Frames table), while keeping fresh installs and upgraded installs aligned.

Changes:

• Add EventId_FrameId_idx (EventId, FrameId) to Frames and drop the previous EventId, Type, and TimeStamp secondary indexes via zm_update-1.39.27.sql.
• Drop several redundant/unused indexes on Logs, Stats, EncoderTemplates, role permission tables, and Monitor_Status via zm_update-1.39.28.sql, matching zm_create.sql.in.
• Bump the schema version to 1.39.28.

File summaries

File	Description
version.txt	Bumps schema version to 1.39.28.
db/zm_update-1.39.27.sql	Adds composite Frames(EventId, FrameId) index and drops redundant Frames secondary indexes.
db/zm_update-1.39.28.sql	Drops additional redundant/unused indexes across several tables (idempotent/guarded).
db/zm_create.sql.in	Updates fresh-install schema to match the post-migration index set.

Review details

• Files reviewed: 4/4 changed files
• Comments generated: 1
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[connortechnology]

Fair catch on the metadata — the title and description were written for the Frames change alone and were never updated when 11bade086 added 1.39.28. Both now cover the full scope: two migrations, version.txt at 1.39.28, and the reasoning for each index in one place rather than split between the description and a follow-up comment.

No code change; 1.39.27 and 1.39.28 are both intended, one per logical change so either can be reverted on its own.