Skip to content

[Enhancement]: Centralize MetaStore to eliminate metadata duplication in MixCoord mode #53578

Description

@tedxu

In MixCoord mode, DataCoord and QueryCoord maintain separate copies
of the same metadata (segments, channels, targets), leading to
inconsistencies, duplicated locking, and redundant metric tracking.

This enhancement introduces a centralized metacache package
(internal/metacache) with MetaView (read-only) and MetaStore
(read-write) interfaces backed by a single struct with sync.RWMutex.
Both DC and QC share one in-process store via constructor injection.

Key changes:

  • Move channel checkpoints into MetaStore with timestamp-based
    deduplication
  • Move segment persistence (file resources, compaction results)
    into MetaStore with metric emission
  • Eliminate segMetricMutation type from DC meta entirely
  • Improve metric emission to track all 3 label dimensions
    (state, level, sorted) instead of just state
  • Simplify DC meta to thin overlay owning only transient runtime
    state (allocations, compaction locks, flush timing, caches)
  • Move QC target management into shared MetaStore
  • Net deletion of ~550 lines of duplicated code

This is phase 1-2 of the metacache consolidation effort.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions