Skip to content

PMM-15249 ethtool name collision. - #1330

Open
JiriCtvrtka wants to merge 14 commits into
mainfrom
PMM-15249-ethtool-name-collision
Open

PMM-15249 ethtool name collision.#1330
JiriCtvrtka wants to merge 14 commits into
mainfrom
PMM-15249-ethtool-name-collision

Conversation

@JiriCtvrtka

@JiriCtvrtka JiriCtvrtka commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

PMM-15249

Problem

On MongoDB 8.3 the getDiagnosticData reply contains systemMetrics.ethtool counters whose names
differ only in characters that prometheusize collapses. A vmxnet3 NIC, for example, reports both
" giant hdr" and " giant hdr", and both map to
mongodb_sys_ethtool_ens192_giant_hdr with a different help string.
The registry then rejects the scrape with collected metric ... was collected before with the same name and label values, so the whole systemMetrics tree is lost, not just the offending counters.

Solution

Colliding fields of one document are now exported under a single canonical metric name, and the new
metric_field label carries the document field each series was built from:

  • Collision detection uses prometheusize itself, because anything less misses cases — fields
    differing only in a trailing special character or in a run of underscores share a name too.
  • The canonical field is the lowest name of the group. BSON order is lost when the document is
    decoded into a map, so the name has to be derived deterministically to stay stable between
    scrapes (and not to renumber when MongoDB stops reporting one of the fields).
  • Whole groups share one name and one help string, which is what the registry requires from a
    metric family.
  • Documents whose keys become a label value (index names, nodeToPDMetrics) are skipped: their keys
    never reach the metric name, and labelling only a part of such a family would make the descriptor
    inconsistent.
  • A cheap isUnambiguousKey pre-check keeps the collision scan off the hot path — almost everything
    MongoDB reports passes through prometheusize unchanged.
  • Label maps are copied instead of mutated in place, so siblings no longer share mutable state.
    Metrics with a unique name are unaffected and do not get the extra label.

Tests

  • testdata/get_diagnostic_data_8.3.json — a reply captured from the MongoDB 8.3.2 instance from
    Collected Metric was collected before with the same name and label values with MongoDB 8.3.1 #1285, trimmed to the relevant subtrees. It is parsed as extended JSON, otherwise arrays decode
    to []any and makeMetrics silently walks past everything array shaped (histogram buckets
    included).
  • All metric tests now gather through a pedantic registry, so duplicated series and inconsistent
    descriptors fail the test instead of the scrape.
  • The full 8.3 payload is gathered both with and without histograms.
  • Unit tests for collidingFields and isUnambiguousKey cover whitespace, other special
    characters, underscore runs and leading/trailing underscores.

@JiriCtvrtka

Copy link
Copy Markdown
Contributor Author

@copilot review

@JiriCtvrtka
JiriCtvrtka force-pushed the PMM-15249-ethtool-name-collision branch from feab3ec to 35abe00 Compare July 27, 2026 09:49
@JiriCtvrtka

Copy link
Copy Markdown
Contributor Author

@copilot review

@JiriCtvrtka

Copy link
Copy Markdown
Contributor Author

@copilot review

@JiriCtvrtka
JiriCtvrtka marked this pull request as ready for review August 4, 2026 06:39
@JiriCtvrtka
JiriCtvrtka requested a review from a team as a code owner August 4, 2026 06:39
@JiriCtvrtka
JiriCtvrtka requested review from ademidoff and maxkondr and removed request for a team August 4, 2026 06:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Collected Metric was collected before with the same name and label values with MongoDB 8.3.1

1 participant