feat: perf test otel on#7792
Draft
pratikmankawde wants to merge 1430 commits into
Draft
Conversation
…tance exported_instance is a Prometheus scrape artifact that only exists when Prometheus scrapes a target with a conflicting instance label. Users running push-based pipelines (remote_write, OTLP) never get this label. service_instance_id is always present via resource_to_telemetry_conversion regardless of ingestion path. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…se8-log-correlation
…tance Merge forward from phase6/7 and apply the same fix to phase9's own dashboards and alerting rules. exported_instance is a Prometheus scrape artifact absent in push-based pipelines; service_instance_id is always present via resource_to_telemetry_conversion. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ase10-workload-validation
…d docs Update the transform/cloudlabels processor to set service_instance_id instead of exported_instance, matching the local Prometheus path where resource_to_telemetry_conversion promotes service.instance.id with that exact label name. Also update source comments and documentation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…se8-log-correlation
…ase9-metric-gap-fill
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ase10-workload-validation
Add missing <cstdint> include for std::uint32_t in Telemetry.h. Add braces around single-line if bodies in Telemetry.cpp. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…hase1c-rpc-integration
…hase2-rpc-tracing
Replace std::transform with std::ranges::transform in Redaction.cpp. Replace std::all_of with std::ranges::all_of in Redaction test. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove unused SpanNames.h from Transactor.cpp (transitively included via TxApplySpanNames.h). Remove unused <ranges> from TraceContextValidation.h (std::ranges::any_of lives in <algorithm>). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…consensus-tracing
Add missing <string_view> include in Consensus.h and RCLConsensus.h. Remove unused SHAMap.h include from RCLConsensus.h. Add braces around single-line if/else bodies in RCLConsensus.cpp. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Rename rippled → xrpld in statsd config prefix. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…phase5-docs-deployment # Conflicts: # src/libxrpl/telemetry/Telemetry.cpp
- Add job_type to otel-naming builtins (standard Prometheus label for job-queue metrics used in node-health dashboard) - Add OpenTelemetry SDK to UBSAN suppressions (intentional unsigned integer overflow in attributemap_hash.h hash computation) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…se8-log-correlation
…ase9-metric-gap-fill # Conflicts: # docker/telemetry/xrpld-telemetry.cfg
…errors - Replace unused observer_result.h with nostd/shared_ptr.h and nostd/unique_ptr.h in MetricsRegistry.h (misc-include-cleaner) - Use auto for cast initializations in MetricsRegistry.cpp and ValidationTracker test (modernize-use-auto) - Run rename scripts: rippled -> xrpld in comments and docs Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ase10-workload-validation
Raise TX error rate threshold from 50% to 95%. Short-lived CI test environments lack pre-funded accounts for complex transactions (AMMCreate, EscrowFinish, NFTokenCreateOffer, etc.), causing expected failures that do not indicate an instrumentation problem. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…se8-log-correlation
The docker/telemetry/alloy/config.alloy node config feeds Grafana Cloud via the OTLP gateway (which promotes resource attributes to labels server-side). Add the resource_group / iac_group build to its metric resource statements so nodes using this Alloy path get the same grouped legend labels the dashboards expect. This is a generic node config (no perf-iac attrs), so iac_group's guard never fires here and only resource_group is built; the iac_group statement is kept for parity and activates automatically on a perf-comparison node. Guards check != "" as well as != nil since OTTL Concat emits a stray separator for an empty element. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
In the Alloy config the spanmetrics connector derives span_calls_total / span_duration_* from the tagged TRACES and sends them straight to the batch processor, bypassing the metric resource statements. The grouped legend labels were built only in metric_statements, so span-derived RED metrics carried no resource_group / iac_group and span-based dashboard panels rendered empty group legends. Build the same labels in trace_statements too (mirroring how the tier attrs are already set in both), so spans carry them before spanmetrics derives from them. The reference otel collector is unaffected: there spanmetrics is a connector feeding the metrics pipeline, which runs transform/legendgroups on all inputs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Panels that group by a distinguishing dimension (consensus_mode, tx_type,
command, method, stage, ...) already fold that dimension's value into the
curated series name via label_replace(..., "series", "<prefix> $1", "<dim>",
"(.*)"). The migration also added a separate [${__field.labels.<dim>}] bracket,
so those legends rendered the value twice, e.g. "proposing [proposing] [...]"
on Consensus Mode Over Time.
Remove the dimension bracket from all 29 affected panel Display names, leaving
${__field.labels.series} [resource_group] [iac_group]; the series name already
conveys the dimension. The migration script no longer emits the bracket either.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ase10-workload-validation
The Display name templated the brackets: "${series} [${resource_group}]
[${iac_group}]". When a group label was absent (e.g. iac_group on a non-perf
node) the "${}" rendered empty, leaving a literal "[]" in the legend. The
empty-bracket cleanup renameByRegex could not remove it: Grafana runs that
transform on the pre-interpolation Display name template, which contains no
"[]" -- the "[]" only appears after variable interpolation, at render time.
Fix it at the source: the collector now bakes the brackets and comma separators
into the group-label VALUES (e.g. resource_group="[node, mainnet]",
iac_group="[branch, role, work-item]"). The Display name becomes bracket-free --
"${series} ${resource_group} ${iac_group}" -- so an absent group is simply an
absent label and renders as nothing. Also switches the intra-group separator
from "-" to ", " (e.g. "[node, mainnet]"). Drop the now-dead cleanup transform
from every panel; value-mapping renameByRegex rules are preserved.
Collector-side label building lives in the paired commits on the collector
config branches. This commit is the dashboard half for the eleven dashboards
this branch owns.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Display name templated the brackets: "${series} [${resource_group}]
[${iac_group}]". When a group label was absent the "${}" rendered empty,
leaving a literal "[]" in the legend, and the cleanup renameByRegex could not
remove it (Grafana runs that transform on the pre-interpolation template, which
has no "[]" -- it only appears after interpolation at render time).
Bake the brackets and comma separators into the group-label VALUES in the
collector instead (resource_group="[node, mainnet]",
iac_group="[branch, role, work-item]"), so the Display name is bracket-free
("${series} ${resource_group} ${iac_group}") and an absent group renders as
nothing. Separator changes from "-" to ", ". Drop the dead cleanup transform.
Covers this branch's local collector config and its three dashboards
(ledger-data-sync, network-traffic, overlay-traffic-detail).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Mirror the local collector change on the Grafana Cloud collector config and the in-repo Alloy config: build the group-label VALUES with brackets and comma separators (resource_group="[node, mainnet]", iac_group="[branch, role, work-item]") in both trace and metric resource statements, so span-derived and native metrics carry the same bracketed values. This lets the dashboards use a bracket-free Display name and avoids the empty "[]" that a templated bracket would leave when a group is absent (Grafana's renameByRegex runs pre-interpolation and cannot strip it). Separator changes from "-" to ", ". Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ase9-metric-gap-fill
…se8-log-correlation
…ase10-workload-validation # Conflicts: # docker/telemetry/grafana/dashboards/ledger-data-sync.json
Update the config.alloy metric_statements comment to describe the current
bracket-baked, ", "-joined group-label values ("[node, mainnet]"), matching the
implementation. Comment-only; no behavior change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…up labels
The collector-built resource_group / iac_group labels did not survive the
dashboards' own PromQL aggregation: panels that sum by(service_instance_id,
xrpl_branch, xrpl_node_role, ...) drop any label not named in the by() clause,
so the derived group labels were aggregated away and the Display name rendered
empty ("Observing" with no node). Building them in the collector also coupled
every dashboard to a lockstep collector redeploy on every node.
Switch the legend Display name to reference the raw labels the panels already
group by, which are always present by construction and need no collector
support:
${__field.labels.series} ${__field.labels.service_instance_id}
${__field.labels.xrpl_branch} ${__field.labels.xrpl_node_role}
${__field.labels.xrpl_work_item}
${__field.labels.X} renders empty for a label a series lacks (verified in
Grafana), so dev/mainnet nodes show just "<name> <node>" and perf nodes add the
branch/role/work-item, with no empty brackets and no double spaces. Verified
live across timeseries, stat, bargauge, piechart, gauge and cross-node
aggregation panels.
Revert the transform/legendgroups processor from the local collector config;
this file is now identical to its pre-change state.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…up labels
The collector-built resource_group / iac_group labels did not survive the
dashboards' own PromQL aggregation: panels that sum by(service_instance_id,
xrpl_branch, xrpl_node_role, ...) drop any label not in the by() clause, so the
derived group labels were aggregated away and the Display name rendered empty.
Switch the legend Display name to the raw labels the panels already group by,
which are always present by construction:
${__field.labels.series} ${__field.labels.service_instance_id}
${__field.labels.xrpl_branch} ${__field.labels.xrpl_node_role}
${__field.labels.xrpl_work_item}
Absent labels render empty (no brackets, no gaps). Covers the eleven dashboards
this branch owns. The matching collector-config revert lands on the branches
that introduced those configs (phase-7 local, phase-10 cloud/alloy).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…figs) The legend now uses raw labels in the dashboard Display name (see the paired dashboard commit), so the collector no longer needs to build resource_group / iac_group. Those derived labels also could not survive the dashboards' sum by() aggregation, which is why the grouped approach was dropped. Revert transform/legendgroups and its transform/cloudlabels datapoint copies from the Grafana Cloud collector config and the in-repo Alloy config; both files are now identical to their pre-change state. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…se8-log-correlation # Conflicts: # docker/telemetry/otel-collector-config.yaml
…ase9-metric-gap-fill
…ase10-workload-validation
Group the node + perf-iac identity labels inside one pair of brackets in the
legend Display name:
${__field.labels.series} [${__field.labels.service_instance_id} ${__field.labels.xrpl_branch} ${__field.labels.xrpl_node_role} ${__field.labels.xrpl_work_item}]
so a series reads e.g. "Queue Depth [validator-0 test:pr:abc123 validator
RIPD-7455]" on a perf node and "Queue Depth [aws-dev-xrpl-1]" on a plain node.
Absent perf-iac labels render empty and collapse to a single space before the
closing bracket (HTML/SVG whitespace collapse) — the only cosmetic artifact on
non-perf nodes; no empty "[]" and no data impact. Verified on a live local
Grafana render.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Group the node + perf-iac identity labels inside one pair of brackets in the legend Display name so a series reads e.g. "Queue Depth [validator-0 test:pr:abc123 validator RIPD-7455]" on a perf node and "Queue Depth [aws-dev-xrpl-1]" on a plain node. Absent perf-iac labels collapse to a single space before the closing bracket; no empty "[]", no data impact. Covers the eleven dashboards this branch owns. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…se8-log-correlation
…ase9-metric-gap-fill
…ase10-workload-validation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Temp. PR.