Skip to content

feature(gcp_pubsub): add extract_tracing_map / inject_tracing_map support#923

Open
jesperanzasolo wants to merge 2 commits into
warpstreamlabs:mainfrom
jesperanzasolo:gcp-pubsub-tracing-maps
Open

feature(gcp_pubsub): add extract_tracing_map / inject_tracing_map support#923
jesperanzasolo wants to merge 2 commits into
warpstreamlabs:mainfrom
jesperanzasolo:gcp-pubsub-tracing-maps

Conversation

@jesperanzasolo

@jesperanzasolo jesperanzasolo commented Jul 8, 2026

Copy link
Copy Markdown

What this adds

  • extract_tracing_map and new_root_span_with_link fields on the gcp_pubsub input.
  • inject_tracing_map field on the gcp_pubsub output.

All three use the existing generic tracing mechanism in public/service/config_extract_tracing.go and public/service/config_inject_tracing.go, so behavior and configuration match every other component that already exposes these fields.

Motivation

GCP Pub/Sub users currently have no config-driven way to propagate a W3C trace context across a Pub/Sub hop. The generic extract/inject tracing mapping already exists in Bento and is wired into kafka_franz (added in v1.17.0), kafka, and the nats components, but gcp_pubsub was never wired up.

Pub/Sub messages carry attributes (a string key/value map), which is the natural carrier for trace context, analogous to Kafka record headers. The input already copies message attributes into Bento metadata, and the output already emits filtered metadata as attributes, so the standard mapping wrappers slot in without any component-specific tracing code.

This is a small parity change rather than a new capability, so no issue was opened first ‚I'm happy to open a tracking issue if that's preferred.

How it mirrors the existing mechanism

  • Input: registers service.NewExtractTracingSpanMappingField() plus its companion service.NewRootSpanWithLinkField() (every existing registrant pairs them: nats, kafka, kafka_franz), and wraps the reader with conf.WrapInputExtractTracingSpanMapping("gcp_pubsub", r) in the constructor. This mirrors kafka_franz (internal/impl/kafka/input_kafka_franz.go), adjusted for the single-message RegisterInput API (WrapInput... rather than WrapBatchInput...). The new_root_span_with_link version tag assumes the next release is 1.19.0 ‚Äî happy to adjust.
  • Output: registers service.NewInjectTracingSpanMappingField() and wraps the writer with conf.WrapBatchOutputExtractTracingSpanMapping("gcp_pubsub", out), matching the kafka output (internal/impl/kafka/output_sarama_kafka.go).

Ordering matches the Kafka precedent: on read, attributes are copied into metadata first and the extraction mapping runs against that populated state; on write, the injection mapping runs before the writer builds Pub/Sub attributes from metadata, so injected span fields flow through to the wire attributes.

The field descriptions and examples come from the shared field constructors, so wording stays consistent with the other components.

Tests

  • go build ./... passes.
  • go test ./internal/impl/gcp/... passes.
  • gofmt -s, goimports -local github.com/warpstreamlabs/bento, and go vet are clean on the changed files; the full golangci-lint run (make lint) was not available in this environment and is left to CI.
  • make docs regenerated website/docs/components/inputs/gcp_pubsub.md and website/docs/components/outputs/gcp_pubsub.md; both are included in this PR.
  • make test-integration was not run ‚ it requires the Pub/Sub emulator, which wasn't available in this environment (mentioning per CONTRIBUTING.md).

No new component-level tests were added, matching the existing convention: the tracing mapping mechanism is tested generically in public/service/config_extract_tracing_test.go and public/service/config_inject_tracing_test.go, and no component (kafka_franz, kafka, nats) ships its own tracing-map test.

Note: the make docs lint step reports pre-existing failures in unrelated DuckDB/SQL components that require a CGO build tag (x_bento_extra) not available in this environment. Those are unaffected by this change; only the two gcp_pubsub docs were regenerated.

Checklist

  • Unit tested (go test ./internal/impl/gcp/...)
  • go build ./... passes
  • Formatted (gofmt -s + goimports -local; full make fmt/make lint left to CI as noted above)
  • Docs regenerated with make docs

No CHANGELOG entry is included: recent history shows changelog entries are batched by maintainers at release time rather than added in feature PRs.

@jesperanzasolo
jesperanzasolo force-pushed the gcp-pubsub-tracing-maps branch from e6e708f to ebfae6b Compare July 8, 2026 19:17
1.19.0 shipped without this change, so the next release carrying it is 1.20.0.
@jesperanzasolo

Copy link
Copy Markdown
Author

Hi @jem-davies @gregfurman, could I get a review here when you have a chance? Thanks!

@gregfurman

Copy link
Copy Markdown
Collaborator

@jesperanzasolo thanks for the ping! Will give this a look in the coming days

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants