feature(gcp_pubsub): add extract_tracing_map / inject_tracing_map support#923
Open
jesperanzasolo wants to merge 2 commits into
Open
feature(gcp_pubsub): add extract_tracing_map / inject_tracing_map support#923jesperanzasolo wants to merge 2 commits into
jesperanzasolo wants to merge 2 commits into
Conversation
jesperanzasolo
force-pushed
the
gcp-pubsub-tracing-maps
branch
from
July 8, 2026 19:17
e6e708f to
ebfae6b
Compare
1.19.0 shipped without this change, so the next release carrying it is 1.20.0.
Author
|
Hi @jem-davies @gregfurman, could I get a review here when you have a chance? Thanks! |
Collaborator
|
@jesperanzasolo thanks for the ping! Will give this a look in the coming days |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this adds
extract_tracing_mapandnew_root_span_with_linkfields on thegcp_pubsubinput.inject_tracing_mapfield on thegcp_pubsuboutput.All three use the existing generic tracing mechanism in
public/service/config_extract_tracing.goandpublic/service/config_inject_tracing.go, so behavior and configuration match every other component that already exposes these fields.Motivation
GCP Pub/Sub users currently have no config-driven way to propagate a W3C trace context across a Pub/Sub hop. The generic extract/inject tracing mapping already exists in Bento and is wired into
kafka_franz(added in v1.17.0),kafka, and thenatscomponents, butgcp_pubsubwas never wired up.Pub/Sub messages carry attributes (a string key/value map), which is the natural carrier for trace context, analogous to Kafka record headers. The input already copies message attributes into Bento metadata, and the output already emits filtered metadata as attributes, so the standard mapping wrappers slot in without any component-specific tracing code.
This is a small parity change rather than a new capability, so no issue was opened first ‚I'm happy to open a tracking issue if that's preferred.
How it mirrors the existing mechanism
service.NewExtractTracingSpanMappingField()plus its companionservice.NewRootSpanWithLinkField()(every existing registrant pairs them:nats,kafka,kafka_franz), and wraps the reader withconf.WrapInputExtractTracingSpanMapping("gcp_pubsub", r)in the constructor. This mirrorskafka_franz(internal/impl/kafka/input_kafka_franz.go), adjusted for the single-messageRegisterInputAPI (WrapInput...rather thanWrapBatchInput...). Thenew_root_span_with_linkversion tag assumes the next release is 1.19.0 — happy to adjust.service.NewInjectTracingSpanMappingField()and wraps the writer withconf.WrapBatchOutputExtractTracingSpanMapping("gcp_pubsub", out), matching thekafkaoutput (internal/impl/kafka/output_sarama_kafka.go).Ordering matches the Kafka precedent: on read, attributes are copied into metadata first and the extraction mapping runs against that populated state; on write, the injection mapping runs before the writer builds Pub/Sub attributes from metadata, so injected span fields flow through to the wire attributes.
The field descriptions and examples come from the shared field constructors, so wording stays consistent with the other components.
Tests
go build ./...passes.go test ./internal/impl/gcp/...passes.gofmt -s,goimports -local github.com/warpstreamlabs/bento, andgo vetare clean on the changed files; the fullgolangci-lintrun (make lint) was not available in this environment and is left to CI.make docsregeneratedwebsite/docs/components/inputs/gcp_pubsub.mdandwebsite/docs/components/outputs/gcp_pubsub.md; both are included in this PR.make test-integrationwas not run ‚ it requires the Pub/Sub emulator, which wasn't available in this environment (mentioning per CONTRIBUTING.md).No new component-level tests were added, matching the existing convention: the tracing mapping mechanism is tested generically in
public/service/config_extract_tracing_test.goandpublic/service/config_inject_tracing_test.go, and no component (kafka_franz,kafka,nats) ships its own tracing-map test.Note: the
make docslint step reports pre-existing failures in unrelated DuckDB/SQL components that require a CGO build tag (x_bento_extra) not available in this environment. Those are unaffected by this change; only the twogcp_pubsubdocs were regenerated.Checklist
go test ./internal/impl/gcp/...)go build ./...passesgofmt -s+goimports -local; fullmake fmt/make lintleft to CI as noted above)make docsNo CHANGELOG entry is included: recent history shows changelog entries are batched by maintainers at release time rather than added in feature PRs.