Skip to content

fix(aiokafka): preserve partition=0 / offset=0 on consumer spans#18342

Open
piochelepiotr wants to merge 5 commits into
mainfrom
fix/aiokafka-falsy-partition-offset
Open

fix(aiokafka): preserve partition=0 / offset=0 on consumer spans#18342
piochelepiotr wants to merge 5 commits into
mainfrom
fix/aiokafka-falsy-partition-offset

Conversation

@piochelepiotr

Copy link
Copy Markdown
Contributor

Summary

  • _on_aiokafka_getone_message set kafka.partition and kafka.message_offset via value or -1, which collapses a legitimate 0 to -1. Single-partition topics (or the first message in any partition) were therefore consistently tagged with -1.
  • Replaced with explicit is not None checks so 0 passes through unchanged.

Test plan

  • Verified the regression locally with the data-streams docker demo (single-partition topic): consumer spans were tagged kafka.partition=-1 / kafka.message_offset=-1 despite real values being 0.
  • CI: existing aiokafka contrib suites should continue to pass.

🤖 Generated with Claude Code

@piochelepiotr piochelepiotr requested review from a team as code owners May 29, 2026 15:46
@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented May 29, 2026

Copy link
Copy Markdown

Codeowners resolved as

tests/snapshots/tests.contrib.aiokafka.test_aiokafka.test_send_and_wait_commit_with_offset.json  @DataDog/apm-python
tests/snapshots/tests.contrib.aiokafka.test_aiokafka.test_send_and_wait_key[None].json  @DataDog/apm-python
tests/snapshots/tests.contrib.aiokafka.test_aiokafka.test_send_and_wait_key[test_key].json  @DataDog/apm-python
tests/snapshots/tests.contrib.aiokafka.test_aiokafka.test_send_and_wait_value[None].json  @DataDog/apm-python
tests/snapshots/tests.contrib.aiokafka.test_aiokafka.test_send_and_wait_value[hueh_hueh_hueh].json  @DataDog/apm-python
tests/snapshots/tests.contrib.aiokafka.test_aiokafka.test_send_and_wait_with_distributed_tracing.json  @DataDog/apm-python
tests/snapshots/tests.contrib.aiokafka.test_aiokafka.test_send_commit.json  @DataDog/apm-python
tests/snapshots/tests.contrib.aiokafka.test_aiokafka_dsm.test_data_streams_aiokafka_enabled.json  @DataDog/apm-python

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f2df60973c

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread ddtrace/_trace/trace_handlers.py Outdated
span.set_tag(MESSAGE_KEY, message_key)

span._set_attribute(PARTITION, message.partition or -1)
span._set_attribute(PARTITION, message.partition if message.partition is not None else -1)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Update aiokafka snapshots for zero values

This change makes single-partition getone() consumer spans emit kafka.partition=0 and first-message spans emit kafka.message_offset=0, but the existing aiokafka snapshot fixtures still expect -1 for those metrics. I checked tests/contrib/aiokafka/utils.py, where create_topic() creates NewTopic(..., 1, 1), and the test_send_and_wait_* / test_send_commit snapshots still contain "kafka.partition": -1.0 and "kafka.message_offset": -1.0, so those snapshot tests will fail unless the fixtures are regenerated or updated alongside this code change.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in df57c98 — regenerated all aiokafka snapshots so they now expect kafka.partition: 0.0 and kafka.message_offset: 0.0 for single-partition / first-message consume spans.

@datadog-datadog-prod-us1-2

datadog-datadog-prod-us1-2 Bot commented May 29, 2026

Copy link
Copy Markdown
Contributor

Pipelines  Tests

Fix all issues with BitsAI

⚠️ Warnings

🚦 8 Pipeline jobs failed

DataDog/apm-reliability/dd-trace-py | build linux serverless: [amd64, cp315-cp315, v113741238-d2b8243-manylinux2014_x86_64, 1]   View in Datadog   GitLab

DataDog/apm-reliability/dd-trace-py | build linux serverless: [amd64, cp315-cp315, v113741491-d2b8243-musllinux_1_2_x86_64, 1]   View in Datadog   GitLab

DataDog/apm-reliability/dd-trace-py | build linux serverless: [arm64, cp315-cp315, v113741357-d2b8243-manylinux2014_aarch64, 1]   View in Datadog   GitLab

View all 8 failed jobs.

ℹ️ Info

No other issues found (see more)

🧪 All tests passed
❄️ No new flaky tests detected

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: a20daf5 | Docs | Datadog PR Page | Give us feedback!

@pr-commenter

pr-commenter Bot commented May 29, 2026

Copy link
Copy Markdown

Benchmarks

Benchmark execution time: 2026-06-11 21:57:59

Comparing candidate commit a20daf5 in PR branch fix/aiokafka-falsy-partition-offset with baseline commit 314cdf9 in branch main.

Found 0 performance improvements and 1 performance regressions! Performance is the same for 384 metrics, 9 unstable metrics.

scenario:span-start

  • 🟥 execution_time [+1.333ms; +1.515ms] or [+8.699%; +9.889%]

@piochelepiotr piochelepiotr reopened this May 29, 2026
@piochelepiotr piochelepiotr force-pushed the fix/aiokafka-falsy-partition-offset branch 2 times, most recently from 16e93ff to 4006398 Compare May 29, 2026 19:53

@brettlangdon brettlangdon left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just a question, but otherwise lgtm

Comment thread ddtrace/_trace/trace_handlers.py Outdated
piochelepiotr and others added 3 commits June 2, 2026 14:34
The aiokafka getone handler reported `kafka.partition` and
`kafka.message_offset` as -1 whenever the real value was 0, because it used
`value or -1` which treats the integer 0 as falsy. Single-partition topics
and the first message in any partition were therefore tagged with -1.

Replace with explicit None checks so that legitimate zero values pass
through unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…of using -1

Addresses reviewer feedback: -1 as a sentinel for unknown partition or offset
provides no real value and is misleading. Producer spans (where partition is not
yet known before broker assignment) and any span with a genuinely absent value
now omit these tags entirely.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@piochelepiotr piochelepiotr force-pushed the fix/aiokafka-falsy-partition-offset branch from e39743c to 0220612 Compare June 2, 2026 20:36
…fset values

After rebasing, consumer span snapshots were reset to main's state which still
had the stale -1.0 sentinel values. The code correctly produces 0.0 (first
message, single partition), so update snapshots to match.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants