Skip to content

libdatadog update to c79d783f#3983

Open
dd-octo-sts[bot] wants to merge 2 commits into
masterfrom
bot/libdatadog-latest
Open

libdatadog update to c79d783f#3983
dd-octo-sts[bot] wants to merge 2 commits into
masterfrom
bot/libdatadog-latest

Conversation

@dd-octo-sts

@dd-octo-sts dd-octo-sts Bot commented Jun 13, 2026

Copy link
Copy Markdown
Contributor

Summary

Automated update of the libdatadog submodule to the latest HEAD.

SHA
Previous $LIBDATADOG_PINNED_SHA
New c79d783f79f4a2d1e637906f3323600c6e2b5b17

Full CI result: ❌ 608 job(s) failed
CI pipeline: https://gitlab.ddbuild.io/DataDog/apm-reliability/dd-trace-php/-/pipelines/118728668


libdatadog Integration Report

libdatadog SHA: c79d783f79f4a2d1e637906f3323600c6e2b5b17
Analysis date: 2026-06-15

Overall status

⚠️ Adapted (API changes fixed)

A single libdatadog FFI signature change accounts for all 608 failing jobs. It was an
ABI mismatch that compiled cleanly but corrupted the stack at runtime, crashing the PHP
process on essentially every request. Adapting our checked-in C header and the one call
site fixes it.

Build & test summary

The delta for this update is small. master pins libdatadog 6760faae (green); this
branch (bot/libdatadog-latest) bumps it to c79d783f, adding exactly 6 commits:

commit relevance to dd-trace-php
c79d783f fix(ffe): honor shared fixture result metadata (#2109) New AssignmentReason::Default enum variant — already adapted by the bot in components-rs/ffe.rs. 0 FFE jobs failed.
bdc0a629 chore(crashtracking)!: remove frame count field (#2114) Removed with_experimental_frame_count; we never call it. No effect.
8907887c fix(trace-utils): mark decoded span maps as deduped (#2110) Additive sidecar-internal behavior. No effect.
aceec12b feat(sidecar): add retry interval configuration (#2106) Root cause — see below.
a21b946a feat(trace-utils)!: change buffer implementation (#2055) change_buffer is dd-trace-js-only (we don't use it); the only PHP-visible change is additive VecMap::{clear,drain}. No effect.
da8cbcb8 feat(shared-runtime)!: use weak waker in trigger (#2050) Internal runtime/fork hardening (macOS-motivated). Verified memory-safe by its own drop test. No effect.

Key observation: none of the 608 traces contained a Rust/C compilation error or a
Rust panic/backtrace — every one was a hard SIGSEGV (exit 139 / signal 11 / core dump)
in the php process itself, across all three sub-products:

  • tracer: php segfaults immediately on running the test suite (e.g.
    test_distributed_tracing segfaults even with datadog.trace.sidecar_trace_sender=0).
  • appsec: php-fpm children exit on signal 11 → HTTP 503 → every integration assertion
    fails; core dumps collected (core.php.*). The appsec extension .phpt suite itself
    passes 343/346, failing only client_init_sidecar.phpt — the one test that exercises
    sidecar session setup.
  • profiler: run-tests.php segfaults on startup.

A universal crash on the path common to all three (sidecar session initialization), with
no compile error and no panic, is the fingerprint of an FFI ABI mismatch.

Non-trivial changes made

Root cause: stale cbindgen header for ddog_sidecar_session_set_config

libdatadog #2106 inserted a new parameter into the FFI function, between
flush_interval_milliseconds and remote_config_poll_interval_millis:

// datadog-sidecar-ffi/src/lib.rs (new)
flush_interval_milliseconds: u32,
retry_interval_milliseconds: u32,   // <-- inserted
remote_config_poll_interval_millis: u32,

The C headers in components-rs/*.h are checked-in artifacts, regenerated only by the
make cbindgen target — not during the extension build. The bot bumped the submodule but
did not regenerate them, so components-rs/sidecar.h still declared the old signature, and
ext/sidecar.c still called it with the old argument list. Because the stale prototype and
the call site agreed, the C code compiled with no error.

At runtime, however, the linked libdatadog symbol reads arguments per the new ABI. Every
argument from the retry_interval slot onward is shifted by one — including the pointer/slice
arguments that follow (log_level, log_path, the remote-config product/capability pointers,
the notify function pointer, the tags vector, hostname/service). libdatadog dereferences these
misaligned (garbage) pointers during session setup → SIGSEGV. Since session config is set up
on essentially every process/request, this crashes everything that loads ddtrace.so (which is
all tracer, appsec and profiler jobs).

Fix (2 files):

  1. components-rs/sidecar.h — added uint32_t retry_interval_milliseconds, after
    uint32_t flush_interval_milliseconds, in the ddog_sidecar_session_set_config
    prototype, matching what make cbindgen would regenerate.

  2. ext/sidecar.c — passed a value for the new parameter at the call site. Used 100
    (milliseconds), which preserves prior behavior: before [BUG] Multiple unique TypeError's are being grouped as the same and have the same fingerprint #2106 the sidecar used
    RetryStrategy::default() whose delay_ms was Duration::from_millis(100). After the
    change the value is threaded through as RetryStrategy::new(5, retry_interval, Exponential, None), so 100 reproduces the previous default exactly. (dd-trace-php has
    no user-facing config for this yet; a dedicated DD_* setting can be wired up separately
    if desired.)

This was the only header-facing API change in the entire delta — verified by diffing
all extern "C" / #[no_mangle] / #[repr(C)] surfaces between 6760faae and c79d783f;
the only hit is retry_interval_milliseconds.

Identified libdatadog issues

None identified. The crash was caused by our stale checked-in header, not a libdatadog
defect. (#2106 is a legitimate, intentional FFI addition; the breakage is on the
integration side because the bot's automated bump does not run make cbindgen.)

Flaky / ignored failures

None. All 608 failures share the single root cause above and are expected to clear once the
sidecar config ABI is realigned. Notable downstream symptoms that are not separate issues:

  • appsec/system-test HTTP 503s and assert resp.statusCode() == 200 failures — secondary
    effects of the php-fpm worker crashing mid-request.
  • "Job failed: execution took longer than 1h0m0s" on the appsec helper-rust job — the test
    harness retries against a continuously-crashing worker until the job times out.

/cc @bwoebi

@dd-octo-sts dd-octo-sts Bot requested review from a team as code owners June 13, 2026 05:00
@dd-octo-sts dd-octo-sts Bot requested review from leoromanovsky and sameerank and removed request for a team June 13, 2026 05:00

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f826cc728b

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread libdatadog
@@ -1 +1 @@
Subproject commit 6760faaeeda1cfcf634410105f93cf7149265592
Subproject commit c79d783f79f4a2d1e637906f3323600c6e2b5b17

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Regenerate the sidecar FFI signature

This libdatadog bump changes ddog_sidecar_session_set_config by inserting retry_interval_milliseconds after flush_interval_milliseconds, but the checked-in components-rs/sidecar.h and the call in ext/sidecar.c still use the old argument list. Because C compiles against the stale header, this will not be caught at compile time; at runtime the new Rust FFI function will read every argument after the flush interval in the wrong slot, so normal sidecar startup can misconfigure intervals/sizes and eventually interpret non-pointer values as strings or callbacks. Please regenerate/update the header and pass the new retry interval at the call site as part of this bump.

Useful? React with 👍 / 👎.

Comment thread components-rs/ffe.rs
AssignmentReason::Static => REASON_STATIC,
AssignmentReason::TargetingMatch => REASON_TARGETING_MATCH,
AssignmentReason::Split => REASON_SPLIT,
AssignmentReason::Default => REASON_DEFAULT,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Handle invalid flag configs as defaults

The new libdatadog revision also adds EvaluationError::FlagConfigurationInvalid, which get_assignment can return for a requested flag whose per-flag config is invalid/unsupported; upstream FFE FFI maps that case to DEFAULT with no error so callers get their supplied default. This wrapper only added the new assignment reason, while the error match below still sends the new error through _ => (ERROR_GENERAL, REASON_ERROR), so those flags will now surface as evaluation errors instead of default evaluations. Please add an explicit arm for the new error variant.

Useful? React with 👍 / 👎.

@datadog-prod-us1-3

datadog-prod-us1-3 Bot commented Jun 13, 2026

Copy link
Copy Markdown

Pipelines  Tests

Fix all issues with BitsAI

⚠️ Warnings

🚦 58 Pipeline jobs failed

DataDog/apm-reliability/dd-trace-php | ASAN test_c: [8.4, arm64]   View in Datadog   GitLab

🧪 1 Test failed

tmp/build_extension/tests/ext/ffe/native_bridge_evaluate.phpt (FFE native bridge evaluates through libdatadog) from php.tmp.build_extension.tests.ext.ffe   View in Datadog (Fix with Cursor)
--
     empty_targeting_key={&#34;valueJson&#34;:&#34;\&#34;empty-targeting-key\&#34;&#34;,&#34;variant&#34;:&#34;empty-target&#34;,&#34;allocationKey&#34;:&#34;alloc-empty-targeting-key&#34;,&#34;reason&#34;:3,&#34;errorCode&#34;:0,&#34;doLog&#34;:true,&#34;providerState&#34;:[],&#34;errorMessage&#34;:null,&#34;hasConfig&#34;:null,&#34;configVersion&#34;:null}
     missing={&#34;valueJson&#34;:&#34;null&#34;,&#34;variant&#34;:null,&#34;allocationKey&#34;:null,&#34;reason&#34;:5,&#34;errorCode&#34;:3,&#34;doLog&#34;:false,&#34;providerState&#34;:[],&#34;errorMessage&#34;:null,&#34;hasConfig&#34;:null,&#34;configVersion&#34;:null}
     type_mismatch={&#34;valueJson&#34;:&#34;null&#34;,&#34;variant&#34;:null,&#34;allocationKey&#34;:null,&#34;reason&#34;:5,&#34;errorCode&#34;:1,&#34;doLog&#34;:false,&#34;providerState&#34;:[],&#34;errorMessage&#34;:null,&#34;hasConfig&#34;:null,&#34;configVersion&#34;:null}
015- parse_error={&#34;valueJson&#34;:&#34;null&#34;,&#34;variant&#34;:null,&#34;allocationKey&#34;:null,&#34;reason&#34;:5,&#34;errorCode&#34;:2,&#34;doLog&#34;:false,&#34;providerState&#34;:[],&#34;errorMessage&#34;:null,&#34;hasConfig&#34;:null,&#34;configVersion&#34;:null}
015&#43; parse_error={&#34;valueJson&#34;:&#34;null&#34;,&#34;variant&#34;:null,&#34;allocationKey&#34;:null,&#34;reason&#34;:5,&#34;errorCode&#34;:7,&#34;doLog&#34;:false,&#34;providerState&#34;:[],&#34;errorMessage&#34;:null,&#34;hasConfig&#34;:null,&#34;configVersion&#34;:null}

DataDog/apm-reliability/dd-trace-php | ASAN test_c: [7.4, arm64]   View in Datadog   GitLab

🧪 1 Test failed

All test failures are known flaky.

❄️ Known flaky: tmp/build_extension/tests/ext/ffe/native_bridge_evaluate.phpt (FFE native bridge evaluates through libdatadog) from PHP.tmp.build_extension.tests.ext.ffe   View in Datadog (Fix with Cursor)
015&#43; parse_error={&#34;valueJson&#34;:&#34;null&#34;,&#34;variant&#34;:null,&#34;allocationKey&#34;:null,&#34;reason&#34;:5,&#34;errorCode&#34;:7,&#34;doLog&#34;:false,&#34;providerState&#34;:[],&#34;errorMessage&#34;:null,&#34;hasConfig&#34;:null,&#34;configVersion&#34;:null}
015- parse_error={&#34;valueJson&#34;:&#34;null&#34;,&#34;variant&#34;:null,&#34;allocationKey&#34;:null,&#34;reason&#34;:5,&#34;errorCode&#34;:2,&#34;doLog&#34;:false,&#34;providerState&#34;:[],&#34;errorMessage&#34;:null,&#34;hasConfig&#34;:null,&#34;configVersion&#34;:null}

Not introduced in this PR.

DataDog/apm-reliability/dd-trace-php | ASAN test_c with multiple observers: [8.0]   View in Datadog   GitLab

View all 58 failed jobs.

ℹ️ Info

No other issues found (see more)

❄️ No new flaky tests detected

🎯 Code Coverage (details)
Patch Coverage: 100.00%
Overall Coverage: 54.08% (-0.04%)

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: b26baf8 | Docs | Datadog PR Page | Give us feedback!

@dd-octo-sts dd-octo-sts Bot requested a review from a team as a code owner June 13, 2026 10:13
@pr-commenter

pr-commenter Bot commented Jun 13, 2026

Copy link
Copy Markdown

Benchmarks [ tracer ]

Benchmark execution time: 2026-06-15 10:38:27

Comparing candidate commit b26baf8 in PR branch bot/libdatadog-latest with baseline commit 46641ee in branch master.

Found 0 performance improvements and 3 performance regressions! Performance is the same for 191 metrics, 0 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:MessagePackSerializationBench/benchMessagePackSerialization

  • 🟥 execution_time [+4.257µs; +6.363µs] or [+3.734%; +5.582%]

scenario:SamplingRuleMatchingBench/benchRegexMatching1

  • 🟥 execution_time [+31.300ns; +125.100ns] or [+2.075%; +8.295%]

scenario:SamplingRuleMatchingBench/benchRegexMatching4

  • 🟥 execution_time [+43.031ns; +163.769ns] or [+2.928%; +11.145%]

@dd-octo-sts dd-octo-sts Bot force-pushed the bot/libdatadog-latest branch from 122e2ec to 3a44de2 Compare June 14, 2026 04:59
@dd-octo-sts dd-octo-sts Bot force-pushed the bot/libdatadog-latest branch from d3d1bae to 9388ac2 Compare June 15, 2026 04:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants