Skip to content

Record Basilisk build toolchain and ABI metadata#1474

Merged
schaubh merged 8 commits into
developfrom
feature/bsk_toolchain_log
Jul 12, 2026
Merged

Record Basilisk build toolchain and ABI metadata#1474
schaubh merged 8 commits into
developfrom
feature/bsk_toolchain_log

Conversation

@schaubh

@schaubh schaubh commented Jul 10, 2026

Copy link
Copy Markdown
Contributor
  • Review: By commit
  • Merge strategy: Merge (no squash)

Description

This PR adds build toolchain and compiled ABI metadata reporting to the installed Basilisk Python package. Basilisk now exposes getBuildInfo() and printBuildInfo() at package import level so users, developers, and future BSK-SDK compatibility checks can inspect the compiler, target platform, Python ABI, SWIG runtime, Eigen settings, runtime linkage, C++ standard library, and selected C/C++ layout canaries used to build the package.

The ABI contract is centralized in architecture/utilities/bskAbiDescriptor.h, which owns the descriptor version, plugin ABI version, canary types, and C/C++ extraction helpers. CMake builds a small compiled probe from this shared contract so the recorded ABI data reflects the actual compiler and target settings rather than only configure-time guesses.

Commits are organized as:

  1. Build metadata and ABI descriptor implementation with tests.
  2. User/developer documentation and release-note snippet.

Verification

Validated locally with the Basilisk virtual environment:

  • pytest -q src/tests/test_buildInfo.py
  • clang-format --dry-run --Werror on the new C/C++ files
  • python -m compileall on generated build-info Python files
  • git diff --check
  • Clean minimal wheel build with CONAN_ARGS="--opNav False --mujoco False --clean"
  • Isolated wheel import check for Basilisk.getBuildInfo() and Basilisk.printBuildInfo()
  • Incremental metadata target and module-only build checks

Added src/tests/test_buildInfo.py to cover metadata shape, public ABI descriptor constants, C layout canaries, platform-specific ABI fields, defensive copy behavior, and formatted build-info output.

Documentation

Updated the build documentation to describe printBuildInfo(), getBuildInfo(), the meaning of the artifact, abi, and diagnostics sections, and how this metadata should and should not be used for future SDK/plugin compatibility decisions.

Added a release-note snippet for the new build toolchain metadata feature.

Reviewers should check the install-build documentation for clarity around:

  • exact-version compatibility still being required today,
  • build-tool metadata versus ABI compatibility metadata,
  • Debug/Release and multi-config generator behavior.

Future work

Use this metadata on the BSK-SDK side to compare plugin build settings against the loaded Basilisk package and emit clear compatibility diagnostics.

If Basilisk starts shipping true multi-slice macOS universal2 binaries, extend the ABI metadata schema to record per-architecture ABI descriptors rather than a single executed architecture.

@schaubh schaubh self-assigned this Jul 10, 2026
@schaubh schaubh requested a review from a team as a code owner July 10, 2026 19:18
@schaubh schaubh added this to Basilisk Jul 10, 2026
@schaubh schaubh added documentation Improvements or additions to documentation enhancement New feature or request build Build system or compilation enhancement labels Jul 10, 2026
@schaubh schaubh moved this to 👀 In review in Basilisk Jul 10, 2026
schaubh added a commit that referenced this pull request Jul 10, 2026
Add a generated build-info path that records the compiler, target, runtime,
Python ABI, SWIG runtime, Eigen settings, and selected C/C++ layout canaries
for an installed Basilisk package.  The metadata is exposed through
Basilisk.getBuildInfo() and Basilisk.printBuildInfo() so users and future
BSK-SDK compatibility checks can inspect the binary that was actually built.

Add architecture/utilities/bskAbiDescriptor.h as the public ABI descriptor
contract shared by Basilisk and SDK-side probes.  The header owns the descriptor
version, plugin ABI version, canary types, and C/C++ extraction helpers so the
two sides do not need to maintain parallel ABI-detection logic.

Generate the runtime ABI payload with a small compiled C/C++ probe.  The probe
uses the same public descriptor header and shared bskPluginAbiSettings interface
as native Basilisk targets, so it observes the same C++ standard, Python limited
API definition, Eigen include/link settings, and compiler behavior.

Wire metadata generation into the CMake build and refresh it for loadable module
targets.  Keep the generation dependency off internal static/shared libraries to
avoid multi-config generated-source dependency issues, while still applying the
shared ABI settings interface to native targets.

Pass relevant Conan and Python build settings into CMake, including the selected
Basilisk version, Conan version, C++ standard/library/runtime settings, build
type, and invoking Python executable.

Add tests covering the public descriptor contract, compiled ABI payload shape,
C layout canaries, platform-specific C++ ABI fields, defensive copy behavior,
and human-readable build-info formatting.
schaubh added a commit that referenced this pull request Jul 10, 2026
schaubh added a commit that referenced this pull request Jul 10, 2026
@schaubh schaubh force-pushed the feature/bsk_toolchain_log branch from ddce204 to 38b4dbe Compare July 10, 2026 19:19

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ddce2047af

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/cmake/bskBuildInfo.cmake
schaubh added a commit that referenced this pull request Jul 10, 2026
schaubh added a commit that referenced this pull request Jul 10, 2026
@schaubh schaubh force-pushed the feature/bsk_toolchain_log branch from 38b4dbe to 0e02212 Compare July 10, 2026 19:46
schaubh added 3 commits July 10, 2026 14:04
Add a generated build-info path that records the compiler, target, runtime,
Python ABI, SWIG runtime, Eigen settings, and selected C/C++ layout canaries
for an installed Basilisk package.  The metadata is exposed through
Basilisk.getBuildInfo() and Basilisk.printBuildInfo() so users and future
BSK-SDK compatibility checks can inspect the binary that was actually built.

Add architecture/utilities/bskAbiDescriptor.h as the public ABI descriptor
contract shared by Basilisk and SDK-side probes.  The header owns the descriptor
version, plugin ABI version, canary types, and C/C++ extraction helpers so the
two sides do not need to maintain parallel ABI-detection logic.

Generate the runtime ABI payload with a small compiled C/C++ probe.  The probe
uses the same public descriptor header and shared bskPluginAbiSettings interface
as native Basilisk targets, so it observes the same C++ standard, Python limited
API definition, Eigen include/link settings, and compiler behavior.

Wire metadata generation into the CMake build and refresh it for loadable module
targets.  Keep the generation dependency off internal static/shared libraries to
avoid multi-config generated-source dependency issues, while still applying the
shared ABI settings interface to native targets.

Pass relevant Conan and Python build settings into CMake, including the selected
Basilisk version, Conan version, C++ standard/library/runtime settings, build
type, and invoking Python executable.

Add tests covering the public descriptor contract, compiled ABI payload shape,
C layout canaries, platform-specific C++ ABI fields, defensive copy behavior,
and human-readable build-info formatting.
@schaubh schaubh force-pushed the feature/bsk_toolchain_log branch from 0e02212 to 284b777 Compare July 10, 2026 20:04
schaubh added 5 commits July 10, 2026 20:48
Link bskBuildInfoProbe against the Python module target selected by the
main Basilisk build. This uses Python3::SABIModule when the limited API
is enabled and Python3::Module otherwise.

The dependency supplies the correct Python import library on Windows,
preventing MSVC from failing to locate python3.lib when Python.h requests
the Stable ABI library. On Linux and macOS, CMake retains the appropriate
platform-specific module-linking behavior.

Verified by rebuilding bskGenerateBuildInfo and running
src/tests/test_buildInfo.py with all 8 tests passing.
Update test_print_build_info to account for the human-readable standard
library label used by the formatter. The compiled ABI metadata stores the
canonical Windows family as "msvc", while printBuildInfo displays it as
"MSVC STL".

Continue checking the canonical labels directly on libc++ and libstdc++
platforms.

Verified with all 8 build information tests and pre-commit checks passing.
Restore Conan dependencies before building and save them immediately after a successful build so later test failures do not discard the cache. Use stable, platform-specific cache keys that are unaffected by ordinary CMake changes, retain compatibility with existing caches, and report sccache hit and miss statistics for Linux, macOS, and Windows builds.
Generate Conan cache keys from the detected toolchain, Conan version, runner architecture, and build options while retaining legacy cache fallbacks. Refresh Conan profiles after restoration, make cache failures nonfatal, and report the 25 slowest pytest cases in CI.
Share Conan dependencies across Python and build variants, restrict cache writes to one producer per platform, and warm reusable base-branch caches after merges to reduce cache churn and eviction.
@schaubh schaubh merged commit 9911687 into develop Jul 12, 2026
19 checks passed
schaubh added a commit that referenced this pull request Jul 12, 2026
Add a generated build-info path that records the compiler, target, runtime,
Python ABI, SWIG runtime, Eigen settings, and selected C/C++ layout canaries
for an installed Basilisk package.  The metadata is exposed through
Basilisk.getBuildInfo() and Basilisk.printBuildInfo() so users and future
BSK-SDK compatibility checks can inspect the binary that was actually built.

Add architecture/utilities/bskAbiDescriptor.h as the public ABI descriptor
contract shared by Basilisk and SDK-side probes.  The header owns the descriptor
version, plugin ABI version, canary types, and C/C++ extraction helpers so the
two sides do not need to maintain parallel ABI-detection logic.

Generate the runtime ABI payload with a small compiled C/C++ probe.  The probe
uses the same public descriptor header and shared bskPluginAbiSettings interface
as native Basilisk targets, so it observes the same C++ standard, Python limited
API definition, Eigen include/link settings, and compiler behavior.

Wire metadata generation into the CMake build and refresh it for loadable module
targets.  Keep the generation dependency off internal static/shared libraries to
avoid multi-config generated-source dependency issues, while still applying the
shared ABI settings interface to native targets.

Pass relevant Conan and Python build settings into CMake, including the selected
Basilisk version, Conan version, C++ standard/library/runtime settings, build
type, and invoking Python executable.

Add tests covering the public descriptor contract, compiled ABI payload shape,
C layout canaries, platform-specific C++ ABI fields, defensive copy behavior,
and human-readable build-info formatting.
schaubh added a commit that referenced this pull request Jul 12, 2026
schaubh added a commit that referenced this pull request Jul 12, 2026
schaubh added a commit that referenced this pull request Jul 12, 2026
Link bskBuildInfoProbe against the Python module target selected by the
main Basilisk build. This uses Python3::SABIModule when the limited API
is enabled and Python3::Module otherwise.

The dependency supplies the correct Python import library on Windows,
preventing MSVC from failing to locate python3.lib when Python.h requests
the Stable ABI library. On Linux and macOS, CMake retains the appropriate
platform-specific module-linking behavior.

Verified by rebuilding bskGenerateBuildInfo and running
src/tests/test_buildInfo.py with all 8 tests passing.
schaubh added a commit that referenced this pull request Jul 12, 2026
Update test_print_build_info to account for the human-readable standard
library label used by the formatter. The compiled ABI metadata stores the
canonical Windows family as "msvc", while printBuildInfo displays it as
"MSVC STL".

Continue checking the canonical labels directly on libc++ and libstdc++
platforms.

Verified with all 8 build information tests and pre-commit checks passing.
schaubh added a commit that referenced this pull request Jul 12, 2026
Restore Conan dependencies before building and save them immediately after a successful build so later test failures do not discard the cache. Use stable, platform-specific cache keys that are unaffected by ordinary CMake changes, retain compatibility with existing caches, and report sccache hit and miss statistics for Linux, macOS, and Windows builds.
@github-project-automation github-project-automation Bot moved this from 👀 In review to ✅ Done in Basilisk Jul 12, 2026
schaubh added a commit that referenced this pull request Jul 12, 2026
Generate Conan cache keys from the detected toolchain, Conan version, runner architecture, and build options while retaining legacy cache fallbacks. Refresh Conan profiles after restoration, make cache failures nonfatal, and report the 25 slowest pytest cases in CI.
@schaubh schaubh deleted the feature/bsk_toolchain_log branch July 12, 2026 17:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

build Build system or compilation enhancement documentation Improvements or additions to documentation enhancement New feature or request

Projects

Status: ✅ Done

Development

Successfully merging this pull request may close these issues.

1 participant