Skip to content

build: include libmtmd in Apple XCFramework (opt-in LLAMA_BUILD_MTMD)#21935

Open
theabecaster wants to merge 1 commit intoggml-org:masterfrom
theabecaster:upstream/libmtmd-xcframework
Open

build: include libmtmd in Apple XCFramework (opt-in LLAMA_BUILD_MTMD)#21935
theabecaster wants to merge 1 commit intoggml-org:masterfrom
theabecaster:upstream/libmtmd-xcframework

Conversation

@theabecaster
Copy link
Copy Markdown

@theabecaster theabecaster commented Apr 15, 2026

The Apple xcframework from build-xcframework.sh is built with LLAMA_BUILD_TOOLS=OFF, so no mtmd_* symbols end up in it. That means Swift / Obj-C consumers on iOS, macOS, tvOS, visionOS can't use vision even though the sources cross-compile fine.

This adds an opt-in LLAMA_BUILD_MTMD option (default OFF). When on and full tools are off, only the tools/mtmd subdirectory gets added so just the library target builds. The CLI exe targets in tools/mtmd/CMakeLists.txt are now gated on LLAMA_BUILD_TOOLS — without that, standalone visionOS xcodebuild fails with install TARGETS given no BUNDLE DESTINATION for MACOSX_BUNDLE executable target 'llama-mtmd-cli'.

build-xcframework.sh turns the option on and adds libmtmd.a to combine_static_libraries.

Headers aren't exposed via the framework module map. mtmd.h / mtmd-helper.h have C++ bits (method-bearing structs, struct mtmd_decoder_pos without the struct tag) that break pure-C clang module precompilation for Swift importers. Consumers declare the ABI via their own extern "C" shim — same workaround people use today.

Tested:

  • build-xcframework.sh passes on macos-15 for all 8 platform slices
  • nm ios-arm64/llama.framework/llama | grep -c ' _mtmd_' → 52
  • Imports into an SPM iOS 17 app (Xcode 16.4, Swift 6 strict concurrency); runtime dlsym(RTLD_DEFAULT, "mtmd_init_from_file") is non-NULL on iPhone 15 Pro
  • Legacy path (LLAMA_BUILD_MTMD=OFF or LLAMA_BUILD_TOOLS=ON) unchanged — full-tools build still pulls mtmd in via tools/

Disclosure: I used Claude to help draft the CMake wording. The design, debugging, and on-device verification are mine.

@theabecaster theabecaster requested review from a team, danbev and ggerganov as code owners April 15, 2026 04:42
@github-actions github-actions bot added build Compilation issues examples labels Apr 15, 2026
@ggml-gh-bot
Copy link
Copy Markdown

ggml-gh-bot bot commented Apr 15, 2026

Hi @theabecaster, thanks for your contribution!

Per our contribution guidelines, the automated PR checker found the following issue(s) that need your attention:

  • AI-generated content: This project does not accept PRs, descriptions or commit messages that are fully or predominantly AI-generated. If you have used AI to assist you in writing code, please make sure to disclose that explicitly.

Please note that maintainers reserve the right to make final decisions on PRs. If you believe there is a mistake, please comment below.

Adds opt-in LLAMA_BUILD_MTMD CMake option so build-xcframework.sh can
link libmtmd.a into the framework binary without pulling in the rest
of tools/ (which doesn't cross-build cleanly to iOS/tvOS/visionOS).

- CMakeLists.txt: new option, default OFF. When on with
  LLAMA_BUILD_TOOLS=OFF, only the tools/mtmd subdir is added.
- tools/mtmd/CMakeLists.txt: gate the CLI exe targets on
  LLAMA_BUILD_TOOLS. Gating on LLAMA_BUILD_COMMON is not enough: it
  defaults ON in standalone builds and visionOS xcodebuild then
  fails with "install TARGETS given no BUNDLE DESTINATION for
  MACOSX_BUNDLE executable target 'llama-mtmd-cli'".
- build-xcframework.sh: turn the option on, pass -DLLAMA_BUILD_MTMD,
  add libmtmd.a to combine_static_libraries.

Module map is not updated to expose mtmd.h / mtmd-helper.h. Those
headers use C++ constructs and reference mtmd_decoder_pos without
the struct tag, which break clang's pure-C module precompilation
path for Swift / Obj-C importers. Consumers declare the ABI via
their own extern "C" shim.

After this, nm on ios-arm64/llama.framework/llama shows 52 _mtmd_
symbols.

AI-assisted: used Claude to help with CMake wording. Design,
debugging, and on-device verification are mine.
@theabecaster theabecaster force-pushed the upstream/libmtmd-xcframework branch from 507dea0 to 0d1ebf7 Compare April 16, 2026 16:50
@theabecaster
Copy link
Copy Markdown
Author

Flagging the disclosure line at the bottom of the PR body / commit message — I used Claude for CMake wording, but the investigation and fix are mine.

For context on motivation: I'm building an iOS app that needs vision-capable on-device inference. Linking against the released xcframework failed because the mtmd_* symbols aren't in the binary; tracing it led back to LLAMA_BUILD_TOOLS=OFF in build-xcframework.sh. I worked around it locally with an extern "C" shim, but the root fix belongs upstream so other Apple-platform consumers don't hit the same wall. Happy to iterate on naming, defaults, or split the commit if preferred.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

build Compilation issues examples

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant