build: include libmtmd in Apple XCFramework (opt-in LLAMA_BUILD_MTMD)#21935
build: include libmtmd in Apple XCFramework (opt-in LLAMA_BUILD_MTMD)#21935theabecaster wants to merge 1 commit intoggml-org:masterfrom
Conversation
|
Hi @theabecaster, thanks for your contribution! Per our contribution guidelines, the automated PR checker found the following issue(s) that need your attention:
Please note that maintainers reserve the right to make final decisions on PRs. If you believe there is a mistake, please comment below. |
Adds opt-in LLAMA_BUILD_MTMD CMake option so build-xcframework.sh can link libmtmd.a into the framework binary without pulling in the rest of tools/ (which doesn't cross-build cleanly to iOS/tvOS/visionOS). - CMakeLists.txt: new option, default OFF. When on with LLAMA_BUILD_TOOLS=OFF, only the tools/mtmd subdir is added. - tools/mtmd/CMakeLists.txt: gate the CLI exe targets on LLAMA_BUILD_TOOLS. Gating on LLAMA_BUILD_COMMON is not enough: it defaults ON in standalone builds and visionOS xcodebuild then fails with "install TARGETS given no BUNDLE DESTINATION for MACOSX_BUNDLE executable target 'llama-mtmd-cli'". - build-xcframework.sh: turn the option on, pass -DLLAMA_BUILD_MTMD, add libmtmd.a to combine_static_libraries. Module map is not updated to expose mtmd.h / mtmd-helper.h. Those headers use C++ constructs and reference mtmd_decoder_pos without the struct tag, which break clang's pure-C module precompilation path for Swift / Obj-C importers. Consumers declare the ABI via their own extern "C" shim. After this, nm on ios-arm64/llama.framework/llama shows 52 _mtmd_ symbols. AI-assisted: used Claude to help with CMake wording. Design, debugging, and on-device verification are mine.
507dea0 to
0d1ebf7
Compare
|
Flagging the disclosure line at the bottom of the PR body / commit message — I used Claude for CMake wording, but the investigation and fix are mine. For context on motivation: I'm building an iOS app that needs vision-capable on-device inference. Linking against the released xcframework failed because the |
The Apple xcframework from
build-xcframework.shis built withLLAMA_BUILD_TOOLS=OFF, so nomtmd_*symbols end up in it. That means Swift / Obj-C consumers on iOS, macOS, tvOS, visionOS can't use vision even though the sources cross-compile fine.This adds an opt-in
LLAMA_BUILD_MTMDoption (default OFF). When on and full tools are off, only thetools/mtmdsubdirectory gets added so just the library target builds. The CLI exe targets intools/mtmd/CMakeLists.txtare now gated onLLAMA_BUILD_TOOLS— without that, standalone visionOS xcodebuild fails withinstall TARGETS given no BUNDLE DESTINATION for MACOSX_BUNDLE executable target 'llama-mtmd-cli'.build-xcframework.shturns the option on and addslibmtmd.atocombine_static_libraries.Headers aren't exposed via the framework module map.
mtmd.h/mtmd-helper.hhave C++ bits (method-bearing structs,struct mtmd_decoder_poswithout thestructtag) that break pure-C clang module precompilation for Swift importers. Consumers declare the ABI via their ownextern "C"shim — same workaround people use today.Tested:
build-xcframework.shpasses on macos-15 for all 8 platform slicesnm ios-arm64/llama.framework/llama | grep -c ' _mtmd_'→ 52dlsym(RTLD_DEFAULT, "mtmd_init_from_file")is non-NULL on iPhone 15 ProLLAMA_BUILD_MTMD=OFForLLAMA_BUILD_TOOLS=ON) unchanged — full-tools build still pulls mtmd in viatools/Disclosure: I used Claude to help draft the CMake wording. The design, debugging, and on-device verification are mine.