You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Companion to the corresponding issues in autobahn-python and crossbar. Reviewing autobahn-python PR #1934 (NVX compiler flags under Yocto/Buildroot cross toolchains) exposed that the WAMP Python projects make cross-build claims in changelogs and docstrings while documenting nothing where users look, and testing nothing in CI.
zlmdb ships a fully vendored LMDB (upstream + lmdb-patches, built via CFFI) and is a hard dependency of Crossbar.io, so it sits directly on the path to "Crossbar.io on Buildroot/Yocto for riscv64". Its cross-build story needs to be written down too.
Important: zlmdb's promise is not autobahn's promise
Worth stating plainly, because copying autobahn's text over would be misleading in the opposite direction:
zlmdb emits no architecture flags at all. Its extra_compile_args are ['-UNDEBUG', '-DHAVE_PATCHED_LMDB=1', '-w'] (build_cffi_lmdb.py:32, src/zlmdb/_lmdb_vendor/_config.py:5) — no -march, no -mtune. So zlmdb is structurally immune to the -march-guessing bug class that #1834/#1934 are
about, and must not claim a fix for a problem it never had.
zlmdb's actual cross-build risks are different:
Can zlmdb build from an sdist at all? It compiles vendored LMDB from the lmdb-upstream submodule plus lmdb-patches. Buildroot and Yocto build from the sdist, not from a git checkout with submodules initialised. If those sources are not complete in the sdist, zlmdb cannot be built from source on any cross target — a hard blocker, and the single most valuable thing a Tier 1 job would establish. This is the same family as [BUG] ReadTheDocs build fails: pip install .[docs] compiles the mandatory LMDB CFFI extension, but RTD omits the lmdb-upstream / deps/flatbuffers submodules #144 (the RTD docs build fails because .[docs] compiles mandatory LMDB-CFFI while lmdb-upstream/deps/flatbuffers are not initialised).
LMDB portability on the target: 64-bit file offsets, page size, and alignment assumptions on rv64gc; and whether our patch set holds.
FlatBuffers: generated code and schema handling in a cross/source build.
Support-status vocabulary (identical across autobahn, zlmdb, crossbar)
Supported — named targets, gated in CI: Tier 1 per-commit, Tier 2 nightly.
Best-effort — expected to work, not gated in CI; bug reports welcome.
Roadmap — intended direction, not yet backed by tests.
Test tiers (vocabulary for the docs; implementation out of scope here)
Tier 1 — cross-build ("baking"): the CFFI/LMDB extension cross-compiles for the target and the artifact is verified (readelf -h → expected machine, plus an import/functional smoke test). Fast; every commit.
Tier 2 — run on target: boot under QEMU system emulation and run zlmdb's own test suite there. Slow; nightly.
Reference target: riscv64 (rv64gc / lp64d) via Buildroot, cross-built on x86-64, glibc first and musl second, booted under qemu-system-riscv64. riscv64 is deliberately chosen: it is a true cross-build (our aarch64 wheels are
built natively under QEMU-user, which never exercises the cross path), and we publish no riscv64 wheels, so every riscv64 user must build from source.
The matrix to publish
artifact
Tier 1 (cross-build)
Tier 2 (run on target)
autobahn
roadmap → supported once the job lands
roadmap
zlmdb
roadmap → supported once the job lands
roadmap
crossbar
roadmap
roadmap (ultimate goal)
Rows move up as tiers land. Nothing may claim more than it can demonstrate.
What to write
README.md — a short "Cross-compilation and embedded targets"
subsection with the current status tag, linking to the installation docs.
docs/installation.rst — the substance:
what building zlmdb from source actually requires (vendored LMDB sources,
a C toolchain, CFFI) and what that means for a cross build;
explicitly: zlmdb passes no architecture flags, so an external
toolchain's own -march/-mtune/-mcpu govern code generation
unmodified — a genuine property worth stating, and distinct from
autobahn's;
sdist completeness expectations for Buildroot/Yocto-style source builds;
the per-target status tags, per the matrix above;
Tier 1 / Tier 2 definitions.
Open question to resolve while writing this
Is the published sdist self-contained for a source build? If yes, say so explicitly — it is exactly the guarantee an embedded integrator needs. If no, that is a bug to file and fix before any cross-build claim can be made at all,
and it likely also explains #144.
Non-goals (separate issues, later)
Implementing the Tier 1 / Tier 2 CI jobs.
Factoring shared cross-build CI machinery into wamp-cicd.
README.md carries a cross-compilation / embedded-targets subsection with
an explicit status tag
docs/installation.rst documents source-build requirements, the
no-architecture-flags property, sdist expectations, per-target status tags,
and the Tier 1 / Tier 2 definitions
The text does not copy autobahn's -march promise, which does not
apply here
Every claim is either backed by an existing test or explicitly tagged roadmap
The status vocabulary and matrix are worded identically to the
autobahn-python and crossbar issues
See also:
Why
Companion to the corresponding issues in autobahn-python and crossbar. Reviewing autobahn-python PR
#1934 (NVX compiler flags under Yocto/Buildroot cross toolchains) exposed that the WAMP Python projects make cross-build claims in changelogs and docstrings while documenting nothing where users look, and testing nothing in CI.
zlmdb ships a fully vendored LMDB (upstream +
lmdb-patches, built via CFFI) and is a hard dependency of Crossbar.io, so it sits directly on the path to "Crossbar.io on Buildroot/Yocto for riscv64". Its cross-build story needs to be written down too.Important: zlmdb's promise is not autobahn's promise
Worth stating plainly, because copying autobahn's text over would be misleading in the opposite direction:
zlmdb emits no architecture flags at all. Its
extra_compile_argsare['-UNDEBUG', '-DHAVE_PATCHED_LMDB=1', '-w'](build_cffi_lmdb.py:32,src/zlmdb/_lmdb_vendor/_config.py:5) — no-march, no-mtune. So zlmdb is structurally immune to the-march-guessing bug class that #1834/#1934 areabout, and must not claim a fix for a problem it never had.
zlmdb's actual cross-build risks are different:
lmdb-upstreamsubmodule pluslmdb-patches. Buildroot and Yocto build from the sdist, not from a git checkout with submodules initialised. If those sources are not complete in the sdist, zlmdb cannot be built from source on any cross target — a hard blocker, and the single most valuable thing a Tier 1 job would establish. This is the same family as [BUG] ReadTheDocs build fails:pip install .[docs]compiles the mandatory LMDB CFFI extension, but RTD omits thelmdb-upstream/deps/flatbufferssubmodules #144 (the RTD docs build fails because.[docs]compiles mandatory LMDB-CFFI whilelmdb-upstream/deps/flatbuffersare not initialised).Support-status vocabulary (identical across autobahn, zlmdb, crossbar)
Test tiers (vocabulary for the docs; implementation out of scope here)
readelf -h→ expected machine, plus an import/functional smoke test). Fast; every commit.Reference target: riscv64 (rv64gc / lp64d) via Buildroot, cross-built on x86-64, glibc first and musl second, booted under
qemu-system-riscv64. riscv64 is deliberately chosen: it is a true cross-build (our aarch64 wheels arebuilt natively under QEMU-user, which never exercises the cross path), and we publish no riscv64 wheels, so every riscv64 user must build from source.
The matrix to publish
Rows move up as tiers land. Nothing may claim more than it can demonstrate.
What to write
README.md— a short "Cross-compilation and embedded targets"subsection with the current status tag, linking to the installation docs.
docs/installation.rst— the substance:a C toolchain, CFFI) and what that means for a cross build;
toolchain's own
-march/-mtune/-mcpugovern code generationunmodified — a genuine property worth stating, and distinct from
autobahn's;
Open question to resolve while writing this
Is the published sdist self-contained for a source build? If yes, say so explicitly — it is exactly the guarantee an embedded integrator needs. If no, that is a bug to file and fix before any cross-build claim can be made at all,
and it likely also explains #144.
Non-goals (separate issues, later)
wamp-cicd.pip install .[docs]compiles the mandatory LMDB CFFI extension, but RTD omits thelmdb-upstream/deps/flatbufferssubmodules #144 or any sdist-completeness bug this analysis uncovers (fileseparately).
Acceptance criteria
README.mdcarries a cross-compilation / embedded-targets subsection withan explicit status tag
docs/installation.rstdocuments source-build requirements, theno-architecture-flags property, sdist expectations, per-target status tags,
and the Tier 1 / Tier 2 definitions
-marchpromise, which does notapply here
roadmap
autobahn-python and crossbar issues