Skip to content

[DOCS] Write down the cross-compilation promise (README + installation docs), with explicit support-status tags #145

Description

@oberstet

See also:


Why

Companion to the corresponding issues in autobahn-python and crossbar. Reviewing autobahn-python PR
#1934 (NVX compiler flags under Yocto/Buildroot cross toolchains) exposed that the WAMP Python projects make cross-build claims in changelogs and docstrings while documenting nothing where users look, and testing nothing in CI.

zlmdb ships a fully vendored LMDB (upstream + lmdb-patches, built via CFFI) and is a hard dependency of Crossbar.io, so it sits directly on the path to "Crossbar.io on Buildroot/Yocto for riscv64". Its cross-build story needs to be written down too.

Important: zlmdb's promise is not autobahn's promise

Worth stating plainly, because copying autobahn's text over would be misleading in the opposite direction:

zlmdb emits no architecture flags at all. Its extra_compile_args are ['-UNDEBUG', '-DHAVE_PATCHED_LMDB=1', '-w'] (build_cffi_lmdb.py:32, src/zlmdb/_lmdb_vendor/_config.py:5) — no -march, no -mtune. So zlmdb is structurally immune to the -march-guessing bug class that #1834/#1934 are
about, and must not claim a fix for a problem it never had.

zlmdb's actual cross-build risks are different:

  1. Can zlmdb build from an sdist at all? It compiles vendored LMDB from the lmdb-upstream submodule plus lmdb-patches. Buildroot and Yocto build from the sdist, not from a git checkout with submodules initialised. If those sources are not complete in the sdist, zlmdb cannot be built from source on any cross target — a hard blocker, and the single most valuable thing a Tier 1 job would establish. This is the same family as [BUG] ReadTheDocs build fails: pip install .[docs] compiles the mandatory LMDB CFFI extension, but RTD omits the lmdb-upstream / deps/flatbuffers submodules #144 (the RTD docs build fails because .[docs] compiles mandatory LMDB-CFFI while lmdb-upstream/deps/flatbuffers are not initialised).
  2. LMDB portability on the target: 64-bit file offsets, page size, and alignment assumptions on rv64gc; and whether our patch set holds.
  3. FlatBuffers: generated code and schema handling in a cross/source build.

Support-status vocabulary (identical across autobahn, zlmdb, crossbar)

  • Supported — named targets, gated in CI: Tier 1 per-commit, Tier 2 nightly.
  • Best-effort — expected to work, not gated in CI; bug reports welcome.
  • Roadmap — intended direction, not yet backed by tests.

Test tiers (vocabulary for the docs; implementation out of scope here)

  • Tier 1 — cross-build ("baking"): the CFFI/LMDB extension cross-compiles for the target and the artifact is verified (readelf -h → expected machine, plus an import/functional smoke test). Fast; every commit.
  • Tier 2 — run on target: boot under QEMU system emulation and run zlmdb's own test suite there. Slow; nightly.

Reference target: riscv64 (rv64gc / lp64d) via Buildroot, cross-built on x86-64, glibc first and musl second, booted under qemu-system-riscv64. riscv64 is deliberately chosen: it is a true cross-build (our aarch64 wheels are
built natively under QEMU-user, which never exercises the cross path), and we publish no riscv64 wheels, so every riscv64 user must build from source.

The matrix to publish

artifact Tier 1 (cross-build) Tier 2 (run on target)
autobahn roadmap → supported once the job lands roadmap
zlmdb roadmap → supported once the job lands roadmap
crossbar roadmap roadmap (ultimate goal)

Rows move up as tiers land. Nothing may claim more than it can demonstrate.

What to write

  1. README.md — a short "Cross-compilation and embedded targets"
    subsection with the current status tag, linking to the installation docs.
  2. docs/installation.rst — the substance:
    • what building zlmdb from source actually requires (vendored LMDB sources,
      a C toolchain, CFFI) and what that means for a cross build;
    • explicitly: zlmdb passes no architecture flags, so an external
      toolchain's own -march/-mtune/-mcpu govern code generation
      unmodified — a genuine property worth stating, and distinct from
      autobahn's;
    • sdist completeness expectations for Buildroot/Yocto-style source builds;
    • the per-target status tags, per the matrix above;
    • Tier 1 / Tier 2 definitions.

Open question to resolve while writing this

Is the published sdist self-contained for a source build? If yes, say so explicitly — it is exactly the guarantee an embedded integrator needs. If no, that is a bug to file and fix before any cross-build claim can be made at all,
and it likely also explains #144.

Non-goals (separate issues, later)

Acceptance criteria

  • README.md carries a cross-compilation / embedded-targets subsection with
    an explicit status tag
  • docs/installation.rst documents source-build requirements, the
    no-architecture-flags property, sdist expectations, per-target status tags,
    and the Tier 1 / Tier 2 definitions
  • The text does not copy autobahn's -march promise, which does not
    apply here
  • Every claim is either backed by an existing test or explicitly tagged
    roadmap
  • The status vocabulary and matrix are worded identically to the
    autobahn-python and crossbar issues

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions