Introduce flavors; CLI rework; Kernel rework; RaspberryPi et al. support#67
Open
rpardini wants to merge 134 commits into
Open
Conversation
Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
- .editorconfig for shell scripts and shell templates Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
…tools tree across invocations Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
- this will allow using a generic/distro kernel instead of building one Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
- hack until it uses the mkosi-supplied kernel for everything - artifacts: look for mkosi-supplied vmlinuz first Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
- self-built kernel from source will come back at a later stage, as a .deb package Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
- just do everything at once Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
…racut/initramfs-tools) Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
- also: show info about modules and full rootfs Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
- later to be reborn as standard apt package Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
- mkosi.finalize: clean up logging a bit - mkosi.postinst: debugs Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
gha: fix Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
gha: pass DEFAULT_FLAVOR_ID as KERNEL_VERSION for publish-combined - retry Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
- `--flavor-id` Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
but logging sucks and no dataclass better, logging still sucks for Full -> Common much better in-package this sucks less cosmetics some static and template rendering and hardcoded cleanup kinda-works kinda works Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
… tree Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
- flavor can produce a hash of contents - incl static + templated files - does NOT include extratrees or packagedirs yet (kernel/tools) - does NOT clean up leftovers yet Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
- deterministic tar for dtb layer too - use epoch as `created` date, so manifests also hit Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
- `tools` and `kernel` are really inputs to mkosi, not outputs Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
- `tools` and `kernel` are really inputs to mkosi, not outputs Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
- BaseFlavor:
- `list_arch_artifacts()` handles vmlinuz, initramfs, and iso directly
- delegates to overridable `add_arch_dtb_artifacts()`
- defaults to including dtbs for aarch64
- but not for trixie-slim (to keep it, well, "slim")
- introduce artifacts.py `OutputArchArtifactType` and `OutputArchArtifact`
- just to type-enforce
- `_collect_arch_artifacts()` is dead, long live `_checksum_files()`
- rework logic in `oci._publish()`
- this is to prepare for extra per-flavor assets, eg "rpi eeprom firmware" for rpi flavor
Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
…_artifact_collect() - for custom stuff done after initramfs build / after artifact collection (mkosi.output/... -> out/) Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
…pi` dir - this is targetting (eeprom-based) RPi Netboot (without u-boot) - implement list_arch_artifacts(), post_mkosi_stage(), post_artifact_collect() Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
- with mixes Path from pathlib and os.path for non-subpaths Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
- refactor to use util.symlink_relative() Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
- comment-change in mkosi.conf to force hash to change and thus all to rebuild Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
…se mkosi-chroot - set `[Build]`.`WithNetwork=yes` so scripts can hit the network if needed - mkosi.postinst is _not_ mkosi.postinst.chroot - See https://github.com/systemd/mkosi/blob/main/mkosi/resources/man/mkosi.1.md#scripts Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
- publish is a release subcommand now
- logging fixes, don't show locals, suppress Docker traces
- TODO: BUILDAH_INSECURE etc
- center on Config (release_ prefix), Docker repasses
- drop `git describe`, always use `v0.0.0-{sha}`
Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
- publish all flavors - last-minute tarball of directories (dtbs, firmware) for GH Release upload - bump actions, drop unused envs, etc - ci: don't fetch-depth:0 as we don't do git describe anymore Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
…ure registries - nee `BUILDAH_INSECURE`, also used for skopeo - mostly for development vs `registry:2` - eg `docker run -p 5001:5000 -it registry:2` - and `REGISTRY_INSECURE=1 uv run captain ... --registry 192.168.66.10:5001 publish` Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
- allows for - `custom/packages.txt`: extra packages to be installed - `custom/mkosi-postinst` and `custom/mkosi-finalize` folders: extra scripts - `custom/files`: extra files in root filesystem - when rendering templates, force newline at the end of each - pass `custom` dir down to Docker Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
- so publish can use GHA-provided docker authentication vs ghcr.io Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
… files) Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
- GHA has skopeo and buildah available in its default runners, but they're old versions - use Docker mode by default also simplifies end-user usage Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
…ifests - makes browsing ghcr.io a bit easier - drop sha/tag from manifest metadata to up SHA-based reuse Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
- otherwise, history between otherwise identical manifests differs for little reason Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
…eterminism) - otherwise, history between otherwise identical manifests differs for little reason Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
- eg `rockchip64-*` matches `rockchip64-vendor` - show sha256's after restore & before save Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
Member
Author
|
Here's a full UART bootlog of a RaspberryPi 4b netboot into CaptainOS |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Introduce flavors; CLI rework; Kernel rework; RaspberryPi et al. support
Flavors
trixie-full- default Debianlinux-generickernel, workingaptpackage managertrixie-slim- uses captainos kernel (built from source), slim profiletrixie-armbian-rpiwith ready-to-netboot firmware produced inrpi-firmwaredirectory
trixie-rockchip64(mainline),trixie-rockchip64-vendor(vendor kernel)trixie-meson64trixie-geniotrixie-fullBaseFlavorclass and a hierarchy of common flavor classes for shared logic and configuration:CommonDebianFlavorfor Debian-based flavors (currently every flavor, but allows for future non-Debian ones)CommonAcpiFlavorfor ACPI/UEFI flavorsCommonArmbianFlavorfor DeviceTree/Armbian-based arm64 flavorscommon-debian > common-acpi > trixie-full(UEFI/ACPI)common-debian > common-armbian > trixie-armbian-*(DeviceTree)CLI
CliContextfor shared options and common CLI behaviorbuild,initramfs,iso,shell,tools,qemu,release,kernel--verboseflag for INFO-level logging--configoption for kernel menuconfig round-tripqemucommand with arm64/UEFI/OVMF support, KVM/HVF accelerationshellsubcommand for interactive bash under DockerKernel Build
make bindeb-pkgfor .deb output6.18.y.amd64and6.18.y.arm64configsmake savedefconfig)--configoptionGitHub Actions
meson64, trixie-rockchip64
build-captainos-with-kernelmainbranchDocker Builder image for mkosi host dependencies
buildercommand; content-hashed per-arch builder imagesuvfor CaptainRelease System
release pull,release tag,release publishBuild Artifacts & Packaging
trixie-slim)Common Debian Flavor
mkosi-chrootin postinst, enableNetwork=yesfor scripts that might need network accesspresetfiles (20-captainos-base, 10-captainos-tink) instead of postinst-based systemctl invocationssystemd-networkd-wait-onlineto accept any interface, 15s timeouttimesyncdretry every 2s for reliabilitytime-sync.target(time-set.targetis not enough)Logging & Debugging
--verboseutil.run()when--verboseFORCE_COLOR=1supportCustomization
custom/packages.txt- extra packagescustom/mkosi-postinst/- extra postinst scriptscustom/mkosi-finalize/- extra finalize scriptscustom/files/- extra root filesystem filesDocumentation
Note: all code written by humans. Documentation and this summary were
Assisted-By: qwen3.5:35b-a3b-coding-nvfp4all commits:
in-docker:prefixdtbmkosi.outputfolder intoout/--flavor-idDpkg::Use-Pty "0";to mkosi sandbox tree--tls-verify=falseregistry:2for testing pushesclick_cliuv run captain ...obtain_builderclickpackagescommon-debian > common-acpi > trixie-fullcommon-debian > common-acpi > trixie-mainlinehas_iso()as TrueConfig::verbose_uvfrom --verbose/Logging.DEBUG; use --quiet otherwiseshutil.get_terminal_sizeto obtain and pass-down-Docker COLUMNSutil.run()if --verbosetrixie-rockchip64-vendorandtrixie-armbian-rpiflavorstrixie-armbian-rpistrays from standard naming as I think we might need to build our own kernel for RPi to achieve small-enough-for-eeprom-netboot eventuallyrun()'s env vars in a Rich Tablerun_in_builder()with bothcommand_and_argsandextra_docker_argscfg.verbose_dockerfor setting envCAPTAIN_VERBOSE=1when relaunching--verbosesets root level as well as current logger to DEBUGshellsubcommandmake bindeb-pkginto a .deb outputpackage_directories()for mkosi'sPackageDirectories=make bindeb-pkgto work cross-archstages(nee_stages) modulebuild_kernel_stage()trixie-slimflavor, using captainos kernel--configfor round-trip menuconfig / savedefconfiguv run captain --verbose --arch amd64 kernel --configCONFIG_MODULE_COMPRESS_ZSTD=yCONFIG_MODULE_COMPRESS_ZSTD=ytrixie-slimflavors (using captainos-kernel)trixie-slimthe default flavorbuild-captainos-with-kernelmainbranchqemucli, with extra support for arm64 & UEFI/OVMFq35machine type for amd64 (it is still not a default)systemd.journald.forward_to_console=1as default prefix kernel cmdlineaudit=0as cmdline prefix defaultuv run captain --verbose --arch=amd64 --flavor-id=trixie-slim qemu -- tink_worker_image=... <...>--are passed directly as kernel cmdline_TINK_PARAMS) for ergonomicsDEFAULT_FLAVOR_ID; make publish-combined a matrix (for both trixie-slim and trixie-full)include_apt(): if true, includes a working apt installinclude_hwdb(): if true, leaves hwdb data inacpidpackage to common_acpi flavorcurl,pciutils,pvtrixie-geniofor Armbian's linux-image-edge-geniouv run captain --arch amd64 kernel --config, then pickCONFIG_BE2NET--loadtodocker buildx buildnativemode (avoid pulling builder for cache hits)createddate, so manifests also hittoolsandkernelare really inputs to mkosi, not outputstoolsandkernelare really inputs to mkosi, not outputslist_arch_artifacts()handles vmlinuz, initramfs, and iso directlyadd_arch_dtb_artifacts()OutputArchArtifactTypeandOutputArchArtifact_collect_arch_artifacts()is dead, long live_checksum_files()oci._publish()firmware-rpidir[Build].WithNetwork=yesso scripts can hit the network if neededgit describe, always usev0.0.0-{sha}REGISTRY_INSECURE=1env var for talking to insecure registriesBUILDAH_INSECURE, also used for skopeoregistry:2docker run -p 5001:5000 -it registry:2REGISTRY_INSECURE=1 uv run captain ... --registry 192.168.66.10:5001 publishcustomfoldercustom/packages.txt: extra packages to be installedcustom/mkosi-postinstandcustom/mkosi-finalizefolders: extra scriptscustom/files: extra files in root filesystemcustomdir down to Dockerbuildah from(more determinism)rockchip64-*matchesrockchip64-vendor