diff --git a/docs/SRU/explanation/further-requirements.rst b/docs/SRU/explanation/further-requirements.rst index 98dc2f455..1c7eee1bb 100644 --- a/docs/SRU/explanation/further-requirements.rst +++ b/docs/SRU/explanation/further-requirements.rst @@ -89,15 +89,13 @@ SRUs ` Documentation ~~~~~~~~~~~~~ -[Insert specifics here: SRU template, what is expected in each section, -etc] - Documentation must be provided in order to meet our :ref:`public documentation requirement ` as well to assist the SRU team to review your upload. This is usually done individually in the description area of the bug, for each bug being fixed by the SRU, and should follow the :ref:`SRU bug template -`. +`. This page contains an annotated template +explaining requirements and providing examples for each section. Explicit is better than implicit: if there's anything a reviewer might find unexpected, calling it out will help us tell the difference between diff --git a/docs/SRU/explanation/principles.rst b/docs/SRU/explanation/principles.rst index c360094da..eaca44696 100644 --- a/docs/SRU/explanation/principles.rst +++ b/docs/SRU/explanation/principles.rst @@ -23,8 +23,9 @@ Minimise regression Behaviour a user reasonably relies upon that works today must not break tomorrow as a result of an update. In a stable release, this includes *any* user interface change! Exception: behaviour that we deem buggy -must necessarily change in order to fix it. [Expand on this]. See `xkcd +must necessarily change in order to fix it. See `xkcd spacebar heater `_. +.. expand the exception with a real world example. In contrast to pre-release versions, official releases of Ubuntu are subject to much wider use, and by a different demographic of users. @@ -67,4 +68,8 @@ Maintain usefulness ~~~~~~~~~~~~~~~~~~~ We make exceptions to keep the distribution useful. E.g. Firefox, -hardware enablement, Internet protocol [need to expand on this]. +hardware enablement, Internet protocol. + +.. + expand the above section with real examples. + diff --git a/docs/SRU/explanation/standard-processes.rst b/docs/SRU/explanation/standard-processes.rst index 24ea51723..5b1b441fd 100644 --- a/docs/SRU/explanation/standard-processes.rst +++ b/docs/SRU/explanation/standard-processes.rst @@ -1,7 +1,7 @@ Standard processes ------------------ -[this section needs cleaning up] +.. this section needs cleaning up - We'd like the minimum process necessary. It should be clear why any process we have is required because of the principles. @@ -29,8 +29,9 @@ Standard processes evidence that this has happened (typically by including the version number of the package tested). -- No iterating in stable releases. [Robie needs to explain what he - means here] +- No iterating in stable releases. + +.. [Robie or anyone else needs to explain what they mean here] .. _explanation-autopkgtest-failures: diff --git a/docs/SRU/reference/bug-template.rst b/docs/SRU/reference/bug-template.rst index 916044c68..aa6571b38 100644 --- a/docs/SRU/reference/bug-template.rst +++ b/docs/SRU/reference/bug-template.rst @@ -25,7 +25,10 @@ and mitigates risk. [ Test Plan ] - * detailed instructions how to reproduce the bug + * detailed instructions how to reproduce the bug. Instructions should be + broken into steps which are clear and actionable. + + * any step that has an expected return, provide the expected returns. * these should allow someone who is not familiar with the affected package to reproduce the bug and verify that the updated package diff --git a/docs/SRU/reference/exception-btrfs-progs.rst b/docs/SRU/reference/exception-btrfs-progs.rst new file mode 100644 index 000000000..7df2ab3ed --- /dev/null +++ b/docs/SRU/reference/exception-btrfs-progs.rst @@ -0,0 +1,107 @@ +.. _btrfs-progs: + +=========== +Btrfs-progs +=========== + +Basic Information +----------------- + +* ``btrfs-progs`` is maintained in GitHub and follows the kernel release cycle. [#f1]_ +* ``btrfs-progs`` states: + + “The *btrfs-progs* of version *X.Y* declare support of kernel features of the same version. New progs on old kernel are expected to work, limited only by features provided by the kernel.” [#f2]_ + +* Releases use tags. As can be seen in the tags, there are no long-term branches, instead continuous moving versions aligning with upstream kernels. [#f3]_ +* Primary functionality of ``btrfs`` is contained in the kernel. +* For minor releases: “A minor version release may happen in the meantime if there are bug fixes or minor useful improvements queued.” [#f4]_ + +Autopkgtest +----------- + +As of 20260128, ``btrfs-progs`` does not have autopkgtests. Reverse run-time dependencies are outlined below. + +Dependency Analysis +~~~~~~~~~~~~~~~~~~~ + +Reverse-Depends +=============== + +* ``apt-btrfs-snapshot``: create btrfs snapshots whenever apt is run (universe) +* ``btrbk``: backup tool for btrfs volumes (universe) +* ``btrfsd`` [amd64 amd64v3 arm64 armhf ppc64el riscv64 s390x]: daemon for btrfs maintenance (universe) +* ``btrfsmaintenance``: scripts automating btrfs maintenance tasks (universe) +* ``calamares-settings-mobian``: calamares + Mobian for small touch screen devices (universe) +* ``freedom-maker``: FreedomBox image maker. Freedombox is a personal cloud server (universe) +* ``golang-github-containerd-btrfs-dev`` (src: ``golang-github-containerd-btrfs``): go bindings for btrfs (universe) +* ``initramfs-tools-devices``: Common initramfs scripts for Ubuntu Core and Classic (universe) +* ``kiwi-dracut-lib`` [amd64 amd64v3 arm64 armhf ppc64el riscv64 s390x] (src: ``kiwi``): modules for dracut for kiwi image builder +* ``kiwi-systemdeps-filesystems`` [amd64 amd64v3 arm64 armhf ppc64el riscv64 s390x] (src: ``kiwi``): kiwi image builder host setup helper (universe) +* ``libblockdev-btrfs3`` (src: ``libblockdev``): library plugin and standalone for btrfs and C++ projects (universe) +* ``libguestfs0t64`` [amd64 amd64v3 arm64 armhf ppc64el riscv64 s390x] (src: ``libguestfs``): shared library for guest disk image management (universe) +* ``mkosi`` +* ``ubuntu-server`` [amd64 amd64v3 arm64 armhf ppc64el riscv64 s390x]: meta +* ``ubuntu-server-minimal`` [amd64 amd64v3 arm64 armhf ppc64el riscv64 s390x]: meta +* ``ubuntu-server-raspi`` [arm64 armhf]: meta + +Reverse-Recommends +================== + +* ``calamares`` [amd64 amd64v3 arm64 armhf ppc64el riscv64 s390x]: alternative graphical installer (universe) +* ``calamares-extensions`` [amd64 amd64v3 arm64 armhf ppc64el riscv64 s390x]: mobile modules for calamares +* ``distrobuilder`` [amd64 amd64v3 arm64 armhf ppc64el riscv64 s390x] +* ``lubuntu-desktop`` [amd64 amd64v3 arm64 armhf ppc64el riscv64 s390x]: meta +* ``lubuntu-desktop-minimal`` [amd64 amd64v3 arm64 armhf ppc64el riscv64 s390x]: meta +* ``ntfs2btrfs`` [amd64 amd64v3 arm64 armhf ppc64el riscv64 s390x]: convert ntfs to btrfs (universe) +* ``ubiquity`` [amd64 amd64v3 arm64 armhf ppc64el riscv64 s390x]: old installer + +Reverse-Build-Depends +===================== + +* ``btrfsd``: yet another btrfs daemon for maintenance tasks (universe) +* ``containerd-app``: daemon to control runc. Runc + containers = can use btrfs +* ``containerd-stable``: as above +* ``docker.io-app``: as above, except Docker +* ``golang-github-containerd-btrfs``: stated above with ``golang-github-containerd-btrfs-dev`` +* ``libguestfs``: stated above with ``libguestfs0t64`` + +.. + +Of these dependencies, only ``libblockdev`` provides direct ``btrfs`` testing mounting ``btrfs`` filesystems, manipulating volumes and snapshots, and validating the plugin path from top level to the kernel module. Of the build dependencies, only ``docker.io-app`` contains any reference to ``btrfs``, and it’s somewhat ancillary. Running the ``docker-in-lxd`` tests configure ``lxd`` with a ``btrfs`` backend. + +The packages broadly show a need to have matching functionality with the kernel. For image building, container runtimes, and libraries it is important to ensure full-functionality with kernel features. For btrfs-specific maintenance scripts and daemons, stability of the interface is important. + +Autopkgtest Requirements and Addition of Tests +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +``Btrfs-progs`` needs to have autopkgtests added to be able to be SRU’d safely. Further, passing ``libblockdev`` tests are of high importance. + +1. Test installation and upgrade of ``btrfs-progs`` (it’s currently seeded on server, so will likely be in any VM, container, or chroot unless explicitly pruned). +2. Test basic filesystem operations: + + 1. Create a bare partition + 2. Ensure partition is unmounted + 3. Mkfs.btrfs the new partition + 4. ``btrfs check --read-only --force $device`` + 5. Ensure partition is mounted + 6. Write a bunch of files + 7. Run ``btrfs filesystem du``, ensure nothing crashes + 8. Run ``btrfs device stats $device`` + 9. All commands need sudo + 10. Tests must be run with ``isolation-machine`` + +The goal of the tests is not to run every command, nor to cover every case. It’s a selection of “low-hanging fruit” that tests that the program is installed and common commands are available. We will not be able to cover all the cases in an autopkgtest, including functions such as subgroups, managing quota, replication, etc. + +Tests must be run with the LTS and HWE kernel. Creating a proper dependency graph that ensures these tests are automatically run on upload of any filesystem tool may not be possible. In which case a single case will be covered for any interim bugfix updates, and manual testing against LTS and HWE done on ``btrfs-progs`` updates. + +Kernel Testing +-------------- + +The Ubuntu Kernel team maintains a test suite that exercises filesystem capabilities. During the coordinated dot release, the kernel team will run these tests against the LTS kernel and the HWE kernel with the new version of ``btrfs-progs`` that has been prepared. Aspirationally, this testing will be done with a package made in a PPA, as there is currently no way to block proposed-migration on an external test. The kernel team will also aid when the new version of any filesystem tool is made available to do similar testing. The Kernel team regularly runs filesystem tests using the version of userspace tools already released, so continuous testing will occur related to kernel updates. Test results from the kernel runs will be posted to the appropriate SRU bugs. + +.. rubric:: Footnotes + +.. [#f1] https://github.com/kdave/btrfs-progs?tab=readme-ov-file#release-cycle +.. [#f2] https://github.com/kdave/btrfs-progs?tab=readme-ov-file#feature-compatibility +.. [#f3] https://github.com/kdave/btrfs-progs/tags +.. [#f4] https://github.com/kdave/btrfs-progs?tab=readme-ov-file#release-cycle \ No newline at end of file diff --git a/docs/SRU/reference/index.rst b/docs/SRU/reference/index.rst index be8b6aa22..8b8d98c6d 100644 --- a/docs/SRU/reference/index.rst +++ b/docs/SRU/reference/index.rst @@ -27,6 +27,7 @@ Learn more about the criteria to accept a package update, about your responsibil special package-specific historical-removals + requirements-closely-coupled-kernel-userspace Testing and SRU verification ---------------------------- diff --git a/docs/SRU/reference/requirements-closely-coupled-kernel-userspace.rst b/docs/SRU/reference/requirements-closely-coupled-kernel-userspace.rst new file mode 100644 index 000000000..cc799ccc4 --- /dev/null +++ b/docs/SRU/reference/requirements-closely-coupled-kernel-userspace.rst @@ -0,0 +1,247 @@ +.. _reference-criteria-closely-coupled-kernel-userspace: + +========================================== +Closely-Coupled Kernel Userspace Releases +========================================== + +Abstract +======== + +In Linux distros, there are a variety of packages that are closely coupled to the Linux kernel. +Examples include filesystem userspace tools, networking utilities, and device drivers. +As the Ubuntu hwe (hardware enablement) kernel moves forward, new capabilities are available +for users, however, the userspace packages are frozen. An example is ``btrfs-progs`` which +is maintained separately in GitHub [#f1]_. From the README in the repository: “The major version +releases are time-based and follow the cycle of the linux kernel releases. The cycle usually takes +2 months [sic: from the upstream kernel release]. A minor version release may happen in the meantime +if there are bug fixes or minor useful improvements queued.” The goal, in the case of ``btrfs-progs``, +is to have userspace and kernel capabilities in lockstep for users. This is not the current case in Ubuntu. + +Userspace packages in Ubuntu also have the issue of being pulled from Debian. If Debian is seeking +to keep userspace and kernel in-line, this will lead to misalignment with Ubuntu kernels. +This is easily shown on an Ubuntu 24.04 Noble Numbat system, where the LTS kernel is 6.8, +and ``btrfs-progs`` is at 6.6.3. The misalignment with kernel and difficulty in maintaining +the package lead to bugs that can cause major repercussions. [#f2]_ [#f3]_ [#f4]_ + +This page specifies the general approach and workflow for packages that are closely coupled +to the Ubuntu kernels. + + +Rationale +========= + +When userspace and kernel are so tightly coupled, users of the `Ubuntu HWE Kernel`_ should have the full +capabilities of that kernel available in the associated user space packages. + +Backporting specific fixes in tightly-coupled kernel userspace packages can be tricky, as bugfixes may be +relying on kernel changes as well. + +Example scenario: a bug is reported against ``btrfs``. It turns out that this issue has a kernel and userspace +component. The kernel portion is addressed in ``$NEW_KERNEL`` in an HWE roll. ``btrfs-progs`` stays on the same +major version and a targeted fix is done. However, due to the mixing of new features and bug fixes related to +new ``btrfs`` work in the kernel and userspace, extracting only the bugfix is difficult. + + +Specification +============= + +High Level Considerations for Inclusion +--------------------------------------- + +Not all packages can be considered a part of the userspace hwe-kernel tight coupling set. +The following are base criteria to see if a package should be considered a part of this specification: + +1. Primary functionally of the package is directly tied to Kernel version + +2. Kernel capabilities added in an HWE roll are *inaccessible* to most users without the userspace package + +3. Upstream development does *not* maintain any “long term stable” branches + +4. Changes to upstream code mix bugfix and features + + 1. As upstream code moves forward, a bug which affects “older” versions may be fixed on top of + already changed code. Thus crafting specific bugfix patches is difficult, and can lead to + incompatible changes with older versions. + + 2. This may lead to situations where a new version is being backported that only contains + enablement for the HWE kernel and no bugs being addressed. This should be safe for users, + but can lead to an extra update. + +5. Upstream projects set *acceptable minimum version compatibility against the kernel*. + + 1. Ex: an upstream project supports *backwards compatibility* with the LTS kernel available + in a given Ubuntu release. + + 2. Bonus points for backwards compatibility against all standard supported kernels, as it means + keeping a consistent version through Ubuntu releases. + +6. Packages are end-user applications not strictly libraries + + 1. Library changes create a huge blast radius. While applications may be called by other applications + (``package-a`` relies on ``ethtool``) there are better guarantees on API compatibility compared to + bumping a SONAME and ABI compatibility. + + 2. As is, it must be proven that a package is *best* served by rolling, and has minimal possible + disruptions to the Ubuntu ecosystem at large. + +Workflow +-------- + +What follows is the general flow of the package, from development to SRU. This includes required steps +and considerations. + +Ubuntu Devel +~~~~~~~~~~~~ + +The Ubuntu kernel is taking a bleeding edge approach, taking as new a version as possible. +This may mean releasing with an RC kernel from upstream. Many userspace packages do not update until +after the release of the upstream kernel, with lag being different for each package. If a time +differential is generally known, it must be documented by the package. Basic steps: + +1. Follow upstream releases as close as possible, ensuring the latest release is in Ubuntu devel +2. If a kernel is released, and no userspace package is available at the time of release, + the package must first be put into Ubuntu Devel and then SRU’d. + + 1. It is acceptable to do an SRU before the development release is complete if it offers enablement + for the kernel provided functionality in ``$PREVIOUS_RELEASE``. + + 1. Ex: Ubuntu Resolute Racoon 26.04 is shipping with kernel 7.0.0, taken from 7.0.0rc7 upstream. + At 26.04 release, userspace tools are not released yet. It is acceptable to get the new tool + into 26.10 Stonking Stingray, and SRU into 26.04 before 26.10 release. + +Ubuntu LTS +~~~~~~~~~~ + +The Ubuntu kernel follows two primary tracks during LTS releases – HWE and Stable. There are also specialized +OEM kernels that are on fixed versions that may differ from HWE and Stable. This leads to a situation where +there are, minimally, two versions of the kernel available. For a package to be able to be backported to a +Stable Release, it must declare support for the Stable LTS kernel. An example upstream README from ``btrfs-progs`` +states: +As the documentation states: + + **Feature compatibility** + + The btrfs-progs of version X.Y declare support of kernel features of the same version. + New progs on old kernel are expected to work, limited only by features + provided by the kernel. [#f5]_ + +If the above condition is met, a package may be backported from Devel into Stable Releases. This must follow standard procedures, not skipping any release unless there is a documented reason. A fictitious example: + +* ``tightly-coupled-package-a`` is at version 1.90 in devel +* Ubuntu Development for 28.10 Winsome Weasel (WW) has begun +* ``tightly-coupled-package-a`` 1.90 defines that it has support for kernel versions starting at 6.15.0 +* The Ubuntu Kernel moves to 7.6 +* ``tightly-coupled-package-a`` follows with version 1.91 enabling features in 7.6 during Ubuntu Development + + * It retains the same earliest starting version of the kernel + +* ``tightly-coupled-pakage-a`` is updated to 1.91 during development +* Ubuntu is released +* At 28.04 VV .2 release + + * The current LTS kernel is 7.4 + * HWE kernel rolls to 7.6 + +* Based on the above criteria, rolling ``tightly-coupled-package-a`` version 1.91 into 28.04 is most desirable +* A team member prepares an SRU, with full-template, for the release of 1.91 to enable the HWE kernel +* They also check for any bugs that may be addressed… +* Bundled in 1.91 are various bugfixes. Because of the enablement of new features, separating the fixes from + the feature work is difficult. +* A bug is reported in ``tightly-coupled-package-a`` on the original version of the package, 1.83, as found + in Ubuntu 26.04 Resolute Racoon. This bug affected 28.04 as well with their version 1.89 +* Version 1.91 upstream specifies it fixes the bug. +* After successfully testing 1.91 on 28.04, a team member prepares uploads for 26.04, as it is the + *next supported release* + + * Note: because this happened after the .2 release, Ubuntu UU is an interim version with only 9 months of support + and has already been deprecated. Therefore it does *not* need to receive an update + +* This follows a standard SRU procedure – do not skip releases, ensure no breaking changes, ensure compatibility + of the tool, especially against both the LTS and HWE kernels. + +Backport/SRU Release Timing +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The Kernel team rolls the HWE kernel roughly 3 months after the release of an interim or the next LTS release. +These coincide with the “dot” releases for an LTS. [#f6]_ This means that tools must also follow this timing. +Due to the time between Ubuntu release and backport of the HWE kernel, a new userspace filesystem tool is likely. +Use the “dot” releases as a milestone, with coordination with the release team and kernel team. + +Things like filesystem tools are often *seeded*, which means needing to be prepared and ready for dot release +candidate images. This is because filesystem tools are made available on the ISOs for users to freely choose +root file systems. + +Documentation +------------- + +All packages meeting the above criteria must be documented in “SRU/Package Specific Notes” [#f7]_. A link must +be added to this page for the documentation, and a reference to this page made in the package specific notes. +The documentation for each package must outline all the information for the package: + +* Release cadence (if publicized) +* Statements of and links to compatibility statements +* Testing documentation + + * Autopkgtests and their coverage + * Kernel integration testing + * Any manual testing steps required for packages. + +* Any *exceptions* that need to be made + + * Ex: inability of running autopkgtests due to environment + +Testing +------- + +Autopkgtests +~~~~~~~~~~~~ + +Tools *should* have autopkgtests. Minimally, these tests must exercise: + +* Installation of the package (in most cases, this will *upgrade* a system) +* Basic operations + +If a package does *not* have autopackage tests, it must be clearly documented why they are *not* appropriate +and what testing will be done to ensure quality and lack of regressions. These tests *must* be run against +the LTS and HWE kernels. + +Kernel Testing +~~~~~~~~~~~~~~ + +In the case of the kernel containing tests exercising the capabilities, including using the userspace tool, +those tests *must* be run. These tests *must* be run against the LTS and HWE kernels. + +Manual Testing +~~~~~~~~~~~~~~ + +If a manual test is required, either due to lack of autopkgtest capability or kernel integration tests, +the manual test must be documented fully. These test plans must be written in such a manner that any individual can run the tests, given reasonable pre-requisites. + + +Included Packages +----------------- + +.. toctree:: + :maxdepth: 1 + + exception-btrfs-progs + + +|br| +|br| + +.. rubric:: Footnotes + +.. [#f1] https://github.com/kdave/btrfs-progs +.. [#f2] https://bugs.launchpad.net/ubuntu/+source/btrfs-progs/+bug/2115454 +.. [#f3] https://bugs.launchpad.net/ubuntu/+source/btrfs-progs/+bug/2091894 +.. [#f4] Note that in Debian it got orphaned, and then salvaged https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1088329 +.. [#f5] https://github.com/kdave/btrfs-progs?tab=readme-ov-file#feature-compatibility +.. [#f6] A “dot” release is, essentially, the kernel roll + a more fully tested ISO. Some packages may follow the “dot” releases due to tight coupling with the ISO (such as ``subiquity``) or internal Canonical teams may use dot releases as milestones to achieve partner commitments. Cloud images are continuously built, tested, and released. +.. [#f7] :ref:`SRU/reference/package-specific ` + +.. _Ubuntu HWE Kernel: https://canonical-kernel-docs.readthedocs-hosted.com/latest/reference/hwe-kernels/ + +.. |br| raw:: html + +
\ No newline at end of file diff --git a/docs/SRU/reference/special.rst b/docs/SRU/reference/special.rst index c98fc8a1f..4071c7574 100644 --- a/docs/SRU/reference/special.rst +++ b/docs/SRU/reference/special.rst @@ -17,6 +17,12 @@ permitted SRU, some of which overlap: * **Hardware enablement:** for Long Term Support releases we regularly want to enable new hardware [:ref:`criteria `]. +* **Closely Coupled Userspace and Kernel HWE:** For Long Term Support releases + the Kernel follows the above Hardware Enablement (HWE) process. This extends + the hardware enablement case to userspace packages that are closely couple to + the kernel, ensuring compatibility between the kernel and userspace components. + [:ref:`criteria `]. + * **Environmental change:** updates that need to be applied to Ubuntu packages to adjust to changes in the environment, server protocols, web services, and similar [:ref:`criteria `].