Pin uv to a patched image and restructure the CI around per-environment registries - #326
Open
filippo-20tab wants to merge 6 commits into
Open
filippo-20tab wants to merge 6 commits into
filippo-20tab wants to merge 6 commits into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ports back to the scaffolding a set of CI and image changes validated on two live projects (
gs1-tendenze/backendandgs1-one/api).1. Pin the uv base image to a patched trixie release
The base image was on the floating tag
ghcr.io/astral-sh/uv:python<version>-bookworm-slim. Astral publishes versionedbookworm-slimimages only up to0.9.0— from0.10.0on, the variant no longer exists — so that floating tag is effectively frozen on an old uv, missing the fixes released in0.11.6(GHSA-pjjw-68hj-v9mw, arbitrary file deletion on uninstall through a malformedRECORD) and0.11.15. Theuvbinary ships in the production image and runs the entrypoint, so the vulnerable version was actually deployed.Pinned to
ghcr.io/astral-sh/uv:{uv_version}-python{python_version}-trixie-slimvia a newuv_versioncookiecutter option, so the version is explicit, auditable and easy to bump. Moving to trixie is required: it is the only distro for which astral still publishes versioned images.Verified: the tag exists,
requires-pythonstill matches, and every apt package used by the Dockerfile (ca-certificates,libpq5,libpq-dev,gcc,libc6-dev,curl,gettext,git,graphviz,openssh-client,postgresql-client) is available in trixie. On a real project the resulting image is also 7.6 MiB smaller.2. One registry repository per environment
A
workflow:rulesblock derivesIMAGE_REPOandIMAGE_REFfrom the ref, andIMAGE_TAGbecomes${CI_REGISTRY_IMAGE}/${IMAGE_REPO}:${IMAGE_REF}:develop…/development:${CI_COMMIT_SHA}main…/staging:${CI_COMMIT_SHA}…/production:v${CI_COMMIT_TAG}(plus:${CI_COMMIT_SHA}for rollback)…/feature:${CI_COMMIT_SHA}This exists to make GitLab's native cleanup policy usable. Its documented algorithm starts by collecting "all tags for a given repository" and then excludes "the N tags based on the
keep_nvalue":keep_napplies per repository, whilecontainer_expiration_policyis a single set of parameters, not a list of rules. Splitting environments into separate repositories is therefore the only way to express "keep the last N per environment, keep every release" natively.It also fixes a real hazard: deployments pin images by immutable SHA, so a moving
:stagingtag protected byname_regex_keepwould not protect the reference the pods actually pull. With this layout,development/stagingdeploy often enough that their live image always sits within the kept window, andproductionis deployed as:v1.2.3— immutable by nature and kept forever byname_regex_keep: ^v.*.The moving
:development,:staging,:productionand:latesttags are no longer needed, so the build job gets simpler, and the layer cache moves to a dedicated…/cacherepository.3. A parallel Check stage, and a build that stops duplicating itself
A
Checkstage runslint(ruff),security(bandit) andaudit(uv-secure) in parallel beforeBuild, for fast, fail-fast feedback.buildwas the only job withoutrules, so every merge request event created amerge_request_eventpipeline that ran only that job — and nobody consumed it, since test, pact and deploy all run in thepushpipeline. Measured on a real project: 242s and 312s wasted per merge request event. It now reuses the existing*pipeline-push-ruleanchor.The build also gets
STORAGE_DRIVER: overlayand a registry layer cache (--cache-fromalways,--cache-toonly ondevelop/main), measured at 113s → 73s warm on a real project.4. Checks and coverage, each in its own script
check.shkeeps every check (ruff, mypy, bandit, and now theuv-secureaudit, for symmetry with the Check stage);test.shis reduced to coverage and reporting.just testdepends onjust check, so the local behaviour is unchanged, and the CI test job callsmypyexplicitly — it is the only place where the project dependencies are installed, which the Check stage deliberately skips.This also fixes a latent bug:
test.shran withset -uo pipefailand no status handling, so a failingcheck.shdid not fail the job — the exit code was the one ofreport.sh.5. The test runner is selectable
TEST_RUNNER_TAG(defaultsaas-linux-small-amd64) is a CI variable used by thetestjob throughtags, andrunner.pynow registers it among the GitLab project variables, so terraform creates it and it can be raised to a bigger runner from the UI without touching the pipeline. Registered unprotected, likeTEST_ENV_FILE, so it applies to feature branches too.Verification
The template renders, the generated
.gitlab-ci.ymlpasses GitLab's CI Lint API, and the existing test suite is green (59 tests).6. The cleanup policy is created by terraform
gitlab_project.mainnow carries acontainer_expiration_policyblock, so every scaffolded project is born with the cleanup already enabled:Combined with the per-environment repositories above, this yields exactly "keep the last 10 of
developmentand ofstaging, keep every release, drop feature images after a week" — becausekeep_nis applied per repository.Note that this is safe from day zero only for a new project: it starts with the per-environment layout, so no live image ever sits unprotected in the root repository. Existing projects need their environments to deploy on the new paths first.