test: migrate tests to mender-server compose files

[rewanrashid-boop]

The backend composition now comes from the mender-server repo, added as a submodule at tests/mender_server. Two overlays in tests/compose/ replace the twelve compose files this repo maintained by hand; 34 files are deleted in total, including cert/, keys/, config/traefik/ and storage-proxy/, all of which mender-server now owns.

Full suite verified locally: 195 collected, 0 failures.

Four things are not obvious from the diff.

Traefik's Docker provider is load-bearing for parallelism. It discovers backends by container label and sees every container on the daemon, not just its own project's. Without the --providers.docker.constraints override in docker-compose.testing.yml, two concurrent namespaces each register the other's services under identical router names and requests round-robin into the wrong environment. Measured, not theorised: every service ended up with two backends.

Compose is invoked with --project-directory pinned to the repo root. Compose otherwise derives it from the first -f file, now under tests/compose/, and silently resolves the root .env and every relative bind mount against the wrong directory. include: paths are resolved the same way, hence written relative to the repo root.

requests_retry() sets a Host header on the session, because Traefik routes on it and tests address the gateway by container IP. The trap: it also retries 5xx, so any test asserting on an error response must pass status_forcelist=[]. This turned test_filetransfer.py's xfail(NotImplementedError) cases into RetryError. Websockets need a different fix again -- the library derives Host from the URL, so passing one via additional_headers emits it twice; a pre-connected socket keeps the URL, and therefore both Host and the TLS SNI name, canonical.

Failover is two compose projects instead of duplicated services. The old file hand-copied every service with a "-2" suffix into one project, which cannot work for the Traefik reason above. Server B is now the same composition in its own project with MENDER_HOSTNAME set, so there is nothing to keep in step with mender-server. B needs its own admin user: useradm's Verify looks up both the user id and the token id in its own database, so a token issued by A can never authorise a request to B, even though the two share a signing key.

Two smaller notes. Mongo runs on tmpfs for speed and per-test isolation; the OS -> Enterprise migration test opts out with persistent_mongo=True because it replaces the backend over a surviving database. XDIST_JOBS_IN_PARALLEL_INTEGRATION drops from 4 to 3, since an environment is now 18 containers (23 for Enterprise) rather than ~14 -- that number wants re-tuning against the real runner.

Known characteristic, not a regression: test_monitor_client.py is the most timing-sensitive file in the suite. Several tests restart services on a QEMU device and poll on short fixed budgets, one as low as 18 seconds. They pass at -n 1 and flake above it.

Ticket: QA-1702

[mender-test-bot]

@rewanrashid-boop, start a full integration test pipeline with:

• mentioning me and start integration pipeline

---

my commands and options

You can prevent me from automatically starting CI pipelines:

• if your pull request title starts with "[NoCI] ..."

You can trigger a client pipeline on multiple prs with:

• mentioning me and start client pipeline --pr mender/127 --pr mender-connect/255

You can trigger a client pipeline for a specific Mender Client release with:

• mentioning me and start client pipeline --release 6.0.x (can be given multiple times)
• by default, a pipeline is triggered for each supported release the component is a part of

You can trigger GitHub->GitLab branch sync with:

• mentioning me and sync

You can print PR statistics for a repository with:

• mentioning me and print fast pr stats (Team stats only)
• mentioning me and print full pr stats (Detailed report)
• options: --repo <repo>, --team <name>, --all-repos, --exclude-drafts, --exclude-user <user>
• mentioning me and print full pr stats --repo mender --all-repos --exclude-drafts

You can deploy a review app with:

• mentioning me and start review app (OS environment)
• mentioning me and start review app enterprise (Enterprise environment)

You can run e2e tests against a deployed review app with:

• mentioning me and start review tests (defaults to os environment)
• mentioning me and start review tests enterprise (for enterprise environment)

You can cherry pick to a given branch or branches with:

• mentioning me and:

 cherry-pick to:
 * 1.0.x
 * 2.0.x

[rewanrashid-boop]

@mender-test-bot start integration pipeline

[mender-test-bot]

Hello 😺 I created a pipeline for you here: Pipeline-2736526728

Build Configuration Matrix

Key	Value
INTEGRATION_REV	pull/2956/head
RUN_TESTS_FULL_INTEGRATION	true

[rewanrashid-boop]

@mender-test-bot start integration pipeline

[mender-test-bot]

Hello 😺 I created a pipeline for you here: Pipeline-2736655678

Build Configuration Matrix

Key	Value
INTEGRATION_REV	pull/2956/head
RUN_TESTS_FULL_INTEGRATION	true

[p-targowicz]

code wise all looking good

[rewanrashid-boop]

@mender-test-bot start integration pipeline

[mender-test-bot]

Hello 😺 I created a pipeline for you here: Pipeline-2747437770

Build Configuration Matrix

Key	Value
INTEGRATION_REV	pull/2956/head
RUN_TESTS_FULL_INTEGRATION	true

[rewanrashid-boop]

Review waits until package which itself needs: mendersoftware/mender-server#2146