♻️ Migrate packages to pyproject.toml build system

[sanderegg]

What do these changes do?

• Migrates all Python packages from setup.py/setup.cfg to pyproject.toml
• Removes legacy setup files and updates Makefiles for new build system
• Adds uv.lock files for reproducible environments
• Updates package dependencies and test configurations
• Cleans up redundant code and simplifies package structure
• Introduces pytest-asyncio for improved async testing
• Refactors Celery library backend and task manager modules
• Updates CI scripts to match new build and test workflow
• No user-facing features changed; internal architecture improved

Related issue/s

N/A

How to test

1. Run make install in the root and each package directory
2. Execute make test or pytest in each package to verify tests pass
3. Check CI pipeline for successful builds and tests
4. Validate that no setup.py/setup.cfg files remain in packages
5. Review package imports and dependencies for correctness

Dev-ops

• Environment variables: No changes
• Database migrations: None
• Manual testing: Not required
• Ops config updates: Not required

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 80.77%. Comparing base (b7d4c78) to head (9262c9a).

❗ There is a different number of reports uploaded between BASE (b7d4c78) and HEAD (9262c9a). Click for more details.

HEAD has 22 uploads less than BASE

Flag	BASE (b7d4c78)	HEAD (9262c9a)
unittests	32	10

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #8853      +/-   ##
==========================================
- Coverage   86.84%   80.77%   -6.07%     
==========================================
  Files        2059      578    -1481     
  Lines       81649    21384   -60265     
  Branches     1459     1334     -125     
==========================================
- Hits        70907    17273   -53634     
+ Misses      10331     3729    -6602     
+ Partials      411      382      -29     

Flag	Coverage Δ
integrationtests	?
unittests	80.77% <100.00%> (-5.47%)	⬇️

Components	Coverage Δ
pkg_aws_library	95.28% <ø> (ø)
pkg_celery_library	76.55% <100.00%> (ø)
pkg_dask_task_models_library	79.37% <ø> (ø)
pkg_models_library	92.89% <ø> (ø)
pkg_notifications_library	84.48% <ø> (ø)
pkg_postgres_database	∅ <ø> (∅)
pkg_service_integration	72.86% <ø> (ø)
pkg_service_library	71.18% <ø> (ø)
pkg_settings_library	90.70% <ø> (ø)
pkg_simcore_sdk	72.31% <ø> (ø)
agent	∅ <ø> (∅)
api_server	∅ <ø> (∅)
autoscaling	∅ <ø> (∅)
catalog	∅ <ø> (∅)
clusters_keeper	∅ <ø> (∅)
dask_sidecar	∅ <ø> (∅)
datcore_adapter	∅ <ø> (∅)
director	∅ <ø> (∅)
director_v2	∅ <ø> (∅)
dynamic_scheduler	∅ <ø> (∅)
dynamic_sidecar	∅ <ø> (∅)
efs_guardian	∅ <ø> (∅)
invitations	∅ <ø> (∅)
payments	∅ <ø> (∅)
resource_usage_tracker	∅ <ø> (∅)
storage	∅ <ø> (∅)
webclient	∅ <ø> (∅)
webserver	∅ <ø> (∅)

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b7d4c78...9262c9a. Read the comment docs.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[mergify]

🧪 CI Insights

Here's what we observed from your CI run for 9262c9a.

❌ Job Failures

Pipeline	Job	Health on master	Retries	🔍 CI Insights	📄 Logs
CI	unit-tests		0	View	View
PR CI	check OAS' are up to date		0	View	View

✅ Passed Jobs With Interesting Signals

Pipeline	Job	Signal	Health on master	Retries	🔍 CI Insights	📄 Logs
CI	integration-tests	Base branch is broken, but the job passed. Looks like this might be a real fix 💪		0	View	View
	system-tests	Base branch is broken, but the job passed. Looks like this might be a real fix 💪		0	View	View

[Copilot]

Pull request overview

This PR migrates several Python packages (and related CI/build tooling) from setup.py/setup.cfg + requirements/*.txt workflows to pyproject.toml with uv dependency groups and standardized Makefile targets.

Changes:

• Replace setuptools-based packaging (setup.py, setup.cfg) with pyproject.toml using uv_build across multiple packages.
• Standardize install/test Makefile targets via scripts/common-package.Makefile (using uv sync) and adjust CI scripts accordingly.
• Update Docker build steps and various internal requirement references to align with the new packaging approach.

Reviewed changes

Copilot reviewed 70 out of 82 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tests/swarm-deploy/requirements/dev.txt	Updates editable installs to match new package layout (drops [migration] extra usage).
tests/swarm-deploy/requirements/ci.txt	Updates CI installs to match new package layout (drops [migration] extra usage).
services/migration/Dockerfile	Switches dependency install step to uv sync with lockfile.
scripts/common-package.Makefile	Introduces standardized install-* and test* targets based on uv sync.
scripts/common-package-extras.Makefile	Adds an optional profile stub/documentation for packages with extras.
packages/simcore-sdk/setup.py	Removes setuptools build script in favor of pyproject.toml.
packages/simcore-sdk/setup.cfg	Removes legacy bumpversion/pytest/mypy config in favor of pyproject.toml.
packages/simcore-sdk/pyproject.toml	Adds pyproject.toml project metadata + dependency groups for simcore-sdk.
packages/simcore-sdk/README.md	Adds minimal usage docs for new Makefile/uv workflow.
packages/simcore-sdk/Makefile	Adapts Makefile to rely on shared common-package targets for installs.
packages/settings-library/setup.py	Removes setuptools build script in favor of pyproject.toml.
packages/settings-library/setup.cfg	Removes legacy bumpversion/pytest config in favor of pyproject.toml.
packages/settings-library/pyproject.toml	Adds pyproject.toml project metadata + dependency groups for settings library.
packages/settings-library/Makefile	Simplifies Makefile to use shared common-package targets.
packages/service-library/setup.py	Removes setuptools build script in favor of pyproject.toml.
packages/service-library/setup.cfg	Removes legacy bumpversion/pytest/mypy config in favor of pyproject.toml.
packages/service-library/pyproject.toml	Adds pyproject.toml project metadata, extras, and dependency groups for service library.
packages/service-library/README.md	Adds minimal usage docs for new Makefile/uv workflow.
packages/service-library/Makefile	Refactors installs/tests and adds bracketed extra targets for aiohttp/fastapi.
packages/service-integration/setup.py	Removes setuptools build script in favor of pyproject.toml.
packages/service-integration/setup.cfg	Removes legacy bumpversion/mypy config in favor of pyproject.toml.
packages/service-integration/pyproject.toml	Adds pyproject.toml project metadata + dependency groups for service-integration.
packages/service-integration/Makefile	Simplifies Makefile to use shared common-package targets (keeps autogen/docker targets).
packages/pytest-simcore/uv.lock	Adds uv lockfile for reproducible installs.
packages/pytest-simcore/pyproject.toml	Updates requires-python and removes setuptools-specific configuration.
packages/pytest-simcore/Makefile	Simplifies Makefile to use shared common-package targets.
packages/postgres-database/setup.py	Removes setuptools build script in favor of pyproject.toml.
packages/postgres-database/setup.cfg	Removes legacy bumpversion/pytest/mypy config in favor of pyproject.toml.
packages/postgres-database/pyproject.toml	Adds pyproject.toml project metadata, scripts, and dependency groups for postgres-database.
packages/postgres-database/docker/Dockerfile	Updates Docker build to install the package without [migration] extra.
packages/postgres-database/README.md	Updates installation guidance to the new Makefile/uv flow.
packages/postgres-database/Makefile	Simplifies Makefile to use shared common-package targets; keeps DB-specific targets.
packages/notifications-library/setup.py	Removes setuptools build script in favor of pyproject.toml.
packages/notifications-library/setup.cfg	Removes legacy bumpversion/pytest/mypy config in favor of pyproject.toml.
packages/notifications-library/pyproject.toml	Adds pyproject.toml project metadata + dependency groups for notifications library.
packages/notifications-library/Makefile	Simplifies Makefile to use shared common-package targets.
packages/models-library/setup.py	Removes setuptools build script in favor of pyproject.toml.
packages/models-library/setup.cfg	Removes legacy bumpversion/pytest/mypy config in favor of pyproject.toml.
packages/models-library/requirements/dev.txt	Updates editable installs to match new package layout (drops [migration] extra usage).
packages/models-library/requirements/ci.txt	Updates CI installs to match new package layout (drops [migration] extra usage).
packages/models-library/pyproject.toml	Adds pyproject.toml project metadata + dependency groups for models library.
packages/models-library/Makefile	Simplifies Makefile to use shared common-package targets; keeps autogen targets.
packages/dask-task-models-library/setup.py	Removes setuptools build script in favor of pyproject.toml.
packages/dask-task-models-library/setup.cfg	Removes legacy bumpversion/pytest config in favor of pyproject.toml.
packages/dask-task-models-library/requirements/_base.in	Removes obsolete conditional dependency for older Python versions.
packages/dask-task-models-library/pyproject.toml	Adds pyproject.toml project metadata + dependency groups for dask-task-models library.
packages/dask-task-models-library/Makefile	Simplifies Makefile to use shared common-package targets.
packages/common-library/uv.lock	Adds uv lockfile for reproducible installs.
packages/common-library/setup.py	Removes setuptools build script in favor of pyproject.toml.
packages/common-library/setup.cfg	Removes legacy bumpversion/pytest/mypy config in favor of pyproject.toml.
packages/common-library/pyproject.toml	Adds pyproject.toml project metadata + dependency groups for common library.
packages/common-library/Makefile	Simplifies Makefile to use shared common-package targets.
packages/celery-library/src/celery_library/backends/_redis.py	Adjusts Redis pipeline execution call in create_group.
packages/celery-library/setup.py	Removes setuptools build script in favor of pyproject.toml.
packages/celery-library/setup.cfg	Removes legacy bumpversion/pytest/mypy config in favor of pyproject.toml.
packages/celery-library/pyproject.toml	Adds pyproject.toml project metadata + dependency groups for celery library.
packages/celery-library/Makefile	Simplifies Makefile to use shared common-package targets.
packages/aws-library/setup.py	Removes setuptools build script in favor of pyproject.toml.
packages/aws-library/setup.cfg	Removes legacy bumpversion/pytest/mypy config in favor of pyproject.toml.
packages/aws-library/pyproject.toml	Adds pyproject.toml project metadata + dependency groups for aws library.
packages/aws-library/Makefile	Simplifies Makefile to use shared common-package targets.
ci/github/unit-testing/simcore-sdk.bash	Removes ad-hoc mypy install; relies on uv-managed dependencies.
ci/github/unit-testing/settings-library.bash	Switches to make test-ci and removes ad-hoc mypy install.
ci/github/unit-testing/service-library.bash	Removes ad-hoc mypy install; relies on uv-managed dependencies.
ci/github/unit-testing/service-integration.bash	Switches to make test-ci.
ci/github/unit-testing/postgres-database.bash	Switches to make test-ci and removes ad-hoc mypy install.
ci/github/unit-testing/notifications-library.bash	Switches to make test-ci.
ci/github/unit-testing/models-library.bash	Switches to make test-ci and removes ad-hoc mypy install.
ci/github/unit-testing/dask-task-models-library.bash	Switches to make test-ci and removes ad-hoc mypy install.
ci/github/unit-testing/common-library.bash	Switches to make test-ci and removes ad-hoc mypy install.
ci/github/unit-testing/celery-library.bash	Switches to make test-ci and removes ad-hoc mypy install.
ci/github/unit-testing/aws-library.bash	Switches to make test-ci and removes ad-hoc mypy install.

Comments suppressed due to low confidence (2)

packages/service-library/README.md:17

• The README references make test-dev, but after the Makefile refactor the standard targets are test / test-ci (and there is no test-dev target in this package). Update the docs to the new target names (and mention the bracketed variants if you keep them).
  packages/settings-library/pyproject.toml:18
• pytest-asyncio is listed under [project].dependencies, which makes it a runtime requirement of simcore-settings-library. Previously it was only part of test requirements; it should likely live under [dependency-groups].dev instead (and be removed from runtime deps) unless the library truly requires pytest at runtime.

[Copilot]

tests[...] targets depend on a tests target, but the shared common-package.Makefile defines test/test-ci (no tests). As-is, make tests and make "tests[aiohttp]" will fail with “No rule to make target 'tests'”. Rename these targets to test[...] (and update the usage header), or add a tests: test alias to preserve the old name.

[Copilot]

The previous setup.py exposed multiple console scripts (e.g. ooil, osparc-service-integrator, simcore-service-*) and a pytest11 plugin entry point. The new pyproject.toml does not define any [project.scripts] or [project.entry-points.pytest11], so these entry points will disappear from installed distributions. Re-add the equivalent entry-point definitions in pyproject.toml to avoid breaking users/CI tooling.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud