Skip to content

BLD: use cibuildwheel to create wheel matrix #21

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
tangkong merged 29 commits into from
Apr 10, 2025
Merged

BLD: use cibuildwheel to create wheel matrix #21

Show file tree
Hide file tree
Changes from 21 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
44754e1
BLD: try cibuildwheel
tangkong Apr 4, 2025
91fbf74
BLD: use cibuildwheel action
tangkong Apr 4, 2025
2b105a7
BLD: make epicscorelibs available before build
tangkong Apr 4, 2025
a8a39bf
BLD: make pyepics available before build
tangkong Apr 4, 2025
d4f0a5f
BLD: pip --no-build-isolation
tangkong Apr 4, 2025
8280bb9
BLD: try explicitly linking LD_LIBRARY_PATH
tangkong Apr 4, 2025
b6d9729
BLD: skip pypy builds
tangkong Apr 4, 2025
41f8d4c
BLD: re-enable wheel repair, expand build matrix
tangkong Apr 7, 2025
62e7197
BLD: adjust before all step, show env for debugging
tangkong Apr 7, 2025
6508706
BLD: move installs to before_build
tangkong Apr 7, 2025
db0e1a7
BLD: set library path right before wheel
tangkong Apr 7, 2025
cdf8e72
BLD: extend library path instead of overwriting
tangkong Apr 7, 2025
3f18074
BLD: clean up unused commands, path setting
tangkong Apr 7, 2025
f80046d
BLD: whoops dangling &&
tangkong Apr 7, 2025
0e8a308
BLD: add windows, macos platforms
tangkong Apr 7, 2025
927b0dc
BLD: use DYLD_LIBRARY_PATH for macos runner
tangkong Apr 7, 2025
7b02faa
BLD: remove windows, pthread not available on windows. Add test impo…
tangkong Apr 7, 2025
dc32d23
BLD: I forgot another &&
tangkong Apr 7, 2025
406817d
BLD: include numpy for conda builds
tangkong Apr 7, 2025
cd059a4
BLD: autoupdate pre-commit
tangkong Apr 7, 2025
7b9db80
BLD/TST: try pinning package version in conda recipe for simplicity o…
tangkong Apr 7, 2025
1a09f48
BLD: fetch-depth=0 to grab all tags
tangkong Apr 8, 2025
4a9357a
BLD: explicitly fetch tags
tangkong Apr 8, 2025
35d630b
BLD: use build to make wheels, try to trigger setuptools_scm
tangkong Apr 8, 2025
502c26e
BLD: use proper isolation flag
tangkong Apr 8, 2025
009107a
BLD: specify all build dependencies
tangkong Apr 8, 2025
d16e97e
BLD: separate build and test workflows, set up framework for twine up…
tangkong Apr 8, 2025
998c149
BLD: rename build wheels workflow
tangkong Apr 8, 2025
b398ed2
BLD: activate wheel build on tag
tangkong Apr 10, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 41 additions & 0 deletions .github/workflows/standard.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,3 +62,44 @@ jobs:
deploy-on-success: ${{ matrix.deploy-on-success || false }}
system-packages: ""
testing-extras: ""

build_wheels:
name: Build wheels on ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest, macos-14]

steps:
- uses: actions/checkout@v4

- name: Build wheels
Copy link
Copy Markdown
Member

@ZLLentz ZLLentz Apr 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the PR here, this ends up building wheels marked at version 0.0.0. Is this expected for the non-tag build?

20 wheels produced in 80 minutes:
  pyca_epics-0.0.0-cp310-cp310-manylinux_2_17_i686.manylinux2014_i686.whl         476 kB
  pyca_epics-0.0.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl     449 kB
  pyca_epics-0.0.0-cp310-cp310-musllinux_1_2_i686.whl                           1,569 kB
  pyca_epics-0.0.0-cp310-cp310-musllinux_1_2_x86_64.whl                         1,432 kB
  pyca_epics-0.0.0-cp311-cp311-manylinux_2_17_i686.manylinux2014_i686.whl         476 kB
  pyca_epics-0.0.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl     449 kB
  pyca_epics-0.0.0-cp311-cp311-musllinux_1_2_i686.whl                           1,569 kB
  pyca_epics-0.0.0-cp311-cp311-musllinux_1_2_x86_64.whl                         1,432 kB
  pyca_epics-0.0.0-cp312-cp312-manylinux_2_17_i686.manylinux2014_i686.whl         477 kB
  pyca_epics-0.0.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl     450 kB
  pyca_epics-0.0.0-cp312-cp312-musllinux_1_2_i686.whl                           1,569 kB
  pyca_epics-0.0.0-cp312-cp312-musllinux_1_2_x86_64.whl                         1,433 kB
  pyca_epics-0.0.0-cp313-cp313-manylinux_2_17_i686.manylinux2014_i686.whl         477 kB
  pyca_epics-0.0.0-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl     450 kB
  pyca_epics-0.0.0-cp313-cp313-musllinux_1_2_i686.whl                           1,569 kB
  pyca_epics-0.0.0-cp313-cp313-musllinux_1_2_x86_64.whl                         1,433 kB
  pyca_epics-0.0.0-cp39-cp39-manylinux_2_17_i686.manylinux2014_i686.whl           470 kB
  pyca_epics-0.0.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl       450 kB
  pyca_epics-0.0.0-cp39-cp39-musllinux_1_2_i686.whl                             1,569 kB
  pyca_epics-0.0.0-cp39-cp39-musllinux_1_2_x86_64.whl                           1,432 kB

Copy link
Copy Markdown
Contributor Author

@tangkong tangkong Apr 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't sorted this bit out yet. I saw this on my fork and hoped that it was just missing tags.

Copy link
Copy Markdown
Contributor Author

@tangkong tangkong Apr 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like all the other cibuildwheel examples end up either collecting the tag ahead of time or setting it statically. I'll deal with this in a similar way, maybe

Copy link
Copy Markdown
Contributor Author

@tangkong tangkong Apr 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some more digging, setuptools_scm generates the version files on import or build. Turns out we had been using the old default wheel build method (python -m pip wheel), and switching this to python -m build generates those files properly.

uses: pypa/cibuildwheel@v2.23.2
env:
CIBW_SKIP: pp*
CIBW_BUILD_FRONTEND: "pip; args: --no-build-isolation"
CIBW_MANYLINUX_X86_64_IMAGE: manylinux2014
CIBW_BEFORE_BUILD: >
pip install setuptools_dso epicscorelibs
CIBW_REPAIR_WHEEL_COMMAND_LINUX: >
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$(python scripts/get_library_paths.py) &&
env | grep LIBRARY &&
auditwheel repair -w {dest_dir} {wheel}
CIBW_REPAIR_WHEEL_COMMAND_MACOS: >
export DYLD_LIBRARY_PATH=$DYLD_LIBRARY_PATH:$(python scripts/get_library_paths.py) &&
env | grep LIBRARY &&
delocate-wheel --require-archs {delocate_archs} -w {dest_dir} -v {wheel}
CIBW_TEST_COMMAND: >
echo "wheel installed" &&
python -c "import pyca; print(pyca)" &&
python -c "import psp; print(psp)"

with:
package-dir: .
output-dir: wheelhouse
config-file: "{package}/pyproject.toml"

- uses: actions/upload-artifact@v4
with:
name: cibw-wheels-${{ matrix.os }}-${{ strategy.job-index }}
path: ./wheelhouse/*.whl
6 changes: 3 additions & 3 deletions .pre-commit-config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ exclude: |

repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.6.0
rev: v5.0.0
hooks:
- id: no-commit-to-branch
- id: trailing-whitespace
Expand All @@ -23,11 +23,11 @@ repos:
- id: debug-statements

- repo: https://github.com/pycqa/flake8.git
rev: 7.0.0
rev: 7.2.0
hooks:
- id: flake8

- repo: https://github.com/timothycrosley/isort
rev: 5.13.2
rev: 6.0.1
hooks:
- id: isort
3 changes: 1 addition & 2 deletions conda-recipe/meta.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,9 @@
{% set package_name = "pyca" %}
{% set import_name = "pyca" %}
{% set data = load_setup_py_data() %}

package:
name: {{ package_name }}
version: {{ data.get('version') }}
version: "3.3.1"

source:
path: ..
Expand Down
11 changes: 11 additions & 0 deletions scripts/get_library_paths.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@

def get_library_paths():
try:
from epicscorelibs.path import lib_path
print(lib_path)
except ImportError:
return


if __name__ == "__main__":
get_library_paths()
Loading