Skip to content

COMP: Make ITKVtkGlue wrapping abi3-compatible - #6715

Merged
hjmjohnson merged 2 commits into
InsightSoftwareConsortium:mainfrom
hjmjohnson:abi3-vtkglue
Sep 6, 2026
Merged

COMP: Make ITKVtkGlue wrapping abi3-compatible#6715
hjmjohnson merged 2 commits into
InsightSoftwareConsortium:mainfrom
hjmjohnson:abi3-vtkglue

Conversation

@hjmjohnson

@hjmjohnson hjmjohnson commented Jul 27, 2026

Copy link
Copy Markdown
Member

Makes ITKVtkGlue's Python wrapping stable-ABI clean, so it can be built with ITK_USE_PYTHON_LIMITED_API=ON — the configuration its FATAL_ERROR previously rejected. Closes #6711.

Scope is deliberately limited to Modules/Bridge/VtkGlue/. No CMake option, Python discovery, or wrapping-macro behaviour changes here; whether ITK should default to abi3 is separate work, tracked in #6757 along with free-threading and abi3t.

Because Limited API calls are also valid in a full-API build, one implementation serves both modes — the module does not become abi3-only, and the FATAL_ERROR is removed rather than inverted.

Verified on macOS 26 / arm64 only. No Linux or Windows verification, and no Slicer test result.

The fix, and a defect in the approach #6711 proposed

VtkGlue.i exchanged pointers with VTK through vtkPythonUtil, whose header chain reaches PyVTKObject.h and touches PyTypeObject members that Py_LIMITED_API hides. The typemaps now use Limited API calls only, parsing VTK's __this__ encoding and reconstructing through Addr=0x....

VTK::WrappingPythonCore is dropped from the wrapping link interface. This is a correctness requirement rather than tidying: that library is built against one libpython, so an extension linking it cannot be version-agnostic.

Defect in the sketch from #6711. The issue proposed PyObject_CallFunction(cls, "s", addr), i.e. vtkImageData("Addr=0x…"). On VTK 9.6.2 this fails, but only for vtkImageData and vtkPolyData, and only in the pointer-to-object direction. The cause is not ABI-related: vtkmodules/util/data_model.py registers @vtkImageData.override with a Python subclass whose __init__(self, **kwargs) is keyword-only, so the positional address string never reaches VTK's C-level reconstruction. vtkImageExport, vtkImageImport, vtkObject and vtkPoints have no override and were unaffected.

The fix is to call __new__ explicitly: it bypasses the override's __init__ while still returning the enhanced ImageData type, with pointer identity preserved.

Neither __this__ nor Addr=0x... is documented VTK API, so PythonVtkGlueABI3EncodingTest exists to fail loudly and specifically if VTK changes either one. PythonVtkGlueRoundTripTest exercises the typemaps end to end.

Environments verified (and what was not)

Platform: macOS 26 / arm64, Apple clang via conda-forge toolchain.
VTK: github.com/slicer/VTK at 6181bb1223bbc499a340a1644f5356e7e152c318 — the SHA Slicer's SuperBuild/External_VTK.cmake pins for its 9.6 series — built with Qt6 6.9.1, full rendering, and Python wrapping.

Both wrapping modes, built and tested separately

Check ITK_USE_PYTHON_LIMITED_API=ON =OFF
Build 8565/8565, 0 FAILED: 8565/8565, 0 FAILED:
ctest -R Python 177/177 177/177
PythonVtkGlueABI3EncodingTest passed passed
PythonVtkGlueRoundTripTest passed passed
Module suffix _ITKVtkGluePython.abi3.so _ITKVtkGluePython.so

pre-commit run --all-files: exit 0.

Artifact audit (limited-API mode)

Property Value
_ITKVtkGluePython.abi3.so undefined Py* symbols 71, all Limited API
PyVTK* / vtkPythonUtil symbols 0
libvtkWrappingPythonCore linked 0
libpython linked none

Multi-interpreter. Extensions were built against pixi Python 3.13.9; the others are Homebrew builds, a different distribution. VTK's Python wrappers were built separately for cp311/cp312/cp313/cp314 against the same VTK C++ libraries, so each row pairs the same ITK .abi3.so with a different version-specific VTK.

Interpreter abi3 module loads Filter result VtkGlue round-trip
3.11.15 yes 7.0 OK
3.12.13 yes 7.0 OK
3.13.9 (build interpreter) yes 7.0 OK
3.14.6 yes 7.0 OK

The 3.11 and 3.14 rows are the substantive ones: that .abi3.so was compiled against cp313 VTK headers yet drove full round-trips against cp311 and cp314 VTK modules. While VTK::WrappingPythonCore was linked, those pairings were structurally impossible.

Downstream

  • Standalone ITK with ITK_WRAP_PYTHON=OFF, Module_ITKVtkGlue=ON: 2730/2730 targets, 0 errors.
  • Slicer: its ITK external project built successfully against this branch, together with ~20 other external projects. No Slicer tests ran — the Slicer build is blocked in GDCM by a conda/system library shadowing problem in the local test environment, independent of this branch (the diff is wrapping-only and Slicer's ITK is ITK_WRAP_PYTHON=OFF).

Not verified: Linux, Windows, any non-arm64 target, and any completed Slicer test suite.

@github-actions github-actions Bot added type:Compiler Compiler support or related warnings type:Infrastructure Infrastructure/ecosystem related changes, such as CMake or buildbots area:Python wrapping Python bindings for a class type:Testing Ensure that the purpose of a class is met/the results on a wide set of test cases are correct area:Bridge Issues affecting the Bridge module area:Core Issues affecting the Core module area:Documentation Issues affecting the Documentation module labels Jul 27, 2026

@dzenanz dzenanz left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good on a glance. Matt and Brad should review this.

Comment thread Modules/Bridge/VtkGlue/wrapping/VtkGlue.i
@hjmjohnson

Copy link
Copy Markdown
Member Author

Force-push a8b614a04430bd48d4d0b: content-only fix for the ITK.macOS.Python failure (no rebase mixed in — the branch was already current with main).

libITKCommon failed to link with undefined _Py_BuildValue / _Py_Dealloc. Fixed by folding into the commit that introduced it.

Root cause

Modules/Core/Common/src/CMakeLists.txt has two uses of _itk_python_target: the INCLUDE_DIRECTORIES source property, and target_link_libraries(ITKCommon PRIVATE ${_itk_python_target}) about 100 lines further down. The commit that replaced the variable with Python3::SABIModule updated only the first, leaving the second to expand to an empty string. ITKCommon then stopped linking Python3::SABIModule and lost its INTERFACE_LINK_OPTIONS, i.e. -undefined dynamic_lookup.

It failed on exactly one configuration because three things have to line up: an undefined CMake variable expands to empty without error, ELF shared objects tolerate undefined symbols, and static archives never resolve them at all. Only a shared macOS build links libITKCommon.dylib strictly enough to notice — hence green Linux Python (177 tests) and green CDash while macOS Python failed at 764/5404 with 0 tests run.

Verified by configuring main and this branch side by side with BUILD_SHARED_LIBS=ON: main emits 98 dynamic_lookup link options, the broken branch 97, and the fixed branch 98 again, with the missing one being libITKCommon. libITKCommon-6.0.1.dylib now links, and nm -u shows _Py_BuildValue and _Py_Dealloc present as permitted undefined symbols.

@hjmjohnson
hjmjohnson requested review from blowekamp and thewtex July 27, 2026 12:05
@hjmjohnson
hjmjohnson marked this pull request as ready for review July 27, 2026 12:27
@greptile-apps

This comment was marked as resolved.

Comment thread Modules/Bridge/VtkGlue/wrapping/VtkGlue.i
@hjmjohnson
hjmjohnson force-pushed the abi3-vtkglue branch 2 times, most recently from c046803 to d542b02 Compare July 27, 2026 13:47
@hjmjohnson

Copy link
Copy Markdown
Member Author

Two force-pushes just landed, split by concern: 33b3a662354 → c046803a5c7 is a plain rebase on main (nothing else), and c046803a5c7 → d542b02554f is content only — the fix for Greptile's P1 unvalidated-pointer finding. The second compare link shows just the two VtkGlue files.

@blowekamp

Copy link
Copy Markdown
Member

What are the implications of this for building against the Python Free Thread ABI. Currently Free Threads are not stable and there is expected to be a separate stable ABI for 3.15:
https://peps.python.org/pep-0803/

@dzenanz

dzenanz commented Jul 28, 2026

Copy link
Copy Markdown
Member

I read this, and the requirements for abi3t are abi3 + no GIL locking. So this PR is a step in the right direction, as best as I can tell.

@hjmjohnson

Copy link
Copy Markdown
Member Author

I do think this is a mandatory step toward abi3t, but I am not an expert on those details. The primary motivator was to remove complexity in the cmake options, and remove unnecessary dependance on non-abi3 code.

@thewtex thewtex left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we necessarily want to remove support for building without abi3. Linux distributions, etc. may want/need to not use abi3.

@thewtex

thewtex commented Aug 4, 2026

Copy link
Copy Markdown
Member

Yes, abi3t will need to be a separate build.

@blowekamp

Copy link
Copy Markdown
Member

I don't think we necessarily want to remove support for building without abi3. Linux distributions, etc. may want/need to not use abi3.

I agree with this. For example for the condo-forge distribution they are still build per python distribution. It is unclear if the SWIG generated code utilizes the extra available interfaces or not in these configurations.

@github-actions github-actions Bot removed area:Core Issues affecting the Core module area:Documentation Issues affecting the Documentation module labels Aug 5, 2026
@hjmjohnson

hjmjohnson commented Aug 5, 2026

Copy link
Copy Markdown
Member Author

@thewtex @blowekamp — thanks both; your comments changed the shape of this PR.

It is now only the ITKVtkGlue fix: one commit, seven files, all under Modules/Bridge/VtkGlue/. ITK_USE_PYTHON_LIMITED_API is untouched, keeps its current default, and no Python-discovery or wrapping-macro behaviour changes here. The earlier revision removed that option; that is withdrawn.

The ABI questions you raised are tracked separately in #6757 — free-threaded CPython, abi3t, whether ITK should default to abi3, and alternative binding generators. Python 3.15 makes the actions much more clear about what to do, so it is a roadmap, not a work item.

Because Limited API calls are also valid in a full-API build, one implementation serves both modes — the module does not become abi3-only, and the FATAL_ERROR is removed rather than inverted. Verified on macOS arm64 with VTK 9.6: both ITK_USE_PYTHON_LIMITED_API=ON and =OFF build 8565/8565 targets and pass 177/177 ctest -R Python.

conda-forge: abi3 is compatible with per-version builds, and does not require them to collapse

Checked against conda-forge's own recipe machinery rather than assumed.

conda-forge supports abi3 explicitly, gated per recipe on an is_abi3 selector. When a feedstock opts in, recipe/example-meta.yaml in python-abi3-feedstock sets:

python_version_independent: true   # [is_abi3]
skip: True                         # [is_abi3 and not is_python_min]

plus a python-abi3 host dependency and an abi3audit test. That produces one version-independent artifact built at python_min, instead of one per Python version.

The relevant point for ITK: this is opt-in. A feedstock that keeps its per-version matrix and simply compiles abi3 binaries is perfectly valid — an abi3 extension loads on any interpreter at or above its floor, so a per-cp3XX package containing one is a strict superset of what that package needs. abi3 makes the per-version matrix unnecessary; it does not make it unusable.

For reference, conda-forge/itk-feedstock (ITK 5.4.7) currently builds per version — 18 .ci_support variants, python 3.10/3.11/3.12 × hdf5 1.14.6/2 × linux_64/linux_aarch64/osx_arm64 — and every one is ____cpython, i.e. no free-threaded variants today.

One correction to how this is sometimes stated. Version-specific builds are not needed "for abi3t under 3.13/3.14" — abi3t does not exist there. Per PEP 803 the Stable ABI is simply unavailable on free-threaded builds:

"Extensions will fail to build for the Stable ABI on free-threaded Python (that is, when both Py_LIMITED_API and Py_GIL_DISABLED preprocessor macros are defined)."

So free-threaded 3.13t/3.14t require a build that is neither abi3 nor abi3t, which is exactly why a version-specific path has to stay reachable.

That changes at 3.15, but not to a single build: PEP 803 recommends shipping both, tagged abi3.abi3t, compiled with Py_LIMITED_API and Py_TARGET_ABI3T respectively. So the end state is two ABI variants per platform rather than one per interpreter version. CMake already models the switch — no new find_package component is needed, since USE_SABI picks Py_TARGET_ABI3T over Py_LIMITED_API when Python3_FREE_THREADED is true and the version is 3.15+ (CMake 4.4). Note CMake does not cover the 3.13t/3.14t gap: it still defines Py_LIMITED_API there, so the guard has to be ours. All captured in #6757.

@hjmjohnson
hjmjohnson requested review from dzenanz and thewtex August 5, 2026 18:23
@dzenanz

dzenanz commented Aug 5, 2026

Copy link
Copy Markdown
Member

I still like this. I don't have time to carefully review.

@hjmjohnson hjmjohnson changed the title COMP: Build all ITK Python wrapping as abi3 (stable ABI) COMP: Make ITKVtkGlue wrapping abi3-compatible Aug 6, 2026
@hjmjohnson

Copy link
Copy Markdown
Member Author

@thewtex would you be able to review this PR?

@thewtex

thewtex commented Aug 24, 2026

Copy link
Copy Markdown
Member

I defer to VTK developers.

@dzenanz
dzenanz requested a review from jadh4v August 24, 2026 14:43
@hjmjohnson

Copy link
Copy Markdown
Member Author

@dzenanz @thewtex Do you know who uses this mechanism and could review?

The purpose for addressing this conversion was to simplify the interplay between sabi vs non-sabi across Python builds. These changes represent using the stable-abi in such a way that it works both with and without the requirement being set.

@dzenanz dzenanz left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that Slicer is a prominent user of this. Another is probably ITK-SNAP. Maybe MITK too?

I am formally approving this, despite not being entirely certain that this works. It is not very likely we will get another review. I am relying on your testing.

Exchange pointers with VTK's Python layer through the `__this__` and
`Addr=0x...` encodings using only Limited API calls, instead of through
vtkPythonUtil, whose header chain accesses PyTypeObject members that
Py_LIMITED_API hides.

Dropping VTK::WrappingPythonCore from the wrapping link interface is
required for correctness, not tidiness: that library is built against one
libpython, so an extension linking it cannot be version-agnostic.

Neither encoding is documented VTK API, so PythonVtkGlueABI3EncodingTest
asserts both still hold and PythonVtkGlueRoundTripTest exercises the
typemaps end to end.

Closes: InsightSoftwareConsortium#6711
VtkGlueRoundTripTest exercises the vtkImageData pair. The vtkImageImport
and vtkImageExport accessors, non-float pixel types, 3D images, object
identity across the address round trip, lifetime after the producing
filter is released, and rejection of unconvertible input were untested.

The test avoids numpy so it runs wherever ITKVtkGlue is wrapped, without
requiring ITKBridgeNumPy.
@hjmjohnson

Copy link
Copy Markdown
Member Author

@dzenanz — you said you liked this but didn't have time to review carefully, and @thewtex deferred to VTK developers. Rather than ask for more review time, I expanded the test coverage and validated the branch on two platforms so the evidence stands on its own.

Every configuration passes, including a pre-PR baseline. Two OSes, two VTK minor versions, both wrapping modes.

Environment LIMITED_API C++ Python Total
Linux x86_64, VTK 9.1, py3.12 OFF 3/3 3/3 8/8
Linux x86_64, VTK 9.1, py3.12 ON 3/3 3/3 8/8
macOS arm64, VTK 9.6.2, py3.13 OFF 3/3 3/3 8/8
macOS arm64, VTK 9.6.2, py3.13 ON 3/3 3/3 8/8
macOS arm64 — main without this PR baseline 3/3 n/a 3/3

The Linux rows are new — the original description was macOS-only. The branch was also rebased onto current main; git range-diff reports the patch byte-identical across 154 commits of upstream churn, and it now compiles alongside the VTK_FUTURE_CONST change from a53fc61.

New test: PythonVtkGlueTypemapCoverageTest

VtkGlue.i declares six Python typemaps. VtkGlueRoundTripTest exercised two of them, so the new test covers the rest and the cases where a typemap can look correct on the happy path:

Area Rationale
GetImporter() / GetExporter() The out vtkImageImport* and out vtkImageExport* typemaps had no Python coverage at all
itk.UC, itk.SS, 3-D itk.F Existing coverage is float/2-D only; a cast correct there can be wrong elsewhere
Object identity Two GetOutput() calls must return the same underlying VTK address — the out typemap rebuilds a wrapper from a raw address, so this is the natural failure mode
Lifetime The wrapper must outlive the producing filter; plus ten repeated round trips
Rejection vtkPoints, a literal "Addr=0x0" string, and an integer must each be refused rather than reinterpreted as a pointer
Scalar-type mismatch A VTK_DOUBLE image must be rejected for an itk.F filter, not silently reinterpreted

It deliberately avoids numpy, so it runs wherever ITKVtkGlue is wrapped without requiring ITKBridgeNumPy.

Worth noting for context: Modules/Bridge/VtkGlue/wrapping/test/ did not exist before this PR. There were no Python tests for VtkGlue at all, so the typemaps have never been exercised in ITK's test suite until now.

Artifact audit, re-verified on the rebased branch

_ITKVtkGluePython.abi3.so, limited-API build:

Property Value
PyVTK* / vtkPythonUtil symbols 0
libvtkWrappingPythonCore linked 0
libpython linked 0
Undefined Py* symbols 72, all Limited API

The OFF build produces _ITKVtkGluePython.so with WrappingPythonCore still linked, as before.

A macOS detour worth recording, since it is not a defect in this PR

The three C++ tests initially segfaulted on macOS. They segfault identically on unpatched main, so I chased it to the end rather than assume.

The cause is Slicer's VTK being a kit build (VTK_ENABLE_KITS=ON, no libvtkCommonCore). A ten-line VTK-only program with no ITK crashes the same way — printing its output, returning from main, then dying in static teardown. Rebuilding VTK non-kit makes both the reproducer and all three ITK tests pass.

So the macOS rows above use a purpose-built VTK 9.6.2: non-kit, rendering enabled, Python-wrapped for the same 3.13.9 interpreter ITK uses, same compiler and deployment target as ITK. Nothing to fix here; recording it so the next person testing VtkGlue against a Slicer VTK does not lose an evening to it.

Separately, this surfaced #6846: ITKVtkGlue cannot configure against a rendering-free VTK because the VTK_RENDERING_BACKEND guard in itk-module-init.cmake tests a VTK 8 variable that VTK 9 never defines, making the non-rendering branch dead code. Unrelated to this PR and out of scope; I will propose the four-line fix after this merges, since both touch the same file.

@dzenanz

dzenanz commented Sep 6, 2026

Copy link
Copy Markdown
Member

CI doesn't do a Windows Python build. Let me do that locally.

@hjmjohnson

Copy link
Copy Markdown
Member Author

CI doesn't do a Windows Python build. Let me do that locally.

You need to explicitly request this; it is not enabled by default. And it requires a VTK build with rendering to be available.

@dzenanz

dzenanz commented Sep 6, 2026

Copy link
Copy Markdown
Member

It does not get built:

dzenan.zukic@ZERATUL3 MINGW64 /m/CDev/ITK-py-namespace
$ ctest -C Release -R Vtk
Test project M:/CDev/ITK-py-namespace
    Start 4600: ITKVtkGlueKWStyleTest
1/4 Test #4600: ITKVtkGlueKWStyleTest .....................   Passed    0.02 sec
    Start 4601: ITKVtkGlueInDoxygenGroup
2/4 Test #4601: ITKVtkGlueInDoxygenGroup ..................   Passed    0.04 sec
    Start 4605: itkVtkMedianImageFilterTest
3/4 Test #4605: itkVtkMedianImageFilterTest ...............   Passed    0.34 sec
    Start 4608: itkVtkConnectedComponentImageFilterTest
4/4 Test #4608: itkVtkConnectedComponentImageFilterTest ...   Passed    0.37 sec

100% tests passed out of 4

Label Time Summary:
ITKVtkGlue          =   0.76 sec*proc (4 tests)
REQUIRES_DISPLAY    =   0.71 sec*proc (2 tests)

Total Test time (real) =   1.04 sec

dzenan.zukic@ZERATUL3 MINGW64 /m/CDev/ITK-py-namespace

ITK's CMakeCache.txt
VTK's CMakeCache.txt

@dzenanz

dzenanz commented Sep 6, 2026

Copy link
Copy Markdown
Member

On my work computer, it gets built:

Dzenan@Ryzenator MINGW64 /c/Misc/ITK-patches-dev-py26
$ ctest -C Release -VV -R Vtk
UpdateCTestConfiguration  from :C:/Misc/ITK-patches-dev-py26/DartConfiguration.tcl
Parse Config file:C:/Misc/ITK-patches-dev-py26/DartConfiguration.tcl
Test project C:/Misc/ITK-patches-dev-py26
Constructing a list of tests
Done constructing a list of tests
Updating test list for fixtures
Added 0 tests to meet fixture requirements
Checking test dependency graph...
Checking test dependency graph end
test 375
    Start 375: ITKVtkGlueKWStyleTest

375: Test command: C:\Dev\KWStyle.exe "-xml" "C:/Misc/ITK-patches-dev/CMake/../Utilities/KWStyle/ITK.kws.xml" "-v" "-o" "C:/Misc/ITK-patches-dev/Modules/Bridge/VtkGlue/ITKKWStyleOverwrite.txt" "-D" "C:/Misc/ITK-patches-dev-py26/Modules/Bridge/VtkGlue/ITKKWStyleFiles.txt" "-gcc"
375: Working Directory: C:/Misc/ITK-patches-dev/CMake/..
375: Test timeout computed to be: 1500
1/7 Test #375: ITKVtkGlueKWStyleTest .....................   Passed    0.15 sec
test 376
    Start 376: ITKVtkGlueInDoxygenGroup

376: Test command: "C:\Program Files\Python312\python.exe" "C:/Misc/ITK-patches-dev/CMake/../Utilities/Doxygen/mcdoc.py" "check" "ITKVtkGlue" "C:/Misc/ITK-patches-dev/Modules/Bridge/VtkGlue/include"
376: Working Directory: C:/Misc/ITK-patches-dev-py26/Modules/Bridge/VtkGlue
376: Test timeout computed to be: 1500
376: 5 headers checked.
2/7 Test #376: ITKVtkGlueInDoxygenGroup ..................   Passed    0.11 sec
test 380
    Start 380: itkVtkMedianImageFilterTest

380: Test command: C:\Misc\ITK-patches-dev-py26\Wrapping\Generators\Python\itk\ITKVtkGlueTestDriver.exe "itkVtkMedianFilterTest" "C:/Misc/ITK-patches-dev-py26/ExternalData/Testing/Data/Input/cthead1.png" "2"
380: Working Directory: C:/Misc/ITK-patches-dev-py26/Modules/Bridge/VtkGlue/test
380: Test timeout computed to be: 1500
3/7 Test #380: itkVtkMedianImageFilterTest ...............Exit code 0xc0000135***Exception:   4.33 sec
test 383
    Start 383: itkVtkConnectedComponentImageFilterTest

383: Test command: C:\Misc\ITK-patches-dev-py26\Wrapping\Generators\Python\itk\ITKVtkGlueTestDriver.exe "itkVtkConnectedComponentImageFilterTest" "C:/Misc/ITK-patches-dev-py26/ExternalData/Testing/Data/Input/cthead1.png"
383: Working Directory: C:/Misc/ITK-patches-dev-py26/Modules/Bridge/VtkGlue/test
383: Test timeout computed to be: 1500
4/7 Test #383: itkVtkConnectedComponentImageFilterTest ...Exit code 0xc0000135***Exception:   0.68 sec
test 518
    Start 518: PythonVtkGlueABI3EncodingTest

518: Test command: C:\Misc\ITK-patches-dev-py26\Wrapping\Generators\Python\itk\itkTestDriver.exe "--add-before-env" "PYTHONPATH" "C:/Misc/ITK-patches-dev-py26/Wrapping/Generators/Python" "--add-before-env" "PYTHONPATH" "C:/Misc/ITK-patches-dev-py26/Wrapping/Generators/Python/itk" "--add-before-libpath" "C:/Misc/ITK-patches-dev-py26/Wrapping/Generators/Python/itk" "C:/Program Files/Python312/python.exe" "C:/Misc/ITK-patches-dev/Modules/Bridge/VtkGlue/wrapping/test/VtkGlueABI3EncodingTest.py"
518: Working Directory: C:/Misc/ITK-patches-dev/Modules/Bridge/VtkGlue/wrapping/test
518: Environment variables:
518:  PYTHONPATH=C:/Libs/vtk-vs22/lib/site-packages
518: Test timeout computed to be: 1500
518: Traceback (most recent call last):
518:   File "C:\Misc\ITK-patches-dev\Modules\Bridge\VtkGlue\wrapping\test\VtkGlueABI3EncodingTest.py", line 39, in <module>
518:     from vtkmodules.vtkCommonDataModel import vtkImageData, vtkPolyData
518:   File "C:\Libs\vtk-vs22\lib\site-packages\vtkmodules\__init__.py", line 106, in <module>
518:     _load_vtkmodules_static()
518:   File "C:\Libs\vtk-vs22\lib\site-packages\vtkmodules\__init__.py", line 104, in _load_vtkmodules_static
518:     import _vtkmodules_static
518: ImportError: DLL load failed while importing _vtkmodules_static: The specified module could not be found.
518: itkTestDriver: Process exited with return value: 1
5/7 Test #518: PythonVtkGlueABI3EncodingTest .............***Failed    0.55 sec
test 519
    Start 519: PythonVtkGlueRoundTripTest

519: Test command: C:\Misc\ITK-patches-dev-py26\Wrapping\Generators\Python\itk\itkTestDriver.exe "--add-before-env" "PYTHONPATH" "C:/Misc/ITK-patches-dev-py26/Wrapping/Generators/Python" "--add-before-env" "PYTHONPATH" "C:/Misc/ITK-patches-dev-py26/Wrapping/Generators/Python/itk" "--add-before-libpath" "C:/Misc/ITK-patches-dev-py26/Wrapping/Generators/Python/itk" "C:/Program Files/Python312/python.exe" "C:/Misc/ITK-patches-dev/Modules/Bridge/VtkGlue/wrapping/test/VtkGlueRoundTripTest.py"
519: Working Directory: C:/Misc/ITK-patches-dev/Modules/Bridge/VtkGlue/wrapping/test
519: Environment variables:
519:  PYTHONPATH=C:/Libs/vtk-vs22/lib/site-packages
519: Test timeout computed to be: 1500
519: Traceback (most recent call last):
519:   File "C:\Misc\ITK-patches-dev\Modules\Bridge\VtkGlue\wrapping\test\VtkGlueRoundTripTest.py", line 30, in <module>
519:     from vtkmodules.vtkCommonDataModel import vtkImageData
519:   File "C:\Libs\vtk-vs22\lib\site-packages\vtkmodules\__init__.py", line 106, in <module>
519:     _load_vtkmodules_static()
519:   File "C:\Libs\vtk-vs22\lib\site-packages\vtkmodules\__init__.py", line 104, in _load_vtkmodules_static
519:     import _vtkmodules_static
519: ImportError: DLL load failed while importing _vtkmodules_static: The specified module could not be found.
519: itkTestDriver: Process exited with return value: 1
6/7 Test #519: PythonVtkGlueRoundTripTest ................***Failed    0.83 sec
test 520
    Start 520: PythonVtkGlueTypemapCoverageTest

520: Test command: C:\Misc\ITK-patches-dev-py26\Wrapping\Generators\Python\itk\itkTestDriver.exe "--add-before-env" "PYTHONPATH" "C:/Misc/ITK-patches-dev-py26/Wrapping/Generators/Python" "--add-before-env" "PYTHONPATH" "C:/Misc/ITK-patches-dev-py26/Wrapping/Generators/Python/itk" "--add-before-libpath" "C:/Misc/ITK-patches-dev-py26/Wrapping/Generators/Python/itk" "C:/Program Files/Python312/python.exe" "C:/Misc/ITK-patches-dev/Modules/Bridge/VtkGlue/wrapping/test/VtkGlueTypemapCoverageTest.py"
520: Working Directory: C:/Misc/ITK-patches-dev/Modules/Bridge/VtkGlue/wrapping/test
520: Environment variables:
520:  PYTHONPATH=C:/Libs/vtk-vs22/lib/site-packages
520: Test timeout computed to be: 1500
520: Traceback (most recent call last):
520:   File "C:\Misc\ITK-patches-dev\Modules\Bridge\VtkGlue\wrapping\test\VtkGlueTypemapCoverageTest.py", line 32, in <module>
520:     from vtkmodules.vtkCommonDataModel import vtkImageData
520:   File "C:\Libs\vtk-vs22\lib\site-packages\vtkmodules\__init__.py", line 106, in <module>
520:     _load_vtkmodules_static()
520:   File "C:\Libs\vtk-vs22\lib\site-packages\vtkmodules\__init__.py", line 104, in _load_vtkmodules_static
520:     import _vtkmodules_static
520: ImportError: DLL load failed while importing _vtkmodules_static: The specified module could not be found.
520: itkTestDriver: Process exited with return value: 1
7/7 Test #520: PythonVtkGlueTypemapCoverageTest ..........***Failed    2.47 sec

The following tests passed:
        ITKVtkGlueKWStyleTest
        ITKVtkGlueInDoxygenGroup

29% tests passed, 5 tests failed out of 7

Label Time Summary:
ITKFFTImageFilterInit    =   3.85 sec*proc (3 tests)
ITKVtkGlue               =   5.27 sec*proc (4 tests)
Python                   =   3.85 sec*proc (3 tests)
REQUIRES_DISPLAY         =   5.01 sec*proc (2 tests)

Total Test time (real) =  11.19 sec

The following tests FAILED:
        380 - itkVtkMedianImageFilterTest (Exit code 0xc0000135) ITKVtkGlue REQUIRES_DISPLAY
        383 - itkVtkConnectedComponentImageFilterTest (Exit code 0xc0000135) ITKVtkGlue REQUIRES_DISPLAY
        518 - PythonVtkGlueABI3EncodingTest (Failed)            ITKFFTImageFilterInit Python
        519 - PythonVtkGlueRoundTripTest (Failed)               ITKFFTImageFilterInit Python
        520 - PythonVtkGlueTypemapCoverageTest (Failed)         ITKFFTImageFilterInit Python
Errors while running CTest
Output from these tests are in: C:/Misc/ITK-patches-dev-py26/Testing/Temporary/LastTest.log
Use "--rerun-failed --output-on-failure" to re-run the failed cases verbosely.

Dzenan@Ryzenator MINGW64 /c/Misc/ITK-patches-dev-py26

In two cases, the problem is:

ITKVtkGlueTestDriver.exe - System Error

The code execution cannot proceed because vtktoken-9.7.dll was not found. Reinstalling the program may fix this problem. 

ITK's CMakeCache.txt
VTK's CMakeCache.txt

@hjmjohnson

Copy link
Copy Markdown
Member Author

Windows validation, filling the gap this PR disclaims. 8/8 in both wrapping
modes
, matching the Linux and macOS rows.

Environment LIMITED_API C++ Python Style Total
Windows 11 x86_64, VTK 9.6.2, py3.12.0 OFF 3/3 3/3 2/2 8/8
Windows 11 x86_64, VTK 9.6.2, py3.12.0 ON 3/3 3/3 2/2 8/8

MSVC 19.38.33145 (VS 2022, toolset 14.38), Ninja, Release x64. VTK 9.6.2
(f49a1dba) built from source: shared, VTK_ENABLE_KITS=OFF, rendering
WANT + RenderingOpenGL2=YES, VTK_WRAP_PYTHON=ON, wrapped for the same
3.12.0 interpreter given to ITK. Branch at 23230cb4be on base 6207a873a0.
Builds were 3463 targets each, 0 errors. The registered test set was checked
against the expected eight names before running, so nothing silently failed to
register.

Artifact audit — the Windows form of the abi3 claim

Property LIMITED_API=OFF LIMITED_API=ON
Python DLL dependency python312.dll python3.dll
vtkWrappingPythonCore absent absent
PyVTK* imports 0 0
vtkPythonUtil imports 0 0
distinct Py* imports 61 72

The ON module imports the stable-ABI forwarder python3.dll and links
python3.lib; OFF imports the version-locked python312.dll and links
python312.lib. This is the one part of the abi3 claim Windows can show
directly that POSIX cannot.

Three Windows-specific notes

The .abi3 suffix check does not work here. Both modes produce plain
_ITKVtkGluePython.pyd. That is not a defect: EXTENSION_SUFFIXES on Windows
is ['.cp312-win_amd64.pyd', '.pyd'], so .pyd already is the stable-ABI
suffix and there is no .abi3.pyd form. On Windows, verify the mode with
dumpbin /dependents and look for python3.dll, not by filename.

vtkWrappingPythonCore is absent in both modes, where the macOS audit
records it still linked in the OFF build. Windows only records a DLL
dependency when symbols are actually imported, and the new typemaps import
none, so the OFF build sheds it too.

The C++ tests need VTK's bin on PATH. Without it they fail with
0xc0000135 (STATUS_DLL_NOT_FOUND) before reaching main. RPATH covers this
on POSIX; nothing does on Windows. The Python tests pass regardless because
CMake sets their environment, so the failure presents as "C++ broken, Python
fine," which reads like a PR defect and is not.

@dzenanz — your two runs, from the caches you attached

Your failures look like configuration rather than anything in this branch.
Three separate causes:

1. vtktoken-9.7.dll was not found / 0xc0000135 on
itkVtkMedianImageFilterTest and itkVtkConnectedComponentImageFilterTest is
the PATH item above. Prepending your VTK's bin directory fixes it; that
alone took my C++ tests from 3 failures to 3 passes with no rebuild.

2. The three ImageToVTK / VTKImageTo C++ tests never registered in your
build.
Your ITK cache has ITK_BUILD_DEFAULT_MODULES:BOOL=OFF together with
Module_ITKTestKernel:INTERNAL=OFF — the variable stayed INTERNAL because it
was set without an explicit type, so ITKTestKernel is off and those tests are
never created. Passing -DModule_ITKTestKernel:BOOL=ON is what makes them
appear.

Worth noting separately: ctest -R Vtk is case-sensitive and does not match
itkImageToVTKImageFilterTest, itkImageToVTKImageFilterRGBTest or
itkVTKImageToImageFilterTest. On my build that filter returns exactly your
seven tests; -R "VtkGlue|ImageToVTK|VTKImageTo" returns all eight. So even
with ITKTestKernel enabled, the narrower filter would still hide them.

3. ImportError: DLL load failed while importing _vtkmodules_static is
because your VTK is a static build — your VTK cache has
BUILD_SHARED_LIBS:BOOL=OFF, which bundles the Python wrappers into a single
_vtkmodules_static module instead of per-module .pyd files. My VTK is
BUILD_SHARED_LIBS=ON and produces 71 individual .pyd modules, which import
without any of this. I did not try to make a static-VTK Python wrapping work;
building VTK shared was simpler and is what the other platforms used.

One more that will bite after those are fixed: your ITK cache has
Module_ITKBridgeNumPy:BOOL=OFF, and PythonVtkGlueRoundTripTest calls
itk.image_from_array. It will fail with module 'itk' has no attribute 'PyBuffer' once it gets past the vtkmodules import.
PythonVtkGlueTypemapCoverageTest is deliberately numpy-free and does not need
it.

Configuration that produced 8/8 here, if useful:

-DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=ON -DBUILD_EXAMPLES=OFF
-DITK_BUILD_DEFAULT_MODULES=OFF
-DModule_ITKVtkGlue=ON
-DModule_ITKTestKernel:BOOL=ON
-DModule_ITKBridgeNumPy:BOOL=ON
-DITK_WRAP_PYTHON=ON
-DITK_USE_PYTHON_LIMITED_API=OFF   # and again with =ON
-DPython3_EXECUTABLE=C:/Python312-x64/python.exe
-DVTK_DIR=<shared VTK build>

Not verified here: the cross-interpreter check, which needs a second VTK
wrapped for a different Python.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:Bridge Issues affecting the Bridge module area:Python wrapping Python bindings for a class type:Compiler Compiler support or related warnings type:Infrastructure Infrastructure/ecosystem related changes, such as CMake or buildbots type:Testing Ensure that the purpose of a class is met/the results on a wide set of test cases are correct

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ITKVtkGlue blocks abi3 / ITK_USE_PYTHON_LIMITED_API builds: wrapping depends on VTK's non-Limited-API Python core

4 participants