Skip to content

Consider attempting CDLL with SONAMEs first #21

Description

@SomeoneSerge

Hi! I noticed gpuctypes uses ctypes.util.find_library to dlopen libraries like libcuda.so by direct paths, instead of relying on the dynamic loader to choose the library by name:

gpuctypes/gpuctypes/cuda.py

Lines 146 to 148 in 77b94ad

_libraries = {}
_libraries['libcuda.so'] = ctypes.CDLL(ctypes.util.find_library('cuda'))
_libraries['libnvrtc.so'] = ctypes.CDLL(ctypes.util.find_library('nvrtc'))

Just loading libraries by name (CDLL("libcuda.so")) might often be already sufficient, because the dynamic loader is fairly configurable and the users may adjust its behaviour as is fit for their specific environment: it respects the LD_LIBRARY_PATH environment variable, it checks for the optional /etc/ld.so.conf, and certain entries in the executable's headers.

Additionally, note that the find_library routine doesn't have any clear semantics and it might return the current host's libraries, or it might randomly return a target platform's libraries accidentally found using e.g. gcc. Perhaps you could make find_library an optional "fallback mode"?

Also note that both CDLL("libcuda.so") and find_library("libcuda.so") require extra steps from the user to work, albeit hidden: on the commonly used distributions those would likely succeed and locate libcuda.so through /etc/ld.so.{cache,conf}, but these may not exist or not contain the records about all of the software available on the system. E.g. in Nixpkgs we're going to patch the referenced file(s) with our own paths. Generally, the solution is to explicitly declare the dependency in advance. Cf. e.g. https://discuss.python.org/t/pep-725-specifying-external-dependencies-in-pyproject-toml/31888 for more context

Thanks

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions