[7oct2023] pybind11 transitive library deps
Table of Contents
Experiencing link difficulties with transitive library dependencies.
1. Update
My original invesgation (below) mistaken. See update cmake headeronly deps.
2. Setup (before Update)
- cmake version 3.25.3
- pybind11 version ???
- nix build (see https://github.com:rconybea/xo-nix2)
Consequences of nix build:
- Each package installed to a separate directory – no "common swimming pool" like
/usr/lib - Implies install directory always distinct from any directory containing build inputs
- Tends to reveal oversights in toolchain, as we'll see below
- Each package installed to a separate directory – no "common swimming pool" like
- pybind library (
xo-pyreflect) with dependency on a separate library (xo-reflect), that in turn has secondary dependencies (xo-refcnt,xo-indentlog). Note thatxo-indentlogis header-only. Expect this cmake script to work:
find_package(pybind11) pybind11_add_module(pyreflect pyreflect.cpp) find_package(reflect CONFIG REQUIRED) target_link_libraries(pyreflect PUBLIC reflect)
3. Problem (before Update)
Instead, link fails. Link line something like:
g++ -fPIC ... -o pyreflect.cpython-311-x86_64-linux-gnu.so /path/to/libreflect.so -lrefcnt -lindentlog
Two problems here:
- directory containing
librefcnt.soisn't on the link line (no-L/path/to/refcnt/dirfor example). libindentlog.sodoes not exist, sinceindentlogis header-only
- directory containing
Looked into intermediate outputs like
lib/cmake/reflectTargets.cmake, excerpt:set_target_properties(reflect PROPERTIES INTERFACE_INCLUDE_DIRECTORIES "${_IMPORT_PREFIX}/include" INTERFACE_LINK_LIBRARIES "indentlog;refcnt" )
It's not obvious how
xo_pyreflectcan know thatindentlogis header-only, whilerefcntisn't (though could presumably extract the relevant libdir fromfind_package()with some work).
4. Workaround (before Update)
Recognize that
pyreflectlink shouldn't needrefcnton the link line, sincelibreflect.sohas aDT_NEEDEDentry for it.$ readelf -d /path/to/libreflect.so Dynamic section at offset 0x17860 contains 34 entries: Tag Type Name/Value 0x0000000000000001 (NEEDED) Shared library: [librefcnt.so.1] ...
When building
pyreflect, suppress transitive dependencies For example:# xo_cxx.cmake macro(xo_pybind11_dependency target dep) find_package(${dep} CONFIG REQUIRED) set_property(TARGET ${dep} PROPERTY INTERFACE_LINK_LIBRARIES "") target_link_libraries(${target} PUBLIC ${dep}) endmacro()
Then in
.cmakeforpyreflect, something equivalent to:pybind11_add_module(pyreflect pyreflect.cpp) xo_pybind11_dependency(pyreflect reflect)