[7oct2023] pybind11 transitive library deps
Table of Contents
Experiencing link difficulties with transitive library dependencies.
1. Update
My original invesgation (below) mistaken. See update cmake headeronly deps.
2. Setup (before Update)
- cmake version 3.25.3
- pybind11 version ???
- nix build (see https://github.com:rconybea/xo-nix2)
Consequences of nix build:
- Each package installed to a separate directory – no "common swimming pool" like
/usr/lib
- Implies install directory always distinct from any directory containing build inputs
- Tends to reveal oversights in toolchain, as we'll see below
- Each package installed to a separate directory – no "common swimming pool" like
- pybind library (
xo-pyreflect
) with dependency on a separate library (xo-reflect
), that in turn has secondary dependencies (xo-refcnt
,xo-indentlog
). Note thatxo-indentlog
is header-only. Expect this cmake script to work:
find_package(pybind11) pybind11_add_module(pyreflect pyreflect.cpp) find_package(reflect CONFIG REQUIRED) target_link_libraries(pyreflect PUBLIC reflect)
3. Problem (before Update)
Instead, link fails. Link line something like:
g++ -fPIC ... -o pyreflect.cpython-311-x86_64-linux-gnu.so /path/to/libreflect.so -lrefcnt -lindentlog
Two problems here:
- directory containing
librefcnt.so
isn't on the link line (no-L/path/to/refcnt/dir
for example). libindentlog.so
does not exist, sinceindentlog
is header-only
- directory containing
Looked into intermediate outputs like
lib/cmake/reflectTargets.cmake
, excerpt:set_target_properties(reflect PROPERTIES INTERFACE_INCLUDE_DIRECTORIES "${_IMPORT_PREFIX}/include" INTERFACE_LINK_LIBRARIES "indentlog;refcnt" )
It's not obvious how
xo_pyreflect
can know thatindentlog
is header-only, whilerefcnt
isn't (though could presumably extract the relevant libdir fromfind_package()
with some work).
4. Workaround (before Update)
Recognize that
pyreflect
link shouldn't needrefcnt
on the link line, sincelibreflect.so
has aDT_NEEDED
entry for it.$ readelf -d /path/to/libreflect.so Dynamic section at offset 0x17860 contains 34 entries: Tag Type Name/Value 0x0000000000000001 (NEEDED) Shared library: [librefcnt.so.1] ...
When building
pyreflect
, suppress transitive dependencies For example:# xo_cxx.cmake macro(xo_pybind11_dependency target dep) find_package(${dep} CONFIG REQUIRED) set_property(TARGET ${dep} PROPERTY INTERFACE_LINK_LIBRARIES "") target_link_libraries(${target} PUBLIC ${dep}) endmacro()
Then in
.cmake
forpyreflect
, something equivalent to:pybind11_add_module(pyreflect pyreflect.cpp) xo_pybind11_dependency(pyreflect reflect)