[question] Conan 2.x Migration - transitive_headers and transitive_libs Trait Defaults

[radonish]

What is your question?

Hello, I've read over the requirements traits documentation and want to ensure the behavior I'm seeing is as desired.

Let's say liba requires libx and liby. Let's also say that all 3 libraries are built shared. liba conanfile.py has:

def requirements(self):
    self.requires(libx/1.0.0)
    self.requires(liby/1.0.0)

In migrating the test_package for liba my first attempt was to simply have the following in test_package's conanfile.py:

def requirements(self):
    self.requires(self.tested_reference_str)

The link of test_package for liba fails due to symbols from libx and liby. (Edit: Note, the test_package is just an empty main() - no actual code/references to the transitive dependencies.)

If I update liba's conanfile.py as follows, test_package can link:

def requirements(self):
    self.requires(libx/1.0.0, transitive_headers=True, transitive_libs=True)
    self.requires(liby/1.0.0, transitive_headers=True, transitive_libs=True)

I think I would have expected the transitive_headers and transitive_libs to be True by default since the libraries are shared. Any insight on the behavior/my assumptions would be appreciated. Thank you!

Have you read the CONTRIBUTING guide?

• I've read the CONTRIBUTING guide

[memsharded]

Hi @radonish

Thanks for your question.

First, the transitive_headers=True are only necessary if the public headers of the package is #including the headers of the libx/1.0 and liby/1.0 respectively. This is not considered the default, as the default is try to hide as an implementation detail the transitive dependencies and expose to the consumers just the public interface of the current package. But some libraries, for example a "mathlib" that uses "eigen" as a base library might want to include eigen in the public headers and expose the eigen matrix types to the end consumer. For this case transitive_headers=True is what needs to be done

transitive_libs=True should be way more exceptional. Conan already manages correctly the propagation of transitive linkage requirements ,taking into account the package_type = shared-library/static-library/header-library. This is done by explicitly defining the package_type or by defining a shared option (the package_type will be implicitly deduced from the option value). With this, Conan can propagate things. You can easily check with:

$ mkdir liba && cd liba && conan new cmake_lib -d name=liba
$ conan create .
$ cd ..
$ mkdir libb && cd libb && conan new cmake_lib -d name=libb -d requires=liba/0.1
$ conan create .

This also works for shared libraries, you can add -o "*:shared=True" to the commands above and it will still work.

So the only case where transitive_libs=True is necessary is when transitive_headers=True and also those header require some linkage of code that is not already propagated automatically by the transitive linkage.

[radonish]

Hello @memsharded, thank you for the quick response.

I ran the commands you gave and do see the expected behavior for those test packages. I'm comparing the generated recipes with the actual recipes my team is using in order to determine what is going wrong.

One key observation when looking at the linker command line for the link failure that occurs in the test_package for our library is that the library paths are there (-L parameters for transitive dependencies are contained) but the specific transitive library path/names are missing. That makes me wonder if it is a package_info problem where the libs for the transitive dependency packages aren't being properly set/advertised.

(In one example our proprietary library - let's call it libx - is simply dependent on the Conan Center's POCO library package. libx builds and links fine, but its test_package fails with linker errors that it cannot find POCO. As noted above, the linker command line contains the -L parameter to the Conan cache for POCO but I do not see any of the POCO libraries called out.)

[radonish]

@memsharded, the issue appears to occur when cross-compiling. An example cross-compile host profile that fails with your liba/libb test is this aarch64 one below:

$ cat ~/.conan2/profiles/profile-aarch64-buildroot-release
{% set TARGET_HOST = "aarch64-buildroot-linux-gnu" %}
{% set SYSROOT = "/tools/toolchains/aarch64-buildroot-linux-gnu_sdk-buildroot/aarch64-buildroot-linux-gnu/sysroot" %}

[settings]
os=Linux
arch=armv8
compiler=gcc
compiler.version=8.4
compiler.libcxx=libstdc++11
compiler.cppstd=gnu11
build_type=Release

[options]

[buildenv]
CONAN_CMAKE_SYSTEM_NAME=Linux
CONAN_CMAKE_SYSTEM_PROCESSOR=aarch64
PATH=+(path)/tools/git_2.42.0/bin:/tools/git_lfs_3.4.0/bin:/tools/python_3.11.4_venv/company_2.0.0/bin:/tools/cmake_3.30.5/bin:/tools/toolchains/aarch64-buildroot-linux-gnu_sdk-buildroot/bin
CC={{ TARGET_HOST }}-gcc
CXX={{ TARGET_HOST }}-g++
STRIP={{ TARGET_HOST }}-strip
LD={{ TARGET_HOST }}-ld
CFLAGS=--sysroot={{ SYSROOT }} -O3 -fPIC
CXXFLAGS=--sysroot={{ SYSROOT }} -O3 -fPIC -std=gnu++11
LDFLAGS=--sysroot={{ SYSROOT }} -L{{ SYSROOT }}/usr/lib -s

It works with a host profile that matches the build profile:

[settings]
os=Linux
arch=x86_64
compiler=gcc
compiler.version=4.8
compiler.libcxx=libstdc++
compiler.cppstd=gnu11
build_type=Release

[options]

[buildenv]
COMPANY_BUILD_HOST=True
CONAN_CMAKE_SYSTEM_NAME=Linux
CONAN_CMAKE_SYSTEM_PROCESSOR=x86_64
PATH=+(path)/tools/git_2.42.0/bin:/tools/git_lfs_3.4.0/bin:/tools/python_3.11.4_venv/company_2.0.0/bin:/tools/cmake_3.30.5/bin
CC=gcc
CXX=g++
CFLAGS=-O3 -s -fPIC
CXXFLAGS=-O3 -s -fPIC -std=gnu++11
LDFLAGS=-s

[radonish]

Here is a build log that shows:

cd liba
conan create . -pr:b profile-x86_64-rhel-release -pr:h profile-aarch64-buildroot-release
cd ../libb
conan create . -pr:b profile-x86_64-rhel-release -pr:h profile-aarch64-buildroot-release

(Changed them to shared libraries within the conanfile.py itself.)

[radonish]

@memsharded, from the log above, the linker command that fails to link libb to liba for the libb test_package is:

/tools/toolchains/aarch64-buildroot-linux-gnu_sdk-buildroot/bin/aarch64-buildroot-linux-gnu-g++ --sysroot=/tools/toolchains/aarch64-buildroot-linux-gnu_sdk-buildroot/aarch64-buildroot-linux-gnu/sysroot -O3 -fPIC -std=gnu++11 -O3 -DNDEBUG --sysroot=/tools/toolchains/aarch64-buildroot-linux-gnu_sdk-buildroot/aarch64-buildroot-linux-gnu/sysroot -L/tools/toolchains/aarch64-buildroot-linux-gnu_sdk-buildroot/aarch64-buildroot-linux-gnu/sysroot/usr/lib -s CMakeFiles/example.dir/src/example.cpp.o -o example   -L/conantest/conan_data2/b/libb859c6370e04f5/p/lib  -L/conantest/conan_data2/b/liba29989e5cd9bde/p/lib  -Wl,-rpath,/conantest/conan_data2/b/libb859c6370e04f5/p/lib:/conantest/conan_data2/b/liba29989e5cd9bde/p/lib /conantest/conan_data2/b/libb859c6370e04f5/p/lib/liblibb.so

I think the problem is related to how the linker command is being constructed. My understanding is we want to use 1 of 2 options:

1. Specify the shared library paths with -L and then reference the desired shared libraries with -l or explicit full paths
2. Use the linker -rpath-link option

Instead the linker command being generated is adding a -L for each package (liba and libb) but no -l or full shared library path is included for liba. The linker is passed the -rpath option but I believe it should be -rpath-link. In fact, I manually executed the linker command above and changed -rpath to -rpath-link and the link succeeded. Note, we'd still want the -rpath flag for runtime use case.

If I change the Conan-generated libb/test_package/conanfile.py from:

def requirements(self):
    self.requires(self.tested_reference_str)

to

def requirements(self):
    self.requires(self.tested_reference_str)
    self.requires("liba/0.1")

The link succeeds because it causes the addition of the full path to liba in the cache to the linker command line.

I haven't dug into why this works OK when I'm not cross-compiling and the build and host profiles are just the default RHEL build tools. Perhaps it has something to do with GCC 4 (build tool version) vs. GCC 8 (host tool version for our buildroot-based cross toolchains) and linker nuances.

Either way, it's starting to feel like this is a cross-compilation test_package linker bug when the dependency tree is made up of shared libraries. It seems to me that the test_package linker command should include the full path to the shared libraries of the transitive dependencies (as is the case for the actual package linker command).

Thank you