ubuntu bionic x86_64: static libc++.a version 9 cannot be used in shared libraries since build without -fPIC

[llvmbot]

Bugzilla Link	43604
Resolution	FIXED
Resolved on	Jul 27, 2021 11:24
Version	unspecified
OS	Linux
Blocks	#48661
Reporter	LLVM Bugzilla Contributor
CC	@chandlerc,@MaskRay,@ldionne,@sylvestre,@tstellar
Fixed by commit(s)	21c24ae

Extended Description

/usr/bin/ld: /usr/lib/llvm-9/lib/libc++.a(algorithm.cpp.o): relocation R_X86_64_32S against .rodata._ZNSt3__16__sortIRNS_6__lessIccEEPcEEvT0_S5_T_' can not be used when making a shared object; recompile with -fPIC /usr/bin/ld: /usr/lib/llvm-9/lib/libc++.a(chrono.cpp.o): relocation R_X86_64_32 against .rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /usr/lib/llvm-9/lib/libc++.a(condition_variable.cpp.o): relocation R_X86_64_32 against .rodata.str1.1' can not be used when making a shared object; recompile with -fPIC /usr/bin/ld: /usr/lib/llvm-9/lib/libc++.a(exception.cpp.o): relocation R_X86_64_32S against symbol _ZTVSt16nested_exception' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /usr/lib/llvm-9/lib/libc++.a(future.cpp.o): relocation R_X86_64_32 against .rodata.str1.1' can not be used when making a shared object; recompile with -fPIC /usr/bin/ld: /usr/lib/llvm-9/lib/libc++.a(hash.cpp.o): relocation R_X86_64_32 against .rodata' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /usr/lib/llvm-9/lib/libc++.a(ios.cpp.o): relocation R_X86_64_32S against symbol _ZTVNSt3__18ios_baseE' can not be used when making a shared object; recompile with -fPIC /usr/bin/ld: /usr/lib/llvm-9/lib/libc++.a(iostream.cpp.o): relocation R_X86_64_32 against .bss' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /usr/lib/llvm-9/lib/libc++.a(locale.cpp.o): relocation R_X86_64_32 against .bss' can not be used when making a shared object; recompile with -fPIC /usr/bin/ld: /usr/lib/llvm-9/lib/libc++.a(memory.cpp.o): relocation R_X86_64_32 against .rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /usr/lib/llvm-9/lib/libc++.a(mutex.cpp.o): relocation R_X86_64_32 against .rodata.str1.1' can not be used when making a shared object; recompile with -fPIC /usr/bin/ld: /usr/lib/llvm-9/lib/libc++.a(new.cpp.o): relocation R_X86_64_32 against symbol _ZTISt9bad_alloc' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /usr/lib/llvm-9/lib/libc++.a(regex.cpp.o): relocation R_X86_64_32 against symbol __gxx_personality_v0' can not be used when making a shared object; recompile with -fPIC /usr/bin/ld: /usr/lib/llvm-9/lib/libc++.a(stdexcept.cpp.o): relocation R_X86_64_32S against symbol _ZTVSt11logic_error' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /usr/lib/llvm-9/lib/libc++.a(string.cpp.o): relocation R_X86_64_32 against .rodata.str1.1' can not be used when making a shared object; recompile with -fPIC /usr/bin/ld: /usr/lib/llvm-9/lib/libc++.a(system_error.cpp.o): relocation R_X86_64_32S against symbol _ZTVNSt3__114error_categoryE' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /usr/lib/llvm-9/lib/libc++.a(thread.cpp.o): relocation R_X86_64_32 against .rodata.str1.1' can not be used when making a shared object; recompile with -fPIC /usr/bin/ld: /usr/lib/llvm-9/lib/libc++.a(vector.cpp.o): relocation R_X86_64_32 against .rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /usr/lib/llvm-9/lib/libc++.a(charconv.cpp.o): relocation R_X86_64_32S against `.rodata' can not be used when making a shared object; recompile with -fPIC

[llvmbot]

assigned to @ldionne

[llvmbot]

libc++ from llvm-8 have no such problem

[chandlerc]

Unsure who all to CC here, but this issue makes the apt.llvm.org Debian packages really broken. The issue persists to this day with LLVM 12.

Basic example of how this works well with libstdc++ but not with libc++:

$ cat plugin_test.cpp
#include <iostream>
__attribute__((visibility("default")))
extern "C" void plugin() {
        std::cout << "Hello World from a plugin!" << std::endl;
}
$ clang++-12 -shared -o plugin.so -fvisibility=hidden plugin_test.cpp -static-libstdc++
$ ldd plugin.so
        linux-vdso.so.1 (0x00007ffcb707c000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f993b614000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f993b5f9000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f993b407000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f993b8c2000)
$ clang++-12 -shared -o plugin.so -fvisibility=hidden plugin_test.cpp -stdlib=libc++ -static-libstdc++
/usr/bin/ld: /tmp/plugin_test-e54e21.o: relocation R_X86_64_32 against undefined symbol `__gxx_personality_v0' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libc++.a(string.cpp.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libc++.a(ios.cpp.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libc++.a(ios.instantiations.cpp.o): relocation R_X86_64_32S against symbol `_ZTVNSt3__115basic_streambufIcNS_11char_traitsIcEEEE' can not be used when making a 
shared object; recompile with -fPIC
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libc++.a(iostream.cpp.o): relocation R_X86_64_32 against `.bss' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libc++.a(locale.cpp.o): relocation R_X86_64_32 against `.bss' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libc++.a(charconv.cpp.o): relocation R_X86_64_32S against `.rodata' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libc++.a(exception.cpp.o): relocation R_X86_64_32S against symbol `_ZTVSt16nested_exception' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libc++.a(memory.cpp.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libc++.a(mutex.cpp.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libc++.a(new.cpp.o): relocation R_X86_64_32 against undefined symbol `_ZTISt9bad_alloc' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libc++.a(stdexcept.cpp.o): relocation R_X86_64_32S against undefined symbol `_ZTVSt11logic_error' can not be used when making a shared object; recompile with -f
PIC
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libc++.a(system_error.cpp.o): relocation R_X86_64_32S against symbol `_ZTVNSt3__114error_categoryE' can not be used when making a shared object; recompile with 
-fPIC
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libc++.a(vector.cpp.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libc++.a(condition_variable.cpp.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libc++.a(thread.cpp.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libc++.a(future.cpp.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Note that libstdc++ isn't one of the DSOs in the first ldd printout. This shows that we are successfully statically linking the archives despite building a shared library.

I've verified that this happens at least with 10, 11, and 12.

Any chance we could fix this? I'm pretty sure this just a CMake setting getting flipped, and so it'd be particularly nice to actually push fresh debian archives with this fix as w/o it the debian packages are completely broken for this use case.

[chandlerc]

I forgot to add that this mostly breaks use of a statically linked libc++ with PIE executables:

$ cat test.cpp
#include <iostream>
int main () {
        std::cout << "Hello World!" << std::endl;
}
$ clang++-12 -o test test.cpp -pie -static-libstdc++
$ ./test
Hello World!
$ ldd test
        linux-vdso.so.1 (0x00007ffe539ec000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f9c05271000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f9c05256000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f9c05064000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f9c054bb000)
$ clang++-12 -o test test.cpp -pie -stdlib=libc++ -static-libstdc++
/usr/bin/ld: /tmp/test-6352e1.o: relocation R_X86_64_32 against undefined symbol `__gxx_personality_v0' can not be used when making a PIE object; recompile with -fPIE
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libc++.a(string.cpp.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a PIE object; recompile with -fPIE
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libc++.a(ios.cpp.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a PIE object; recompile with -fPIE
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libc++.a(ios.instantiations.cpp.o): relocation R_X86_64_32S against symbol `_ZTVNSt3__115basic_streambufIcNS_11char_traitsIcEEEE' can not be used when making a 
PIE object; recompile with -fPIE
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libc++.a(iostream.cpp.o): relocation R_X86_64_32 against `.bss' can not be used when making a PIE object; recompile with -fPIE
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libc++.a(locale.cpp.o): relocation R_X86_64_32 against `.bss' can not be used when making a PIE object; recompile with -fPIE
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libc++.a(charconv.cpp.o): relocation R_X86_64_32S against `.rodata' can not be used when making a PIE object; recompile with -fPIE
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libc++.a(exception.cpp.o): relocation R_X86_64_32S against symbol `_ZTVSt16nested_exception' can not be used when making a PIE object; recompile with -fPIE
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libc++.a(memory.cpp.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a PIE object; recompile with -fPIE
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libc++.a(mutex.cpp.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a PIE object; recompile with -fPIE
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libc++.a(new.cpp.o): relocation R_X86_64_32 against undefined symbol `_ZTISt9bad_alloc' can not be used when making a PIE object; recompile with -fPIE
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libc++.a(stdexcept.cpp.o): relocation R_X86_64_32S against undefined symbol `_ZTVSt11logic_error' can not be used when making a PIE object; recompile with -fPIE
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libc++.a(system_error.cpp.o): relocation R_X86_64_32S against symbol `_ZTVNSt3__114error_categoryE' can not be used when making a PIE object; recompile with -fP
IE
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libc++.a(vector.cpp.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a PIE object; recompile with -fPIE
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libc++.a(condition_variable.cpp.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a PIE object; recompile with -fPIE
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libc++.a(thread.cpp.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a PIE object; recompile with -fPIE
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libc++.a(future.cpp.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a PIE object; recompile with -fPIE
clang: error: linker command failed with exit code 1 (use -v to see invocation)

All of this testing is done on a vanilla Ubuntu 20.04 (LTS) install with the https://apt.llvm.org/llvm.sh script used to install the relevant packages.

[ldionne]

I'm digging through this issue now. I believe the following commit is relevant for context:

commit 30f305efe27928b15d24c7670cce54d26dd2e348
Author: Louis Dionne <ldionne@apple.com>
Date:   Thu Mar 14 14:38:38 2019 +0000

    [libc++] Do not force building with -fPIC (re-applying)

    Summary:
    In r355746, we stopped forcing to build with -fPIC because that should
    be specified by the CMAKE_POSITION_INDEPENDENT_CODE option at CMake
    configure time (and by default -fPIC is used for shared libraries anyways).

    However, r355746 had to be reverted in r355756 because we were not
    actually building the shared library with -fPIC. The reason is that
    we were sharing an object library between the static and the shared
    library, which caused flags for static libraries to be used when
    building object files that were going to be used for a shared library.

    Since this was resolved by r356150, we can stop forcing -fPIC again.

    Differential Revision: https://reviews.llvm.org/D59250

    llvm-svn: 356153

Basically, I removed -fPIC from both the static and the shared libc++ builds so that the CMake default would be used. The CMake defaults are -fPIC for shared libraries, and nothing for static libraries. Hence, that commit changed the static library from being built with -fPIC to not being built with -fPIC (but there was no change for the shared library).

At the time, it seemed excessive to force whoever's building libc++ (e.g. vendors) to use -fPIC if they don't want to. The idea was to use -DCMAKE_POSITION_INDEPENDENT_CODE=ON if one wanted to build with PIC enabled, as intended by CMake. I'm still thinking this is probably the right thing to do, since not everyone wants to build with -fPIC (for example, Apple does not, even for the shared library).

So, I acknowledge 100% that the problem you're seeing exists and needs to be fixed. I'm just not certain whether:

(1) we should start enforcing -fPIC unconditionnally in libc++ again, or
(2) modify the build scripts such that we set -DCMAKE_POSITION_INDEPENDENT_CODE=ON when we build for the LLVM release

I'd be leaning towards (2) since that would bring back the ability to use libc++.a as released by LLVM in shared libraries and PIE executables without forcing all vendors to go down the same route.

On the other hand, I'm not super well versed in the whole PIE/PIC story, so I read this to try and get a better idea of the tradeoffs: https://lists.debian.org/debian-devel/2016/05/msg00309.html.

This seems to suggest that nowadays, avoiding -fPIC might just not be worth it, and so perhaps it would be reasonable to simply force -fPIC down everybody's throat for the sake of consistency. I'm certainly open to that option if someone knowledgeable can argue for it. Do we lose anything at all besides by enabling -fPIC?

[chandlerc]

I'm digging through this issue now. I believe the following commit is
relevant for context:

commit 30f305efe27928b15d24c7670cce54d26dd2e348
Author: Louis Dionne <ldionne@apple.com>
Date:   Thu Mar 14 14:38:38 2019 +0000

    [libc++] Do not force building with -fPIC (re-applying)

    Summary:
    In r355746, we stopped forcing to build with -fPIC because that

should
be specified by the CMAKE_POSITION_INDEPENDENT_CODE option at CMake
configure time (and by default -fPIC is used for shared libraries
anyways).

    However, r355746 had to be reverted in r355756 because we were not
    actually building the shared library with -fPIC. The reason is that
    we were sharing an object library between the static and the shared
    library, which caused flags for static libraries to be used when
    building object files that were going to be used for a shared

library.

    Since this was resolved by r356150, we can stop forcing -fPIC again.

    Differential Revision: https://reviews.llvm.org/D59250

    llvm-svn: 356153

Basically, I removed -fPIC from both the static and the shared libc++
builds so that the CMake default would be used. The CMake defaults are
-fPIC for shared libraries, and nothing for static libraries. Hence, that
commit changed the static library from being built with -fPIC to not being
built with -fPIC (but there was no change for the shared library).

At the time, it seemed excessive to force whoever's building libc++ (e.g.
vendors) to use -fPIC if they don't want to. The idea was to use
-DCMAKE_POSITION_INDEPENDENT_CODE=ON if one wanted to build with PIC
enabled, as intended by CMake. I'm still thinking this is probably the right
thing to do, since not everyone wants to build with -fPIC (for example,
Apple does not, even for the shared library).

So, I acknowledge 100% that the problem you're seeing exists and needs to be
fixed. I'm just not certain whether:

(1) we should start enforcing -fPIC unconditionnally in libc++ again, or
(2) modify the build scripts such that we set
-DCMAKE_POSITION_INDEPENDENT_CODE=ON when we build for the LLVM release

I'd be leaning towards (2) since that would bring back the ability to use
libc++.a as released by LLVM in shared libraries and PIE executables without
forcing all vendors to go down the same route.

I might go with (2) if possible for patching the LLVM-12 release, and I would really love to see a patch release with this fixed. It seems a strict improvement at low cost (only the .a files change).

On the other hand, I'm not super well versed in the whole PIE/PIC story, so
I read this to try and get a better idea of the tradeoffs:
https://lists.debian.org/debian-devel/2016/05/msg00309.html.

This seems to suggest that nowadays, avoiding -fPIC might just not be
worth it, and so perhaps it would be reasonable to simply force -fPIC down
everybody's throat for the sake of consistency. I'm certainly open to that
option if someone knowledgeable can argue for it. Do we lose anything at all
besides by enabling -fPIC?

I largely agree with this assessment. I think that building .o files and .a files with PIC always (on Linux ABIs at least) and then letting the linker do its thing for .so and executables is the best choice.

At the least, it'd be nice of users that build their own libc++ got the right defaults out of the box here, and the right default seems almost certainly for PIC .a files -- otherwise PIE isn't ever really an option.

[MaskRay]

The -fPIE/-fPIC distinction is ELF specific. The only differences are:

• -fPIE can use more optimized thread-local storage models. (Windows/Darwin only support one model and have no such complexity.)
• An external linkage function/variable with STB_GLOBAL binding and STV_DEFAULT visibility is interposable by default.
  (Very unfortunate traditional interpretation) GCC suppresses interprocedural optimizations (including inlining) for non-vague-linkage functions for -fPIC.
  Interprocedural optimizations are enabled for GCC -fPIE.

The main CMake file uses -fno-semantic-interposition by default now (https://reviews.llvm.org/D102453) so the second benefit is moot.
The first actually matters a bit for certain applications but probably small for libc++.

I know very little about CMake and just learned CMAKE_POSITION_INDEPENDENT_CODE.
Since there is a standard CMake variable, perhaps we should move our LLVM_ENABLE_PIC logic to CMAKE_POSITION_INDEPENDENT_CODE instead...
There is a difference about different defaults:
LLVM_ENABLE_PIC is on by default (the default libLLVM*.a/libclang*.a get -fPIC instead of -fPIE) while (I guess) CMAKE_POSITION_INDEPENDENT_CODE is off for certain build configurations?

llvm/CMakeLists.txt
410:option(LLVM_ENABLE_PIC "Build Position-Independent Code" ON)

I think defaulting CMAKE_POSITION_INDEPENDENT_CODE to on makes a lot of sense...

[sylvestre]

Try to rebuild -12 packages (apt.llvm.org) with
-DCMAKE_POSITION_INDEPENDENT_CODE=ON
(stage 2)

but I am getting the same issue:

echo '#include
int main () {
std::cout << "Hello World!" << std::endl;
}'>test.cpp

clang++-12 -o test test.cpp -pie -stdlib=libc++ -static-libstdc++

/usr/bin/ld: /tmp/test-75d1cf.o: relocation R_X86_64_32 against undefined symbol __gxx_personality_v0' can not be used when making a PIE object; recompile with -fPIE /usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu/libc++.a(string.cpp.o): relocation R_X86_64_32 against .rodata.str1.1' can not be used when making a PIE object; recompile with -fPIE
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu/libc++.a(ios.cpp.o): relocation R_X86_64_32 against .rodata.str1.1' can not be used when making a PIE object; recompile with -fPIE /usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu/libc++.a(ios.instantiations.cpp.o): relocation R_X86_64_32S against symbol _ZTVNSt3__115basic_streambufIcNS_11char_traitsIcEEEE' can not be used when making a PIE object; recompile with -fPIE
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu/libc++.a(iostream.cpp.o): relocation R_X86_64_32 against .bss' can not be used when making a PIE object; recompile with -fPIE /usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu/libc++.a(locale.cpp.o): relocation R_X86_64_32 against .bss' can not be used when making a PIE object; recompile with -fPIE
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu/libc++.a(charconv.cpp.o): relocation R_X86_64_32S against .rodata' can not be used when making a PIE object; recompile with -fPIE /usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu/libc++.a(exception.cpp.o): relocation R_X86_64_32S against symbol _ZTVSt16nested_exception' can not be used when making a PIE object; recompile with -fPIE
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu/libc++.a(memory.cpp.o): relocation R_X86_64_32 against .rodata.str1.1' can not be used when making a PIE object; recompile with -fPIE /usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu/libc++.a(mutex.cpp.o): relocation R_X86_64_32 against .rodata.str1.1' can not be used when making a PIE object; recompile with -fPIE
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu/libc++.a(new.cpp.o): relocation R_X86_64_32 against undefined symbol _ZTISt9bad_alloc' can not be used when making a PIE object; recompile with -fPIE /usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu/libc++.a(stdexcept.cpp.o): relocation R_X86_64_32S against undefined symbol _ZTVSt11logic_error' can not be used when making a PIE object; recompile with -fPIE
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu/libc++.a(system_error.cpp.o): relocation R_X86_64_32S against symbol _ZTVNSt3__114error_categoryE' can not be used when making a PIE object; recompile with -fPIE /usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu/libc++.a(vector.cpp.o): relocation R_X86_64_32 against .rodata.str1.1' can not be used when making a PIE object; recompile with -fPIE
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu/libc++.a(condition_variable.cpp.o): relocation R_X86_64_32 against .rodata.str1.1' can not be used when making a PIE object; recompile with -fPIE /usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu/libc++.a(thread.cpp.o): relocation R_X86_64_32 against .rodata.str1.1' can not be used when making a PIE object; recompile with -fPIE
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu/libc++.a(future.cpp.o): relocation R_X86_64_32 against .rodata.str1.1' can not be used when making a PIE object; recompile with -fPIE /usr/bin/ld: /tmp/test-75d1cf.o: warning: relocation in read-only section .text'
clang: error: linker command failed with exit code 1 (use -v to see invocation)

[chandlerc]

Some of those errors are from your test, try:

clang++-12 -o test -fPIC test.cpp -pie -stdlib=libc++ -static-libstdc++

The -fPIC (or -fPIE, not a big diff here) impacts test.cpp` and will fix some of the errors.

The others do look like the same bug here. But the path to libc++.a looks like the system libc++.a... is this after installing the new package?

Is the stage2 build where you're adding the CMake flag the one that is building the runtimes, or just the stage2 compiler? Maybe need to pass the CMake flags specifically to the runtimes part of the build (which can happen even in stage1)

[tstellar]

Can this be fixed by just setting the necessary CMake variable when building the debian packages?

[chandlerc]

Can this be fixed by just setting the necessary CMake variable when building
the debian packages?

Somewhere, yes. Not sure where the CMake variables are set when building the debian packages. If you point me at the script I can try to suggest how to fix things.

[sylvestre]

Here is the patch that I applied:

diff --git a/debian/rules b/debian/rules
index a5d9445..ad76399 100755
--- a/debian/rules
+++ b/debian/rules
@@ -395,6 +395,7 @@ override_dh_auto_configure: preconfigure
-DCMAKE_VERBOSE_MAKEFILE=ON
-DCMAKE_BUILD_TYPE=RelWithDebInfo
-DCMAKE_CXX_FLAGS_RELWITHDEBINFO="$(opt_flags)" \

•   -DCMAKE_POSITION_INDEPENDENT_CODE=ON \
    -DLLVM_LINK_LLVM_DYLIB=ON \
    -DLLVM_INSTALL_UTILS=ON \
    -DLLVM_VERSION_SUFFIX= \
  

sources are:
https://salsa.debian.org/pkg-llvm-team/llvm-toolchain/-/tree/12/debian

you can find the tarballs:
https://apt.llvm.org/unstable/pool/main/l/llvm-toolchain-12/

for example:
dget -x https://apt.llvm.org/unstable/pool/main/l/llvm-toolchain-12/llvm-toolchain-12_12.0.1~%2b%2b20210604062548%2bf2ce10d14b7c-1~exp1~20210604043255.98.dsc
cd llvm-toolchain-12*

dpkg-buildpackage -nc -b

(takes a while)

[chandlerc]

Here is the patch that I applied:

diff --git a/debian/rules b/debian/rules
index a5d9445..ad76399 100755
--- a/debian/rules
+++ b/debian/rules
@@ -395,6 +395,7 @@ override_dh_auto_configure: preconfigure
-DCMAKE_VERBOSE_MAKEFILE=ON
-DCMAKE_BUILD_TYPE=RelWithDebInfo
-DCMAKE_CXX_FLAGS_RELWITHDEBINFO="$(opt_flags)" \

•   -DCMAKE_POSITION_INDEPENDENT_CODE=ON \
    -DLLVM_LINK_LLVM_DYLIB=ON \
    -DLLVM_INSTALL_UTILS=ON \
    -DLLVM_VERSION_SUFFIX= \
  

You'll also want to add it to the bootstrap passthrough flag list I would guess?

I would love to send you patches to this file, but there is no license and so I'm unsure how to do so... While patching this, maybe add the fix suggested here so that the resulting archive also works out of the box? llvm/llvm-bugzilla-archive#46321 #c5

[sylvestre]

You'll also want to add it to the bootstrap passthrough flag list I would
guess?
Arf, yeah, you are probably correct. This is biting me one more time :(

I just started a new build to see if it fixes the issue.

I would love to send you patches to this file, but there is no license and
so I'm unsure how to do so...
The license is the same as llvm
https://salsa.debian.org/pkg-llvm-team/llvm-toolchain/-/blob/12/debian/copyright#L27
(to simplify applying patches upstream)

While patching this, maybe add the fix
suggested here so that the resulting archive also works out of the box?
llvm/llvm-bugzilla-archive#46321 #c5

Sure, I will have a look.

[sylvestre]

Chandler, failed again -even with CLANG_BOOTSTRAP_PASSTHROUGH

diff --git a/debian/rules b/debian/rules
index a5d9445..a1201cb 100755
--- a/debian/rules
+++ b/debian/rules
@@ -395,6 +395,7 @@ override_dh_auto_configure: preconfigure
-DCMAKE_VERBOSE_MAKEFILE=ON
-DCMAKE_BUILD_TYPE=RelWithDebInfo
-DCMAKE_CXX_FLAGS_RELWITHDEBINFO="$(opt_flags)" \

•   -DCMAKE_POSITION_INDEPENDENT_CODE=ON \
    -DLLVM_LINK_LLVM_DYLIB=ON \
    -DLLVM_INSTALL_UTILS=ON \
    -DLLVM_VERSION_SUFFIX= \
  

@@ -417,7 +418,7 @@ override_dh_auto_configure: preconfigure
-DLLVM_POLLY_LINK_INTO_TOOLS=ON
-DBOOTSTRAP_CMAKE_CXX_FLAGS='$(BOOTSTRAP_CXXFLAGS_EXTRA)'
-DBOOTSTRAP_CMAKE_C_FLAGS='$(BOOTSTRAP_CFLAGS_EXTRA)' \

•   -DCLANG_BOOTSTRAP_PASSTHROUGH="CMAKE_INSTALL_PREFIX;CMAKE_VERBOSE_MAKEFILE;CMAKE_BUILD_TYPE;CMAKE_CXX_FLAGS_RELWITHDEBINFO;LLVM_LINK_LLVM_DYLIB;LLVM_INSTALL_UTILS;LLVM_VERSION_SUFFIX;LLVM_ENABLE_SPHINX;SPHINX_WARNINGS_AS_ERRORS;LLVM_BUILD_LLVM_DYLIB;LLVM_ENABLE_RTTI;LLVM_ENABLE_FFI;LIBCLANG_LIBRARY_VERSION;ENABLE_LINKER_BUILD_ID;POLLY_BUNDLED_JSONCPP;LLVM_EXPERIMENTAL_TARGETS_TO_BUILD;LLVM_USE_PERF;LLVM_ENABLE_ASSERTIONS;LLVM_BINUTILS_INCDIR;LLVM_HOST_TRIPLE;LLVM_COMPILER_CHECKED;COMPILER_RT_BUILD_BUILTINS;LIBOMP_LIBFLAGS;CMAKE_SHARED_LINKER_FLAGS;PYTHON_EXECUTABLE;LLVM_ENABLE_Z3_SOLVER;LLVM_POLLY_LINK_INTO_TOOLS;CLANG_VENDOR"
  

•   -DCLANG_BOOTSTRAP_PASSTHROUGH="CMAKE_INSTALL_PREFIX;CMAKE_VERBOSE_MAKEFILE;CMAKE_BUILD_TYPE;CMAKE_CXX_FLAGS_RELWITHDEBINFO;LLVM_LINK_LLVM_DYLIB;LLVM_INSTALL_UTILS;LLVM_VERSION_SUFFIX;LLVM_ENABLE_SPHINX;SPHINX_WARNINGS_AS_ERRORS;LLVM_BUILD_LLVM_DYLIB;LLVM_ENABLE_RTTI;LLVM_ENABLE_FFI;LIBCLANG_LIBRARY_VERSION;ENABLE_LINKER_BUILD_ID;POLLY_BUNDLED_JSONCPP;LLVM_EXPERIMENTAL_TARGETS_TO_BUILD;LLVM_USE_PERF;LLVM_ENABLE_ASSERTIONS;LLVM_BINUTILS_INCDIR;LLVM_HOST_TRIPLE;LLVM_COMPILER_CHECKED;COMPILER_RT_BUILD_BUILTINS;LIBOMP_LIBFLAGS;CMAKE_SHARED_LINKER_FLAGS;PYTHON_EXECUTABLE;LLVM_ENABLE_Z3_SOLVER;LLVM_POLLY_LINK_INTO_TOOLS;CLANG_VENDOR;CMAKE_POSITION_INDEPENDENT_CODE"
    FOUND_VERSION=`grep LLVM_VERSION_STRING build-llvm/include/llvm/Config/llvm-config.h|cut -d\" -f2`; \
    if ! echo "$(LLVM_VERSION_FULL)"|grep "$$FOUND_VERSION"; then \
            echo "mistmatch of version. Found: $$FOUND_VERSION / Expected: $(LLVM_VERSION_FULL)"; \
  

$ clang++-12 -o test -fPIC test.cpp -pie -stdlib=libc++ -static-libstdc++
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu/libc++.a(string.cpp.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a PIE object; recompile with -fPIE