UNDER CONSTRUCTION

DSO undef and non-exporteddef

If a STB_GLOBAL symbol referenced by a DSO is defined inrelocatable object files but not exported, should the--no-allow-shlib-undefined feature report an error? You maywant to check out Dependencyrelated linker options for a discussion of this option and the symbolexporting rule.

For quite some time, the --no-allow-shlib-undefinedfeature has been implemented in lld/ELF as follows:

1
2
3
4
5
6
7
8
9
10
11
12

for (SharedFile *file : ctx.sharedFiles) {
  bool allNeededIsKnown =
      llvm::all_of(file->dtNeeded, [&](StringRef needed) {
        return symtab.soNames.count(CachedHashStringRef(needed));
      });
  if (!allNeededIsKnown)
    continue;
  for (Symbol *sym : file->requiredSymbols)
    if (sym->isUndefined() && !sym->isWeak())
      diagnose("undefined reference due to --no-allow-shlib-undefined: " +
               toString(*sym) + "\n>>> referenced by " + toString(file));
}

Recently I noticed that GNU ld implemented a related error in April2003 (discussion).

1
2
3
4

echo '.globl _start; _start: call shared' > main.s && clang -c main.s
echo '.globl shared; shared: call foo' > a.s && clang -shared -fpic a.s -o a.so
echo '.globl foo; foo:' > def.s && clang -c def.s && clang -shared def.o -o def.so
echo '.globl foo; .hidden foo; foo:' > def-hidden.s && clang -c def-hidden.s

1
2
3
4

% ld.bfd main.o a.so def.o
% ld.bfd main.o a.so def-hidden.o
ld.bfd: a.out: hidden symbol `foo' in def-hidden.o is referenced by DSO
ld.bfd: final link failed: bad value

A non-local default or protected visibility symbol can satisfy a DSOreference. The linker will export the symbol to the dynamic symboltable. Therefore, ld.bfd main.o a.so def.o succeeds asintended.

We encounter an error forld.bfd main.o a.so def-hidden.o because a symbol withhidden visibility cannot be exported, and it's unable to satisfy thereference in a.so at run-time.

Here is another interesting case: we use a version script to changethe binding of a defined symbol to STB_LOCAL, causing it tobe unable to satisfy the reference in a.so at run-time. GNUld also reports an error in this case.

1
2
3

% ld.bfd --version-script=local.ver main.o a.so def.o
ld.bfd: a.out: local symbol `foo' in def.o is referenced by DSO
ld.bfd: final link failed: bad value

My recent commit https://github.com/llvm/llvm-project/commit/1981b1b6b92f7579a30c9ed32dbdf3bc749c1b40strengthened LLD's --no-allow-shlib-undefined to detectcases in which the non-exported definitions are garbage-collected. Ihave also proposed enhancements in https://github.com/llvm/llvm-project/pull/70769 to covernon-garbage-collected cases.

DSO undef, non-exporteddef, and DSO def

A variation of the scenario mentioned above occurs when a DSOdefinition is also present. Even if the executable does not exportfoo, another DSO (def.so) may provide it. GNUld's check allows for this case.

1
2

ld.bfd main.o a.so def-hidden.o def.so  # succeeded
ld.lld main.o a.so def-hidden.o def.so  # failed after commit 1981b1b6b92f7579a30c9ed32dbdf3bc749c1b40

It turns out that https://github.com/llvm/llvm-project/commit/1981b1b6b92f7579a30c9ed32dbdf3bc749c1b40unexpectedly strengthened --no-allow-shlib-undefined toalso catch this ODR violation. More precisely, when all three conditionsare met, the new --no-allow-shlib-undefined code reports anerror.

• There is a DSO undef that can be satisfied by a definition fromanother DSO (referred to as SharedSymbol in lld/ELF).
• The SharedSymbol is overridden by a non-exported(usually of hidden visibility) definition in a relocatable object file(Defined).
• The section containing the Defined is garbage-collected(it is not part of .dynsym and is not marked as live).

An exported symbol is a GC root, making its section live. Anon-exported symbol, however, can be discarded when its section isdiscarded.

So, is this error legitimate? At run-time, the undefined symbolfoo in a.so will be bound todef.so, even if the executable does not exportfoo, so we are fine. This suggests that the--no-allow-shlib-undefined code probably should not reportan error.

However, both def-hidden.o and def.sodefine foo, and we know the definitions are different andless likely benign. At the very least, they are not exactly the same dueto different visibilities or one being localized by a versionscript.

A real-world report boils down to

1
2
3
4
5
6
7

% ld.lld @response.txt -y _Znam
...
libfdio.so: reference to _Znam
libclang_rt.asan.so: shared definition of _Znam
libc++.a(stdlib_new_delete.cpp.obj): definition of _Znam
ld.lld: error: undefined reference due to --no-allow-shlib-undefined: _Znam
>>> referenced by libfdio.so

How does libfdio.so obtain a reference to_Znam? Well, libfdio.so is linked against bothlibclang_rt.asan.so and libc++.a. Due tosymbol processing rules, the definition fromlibclang_rt.asan.so takes precedence. (See Symbol processing#Sharedobject overriding archive.)

An appropriate solution is to replace libc++a with anAddressSanitizer-instrumented version that does not define_Znam.

I have also encountered issues stemming from the combination ofmultiple definitions from libgcc.a (with hidden visibility)and libclang_rt.builtins.a (with default visibility),relying on archive member extraction rules.

1
2
3
4
5
6
7
8

% ld.lld @response.txt -y __divti3
...
a.so: reference to __divti3
libgcc.a(_divdi3.o): definition of __divti3
libc++.so: shared definition of __divti3
# A lazy symbol in libclang_rt.builtins.a is not reported by -y
ld.lld: error: undefined reference due to --no-allow-shlib-undefined: __divti3
>>> referenced by a.so

a.so is linked against libc++.so andlibclang_rt.builtins.a and obtains a reference to__divti3 due to libc++.so. For the executablelink, the undesired situation arises as the definition inlibgcc.a takes precedence. What we actually want is forlibgcc.a to provide the missing components fromlibclang_rt.builtins.a.

Some users compile relocatable object files with-fvisibility=hidden to disallow dynamic linking. However,when their system includes specific shared objects, it increases therisk of conflicting multiple definition symbols.

While this additional check introduced in https://github.com/llvm/llvm-project/commit/1981b1b6b92f7579a30c9ed32dbdf3bc749c1b40may not perfectly fit into --no-allow-shlib-undefined, Ibelieve it has value. As a result, I have proposed --[no-]allow-non-exported-symbols-shared-with-dso.

Technically, the check can be extended to default visibility to catchall link-time symbol interposition. However, I suspect that there are alot of benign violations and in the absence of an ignore list mechanism,this extension will not be useful.