This article describes how to detect C++One Definition Rule(ODR) violations. There are many good resources on the Internet about how ODR violations can introduce subtle bugs, so I will not repeat that here.Debug information basedgold--detect-odr-violationsIn2007, the gold linker implemented an option--detect-odr-violationsto detect ODR violation based on debug information. This option collects symbols starting with_Zand finds twoSTB_WEAKdefinitions with differentst_sizeor differentst_typevalues. These symbols are candidates of ODR violation.gold parses DWARF line tables. For a candidate, if bo ...继续阅读 (65)

This article describes how to detect C++One Definition Rule(ODR) violations. There are many good resources on the Internet about how ODR violations can introduce subtle bugs, so I will not repeat that here.Debug information basedgold--detect-odr-violationsIn2007, the gold linker implemented an option--detect-odr-violationsto detect ODR violation based on debug information. This option collects symbols starting with_Zand finds twoSTB_WEAKdefinitions with differentst_sizeor differentst_typevalues. These symbols are candidates of ODR violation.gold parses DWARF line tables. For a candidate, if bo ...继续阅读 (66)

glibc 2.3.4 introduced_FORTIFY_SOURCEin2004to catch security errors due to misuse of some C library functions. The initially supported functions wasfprintf, gets, memcpy, memmove, mempcpy, memset, printf, snprintf, sprintf, stpcpy, strcat, strcpy, strncat, strncpy, vfprintf, vprintf, vsnprintf, vsprintfand focused on buffer overflow and dangerous printf%nuses. The implementation leverages inline functions and__builtin_object_size(see[PATCH] Object size checking to prevent (some) buffer overflows). More functions were added over time and__builtin_constant_pwas used as well. As of 2022-11 glibc ...继续阅读 (69)

glibc 2.3.4 introduced_FORTIFY_SOURCEin2004to catch security errors due to misuse of some C library functions. The initially supported functions wasfprintf, gets, memcpy, memmove, mempcpy, memset, printf, snprintf, sprintf, stpcpy, strcat, strcpy, strncat, strncpy, vfprintf, vprintf, vsnprintf, vsprintfand focused on buffer overflow and dangerous printf%nuses. The implementation leverages inline functions and__builtin_object_size(see[PATCH] Object size checking to prevent (some) buffer overflows). More functions were added over time and__builtin_constant_pwas used as well. As of 2022-11 glibc ...继续阅读 (75)

I was asked about a segfault related to lld linked musl libc.so on PowerPC64./usr/lib/ld-musl-powerpc64le.so.1 /path/to/thingworked. The kernel ELF loader loads rtld and rtld loads the executable./path/to/thingsegfaulted. The kernel ELF loader loads both rtld and the executable.Therefore the bug is likely due to a difference between the two modes.The section and program header dump from readelf looked like the following. I annotated the interesting lines with!!!.1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636465666768697071 ...继续阅读 (75)

I was asked about a segfault related to lld linked musl libc.so on PowerPC64./usr/lib/ld-musl-powerpc64le.so.1 /path/to/thingworked. The kernel ELF loader loads rtld and rtld loads the executable./path/to/thingsegfaulted. The kernel ELF loader loads both rtld and the executable.Therefore the bug is likely due to a difference between the two modes.The section and program header dump from readelf looked like the following. I annotated the interesting lines with!!!.1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636465666768697071 ...继续阅读 (74)

Updated in 2022-10.In January I wroteCompressed debug sections. The venerable zlib shows its age and there are replacements which are better in every metric except adoption and a larger memory footprint. The obvious choice was Zstandard, but I was not so confident about adoptinig it and solving the ecosystem issue. At any rate, I slowly removed some legacy.zdebugsupport from llvm-project so that a new format could be more easily introduced.In June, Cole Kissane posted[RFC] Zstandard as a second compression method to LLVMon LLVM discourse forums. I learned that other folks were investigating a ...继续阅读 (61)

Updated in 2022-10.In January I wroteCompressed debug sections. The venerable zlib shows its age and there are replacements which are better in every metric except adoption and a larger memory footprint. The obvious choice was Zstandard, but I was not so confident about adoptinig it and solving the ecosystem issue. At any rate, I slowly removed some legacy.zdebugsupport from llvm-project so that a new format could be more easily introduced.In June, Cole Kissane posted[RFC] Zstandard as a second compression method to LLVMon LLVM discourse forums. I learned that other folks were investigating a ...继续阅读 (74)

Note: The article will likely get frequent updates in the next few days.This article describes some approaches to distribute debug information. Commands below will use two simple C files for demonstration.12345678cat > a.c 继续阅读 (68)

Note: The article will likely get frequent updates in the next few days.This article describes some approaches to distribute debug information. Commands below will use two simple C files for demonstration.12345678cat > a.c 继续阅读 (64)

I recently revampedCompetitive programming in Nim. In short, I can create a C amalgamation from a Nim program and submit the C source code to various competitive programming websites.Then I use a Clang based tool to shorten the C source code. It does two things:Shorten function, variables, and type namesUse theclangFormatlibrary to remove some whitespaceFor the first step, the tool uses a derivedASTFrontendActionto traverse the AST twice, one for collecting function/var/type names and the other for renaming. Buildingclang::CompilerInstancefrom command lines needs some boilerplate. An alternati ...继续阅读 (71)

I recently revampedCompetitive programming in Nim. In short, I can create a C amalgamation from a Nim program and submit the C source code to various competitive programming websites.Then I use a Clang based tool to shorten the C source code. It does two things:Shorten function, variables, and type namesUse theclangFormatlibrary to remove some whitespaceFor the first step, the tool uses a derivedASTFrontendActionto traverse the AST twice, one for collecting function/var/type names and the other for renaming. Buildingclang::CompilerInstancefrom command lines needs some boilerplate. An alternati ...继续阅读 (71)

llvm-project 15 was just released. I added some lld/ELF notes tohttps://github.com/llvm/llvm-project/blob/release/15.x/lld/docs/ReleaseNotes.rst. Here I will elaborate on some changes.-z pack-relative-relocsis now available to supportDT_RELRfor glibc 2.36+. (D120701) This exciting size reduction optimization calledDT_RELRfinally landed in glibc. The linker option is all you need to use to enable this optimization. It took me a lot of energy to lobby for the feature in glibc. The benefit will pay off. SeeRelative relocations and RELR.--package-metadata=has been added to create package metadata ...继续阅读 (75)

llvm-project 15 was just released. I added some lld/ELF notes tohttps://github.com/llvm/llvm-project/blob/release/15.x/lld/docs/ReleaseNotes.rst. Here I will elaborate on some changes.-z pack-relative-relocsis now available to supportDT_RELRfor glibc 2.36+. (D120701) This exciting size reduction optimization calledDT_RELRfinally landed in glibc. The linker option is all you need to use to enable this optimization. It took me a lot of energy to lobby for the feature in glibc. The benefit will pay off. SeeRelative relocations and RELR.--package-metadata=has been added to create package metadata ...继续阅读 (69)

In GCC and Clang, there are three major options specifying the architecture and microarchitecture the generated code can run on. The general semantics are described below, but each target machine may assign different semantics.-march=X: (execution domain) Generate code that can use instructions available in the architecture X-mtune=X: (optimization domain) Optimize for the microarchitecture X, but does not change the ABI or make assumptions about available instructions-mcpu=X: Specify both-march=and-mtune=but can be overridden by the two options. The supported values are generally the same as- ...继续阅读 (72)

In GCC and Clang, there are three major options specifying the architecture and microarchitecture the generated code can run on. The general semantics are described below, but each target machine may assign different semantics.-march=X: (execution domain) Generate code that can use instructions available in the architecture X-mtune=X: (optimization domain) Optimize for the microarchitecture X, but does not change the ABI or make assumptions about available instructions-mcpu=X: Specify both-march=and-mtune=but can be overridden by the two options. The supported values are generally the same as- ...继续阅读 (65)

tl;dr "Easy Anti-Cheat"'s incompatibility with glibc 2.36 provides shared objects (libc.so.6,ld-linux-x86_64.so.2) is an instance of Hyrum's law.On 2022-08-02 glibc 2.36 wasreleased.On the same day the x86-64 package wasmoved to[core]on Arch Linux.On 2022-08-03 Jelgnum reported that with the new glibc, "Easy Anti-Cheat" cannot load the anti-cheat module (GLIBC update broke EAC for most games that use it).Multiple Arch Linux game users confirmed the problem.Frogging101 bisected the problem to the glibc commitDo not use --hash-style=both for building glibc shared objects.The problem led to heate ...继续阅读 (60)

tl;dr "Easy Anti-Cheat"'s incompatibility with glibc 2.36 provides shared objects (libc.so.6,ld-linux-x86_64.so.2) is an instance of Hyrum's law.On 2022-08-02 glibc 2.36 wasreleased.On the same day the x86-64 package wasmoved to[core]on Arch Linux.On 2022-08-03 Jelgnum reported that with the new glibc, "Easy Anti-Cheat" cannot load the anti-cheat module (GLIBC update broke EAC for most games that use it).Multiple Arch Linux game users confirmed the problem.Frogging101 bisected the problem to the glibc commitDo not use --hash-style=both for building glibc shared objects.The problem led to heate ...继续阅读 (66)

On 2022-07-07, I added a RISC-V linker relaxation framework in ld.lld and implementedR_RISCV_ALIGN/R_RISCV_CALL/R_RISCV_CALL_PLTrelaxation. The changes will be included in the next llvm-project release 15.0.0. This post describes the implementation.SeeThe dark side of RISC-V linker relaxationfor more information about RISC-V linker relaxation.Problemsld.lld performs these steps (simplified):Parse command line optionsFind and scan input files (.o, .so, .a), interleaved with symbol resolutionCall LLVM LTO to get ELF object filesGlobal transforms (section based garbage collection, identical code ...继续阅读 (76)

On 2022-07-07, I added a RISC-V linker relaxation framework in ld.lld and implementedR_RISCV_ALIGN/R_RISCV_CALL/R_RISCV_CALL_PLTrelaxation. The changes will be included in the next llvm-project release 15.0.0. This post describes the implementation.SeeThe dark side of RISC-V linker relaxationfor more information about RISC-V linker relaxation.Problemsld.lld performs these steps (simplified):Parse command line optionsFind and scan input files (.o, .so, .a), interleaved with symbol resolutionCall LLVM LTO to get ELF object filesGlobal transforms (section based garbage collection, identical code ...继续阅读 (85)

UNDER CONSTRUCTIONDeterministic builds with clang and llddescribes several degrees of build determinism. Here is the description of local determinism:Like incremental basic determinism, but builds are also independent of the name of the build directory. Builds of the same source code on the same machine produce exactly the same output every time, independent of the location of the source checkout directory or the build directory.This is often a stretch goal for a build system. For example, many autotools builds use absolute paths. CMake's Ninja generator uses absolute paths (https://gitlab.kit ...继续阅读 (66)

UNDER CONSTRUCTIONDeterministic builds with clang and llddescribes several degrees of build determinism. Here is the description of local determinism:Like incremental basic determinism, but builds are also independent of the name of the build directory. Builds of the same source code on the same machine produce exactly the same output every time, independent of the location of the source checkout directory or the build directory.This is often a stretch goal for a build system. For example, many autotools builds use absolute paths. CMake's Ninja generator uses absolute paths (https://gitlab.kit ...继续阅读 (81)

In recent ISO C++ standards, [depr.c.headers] describes how a C headername.his transformed to the corresponding C++cnameheader. There is a helpful example:[ Example: The header assuredly provides its declarations and definitions within the namespace std. It may also provide these names within the global namespace. The headerassuredly provides the same declarations and definitions within the global namespace, much as in the C Standard. It may also provide these names within the namespace std. — end example ]"may also" in the wording allows implementations to provide mix-and-match, e.g.std::exi ...继续阅读 (72)

In recent ISO C++ standards, [depr.c.headers] describes how a C headername.his transformed to the corresponding C++cnameheader. There is a helpful example:[ Example: The header assuredly provides its declarations and definitions within the namespace std. It may also provide these names within the global namespace. The headerassuredly provides the same declarations and definitions within the global namespace, much as in the C Standard. It may also provide these names within the namespace std. — end example ]"may also" in the wording allows implementations to provide mix-and-match, e.g.std::exi ...继续阅读 (66)

Recently I have fixed two glibc rtld bugs related to early GOT relocation for retro-computing architectures: m68k and powerpc32. They are related to the obscurePI_STATIC_AND_HIDDENmacro which I am going to demystify.In 2002,PI_STATIC_AND_HIDDENwas introducedinto glibc rtld (runtime loader). This macro indicates whether accesses to the following types of variables need dynamic relocations.static specifier:static int a;(STB_LOCAL)hidden visibility attribute:__attribute__((visibility("hidden"))) int a;(STB_GLOBAL STV_HIDDEN),__attribute__((weak, visibility("hidden"))) int a;(STB_WEAK STV_HIDDEN)P ...继续阅读 (80)

This post has some notes about unwinding through a signal handler. You may want to readStack unwindingfirst.12345678910111213141516171819202122232425262728293031// a.c#define_GNU_SOURCE#include#include#include#include#includestaticvoidhandler(intsigno){unw_context_tcontext;unw_cursor_tcursor;unw_getcontext(&context;);unw_init_local(&cursor;, &context;);unw_word_tpc, sp;do{unw_get_reg(&cursor;, UNW_REG_IP, &pc;);unw_get_reg(&cursor;, UNW_REG_SP, &sp;);printf("pc=0x%016zx sp=0x%016zx", (size_t)pc, (size_t)sp);Dl_info info = {};if(dladdr((void*)pc, &info;))printf(" %s:%s", inf ...继续阅读 (69)

Symbol related--trace-symbol=Alias:-y symPrint files which define or reference the specified non-local symbol. There are three types of definitions: definition in a relocatable object file, definition in a shared object, definition in a lazy object file/archive.12345% ld.lld -y foo a1.a a.o a2.soa1.a(a1.o): lazy definition of fooa.o: reference to fooa1.a(a1.o): definition of fooa2.so: shared definition of fooFile related--traceAlias:-tPrint processed files: relocatable object files, shared objects, and extracted lazy object files/archive members. Note that unextracted lazy object files/archive ...继续阅读 (67)

llvm-project 14 will be released soon. I added some lld/ELF notes tohttps://github.com/llvm/llvm-project/blob/release/14.x/lld/docs/ReleaseNotes.rst. Here I will elaborate on some changes.--export-dynamic-symbol-listhas been added. (D107317) When I added--export-dynamic-symbolto GNU ld, H.J. Lu asked me to add this option. I asked myself whether this was necessary but then realized this may help deprecate--dynamic-listin the long term.--dynamic-listis confusing. It has a different semantics for executables and shared objects. The symbolic intention for shared objects isn't clear.--why-extracth ...继续阅读 (72)

Programming language behaviorFORTRAN 77 COMMON blocks compiled to COMMON symbols. You could declare a COMMON block in more than one file, with each specifying the number, type, and size of the variable. The linker allocated enough space to satisfy the largest size.This feature was somehow ported to C. Unix C compilers traditionally permitted a variable using tentative definition in different compilation units and the linker would allocate enough space without reporting an error.This behavior is constrast to both C and C++ standards, but GCC and Clang traditionally defaulted to-fcommonfor C. GC ...继续阅读 (74)

Recently I have fixed two glibc rtld bugs related to early GOT relocation for retro-computing architectures: m68k and powerpc32. They are related to the obscurePI_STATIC_AND_HIDDENmacro which I am going to demystify.In 2002,PI_STATIC_AND_HIDDENwas introducedinto glibc rtld (runtime loader). This macro indicates whether accesses to the following types of variables need dynamic relocations.static specifier:static int a;(STB_LOCAL)hidden visibility attribute:__attribute__((visibility("hidden"))) int a;(STB_GLOBAL STV_HIDDEN),__attribute__((weak, visibility("hidden"))) int a;(STB_WEAK STV_HIDDEN)P ...继续阅读 (71)