一如既往，主要在工具链领域耕耘。给这些high-profileOSS贡献的时候，希望透过这个微小的角度改变世界。llvm-projectEnabled--features=layering_checkfor Bazel's llvm andclang projectsImplementedLLVM_ENABLE_REVERSE_ITERATIONforStringMapAddedllvm::xxh3_64bitsand adopted it inllvm/clang/lldReviewed many commits. A lot of people don't add aReviewed By:tag. Anyway, counting commits with the tag cangive an underestimate.12% git shortlog -sn 2679e8bba3e166e3174971d040b9457ec7b7d768...main --grep 'Reviewed .*MaskRay' | awk '{s+=$1}END{print s}'395Many GitHub pull requests are not counted.I made 700+ commits th ...继续阅读 (78)

For approximately 10 years, reviews.llvm.org functioned as the codeview site for the LLVM project, utilizing a Phabricator instance. Thiswebsite hosted numerous invaluable code review discussions. However,following LLVM's transitionto GitHub pull requests, there arises a necessity for a read-onlyarchive of the existing Phabricator instance.The intent is to eliminate a SQL engine. Phabicator operates on a complexdatabase scheme. To minimize time investment, the most feasibleapproach seems to involve downloading the static HTML pages andemploying a lightweight scraping process.Raphaël Gomès deve ...继续阅读 (60)

Since 2012, LLVM has relied on its self-hosted Phabricator instanceon Google Cloud Platform for code review, but now it's making atransition to GitHub pull requests. In this post, I'll share myperspective on this switch, highlighting GitHub offers significantbenefits in some areas while having major drawbacks in the reviewprocess.I may update this article as the process stabilizes further.Transition to GitHub pullrequestsThe move to GitHub pull requests has been a topic of discussion overthe past few years. Several lengthy threads on the subject haveemerged:November 2019 EnableContributions Th ...继续阅读 (55)

Let's begin with a Linux x86-64 example involving global variablesexhibiting various properties such as read-only versus writable,zero-initialized versus non-zero, and more.1234567#includeconstintro =1;intw0, w1 =1;int*constpw0 = &w0;intmain(){printf("%d %d %d %p\n", ro, w0, w1, pw0);}12345678910111213141516171819202122232425% clang -c -fpie a.c% clang -pie -fuse-ld=lld -Wl,-z,separate-loadable-segments a.o -o a% objdump -wt a | grep -P 'main|w[01]|ro$'00000000000010f0 g F .text 000000000000002e main0000000000003044 g O .bss 0000000000000004 ...继续阅读 (64)

UNDER CONSTRUCTIONThis article describes linker notes about Portable Executable (PE)and Common Object File Format (COFF) used on Windows and UEFIenvironments.In ELF, an object file can be a relocatable file, an executable file,or a shared object file. On Windows, the term "object file" usuallyrefers to relocatable files like ELF. Such files use the Common ObjectFile Format (COFF) while image files (e.g. executables and DLLs) use thePortable Executable (PE) format.Input filesThe input files can be object files, archive files, and importlibraries. GNU ld and lld-link allow linking against DLL fi ...继续阅读 (55)

UNDER CONSTRUCTIONDSO undef and non-exporteddefIf aSTB_GLOBALsymbol referenced by a DSO is defined inrelocatable object files but not exported, should the--no-allow-shlib-undefinedfeature report an error? You maywant to check out Dependencyrelated linker optionsfor a discussion of this option and the symbolexporting rule.For quite some time, the--no-allow-shlib-undefinedfeature has been implemented in lld/ELF as follows:123456789101112for(SharedFile *file : ctx.sharedFiles) {boolallNeededIsKnown =llvm::all_of(file->dtNeeded, [&](StringRef needed) {returnsymtab.soNames.count(CachedHas ...继续阅读 (77)

AddressSanitizer (ASan) is a compiler technology that detectsaddressability-related memory errors with some additional checks. Itconsists of two components: compiler instrumentation and a runtimelibrary. To put it simply,The compiler instruments global variables, stack frames, and heapallocations to monitor shadow memory.The compiler also instruments memory access instructions to verifyshadow memory.In case of an error, the inserted code invokes a callback(implemented in the runtime library) to report the error along with astack trace. Typically, the program will terminate after displaying the ...继续阅读 (71)

Clang is a C/C++ compiler that generates LLVM IR and utilitizes LLVMto generate relocatable object files. Using the classic three-stagecompiler structure, the stages can be described as follows:1C/C++ =(front end)=> LLVM IR =(middle end)=> LLVM IR (optimized) =(back end)=> relocatable object fileIf we follow the representation of functions and instructions, a moredetailed diagram looks like this:1C/C++ =(front end)=> LLVM IR =(middle end)=> LLVM IR (optimized) =(instruction selector)=> MachineInstr =(AsmPrinter)=> MCInst =(assembler)=> relocatable object fileLLVM and Clang are designed as a co ...继续阅读 (85)

Since 2012, LLVM has relied on its self-hosted Phabricator instance(on Google Cloud Platform) for code review, but now it's making atransition to GitHub pull requests. In this post, I'll share myperspective on this switch, highlighting GitHub offers significantbenefits in some areas while having major drawbacks in the reviewprocess.I may update this article as the process stabilizes further.Transition to GitHub pullrequestsThe move to GitHub pull requests has been a topic of discussion overthe past few years. Several lengthy threads on the subject haveemerged:November 2019 EnableContributions ...继续阅读 (69)

UNDER CONSTRUCTIONThis article describes some notes about MIPS with a focus on GCC,binutils, and LLVM/Clang.ISAsEarlier ISAs, such as MIPS I, MIPS II, MIPS III, and MIPS IV, can beselected bygcc -march=mips[1-4]. MIPS V has noimplementation and GCC doesn't support-march=mips5.Successive versions are split into MIPS32 and MIPS64, which can beselected by-march=mips{32,64}and-march=mips{32,64}r{2,3,5,6}.-march=mips{32,64}r4is not supported.Release 6 is very different from prior releases.-march=mips{32,64}r2define__mips_isa_revto 2.-march=mips{ ...继续阅读 (78)

clangDriver is the library implementing the compiler driver forClang. It utilitized LLVMOption to process command line options. Asoptions are processed when required, as opposed to use a largeswitch, Clang gets the ability to detect unused options.When an option is checked with APIs such ashasArgandgetLastArg, its "claimed" bit is set.123456789templateArg *getLastArg(OptSpecifiers ...Ids)const{Arg *Res =nullptr;for(Arg *A : filtered(Ids...)) {Res = A;Res->claim();}returnRes;}After all options are processed, Clang reports a-Wunused-command-line-argumentdiagnostic for eachunc ...继续阅读 (98)

LLVM 17 will soon be relased. This post provides a summary of mycontributions in this release cycle to record my learning progress.Ported-fsanitize=functionto non-x86 architectures andthe C language.lld maintenance. See lld 17 ELF changesLLVM binary utility maintenance, e.g.adopted llvm-readobjstyle ObjectFile specific dumpers[llvm-objdump][X86] Add @plt symbols for .plt.got[llvm-readobj] Printfor relocation target with an empty namecompiler-rt maintenance, e.g.changed lsan to work with high-entropy ASLR for x86-64 Linuxremoved crypt and crypt_rinterceptorsto work with glibcFixed ~20 places wh ...继续阅读 (74)

I do not use Apple products myself, but I sometimes delve into Mach-Odue to my interest in object file formats. Additionally, my LLVM/Clangchanges sometimes require some understanding of Mach-O. Occasionally, Ineed to understand the format to some extent to work around its quirks(the old format inherited many problems of "a.out").Recently, there has been interest (from Oleksii Lozovskyi) inenablingXRay, a functioncall tracing system in LLVM, to work on Apple systems. Intrigued bythis, I decided to delve into the details and investigate the necessarychanges. XRay supports many 64-bit architectu ...继续阅读 (81)

When linking an oversized executable, it is possible to encountererrors such asrelocation truncated to fit: R_X86_64_PC32 against `.text'(GNU ld) orrelocation R_X86_64_PC32 out of range(ld.lld).These diagnostics are a result of the relocation overflow check, afeature in the linker.123456% gcc -fuse-ld=bfd @response.txt...a.o: in function `_start':(.text+0x0): relocation truncated to fit: R_X86_64_PC32 against `.text'% gcc -fuse-ld=lld @response.txtld.lld: error: a.o:(.text+0x0): relocation R_X86_64_PC32 out of range: -2147483649 is not in [-2147483648, 2147483647]; references ...继续阅读 (86)

This article describes target-specific details about AArch64 in ELFlinkers. AArch64 is the 64-bit execution state for the Arm architecture.The AArch64 execution state runs the A64 instruction set. The AArch32and AArch64 execution states use very different instruction sets, somany pieces of software use two ports for the two execution states ofthe Arm architecture.There were the "ARM architecture" and the "ARM instruction set",leading to many software projects using "ARM" or "arm" as their portnames. In 2011, ARMv8 introduced two execution states, AArch32 andAArch64. The previous instruction se ...继续阅读 (79)

LLVM 17 will be released. As usual, I maintain lld/ELF and have addedsome notes to https://github.com/llvm/llvm-project/blob/release/17.x/lld/docs/ReleaseNotes.rst.Here I will elaborate on some changes.When--threads=is not specified, the number ofconcurrency is now capped to 16. A large--thread=can harmperformance, especially with some system malloc implementations likeglibc's. (D147493)--remap-inputs=and--remap-inputs-file=are added to remap input files. (D148859)--lto=is now available to supportclang -funified-lto(D123805)--lto-CGO[0-3]is now available to controlCodeGenOpt::Levelindependent ...继续阅读 (94)

LLVM 17 will be released. As usual, I maintain lld/ELF and have addedsome notes to https://github.com/llvm/llvm-project/blob/release/17.x/lld/docs/ReleaseNotes.rst.Here I will elaborate on some changes.When--threads=is not specified, the number ofconcurrency is now capped to 16. A large--thread=can harmperformance, especially with some system malloc implementations likeglibc's. (D147493)--remap-inputs=and--remap-inputs-file=are added to remap input files. (D148859)--lto=is now available to supportclang -funified-lto(D123805)--lto-CGO[0-3]is now available to controlCodeGenOpt::Levelindependent ...继续阅读 (85)

C/C++ projects can benefit from using precompiled headers to improvecompile time. GCC addedsupport for precompiled headersin 2003 (version 3.4), andthe current documentation can be found at https://gcc.gnu.org/onlinedocs/gcc/Precompiled-Headers.html.Even with the emergence of C++ modules, precompiled headers remainrelevant for several reasons:Precompiled headers share implementation aspects with modules (e.g.,AST serialization in Clang).Many C++ projects rely on the traditional compilation model and arenot converted to C++ modules.Modules may possibly use some preamble-like technology to accel ...继续阅读 (98)

C/C++ projects can benefit from using precompiled headers to improvecompile time. GCC addedsupport for precompiled headersin 2003 (version 3.4), andthe current documentation can be found at https://gcc.gnu.org/onlinedocs/gcc/Precompiled-Headers.html.This article focuses on Clang precompiled headers (PCH). Let's beginwith an example.1234567891011121314151617181920cat > a1.cc <<'eof'#include "b.hh"intfa1() {returnfb(); }eofcat > a.cc <<'eof'#include "b.hh"int fa1();intmain() {returnfa1() + fb(); }eofcat > b.hh <<'eof'#ifndef B_HH#define B_HHinlin ...继续阅读 (103)

This article describesSHF_ALLOC|SHF_COMPRESSEDsectionsin ELF and a proposed linker option--compress-sectionstocompress arbitrary sections.Metadata sectionsIn recent years, metadata sections have gained more uses. Some usersmay find compression attractive if they weigh file sizes overdecompression overhead.Some metadata sections are non-SHF_ALLOC, like DWARFdebug sections. They may be unused by a runtime library but provideadditional information for an analysis tool to inspect the content.Other metadata sections have theSHF_ALLOCflag andtherefore are part of aPT_LOADsegment. When the programis ...继续阅读 (77)

UNDER CONSTRUCTIONThis article describesSHF_ALLOC|SHF_COMPRESSEDsectionsin ELF and a proposed linker option--compress-sectionstocompress arbitrary sections.Metadata sectionsIn recent years, metadata sections have gained more uses. Some usersmay find compression attractive if they weigh file sizes overdecompression overhead.Some metadata sections are non-SHF_ALLOC, like DWARFdebug sections. They may be unused by a runtime library but provideadditional information for an analysis tool to inspect the content.Other metadata sections have theSHF_ALLOCflag andtherefore are part of aPT_LOADsegment. W ...继续阅读 (85)

For a user who only uses one C++ standard library, such as libc++,there are typically three compatibility goals, each with increasingcompatibility requirements:Can the program, built with a specific version of libc++, work withan upgraded libc++ shared object (DSO)?Can an executable and its DSOs be compiled with different versionsof libc++ headers?Can two relocatable object files, compiled with different versionsof libc++ headers, be linked into the same executable or DSO?If we replace "different libc++ versions" with a mixture of libc++and libstdc++, we encounter additional goals:Can the prog ...继续阅读 (88)

For a user who only uses one C++ standard library, such as libc++,there are typically three compatibility goals, each with increasingcompatibility requirements:Can the program, built with a specific version of libc++, work withan upgraded libc++ shared object (DSO)?Can an executable and its DSOs be compiled with different versionsof libc++ headers?Can two relocatable object files, compiled with different versionsof libc++ headers, be linked into the same executable or DSO?If we replace "different libc++ versions" with a mixture of libc++and libstdc++, we encounter additional goals:Can the prog ...继续阅读 (91)

I do not use Apple products, but I sometimes like investigatingMach-O as an object file format and my llvm-project changes sometimesneed to work around the quirks.LLVM has a function call tracing system called XRay. It supports manyarchitectures on Linux and some BSDs but does not support Apple systems.If the target triple isx86_64-apple-darwin*, you maynotice that Clang will allow you to perform compilation, but linkingwill fail. For other architectures, Clang will reject it.123456% clang --target=x86_64-apple-darwin -fxray-instrument -fxray-instruction-threshold=1 -c a.c% clang --t ...继续阅读 (85)

UNDER CONSTRUCTIONThis article describes target-specific details about AArch32 in ELFlinkers. I described AArch64 in a previousarticle.AArch32 is the 32-bit execution state for the Arm architecture andruns the A32 and T32 instruction sets. A32 refers to the old ISA with a32-bit fixed width, while T32 refers to the mixed 16-bit and 32-bitThumb2 instructions."AArch32", "A32", and "T32" are new names. Many projects use "ARM","Arm", or "arm" as their port name.ABI documentsELFfor the Arm® ArchitectureProcedureCall Standard for the Arm® ArchitectureC++ABI for the Arm® ArchitectureExceptionHandling ...继续阅读 (72)

For a GCC or Clang command, there is typically one primary outputfile, specified by-oor the default (a.outora.exe). There can also be temporary files and auxiliaryfiles.Primary output fileWe can specify the primary output file with the-ooption. When unspecified, a default output file name is inferred fromthe input files and the final phase. When the final phase is linking,the default output file name isa.outora.exe.1234gcc -S d/a.c# a.sgcc -c d/a.c# a.ogcc d/a.c# a.outgcc d/a.c -o e/a# e/aFor-Sand-c, specifying-oin the presence of more than one input files leads to an error.123% gcc -c d/ ...继续阅读 (75)

When linking an oversized executable, it is possible to encountererrors such asrelocation truncated to fit: R_X86_64_PC32 against `.text'(GNU ld) orrelocation R_X86_64_PC32 out of range(ld.lld).These diagnostics are a result of the relocation overflow check, afeature in the linker.This article aims to explain why such issues can occur and providesinsights on how to mitigate them.Certain groups prefer static linking or mostly static linking for thesake of deployment convenience and performance. In scenarios where thedistributed program contains a significant amount of code (related:software blo ...继续阅读 (77)

UNDER CONSTRUCTIONThis archive provides a description of popular assemblers and theirarchitecture-specific differences.AssemblersGCC generates assembly code and invokes GNU Assembler (also known as"gas"), which is part of GNU Binutils, to convert the assembly code intomachine code. The GCC driver is also capable of accepting assembly inputfiles. Due to GCC's widespread usage, GNU Assembler is arguably the mostpopular assembler.Within the LLVM project, the LLVM integrated assembler is a librarythat is linked by Clang, llvm-mc, and lld (for LTO purposes) to generatemachine code. It supports a wi ...继续阅读 (85)

For a GCC or Clang command, there is typically one primary outputfile, specified by-oor the default (a.outora.exe). There can also be temporary files and auxiliaryfiles.Primary output fileWe can specify the primary output file with the-ooption. When unspecified, a default output file name is inferred fromthe input files and the final phase. When the final phase is linking,the default output file name isa.outora.exe.1234gcc -S d/a.c# a.sgcc -c d/a.c# a.ogcc d/a.c# a.outgcc d/a.c -o e/a# e/aTemporary filesFor compilation and linking in one command, the relocatable objectfiles are temporary files ...继续阅读 (78)

UNDER CONSTRUCTIONThis article describes target-specific details about AArch32 in ELFlinkers. I described AArch64 in a previousarticle.AArch32 is the 32-bit execution state for the Arm architecture andruns the A32 and T32 instruction sets. A32 refers to the old ISA with a32-bit fixed width, while T32 refers to the mixed 16-bit and 32-bitThumb2 instructions."AArch32", "A32", and "T32" are new names. Many projects use "ARM","Arm", or "arm" as their port name.ABI documentsELFfor the Arm® ArchitectureProcedureCall Standard for the Arm® ArchitectureC++ABI for the Arm® ArchitectureExceptionHandling ...继续阅读 (87)