This post explores how GNU Assembler and LLVM integrated assemblergenerate relocations, an important step to generate a relocatable file.Relocations identify parts of instructions or data that cannot be fullydetermined during assembly because they depend on the final memorylayout, which is only established at link time or load time. These areessentially placeholders that will be filled in (typically with absoluteaddresses or PC-relative offsets) during the linking process.Relocation generation: thebasicsSymbol references are the primary candidates for relocations. Forinstance, in the x86-64 in ...继续阅读

In my previous post,RelocationGeneration in Assemblers, I explored some key concepts behindLLVM’s integrated assemblers. This post dives into recent improvementsI’ve made to refine that system.The LLVM integrated assembler handles fixups and relocatableexpressions as distinct entities. Relocatable expressions, inparticular, are encoded using theMCValueclass, whichoriginally looked like this:12345classMCValue{constMCSymbolRefExpr *SymA =nullptr, *SymB =nullptr;int64_tCst =0;uint32_tRefKind =0;};In this structure:RefKindacts as an optional relocation specifier,though only a handful of ...继续阅读

This post describes how to compile a single C++ source file to anobject file with the Clang API. Here is the code. It behaves like asimplifiedclangexecutable that handles-cand-S.1cat> main.cc // EmitObjAction#includeDriverCompilation.h>#includeDriverDriver.h>#includeFrontendCompilerInstance.h>#includeFrontendFrontendOptions.h>#includeConfigllvm-config.h>// LLVM_VERSION_MAJOR#includeSupportTargetSelect.h>// LLVMInitialize*#includeSupportVirtualFileSystem.h>usingnamespaceclang;constexprllvm::StringRef kTargetTriple ="x86_64-unknown-linux-gnu";namespace{structDiagsSav ...继续阅读

LLVM 20 will be released. As usual, I maintain lld/ELF and have addedsome notes to https://github.com/llvm/llvm-project/blob/release/20.x/lld/docs/ReleaseNotes.rst.I've meticulously reviewed nearly all the patches that are not authoredby me. I'll delve into some of the key changes.-z nosectionheaderhas been implemented to omit thesection header table. The operation is similar tollvm-objcopy --strip-sections. (#101286)--randomize-section-padding=is introducedto insert random padding between input sections and at the start of eachsegment. This can be used to control measurement bias in A/Bexperi ...继续阅读

Followed this guide: https://www.patrickthurmond.com/blog/2023/12/11/commenting-is-available-now-thanks-to-giscusAdd the following tolayout/_partial/article.ejs12345678910111213141516171819<% if (!index && post.comments) { %><% } %>Unfortunately comments from Disqus have not been migrated yet. Ifyou've left comments in the past, thank you. Apologies they are nowgone.While you can create Github Discussions via GraphQL API, I haven'tfound a solution that works out of the box. https://www.davidangulo.xyz/posts/dirty-ruby-script-to-migrate-comments-from-disqus-to-giscus/provides a Ruby s ...继续阅读

...继续阅读

Clang provides a few options to generate timing report. Among them,-ftime-reportand-ftime-tracecan be used toanalyze the performance of Clang's internal passes.-fproc-stat-reportrecords time and memory on spawnedprocesses (ld, and gas if-fno-integrated-as).-ftime-trace, introduced in 2019, generates Clangtiming information in the Chrome Trace Event format (JSON). The formatsupports nested events, providing a rich view of the front end.-ftime-report: The option name is borrowed fromGCC.This post focuses on the traditional-ftime-report,which uses a line-based textual format.Understanding-ftime-r ...继续阅读 (4)

一如既往，主要在工具链领域耕耘。BlogWrote 31 blog posts (including this one, mainly about toolchains) andrevised many posts initially written between 2020 and 2024.6 posts managed to get more than 50 points on Hacker News. (https://news.ycombinator.com/from?site=maskray.me).GCCMade 5 commits to GCC and earned committer access. Added the inlineasm constraint "Ws" to the x86 port for raw symbolnames.llvm-projectReviewed many many patchesOfficial maintainer status on the MC layer and binary utilitiesAdded a script update_test_body.pyto generate elaborated IR and assembly tests (#89026)MCMade some MCand assembler ...继续阅读 (7)

In debuggers, stepping into a function with arguments that involvefunction calls may step into the nested function calls, even if they aresimple and uninteresting, such as those found in the C++ STL.GDBConsider the following example:1234567891011121314#include#include#includeusingnamespacestd;voidfoo(inti,intj){printf("%d %d\n", i, j);}intmain(){autoi =make_unique(3);vector v{1,2};foo(*i, v.back());// step into}When GDB stops at thefoocall, thestep(s) command will step intostd::vector::backandstd::unique_ptr::operator*. While you can executefinish(fin) and execute ...继续阅读 (9)

Onhttps://x.com/settings/, clickMore -> Settings and privacy -> Download an archive of your data.Wait for a message from x.com: "@XXXyour X data is ready" Download thearchive.1cpdata/tweets.js tweets.tsChange the first line fromwindow.YTD.tweets.part0 = [tolet part0 = [, and append12345678910111213141516171819202122232425262728import{unescape}from"@std/html/entities";letout = part0.map(tw=>[newDate(tw.tweet.created_at), tw.tweet.full_text])out.sort((a,b) =>a[0] - b[0])letyy0 =0, mm0 =0, str =''for(leti=0, j=0; i<=out.length; i++) {letd = i<$1>")}\n`}Then rund ...继续阅读 (8)

Both compiler developers and security researchers have builtdisassemblers. They often prioritize different aspects. Compilertoolchains, benefiting from direct contributions from CPU vendors, tendto offer more accurate and robust decoding. Security-focused tools, onthe other hand, often excel in user interface design.For quick disassembly tasks,rizinprovides a convenient command-line interface.123% rz-asm -a x86 -b 64 -d 4829c390sub rbx, raxnop-a x86can be omitted.llvm-mcWithin the LLVM ecosystem, llvm-objdump serves as a drop-inreplacement for the traditional GNU objdump, leveraging instructio ...继续阅读 (9)

Both compiler developers and security researchers have builtdisassemblers. They often prioritize different aspects. Compilertoolchains, benefiting from direct contributions from CPU vendors, tendto offer more accurate and robust decoding. Security-focused tools, onthe other hand, often excel in user interface design.For quick disassembly tasks,rizinprovides a convenient command-line interface.123% rz-asm -a x86 -b 64 -d 4829c390sub rbx, raxnop-a x86can be omitted.Within the LLVM ecosystem, llvm-objdump serves as a drop-inreplacement for the traditional GNU objdump, leveraging instructioninform ...继续阅读 (11)

The Google C++ Style is widely adopted by projects. It contains abrace omission guideline in Loopingand branching statements:For historical reasons, we allow one exception to the above rules:the curly braces for the controlled statement or the line breaks insidethe curly braces may be omitted if as a result the entire statementappears on either a single line (in which case there is a space betweenthe closing parenthesis and the controlled statement) or on two lines(in which case there is a line break after the closing parenthesis andthere are no braces).123456789// OK - fits on one line.if(x = ...继续阅读 (14)

LLD, the LLVM linker, is a mature and fast linker supporting multiplebinary formats (ELF, Mach-O, PE/COFF, WebAssembly). Designed as astandalone program, the code base relies heavily on global state, makingit less than ideal for library integration. As outlined in RFC:Revisiting LLD-as-a-library design, two main hurdles exist:Fatal errors: they exit the process without returning control to thecaller.Global variable conflicts: shared global variables do not allow twoconcurrent invocation.I understand that calling a linker API could be convenient,especially when you want to avoid shipping anothe ...继续阅读 (16)

LLVM's C++ API doesn't offer a stability guarantee. This meansfunction signatures can change or be removed between versions, forcingprojects to adapt.On the other hand, LLVM has an extensive API surface. When a librarylikellvm/lib/Yrelies functionality from another library,the API is often exported in header files underllvm/include/llvm/X/, even if it is not intended to beuser-facing.To be compatible with multiple LLVM versions, many projects rely on#ifdirectives based on theLLVM_VERSION_MAJORmacro. This post explores the specific techniques used by ccls to ensurecompatibility with LLVM versio ...继续阅读 (13)

Aftermigrating fromVim to Emacsas my primary C++ editor in 2015, I switched from Vimto Neovim for miscellaneous non-C++ tasks as it is more convenient in aterminal. Customizing the editor with a language you are comfortablewith is important. I found myself increasingly drawn to Neovim'sterminal-based simplicity for various non-C++ tasks. Recently, I'verefined my Neovim setup to the point where I can confidently migrate myentire C++ workflow away from Emacs. This post explores the keyimprovements I've made to achieve this seamless transition.Key mappingI've implemented custom functions that sim ...继续阅读 (17)

My C++ language server ccls implements a semantic highlightingfeatures through the non-standard messages$ccls/publishSemanticHighlightand$ccls/publishSkippedRanges, supported by both emacs-cclsand vscode-ccls.For years, my primary editor was Emacs. In 2018, I created emacs-ccls(a fork of emacs-cquery) and enjoyed its rainbow semantic highlighting.My setup also relied heavily on two features:Bolding and underlining variables of static duration storageItalicizing member functions and variables12(setqccls-sem-highlight-method 'font-lock)(ccls-use-default-rainbow-sem-highlight)My elisp skills ...继续阅读 (19)

LLVM 19 will soon be released. This post provides a summary of mycontributions in this release cycle to record my learning progress.LLVM MC: Integratedassembler improvements in LLVM 19lld/ELF:lld 19 ELFchangesAArch64[AArch64]Implement -fno-plt for SelectionDAG/GlobalISel[AArch64]Support optional constant offset for constraint "S"[AArch64AsmParser]Allow branch target symbol to have a shift/extend modifier name[MC,AArch64]Create mapping symbols with non-unique namesARM[ARM,MC]Support FDPIC relocations. See MMU-less systems andFDPICRISC-V[RISCV]Support constraint "s"[MC,RISCV]Create mapping symbo ...继续阅读 (41)

LLVM 19 will be released. As usual, I maintain lld/ELF and have addedsome notes to https://github.com/llvm/llvm-project/blob/release/19.x/lld/docs/ReleaseNotes.rst.I've meticulously reviewed nearly all the patches that are not authoredby me. I'll delve into some of the key changes.EI_OSABIin the output is now inferred from inputobject files. (#97144)--compress-sections={none,zlib,zstd}[:level]is added to compress matched output sections without theSHF_ALLOCflag. (#84855) (#90567)The default compression level for zlib is now independent of linkeroptimization level (Z_BEST_SPEED).zstd ...继续阅读 (33)

In object files, certain code patterns incorporate data directlywithin the code or transit between instruction sets. This practice canpose challenges for disassemblers as data might be mistakenlyinterpreted as code, leading to nonsensical output. In addition, codefrom instruction set A might be disassembled as instruction set B. Toaddress the issues, some architectures define mapping symbols todescribe state transition. Let's explore this concept using an AArch32code example:1234567891011121314151617181920212223.textadr r0, .LJTI0_0vldr d0, .LCPI0_0bl thumb_callee.LBB0_1:nop.LBB0_2:nop. ...继续阅读 (41)

The output ofld.lld -vincludes a message "compatiblewith GNU linkers" to address detectionmechanism used by GNU Libtool. This problem is described by Softwarecompatibility and our own "User-Agent" problem.The latestm4/libtool.m4continues to rely on aGNUcheck.12345678910[AC_CACHE_CHECK([if the linker ($LD) is GNU ld], lt_cv_prog_gnu_ld,[# I'd rather use --version here, but apparently some GNU lds only accept -v.case `$LD -v 2>&1 ...继续阅读 (53)

Within the LLVM project, MC is a library responsible for handlingassembly, disassembly, and object file formats. [Intro to the LLVM MCProject], which was written back in 2010, remains a good source tounderstand the high-level structures.In the latest release cycle, substantial effort has been dedicated torefining MC's internal representation for improved performance andreadability. These changes havedecreased compiletime significantly. This blog post will delve into the details,providing insights into the specific changes.MCAssemblerandMCAsmLayoutMCAssemblermanages assembler states (includings ...继续阅读 (45)

GNU ld's output section layout is determined by a linker script,which can be either internal (default) or external (specified with-Tor-dT). Within the linker script,SECTIONScommands define how input sections are mapped intooutput sections.Input sections not explicitly placed bySECTIONScommands are termed "orphansections".Orphan sections are sections present in the input files which are notexplicitly placed into the output file by the linker script. The linkerwill still copy these sections into the output file by either finding,or creating a suitable output section in which to place the orphane ...继续阅读 (43)

The ELF object file format is adopted by many UNIX-like operatingsystems. Understanding its specifications can be an interestingjourney.The format consists of the generic specification, processor-specificspecifications, and OS-specific specifications. Three key documentsoften surface when searching for the generic specification:Tool Interface Standard (TIS) Portable Formats Specification,version 1.2onhttps://refspecs.linuxfoundation.org/SystemV Application Binary Interface - DRAFT - 10 June 2013onwww.sco.comOracle Solaris Linkers and Libraries GuideThe TIS specification breaks ELF into the gen ...继续阅读 (56)

The Linux kernel is written in C, but it also leverages extensionsprovided by GCC. In 2022, it moved from GCC/Clang-std=gnu89to-std=gnu11. This articleexplores my notes on how these GNU extensions are utilized within thekernel.Statement expressionsStatementexpressionsare commonly used in macros.LocallabelsSome macros use this extension to restart a for loop in a macro'sreplacement list.123456789101112131415161718192021222324// include/linux/wait.h#define___wait_event(wq_head, condition, state, exclusive, ret, cmd) \({ ...继续阅读 (53)

tl;dr Clang 19 will remove the-mrelax-alldefault at-O0, significantly decreasing the text section size forx86.Span-dependent instructionsIn assembly languages, some instructions with an immediate operandcan be encoded in two (or more) forms with different sizes. On x86-64, aJMP/JCC (jumps) can be encoded either in 2 bytes with a 8-bit relativeoffset or 6 bytes with a 32-bit relative offset. The short form ispreferred because it takes less space. However, when the target of thejump is too far away, the long form must be used.1234ja foo # jump near if above, 77ja foo # jump short if above, ...继续阅读 (60)

QOI, the Quite OK Image format,has been gaining in popularity. Chris Wellons offers a great analysis.QOI's key advantages is its simplicity. Being a byte-oriented formatwithout entropy encoding, it can be further compressed with generic datacompression programs like lz4, xz, and zstd. PNG, on the other hand,uses DEFLATE compression internally and is typically resistant tofurther compression. By applying a stronger compression algorithm on QOIoutput, you can often achieve a smaller file size compared to PNG.Lasse Collin has shared some effective options for compressinguncompressed BMP/TIFF file ...继续阅读 (52)

QOI, the Quite OK Image format,has been gaining in popularity. Chris Wellons offers a great analysis.QOI's key advantages is its simplicity. Being a byte-oriented formatwithout entropy encoding, it can be further compressed with genericcodecs like lz4, xz, and zstd. PNG, on the other hand, uses DEFLATEcompression internally and is typically resistant to furthercompression. By applying a stronger compression algorithm on QOI output,you can often achieve a smaller file size compared to PNG.Lasse Collin has shared some effective options for compressinguncompressed BMP/TIFF files. I tested them on ...继续阅读 (54)

ELF's design emphasizes naturalsize and alignment guidelines for its control structures. Whileensured efficient processing in the old days, this can lead to largerfile sizes. I propose "Light ELF" (EV_LIGHT, version 2) – awhimsical exploration inspired by Light Elves of Tolkien's legendarium(who had seen the light of the Two Trees in Valinor).In a light ELF file, thee_versionmember of the ELFheader is set to 2.EV_CURRENTremains 1 for backwardcompatibility.123#defineEV_NONE 0#defineEV_CURRENT 1#defineEV_LIGHT 2When linking a program, traditional ELF (version 1) and light ELF(version 2) files ca ...继续阅读 (48)

ELF's design emphasizes natural sizeand alignment guidelines for its control structures. However, thisapproach has substantial size drawbacks.In a release build of llvm-project(-O3 -ffunction-sections -fdata-sections, the sectionheader tables occupy 13.4% of the.ofile size.I propose an alternative section header table format that is signaledbye_shentsize == 0in the ELF header.e_shentsize == sizeof(Elf64_Shdr)(or the 32-bitcounterpart) selects the traditional section header table format.nshdrdenotes the number of sections (includingSHN_UNDEF). The compact section header table (located ate_shoff ...继续阅读 (54)