This is an old revision of the document!

Session Leader: jhb

Notes: https://hackmd.io/BjHsH7MzTLuAcUF_P7JJxA#

# Toolchain and Build

* basic build requirements

• compiler, etc. to bootstrap a new release
• Guarantee: latest X release can build stable/X+1
• Willing to guarantee: X-1 and X-2 userland will work on X-CURRENT kernel

* cross-building from a non-FreeBSD host

• patch to build without depending on tools in $PATH
• Can we enumerate all build tools?
• Once it works, add CI
• Avoid host $PATH
• Wrapper scripts for non-FreeBSD hosted builds
• arichardson patch is ~ +1500 -69
  • https://github.com/arichardson/cheribsd/commits/crossbuild-aug2018

* META_MODE

• How do we finally turn it on? – src.conf(5)
• Can you use this without filemon.ko (cross-build)?
  • Would have to check in the dependencies, but wouldn't update them when making changes

* LLVM sanitizers

• more architectures (aarch64 and mips64 ASAN in review)
• amd64 ASAN LLVM tests don't all pass
• Need a matrix sowing what works
  • asan + threads issue?
• kernel ubsan? (relatively straightforward)
• Couple test suite with CI with sanitizers
• which ones do we want?
  • ASAN
  • UBSAN
  • TSAN
  • profile?
  • xray?
  • msan?
• Build infrastructure (make goo?) e.g. WITH_ASAN
• Separate ASAN-built libraries as an option?

* SSP in GENERIC-DEBUG? WITH_SSP=yes? * LLD

• Current state
  • bootstrap + system linker on amd64, arm64
  • bootstrap linker on i386, armv7
• Near future (12.0 if possible, else 13)
  • system linker on i386, armv7
  • armv7 flip on and watch for ports fallout
  • MIPS64? (requires -z notext)

* Userland LTO

• requires llvm-ar, possibly llvm-nm
• requires rtld in 11+
• option for base system? (WITH_LTO)
• integration/deployment questions
  • -flto produces .o that contain llvm IR (not ELF)
  • llvm-ar emits llvm IR archives
  • new extension for LTO lib vs new libpath vs fat .a

* Kernel LTO

• barfed on Dtrace (ctfconvert is unhappy)
  • OpenBSD approach might address this
• debugging and observability suffers
• Need to be careful with optimizing out “unnecessary” memory zeroing etc.
• Link some parts with -flto and not others?
• -flto=thin vs -flto

* C11 Atomics vs <machine/atomic.h>

• marginally better asm
• ordering with I/O (bus_space_barrier and bus_dmamap_sync)
• issue with using C11 memory_order_consume to implement RCU in the Linux kernel (papers at http://www.rdrop.com/users/paulmck/RCU/)

* Enabling compiler support for mitigation features

• per-arch considerations
• CFI – needs someone to actually try and test
• safe stack – ^^^
• PIE binaries (WITH_PIE)

* Compressed debug sections / .gdb_index

• Enable on supported toolchains – needs someone to do and test, probably shouldn't have WITH/WITHOUT, just enable by default

* Switch llvm back to dwarf4 by default

• emaste should do this tonight during the hacking lounge

* clang support for more architectures?

• RISC-V (can build test application with clang 8, not link)
• MIPS? (do we care about mips32)
  • mips32 TLS segfaults
  • TLS model is always dynamic
• powerpc

* Using newer features (like ifuncs / optimized ifuncs)

• Ok to start ditching GCC 4.2.x / ld.bfd 2.17.50
• Requiring vs optionally using features in only one modern toolchain
• Should we support both toolchains on platforms that support both? Generally yes.
• Defer question about desupporting modern alternate toolchain.

* Starting to use C++11 in userland on architectures? * Using external toolchain more often (e.g. in regular amd64 development)

• Manual override to force existing system toolchain
• Actual testing / documentation of llvm-xtoolchain port

* LLVM vs elftoolchain

• ar, nm - for LTO and large .a
  • bug fix needed for elftoolchain, possibly new functionality to be added
• size, objcopy, readelf, strings - leave as elftc

* External GCC (base/*) * Killing src/contrib/gcc * Killing src/contrib/binutils * Working with compiler-provided headers (std*.h, etc.)