Toolchain and Build

basic build requirements
- compiler, etc. to bootstrap a new release
- Guarantee: latest X release can build stable/X+1
- Willing to guarantee: X-1 and X-2 userland will work on X-CURRENT kernel
cross-building from a non-FreeBSD host
- patch to build without depending on tools in $PATH
- Can we enumerate all build tools?
- Once it works, add CI
- Avoid host $PATH
- Wrapper scripts for non-FreeBSD hosted builds
- arichardson patch is ~ +1500 -69
  - https://github.com/arichardson/cheribsd/commits/crossbuild-aug2018
META_MODE
- How do we finally turn it on? – src.conf(5)
- Can you use this without filemon.ko (cross-build)?
  - Would have to check in the dependencies, but wouldn't update them when making changes
- LLVM sanitizers
  - more architectures (aarch64 and mips64 ASAN in review)
  - amd64 ASAN LLVM tests don't all pass
  - Need a matrix sowing what works
    - asan + threads issue?
  - kernel ubsan? (relatively straightforward)
  - Couple test suite with CI with sanitizers
  - which ones do we want?
    - ASAN
    - UBSAN
    - TSAN
    - profile?
    - xray?
      - msan?
  - Build infrastructure (make goo?) e.g. WITH_ASAN
  - Separate ASAN-built libraries as an option?
SSP in GENERIC-DEBUG? WITH_SSP=yes?
LLD
- Current state
  - bootstrap + system linker on amd64, arm64
  - bootstrap linker on i386, armv7
- Near future (12.0 if possible, else 13)
  - system linker on i386, armv7
  - armv7 flip on and watch for ports fallout
  - MIPS64? (requires -z notext)
Userland LTO
- requires llvm-ar, possibly llvm-nm
- requires rtld in 11+
- option for base system? (WITH_LTO)
- integration/deployment questions
  - -flto produces .o that contain llvm IR (not ELF)
  - llvm-ar emits llvm IR archives
  - new extension for LTO lib vs new libpath vs fat .a
Kernel LTO
- barfed on Dtrace (ctfconvert is unhappy)
  - OpenBSD approach might address this
- debugging and observability suffers
  - Need to be careful with optimizing out “unnecessary” memory zeroing etc.
  - Link some parts with -flto and not others?
  - -flto=thin vs -flto
C11 Atomics vs <machine/atomic.h>
- marginally better asm
- ordering with I/O (bus_space_barrier and bus_dmamap_sync)
- issue with using C11 memory_order_consume to implement RCU in the Linux kernel (papers at http://www.rdrop.com/users/paulmck/RCU/)
Enabling compiler support for mitigation features
- per-arch considerations
- CFI – needs someone to actually try and test
- safe stack – ^^^
- PIE binaries (WITH_PIE)
Compressed debug sections / .gdb_index
- Enable on supported toolchains – needs someone to do and test, probably shouldn't have WITH/WITHOUT, just enable by default
Switch llvm back to dwarf4 by default
- emaste should do this tonight during the hacking lounge
clang support for more architectures?
- RISC-V (can build test application with clang 8, not link)
- MIPS? (do we care about mips32)
  - mips32 TLS segfaults
  - TLS model is always dynamic
- powerpc
Using newer features (like ifuncs / optimized ifuncs)
- Ok to start ditching GCC 4.2.x / ld.bfd 2.17.50
- Requiring vs optionally using features in only one modern toolchain
- Should we support both toolchains on platforms that support both? Generally yes.
- Defer question about desupporting modern alternate toolchain.
Starting to use C++11 in userland on architectures?
Using external toolchain more often (e.g. in regular amd64 development)
- Manual override to force existing system toolchain
- Actual testing / documentation of llvm-xtoolchain port
LLVM vs elftoolchain
- ar, nm - for LTO and large .a
  - bug fix needed for elftoolchain, possibly new functionality to be added
- size, objcopy, readelf, strings - leave as elftc
External GCC (base/*)
Killing src/contrib/gcc
Killing src/contrib/binutils
Working with compiler-provided headers (std*.h, etc.)