(from https://goo.gl/vhswUP, a.k.a., https://hackmd.io/EwQwRgJgZgnAxgdgLQBYYwAysgUyTANmBlQFYwYIJSEMRScg)

Build systems, compilers, linkers, oh my!

  • decouple thread0 allocation from jemalloc
    • sanitizer integration
    • malloc not coupled to csu (could have libjemalloc)
    • ability to replace with debug malloc, secure malloc, etc.
  • kernel IFUNC support - armv8.1 atomic instructions
  • Investigate autofdo (pgo)
  • Empirical questions out of pgo
  • Collect profile data from (buildworld libraries? package build cluster?)
  • Incrementally commit CHERIbuild changes
  • Cross Building from non-FreeBSD / CHERIBuild
  • Make, meta-mode, etc.
  • Build system replacement
  • LLD integration
  • External toolchain
  • IRification
  • Static analysis and sanitizers
  • Ed gave an update on LLD status in FreeBSD
    • default on arm64 now
    • 20k/27k ports build on amd64
    • building ports infrastructure to set USES_BFD or the like
    • functional on armv7, arm64, amd64, does not support armv4/armv5/armv6 yet
    • i386 has bugs
    • MIPS in progress: possible to link & boot with some hacks/workarounds
      • rtld currently doesn't support .rel relocations
      • hacks to change .rel to .rela could be replaced with proper rtld support
  • Can we ship LLVM tools (opt, llvm-config etc.) by default?
    • Concerns with ABI stability - not provided by LLVM
    • Problem: we already have /usr/lib/clang/4.0.0/ with no explicit ABI disclaimers…
    • Post-summit update: not a concern right now, all static libs except for libclang_rt.asan-x86_64.so but can consider it not part of the supported ABI.
  • Build LLVM + Clang + cheribsd from MacOS or Linux
  • Now also builds postgres etc.
  • Wrote cmake build infrastructure for awk, sed, rmdir, etc. (build tools) then invoke bmake
  • Managed to get a FreeBSD MIPS build w/ cheribsd
  • Problem with generator tools (e.g. csh) that are not done in their own phase - they build in the same directory at the same time as the main binary
  • Separate build tool to own Makefile?
  • Incrementally migrate changes into FreeBSD upstream?
    • hardcoded tool paths in tool-building shell scripts
  • CheriBSD increasingly building with external toolchain
  • Plan to migrate platforms using GCC today to external toolchain
  • For 12.0 - either use Clang or require xtoolchain
  • Release engineering / reproducibility, need to store xtoolchain packages used for release
  • Future: may treat local toolchain as special case of external toolchain?
  • No current, concrete plans
  • If anyone wrote CMake files for FreeBSD they would find some willing testers :)
  • 2/4 major sanitizers mostly work on FreeBSD (dsan, ubsan)
  • Question: should certain sanitizers be a matter of tier support?
  • C source-to-source transformation for dynamic validation
  • CADETS project includes work to build some/all of the base system via LLVM IR
  • use cases:
    • instrumentation (for security, dynamic validation, coverage…)
    • LTO (overlap with existing LTO efforts?)
    • make callgraph
  • pgo clang takes a very long time
    • build llvm
    • run through test inputs
    • build llvm again
  • Google's autofdo can generate profile data from perf data
  • Q: how portable is profile data?
    • A: pretty portable!
    • could we build profile data on e.g., the package builders for distribution in the source tree?
  • Current analysis is based on gcov.
  • Coverage is statement based, would like condition decision based coverage.
    • Lack of opensource tools means no ability to run under travis, etc
    • Hard to know if we're writing correct/sufficient tests without coverage
    • Support for “IR everywhere” may help in writing new/better tools
  • devsummit/201708/toolchain.txt
  • Last modified: 2017/08/09 15:47
  • by EdMaste