Building on non-FreeBSD hosts

Until recently, FreeBSD could only be built on a FreeBSD host. However, most CI systems do not include FreeBSD build slaves by default. If we allow FreeBSD to be compiled on a non-FreeBSD host (current target is Linux and macOS) it is easier to use these CI systems. Furthermore, it allows developers who do not have a FreeBSD system at work to use that to work on FreeBSD.

Since Linux and macOS don't include a version of bmake by default, building should be done using the script tools/build/make.py which will bootstrap bmake before attempting to build. Additionally it will set the required XCC,XLD etc variables that are needed to build with an ExternalToolchain if not opting to use the in-tree LLVM. If you have a recent version of bmake installed using homebrew or another package manager you can also set the required variables manually (the easiest way to obtain them is by running tools/build/make.py --debug). The script also passes any additional command arguments, like -j X or targets, to bmake.

Usage:

MAKEOBJDIRPREFIX=... tools/build/make.py buildworld TARGET=foo TARGET_ARCH=bar ...

tools/build/make.py should work out-of-the box on macOS with llvm installed via homebrew:

brew install llvm

You will need a recent version of clang (ideally 10.0) and the libarchive + libbz2 headers (libarchive-dev and libbz2-dev packages)

Instead of installing clang via your system's package manager, you can pass --bootstrap-toolchain to tools/build/make.py to build the in-tree LLVM and use it for the cross-build. This more closely matches building natively on FreeBSD (where the in-tree LLVM is used unless it is the same version as the existing system build of the in-tree LLVM), but will be slower than using a pre-built LLVM for your OS.

• Some build tools depended on host headers being compatible with target headers (e.g. localedef). This required some refactoring of the xlocale headers to fix properly.
• The version of clang contained in the FreeBSD source tree could not be bootstrapped with the makefiles from base since the config headers depended on FreeBSD
  • Could have built the in-tree clang using CMake+ninja (which should also be much faster than the current Makefile solution), but that would have been a lot of effort
  • Initial solution was to just use upstream clang as an external toolchain until the config headers and build system were patched to support Linux and macOS
    • However, some features such as ifunc-noplt are not in upstream clang
• The build system often falls back to invoking binaries in $PATH
  • This can cause strange build errors if that binary was not bootstrapped and an incompatible versions is executed
  • Can also happen when building on FreeBSD (e.g. running dtc from an old FreeBSD version)
  • Building with a clean $PATH D16815 fixes this problem since it requires all bootstrap tools to be listed explicitly