Differences

This shows you the differences between two versions of the page.

--- buildingonnonfreebsd [2020/09/30 12:25] – Update for current status AlexRichardson
+++ buildingonnonfreebsd [2023/10/02 18:13] (current) – redirect to handbook MateuszPiotrowski
@@ Line 1: / Line 1: @@
 ====== Building on non-FreeBSD hosts ======
-Until recently, %%FreeBSD%% could only be built on a %%FreeBSD%% host. However, most CI systems do not include %%FreeBSD%% build slaves by default.
+<callout type="info">
-If we allow %%FreeBSD%% to be compiled on a non-%%FreeBSD%% host (current target is Linux and macOS) it would be much easier to use these CI systems.
+**Moved to handbook**
-Furthermore, it would allow developers who do not have a %%FreeBSD%% system at work to use that to work on %%FreeBSD%%.
-===== Why crossbuilding from non-FreeBSD didn't work =====
+The building process is now documented in the %%FreeBSD%% Handbook. https://docs.freebsd.org/en/books/handbook/cutting-edge/#building-on-non-freebsd-hosts
+</callout>
-  * Some build tools depend on host headers being compatible with target headers (e.g. localedef). This would require some refactoring of the xlocale headers to fix properly.
+Until recently, %%FreeBSD%% could only be built on a %%FreeBSD%% host. However, most CI systems do not include %%FreeBSD%% build slaves by default.
-  * The version of clang contained in the %%FreeBSD%% source tree cannot be bootstrapped with the makefiles from base since the config headers depend on %%FreeBSD%%
+If we allow %%FreeBSD%% to be compiled on a non-%%FreeBSD%% host (current target is Linux and macOS) it is easier to use these CI systems.
-    * Could build the bootstrap clang using CMake+ninja (which should also be much faster than the current Makefile solution), but that will be a lot of effort
+Furthermore, it allows developers who do not have a %%FreeBSD%% system at work to use that to work on %%FreeBSD%%.
-    * Current solution is to use upstream clang
-      * However, some features such as ifunc-noplt are not in upstream clang
-  * The build system often falls back to invoking binaries in $PATH
-    * This can cause strange build errors if that binary was not bootstrapped and an incompatible versions is executed
-    * can also happen when building on %%FreeBSD%% (e.g. running dtc from an old %%FreeBSD%% version)
-    * Building with a clean $PATH [[https://reviews.freebsd.org/D16815|D16815]] fixes this problem since it requires all bootstrap tools to be listed explicitly
-===== Current status =====
+===== Instructions for building on Linux/macOS =====
-As of September 2020 all required patches have been merged to HEAD.
-==== Instructions for building on Linux/macOS ====
 Since Linux and macOS don't include a version of bmake by default, building should be done using the script ''%%tools/build/make.py%%'' which will bootstrap bmake before attempting to build.
-Additionally it will set the required XCC,XLD etc variables that are needed to build with an [[ExternalToolchain]].
+Additionally it will set the required XCC,XLD etc variables that are needed to build with an [[ExternalToolchain]] if not opting to use the in-tree LLVM.
-If you have a recent version of bmake installed using homebrew or another package manager you can also set the required variables manually (the easiest way to obtain them is by running ''%%tools/build/make.py --debug%%'')
+If you have a recent version of bmake installed using homebrew or another package manager you can also set the required variables manually (the easiest way to obtain them is by running ''%%tools/build/make.py --debug%%''). The script also passes any additional command arguments, like -j X or targets, to bmake.
 Usage:
 <code>
-MAKEOBJDIRPREFIX=... ./tools/build/make.py buildworld TARGET=foo TARGET_ARCH=bar ...
+MAKEOBJDIRPREFIX=... tools/build/make.py buildworld TARGET=foo TARGET_ARCH=bar ...
 </code>
-=== Prerequisites on macOS ===
+==== Prerequisites on macOS ====
-''%%tools/build/make.py%%'' should work out-of-the box on macOS with clang and libarchive installed via homebrew:
+''%%tools/build/make.py%%'' should work out-of-the box on macOS with llvm installed via homebrew:
 <code>
-brew install clang libarchive
+brew install llvm
 </code>
-=== Prerequisites on Linux ===
+==== Prerequisites on Linux ====
-You will need a recent version of clang (ideally 10.0) and the libarchive + libbz2 headers.
+You will need a recent version of clang (ideally 10.0) and the libarchive + libbz2 headers (libarchive-dev and libbz2-dev packages)
+==== Using the in-tree toolchain ====
+Instead of installing clang via your system's package manager, you can pass ''%%--bootstrap-toolchain%%'' to ''%%tools/build/make.py%%'' to build the in-tree LLVM and use it for the cross-build. This more closely matches building natively on %%FreeBSD%% (where the in-tree LLVM is used unless it is the same version as the existing system build of the in-tree LLVM), but will be slower than using a pre-built LLVM for your OS.
+===== History =====
+==== Why crossbuilding from non-FreeBSD didn't work ====
+  * Some build tools depended on host headers being compatible with target headers (e.g. localedef). This required some refactoring of the xlocale headers to fix properly.
+  * The version of clang contained in the %%FreeBSD%% source tree could not be bootstrapped with the makefiles from base since the config headers depended on %%FreeBSD%%
+    * Could have built the in-tree clang using CMake+ninja (which should also be much faster than the current Makefile solution), but that would have been a lot of effort
+    * Initial solution was to just use upstream clang as an external toolchain until the config headers and build system were patched to support Linux and macOS
+      * However, some features such as ifunc-noplt are not in upstream clang
+  * The build system often falls back to invoking binaries in $PATH
+    * This can cause strange build errors if that binary was not bootstrapped and an incompatible versions is executed
+    * Can also happen when building on %%FreeBSD%% (e.g. running dtc from an old %%FreeBSD%% version)
+    * Building with a clean $PATH [[https://reviews.freebsd.org/D16815|D16815]] fixes this problem since it requires all bootstrap tools to be listed explicitly
-===== Open issues =====
+==== Issues ====
 | Name | STATUS | Owner | Review | Commits | Comments |
 | MK_PMC broken | Committed | AlexRichardson | [[https://reviews.freebsd.org/D17682|D17682]] | [[https://reviews.freebsd.org/rS339880|r339880]] | jevents doesn't emit output on macOS |
 | MK_BSNMP broken | Committed |  | [[https://reviews.freebsd.org/D21605|D21605]] | [[https://reviews.freebsd.org/rS352220|r352220]] | The makefiles for libsnmp use ls -D which is not supported on macOS+Linux |
-| MK_CDDL broken | Needs review | AlexRichardson | [[https://reviews.freebsd.org/D25979|D25979]] |  | Some of the CDDL bootstrap tools don't compile yet |
+| MK_CDDL broken | Committed | AlexRichardson | [[https://reviews.freebsd.org/D25979|D25979]], [[https://reviews.freebsd.org/D26193|D26193]] and [[https://github.com/openzfs/zfs/pull/10863|openzfs/zfs#10863]] | [[https://reviews.freebsd.org/rS364022|r364022]] and [[https://reviews.freebsd.org/rS365901|r365901]] | Some of the CDDL bootstrap tools don't compile yet |
 | MK_BOOT broken | Committed |  | ? | ? | BOOT contains some assembly files that cannot be built with clang |
 | MK_TESTS broken | Committed | AlexRichardson | [[https://reviews.freebsd.org/D24788|D24788]] | [[https://reviews.freebsd.org/rS360894|rS360894]] | at least one of the test makefiles uses ''%%dd status=none%%'' which is not supported on Linux/macOS |
 | MK_LOCALES broken | Committed | AlexRichardson | [[https://reviews.freebsd.org/D25229|D25229]] | [[https://reviews.freebsd.org/rS363218|rS363218]] | localedef requires the host xlocale headers to be compatible with the target. Would need to namespace those headers to avoid conflicting with the host |
-| Clang+LLD cannot be built | Not started/Bad idea |  |  |  | This requires tablegen to be bootstrapped but won't work with the existing makefiles/config.h. Using an external toolchain is preferable |
+| Clang+LLD cannot be built | Committed |  | [[https://reviews.freebsd.org/D31057|D31057]] | [[https://reviews.freebsd.org/R10:31ba4ce8898f9dfa5e7f054fdbc26e50a599a6e3|31ba4ce8898f]] |  |
 | Building without inheriting $PATH | Committed | AlexRichardson | [[https://reviews.freebsd.org/D16815|D16815]] and [[https://reviews.freebsd.org/D16814|D16814]] | [[https://reviews.freebsd.org/rS340157|r340157]] | This avoids invoking binaries that are incompatible |
-| Add compatibility headers for bootstrap phase | Needs review | AlexRichardson | [[https://reviews.freebsd.org/D16815|D16815]] and [[https://reviews.freebsd.org/D16814|D16814]] |  | Allows bootstrapping tools without making changing the source code to be portable |
+| Add compatibility headers for bootstrap phase | Committed | AlexRichardson | [[https://reviews.freebsd.org/D16815|D16815]] and [[https://reviews.freebsd.org/D16814|D16814]] | [[https://reviews.freebsd.org/rS364757|r364757]] | Allows bootstrapping tools without making changing the source code to be portable |
+| Add wrapper script to bootstrap bmake | Committed | AlexRichardson | [[https://reviews.freebsd.org/D16767|D16767]] | [[https://reviews.freebsd.org/rS365949|r365949]] | Needed since macOS/Linux don't include bmake by default |
 ----
-{{tag>Project Project}}
+{{tag>HowTo}}