Here is a list of issues that will lead at a moment or another to a failure about cross compilation with the current ports tree:

• .if exists(/path/something): lots of things are tested against the host system, some may be tested against the destination chroot and some may be looking at the host (because needed for building)

Clang is a cross-compiler out of the box. The desired target triple must be specified, for example -triple arm-unknown-freebsd. You may also want to explicitly add some other flags, such as -march and so on. If you are using clang's internal assembler, then you can generate .o files natively, however you will still currently need a linker specially compiled for your target platform. Ideally, the whole of binutils will need to be built for the target - clang should automatically use {triple}-ld and {triple}-as if it exists, in preference to the normal ld and as.

Ports' {BUILD,EXTRACT,FETCH,PATCH,PKG}_DEPENDS by convention use ${progname}:categ/portname to identify dependency on a particular tool that must be executable within the corresponding build stage. These may be useful in determining which dependencies must be built for the host.

http://people.freebsd.org/~trhodes/ports/bsd.crossbuild.mk

.if defined(PORT_TARGET_ARCH)

.if defined(GNU_CONFIGURE)
CONFIGURE_TARGET=${PORT_TARGET_ARCH}-freebsd
CONFIGURE_ARGS+=--target ${PORT_TARGET_ARCH}-freebsd --host ${PORT_TARGET_ARCH}-freebsd
.else
__Y=${PORT_TARGET_ARCH}-freebsd-
AR      = ${__Y}ar
RANLIB  = ${__Y}ranlib
AS      = ${__Y}as
NO_CFT=1
CXX     = ${__Y}cxx
CC      = ${__Y}cc
CPP     = ${__Y}cpp
#FC     = ${__Y}f77  # notyet
LD      = ${__Y}ld
OBJC    = ${__Y}cc
.endif
.endif

http://blog.chris.tylers.info/index.php?/archives/249-Fedora-ARM-PandaStack.html (broken)

Use lots of target hardware in a cluster connected to a NFS server. Farm out the work across the cluster.

mav@ has made a SheevaPlug available online for building arm packages. It uses NFS storage but the networking is currently suffering performance problems.

Hardware from the NLM multi-threaded MIPS boxes in the Netperf cluster might be available for trying the same sort of thing.

Another possibility is to run the compiler on the host, but everything else in the emulator. There are a few ways of doing this. The simplest is to use distcc, but that still does the preprocessing in the emulator, which is not ideal, but faster than doing the entire compile there. If they share a filesystem (e.g. an NFS server in the host) then you could probably replace cc, c++, and so on in the emulated environment with shell scripts that ran the cross compiler on the host (just needs to pass the arguments down a socket, so a wrapper around netcat would probably work).

This idea comes from the paper: http://www.mewburn.net/luke/papers/build.sh.pdf

Krister Walfridsson has proposed building the packages natively in an emulator, using optimizations such as only emulating user-mode, and capturing system calls and running those natively (after manipulating the arguments).

An enhancement to that is to detect if an emulated program is gcc (for example), and invoking the host's cross-compiler natively for that case.

The initial progress looks very promising, with an arm emulator compiling six times faster than the native platform.

QEMU user mode information:

http://qemu.weilnetz.de/qemu-tech.html#User-emulation-specific-details

http://static.usenix.org/event/usenix05/tech/freenix/full_papers/bellard/bellard.pdf

QEMU supports user mode emulation in order to run a Linux process compiled for one target CPU on another CPU.

At the CPU level, user mode emulation is just a subset of the full system emulation. No MMU simulation is done because QEMU supposes the user memory mappings are handled by the host OS. QEMU includes a generic Linux system call converter to handle endianness issues and 32/64 bit conversions. Because QEMU supports exceptions, it emulates the target signals exactly. Each target thread is run in one host thread.

Endian independent (EI) UFS:

pjd@'s diffs for read-only EI UFS

FreeBSD Qemu User Mode HowTo

FreeBSD Qemu User Mode ToDo list

Using the following method (proposed by NgieCooper) should permit cross-building with build systems that don't work within “sane” constraints like autoconf does.

1. Run make installworld / make distributeworld (use etcupdate/mergemaster for this?) in order to install the target system into a chroot.
2. Nullfs in some directories from the target host in order to deal with host tools ({BUILD,EXTRACT,FETCH,PATCH,PKG}_DEPENDS).
3. Copy in host binaries that need to delve into kernel structures on the host system in a separate directory (/sbin/ifconfig, etc).

This process will be executed for everything that's a build tool for the target package ({BUILD,EXTRACT,FETCH,PATCH,PKG}_DEPENDS).

1. Build all necessary tools ({BUILD,EXTRACT,FETCH,PATCH,PKG}_DEPENDS) for the host system statically.
2. chroot into encapsulated world, setting the PATH and UNAME_* vars appropriately. The code in the fake_target_host function in FreeNAS effectively does that.
3. Build all tool packages statically.

This process will be executed for everything that's a runtime package for the target package ({LIB,RUN,PKG}_DEPENDS).

1. Build all necessary host-system packages that are tools (see “Process for each host-system package”).
2. chroot into encapsulated world, similar to “Process for each host-system package”.
3. Build all packages dynamically.

1. No modifications need to be for the bulk majority of the package build systems out there.
2. Package build changes won't spam the running system.
3. Package build/install changes to the encapsulated world can be better audited potentially.

1. Is a FreeBSD specific solution and can't be shared with other Unix distros.
2. Requires nullfs.
3. Requires lots of disk space:
   • Some packages will now be built twice because they'll need to be built for the host and the target (python for instance).
   • Toolchain will be fatter.
4. Is considerably more complicated overall than making everything conform to autoconf “sanity”.
5. Issues with ({BUILD,EXTRACT,..}_DEPENDS) being implied as runtime dependencies ({LIB,PKG,RUN}_DEPENDS) need to be fixed. This should be done regardless as it allows package-only systems to be considerably smaller.
6. Explicit AS, CC, LD, etc setting needs to be fixed (should be addressed with the clang porting effort).
7. Explicit $PATH setting needs to be fixed.
8. Some packages that do tricky things trying to detect the host system (e.g. net-mgmt/net-snmp) might not necessarily work in all cases.
9. Some packages might need to be fixed to build NO_SHARED, etc.