How to boot FreeBSD from NFS?

Heavily based on https://oshogbo.vexillium.org/blog/28/

You may want to look at the handbook first: https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/makeworld.html

1. Get the FreeBSD source code. I'll call the root directory of our local source code copy freebsdsrc/.

git clone https://github.com/freebsd/freebsd freebsdsrc

1. Go to the directory with the source code.

cd freebsdsrc

1. Go to the directory with the kernel's configuration code.

# If your system is 32-bit.
cd sys/i386/conf/
# If your system is 64-bit.
cd sys/amd64/conf/

1. Get a copy of the GENERIC kernel (I'll call my copy CUSTOM0MP). It will be the base for our customisations.

cp GENERIC CUSTOM0MP

1. Now it is time to build the kernel:

make -j$(sysctl -n hw.ncpu) buildkernel KERNCONF=CUSTOM0MP -DNO_CLEAN

Estimated time on Hasee Q540S running on a one processor: 2 hours.
Estimated time on Dell L702X running on 8 processors: 19 minutes.

1. Go to the freebsdsrc/ directory and build the world:

make -j$(sysctl -n hw.ncpu) buildworld KERNCONF=CUSTOM0MP -DNO_CLEAN

Estimated time on Hasee Q540S running on a one processor: 8 hours.
Estimated time on Dell L702X running on 8 processors: 98 minutes.

1. Let's configure the destination directory for the root file system of your new FreeBSD. I chose the /usr/home/0mp/CUSTOM0MP. Just add the following lines to /etc/src.conf to set it up:

.if ${KERNCONF} == "CUSTOM0MP"
DESTDIR?=/usr/home/0mp/CUSTOM0MP
MODULES_OVERRIDE=md ufs
.endif

Remember to use spaces not tabs if you wish to indent the code.

1. Create the root file system now:

sudo make distrib-dirs KERNCONF=CUSTOM0MP

Estimated time on Hasee Q540S running on a one processor: a few seconds.

make distribution KERNCONF=CUSTOM0MP

Estimated time on Hasee Q540S running on a one processor: 3 minutes.

1. Now it's time to install the world and the kernel.

sudo make installworld KERNCONF=CUSTOM0MP

Estimated time on Hasee Q540S running on a one processor: 5 minutes.

sudo make installkernel KERNCONF=CUSTOM0MP

Estimated time on Hasee Q540S running on a one processor: a few seconds.

1. Done. Now it's time to configure you VirtualBox.

1. Open File > Preferences > Network > Host-only Networks.
2. Click the Add host-only network (Ins) button with a flag and a plus.
3. Select your new network (I'll assume its name is vboxnet0) and click the Edit host-only network (Space) screwdriver button.
4. Open DHCP Server and uncheck Enable Server.

1. Go Machine > New to open the Create Virtual Machine window.
2. Call your FreeBSD machine however you wish. Choose the proper version of FreeBSD - pay attention to the Version (32-bit or 64-bit).
3. Give your machine some RAM.
4. Add a reasonable hard drive.
5. Select your newly created machine and open Settings > Network > Adapter 1.
6. Check Enable Network Adapter.
7. Choose Host-only Adapter from the Attached to: list. Select vboxnet0 in the Name: field.
8. Click Advanced and put down the value of the MAC Address: field somewhere.

The VirtualBox configuration is also covered by [the oshogbo's tutorial] in the VIRTUALBOX CONFIGURATION section.

1. Enable TFTP.

Uncomment the

tftp dgram udp wait root /usr/libexec/tftpd tftpd -l -s /tftpboot

line from the /etc/inetd.conf file.

Edit it so that /tftpboot points to your kernel's root directory. In my case the line looks like this:

tftp    dgram   udp     wait    root    /usr/libexec/tftpd      tftpd -l -s /usr/home/0mp/CUSTOM0MP

1. Add inetd_enable="YES" to /etc/rc.conf and restart the service:

sudo service inetd restart

1. Now let's get a copy of pxeboot.

sudo cp /boot/pxeboot /usr/home/0mp/CUSTOM0MP/boot/pxeboot

1. Test if everything is working fine. Run:

tftp localhost

Now in type get boot/pxeboot in the tftp prompt. You should get a message similar to:

Received 268289 bytes during 0.1 seconds in 526 blocks

You've just downloaded a copy of pxeboot to the current folder - you should probably delete it. See FAQ if you've got pxeboot files scattered around the system.

My custom0mp machine will be assigned the 192.168.56.103 IP address.

1. Add this line to the /etc/exports file:

/usr/home/0mp/CUSTOM0MP -alldirs -maproot=0 -network 192.168.56.0 -mask 255.255.255.0

1. Add this line to /etc/rc.conf:

nfs_server_enable="YES"

1. Restart the NFS server:

sudo service nfsd restart

1. Check if everything is working fine. Run:

showmount -e

You should get an input like:

Exports list on localhost:
/usr/home/0mp/CUSTOM0MP                192.168.56.0 

1. Set up what and where to mount. Add the following line to /usr/home/0mp/CUSTOM0MP/etc/fstab:

192.168.56.1:/usr/home/0mp/CUSTOM0MP / nfs rw 0 0

1. Set up loader.conf and rc.conf.

Add the following lines to /usr/home/0mp/CUSTOM0MP/boot/loader.conf:

autoboot_delay="3"
beastie_disable="YES"
boot.nfsroot.options="nolockd,tcp"

Add the following lines to /usr/home/0mp/CUSTOM0MP/etc/rc.conf:

sshd_enable="YES"
sendmail_enable="NO"
sendmail_submit_enable="NO"
sendmail_outbound_enable="NO"
sendmail_msp_queue_enable="NO"
cron_enable="NO"
devd_enable="NO"

1. Install isc-dhcpd. For example with:

sudo pkg install isc-dhcp43-server

1. Modify /usr/local/etc/dhcpd.conf to suit your needs. Here's mine configuration:

default-lease-time 600;
max-lease-time 7200;

ddns-update-style none;

# Set the subnet and the mask values the same as in /etc/exports.
subnet 192.168.56.0 netmask 255.255.255.0 {
    authoritative;
    use-host-decl-names on;
    option subnet-mask 255.255.255.0;
    option broadcast-address 192.168.56.255;

    host custom0mp {
        hardware ethernet 08:00:27:22:A9:64;

        # The fixed IP address assigned to the virtual machine.
        fixed-address 192.168.56.103;

        # The path to the boot loader; it is relative to the root-path below.
        filename "boot/pxeboot";

        # The IP address that provides the NFS.
        next-server 192.168.56.1;

        # The path to the root directory over the NFS.
        option root-path "/usr/home/0mp/CUSTOM0MP/";
    }
}

1. Add these lines to /etc/rc.conf:

dhcpd_enable="YES"
dhcpd_ifaces="vboxnet0"

1. Start the DHCP deamon:

sudo service isc-dhcpd restart

This part is also covered in detail by the oshogbo's tutorial in the DHCP SERVER section.

1. Reboot your host FreeBSD (just to be sure that everything is configured).
2. Start VirtualBox.
3. Start the virtual machine to activate vboxnet0.

This step is needed because DHCP probably failed to start since there had been no vboxnet0 interface before a virtual machine was launched. Now, as VirtualBox host-only network is up and running we want to restart the DHCP server.

1. Restart the DHCP server

sudo service isc-dhcpd restart

It might fail if vboxnet0 is not yet available.

1. Start your virtual machine. Hopefully, it will boot over the network.

You might need to restart your machine if the DHCP server isn't yet fully functional.

1. A potentially useful thread on the FreeBSD forum about mounting from NFS (Can't NFS mount exported root).
2. Here's my NFS related section in /etc/rc.conf:

inetd_enable="YES"
nfs_server_enable="YES"
dhcpd_enable="YES"
dhcpd_ifaces="vboxnet0"

1. Why do I have so many pxeboot files scattered around? Whenever you do this to test whether your NFS is bootable:

1. Run tftp localhost.
2. Enter get boot/pxeboot when prompted by tftp.
3. You should get a message like this:

Received 333825 bytes during 0.0 seconds in 655 blocks

You actually save a copy of the pxeboot file in the current working director when you do this. Hence the presence of those pxeboot files all over your system.

1. Why do the boot process freeze? You might see something similar to

NFS ROOT: 192.168.56.1:/usr/home/0mp/CUSTOM0MP/

In my case I had to leave it for 3 hours to be able to log in on Hasee Q540S. It takes about a minute to boot over NFS on Dell L702X however.

1. Why can't I add a user on my virtual machine? If you are getting an error similar to this one:

passwd: could not lock the passwd file: Operation not supported

you might be missing boot.nfsroot.options="nolockd,tcp" from the boot/loader.conf file of your NFS.

If it doesn't help then try to tinker with rpc.lockd(8) and rpc.statd(8); I stumbled upon them as I was looking for the solution myself.

• FreeBSD diskless on VirtualBox.
• FreeBSD Development for Smarties: The quest for a better kernel development environment.