BootFreebsdFromNfs [FreeBSD WIKI]

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== How to boot FreeBSD from NFS? ======

//Heavily based on [[https://oshogbo.vexillium.org/blog/28/]]//

===== Steps =====

==== Build FreeBSD from source ====

You may want to look at the handbook first: https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/makeworld.html

  - Get the %%FreeBSD%% source code. I'll call the root directory of our local source code copy ''%%freebsdsrc/%%''.

<code>
git clone https://github.com/freebsd/freebsd freebsdsrc
</code>

  - Go to the directory with the source code.

<code>
cd freebsdsrc
</code>

  - Go to the directory with the kernel's configuration code.

<code>
# If your system is 32-bit.
cd sys/i386/conf/
# If your system is 64-bit.
cd sys/amd64/conf/
</code>

  - Get a copy of the **GENERIC** kernel (I'll call my copy //CUSTOM0MP//). It will be the base for our customisations.

<code>
cp GENERIC CUSTOM0MP
</code>

  - Now it is time to build the kernel:

<code>
make -j$(sysctl -n hw.ncpu) buildkernel KERNCONF=CUSTOM0MP -DNO_CLEAN
</code>

//Estimated time on Hasee Q540S running on a one processor: 2 hours.//\\ //Estimated time on Dell L702X running on 8 processors: 19 minutes.//

  - Go to the ''%%freebsdsrc/%%'' directory and build the world:

<code>
make -j$(sysctl -n hw.ncpu) buildworld KERNCONF=CUSTOM0MP -DNO_CLEAN
</code>

//Estimated time on Hasee Q540S running on a one processor: 8 hours.//\\ 
//Estimated time on Dell L702X running on 8 processors: 98 minutes.//

  - Let's configure the destination directory for the root file system of your new %%FreeBSD%%. I chose the ''%%/usr/home/0mp/CUSTOM0MP%%''. Just add the following lines to ''%%/etc/src.conf%%'' to set it up:

<code>
.if ${KERNCONF} == "CUSTOM0MP"
DESTDIR?=/usr/home/0mp/CUSTOM0MP
MODULES_OVERRIDE=md ufs
.endif
</code>

//Remember to use spaces not tabs if you wish to indent the code.//

  - Create the root file system now:

<code>
sudo make distrib-dirs KERNCONF=CUSTOM0MP
</code>

//Estimated time on Hasee Q540S running on a one processor: a few seconds.//

<code>
make distribution KERNCONF=CUSTOM0MP
</code>

//Estimated time on Hasee Q540S running on a one processor: 3 minutes.//

  - Now it's time to install the world and the kernel.

<code>
sudo make installworld KERNCONF=CUSTOM0MP
</code>

//Estimated time on Hasee Q540S running on a one processor: 5 minutes.//

<code>
sudo make installkernel KERNCONF=CUSTOM0MP
</code>

//Estimated time on Hasee Q540S running on a one processor: a few seconds.//

  - Done. Now it's time to configure you [[/VirtualBox|VirtualBox]].

==== Configure VirtualBox ====

=== Configure VirtualBox network ===

  - Open **File** > **Preferences** > **Network** > **Host-only Networks**.
  - Click the **Add host-only network (Ins)** button with a flag and a plus.
  - Select your new network (I'll assume its name is ''%%vboxnet0%%'') and click the **Edit host-only network (Space)** screwdriver button.
  - Open **DHCP Server** and uncheck **Enable Server**.

=== Configure a virtual machine ===

  - Go **Machine** > **New** to open the **Create Virtual Machine** window.
  - Call your %%FreeBSD%% machine however you wish. Choose the proper version of %%FreeBSD%% - pay attention to the **Version** (32-bit or 64-bit).
  - Give your machine some RAM.
  - Add a reasonable hard drive.
  - Select your newly created machine and open **Settings** > **Network** > **Adapter 1**.
  - Check **Enable Network Adapter**.
  - Choose **Host-only Adapter** from the **Attached to:** list. Select //vboxnet0// in the **Name:** field.
  - Click **Advanced** and put down the value of the **MAC Address:** field somewhere.

The %%VirtualBox%% configuration is also covered by [the oshogbo's tutorial] in the **VIRTUALBOX CONFIGURATION** section.

==== Set up TFTP server ====

  - Enable TFTP.

Uncomment the

<code>
tftp dgram udp wait root /usr/libexec/tftpd tftpd -l -s /tftpboot
</code>

line from the ''%%/etc/inetd.conf%%'' file.

Edit it so that ''%%/tftpboot%%'' points to your kernel's root directory. In my case the line looks like this:

<code>
tftp    dgram   udp     wait    root    /usr/libexec/tftpd      tftpd -l -s /usr/home/0mp/CUSTOM0MP
</code>

  - Add ''%%inetd_enable="YES"%%'' to ''%%/etc/rc.conf%%'' and restart the service:

<code>
sudo service inetd restart
</code>

  - Now let's get a copy of ''%%pxeboot%%''.

<code>
sudo cp /boot/pxeboot /usr/home/0mp/CUSTOM0MP/boot/pxeboot
</code>

  - Test if everything is working fine. Run:

<code>
tftp localhost
</code>

Now in type ''%%get boot/pxeboot%%'' in the tftp prompt. You should get a message similar to:

<code>
Received 268289 bytes during 0.1 seconds in 526 blocks
</code>

You've just downloaded a copy of ''%%pxeboot%%'' to the current folder - you should probably delete it. See FAQ if you've got ''%%pxeboot%%'' files scattered around the system.

==== Set up NFS server ====

My **custom0mp** machine will be assigned the 192.168.56.103 IP address.

  - Add this line to the ''%%/etc/exports%%'' file:

<code>
/usr/home/0mp/CUSTOM0MP -alldirs -maproot=0 -network 192.168.56.0 -mask 255.255.255.0
</code>

  - Add this line to ''%%/etc/rc.conf%%'':

<code>
nfs_server_enable="YES"
</code>

  - Restart the NFS server:

<code>
sudo service nfsd restart
</code>

  - Check if everything is working fine. Run:

<code>
showmount -e
</code>

You should get an input like:

<code>
Exports list on localhost:
/usr/home/0mp/CUSTOM0MP                192.168.56.0 
</code>

  - Set up what and where to mount. Add the following line to ''%%/usr/home/0mp/CUSTOM0MP/etc/fstab%%'':

<code>
192.168.56.1:/usr/home/0mp/CUSTOM0MP / nfs rw 0 0
</code>

  - Set up ''%%loader.conf%%'' and ''%%rc.conf%%''.

Add the following lines to ''%%/usr/home/0mp/CUSTOM0MP/boot/loader.conf%%'':

<code>
autoboot_delay="3"
beastie_disable="YES"
boot.nfsroot.options="nolockd,tcp"
</code>

Add the following lines to ''%%/usr/home/0mp/CUSTOM0MP/etc/rc.conf%%'':

<code>
sshd_enable="YES"
sendmail_enable="NO"
sendmail_submit_enable="NO"
sendmail_outbound_enable="NO"
sendmail_msp_queue_enable="NO"
cron_enable="NO"
devd_enable="NO"
</code>

=== Configure a DHCP server ===

  - Install ''%%isc-dhcpd%%''. For example with:

<code>
sudo pkg install isc-dhcp43-server
</code>

  - Modify ''%%/usr/local/etc/dhcpd.conf%%'' to suit your needs. Here's mine configuration:

<code>
default-lease-time 600;
max-lease-time 7200;

ddns-update-style none;

# Set the subnet and the mask values the same as in /etc/exports.
subnet 192.168.56.0 netmask 255.255.255.0 {
    authoritative;
    use-host-decl-names on;
    option subnet-mask 255.255.255.0;
    option broadcast-address 192.168.56.255;

    host custom0mp {
        hardware ethernet 08:00:27:22:A9:64;

        # The fixed IP address assigned to the virtual machine.
        fixed-address 192.168.56.103;

        # The path to the boot loader; it is relative to the root-path below.
        filename "boot/pxeboot";

        # The IP address that provides the NFS.
        next-server 192.168.56.1;

        # The path to the root directory over the NFS.
        option root-path "/usr/home/0mp/CUSTOM0MP/";
    }
}
</code>

  - Add these lines to ''%%/etc/rc.conf%%'':

<code>
dhcpd_enable="YES"
dhcpd_ifaces="vboxnet0"
</code>

  - Start the DHCP deamon:

<code>
sudo service isc-dhcpd restart
</code>

This part is also covered in detail by [[https://oshogbo.vexillium.org/blog/28/|the oshogbo's tutorial]] in the **DHCP SERVER** section.

==== Boot FreeBSD over NFS ====

  - Reboot your host %%FreeBSD%% (//just to be sure that everything is configured//).
  - Start [[/VirtualBox|VirtualBox]].
  - Start the virtual machine to activate //vboxnet0//.

This step is needed because DHCP probably failed to start since there had been no ''%%vboxnet0%%'' interface before a virtual machine was launched. Now, as [[/VirtualBox|VirtualBox]] host-only network is up and running we want to restart the DHCP server.

  - Restart the DHCP server

<code>
sudo service isc-dhcpd restart
</code>

It might fail if //vboxnet0// is not yet available.

  - Start your virtual machine. Hopefully, it will boot over the network.

You might need to restart your machine if the DHCP server isn't yet fully functional.

===== Extras =====

  - [[https://forums.freebsd.org/threads/42172/|A potentially useful thread on the FreeBSD forum about mounting from NFS (Can't NFS mount exported root).]]
  - Here's my NFS related section in ''%%/etc/rc.conf%%'':

<code>
inetd_enable="YES"
nfs_server_enable="YES"
dhcpd_enable="YES"
dhcpd_ifaces="vboxnet0"
</code>

===== FAQ =====

  - Why do I have so many ''%%pxeboot%%'' files scattered around? Whenever you do this to test whether your NFS is bootable:

    - Run ''%%tftp localhost%%''.
    - Enter ''%%get boot/pxeboot%%'' when prompted by ''%%tftp%%''.
    - You should get a message like this:

<code>
Received 333825 bytes during 0.0 seconds in 655 blocks
</code>

You actually save a copy of the ''%%pxeboot%%'' file in the current working director when you do this. Hence the presence of those ''%%pxeboot%%'' files all over your system.

  - Why do the boot process freeze? You might see something similar to

<code>
NFS ROOT: 192.168.56.1:/usr/home/0mp/CUSTOM0MP/
</code>

In my case I had to leave it for 3 hours to be able to log in on Hasee Q540S. It takes about a minute to boot over NFS on Dell L702X however.

  - Why can't I add a user on my virtual machine? If you are getting an error similar to this one:

<code>
passwd: could not lock the passwd file: Operation not supported
</code>

you might be missing ''%%boot.nfsroot.options="nolockd,tcp"%%'' from the boot/loader.conf file of your NFS.

If it doesn't help then try to tinker with [[https://lists.freebsd.org/pipermail/freebsd-fs/2008-December/005540.html|rpc.lockd(8) and rpc.statd(8); I stumbled upon them as I was looking for the solution myself]].

===== Resources =====

  * [[http://oshogbo.vexillium.org/blog/28/|FreeBSD diskless on VirtualBox.]]
  * [[https://www.bsdcan.org/2010/schedule/attachments/144_bsdcan2010_lstewart_smarties.pdf|FreeBSD Development for Smarties: The quest for a better kernel development environment.]]

----

{{tag>HowTo}}