Installing FreeBSD Root on ZFS using a FreeBSD MBR Slice

This is an old revision of the document!

Fails to boot

BTX loader 1.00 BTX version is 1.02 Consoles: internal video/keyboard BIOS drive C: is disk0 BIOS 638kB/2094720kB available memory

FreeBSD/i386 bootstrap loader, Revision 1.1 (root@, Sat Jul 18 10:58:41 UTC 2009

can't load 'kernel'

Type '?' for a list of commands, 'help' for more detailed help.

Boot FreeBSD install CD/DVD

Choose Fixit option in sysinstall

Create Slice using gpart

Fixit# gpart show ad0
=>       63  625142385  ad0  MBR  (289G)
         63  209712447    1  !7  (100G)
  209712510     417690    2  !136  (204M)
  210130200  415012248       - free -  (198G)

Fixit# gpart add -b 210130200  -s 415012248 -t freebsd ad0
ad0s3 added

Fixit# gpart show ad0
=>       63  625142385  ad0  MBR  (289G)
         63  209712447    1  !7  (100G)
  209712510     417690    2  !136  (204M)
  210130200  415012248    3  freebsd  (198G)

Sizes and offsets are specified in sectors (1 sector is typically 512 bytes).

Load ZFS kernel module

Fixit# kldload /mnt2/boot/kernel/opensolaris.ko
Fixit# kldload /mnt2/boot/kernel/zfs.ko

Create ZFS Pool zroot

Fixit# mkdir /boot/zfs
Fixit# zpool create zroot /dev/ad0s3
Fixit# zpool set bootfs=zroot zroot

NOTE: zroot is the name of the ZFS Pool, it could be anything (i.e. tank, data)

Create ZFS Swap volume

Fixit# zfs create -V 2gb zroot/swap
Fixit# zfs set org.freebsd:swap=on zroot/swap
Fixit# zfs set checksum=off zroot/swap

Install boot Manager

Fixit# gpart bootcode -b /mnt2/boot/boot0 ad0
ad0 has bootcode

Install ZFS boot

- Export zroot, before installing boot code

Fixit# zpool export zroot

- Install the boot1 stage:

Fixit# dd if=/mnt2/boot/zfsboot of=/dev/ad0s3 count=1

- Install the boot2 zfs stage into the convienient hole in the ZFS filesystem on-disk format which is located just after the ZFS metadata (this is the seek=1024).

Fixit# dd if=/mnt2/boot/zfsboot of=/dev/ad0s3 skip=1 seek=1024

- Import zroot to continue the install

Fixit# zpool import zroot

Create ZFS filesystem hierarchy

The fletcher4 algorithm should be more robust than the fletcher2 algorithm.

Fixit# zfs set checksum=fletcher4					zroot

Fixit# zfs create -o compression=on	-o exec=on	-o setuid=off	zroot/tmp
Fixit# chmod 1777 /zroot/tmp

Fixit# zfs create							zroot/usr
Fixit# zfs create							zroot/usr/home
Fixit# cd /zroot ; ln -s /usr/home home

Fixit# zfs create -o compression=lzjb 			-o setuid=off	zroot/usr/ports
Fixit# zfs create -o compression=off	-o exec=off	-o setuid=off	zroot/usr/ports/distfiles
Fixit# zfs create -o compression=off	-o exec=off	-o setuid=off	zroot/usr/ports/packages

Note:

If you use nullfs or nfs to mount /usr/ports to different locations/servers, you will also need to nullfs/nfs mount /usr/ports/distfiles and/or /usr/ports/packages.

Fixit# zfs create -o compression=lzjb 	-o exec=off	-o setuid=off	zroot/usr/src

Fixit# zfs create							zroot/var
Fixit# zfs create -o compression=lzjb 	-o exec=off	-o setuid=off	zroot/var/crash
Fixit# zfs create			-o exec=off	-o setuid=off	zroot/var/db
Fixit# zfs create -o compression=lzjb 	-o exec=on	-o setuid=off	zroot/var/db/pkg
Fixit# zfs create			-o exec=off	-o setuid=off	zroot/var/empty
Fixit# zfs create -o compression=lzjb 	-o exec=off	-o setuid=off	zroot/var/log
Fixit# zfs create -o compression=gzip 	-o exec=off	-o setuid=off	zroot/var/mail
Fixit# zfs create			-o exec=off	-o setuid=off	zroot/var/run
Fixit# zfs create -o compression=lzjb 	-o exec=on	-o setuid=off	zroot/var/tmp
Fixit# chmod 1777 /zroot/var/tmp

Note:

Compression may be set to on, off, lzjb, gzip, gzip-N (where N is an integer from 1 (fastest) to 9 (best compression ratio. gzip is equivalent to gzip-6).
Compression will cause some latency when accessing files on the ZFS filesystems. Use compression on ZFS filesystems which will not be accessed that often.

Install FreeBSD to zroot

Fixit# cd /dist/8.0-*
Fixit# export DESTDIR=/zroot
Fixit# for dir in base catpages dict doc games info lib32 manpages ports; \
  do (cd $dir ; ./install.sh) ; done
Fixit# cd src ; ./install.sh all
Fixit# cd ../kernels ; ./install.sh generic
Fixit# cd /zroot/boot ; cp -Rlp GENERIC/* /zroot/boot/kernel/

Make /var/empty readonly

Fixit# zfs set readonly=on zroot/var/empty

chroot into /zroot

Fixit# chroot /zroot

Create /etc/rc.conf

Fixit# echo 'zfs_enable="YES"' > /etc/rc.conf
Fixit# echo 'hostname="beastie.mydomain.local"' >> /etc/rc.conf
Fixit# echo 'ifconfig_re0="DHCP"' >> /etc/rc.conf

Note:

Replace re0 with the name of the Network interface for the new system

Create /boot/loader.conf

Fixit# echo 'zfs_load="YES"' > /boot/loader.conf
Fixit# echo 'vfs.root.mountfrom="zfs:zroot"' >> /boot/loader.conf

Change root's password

Fixit# passwd

Set the local time zone

Fixit# tzsetup

Create /etc/mail/aliases.db

Fixit# cd /etc/mail
Fixit# make aliases

Exit from the /zroot

Fixit# umount /dev
Fixit# exit

Install zpool.cache to the ZFS filesystem

Fixit# cp /boot/zfs/zpool.cache /zroot/boot/zfs/zpool.cache

Create /etc/fstab

Fixit# cat << EOF > /zroot/etc/fstab
# Device			Mountpoint		FStype	Options		Dump	Pass#
EOF

export LD_LIBRARY_PATH

Fixit# export LD_LIBRARY_PATH=/mnt2/lib

Change mount points for zroot pool

Fixit# zfs set mountpoint=legacy zroot
Fixit# zfs set mountpoint=/tmp zroot/tmp
Fixit# zfs set mountpoint=/usr zroot/usr
Fixit# zfs set mountpoint=/var zroot/var

Unmount all zfs filesystems

Fixit# zfs unmount -a

Exit Fixit mode, and then sysinstall. Remove the FreeBSD install CD and the system will boot using the ZFS root.

Installing FreeBSD Root on ZFS using a FreeBSD MBR Slice

Creating a bootable ZFS Filesystem

Installing FreeBSD to the ZFS filesystem

Finish install

FreeBSD WIKI