Table of Contents

Installing FreeBSD Root on ZFS (RAIDZ1) using GPT

Creating a bootable ZFS Filesystem

  1. Boot FreeBSD install DVD or USB Memstick
  1. Choose Fixit option in sysinstall
  1. Create GPT Disks
Fixit# gpart create -s gpt ad0
Fixit# gpart create -s gpt ad1
Fixit# gpart create -s gpt ad2
  1. Create the boot, swap and zfs partitions

Create 3 partitions on drives ad0, ad1 and ad2. The first partition contains the gptzfsboot loader which is able to recognize and load the loader from a ZFS partition. The second partition is a 4 GB swap partition. The third partition is the partition containing the zpool (60GB). 

Fixit# gpart add -b 34 -s 128k -t freebsd-boot ad0
Fixit# gpart add -s 4G -t freebsd-swap -l swap0 ad0
Fixit# gpart add -s 60G -t freebsd-zfs -l disk0 ad0
Fixit# gpart add -b 34 -s 128k -t freebsd-boot ad1
Fixit# gpart add -s 4G -t freebsd-swap -l swap1 ad1
Fixit# gpart add -s 60G -t freebsd-zfs -l disk1 ad1
Fixit# gpart add -b 34 -s 128k -t freebsd-boot ad2
Fixit# gpart add -s 4G -t freebsd-swap -l swap2 ad2
Fixit# gpart add -s 60G -t freebsd-zfs -l disk2 ad2
Note:
  1. While a ZFS Swap Volume can be used instead of the freebsd-swap partition, crash dumps can't be created on the ZFS Swap Volume.
  2. Sizes and offsets are specified in sectors (1 sector is typically 512 bytes) but can also be specified in MB or GB by adding an 'M' or 'G' suffix.
  1. Install the Protective MBR (pmbr) and gptzfsboot loader to both drives
Fixit# gpart bootcode -b /mnt2/boot/pmbr -p /mnt2/boot/gptzfsboot -i 1 ad0
Fixit# gpart bootcode -b /mnt2/boot/pmbr -p /mnt2/boot/gptzfsboot -i 1 ad1
Fixit# gpart bootcode -b /mnt2/boot/pmbr -p /mnt2/boot/gptzfsboot -i 1 ad2

This may fail with an “operation not permitted” error message, since the kernel likes to protect critical parts of the disk. If this happens for you, run: 

Fixit# sysctl kern.geom.debugflags=0x10
  1. Load ZFS kernel module
Fixit# kldload /mnt2/boot/kernel/opensolaris.ko
Fixit# kldload /mnt2/boot/kernel/zfs.ko
  1. Create ZFS Pool zroot
Fixit# mkdir /boot/zfs
Fixit# zpool create zroot raidz1 /dev/gpt/disk0 /dev/gpt/disk1 /dev/gpt/disk2
Fixit# zpool set bootfs=zroot zroot
Note:
  1. zroot is the name of the ZFS Pool, it could be anything (i.e. tank, data, …)
  2. A raidz1 pool requires a minimum of 3 disks and a maximum of 9 disks

Installing FreeBSD to the ZFS filesystem

  1. Create ZFS filesystem hierarchy

The fletcher4 algorithm should be more robust than the fletcher2 algorithm. 

Fixit# zfs set checksum=fletcher4					zroot
Fixit# zfs create -o compression=on	-o exec=on	-o setuid=off	zroot/tmp
Fixit# chmod 1777 /zroot/tmp
Fixit# zfs create							zroot/usr
Fixit# zfs create							zroot/usr/home
Fixit# cd /zroot ; ln -s /usr/home home
Fixit# zfs create -o compression=lzjb 			-o setuid=off	zroot/usr/ports
Fixit# zfs create -o compression=off	-o exec=off	-o setuid=off	zroot/usr/ports/distfiles
Fixit# zfs create -o compression=off	-o exec=off	-o setuid=off	zroot/usr/ports/packages
Note:

If you use nullfs or nfs to mount /usr/ports to different locations/servers, you will also need to nullfs/nfs mount /usr/ports/distfiles and/or /usr/ports/packages. 

Fixit# zfs create -o compression=lzjb 	-o exec=off	-o setuid=off	zroot/usr/src
Fixit# zfs create							zroot/var
Fixit# zfs create -o compression=lzjb 	-o exec=off	-o setuid=off	zroot/var/crash
Fixit# zfs create			-o exec=off	-o setuid=off	zroot/var/db
Fixit# zfs create -o compression=lzjb 	-o exec=on	-o setuid=off	zroot/var/db/pkg
Fixit# zfs create			-o exec=off	-o setuid=off	zroot/var/empty
Fixit# zfs create -o compression=lzjb 	-o exec=off	-o setuid=off	zroot/var/log
Fixit# zfs create -o compression=gzip 	-o exec=off	-o setuid=off	zroot/var/mail
Fixit# zfs create			-o exec=off	-o setuid=off	zroot/var/run
Fixit# zfs create -o compression=lzjb 	-o exec=on	-o setuid=off	zroot/var/tmp
Fixit# chmod 1777 /zroot/var/tmp
Note:
  1. Compression may be set to on, off, lzjb, gzip, gzip-N (where N is an integer from 1 (fastest) to 9 (best compression ratio. gzip is equivalent to gzip-6).
  2. Compression will cause some latency when accessing files on the ZFS filesystems. Use compression on ZFS filesystems which will not be accessed that often.
  1. Install FreeBSD to zroot
Fixit# cd /dist/8.0-*
Fixit# export DESTDIR=/zroot
Fixit# for dir in base catpages dict doc games info lib32 manpages ports; \
  do (cd $dir ; ./install.sh) ; done
Fixit# cd src ; ./install.sh all
Fixit# cd ../kernels ; ./install.sh generic
Fixit# cd /zroot/boot ; cp -Rlp GENERIC/* /zroot/boot/kernel/
  1. Make /var/empty readonly
Fixit# zfs set readonly=on zroot/var/empty
  1. chroot into /zroot
Fixit# chroot /zroot
  1. Create /etc/rc.conf
Fixit# echo 'zfs_enable="YES"' > /etc/rc.conf
Fixit# echo 'hostname="beastie.mydomain.local"' >> /etc/rc.conf
Fixit# echo 'ifconfig_re0="DHCP"' >> /etc/rc.conf
Note:

Replace re0 with the name of the Network interface for the new system

  1. Create /boot/loader.conf
Fixit# echo 'zfs_load="YES"' > /boot/loader.conf
Fixit# echo 'vfs.root.mountfrom="zfs:zroot"' >> /boot/loader.conf
  1. Change root's password
Fixit# passwd
  1. Set the local time zone
Fixit# tzsetup
  1. Create /etc/mail/aliases.db
Fixit# cd /etc/mail
Fixit# make aliases
  1. Exit from the /zroot
Fixit# umount /dev
Fixit# exit
  1. Install zpool.cache to the ZFS filesystem
Fixit# cp /boot/zfs/zpool.cache /zroot/boot/zfs/zpool.cache

Finish install

  1. Create /etc/fstab
Fixit# cat << EOF > /zroot/etc/fstab
# Device			Mountpoint		FStype	Options		Dump	Pass#
/dev/gpt/swap0			none			swap	sw		0	0
/dev/gpt/swap1			none			swap	sw		0	0
/dev/gpt/swap2			none			swap	sw		0	0
EOF
  1. export LD_LIBRARY_PATH 
Fixit# export LD_LIBRARY_PATH=/mnt2/lib
  1. Unmount all zfs filesystems
Fixit# zfs unmount -a
  1. Change mount points for zroot pool 
Fixit# zfs set mountpoint=legacy zroot
Fixit# zfs set mountpoint=/tmp zroot/tmp
Fixit# zfs set mountpoint=/usr zroot/usr
Fixit# zfs set mountpoint=/var zroot/var
  1. Exit Fixit mode, and then sysinstall. Remove the FreeBSD install DVD/Memstick and the system will boot using the ZFS root.