Installing and Running a Gentoo Linux guest in bhyve

This guide will help you install and run a Gentoo Linux guest in bhyve. I assume you have familiarity installing Gentoo, and have read the Handbook section FreeBSD as a Host with bhyve.

1. You must be root.

1. Setup bhyve and a network interface as described in the handbook under “Preparing the Host.” I assume tap0 and bridge0 for this guide.

1. Create a zfs volume to store the Gentoo disk. In the handbook see “Using ZFS with bhyve Guests.” For example, to create a 32 GB volume under the zroot zpool, do this: zfs create -V32G -o volmode=dev zroot/gentoo

1. install grub-bhyve pkg install grub-bhyve

1. create a directory on the host for all of your Gentoo bhyve-specific stuff: I assume /root/gentoo for this guide.

1. download the Gentoo minimal install CD. The file path should be something like /root/gentoo/install-amd64-minimal-20190129T214502Z.iso

Create /root/gentoo/device.map. Do not use ~ in your paths, it doesn't work correctly with grub-bhyve.

(cd0) /root/gentoo/install-amd64-minimal-20190129T214502Z.iso
(hd0) /dev/zvol/zroot/gentoo

1. now run grub-bhyve. In this example I'm giving the guest 8GB RAM, and calling the vm gentoo

grub-bhyve -m /root/gentoo/device.map -r cd0 -M 8192M -d /grub gentoo

1. select the first boot option in grub or let the timeout expire. If you see a grub prompt and no boot options, something is wrong.
2. grub-bhyve will return you to a terminal: that's good.

1. now that we've loaded the installer kernel via Grub, run bhyve. For this command I'm giving the guest 6 cores and 8GB RAM, and continuing to call the vm gentoo

bhyve -AHP -s 0:0,hostbridge -s 1:0,lpc -s 2:0,virtio-net,tap0 -s 3:0,virtio-blk,/dev/zvol/zroot/gentoo -s 4:0,ahci-cd,/root/gentoo/install-amd64-minimal-20190129T214502Z.iso -l com1,stdio -c 6 -m 8192M gentoo

1. at this point you should be sitting at a livecd prompt, ready to install Gentoo.

1. When partitioning the disk, the ZFS disk volume in Linux is /dev/vda.
2. Make sure you open your firewall on the host for the guest with something like this:

pass quick on tap0 all
pass quick on bridge0 all

1. Helpful config options for the kernel:
   1. Set the config options recommended in the Gentoo handbook first. https://wiki.gentoo.org/wiki/Handbook:AMD64/Installation/Kernel#Activating_required_options
   2. Add these options for a Virtualized Linux guest: https://wiki.gentoo.org/wiki/QEMU/Linux_guest#Kernel
   3. If you want the hardware random number generator from your host machine: Y Device Drivers->Character Devices->Hardware Random Number Generator Core Support->VirtIO Random Number Generator support

1. For the clock to be the same on both the host and the guest, do this.
   1. set your timezone to the same timezone as the host, as described in the Gentoo handbook.
   2. edit /etc/conf.d/hwclock:

clock="local"
clock_hctosys="YES"
clock_systohc="NO"

1. The terminal size defaults to 80×25 which is horrible when you're trying to configure kernel options with make menuconfig. Since Linux thinks this is a serial console, we can set the size like this:

stty rows 50 cols 100
dmesg

the dmesg seems to help it expand.

1. Some helpful info here: https://github.com/churchers/vm-bhyve/issues/186

1. we're using BIOS, not UEFI, so edit /etc/portage/make.conf: GRUB_PLATFORMS="pc efi-64"
2. install grub2 and make sure it got the options right: emerge --ask --verbose sys-boot/grub:2
3. install grub to the “disk” grub-install /dev/vda
4. edit /etc/default/grub:

GRUB_TERMINAL=console
GRUB_GFXPAYLOAD_LINUX=text

1. run mkconfig: grub-mkconfig -o /boot/grub.cfg

1. exit out of your chroot as described in the Gentoo handbook, and shutdown the installer environment cleanly.
2. kill the vm: bhyvectl --destroy --vm=gentoo
3. Startup grub-bhyve and point it to the /boot partition on your Gentoo drive. I created my /boot on partition 2 of the disk, so it is -r hd0,2. I'm giving the vm 8GB of RAM.

grub-bhyve -m /root/gentoo/device.map -M 8192M -r hd0,2 -d /grub gentoo

1. Startup bhyve with 6 cores and 8 GB of RAM: bhyve -AHP -s 0:0,hostbridge -s 1:0,lpc -s 2:0,virtio-net,tap0 -s 3:0,virtio-blk,/dev/zvol/zroot/gentoo -s 4:0,virtio-rnd,/dev/random -l com1,stdio -c 6 -m 8192M gentoo