bhyve, pronounced “beehive” is a hypervisor/virtual machine manager for FreeBSD that supports most Intel and AMD processors that report the “POPCNT” (POPulation Count) processor feature in dmesg(8).
Most Intel Atom C2000, Celeron, Core i3, i5, i7 and related Xeon processors support bhyve but early “Core” processors may only support one virtual machine.
Most “Barcelona” class and newer AMD processors support bhyve but some, notably the “Kuma” core processors include POPCNT but lack the required “NRIPS” (Next RIP Save) feature.
A: The developers mercifully retired “BHyVe” and simply refer to it as “bhyve”, after the utility.
A: Begin with the FreeBSD Handbook chapter and manual page.
Yes. See the xhyve, Pluribus Netvisor, and bhyve in Illumos-based distributions efforts.
A: bhyve supports any version of FreeBSD i386/amd64 with VirtIO support. OpenBSD, NetBSD and GNU/Linux are supported using the sysutils/grub2-bhyve port.
UEFI support is available for FreeBSD, OpenBSD, GNU/Linux, SmartOS and Windows:
Tested GNU/Linux distributions include:
See the grub2-bhyve page for detailed sysutils/grub2-bhyve instructions and for best results use FreeBSD CURRENT with the -w bhyve flag.
To load a FreeBSD kernel from disk image vm0.img with 256M RAM and the name vm0:
/usr/sbin/bhyveload -m 256 -d ./vm0.img vm0
This will show the FreeBSD loader screen and you should see the device /dev/vmm/vm0
To boot the VM with 2 vCPUs, the same 256M RAM and the tap0 network interface:
/usr/sbin/bhyve -c 2 -m 256 -A -H -P \ -s 0:0,hostbridge \ -s 1:0,virtio-net,tap0 \ -s 2:0,ahci-hd,./vm0.img \ -s 31,lpc -l com1,stdio \ vm0
After the VM has been shutdown, its resources can be reclaimed with:
/usr/sbin/bhyvectl --destroy --vm=vm0
The included method booting FreeBSD virtual machines is the /usr/share/examples/bhyve/vmrun.sh script:
fetch ftp://ftp.freebsd.org/pub/FreeBSD/snapshots/VM-IMAGES/11.0-STABLE/amd64/Latest/FreeBSD-11.0-STABLE-amd64.raw.xz unxz FreeBSD-11.0-STABLE-amd64.raw.xz sudo sh /usr/share/examples/bhyve/vmrun.sh -d FreeBSD-11.0-STABLE-amd64.raw vm0
FreeNAS also offers a graphical bhyve management interface: http://www.freenas.org/
See the UEFI and bhyve Windows Virtual Machines wiki pages.
A: Yes
A: Yes, but you must enable it: look for the GUI or vhv.allow or vhv.enable options, depending on your version of VMware. VMware Fusion: Processors & Memory: Advanced options: Enable hypervisor applications in this virtual machine
Confirmed with VMware Workstation, VMware Player, VMware Fusion and ESXi. Note that you must stop all running VMs before you can enable EPT pass-through.
A: Maybe. You must enable it and note the known limitations. Please let us know if you are successful.
A: No. See this thread for more information.
A: VirtualBox does not, but Hyper-V has nested EPT available to Windows Insiders.
A: bhyve does not currently support reading foreign disk image formats directly, so one will have to convert existing disk images to a raw image using qemu-img. This can be accomplished by doing the following on a FreeBSD system:
$ pkg install qemu-devel $ qemu-img convert -f vmdk -O raw virtual_box_image.vmdk bhyve_raw_image.img
VMDK, QCOW, QCOW2, VDI and more images can be converted to raw images using the qemu-img(1) utility in the emulators/qemu-devel port.
A: See OmniOS.
A: The vmm.ko loadable kernel module, the libvmmapi library, the bhyve, bhyveload and bhyvectl utilities.
In total these binaries are under 500K in size.
A: Yes, on Intel CPUs. See pci_passthru.
A: Yes, and they provide non-blocking I/O for higher performance.
CD: -s 4,ahci-cd,/path/to/image.iso
HD: -s 4,ahci-hd,/path/to/disk.img
A: Yes, and it can be accessed several ways including as a nmdm nullmodem terminal or a pty. stdio output can also be sent to a terminal multiplexer like tmux or screen. Example nmdm syntax is:
-s 31,lpc -l com1,stdio
Becomes:
-s 31,lpc -l com1,nmdm0A
Before starting the VM, run:
kldload nmdm
Start the VM but note that it will not display any output when booted. Connect to the serial interface with:
cu -l /dev/nmdm0B -s 9600
Notes:
A is connected to B
nmdmN is a a 32 bit unsigned integer so you have plenty to use.
ptys can be created with socat.
This functionality is included in the FreeBSD 10.0 release.
A: Yes, the kern.vm_guest sysctl will report if a FreeBSD guest is under KVM or bhyve.
A: The source is in FreeBSD SVN source repository: sys/amd64/vmm/, usr.sbin/bhyve/, usr.sbin/bhyveload/, usr.sbin/bhyvectl/, and lib/libvmmapi/
# kldload vmm vmx_init: processor does not support VMX operation module_register_init: MOD_LOAD (vmm, 0xffffffff81e17169, 0) error 6
# sh /usr/share/examples/bhyve/vmrun.sh ... Launching virtual machine "... vm_create: Device not configured
vmcreate: Invalid argument
A: This indicates that your /etc/ttys is not properly configured for serial output for use with bhyve.
ttyu0 "/usr/libexec/getty 3wire.9600" vt100 on secure
A: vmrun.sh creates disk images using the truncate(1) command and as per the tar(1) manual page, “There is not yet any support for multi-volume archives or for archiving sparse files.”
A: Access points cannot determine the destination VMs and only see the host.
A: You are probably attempting to boot an older version of FreeBSD on a zpool that you created with a newer version of FreeBSD and the available ZFS features are not available. A safe set of features to use with FreeBSD 9.2 and 10.0 are:
zpool create -d -o feature@async_destroy=enabled -o feature@empty_bpobj=enabled -o feature@lz4_compress=enabled -o ...
A: This can happen if your kernel and world are out of sync.
A: Yes: The gdb interface.
A: The bottom most hypervisor needs to run the virtual interface in promiscuous mode in order for packets destined for an ethernet address other than that of the first level guest to be received by the virtual interface.
A: The most popular places to discuss bhyve are the freebsd-virtualization@freebsd.org mailing list and the #bhyve channel on irc.libera.chat.
A: First and foremost, by testing bhyve on your unique hardware and in every configuration you can imagine. There could easily be remaining bugs and it is in everyone's best interests for these to be resolved ASAP.
If you think you have an issue, considering asking about it on IRC. If you know you have an issue, consider posting it on the FreeBSD Virtualization mailing list.
Testing ideas:
A: If your host or VM panics and enters the debugger, the information you provide can be very useful for revealing and reproducing bugs. Typing “bt” at the debugger will produce a backtrace and “show thread” on various hex strings may yield more. Please capture this information and use your best judgement in sending it to the bhyve developers or the freebsd-virtualization mailing list. If in doubt, I am happy to relay reports (address below).
Do consider using the sysutils/panicmail port with bhyve.