Email: amracks@gmail.com

…

ContainerLinux in bhyve

Download the raw container linux image from CoreOS.

fetch https://stable.release.core-os.net/amd64-usr/current/coreos_production_image.bin.bz2
fetch https://stable.release.core-os.net/amd64-usr/current/coreos_production_image.bin.bz2.DIGESTS
fetch https://stable.release.core-os.net/amd64-usr/current/coreos_production_image.bin.bz2.DIGESTS.asc
fetch https://stable.release.core-os.net/amd64-usr/current/coreos_production_image.bin.bz2.DIGESTS.sig
fetch https://stable.release.core-os.net/amd64-usr/current/coreos_production_image.bin.bz2.sig

Verify the image

Optionally create a ZFS volumne for the disk drive

zfs create -V 32G -s -o refreservation=none -o volmode=dev zroot/coreos

bzcat the image into your new drive (this can be a file if not using ZFS)

bzcat coreos_production_image.bin.bz2 > /dev/zvol/zroot/coreos

Optionally snapshot if you are using ZFS

zfs snap zroot/coreos@vanilla

At this point you can boot the container linux VM, but it will not be “provisioned”, will not be accessible, and is of little use. Read on for how to provision an instance of this VM.

bhyve -c 1 -m 4G -A -H -P \
  -s 0,hostbridge \
  -s 3,ahci-hd,/dev/zvol/zroot/coreos \
  -s 31,lpc \
  -l com1,stdio \
  -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
  coreos

There are a couple of options here for accessing and configuring the VM: 1. You can manually boot of a linux ISO and add your ssh key to ~core/.ssh/authorized_keys 1. You can provision the VM via a network-hosted config

I'm using nginx to host my configuration. You can possibly use FTP if you don't want to install any ports or packages, I have not tried it.

pkg install nginx

The old yml based cloud config is deprecated, you know how to “transpile” your yml to json using the container-linux-config-transpiler. It builds fine with go and gmake installed. Perhaps a port is in order.

https://github.com/coreos/container-linux-config-transpiler

Write your provisioning configuration in yml then convert to json and host on some network.

/usr/local/www/nginx/coreos.yml:

ct < /usr/local/www/nginx/coreos.yml > /usr/local/www/nginx/coreos.ign

To point your VM at your cloud config, hit 'e' immediately after starting the VM, then add the following paramters to the last line of the boot prompt.

coreos.first_boot=1 coreos.config.url=http://<host_or_ip>/coreos.ign

then ctl-x to boot with these paramters

bhyve -c 1 -m 4G -A -H -P \
   -s 0,hostbridge \
   -s 2,ahci-cd,/usr/home/amarks/distfiles/systemrescuecd-x86-5.2.2.iso \
   -s 3,ahci-hd,/dev/zvol/zrt/core01 \
   -s 4,virtio-net,tap1 \
   -s 29,fbuf,w=1280,h=1024,tcp=0.0.0.0:5900 \
   -s 30,xhci,tablet \
   -s 31,lpc \
   -l com1,stdio \
   -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
   core01

mount /dev/sda1

tar xf /media/coreos/grub/grub.cfg.tar

Important parts of grub.cfg

# Anything else the OEM adds should use this variable.
set linux_append=""

# Search for the OEM partition, load additional configuration if found.
if [ "$secure_boot" = "0" ]; then
   search --no-floppy --set oem --part-label OEM --hint "$root"
      if [ -n "$oem" -a -f "($oem)/grub.cfg" ]; then
             source "($oem)/grub.cfg"
      fi
fi

# Assemble the options applicable to all the kernels below
set linux_cmdline="rootflags=rw mount.usrflags=ro consoleblank=0 $linux_root $linux_console $first_boot $randomize_disk_guid $e
xtra_options $oem $linux_append"

This shows that linux_append should be used to add arguments to the Linux command line, and if there is a grub.cfg at the root of the OEM partition, it will be sourced, so that is where we can set linux_append.

Find the OEM partition using /dev/disk/by-partlabel

ls -l /dev/disk/by-partlabel

lrwxrwxrwx 1 root root 10 Aug 19 13:42 BIOS-BOOT -> ../../sda2
lrwxrwxrwx 1 root root 10 Aug 19 13:42 EFI-SYSTEM -> ../../sda1
lrwxrwxrwx 1 root root 10 Aug 19 13:42 OEM -> ../../sda6
lrwxrwxrwx 1 root root 10 Aug 19 13:43 OEM-CONFIG -> ../../sda7
lrwxrwxrwx 1 root root 10 Aug 19 13:42 ROOT -> ../../sda9
lrwxrwxrwx 1 root root 10 Aug 19 13:42 USR-A -> ../../sda3
lrwxrwxrwx 1 root root 10 Aug 19 13:42 USR-B -> ../../sda4

/dev/sda6 is labeled OEM

mount /dev/sda6 /media
vi /media/grub.conf

Container Linux specific options are documented here: https://coreos.com/os/docs/latest/booting-with-pxe.html

linux_append="coreos.autologin coreos.first_boot=1 coreos.config.url=http://<IP Address>/config.json"

Shutdown

shutdown -h now