As of FreeBSD 12, VIMAGE is turned on by default. This lets jails have private network stacks, which is useful.
Here's an example /etc/jail.conf. Previously, the basic assumption here was that you're using vimage, specifying a custom MAC, and using dhclient(8) to pull a dynamic address. Given ordering requirements resulting from BZ#237656, experimentation with dhclient is on hold, so this version uses static addressing.
This assumes jail root in /var/jail/{foo,bar} and an already-extant
bridge0.
exec.prestart = "ifconfig epair${ep} create up";
exec.prestart += "ifconfig $bridge addm epair${ep}a";
exec.start = "/sbin/ifconfig epair${ep}b link $mac";
exec.start += "/sbin/ifconfig epair${ep}b 10.0.0.${ep}/24";
exec.start += "/sbin/route add -net default 10.0.0.1";
exec.start += "/bin/sh /etc/rc";
# positioning of this is up in the air at present - ideally we run
# prior to /etc/rc but this needs testing, hence static addressing,
# above.
#exec.? += "dhclient epair${ep}b";
# We're shutting down this way so that services using the network
# conclude before we rip out the network. Once the race condition
# bug noted above is resolved, we'll be able to shut down in
# exec.stop as normal, and likely ignore the -vnet line entirely.
exec.prestop = "/usr/sbin/jexec ${name} /bin/sh /etc/rc.shutdown";
exec.prestop += "/sbin/ifconfig epair${ep}b -vnet ${name}";
exec.poststop = "ifconfig $bridge deletem epair${ep}a";
exec.poststop += "ifconfig epair${ep}a destroy";
exec.clean;
mount.devfs;
devfs_ruleset = 110;
exec.system_user = "root";
exec.jail_user = "root";
vnet;
persist;
$bridge = "bridge0";
path = "/var/jail/$name";
vnet.interface = "epair${ep}b";
foo {
$ep = 2;
$mac = "00:00:00:00:00:00";
}
bar {
$ep = 3;
$mac = "00:00:00:00:00:01";
}
My /etc/devfs.rules turns on bpf, for dhclient - this is based on
the jail example in /etc/defaults/devfs.rules:
[jail_devfs=110] add include $devfsrules_hide_all add include $devfsrules_unhide_basic add include $devfsrules_unhide_login #add path fuse unhide #add path zfs unhide add path 'bpf*' unhide
The docs don't seem to say how to create a bridge and give it a static address, and
- You can't have multiple ifconfig lines.
- The order matters - you can't have addm first.
So, here's how I'm specifying this in /etc/rc.conf:
cloned_interfaces="bridge0" ifconfig_bridge0="inet 10.0.0.2 netmask 0xffffff00 addm em0" ifconfig_em0="up"
In case your system suffers from this, at least Intel NICs can experience a delay when adding the first epair to a bridge. In this case, attaching a “sacrificial” epair on boot should prevent subsequent impact:
cloned_interfaces="bridge0 epair0" ifconfig_bridge0="inet 10.0.0.2 netmask 0xffffff00 addm em0 addm epair0a" ifconfig_em0="up"