200808 DevSummit Cambridge

• All commits go through vimage-commit2 branch step by step. Julian and Marko bringing in the basic set and reviewers are going to finalize and produce the patch.
• RobertWatson, BrooksDavis, KrisKennaway volunteered to also review patches.
• BjoernZeeb volunteered to keep an eye on the schedule, progress, …
• After step 2 we are going to provide a clean p4 branch people can use to branch their own working area off to virtualize other subsystems like vimage.sysvipc. If you want to work on anything before this use the 'vimage' p4 branch (//depot/projects/vimage/…).

• Step 1.5 and step 2 are planned to take about 2 weeks each with step 3 being done at latest before November 1st preferably before EuroBSDCon2008. This would give us about the following deadlines:
  • Step 1.5: Sep 1st
  • Step 2: Sep 15th
  • Step 3: ideally Sep 30th, in case thing go bad and move, Oct 13, worst Halloween.
• We are going to reflect on this at the upcoming EuroBSDCon2008 DevSummit.

1. [DONE] Apply the V_ macro prefix:

• NOP (non functional) change.
• Invalidates most of the patches out there.

1.5. Further style changes (NOP) bringing in more infrastructure:

1. [DONE] Resolve file local statics with conflicting names.
2. Import INIT_VNET_* as NOPs along with net/vnet.h, netinet/vinet.h, netinet6/vinet6.h, netgraph/vnetgraph.h, …, move #defins from vimage.h to those header files, make the macros NOPs in vimage.h. Put things into present header files.
3. Import “V_SYSCTLs”, make them behave SYSCTLs w/o vimage for the moment. update man pages.

2. Move the globals into a struct (g.v):

• Make sure the macros from step 1.5 are no longer NOPs.
• Bring in per module constructor functions.
• Move globals to structs.
• Make it possible to revert to globals for the commit and remove that shortly afterwards.

3. Apply virtualization (g→v):

• Add destructor functions.
• Add iterator macros.
• Per vimage pointers.
• INIT_VNET_INET would start doing something.

4. Finish more subsystems

• That is where your hands are needed and when you need to know what to do!
• Reorder variables in structs for cache locality.
• James Gritton's management framework.
  • User space application will be called jail(8). There will be a switch to flip the behavior from classical to new style.
  • Syscalls will be called jail_*.
  • In kernel structure will be called whatever they will be called. He who implements decides.
  • We want an explicit opt-in for any recursive changes.

You can use nm(1) on your kernel/object files. Use something like: nm *.o | awk '/ [bd] / { print $3 }' | sort | uniq -c | awk '{ if ($1 > 1) { print $0 } }'

1. Turn off INVARIANTS/INVARIANT_SUPPORT (since this records line numbers everywhere).

cd /sys/<arch>/conf
make LINT

Generate a VIMAGE_LINT (or an equivalent for GENERIC):

include         LINT
ident           VIMAGE-LINT
nooptions       INVARIANTS
nooptions       INVARIANT_SUPPORT

1. Bump __FreeBSD_version in unpatched tree (to match the patched tree) if needed.
2. __LINE__ appears in some .o files via macros, so whitespace needs to be added to unpatched files that are modified by e.g. addition of #include <sys/vimage.h>. Prepending blank lines to the top of the file should be good enough at least for most of them. If it is not, for example at a later stage, you need to prepare a 'counterpatch' that adds the same whitespace to the plain kernel as the vimage diff adds new lines to its tree. You may also need to add some temporary whitespace to the vimage tree in case you remove lines and do not add new ones back.
3. Build kernel once with the whitespace and once with the real patch.

config VIMAGE_LINT
cd ../compile/VIMAGE_LINT
make cleandepend __MAKE_CONF=/dev/null SRCCONF=/dev/null
make depend __MAKE_CONF=/dev/null SRCCONF=/dev/null
make __MAKE_CONF=/dev/null SRCCONF=/dev/null

or run following from src/

make buildkernel KERNCONF=VIMAGE_LINT __MAKE_CONF=/dev/null SRCCONF=/dev/null

1. Strip the *.o files in sys/<arch>/compile/VIMAGE_LINT or obj/sys/VIMAGE_LINT (subsitute the . argument to find in that case):

find . -type f -name "*.o" -print | xargs strip

1, Run md5 on each .o file and save the output depending on the tree.

find . -type f -name "*.o" -print | xargs m5 | sort > /tmp/VIMAGE_LINT.{plain,vimage}

1. Comparing the md5 output of both runs, checksums of *.o should only differ in vers.o, which records the timestamp of the kernel build.

diff -u /tmp/VIMAGE_LINT.plain /tmp/VIMAGE_LINT.vimage

• …
• …
• …

• Jail is useful for name for virtualization for “branding”
• You have jail safe file systems, but you should be able to only allow some file systems to be mounted in a jail
• Perhaps only allow jexec instead of jattach since jattach is “ugly” and leave things from outside
• You want jails without processes hanging around e.g. if “jails” which just does forwarding
• Perhaps keep jail(8) for compat, and new jcreate(8) for advanced fancy jails. jail(8) is just a jcreate(8) wrapper

http://people.freebsd.org/~bz/200805DevSummit/#vimageII