200808 DevSummit Cambridge

• All commits go through vimage-commit2 branch step by step. Julian and Marko bringing in the basic set and reviewers are going to finalize and produce the patch.
• RobertWatson, BrooksDavis, KrisKennaway volunteered to also review patches.
• BjoernZeeb volunteered to keep an eye on the schedule, progress, …
• After step 2 we are going to provide a clean p4 branch people can use to branch their own working area off to virtualize other subsystems like vimage.sysvipc. If you want to work on anything before this use the 'vimage' p4 branch (//depot/projects/vimage/…).

• Step 1.5 and step 2 are planned to take about 2 weeks each with step 3 being done at latest before November 1st preferably before EuroBSDCon2008. This would give us about the following deadlines:
  • Step 1.5: Sep 1st
  • Step 2: Sep 15th
  • Step 3: ideally Sep 30th, in case thing go bad and move, Oct 13, worst Halloween.
• We are going to reflect on this at the upcoming EuroBSDCon2008 DevSummit.

*REALITY* Step 2 was committed DECEMBER 10

1. [DONE] Apply the V_ macro prefix:

• NOP (non functional) change.
• Invalidates most of the patches out there.

1.5. [DONE] Further style changes (NOP) bringing in more infrastructure:

1. [DONE] Resolve file local statics with conflicting names.
2. [DONE] Import INIT_VNET_* as NOPs along with net/vnet.h, netinet/vinet.h, netinet6/vinet6.h, netgraph/vnetgraph.h, …, move #defins from vimage.h to those header files, make the macros NOPs in vimage.h. Put things into present header files.
3. [DONE]Import “V_SYSCTLs”, make them behave SYSCTLs w/o vimage for the moment. update man pages.

2.[DONE] Move the globals into a struct (g.v):

• Make sure the macros from step 1.5 are no longer NOPs.
• Bring in per module constructor functions.
• Move globals to structs.
• Make it possible to revert to globals for the commit and remove that shortly afterwards.

3. Apply virtualization (g→v):

• Add destructor functions.
• Add iterator macros.
• Per vimage pointers.
• INIT_VNET_INET would start doing something.

4. Finish more subsystems

• That is where your hands are needed and when you need to know what to do!
• Reorder variables in structs for cache locality.
• James Gritton's management framework.
  • User space application will be called jail(8). There will be a switch to flip the behavior from classical to new style.
  • Syscalls will be called jail_*.
  • In kernel structure will be called whatever they will be called. He who implements decides.
  • We want an explicit opt-in for any recursive changes.

I (julian) have been testing going backwards and forwards using the VIMAGE_GLOBALS option and have see no significant performance changes between the original code and the same code compiled with a set of global structures. Looking at the generated code I can see that the compiler is resolving the locations of each field in the structures and the resulting instructions are almost identical with only the addesses being differnet. This gives me confidence that when VIMAGE is compiled out the performance numbers are in practice, as we had postulated, identical to teh original numbers. This makes VIMAGE a much more attractive feature if those who do need the performance and don't need multiple contexts can be confident we will not be hurting them.

You can use nm(1) on your kernel/object files. Use something like:

nm *.o | awk '/ [bd] / { print $3 }' | sort | uniq -c | awk '{ if ($1 > 1) { print $0 } }'

1. Turn off INVARIANTS/INVARIANT_SUPPORT (since this records line numbers everywhere).

cd /sys/<arch>/conf
make LINT

Generate a VIMAGE_LINT (or an equivalent for GENERIC):

include         LINT
ident           VIMAGE-LINT
nooptions       INVARIANTS
nooptions       INVARIANT_SUPPORT

1. Bump __FreeBSD_version in unpatched tree (to match the patched tree) if needed.
2. __LINE__ appears in some .o files via macros, so whitespace needs to be added to unpatched files that are modified by e.g. addition of #include <sys/vimage.h>. Prepending blank lines to the top of the file should be good enough at least for most of them. If it is not, for example at a later stage, you need to prepare a 'counterpatch' that adds the same whitespace to the plain kernel as the vimage diff adds new lines to its tree. You may also need to add some temporary whitespace to the vimage tree in case you remove lines and do not add new ones back. To avoid to much empty line shuffling you may want to change the lock line number to a static like:

--- sys/sys/lock.h.orig      2008-09-25 22:21:06.000000000 +0000
+++ sys/sys/lock.h   2008-09-25 22:21:49.000000000 +0000
@@ -132,4 +132,4 @@ struct lock_class {
 #if LOCK_DEBUG > 0
 #define        LOCK_FILE       __FILE__
-#define        LOCK_LINE       __LINE__
+#define        LOCK_LINE       0xc0defeed
 #else

1. Build kernel once with the whitespace and once with the real patch.

config VIMAGE_LINT
cd ../compile/VIMAGE_LINT
make -s cleandepend __MAKE_CONF=/dev/null SRCCONF=/dev/null
make -s depend __MAKE_CONF=/dev/null SRCCONF=/dev/null
make -s __MAKE_CONF=/dev/null SRCCONF=/dev/null

or run following from src/

make buildkernel KERNCONF=VIMAGE_LINT __MAKE_CONF=/dev/null SRCCONF=/dev/null

1. Strip the *.o files in sys/<arch>/compile/VIMAGE_LINT or obj/sys/VIMAGE_LINT (subsitute the . argument to find in that case):

find . -type f -name "*.o" -print | xargs -n 1 strip

1. Run md5 on each .o file and save the output depending on the tree.

find . -type f -name "*.o" -print | xargs md5 | sort > /tmp/VIMAGE_LINT.{plain,vimage}

1. Comparing the md5 output of both runs, checksums of *.o should only differ in vers.o, which records the timestamp of the kernel build.

diff -u /tmp/VIMAGE_LINT.plain /tmp/VIMAGE_LINT.vimage

• …
• …
• …

• Jail is useful for name for virtualization for “branding”
• You have jail safe file systems, but you should be able to only allow some file systems to be mounted in a jail
• Perhaps only allow jexec instead of jattach since jattach is “ugly” and leave things from outside
• You want jails without processes hanging around e.g. if “jails” which just does forwarding
• Perhaps keep jail(8) for compat, and new jcreate(8) for advanced fancy jails. jail(8) is just a jcreate(8) wrapper

http://people.freebsd.org/~bz/200805DevSummit/#vimageII