vimage:notes200808devsummit

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision		 Previous revision
vimage:notes200808devsummit [2008/08/23 15:45] – Add howto identify file static globals with collliding names; Submitted by: kris (magic changed by me) BjoernZeebvimage:notes200808devsummit [2014/10/12 22:42] (current) – renamed from Image/Notes200808DevSummit CraigRodrigues
Line 16: Line 16:
   * We are going to reflect on this at the upcoming EuroBSDCon2008 [[/DevSummit|DevSummit]].   * We are going to reflect on this at the upcoming EuroBSDCon2008 [[/DevSummit|DevSummit]].
  
 +*REALITY* Step 2 was committed DECEMBER 10
 ==== The 4.5 steps ==== ==== The 4.5 steps ====
  
Line 22: Line 23:
   * Invalidates most of the patches out there.   * Invalidates most of the patches out there.
  
-1.5. Further style changes (NOP) bringing in more infrastructure:+1.5. [DONE] Further style changes (NOP) bringing in more infrastructure:
   - [DONE] Resolve file local statics with conflicting names.   - [DONE] Resolve file local statics with conflicting names.
-  - Import INIT_VNET_* as NOPs along with net/vnet.h, netinet/vinet.h, netinet6/vinet6.h, netgraph/vnetgraph.h, ..., move #defins from vimage.h to those header files, make the macros NOPs in vimage.h. Put things into present header files. +  - [DONE] Import INIT_VNET_* as NOPs along with net/vnet.h, netinet/vinet.h, netinet6/vinet6.h, netgraph/vnetgraph.h, ..., move #defins from vimage.h to those header files, make the macros NOPs in vimage.h. Put things into present header files. 
-  - Import "V_SYSCTLs", make them behave SYSCTLs w/o vimage for the moment. update man pages.+  - [DONE]Import "V_SYSCTLs", make them behave SYSCTLs w/o vimage for the moment. update man pages.
  
-2. Move the globals into a struct (g.v):+2.[DONE]  Move the globals into a struct (g.v):
   * Make sure the macros from step 1.5 are no longer NOPs.   * Make sure the macros from step 1.5 are no longer NOPs.
   * Bring in per module constructor functions.   * Bring in per module constructor functions.
Line 47: Line 48:
     * In kernel structure will be called whatever they will be called. He who implements decides.     * In kernel structure will be called whatever they will be called. He who implements decides.
     * We want an explicit opt-in for any recursive changes.     * We want an explicit opt-in for any recursive changes.
 +
 +===== Progress report   Jan 31 09 =====
 +
 +I (julian) have been testing going backwards and forwards using the VIMAGE_GLOBALS option and have see no significant performance changes between the original code and the same code compiled with a set of global structures. Looking at the generated code I can see that the compiler is resolving the locations of each field in the structures and the resulting instructions are almost identical with only the addesses being differnet.  This gives me confidence that when VIMAGE is compiled out the performance numbers are in practice, as we had postulated, identical to teh original numbers. This makes VIMAGE a much more attractive feature if those who do need the performance and don't need multiple contexts can be confident we will not be hurting them.
  
 ===== HOWTO identify file static global variables with colliding names ===== ===== HOWTO identify file static global variables with colliding names =====
  
-You can use nm(1) to do this on you kernel/object files. Use something like: +You can use nm(1) on your kernel/object files. Use something like:
-''%%nm *.o | awk '/ [bd] / { print $3 }' | sort | uniq -c | awk '{ if ($1 > 1) { print $0 } }'%%''+
  
-===== HOWTO verify that the pure style changes are all right =====+<code> 
 +nm *.o | awk '/ [bd] / { print $3 }' | sort | uniq -c | awk '{ if ($1 > 1) { print $0 } }' 
 +</code> 
 + 
 +===== HOWTO verify that the pure style changes are all right, aka. The MD5 Dance ===== 
 + 
 +  - Turn off INVARIANTS/INVARIANT_SUPPORT (since this records line numbers everywhere). 
 + 
 +<code> 
 +cd /sys/<arch>/conf 
 +make LINT 
 +</code> 
 + 
 +Generate a VIMAGE_LINT (or an equivalent for GENERIC): 
 + 
 +<code> 
 +include         LINT 
 +ident           VIMAGE-LINT 
 +nooptions       INVARIANTS 
 +nooptions       INVARIANT_SUPPORT 
 +</code> 
 + 
 +  - Bump ''%%__FreeBSD_version%%'' in unpatched tree (to match the patched tree) if needed. 
 +  - ''%%__LINE__%%'' appears in some .o files via macros, so whitespace needs to be added to unpatched files that are modified by e.g. addition of #include %%<%%sys/vimage.h>. Prepending blank lines to the top of the file should be good enough at least for most of them. If it is not, for example at a later stage, you need to prepare a 'counterpatch' that adds the same whitespace to the plain kernel as the vimage diff adds new lines to its tree. You may also need to add some temporary whitespace to the vimage tree in case you remove lines and do not add new ones back. To avoid to much empty line shuffling you may want to change the lock line number to a static like: 
 + 
 +<code> 
 +--- sys/sys/lock.h.orig      2008-09-25 22:21:06.000000000 +0000 
 ++++ sys/sys/lock.h   2008-09-25 22:21:49.000000000 +0000 
 +@@ -132,4 +132,4 @@ struct lock_class { 
 + #if LOCK_DEBUG > 0 
 + #define        LOCK_FILE       __FILE__ 
 +-#define        LOCK_LINE       __LINE__ 
 ++#define        LOCK_LINE       0xc0defeed 
 + #else 
 +</code> 
 + 
 +  - Build kernel once with the whitespace and once with the real patch. 
 + 
 +<code> 
 +config VIMAGE_LINT 
 +cd ../compile/VIMAGE_LINT 
 +make -s cleandepend __MAKE_CONF=/dev/null SRCCONF=/dev/null 
 +make -s depend __MAKE_CONF=/dev/null SRCCONF=/dev/null 
 +make -s __MAKE_CONF=/dev/null SRCCONF=/dev/null 
 +</code> 
 + 
 +or run following from src/ 
 + 
 +<code> 
 +make buildkernel KERNCONF=VIMAGE_LINT __MAKE_CONF=/dev/null SRCCONF=/dev/null 
 +</code> 
 + 
 +  - Strip the *.o files in sys/%%<%%arch>/compile/VIMAGE_LINT or obj/sys/VIMAGE_LINT (subsitute the . argument to find in that case): 
 + 
 +<code> 
 +find . -type f -name "*.o" -print | xargs -n 1 strip 
 +</code> 
 + 
 +  - Run md5 on each .o file and save the output depending on the tree. 
 + 
 +<code> 
 +find . -type f -name "*.o" -print | xargs md5 | sort > /tmp/VIMAGE_LINT.{plain,vimage} 
 +</code>
  
-  - Turn off INVARIANTS/INVARIANT_SUPPORT (since this records line numbers ever ywhere). 
-  - Bump ''%%__FreeBSD_version%%'' in unpatched tree (to match the patched tree). 
-  - ''%%__LINE__%%'' appears in some .o files via macros, so whitespace needs to be added to unpatched files that are modified by e.g. addition of #include %%<%%sys/vimage.h>. Prepending blank lines to the top of the file should be good enough at least for most of them. 
-  - Build kernel once with the whitespace and once with the real patch. Strip the *.o files and run md5 on each file in obj/sys/KERNANEM and save the output to a file. 
   - Comparing the md5 output of both runs, checksums of *.o should only differ in vers.o, which records the timestamp of the kernel build.   - Comparing the md5 output of both runs, checksums of *.o should only differ in vers.o, which records the timestamp of the kernel build.
 +
 +<code>
 +diff -u /tmp/VIMAGE_LINT.plain /tmp/VIMAGE_LINT.vimage
 +</code>
  
 ===== Unvirtualized (or not finished) parts of the network stack ===== ===== Unvirtualized (or not finished) parts of the network stack =====
  • vimage/notes200808devsummit.1219506333.txt.gz
  • Last modified: 2008/08/23 15:45
  • by BjoernZeeb