New kernel configuration system

Wojciech A. Koszek
wkoszek@FreeBSD.org
http://FreeBSD.czest.pl/~wkoszek/

Current kernel configuration system has its roots in old BSD days. It's similar to sysinstall – it does the job, but the FreeBSD world would be a bit better with having different/better config(8). The goal of the page is to list config(8) limitations and problems hitting us in FreeBSD-9 times. This may be a root of “config(8) suck less” project.

This page contains some ideas about config(8) in random
order. Some are outdated, but later ones are relatively
new. Keep that in mind while studying this page.

Current implementation of kernel configuration tool has a lot of problems and shortcomings that make a life of typical FreeBSD user much hard. Below I try to address them:

• build all .c files just once, and use object files to create the kernel and modules
  • right now we actually compile kernel twice–for the kernel and modules separately
• support dependencies – right now when some option is missing, you'll get link time error
  • try to keep nice structure of conf/files, since it's relatively easy to insert file there
  • this is stricly related with entry above
  • do NOT block advanced users from providing their own .o (custom) files if they want it (Peter)
• support out-of-tree module build
  • or at least be smart enough to let us to remove (or limit) src/sys/modules; stay away from empty opt_*.h recreation.
• support “kernel will accept this KLD for sure and won't panic”:
  • compile the kernel against particular kernel – if the kernel has INVARIANTS, KLD should be compiled with INVARIANTS as well.

• DSL implemented:
  • nooption nosomething
  • enable disable
  • …and other don't work well.

• be able to configure cross-arch kernels

• include whole src/sys stuff (imp@)
  • include only .c files, from which kernel has been built (des@)

• INCLUDE_CONFIG_FILE should include comments in the generated output

Proposed syntax for the new kernel configurator a.k.a config2:

component kernel {
         files {
                kern_main.c,
                kern_exec.c,
                [...]
         };
};

component md {
         files {
                md.c
         };
};

component mii {
          files {
                 mii.c
          };
};

component acphy {
          files {
                   dev/mii/acphy.c
          };
          depends {
                   mii
          };  
};

component amphy {
          files {
                 dev/mii/amphy.c
          };
          depends {
                   mii
          }; 
};

component miibus {
          embed {
                   mii, amphy, acphy
          };
};

component em {
          compile_with "${NORMAL_C} -I$S/dev/e1000";    // compile_with can take block or a string
          files {
                  dev/e1000/if_em.c
                  dev/e1000/if_lem.c
                  dev/e1000/if_igb.c
                  dev/e1000/e1000_80003es2lan.c
                  dev/e1000/e1000_82540.c

                  [...]
           };
};

component ipwibssfw {
           compile_with {                            // compile_with as a block + possibility of line breaking
                   ${AWK} -f $S/tools/fw_stub.awk ipw_ibss.fw:ipw_ibss:130
                       -lintel_ipw -mipw_ibss -c${.TARGET}
           };

           clean "ipwibssfw.c";                      // I think we could either accept {} block or a string for single statements
                  
           no_implicit_rule;
           before_depend;
           local (XXWKOSZEK: need to figure out what that is)
        
};

Keyword component would define what functionality is provided.

Keyword files would be used to start a block with c files enlisted.

Keyword embed just picks the content of previous blocks and merges them in a current block.

Keyword compile_with sets the generation command

Keywords “no_implicit_rule”, “before_depend”, “local” stay as they were.

static kernel;                 // this could be implicit or not
static md;                     

makeoptions "DEBUG=-g"

options         SCHED_ULE               # Some options would stay
options         PREEMPTION              #
options         INET                    #
options         INET6                   #

options         NFSCL                   # but some COULD probably be just
options         NFSD                    # included as "static nfscl"
options         NFSLOCKD                # Why would NFS be included as an "options?"

With “components” and dependencies, src/sys/modules could go away, so modules would be built 'for free'. And kernel configuration would only enlist things necessary for components that we want to link staticly.

In my view, we should use the same parsing for src/sys/conf/files' and src/sys/<arch>/conf/KERNCONF, so that

cat src/sys/conf/files src/sys/amd64/conf/KERNCONF > kconf
config2 -f kconf

is possible. The reason is that for development, I'd like to have kern_something.c included from my KERNCONF and the whole case with parsing would just be simplified.

The idea which Peter prototyped already in the past was to use config(8) to dump configuration data in the form of easily parseable tables, which would later get consumed by scripts.

It would be scripts that generate Makefiles and other metadata files necessary by a boot process.

This is for the case where we want to preserve as much as the existing file format as possible.

Lets forget about config(8) for now. Let config2(8) be a new configuration tool.

We meet a case where:

- newconfig(8)-v1 operates on GENERIC-v1 and generates only Makefile for kernel and modules.

- BadCommiter brings “option TWICE_AS_FAST”, which makes your kernel compile and work twice as fast – it is based on kernel and newconfig(8) new features, new file generation, new parsing etc..

- newconfig(8) gets its version number bumped and becomes newconfig(8)-v2. TOGETHER with newconfig(8), configuration file becomes GENERIC-v2. Both Stuff gets commited.

There are 4 cases:

Everything works – user didn't update his /usr/src after BadCommiter's commit; newconfig(8) and GENERIC versions match. User doesn't get new features.

User tries to use old newconfig(8) with new GENERIC.

Version between the tool and a file doesn't match. User could get an error message about a need to recompile newconfig(8) or a chance to ignore the kernel option entirely.

I'd like to solve it so that no recompile is necessary. BadCommiter can, in theory, bring enough meta data for newconfig(8) so that the tool will know how to handle this case.

This is “OK” case – new tool should be able to process old configuration files. Have we ever tried to remove config(8) functionality, so that this problem should be addressed as well?

User updates his /usr/src; he reads UPDATING and sees a message about newconfig(8) changes. He updates newconfig(8) and reconfigures his kernel with new options.

This case is 'the correct one', but is problematic for developers, who tend to keep dozens of src/sys/… tree around, since newconfig(8) has to be updated for them in order to get their kernels to compile.

There are several solutions for the problem.

The first one is inspired by mosty pkgsrc, which builds bmake as a first stage of its bootstrap procedure. bmake is used later to perform the whole pkgsrc bootstrap tasks. Couldn't we repeat this behavior?

No. At least not directly.

Why?

Because our config(8) sources live in src/usr.sbin, and since kernel developers don't want bring whole /usr/src on their development machines, they just copy src/sys/... and start having it.

How could this get solved?

My first idea is quite simple, a bit non-standard but should work pretty well, if some caution is taken.

Why not to bring config(8) into src/sys/config/… instead of src/usr.sbin?

This of course doesn't stop us from having a configuration tool in /usr/sbin. It just means sources would live somewhere else. We already depend on kernel bits in order to build world (let me point to sbuf as being an example).

What would it give us?

We could just teach newconfig(8) to enter src/sys/config/…, rebuild its newer brother and use it later – each time version mismatch is detected.

The installworld target shouldn't get more complicated – it's a matter of changing paths

Wouldn't it help us to solve more (all?) configuration problems?

Any simpler ideas?