Toolchain, Build and Packaging of Base

BSDCam session, 1pm - 2:45pm, August 26.

• LLVM Instrumentation
• Configuration
  • Modules by default
  • Cleanup
• Static Analysis (operational issues)
• Tinderboxes
• Incremental Build
• Packaging Base
• External Toolchain Status
• Removing GPL from the toolchain
• Building old FreeBSD on Head (and vice versa)

Traditional build turns a .c file into a .o file and then link the .o files. In the LLVM IR instrumentation world, you turn .c files into .ll/bc files, modify them, link them (then run link-time optimisation), then generate .o files, then link. In SOAAP, the instrumentation happens after the IR-link phase, for TESLA we run static analysis of the .c files, then instrument individual files.

There's no good way in make to express different routes from .c to .o. We want to be able to compose many of these things.

Things like TESLA and SOAAP require annotations, which are macros and can be compiled out. Things like the clang static analyser or KINT just check C-language semantics.

Some things in ports are also interesting to check, so it would be nice if we had a ${CC} and ${LD} that invoked some of these as part of a normal build. Some check API usage, including Apple kernel API checks, which could have some things.

How much annotation are we willing to accept in the kernel for documenting invariants and so on? Warren reminds us that having specification close to the source code is very useful. Robert proposes that we're willing to accept any changes that provide specs into the code, as long as we can run the checks that verify them.

Configuration should be done by setting variables, not by executable make rules.

Kernel builds are currently very special - a generated rule for every .c file - but the kernel is just an ELF image.

Can we do the build-as-non-root and then boot in QEMU and so on to run regression tests? Should the tinderboxes run the static analyser?

DES is going to add scheduling to tinderboxes. For example, when secteam wants to test a security bug fix they will get to interrupt normal builds and build things.

Tinderbox should become a generic infrastructure for doing builds, testing, and static analysis. DES volunteers to own this gradual progression, but requires hosting for some machines.

Touching certain headers forces a full rebuild. Our dependency tracking is hard because some headers are in src/include.

Probably a better discussion for Malta?

Current version allows ${XCC}, ${XLD} and so on to replace compiler, binutils, and so on. This is important because new clang will need building with clang soon (clang 3.4 uses C++11 so won't build with gcc shipped in 9.x).

Currently no support for using X* to build the toolchain. This is useful for bootstrapping and so on.

How useful is this for building FreeBSD on Linux? You'll need to build a new LLVM (unless you have one in packages) and a binutils that targets FreeBSD. Requires bmake and so on as well - someone must enumerate everything that we need for the build.

Our GCC and libstdc++ are from 2007. They predate the current C/C++ standards and are a pain to maintain. Big C++ ports like LibreOffice don't work with our libstdc++. We really, really don't want to be supporting a 2007 release in 2017 (by which time C++14 will be out) when 10.x is EOL'd.

Proposed plan:

• Remove gcc / libstdc++ from 10.0 on platforms where clang is cc
• Migrate all platforms to either clang or an external toolchain by 11
• Fix remaining ports to use lang/gcc to build (mostly done?)

No objections (except lots of people on mailing lists)…

Brooks says: Yes. Umm. Probably work if you use fmake from ports… Bapt says it works. Or, at least, did. Documentation is probably required - the switch to bmake broke it (a yacc update broke it previously). Is this something we want to be checking automatically? We don't have automatic testing and we don't promise that it will work, but it's quite nice when it works.