====== Package Building for the FreeBSD.org Cluster ======

===== The Big Picture =====
The basic idea is that we need to make certain that when port upgrades happen on cluster machines, we need to be able to somewhat reasonably guarantee that the ports being upgraded are compatible with the existing ports, and nothing is missed.
poudriere handles this quite nicely and reliably.

===== Poudriere =====
==== Local Setup ====
The cluster package build repository is separate from the default clusteradm subversion repository.  It is located at: ''%%svn+ssh://pb00.FreeBSD.org/var/svn/pkgbuild/keg%%''

This repository contains the ///usr/local/etc/poudriere.d/// directory on pb00.%%FreeBSD%%.org.

This directory contains all the magic needed for %%FreeBSD%%.org cluster machine package builds as noted in the Local Configuration and Local Scripts sections.

As we already know, pre-built packages were never one-size-fits-all.  What I have tended to do when setting up such package build systems is to do package builds based on either a per-machine basis, or based on a set of machine roles.  Both of these have their pros and cons:

  * With per-role builds, time spent building packages is reduced, however you lose flexibility with machine-independent configuration.
  * With per-machine builds, more time is spent building (potentially duplicate) packages, however each can have independent configurations.

My preference has always been per-machine package builds.

==== Local Configurations ====
///usr/local/etc/poudriere.d/// is set up according to what poudriere expects to find, and where it will look.  These files are all under revision control, for the inevitable case of needing to look back to see what changed with a port build options, or needing to revert an entire set of changed options.

Each build has its own:
  * //options/// directory
  * //make.conf// file
  * //pkglist.txt// file

These are linked to a particular build by prefixing the jail name and target machine name, for example:
  * //10-amd64-pb00-make.conf//
  * //10-amd64-pb00-options///
  * //10-amd64-pb00-pkglist.txt//

In this case, //10-amd64// is the name of the jail used for the port build and //pb00// is the %%FreeBSD%%.org cluster machine that will be the consumer of the resulting packages.

The idea for jail names is effectively //${BRANCH_VERSION}-${ARCH}//

The idea for the "sets" poudriere will build is effectively the machine hostname, less "freebsd.org".  In cases where there are multiple machines per site, or a machine hostname is site specific, the "." is replaced with "-", for example: //cook-isc//

==== Local Scripts ====
There are two scripts right now to help automate the maintenance of package builds:

  * //clusterbuild.sh//: Runs the package build for all configured systems.
  * //regen.sh//: Updates the ports tree and, if TERM is set, runs **make config** for the ports in the //pkglist.txt// files (and dependencies).

There are a few configuration files that are used by the scripts:
  * //jails.conf//: List of jails to map to build "sets"
  * //sets-${jail}.conf//: List of target machines to build using a particular build jail.

There is also a //pkgbootstrap.sh// script that is intended to fetch the sample //pkg.conf// file and update as necessary, as well as the public //pkg.cert// file from pbdist.%%FreeBSD%%.org, and put them into the correct locations.  After fetching the files, the SHA256 is calculated and verified against known values.

===== Package Distribution =====
==== Local Setup ====
pb00.%%FreeBSD%%.org has a single jail for web services, pbdist.%%FreeBSD%%.org.
Its web root directory is a read-only nullfs(5) mount from pb00.%%FreeBSD%%.org, containing an empty //index.html// file, a sample //pkg.conf// file, and the public certificate file (//pkg.cert//) for the SSL key used to sign the packages.

Note:  If ever changing the //pkg.cert// (or the signing key) or //pkg.conf//, the SHA256 sums must be updated in //pkgbootstrap.sh//.

==== Package Repository Creation ====
poudriere will automatically create the //repo.txz// using the signing key after a package build completes.  In general, there should be nothing to do here, but if manually signing a package build is needed, this can be done as:

<code>
# pkg repo /poudrieredata/data/packages/${JAIL}-cluster-${TARGET_HOST}/ /root/ssl/pkg.key
</code>

===== Pkgng =====
==== Cluster Server Bootstrapping ====
A "bootstrapping a system for cluster pkgng builds" example:

<code>
root@pb00:~ # fetch http://pbdist.freebsd.org/pkgbootstrap.sh
pkgbootstrap.sh                               100% of 1465  B 1346 kBps 00m00s
root@pb00:~ # sh ./pkgbootstrap.sh 
/tmp/pkg.cert                                 100% of 1491  B 4282 kBps 00m00s
/tmp/pkg.conf                                 100% of  479  B 1480 kBps 00m00s
SHA256 for pkg.conf and pkg.cert match.
Creating /etc/ssl/ directory for pkg.cert...
Installing the public certificate file...
Done.
Running FreeBSD:10 amd64
This hostname is: pb00
Installing pkg.conf...
Done.
Cleaning up.
Done.
root@pb00:~ # grep ^PACKAGESITE /usr/local/etc/pkg.conf
PACKAGESITE         : https://pbdist.FreeBSD.org/10-amd64-cluster-pb00
root@pb00:~ # pkg update
Updating repository catalogue
digests.txz
100% 2280     2.2KB/s   2.2KB/s   00:00    
packagesite.txz
100% 7580     7.4KB/s   7.4KB/s   00:00    
Incremental update completed, 0 packages processed:
0 packages updated, 0 removed and 21 added.
</code>