This page is obsolete; CI testing is now done via Jenkins

This page is part of the TestSuite project and provides a description of the testing cluster offered by the FreeBSD project.

The test suite cluster is composed of various machines (nodes) with pretty much identical configurations. This is on purpose so that the failure of a single machine does not leave the whole cluster unusable.

Each node has the following:

• A web server (nginx) to serve the test results for that node as well as the index page for the test suite, with pointers to all other nodes.

• A background loop that iterates over various release configurations. These configurations are specific to each host so that every host can build different platform/release combinations.

• An iteration process within the loop that fetches the FreeBSD source tree for a specific branch, issues a full release build, sets up a virtual machine with the new release, boots the virtual machine, runs the test suite inside the virtual machine, extracts the test suite results for the run and generates HTML reports for that run.

• A background cron job that fetches all configuration files and scripts and self-updates the host.

This whole process is controlled by the autotest scripts, which are further described in their README file.

• kyua1: Master node serving the frontpage of the test suite. This node also acts as a “canary” for autotest itself and only builds amd64 HEAD in a fast paced (NO_CLEAN=yes) mode for quicker turnaround of changes to the autotest code. Because of the unclean builds, it's possible for this machine to report false negatives for tests so please compare the results with other machines first.

• kyua2: Slave node building i386 HEAD and stable/10. Also serves the frontpage.

• kyua3: Slave node building amd64 HEAD and stable/10. Also serves the frontpage.

Any changes to these machines should be deployed via changes to their configuration files in the autotest/node/configs directory. No manual intervention on the host itself should be necessary.

The procedure is the following:

1. Apply the desired change to the configuration and/or the autotest code.

1. Test the changes locally if at all possible. Using ./setup -X <step> is an option to execute individual parts of the autotest systme.

1. Commit the changes and get the revision number R.

1. The kyua1/host.conf file (canary) is set to fetch autotest from HEAD so within an hour, your changes will be pulled up by kyua1 and a few minutes/hours later, a testing iteration will complete with your changes.

1. If the iteration finished successfully, proceed; otherwise fix, commit and wait again.

1. Update AUTOTEST_REVISION in the host.conf of all other nodes to match R. This will push this specific autotest release (and nothing newer) to those machines.

1. Commit the changes. All should be good and within a few hours your modifications should propagate to all other machines. Note that because regular workers perform clean builds, the latency from push to results is quite long.