Differences

This shows you the differences between two versions of the page.

--- tinderbox [2008/01/26 22:33] – fix typo MarkLinimon
+++ tinderbox [2022/10/06 02:57] (current) – Chase page rename KubilayKocak
@@ Line 1: / Line 1: @@
-====== Tinderbox ======
+===== Tinderbox =====
-This article describes the %%FreeBSD%% source tinderbox.
+This article describes the %%FreeBSD%% source tinderbox.  To learn more about what it does, take a look at the [[http://tinderbox.freebsd.org/|current FreeBSD src build status]].
-===== Overview =====
 The tinderbox ecosystem consists of:
@@ Line 13: / Line 11: @@
   * A web server that keeps a complete set of tinderbox logs and displays an up-to-date summary
-The tinderbox scripts were developed (and are still maintained) by %%DagErlingSm%%ørgrav.  The original version was a shell script that ran on ''%%bento.freebsd.org%%'' once a day, after one too many Alpha-breaking commits.  It was replaced with the current Perl implementation in early 2003.
+===== History =====
-All scripts and configuration files are kept in [http://www.freebsd.org/cgi/cvsweb.cgi/projects/tinderbox/ projects/tinderbox/].
+tinderbox scripts were developed and are maintained by %%DagErlingSm%%ørgrav. The original version was a shell script that ran on ''%%bento.freebsd.org%%'' once a day, after one Alpha-breaking commit too many.  It was replaced with the current Perl implementation in early 2003.
-===== The tinderbox script =====
+All scripts and configuration files are kept in [[http://svnweb.freebsd.org/base/user/des/tinderbox/|base/user/des/tinderbox]].
+===== Dashboard & Build Status =====
+**Note:** The service described here is no longer operational and now redirects to the %%FreeBSD%% [[Jenkins]] Continuous Integration service: [[https://ci.freebsd.org/tinderbox|Tinderbox-style view]].
+Summaries and logs from the official build servers are made available online at [[http://tinderbox.freebsd.org/]].
+Currently, this site is hosted by %%DagErlingSm%%ørgrav. The site is set up as follows:
+  * A cron job checks the build servers at regular intervals and downloads any new log files using ''%%rsync%%''.
+  * [[Apache]] is set up to use ''%%index.cgi%%'' as ''%%DirectoryIndex%%''.
+  * A [[https://www.varnish-cache.org/|Varnish]] instance in front of Apache ensures that ''%%index.cgi%%'' does not need to run more than once every two minutes.
+===== tinderbox script =====
 This section has not yet been written.  See the ''%%tinderbox(1)%%'' manual page for details.
-===== The tbmaster script =====
+===== tbmaster script =====
 This section has not yet been written.  See the ''%%tbmaster(1)%%'' manual page for details.
-===== The index.cgi script =====
+===== Configuration Files =====
 This section has not yet been written.
-===== Configuration files =====
+===== index.cgi script =====
-This section has not yet been written.
+The ''%%index.cgi%%'' script generates the HTML summary.  Although intended to be used as a CGI script (as the name indicates), it can also be run from the command line, or from a cron job, in which case it will look for logs in the directory in which the script itself is located.  It will automatically detect the context, and output HTTP headers only if run as a CGI script.
-===== Official Build Servers =====
+The script outputs strict XHTML 1.0 code and uses CSS for styling.
-The official tinderbox build servers are hosted by [http://www.sentex.ca/ Sentex Data Communications], who also host the [http://www.freebsd.org/projects/netperf/cluster.html %%FreeBSD%% netperf cluster].
+==== MAIN ====
-There are currently three build servers:
+The script's entry point is in the named block ''%%MAIN%%''.
-  * **freebsd-current.sentex.ca**: builds HEAD for every tier 1, 2 and 3 platform, currently manages a full run in about five hours.
+The script starts by attempting to verify that it is running on the official tinderbox web site.  If it isn't, the page it produces will include text to that effect, and a link to the official site.
-  * **freebsd-stable.sentex.ca**: builds RELENG_7 and still-supported 7.x branches for every tier 1 and 2 platform, currently manages a full run in about ten hours.
-  * **freebsd-legacy.sentex.ca**: builds RELENG_5, RELENG_6 and still-supported 5.x and 5.6 release branches for every tier 1 platform, currently manages a full run in about thirteen hours.
-The "tiers" used here are not quite the same as those [http://www.freebsd.org/doc/en_US.ISO8859-1/articles/committers-guide/archs.html defined in the Committers' Guide]:
+It then scans the log directory to get an inventory of configurations, branches and architectures for which log files exist, to avoid hard-coding a list into the script and potentially ending up with blank rows or columns.  This information is derived from the names of the log files, which match the following pattern:
-  * **Tier 1**: ''%%amd64%%'', ''%%i386/i386%%'', ''%%i386/pc98%%'', ''%%sparc64/sparc64%%''
+<code>
-  * **Tier 2**: ''%%ia64%%'', ''%%powerpc%%''
+tinderbox-$config-$branch-$arch-$machine.{brief,full}
-  * **Tier 3**: ''%%arm%%'', ''%%sparc64/sun4v%%''
+</code>
-  * **Tier 4**: ''%%mips%%'', ''%%s390%%''
-===== Official Summary Site =====
+The configurations used on the official tinderbox build servers are named for the branches they build; for instance, the ''%%releng_6%%'' configuration is used to build ''%%RELENG_6%%'' as well as all still-supported release branches.
-Summaries and logs from the official build servers are made available online at [[[http://tinderbox.freebsd.org/]]].
+Once this has been determined, the script generates a prologue, calls ''%%do_config()%%'' for each configuration, and generates an epilogue.
-Currently, this site is hosted by %%DagErlingSm%%ørgrav.  The site is set up as follows:
+==== do_config() ====
-  * A cron job checks the build servers at regular intervals and downloads any new log files using ''%%rsync%%''.
+The ''%%do_config()%%'' subroutine generates HTML code for a single tinderbox configuration.
-  * Apache is set up to use ''%%index.cgi%%'' as ''%%DirectoryIndex%%''.
-  * A [http://www.varnish-cache.org/ Varnish] instance in front of Apache ensures that ''%%index.cgi%%'' does not need to run more than once every two minutes.
-===== Room for Improvement =====
+It first generates a header row, then iterates over each branch build with this particular configuration, producing a single row of results for each in the following manner:
+  * For each architecture,
+    * For each machine within that architecture,
+      * If a //brief// log file exists:
+        * Call ''%%success()%%'' to determine the outcome of the build
+        * Output the modification date
+        * Output the size of the //brief// log file with a link to the file itself
+        * If a //full// log file also exists:
+          * Output the size of the //full// log file with a link to the file itself
+      * Otherwise:
+        * Output ''%%n/a%%''
+==== success() ====
+The ''%%success()%%'' subroutine scans a //brief// log file to determine whether the build was successful.  This is done quite simply by searching for the string "tinderbox run completed".
+==== Build Sorting ====
+Configurations and branches are sorted according to their //branch rank//, which is computed as follows:
+  * ''%%HEAD%%'' and ''%%CURRENT%%'' have rank 9999
+  * ''%%RELENG_x%%'' has rank xx99
+  * ''%%RELENG_x_y%%'' has rank xxyy
+In practical terms, this means that ''%%HEAD%%'' always ranks highest, and that ''%%RELENG%%'' branches are ranked in numerical order, with each ''%%STABLE%%'' branch ranking higher than the release branches forked off of it.  For instance, for %%FreeBSD%% 6, the order will be, from highest-ranking to lowest-ranking:
+  * ''%%RELENG_6%%'' (branch rank 699)
+  * ''%%RELENG_6_3%%'' (branch rank 603)
+  * ''%%RELENG_6_2%%'' (branch rank 602)
+  * ''%%RELENG_6_1%%'' (branch rank 601)
+  * ''%%RELENG_6_0%%'' (branch rank 600)
+==== Colors ====
+The summary script uses CSS to apply different colors to each cell in the table.  Successful builds are displayed with green text, failed builds with red text.  The color fades as time passes since the corresponding build, with every half-hour bringing it one step closer to gray.
+===== Official Build Servers =====
+The official tinderbox build servers are hosted by [[http://www.sentex.ca/|Sentex Data Communications]], who also host the [[RobertWatson/netperf|FreeBSD netperf cluster]].
+There are currently three build servers:
+  * **freebsd-current.sentex.ca**: builds
+    * ''%%HEAD%%'' for amd64, arm, i386, pc98, ia64, mips, powerpc, and sparc64; this takes about four hours.
+    * ''%%RELENG_8%%'' for amd64, arm, i386, pc98, ia64, mips, powerpc and sparc64; this takes about three and a half hours.
+  * **freebsd-stable.sentex.ca**: builds RELENG_7 and supported 7.x branches for amd64, i386, pc98, ia64, powerpc and sparc64; each branch takes about six hours.
+  * **freebsd-legacy.sentex.ca**: builds RELENG_6 and supported 6.x branches for amd64, i386, pc98 and sparc64; each branch takes about three hours.
+===== Future Work =====
 Suggested improvements:
-  * improve the logic ''%%tbmaster%%'' uses to extract the error message from the full log to put in the brief log.
+  * Improve the logic ''%%tbmaster%%'' uses to extract the error message from the full log to put in the brief log.
-  * when a build finishes, keep the source and object tree along with the log file, and filter the log file through a CGI script that adds markup linking every file name to the corresponding file in the source or object tree, and every compiler error or warning with the file and line it references.
+  * When a build finishes, keep the source and object tree along with the log file, and filter the log file through a CGI script that adds markup linking every file name to the corresponding file in the source or object tree, and every compiler error or warning with the file and line it references.
-  * instead of checking out a separate source tree for each platform, use a single source tree per iteration
+  * Instead of checking out a separate source tree for each platform, use a single source tree per iteration
-  * store results (though perhaps not full logs) in a database
+  * Store results (though perhaps not full logs) in a database
-  * implement a better build scheduling algorithm which
+  * Implement a better build scheduling algorithm which
     * avoids building branches that haven't changed since the last build
     * prioritizes some branches over others, e.g. security branches over stable branches over head
-  * implement a patch-testing service where people with appropriate credentials can submit a patch and schedule a build run with that patch applied
+  * Implement a patch-testing service where people with appropriate credentials can submit a patch and schedule a build run with that patch applied
+  * One weakness of the current summary script is that it relies on colors to distinguish successful from failed builds.  I have tried to find a better way&mdash;adding "FAIL" or "OK" to each table cell depending on the outcome of the corresponding build, for instance&mdash;but everything I've come up with has increased the size of the summary page to the point where it won't fit in a browser without scrolling, even on 1600x1200.
+See %%DagErlingSm%%ørgrav/%%LightMyFire%% for more information on this subject.
+----
+{{tag>Services}}
+{{tag>Stale}}