Show pageOld revisionsBacklinksBack to top This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong. ===== Tinderbox ===== This article describes the %%FreeBSD%% source tinderbox. To learn more about what it does, take a look at the [[http://tinderbox.freebsd.org/|current FreeBSD src build status]]. The tinderbox ecosystem consists of: * A build script, ''%%tinderbox%%'' that automates checking out a specific version of the %%FreeBSD%% ''%%src%%'' tree and building it * A supervisor script, ''%%tbmaster%%'', that runs and monitors individual ''%%tinderbox%%'' instances, logs their output, and mails out failure notices * A CGI script, ''%%index.cgi%%'', that reads a set of ''%%tbmaster%%'' logs and presents an easy-to-read HTML summary * A set of build servers that continuously test the tip of the most important %%FreeBSD%% branches * A web server that keeps a complete set of tinderbox logs and displays an up-to-date summary ===== History ===== tinderbox scripts were developed and are maintained by %%DagErlingSm%%ørgrav. The original version was a shell script that ran on ''%%bento.freebsd.org%%'' once a day, after one Alpha-breaking commit too many. It was replaced with the current Perl implementation in early 2003. All scripts and configuration files are kept in [[http://svnweb.freebsd.org/base/user/des/tinderbox/|base/user/des/tinderbox]]. ===== Dashboard & Build Status ===== **Note:** The service described here is no longer operational and now redirects to the %%FreeBSD%% [[Jenkins]] Continuous Integration service: [[https://ci.freebsd.org/tinderbox|Tinderbox-style view]]. Summaries and logs from the official build servers are made available online at [[http://tinderbox.freebsd.org/]]. Currently, this site is hosted by %%DagErlingSm%%ørgrav. The site is set up as follows: * A cron job checks the build servers at regular intervals and downloads any new log files using ''%%rsync%%''. * [[Apache]] is set up to use ''%%index.cgi%%'' as ''%%DirectoryIndex%%''. * A [[https://www.varnish-cache.org/|Varnish]] instance in front of Apache ensures that ''%%index.cgi%%'' does not need to run more than once every two minutes. ===== tinderbox script ===== This section has not yet been written. See the ''%%tinderbox(1)%%'' manual page for details. ===== tbmaster script ===== This section has not yet been written. See the ''%%tbmaster(1)%%'' manual page for details. ===== Configuration Files ===== This section has not yet been written. ===== index.cgi script ===== The ''%%index.cgi%%'' script generates the HTML summary. Although intended to be used as a CGI script (as the name indicates), it can also be run from the command line, or from a cron job, in which case it will look for logs in the directory in which the script itself is located. It will automatically detect the context, and output HTTP headers only if run as a CGI script. The script outputs strict XHTML 1.0 code and uses CSS for styling. ==== MAIN ==== The script's entry point is in the named block ''%%MAIN%%''. The script starts by attempting to verify that it is running on the official tinderbox web site. If it isn't, the page it produces will include text to that effect, and a link to the official site. It then scans the log directory to get an inventory of configurations, branches and architectures for which log files exist, to avoid hard-coding a list into the script and potentially ending up with blank rows or columns. This information is derived from the names of the log files, which match the following pattern: <code> tinderbox-$config-$branch-$arch-$machine.{brief,full} </code> The configurations used on the official tinderbox build servers are named for the branches they build; for instance, the ''%%releng_6%%'' configuration is used to build ''%%RELENG_6%%'' as well as all still-supported release branches. Once this has been determined, the script generates a prologue, calls ''%%do_config()%%'' for each configuration, and generates an epilogue. ==== do_config() ==== The ''%%do_config()%%'' subroutine generates HTML code for a single tinderbox configuration. It first generates a header row, then iterates over each branch build with this particular configuration, producing a single row of results for each in the following manner: * For each architecture, * For each machine within that architecture, * If a //brief// log file exists: * Call ''%%success()%%'' to determine the outcome of the build * Output the modification date * Output the size of the //brief// log file with a link to the file itself * If a //full// log file also exists: * Output the size of the //full// log file with a link to the file itself * Otherwise: * Output ''%%n/a%%'' ==== success() ==== The ''%%success()%%'' subroutine scans a //brief// log file to determine whether the build was successful. This is done quite simply by searching for the string "tinderbox run completed". ==== Build Sorting ==== Configurations and branches are sorted according to their //branch rank//, which is computed as follows: * ''%%HEAD%%'' and ''%%CURRENT%%'' have rank 9999 * ''%%RELENG_x%%'' has rank xx99 * ''%%RELENG_x_y%%'' has rank xxyy In practical terms, this means that ''%%HEAD%%'' always ranks highest, and that ''%%RELENG%%'' branches are ranked in numerical order, with each ''%%STABLE%%'' branch ranking higher than the release branches forked off of it. For instance, for %%FreeBSD%% 6, the order will be, from highest-ranking to lowest-ranking: * ''%%RELENG_6%%'' (branch rank 699) * ''%%RELENG_6_3%%'' (branch rank 603) * ''%%RELENG_6_2%%'' (branch rank 602) * ''%%RELENG_6_1%%'' (branch rank 601) * ''%%RELENG_6_0%%'' (branch rank 600) ==== Colors ==== The summary script uses CSS to apply different colors to each cell in the table. Successful builds are displayed with green text, failed builds with red text. The color fades as time passes since the corresponding build, with every half-hour bringing it one step closer to gray. ===== Official Build Servers ===== The official tinderbox build servers are hosted by [[http://www.sentex.ca/|Sentex Data Communications]], who also host the [[RobertWatson/netperf|FreeBSD netperf cluster]]. There are currently three build servers: * **freebsd-current.sentex.ca**: builds * ''%%HEAD%%'' for amd64, arm, i386, pc98, ia64, mips, powerpc, and sparc64; this takes about four hours. * ''%%RELENG_8%%'' for amd64, arm, i386, pc98, ia64, mips, powerpc and sparc64; this takes about three and a half hours. * **freebsd-stable.sentex.ca**: builds RELENG_7 and supported 7.x branches for amd64, i386, pc98, ia64, powerpc and sparc64; each branch takes about six hours. * **freebsd-legacy.sentex.ca**: builds RELENG_6 and supported 6.x branches for amd64, i386, pc98 and sparc64; each branch takes about three hours. ===== Future Work ===== Suggested improvements: * Improve the logic ''%%tbmaster%%'' uses to extract the error message from the full log to put in the brief log. * When a build finishes, keep the source and object tree along with the log file, and filter the log file through a CGI script that adds markup linking every file name to the corresponding file in the source or object tree, and every compiler error or warning with the file and line it references. * Instead of checking out a separate source tree for each platform, use a single source tree per iteration * Store results (though perhaps not full logs) in a database * Implement a better build scheduling algorithm which * avoids building branches that haven't changed since the last build * prioritizes some branches over others, e.g. security branches over stable branches over head * Implement a patch-testing service where people with appropriate credentials can submit a patch and schedule a build run with that patch applied * One weakness of the current summary script is that it relies on colors to distinguish successful from failed builds. I have tried to find a better way—adding "FAIL" or "OK" to each table cell depending on the outcome of the corresponding build, for instance—but everything I've come up with has increased the size of the summary page to the point where it won't fit in a browser without scrolling, even on 1600x1200. See %%DagErlingSm%%ørgrav/%%LightMyFire%% for more information on this subject. ---- {{tag>Services}} {{tag>Stale}} tinderbox.txt Last modified: 2022/10/06 02:57by KubilayKocak