Differences

This shows you the differences between two versions of the page.

--- summerofcode2012:automatedkernelcrashreportingsystem [2012/06/12 11:15] – TzanetosBalitsaris
+++ summerofcode2012:automatedkernelcrashreportingsystem [2012/08/22 21:36] (current) – Add information about the bugs_id column. The "Database" and the "Recognize phase" sections has been affected. TzanetosBalitsaris
@@ Line 3: / Line 3: @@
 ===== Project description =====
-This project aims to develop a system for automated kernel crash reports for the %%FreeBSD%% Operating System. When a %%FreeBSD%% system has a crash and responsible is a kernel panic, a crash reporter program will collect some necessary data from the crashed system and deliver them automatically in the form of a report to a Central Collector machine. The Central Collector machine will receive these reports, store them in a database, recognize which crash reports refer to previously reported bugs, and provide two web pages, one for the Community and one for the Developers of %%FreeBSD%%, each with different features.
+This project aims to develop an information system for automated kernel crash reports for the %%FreeBSD%% Operating System. When a %%FreeBSD%% system has a crash and responsible is a kernel panic, a crash reporter program will collect some necessary data from the crashed system and deliver them automatically in the form of a report to a Central Collector machine. The Central Collector machine will receive these reports, check them for validity, store them in a database, recognize which crash reports refer to previously reported bugs, and provide a dynamic website for the presentation of the collected data with different features for the community and the developers of %%FreeBSD%%.
 ===== Approach to solving the problem =====
-The approach of solving the problem is the division of the project into two parts. The first part is related with any %%FreeBSD%% system in the world that may crash (Client Side part). The second part is related with a central machine that collects the crash reports from the crashed systems (Server Side part).
+The approach of solving the problem is the division of the project in two parts. The first part is related with any %%FreeBSD%% system in the world that may crash (Client Side part). The second part is related with a central machine that collects the crash reports from the crashed systems (Server Side part).
 ==== Client Side part ====
-In this part, the main goal is to collect the debugging information from the crashed %%FreeBSD%% system and send them to the Central Collector machine which is responsible for storing the crash reports.
+In this part, the main goals are to locate, collect, form, and transfer the debugging information from the crashed %%FreeBSD%% system to the Central Collector machine which is responsible for storing the crash reports.
 The introduction of three new variables in the ///etc/rc.conf// file is necessary. These variables give the freedom to the System Administrator to configure the behavior of the crash reporting system. These three variables are called **crashreport_enable**, **crashreport_program** and **crashreport_email** and their default values are specified in the ///etc/defaults/rc.conf// file.
@@ Line 34: / Line 34: @@
 === Automated behavior ===
-The automated behavior refers to the automatic reporting of the kernel crash during the first multi-user boot after the crash. In every boot, the ///etc/rc.d/savecore// shell script executes the ///sbin/savecore// program that checks for a crash dump in the specified //dumpdev// device. If ///sbin/savecore// finds a crash dump, it extracts the crash dump to the //dumpdir// directory. If the type of the crash dump was a //full memory dump// or a //minidump//, the //dumpdir// directory will be filled from two files. The //vmcore.X// and //info.X//. If the type of the crash dump was a //textdump//, the //dumpdir// directory will be filled from a tarred file called //textdump.tar.X// that contains a bunch of //.txt// files. Subsequently, ///etc/rc.d/savecore// checks the //rc// variable //crashinfo_enable//. If this variable is set to "YES", then ///etc/rc.d/savecore// executes the ///usr/sbin/crashinfo// shell script that analyzes the extracted core dump and generates a text file in the //dumpdir// directory. This file is called //core.txt.X// and contains valuable debugging information. Likewise, ///etc/rc.d/savecore// checks the //rc// variable //crashreport_enable//. If this variable is set to "YES", then ///etc/rc.d/savecore// executes the ///usr/sbin/crashreport//. After the execution of the //usr/sbin/crashreport// the booting process continues normally.
+The automated behavior refers to the automatic reporting of the kernel crash during the first multi-user boot after the crash. In every boot, the ///etc/rc.d/savecore// shell script executes the ///sbin/savecore// program that checks for a crash dump in the specified //dumpdev// device. If ///sbin/savecore// finds a crash dump, it extracts the crash dump to the //dumpdir// directory. If the type of the crash dump was a //full memory dump// or a //minidump//, the //dumpdir// directory will be filled from two files. The //vmcore.X// and //info.X//. If the type of the crash dump was a //textdump//, the //dumpdir// directory will be filled from a tarred file called //textdump.tar.X// and from a file called //info.X//. The tarred file //textdump.tar.X// contains a bunch of //.txt// files (up to 5) that provide valuable debugging information. Subsequently, ///etc/rc.d/savecore// checks the //rc// variable //crashinfo_enable//. If this variable is set to "YES", then ///etc/rc.d/savecore// executes the ///usr/sbin/crashinfo// shell script that analyzes the extracted core dump (so it works only with //full memory dumps// and //minidumps//) and generates a text file in the //dumpdir// directory. This file is called //core.txt.X// and contains valuable debugging information. Likewise, ///etc/rc.d/savecore// checks the //rc// variable //crashreport_enable//. If this variable is set to "YES", then ///etc/rc.d/savecore// executes the ///usr/sbin/crashreport//. The crashreport program will search by default the ///var/crash// directory for a file called //core.txt.X// or //textdump.tar.X//. If both files exist, it will select the one with the higher X number. From the selected file, it will extract the needed information and use them to form the crash report. The crash report is a XML file with tags that the Central Collector machine can recognize. Finally, it will send the report to the Central Collector machine using the SSH protocol. After the execution of the //usr/sbin/crashreport// the booting process continues normally.
 === Manual behavior ===
-The manual behavior refers to the invocation of the ///usr/sbin/crashreport// explicitly from the System Administrator. This behavior is useful in the case that the System Administrator did not have enabled the variable //crashreport_enable// for various reasons when the system had a crash and now he/she wants to report the kernel crash. Also, it is useful for sending more debugging information when asked from a %%FreeBSD%% developer. Only the superuser will be able to execute successfully the crashreport program.
+The manual behavior refers to the invocation of the ///usr/sbin/crashreport// explicitly from the System Administrator. This behavior is useful in the case that the System Administrator did not have enabled the variable //crashreport_enable// for various reasons when the system had a crash and now he/she wants to report the kernel crash. Only the superuser will be able to execute successfully the crashreport program.
 === Anatomy of crashreport ===
@@ Line 45: / Line 45: @@
 **-d** __dumpdir__
-Specifiy an alternate core dump directory that contains the debugging information. The default dump directory is ///var/crash//.
+Specify an alternate core dump directory that contains the debugging information. The default dump directory is ///var/crash//.
-**-f** __file__
+**-f** __filename__
-Specify the name of the file that contains the debugging information. Normally, it is either //core.txt.x// or //textdump.tar.x//
+Specify the name of the file that contains the debugging information. Normally, it is either //core.txt.X// or //textdump.tar.X//
 **-e** __email__
 Specify the email address of the System Administrator that it will be used for contact from the Central Collector machine. Normally, the email address is specified in the variable //crashreport_email// of the ///etc/rc.conf// file.
-If none of the above options is specified, crashreport is capable of finding the right information on his own. That is, it can be invoked from the superuser without any options, and it will locate, collect and send the report without any further user interaction. It is important to mention that only the Superuser (root - uid 0) will be able to execute crashreport successfully. One technical reason is that the generated files //core.txt.x// and //textdump.tar.x// have read and write access only for root. Also, it is better only the System Administrator of the machine to be able to report a kernel crash and not every individual user that may have access to the machine.
+If none of the above options is specified, crashreport is capable of finding the right information on his own. That is, it can be invoked from the superuser without any options, and it will locate, collect and send the report without any further user interaction. It is important to mention that only the Superuser (root - uid 0) will be able to execute crashreport successfully. One technical reason for that is that the generated files //core.txt.X// and //textdump.tar.X// have read and write access only for root. Also, it is better only the System Administrator of the machine to be able to report a kernel crash and not every individual user that may have access to the machine.
-In the first version of crashreport (known as kcrashreporter), we wanted to instruct crashreport to generate the needed debugging information on his own. This is not an issue anymore because the script ///usr/sbin/crashinfo// does this job pretty well. Also as we mentioned above, it is invoked from the rc.d savecore script when savecore finds and extracts successfully a new core dump. So if the crash dump type is a //full memory dump// or a //minidump//, crashinfo will generate a //core.txt.x// file that will contain valuable debugging information. In the case of a //textdump//, the job is done without the aid of crashinfo.
+In the first version of crashreport (known as kcrashreporter), we wanted to instruct crashreport to generate the needed debugging information on his own. This is not an issue anymore because the script ///usr/sbin/crashinfo// does this job pretty well. Also as we mentioned above, it is invoked from the rc.d savecore script when savecore finds and extracts successfully a new core dump. So if the crash dump type is a //full memory dump// or a //minidump//, crashinfo will generate a //core.txt.X// file that will contain valuable debugging information. In the case of a //textdump//, the //usr/sbin/crashinfo// script does not take place but the file //textdump.tar.X// contains the debugging information that we need.
 An essential functionality of crashreport is the method that sends the report to the Central Collector machine. It uses the SSH protocol to transfer securely the report from the client to the server. Specifically, it uses the //scp// tool of %%OpenSSH%% with public key authentication. The configuration options can be divided in 2 parts, one for the client and one for the server. For the server machine we did the following:
-  * Register a %%DynDNS%% domain (akcrs.dyndns.org) that points to the (dynamic) IP address of the server machine (currently my Desktop computer). This domain will be used from the //scp// tool as the hostname field and as the portal to the Community and Developers web pages ([[http://akcrs.dyndns.org|akcrs.dyndns.org]]). *Create a new user (named //reporter//). This user will be used from the //scp// tool to transfer the report and store it in his home directory. *As the user //reporter//, we generate with //ssh-keygen// a pair of keys using the RSA algorithm. This is the user pair of keys. The public key is placed in the file //authorized_keys// of his //.ssh// directory (~/.ssh). The private key is distributed with crashreport. *We check the public key of the ///etc/ssh/ssh_host_ecdsa_key.pub// and distribute it with crashreport in order to register the fingerprint as known hosts in the Client side.
+  * Register a %%DynDNS%% domain (akcrs.dyndns.org) that points to the (dynamic) IP address of the server machine (currently my Desktop computer). This domain will be used from the //scp// tool as the hostname field and as the domain name for the website ([[http://akcrs.dyndns.org|akcrs.dyndns.org]]). *Create a new user (named //reporter//). This user will be used from the //scp// tool to transfer the report and store it in the directory ///var/spool/crashreports// that he owns. *As the user //reporter//, we generate with //ssh-keygen// a pair of keys using the RSA algorithm. This is the user pair of keys. The public key is placed in the file //authorized_keys// of his //.ssh// directory (~/.ssh). The private key is distributed with crashreport. *We check the public key of the ///etc/ssh/ssh_host_ecdsa_key.pub// and distribute it with crashreport in order to register the fingerprint as known hosts in the Client side.
 For the client, everything about the method for sending the report is integrated inside the crashreport script. That is, the host public key, the user private key and the //scp// command.
@@ Line 66: / Line 66: @@
 ==== Server Side part ====
-In this part, the main goals are to receive the crash reports, check them, store them in a database, recognize reports that refer to the same issue, assign a value that shows their importance, ask the submitters for further information when needed, and provide two web pages, one for the community and one for the developers.
+In this part, the main goals are to receive the crash reports, check them for validity, recognize the reports that refer to the same issue as previously logged reports, store them in a database, and provide one dynamic website with different features for the community and the developers.
-In order to accomplish these goals, we have to possess a %%FreeBSD%% machine that will act as the Central Collector machine, and has installed a DBMS for the manipulation of the database, a Web Server in order to provide the two web pages, and a program that will handle all the crucial tasks of the system. From now on, we will call it (informally) **handler**.
+The above goals define the functions of the Central Collector machine. The functions of the Central Collector machine can be divided in two parts:
+  * Organization of data
+  * Presentation of data
-About the actual machine, we will have to find a machine that will be able to run the mentioned software and has a stable internet connection 24 hours a day and 7 days a week. Maybe we can use the same machine that serves the official %%FreeBSD%% website or something similar. In the duration of the %%GSoC%%, I will host it on a x86 personal computer.
+The Organization of data part is divided in the following phases:
+  * Receive
+  * Check
+  * Recognize
+  * Store
-About the database, we will use a database management system and specifically the ORDBMS %%PostgreSQL%%. We choose Postgres because it is released under the %%PostgreSQL%% license, a license similar to the BSD license, it is a free and open source software, and it has high scores in various benchmarks. The schema of the database will be somewhat of the following form:
+The Presentation of data is done via the dynamic website.
+The //crashreportd// program is responsible to execute the organization of data, and the website the presentation of data.
+=== Database ===
+About the database that will store the received crash reports and any other data that will be used in the server side part, we will use a database management system and specifically the ORDBMS %%PostgreSQL%%. We choose Postgres because it is released under the %%PostgreSQL%% license, a license similar to the BSD license, it is a free and open source software, and it has high scores in various benchmarks. The schema of the database is the following:
 <code sql>
-Bugs(ID, State, Votes, Notes)
+Users(id, email, password, is_developer)
-Submitters(ID, Email, Password)
+Bugs(id, state, reported)
-Reports(ID, Bug_ID, Submitters_ID, Date, Backtrace, SystemRelatedInfo)
+Reports(id, bug_id, user_id, received_datetime, confirmation_code, confirmed, bugs_id, crashdata)
 </code>
-There are three tables, the table Reports, the table Bugs and the table Submitters. The table Reports stores all the crash reports that arrive in the system, while the table Bugs stores the unique crash reports. That is, if we have in Reports more than one report that is similar with other, then they share the same //Bug_ID// and there is only one tuple for this issue in Bugs. The table Submitters stores the email address along with a generated password and a unique ID for every submitter that send a report, either automatically or manually.
+There are three tables, the table Reports, the table Bugs and the table Users. The table Reports stores the data related with the crash reports that arrive in the system, while the table Bugs exists to indicate the problems that had been reported. For example, if we have in Reports more than one report that is similar with other, that means that both reports refers to the same problem. For this reason, we assign to them the same //bug_id// and we store in the Bugs table only one tuple that indicates the problem. As a rule, one report refers to exactly one bug, while one bug may be referred from more than one reports. Also, the table Bugs should always have equal or less tuples than Reports. The table Users stores the email address along with a generated password and a unique ID for every submitter that send a report, either automatically or manually.
-The primary key of Reports is the column named //ID// and the columns //Bug_ID// and //Submitters_ID// are foreign keys that references the columns //ID// of Bugs and //ID// of Submitters respectively. The column //Date// stores the date and time that the report was received. The column //%%SystemRelatedInfo%%// will probably span in more than one columns, each of them describing a characteristic of the panicked system that sent the report.
+The primary key of Reports is the column named //id// and the columns //bug_id// and //users_id// are foreign keys that references the columns //id// of Bugs and //id// of Users respectively. The column //received_datetime// stores the date and time that the report was received. The column //confirmation_code// stores the code that is sent to the submitter in order to confirm his/her report. The column //confirmed// indicates whether this report has been confirmed or not. The column //bugs_id// is used from the recognize phase when crashreportd determine that the report refers to more than one logged bugs (more details in the Recognize phase). The column //crashdata// is not a single column in the actual schema, but it is here to represent the columns related with the data that has been received from the panicked system that sent the report. For example, some columns are //panic// which keep the panic message, //backtrace// which keep the backtrace, //ps -axl// which keep the output of the command ps with the switches a, x and l.
-The primary key of Bugs is the column named //ID//. The column //State// defines the current state of the bug. This column can take only one of the following values:
+The primary key of Bugs is the column named //id//. The column //state// defines the current state of the bug. This column can take only one of the following values:
 ***Open**: This is the initial state of a bug. This state declares that there is at least one report that refers to it and the bug has not been reviewed from a %%FreeBSD%% developer yet.
@@ Line 94: / Line 106: @@
 The bug states that described above have many similarities with the states of Problem Reports in the GNATS bug tracking system that is currently used from the %%FreeBSD%% project.
-The column //Votes// is an integer that shows the importance of a bug based on the number of reports that arrive to the system and point to a particular bug. For example, if we have ten different reports that refer to the same bug, then the //Votes// column of this bug will have the value 10. The presence of the column //Votes// is for performance reasons, for easier manipulation of the data and logical design. Instead of counting the entries of a specific //Bug_ID// in the Reports table every time that we want to query the database for the most frequent bug report or something similar, we simply check the //Votes// column. Finally, the column //Notes// can be used only from the %%FreeBSD%% developers in order to inform the submitters of a particular bug. In this column, a %%FreeBSD%% developer may write valid //kgdb// commands that he/she wants to be executed from the submitter.
+The column //reported// is an integer that shows the importance of a bug based on the number of reports that arrive to the system and point to a particular bug. For example, if we have ten different reports that refer to the same bug, then the //reported// column of this bug will have the value 10. The presence of the column //reported// is for performance reasons, for easier manipulation of the data and logical design. Instead of counting the entries of a specific //bug_id// in the Reports table every time that we want to query the database for the most frequent bug report or something similar, we simply check the //reported// column.
-About the two Web pages, I prefer the Apache Web Server as the software that will be responsible to serve them. The two Web pages refer to two different groups. The first Web page will be accessed by everyone from everywhere and it will show the list of received reports along with some information about them. That is, the backtrace (not full backtrace in the case that exists), the Operating System and its release (for example, %%FreeBSD%% 9.0-STABLE), and the type of hardware platform of the panicked system (for example, amd64). This page will be called Community Web page. The second Web page will be accessible only by giving a valid username and password (only %%FreeBSD%% developers will have one) and when you are logged in, you have the ability to view all the data in the database and update some of them. This page will be called Developers Web page.
+=== Organization of Data ===
-I would like to make a pause and mention two things about the privacy on the Community web page. Because the backtrace can reveal some sensitive information about the user that made the report, we will only make visible the function names of every function call and the file that they are located. That is, without the arguments that potentially contain private information. I think that with this way, the user can still view useful information about the crash (which function is responsible) without giving away further information.
+== Receive phase ==
-Finally, about the handler. The handler will be a program written in PHP in conjunction with SQL that can be described as the core of the Central Collector machine. Handler will be an internal program that performs the following tasks:
+In this phase we define the way that crashreportd program will understand that a new crash report has arrived in the system. The crashreportd program is a daemon that checks constantly the directory where the crash reports arrive (///var/spool/crashreports//). If the directory is not empty, it makes a list of the detected reports and start examining them one by one.
-*Receive the crash reports
+== Check phase ==
-*Check the crash reports for security issues
-*Store the reports in the database
-*Match same reports
-*Provide a mechanism for votes
-*Sends emails to the submitters, either for feedback or informing them about the status of their report(s)
-About the reception of the crash reports, we define the way that the handler will understand that a new crash report arrived in the system. This is related with the used protocol and method. A crash report is contained in a single ASCII file.
+This second phase exists primarily for security reasons. Our goal is to identify if a report is really a valid crash report and not junk that wants to pollute our system. In order to achieve this, we perform a series of checks on the examined report. A valid crash report is a tar.gz file with filename a strict naming pattern. The gunzip file contains only one file, which has a filename of the same pattern but ends with //.xml//. Also, the XML file should comply against a DTD that checks if the XML file is a well formed and valid crash report. If these checks are successful, then crashreportd sends a confirmation email to the submitter. This is the last check that will determine that the crash report is valid.
-About the check of the reports, this action exists primarily for security reasons. Firstly, the system checks if an email address is bundled with the crash report and sends a confirmation email to the submitter to the address listed. The confirmation email will include a link that should be followed in a period of time in order to consider the report valid. If the submitter follow the link in this period, then the report becomes valid. If the email address of the submitter is not listed in the database, the system will add the email address along with a generated password and a unique id to the Submitters table. After the confirmation of the report, the system will examine the report for any malicious code and it will split the single text file in multiple parts ready to be stored in the database. Finally, the system will inform the submitter via a second email about his/her new account in the case that the system just registered it and send a link of his/her report. The method of the confirmation email will reduce the chances of storing multiple fake reports sent from an automated script.
+== Recognize phase ==
-Now that we are pretty sure that the received report is clean and valid, the handler actually stores the report in the database. In order to store the report, the system needs to define a //Bug_ID// for the report. About the matching of the same reports and the mechanism of votes, when a crash report arrive in the machine, the handler is responsible to recognize if there is any similar report already in the database. If yes, the handler match them under the same //Bug_ID// and increase the //Votes// field by one. Otherwise, it assigns a new //Bug_ID// and the value 1 in //Votes//.
+The third phase is the recognize phase. Now that we know that the report is syntactically valid (we perform this phase without knowing that the report has been confirmed thought), crashreportd will try to determine if the report refers to a problem that previously logged reports had been referred too. If the report is similar with a previously logged bug, then we will assign the //bug_id// to the new report. If the report is not similar with any previously logged bug, then we will create a new bug in the Bugs table and we will assign the id of this bug to the report. If the report is similar with more than one logged bugs, then we will store the IDs of these bugs in the column //bugs_id//. This is an indication that our algorithm is not very accurate. The column //bugs_id// could be used from a developer in order to understand which bugs the algorithm thought that the report may refers to, and manually set the correct bug or create a new one.
-In this point, I would like to explain in more details the process of how we will actually decide if a new bug report is the same as a previously logged bug report because it is a crucial of the system. Firstly, the whole process is based on the following assumptions:
+The process of how we will actually decide if the new report is the same as a previously logged report is a crucial part of the system. Firstly, the whole process is based on the following assumptions:
-  - Every backtrace produced from a kernel panic, contains a function call to //panic()//.
+  - Every backtrace contains stack frames that identify the source of the problem, and stack frames that are in common with every stack backtrace.
-  - Function names that contain the word //trap// can be found only before the call of //panic()//.
+  - Stack frames that have as a function name a name that contains the words //syscall//, //panic//, //trap//, //lock//, //sleep// or //??// are insignificant.
-  - Function names that contain the word //syscall// can be found only in the first frames of a backtrace.
+  - Stack frames that have as a function name a name that does not contain the above words and lie between the last insignificant stack frame and the first insignificant stack frame after the significant stack frames (while searching from bottom to top).
-The above assumptions have been derived from examining the backtraces of different kernel panics reported to the GNATS database. Driven by those assumptions and the comments of developers in every PR, we were led to the following conclusion:
+The above assumptions have been derived from examining the backtraces of different kernel panics reported to the GNATS database, and from the problem reports found from the kernel stress test suite [[http://people.freebsd.org/~pho/stress/log/ls.html|Stress 2 project]].
-  * The frames that play the leading role of identifying a bug, lie between the first function that contains the word 'trap' (if it exists, otherwise panic(%%))%%, and the last function that contains the word 'syscall'. (count from bottom to top)
 For example, consider the following backtrace:
@@ Line 143: / Line 148: @@
 </code>
-From this backtrace, the important frames are from #2 to #10. The most important frame is the frame #2. The frames #0, #1 and #11, #12 are not vital in identifying the bug.
+From this backtrace, the significant frames are from #2 to #10. The most significant frame is the frame #2. The frames #0, #1 and #11, #12 are not vital in identifying the bug. How do we found that? We start searching from the bottom stack frame (i.e #12) up to the top. The #12 stack frame has a function name that contains the word syscall, so we mark it as insignificant. We continue with the #11 stack frame that also has the syscall word. We move to the #10 stack frame that does not contain any insignificant word. So it is the first significant stack frame that we found. We continue with the same manner up to the stack frame #1 that is the first time that we encounter an insignificant frame after our significant frames. This marks the end of our algorithm.
 Also, consider the following backtrace:
@@ Line 171: / Line 176: @@
 </code>
-From this backtrace, the important frames are from #6 to #17. The most important frame is the frame #6. The frames #0, #1, #2, #3, #4, #5 and #18, #19, #20 are not vital in identifying.
+From this backtrace, the significant frames are from #6 to #17. The most significant frame is the frame #6. The frames #0, #1, #2, #3, #4, #5 and #18, #19, #20 are not vital in the identification.
-Based on the above things, we decided that the most significant factors for matching two reports are:
+After examining the reports, we decided that the factors that will influence the matching of two reports are:
-*the Panic Message
+*the panic message
-*the Function before the call of the //panic()// function or the call of a function that contains the word //trap//.
+*the significant frames of the backtrace
-Also, a frame in the bottom of the stack is the less significant frame, while the frame in the top of the stack is the most significant frame.
-Based on all these things, we concluded that in order to say that 2 reports refer to the same bug, they should have at least the panic message and the function before a //trap// function or //panic()//, exactly the same.
+From the significant stack frames, we say that the stack frame in the bottom is the less significant frame, while the frame in the top is the most significant frame.
-In order to deal with different function names from release to release and from architecture to architecture, we have to perform a more flexible approach than comparing two strings character by character and return true if they are exactly the same, otherwise false.
+In order to deal with different function names from release to release and from architecture to architecture, we have to perform a more flexible approach than comparing two strings character by character and return true if they are exactly the same or false if they are not.
 For this reason, we add the notion of percentage in every factor. If two reports are exactly the same, then the matching returns a 100% similarity.
-For the two most significant factors, we choose that the panic message costs 25% and the backtrace costs 75%.
+For the two factors, we choose that the panic message costs 25% and the backtrace costs 75%.
-For the backtrace, the most important frame (the top one before panic() or 'trap') costs 25% and all the others a 50%.
+For the backtrace, the most significant frame costs 25% and all the others a 50%.
-For the remaining frames, we allocate symmetrically this 50% percent based on the place in the stack.
+For the remaining significant stack frames, we allocate incrementally and symmetrically this 50% percentage based on their place in the stack.
-For every match, we calculate the similarity and if it is above 50% then these reports refer to the same bug.
+For every match, we calculate the similarity and if it is above 50%, then these reports refer to the same bug.
-For an implementation of the matching reports process that described above, check [[http://aetos.it.teithe.gr/~tzabal/gsoc/algo|this]] demo that is written in PHP.
+For a better understanding of the recognize phase, it is better to check the actual source code that is written in Python.
-About the interaction of the system with the submitters, we can say that this will be achieved through emails using the SMTP protocol. The handler will send a confirmation email to the submitter of every report, send an email informing that the report was submitted successfully and a link to the reports web page in the Community Web page. Also, it will send an email when asked from a %%FreeBSD%% developer asking for more debugging information.
+== Store phase ==
+The last phase is the store phase where crashreportd stores the report in the database as unconfirmed. We create a new user in Users if the email address of the report does not exist in our system and we create a new bug in Bugs if in the recognize phase we determined that this report refers to a new bug. Otherwise we assign the appropriate user_id and bug_id. This unconfirmed report will not be treated as a report until the user confirms it. That is, it will not presented in the website or compared against another report in the recognize phase.
+=== Presentation of Data ===
+== Website ==
+The website is a dynamic website responsible to present the data that we have collected to the public. Everyone is able to view the received crash reports along with their crash information and view the identified bugs. Only the submitters and the %%FreeBSD%% developers will be able to log in into the system. The developers can perform further actions like evaluating a crash report, changing the state of a bug and have access to the email address of submitters in order to contact them. In the future, more features can be implemented.
+=== Central Collector Machine ===
+In the duration of the development of the project and until the %%FreeBSD%% project accept the system, the Central Collector machine will be my desktop computer (probably this will change in the near future) running %%FreeBSD%% 9.0-RELEASE with a stable ADSL connection. The role of the DBMS will be played from the %%PostgreSQL%% Server 9.0.8, the Apache HTTP Server 2.22.22 will serve the website, and the Python 2.7.3 will execute the crashreportd program and the Pyramid powered website.
 ==== Toolbox ====
-Our toolbox contains (Bourne) Shell programming for the construction of the crashreport program, the %%FreeBSD%% Operating System (a RELEASE version) for the Operating System of the Central Collector machine, the Apache Web Server as the Web Server that will serve the two web pages, the use of the ORDBMS %%PostgreSQL%% for the management of our database, the use of the PHP programming language with the SQL programming language for the construction of functionalities of the Central Collector machine.
+Our toolbox contains (Bourne) Shell programming for the construction of the crashreport program, the %%FreeBSD%% Operating System (a RELEASE version) for the Operating System of the Central Collector machine, the Apache HTTP Server as the Web Server that will serve the two web pages, the use of the ORDBMS %%PostgreSQL%% for the management of our database, the use of the Python programming language along with a set of libraries for the construction of the crashreportd program and the web framework Pyramid for the dynamic website.
 ===== Deliverables =====
-A fully functioning system for automated and manual kernel crash reports for the %%FreeBSD%% project.
+An information system for automated and manual kernel crash reports for the %%FreeBSD%% project.
 ===== Milestones =====
-Is currently under review.
+May 21 - June 03 (Week 01, 02)
+*Implementation of the crash reporter program (client-side)
+June 04 - June 17 (Week 03, 04)
+*Implementation of the rc.d script (server-side)
+June 18 - July 01 (Week 05, 06)
+*Connecting the client-side with the server-side
+*XML crash reports
+July 02 - July 15 (Week 07, 08)
+*Implementation of the receive phase
+July 16 - July 22 (Week 09)
+*Implementation of the check phase
+July 23 - July 29 (Week 10)
+*Implementation of the store phase
+July 30 - August 05 (Week 11)
+*Implementation of the recognize phase
+*Reorganization of server-side code
+August 06 - August 12 (Week 12)
+*Construction of the dynamic website that will present the data
+August 13 - August 19 (Week 13)
+*Updates for the website
+*Documentation
+*Testing the whole system
 ===== Test Plan =====
-(List of steps you plan to use to test your work, as discussed with your mentor)
+==== How to test the Client Side part ====
+The recommended way to test the Client Side part is to create a virtual machine (ex. using [[/VirtualBox|VirtualBox]]) and install a %%FreeBSD%% version of your choice. Then, you can download the source in /usr/src from the socsvn repository and build the system. In the end, you will have a virtual machine running %%FreeBSD%% 10.0-CURRENT (at the moment a 2 months old revision) with this project's added code. The following is a guide that describes the steps after creating your %%FreeBSD%% virtual machine.
+<code bash>
+# Once installation is complete, obtain the Ports Collection
+portsnap fetch
+portsnap extract
+portsnap update
+# Install Subversion from the Ports Collection
+cd /usr/ports/devel/subversion
+make -DBATCH
+make install
+make clean
+# Check out the project's client side code into /usr/src (your Working Copy)
+svn checkout https://socsvn.freebsd.org/socsvn/soc2012/tzabal/client-side/akcrs-head/ /usr/src
+# The following is from: http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/makeworld.html
+cd /usr/src
+make buildworld
+make buildkernel
+make installkernel
+shutdown -r now
+# Boot in single-user mode
+mount -u /
+mount -a -t ufs
+adjkerntz -i
+mergemaster -p
+cd /usr/src
+make installworld
+mergemaster
+reboot
+# Periodically update your working copy
+cd /usr/src
+svn update
+# In the case of minor changes, install manually
+# Example of having only the usr.sbin/crashreport updated
+cd usr.sbin/crashreport
+make install
+</code>
 ===== The Code =====
-https://socsvn.freebsd.org/socsvn/soc2012/tzabal/
+http://svnweb.freebsd.org/socsvn/soc2012/tzabal/
 ===== Useful links =====