• Student: Aniket Pandey (aniketp@freebsd.org)

• Mentors:
  • Alan Somers (asomers@freebsd.org)

• Robert N. M. Watson (rwatson@freebsd.org)

• George V. Neville Neil (gnn@freebsd.org)

The Audit Test-Suite can be found here: AuditTestSuite

• The audit(4) sub-directory within FreeBSD source tree: tests/sys/audit

• The auditpipe(4) sub-directory within FreeBSD source tree: tests/sys/auditpipe

• The praudit(1) sub-directory within FreeBSD source tree: usr.sbin/praudit/tests

FreeBSD is a rapidly developing operating system with an extreme focus on advanced security & networking features. For an OS with a widespread usage and development, testing and monitoring of security regressions becomes a critical measure. FreeBSD has an audit subsystem which is responsible for monitoring a variety of security-relevant system events, such as user-logins, configuration changes, file system & network access. Although the audit framework is indispensable for security conscious organisations running FreeBSD servers, currently there is no tool to test its reliability and the intended behaviour.

The project aims to develop a regression test-suite, which will evaluate the audit framework for proper logging of most auditable system calls classified in TCP/IP & UDP sockets, File I/O, process control and device management, along with the semantics of audit trail's BSM/XML/text output.

BSM tokens can be obtained via synchronous I/O multiplexing on a special clonable device /dev/auditpipe, by configuring various preselection parameters for local mode auditing with the provided IOCTLs. Several libbsm(3) APIs and functions within the FreeBSD kernel can be used to analyse syscall tokens in the audit record. Finally, kyua(7)'s run-time engine will be used to automate regression testing of entire operating system at once, audit(4) included.

This application would consist of a set of test programs written in atf-c(3), containing independent test cases for triggering similar system calls, e.g open(2) & openat(2). Each program would test a wide variety of functionalities of corresponding system calls. Testing scenarios would ensure that the audit record contains all expected parameters, e.g the arguments, valid argument types, return values etc. The testing will be done for various success and failure modes, with cross-checking for appropriate errno codes in case of failure mode.

FreeBSD uses OpenBSM, an implementation of Sun's BSM event auditing file format and API. OpenBSM is made up of a plethora of tools including audit-viewer applications, e.g praudit(1) & auditreduce(1) as well as the libbsm(3) library to provide an interface to BSM audit record stream. The auditd(8) daemon is responsible for managing the kernel’s audit(4) mechanism and also rotates the log files whenever required.

FreeBSD already has user-space support for audit system. The audit daemon, auditd(8) can be enabled by adding the following line to /etc/rc.conf:

echo ‘auditd_enable=“YES”’ >> /etc/rc.conf

A safer way is to use sysrc(8)

sysrc auditd_enable=“YES”

Then start (and stop) the audit daemon, with a new audit trail:

service auditd {one}start && audit -n;
service auditd {one}stop

dir=‘/etc/security’

Audit configuration is defined in ${dir}/audit_control. Various event selection expressions are defined here, also found in ${dir}/audit_class, which lets a user configure the events to be audited. For example,

• lo for login-logout
• nt for network communication
• pc for process control
• ad for administration

${dir}/audit_user specifies events to be audited for each system user, like login attempts. ${dir}/audit_event contains a list of all auditable system events.

The test program here is an initial attempt at creating independent tests for similar system calls which can then be automated by Kyua.
The independence is achieved by following these steps:-

1. In a situation where the system does not have the audit system enabled, the tests would usually fail or skip. However, the audit daemon can be started (and stopped) by “service auditd one{start, stop}” without modifying /etc/rc.conf. This eliminates the need for a setup script as described in the deprecated test plan.

1. Create separate functions for the startup and termination of the audit daemon. This step also takes into account the possibility of auditd(8) running beforehand. If so, the process termination is avoided to maintain the current state of the machine. The running status can be checked by service auditd onestatus.

1. Audit system also allows live auditing via auditpipe(4) for IDS and testing purpose. This enables us to open /dev/auditpipe and verify the proper audit of the system call in question. Usually in cases like these, poll(2) helps in confirming if the opened file descriptor is ready to perform I/O operations.

1. The system may not have the proper settings in /etc/security. This would fail the events not covered by “audit_control” configuration file. However, auditpipe(4) describes quite a few ioctl(2) requests in audit_ioctl.h for such scenarios. getauclassnam(3) can be used to obtain required information from the audit_class(5) database.

• Set of tests for a small subset of the system calls supported by OpenBSM.
• Initial work done on writing Kyua compatible test programs, presumably using atf-c-api(3).

• A comprehensive and automated test-suite covering “most” system calls.
• Stand-alone tests for all system calls tested till now, to facilitate regression testing of the entire operating system at once.
• Set of test cases for checking file-format of audit-trails (text/XML/BSM).

• Conclusion of independent and stand-alone tests for audit system, arranged in the appropriate directory structure.
• atf-sh(3) & atf-c(3) tests for all audit tracepoints integrated into Kyua framework.
• An attempt will be made to test the integrity of audit_control(5) configuration file. (Stretch Goal)

1. Evaluating the proper audit of these system calls in either success or failure mode (or both) was not possible due to administrative and security issues.
2. BSD libc converts open(2) to openat(2) to optimize resource usage when certain arguments are passed. The testing was done using syscall(2).
3. This Wiki's general construct was heavily borrowed from SmokeTestingOfBaseUtilities and TCP-IP-RegressionTestSuite, authored by ShivanshRai.

I can't thank my mentors enough for all the help they provided during the summers. Also, thanks to ShivanshRai, MateuszPiotrowski & DomagojStolfa for helping me at various stages of my project when I was stuck. It has been a valuable learning experience and definitely something way more than I could have expected for a summer! I look forward to continue the work on improving the audit(4) subsystem and also exploring various other areas of FreeBSD.

And thanks to The FreeBSD Project for accepting this project and welcoming me into the community!