FreeBSD is an Open-Source Operating System, with advanced features, a large community, and a small footprint. FreeBSD can run on multiple platforms, and has over 33,000 packages.

In this document, we will go over the use cases of FreeBSD for scientific purposes.

FreeBSD has multiple subsystems that makes it perfect for scientific computing; here’s a basic list of what it offers:

ZFS is an advanced filesystem with volume management capabilities. If there’s a filesystem feature that you want, then ZFS has it!

Some of ZFS’ features make it perfect for scientific computing, such as:

FreeBSD Jails are a mechanism to create systems isolated from the host, that have their own root directory, process tree and (optionally) network stack.

Jails are an amazing utility if you want to run:

• Different versions of FreeBSD
• Different versions of a specific runtime (Python, Ruby, Node, etc)
• Different environments for projects

They are known as “the old-school containers”, if you’ve heard of that buzz-word. FreeBSD Jails have been around since March 14, 2000, so they are VERY stable and secure!

You can learn about the basics of FreeBSD Jails in the FreeBSD Handbook: Jails and Containers

To setup a Jail right now, you can either:

1. Setup a Jail manually
   • VNET Jail HowTo
   • VNET Jail HowTo Part 2: Networking

1. Use Jail Managers Check the list of Jail managers: Handbook and Wiki

The DTrace facility allows users to locate performance bottlenecks in production systems. In addition to diagnosing performance problems, DTrace can be used to help investigate and debug unexpected behavior in both the FreeBSD kernel and userland programs.

More importantly, the DTrace facility traces programs without the need to restart them.

Let’s say a scientist informs you that their program is “not working”. As a system operator, you understand that the software is doing something wrong, but need to dig deeper to understand what the root cause is.

Assuming the misbehaving program’s name is do_science, we can do the following:

• we can start by tracing the userland itself, asking DTrace to print the User Stack (ustack) every time we hit a function entry point (entry)

# dtrace -n 'pid$target:::entry{ustack()}' -p `pgrep do_science`
[…]

  0  85517                      fopen:entry                                                                                                                                                                            
              libc.so.7`fopen+0x1                                                                                                                                                                                      
              do_science`initialize+0x2b                                                                                                                                                                               
              do_science`main+0x14                                                                                                                                                                                     
              do_science`_start+0x100                                                                                                                                                                                  
              `0x821408008                                                                                                                                                                                             
[…]

  1  83667                      sleep:entry 
              libc.so.7`sleep+0x1
              do_science`initialize+0x12
              do_science`main+0x14
              do_science`_start+0x100
              `0x821408008

We see that the program is trying to open a file using fopen and then goes into sleep. Another question rises: which file?

• Using the DTrace filters again, we can trigger the fopen function’s probe at entry, and ask DTrace to print the first argument of that function, which points to the file.

# dtrace -n '::fopen:entry{trace(copyinstr(arg0))}' -p `pgrep do_science`
dtrace: description '::fopen:entry' matched 1 probe
CPU     ID                    FUNCTION:NAME
  0  85517                      fopen:entry   foo.txt                          
  0  85517                      fopen:entry   foo.txt

Ah, apparently the foo.txt file is missing, which caused the program to “not work”.

To learn more about DTrace check the Handbook and wiki (One-Liners).

The FreeBSD Ports tree is a collection of software ported to FreeBSD. There are over 30,000 ports.

All ports have categories, including scientific ones!

Many ports have multiple categories.

More importantly, you can port software to FreeBSD and submit your work for addition to the ports tree, to make it available on every FreeBSD machine out there, including yours!

If you do have your own software, you can setup a FreeBSD Ports tree for yourself, which can be distributed as a custom FreeBSD Package Repository, so all your scientists can do # pkg install YourAwesomeSoftware to install the latest and greatest version of your program.

For porting to FreeBSD, check the Porters’ Handbook and Poudriere.

In a scientific environment, resources are always scarce and limiting their use is hard. FreeBSD provides a set of facilities to manage resources, such as the old-school Login Classes as well as the modern Resource Limits Database.

If you’ve already configured jail proj0, you can limit its memory (RAM) resource using the rctl command:

# rctl -a jail:proj0:memoryuse:deny=20G/jail

Another amazing facility is the CPU Sets, where specific CPUs can be bound to a Jail, so they don’t bother others!

Print Jails: their JIDs, names and CPUSet IDs

# jls jid name cpuset.id
1 proj0 5

Jail proj0 has JID 1 and CPUSet ID 5

Print the current CPUSet policy of Jail with ID 1:

# cpuset -g -j 1
jail 1 mask: 0, 1, 2, 3, 4, 5, 6, 7
jail 1 domain policy: first-touch mask: 0

Assign CPUSet 5 with the CPUs numbered 6 to 7

# cpuset -l 6-7 -s 5

done!

FreeBSD comes with tools to manage the realtime/idletime priority of processes. This allows administrators to run tools in the background without disturbing other processes.

Priority is an integer between 0 and 31, where 0 is highest.

Example: Run build_database.sh in the background using Idle Priority

# idprio 31 ./build_database.sh

Run reconnect_nodes.sh with highest priority

# rtprio 31 ./reconnect_nodes.sh

Here's a list of networking features that come out-of-the-box with FreeBSD:

• Ethernet Bridges
• VLAN
• Link Aggregation / Bonding
• Link Failover
• Address Redundancy
• IP-in-IP Tunnels (gif(4) / gre(4))
• ipsec(4)
• WireGuard
• VXLan
• Virtualized Networking (VNET)
• Graph-based Networking (NetGraph)
• ipfw — firewall, traffic shaper, packet scheduler, in-kernel NAT
• pf — packet filter originally from OpenBSD (pfctl(8), pf.conf(5))
• blacklistd

As you have guessed, FreeBSD has awesome infrastructure to help administrators run their operations.

However, most scientific applications are not written with FreeBSD in mind. Most of the time they are written for Linux. In some cases, they are written for runtimes that work only on Linux, such as Docker.

There are two common solutions for these problems:

FreeBSD can run Linux binaries using Linuxulator, which emulates the Linux ABI. If you want to run Linux Jails, check the following guide: FreeBSD Jail booting & running Devuan GNU+Linux with OpenRC and LinuxJails.

Please keep in mind that you might still have some issues, such as crashes while debugging (for example, invoking jstack to debug Java applications)

A more time-consuming but stable approach is to port the software to FreeBSD.

Data Science on FreeeBSDARM64 by Maciej Czekaj