Basic knowledge of FreeBSD system administration is assumed. If you've never compiled and installed a FreeBSD kernel, this article may be hard to follow, since it does not provide all the necessary details. Study the FreeBSD Handbook first, especially chapter 8: 'Configuring the FreeBSD kernel'. The same goes for apache and tomcat, if you've never set up an apache server or a tomcat server this document will not be all inclusive.

This document will assume these things:

• Your internet facing interface is em0
• Your internal interface is em1
• Your external IP is 192.168.42.5/24
• Your internal IP is 192.168.69.1/24

All services that listen on external network interfaces are contained within jails, except for sshd. Each service has it's own jail, and there is also a base jail. The jails will listen on an internal interface, this interface doesn't have to be connected to a LAN.

• buildserver at 192.168.69.10 - contains a base jail which will be used to build the packages for the rest of the other jails;
• webserver at 192.168.69.20 - contains the Apache httpd, listening on port 8080 for HTTP connections and on port 8443 for HTTPS connections;
• appserver at 192.168.69.21 - contains the Tomcat web server, listening on port 7001 for AJP (v1.3) connections;

The setup described has been tested with the following software versions:

• FreeBSD 6.2p6 (amd64)
• ezjail 2.0.1
• Apache 2.2.8
• Tomcat 6.0.14
• mod_jk 1.2.26

If you haven't done so already, setup a FreeBSD 6 system.

• Secure configuration
  If you want your system to be secure, make sure your configuration process is also secure. Consider completing the configuration of the system before allowing any incoming connections. E.g. configure it while it's physically near your desk and disconnected from the Internet. This avoids that your system is compromised while being temporarily misconfigured.

Some tips for FreeBSD installation:

• Use the most recent supported errata fix branch. At the time of writing, this is RELENG_6_2; see the FreeBSD Release Engineering page to determine the current one.
• Perform a Custom installation.
• Stick /tmp on a separate partition or make it link to /var/tmp and mount it with the noexec and nosuid flags.
• Consider having a separate /usr/jails partition.
• Distributions: Choose the Minimal config. Include src and ports.
• Don't configure the machine as a gateway.
• Disable inetd.
• Disable NFS and anonymous FTP.
• Disable Linux compatibility.

After the installation:

• Install a custom kernel (instead of GENERIC). Trim your kernel configuration to the minimum, excluding things like NFS. Test it with nextboot(8).
• Install portaudit, which checks (to be) installed ports against the on-line ports vulnerabilities database via pkg_add -r portaudit.
• Install and use sudo instead of working as root all the time.
• Configure a firewall on your machine.
• Configure your system for mail sending only, see section 26.8 in the FreeBSD Handbook.
• Turn off password authentication in sshd_config and use ssh keys to login.

We will set up the pf firewall to do simple natting and redirection, this is needed so that the jails will be able to access resources from the internet. We will also use pf to let outside hosts access the webserver

[Step 1] Set up your /etc/pf.conf to nat and redirect the ports:

Modify /etc/pf.conf so that only these lines are uncommented:

# Macros: define common values, so they can be referenced and changed easily.
ext_if="em0"                                            # The external interface
int_if="em1"                                            # The internal interface
external_addr="192.168.42.5"                            # Your public IP address
internal_net="192.168.69.0/24"                          # Your internal subnet
# Translation: specify how addresses are to be mapped or redirected.
# nat: packets going out through $ext_if with source address $internal_net will
# get translated as coming from the address of $ext_if, a state is created for
# such packets, and incoming packets will be redirected to the internal address.
nat on $ext_if from $internal_net to any -> ($ext_if)
# rdr: packets coming in on $ext_if with destination $external_addr:1234 will
# be redirected to 10.1.1.1:5678. A state is created for such packets, and
# outgoing packets will be translated as coming from the external address.
rdr on $ext_if proto tcp from any to $external_addr/32 port 80 -> 192.168.69.11 port 8080
rdr on $ext_if proto tcp from any to $external_addr/32 port 443 -> 192.168.69.11 port 8443
# Make sure we don't block any traffic
pass in all
pass out all

[Step 2] Enable pf

Enable pf:

# pfctl -e -f /etc/pf.conf

Add this line to your /etc/rc.conf so that pf will start after your system reboots:

pf_enable="YES"

[Step 1] Install ezjail using the ports:

# cd /usr/ports/sysutils/ezjail/ && make install

[Step 2] Create an ezjail.conf file:

# cp /usr/local/etc/ezjail.conf.sample /usr/local/etc/ezjail.conf
# vi /usr/local/etc/ezjail.conf

You want to set the ezjail_ftphost to a FreeBSD ftp server near your server (ftp.nl.freebsd.org in my case) and you probably also want to set ezjail_uglyperlhack to yes, this will save you a lot of headaches with perl script you might want to run later. You may also want to adjust some other settings, if so you may want to read ezjail.conf(5). For example, all jails are by default placed under /usr/jails. Modify the ezjail_jaildir setting to change this.

[Step 3] Now have ezjail create a base jail, containing all directories and files that will be shared among the different jails.
Since this will run make world it may take a few hours.

# ezjail-admin install -p

The -p option will install ports in the base jail. For more information, please study the ezjail-admin(1) man page.

[Step 4] Make sure the jails will be started automatically when your machine starts up. Edit your /etc/rc.conf file and add the following line:

ezjail_enable="YES"

When you create a new jail, you can indicate of which flavour it should be. The flavour indicates which modifiable directories and files will be copied in the jail and contains a script to be run when the jail starts up for the very first time.

[Step 5] Configure the default flavour

Modify /usr/jails/flavours/default/etc/rc.conf to apply some default restrictions:

rpcbind_enable="NO"             # Disable the RPC daemon
cron_flags="$cron_flags -J 15"  # Prevent lots of jails running cron jobs at the same time
syslogd_flags="-ss"             # Disable syslogd listening for incoming connections
sendmail_enable="NONE"          # Completely disable sendmail
clear_tmp_enable="YES"          # Clear /tmp at startup

Copy the /etc/resolv.conf to the default jail so that we can resolve from within the jails:

# cp /etc/resolv.conf /usr/jails/flavours/default/etc

Next, we will want to create two custom flavours:

• webserver, for jails running a webserver (Apache 2.2 in our case);
• appserver, for jails running an application server (Tomcat 6, in our case).

[Step 6] Create some new flavours by copying the default flavour:

# cd /usr/jails/flavours/
# cp -Rp default webserver
# cp -Rp default appserver
# mkdir -p webserver/pkg
# mkdir -p appserver/pkg

• (i) Although it's not strictly necessary to use custom flavours, it's good practice, making it easier to resurrect a deleted jail or to instantiate a jail similar to another one.

[Step 7] Create a build jail.

We need to create a build jail first so that we can build the packages for the other jails:

# ezjail-admin create -f default build 192.168.69.10

You can now start the build jail by entering this command:

# /usr/local/etc/rc.d/ezjail.sh start build

Now we're going to install the jail in which Apache 2.2 will run and configure it. I'll only describe the configuration which is relevant to connecting to tomcat.

Enter the build jail and enter these commands to build the packages needed for the webserver jail:

# mkdir -p /var/ports/packages
# cd /usr/ports/www/mod_jk-apache2
# make distclean
# make package-recursive

Exit the jail and copy the packages to the flavour:

# cp /usr/jails/build/var/ports/packages/* /usr/jails/flavours/webserver/pkg

Creating the webserver will use almost the same command as creating the build jail:

# ezjail-admin create -f webserver webserver 192.168.69.20

This will create a basic jail and then add the packages in the pkg directory situated in the webserver flavour root. The webserver jail is then started by entering this command:

# /usr/local/etc/rc.d/ezjail.sh start webserver

Now we'll alter the apache configuration so that it will connect to the tomcat jail. Create /usr/local/etc/apache22/workers.properties and add these lines:

worker.list=tomcat                              # The name of the tomcat server
worker.tomcat.port=7001                         # The port of the tomcat server
worker.tomcat.host=192.168.69.21                # The host of the tomcat server
worker.tomcat.type=ajp13                        # The type of the connection
worker.tomcat.lbfactor=1                        # The weight of the tomcat server

Now modify the httpd.conf and be sure to add these lines:

Listen 192.168.69.20:8080                                       # Listen to the right IP/Port combination
# In the module section
LoadModule jk_module          libexec/apache22/mod_jk.so        # Load the mod_jk module
# mod_jk basic configuration
<IfModule mod_jk.c>
        JkWorkersFile etc/apache22/workers.properties           # Set the worker.properties file
        JkLogFile  /var/log/jk.log                              # Set the jk log
        JkShmFile  /var/log/jk-runtime-status                   # Set the status file
        JkLogLevel error                                        # Set the log level
</IfModule>

To mount your tomcat application on the root directory of a site add this line to the virtualhost:

JkMount /* tomcat

Where tomcat is the name you defined in workers.properties.

Other tips for configuring apache are:

• Remove all unneeded mods, this will make the risk of vulnerabilities in the webserver.
• Remove the default website and setup your own default virtual host.
• Use either the Location directive or tomcat for access control.
• Make apache listen to 192.168.69.20:8443 if you add SSL support.

After finishing configuration you are done with the apache2.2 configuration.

Now we're going to install the jail in which Tomcat 6 will run and configure it. Because the configuration of tomcat is very specific to your application I'll only show you how to bind it to the right IP.

Enter the build jail, deinstall all the old packages and enter these commands to build the packages needed for the webserver jail:

# cd /usr/ports/www/tomcat6
# make distclean
# make package-recursive

During the build process you'll get a prompt about the ports system not being able to fetch the files it needs to build java. Follow the instructions on screen and then repeat the above commands but without the make distclean.

Exit the jail and copy the packages to the flavour:

# cp /usr/jails/build/usr/ports/packages/* /usr/jails/flavours/appserver/pkg

Creating the appserver will use almost the same command as creating the build jail:

# ezjail-admin create -f appserver appserver 192.168.69.21

This will create a basic jail and then add the packages in the pkg directory situated in the appserver flavour root. The appserver jail is then started by entering this command:

# /usr/local/etc/rc.d/ezjail.sh start appserver

To make tomcat bind to the right port, find this line and change it:

<Connector port="7001" protocol="AJP/1.3"  redirectPort="8443" />

• "The Jail Subsystem" - Chapter 4 in the FreeBSD Architecture Handbook
• "Jails under FreeBSD 6" - Article at 'The FreeBSD Diary'
• "Virtualization with FreeBSD Jails" - ONLamp (O'Reilly) article, discusses FreeBSD 5
• ezjail homepage
• "FreeBSD jails" - Wikipedia article
• "FreeBSD Jail Software and Docs" by Stef Walter
• Man pages: jail(8) jls(8) jexec(8) rc.conf(5)

• "Apache HTTP Server" - Section 27.7 in the FreeBSD Handbook
• Apache httpd homepage
• "Apache httpd 2.2 vulnerabilities" - at apache.org

• Apache Tomcat homepage
• "The AJP Connector" - Part of Tomcat 6 Configuration Reference

• mod_jk documentation

Potential improvements for this article are:

• Add one or more diagrams. At least one that shows the host environment and the jails, with their IP addresses.
• Discuss ezjail-admin install -i.