FreeBSD Jails

FreeBSD jails are a sophisticated chroot environment. Both the man page (man(8) jail) and the Handbook pages are very good, but it is my hope that this page might save someone a bit of googling for answers. You will probably notice sentences copied from both sources, and although a good 90-95 percent of things here could truthfully begin with, "As the man page suggests," or, "As the Handbook suggests," it is usually left out for brevity's sake.

This page is about creating a single jail on a system. If you are planning to have multiple jails, then see my page about using nullfs to share resources among several jails. Rather than duplication, I also cover a couple of other topics there not covered here, such as renaming a jail and using multiple IP addresses in a jail.

We are going to work on a host with an address of 192.168.1.50. We will make a jail, called myjail, with an address of 192.168.1.51. We will store it in a /jails directory that we'll create, so its path will be /jails/myjail. It will have the fully qualified domain name of myjails.example.com

Start by creating the directory tree. If you're doing this as root (almost all of this will require either being root or using sudo), and are using root's default tcsh shell, use setenv D to set the jail directory's variable to $D. If using a Bourne style shell, for example, with sudo, change the setenv D /jails/myjail to D=/jails/myjail
setenv D /jails/myjail
mkdir -p $D
cd /usr/src
make buildworld
make installworld DESTDIR=$D
make distribution DESTDIR=$D
mount -t devfs devfs $D/dev

If you've recently run a buildworld, you may be able to skip the make buildworld setup. You can try going directly to the make installworld line. If it succeeds, great, if it fails, then remove what did get installed in the jail, run buildworld, and then run the make installworld line again.

Also, as mounting devfs does have some security risks, look in /etc/defaults/devfs.rules for the section marked jail. At time of writing it begins with a section [devfsrules_jail=4]. Copy that section to /etc/devfs.rules. (You will probably have to create /etc/devfs.rules.) Now you can apply it with
devfs -m /jails/myjail/dev rule -s 4 applyset

Sometimes this step fails with a no such process error. I've not been able to track down why it sometimes works and sometimes doesn't, and various solutions I've seen haven't applied to my situation. In every case when this has happened to me, once I add the proper lines to /etc/rc.conf (as mentioned below), and start the jail, the ruleset is followed. I suspect typos on my part, but have yet to confirm that.

Copy over the host's /etc/resolv.conf to the jail.
cp /etc/resolv.conf /jails/myjail/etc/

Enter the jail for some configuration. The man page suggests using the hostname and ip for this command, but I've been able to do this with
jail -c path=/jails/myjail command=/bin/sh

This will leave you at a root prompt inside the jail. Set root's password with the passwd command. I also like to add a user with the adduser command, making sure I put the user in the wheel group so that said user can su to root.

Run newaliases. It may or may not complain about no fully qualified domain name, but that's not important at this point.

newaliases

While still in the jail run
tzsetup

to set your time zone.

I then edit /etc/rc.conf in the jail to add the hostname and enable ssh.
hostname="myjail.example.com"
sshd_enable="YES"

If this is a test jail, you don't even need the FQDN, hostname="myjail" will also work. You can now exit the jail by typing exit at the # prompt.

Create an alias for the interface that the jail will use. In this case, we'll say the machine's interface is em0, and it already has an address of 192.168.1.50. We'll now create an alias of 192.168.1.51 for the jail to use.
ifconfig em0 alias 192.168.1.51/32

Add a line for the alias to /etc/rc.conf, so that it will persist upon reboot.
ifconfig_em0_alias0="inet 192.168.1.51/32"

The man page mentions creating an /etc/jail.conf. When I first began using jails, the information in that file was added to rc.conf, and I didn't know about jail.conf till I recently reread the man page. I don't see mention of jail.conf in the Handbook either, so for now, I'm continuing to just add the following entries to /etc/rc.conf on the host. In addition, some quick testing indicated that it could be problematic with FreeBSD-9.2--for example, as far as I can tell, my syntax was correct, but the jail ignored all directives that I put in /etc/jail.conf.

There is a sysutils/jail2 port that according to its description, uses /etc/jail.conf, and seems to require jail2_enable rather than jail_enable, in /etc/rc.conf, but I haven't tested it.

FreeBSD-10 is a bit more insistent, giving warning messages suggesting the user switch to /etc/jail.conf. Firstly, let's deal with 9.x

Add the following to /etc/rc.conf
jail_enable="YES"
jail_list="myjail"
jail_myjail_hostname="myjail.example.com"
jail_myjail_rootdir="/jails/myjail"
jail_myjail_ip="192.168.1.51"
jail_myjail_devfs_enable="YES"
jail_myjail_devfs_ruleset="devfsrules_jail"
jail_myjail_exec="/bin/sh /etc/rc"

If using FreeBSD 10, use /etc/jail.conf as default. Leave the line jail_enable="YES" in /etc/rc.conf. The rest of the information will be in /etc/jail.conf. In this case, it reads
myjail {
	path = /jails/myjail;
	mount.devfs;
	devfs_ruleset = 4;
	host.hostname = myjail.example.com;
	ip4.addr = 192.168.1.51;
	exec.start = "/bin/sh /etc/rc";
	exec.stop = "/bin/sh /etc/rc.shutdown";
}

(As a quick note, man(5) jail.conf indicates that one can use wild card paramaters at the beginning of the file to apply to all jails, then customize each jail. I haven't worked with this yet.)

The above configurations assume that you copied the /etc/defaults/devfs.rules section for jails to /etc/devfs.rules. For the ruleset name, use whatever you have in /etc/devfs.rules in /etc/rc.conf (FreeBSD-9x) and the number, in this case, "4" in /etc/jail.conf for FreeBSD-10. If you have multiple jails, you would add them to the jail_list line, without commas, e.g.,
jail_list="myjail myotherjail"

If running Zabbix, Postgresql, and no doubt, some other services, (though those are the only two that I know of), note that they require that security.jail.sysvipc_allowed be set to 1, rather than its default of 0. This has to be set per jail. So, if you need to run one of these services add
jail_myjail_parameters="allow.sysvipc=1"
to /etc/rc.conf in FreeBSD-9x and
allow.sysvipc = 1

to a jail's entry stanza in /etc/jail.conf in FreeBSD-10.

There are some security issues with changing that setting, so the reader should investigate the risk for their particular situation.

You should now be able to start the jail with the command
service jail start myjail

You should now be able to reach it by ssh, and once inside the jail, be able to use pkg_add (or pkg install if using pkgng) to install packages. Although ports are not installed, you can add them if desired with portsnap.

You should be able to start and stop a jail with the service command. To stop the jail
service jail stop myjail

Another way to shut down a jail is to use the jexec command. First run
jls

which will list the jail, including its JID.
JID   	IP Address  	Hostname		Path   		
2     	192.168.1.51 	myjail.example.com	/jails/myjail 

The jail can then be shutdown with
jexec 2 kill -TERM -1

to /etc/rc.conf. Lately, I've found that this sometimes doesn't work. Again, assuming that the JID is 2, an alternative command, that has continued to work for me is
jail -r 11