Before continuing, be sure to read part 1 of this blogpost.
What you will need
1 at least 1GB USB stick for the FreeBSD installer image
1 or more xGB USB sticks for the boot files and encryption keys
You should create multiple copies of the USB stick that holds the boot files and encryption keys. If you lose the stick or the data gets corrupted and you don’t have another copy, all your data stored on the encrypted disks is lost.
Booting the FreeBSD Installer
I’m using a USB stick with the FreeBSD 10 memstick image to boot into the FreeBSD installer. See here for a Mac OS X guide on how to get the memstick image onto a USB stick.
Now after your system finishes booting from the USB stick, it should present you with a blue, text-based installer giving you three options:
We will start by dropping into the shell and run su - to get a root shell.
I will assume that your server is connected to your LAN during the installation.
That way we can start an SSH daemon from the installer image and use our Mac or PC to enter the setup commands or copy files to the server.
So, on the shell on your server, run
to identify your network interface name. In my case it’s bge0.
to get an IPv4 address, in my case 192.168.1.45.
If your LAN does not offer you an IP address via DHCP, run man ifconfig and read up on how to configure a network card manually.
Let’s say your server is now connected to your LAN and has an IPv4 address. We can now start an SSH daemon by running:
The root password you are asked to enter is just for the installer; it’s not the root password you will use later for your installation.
Now to login to the installer image by running ssh root@<ip-address-of-your-server> on your Mac or PC.
This guide should also work if you enter all the commands on the command line yourself, but doing it over SSH is more convenient.
Identifying your disks
Now let’s see which storage devices are connected to your server:
In my case I have four hard disks and one USB stick (da0)
We’ll create two zpools, one for the OS installation and for data.
In my case I’ll use the disks ada0 and ada1 for the OS and ada2 and ada3 for my data.
The device names are probably different on your system. Please consult FreeBSD Disk device names to find out how FreeBSD names attached storage devices.
We will start by writing random data to the two operating system disks.
This will take a very long time, depending on how big your disks are.
Now let’s start partitioning the disks. This is what the layout will look like in the end:
| Hard Disk Device | Partition 1 | Partition 2 |
| ada0 | ada0p1 freebsd-swap | ada0p2 freebsd-zfs OS installation |
| ada1 | ada1p1 freebsd-swap | ada1p2 freebsd-zfs OS installation |
As I said, we are going to store the bootcode, kernel and keyfiles on a USB stick, so there is no need for a boot partition.
You might also notice that we’ll create separate swap partitions and won’t use a ZVOL for swap. Here is why.
The next steps will destroy any data on your drives:
To better understand the following commands it would be a good idea to read the manpage of gpart: man gpart.
Clean the drives of existing partition tables:
If you get a message like gpart: arg0 'ada2': Invalid argument, that’s fine and you can ignore it. It just means that there was no partition table on the disk anyway.
Create a GPT partition table on each disk:
Nowadays, disks (especially very large ones) use a sector format called “Advanced Format”. Long story short, even if you don’t have Advanced Format disks, we are going to align the partitions with 4K sectors. This blog post explains it quite well in a ZFS/FreeBSD context.
Next we are going to create the swap partition. You will have to choose how big your swap partition is going to be. I’ll create a swap partition of the same size as the memory I’m planning on having in the server, so I’ll choose 16GB. You might have different needs. https://wiki.freebsd.org/SystemTuning#Swap suggests the following:
Size swap space to approximately twice the size of main memory on systems with less than 4GB RAM and the size of main memory for systems with more than 4GB. If in doubt, allocate more swap; allocating insufficient swap is far worse than allocating too much. If the system has multiple disks, reduce swap I/O contention by spreading swap across the disks, ideally in equally sized partitions.
We are using labels so we can more easily replace hardware later.
Now for the OS partition. Because I still have a few spare 160GB drives, I’m only going to use 140GB for the OS partition, so that if one of the current disks fails, I can easily replace it with one of my 160GB spares.
If you want to use all the remaining space on your disks (which is what I would normally do), run this instead:
What have we done?
We will come back to the two empty data disks later.
SWAP Raid 1
Since we have two OS disks that…
We are going to use GELI for the encryption. Basically it will encrypt each sector transparently. ZFS itself doesn’t know it is being encrypted:
| ZFS |
| GELI Encryption |
| Physical Hard Disk |
Load the kernel modules that are needed for GEOM and ZFS:
Operating System Partitions
Insert the USB stick that you plan to use as a boot device. Mine is da1.
You will usually see some debug output about the just-connected USB stick on the server shell (not via SSH). It should also show the device name.