This document describes how to configure a Linux system to run with an encrypted filesystem. I have not yet tried to install a system with the root filesystem on an encrypted device. If I ever do, I will post the results here, but until then, you could try to combine information from this file with the Loopback Root Filesystem HOWTO.
So this is merely a description of how I managed to get things set up. The system was RedHat 7.3 and the kernel was 2.4.20. If things do not work for you, I probably cannot help you.
The text presented here describes how to set up an encrypted filesystem using the crypto loopback. Since kernel 2.6 a much better way of achieving this is by using the device-mapper crypto support which is in the kernel. The device-mapper is a system that takes one block-device (such as /dev/hdh1), adds a layer of encryption and presents this as another block-device. Of course if you really want to have your encrypted data in a file residing on the filesystem (instead of using an entire hard-drive partition), you would still need the loopback driver to turn your file into a block-device. On the other hand, if you want to encrypt an entire hard-drive partition, the device-mapper handles the job nicely, and is much more clean compared to the cryptoloop (which has some issues, especially with journalling file systems).
I have not yet had the time to describe setting up an encrypted filesystem with dmcrypt, but I would still suggest you check out this option if you want an encrypted filesystem. If I get arround to writing an article about dmcrypt, I will post it here.
It seems the international patches finally made it into the vanilla kernel (at least as of 2.4.22). I have not yet testet this, and I guess it uses the original Cipher-AES, not the pentium-optimized version, so you might still want to get a patch to get the best performance.
I assume the kernel-source is in /usr/src/linux/. It may be a good idea to fetch a stock kernel from kernel.org or some mirror-site, to be sure the patches will apply cleanly.
Change to the /usr/src/ directory
[root@crypto root]# cd /usr/src/
Now use wget, ncftp or a browser or something to get the appropriate kernel patches. Since we are using a 2.4.20-kernel, we are fetching the patch-int-2.4.20.1.bz2.
Now, we apply the patch:
[root@crypto src]# cd linux/ [root@crypto linux]# bzcat ../patch-int-2.4.20.1.bz2 | patch -p1 patching file arch/alpha/config.in patching file arch/arm/config.in patching file arch/cris/config.in patching file arch/i386/config.in patching file arch/ia64/config.in patching file arch/m68k/config.in patching file arch/mips/config-shared.in patching file arch/parisc/config.in patching file arch/ppc/config.in patching file arch/ppc64/config.in patching file arch/s390/config.in patching file arch/s390x/config.in patching file arch/sh/config.in patching file arch/sparc/config.in patching file arch/sparc64/config.in patching file crypto/ciphers/cipher-3des.c patching file crypto/ciphers/cipher-aes.c patching file crypto/ciphers/cipher-blowfish.c patching file crypto/ciphers/cipher-blowfish_old.c patching file crypto/ciphers/cipher-cast5.c patching file crypto/ciphers/cipher-des.c patching file crypto/ciphers/cipher-dfc.c patching file crypto/ciphers/cipher-gost.c patching file crypto/ciphers/cipher-idea.c patching file crypto/ciphers/cipher-mars.c patching file crypto/ciphers/cipher-null.c patching file crypto/ciphers/cipher-rc5.c patching file crypto/ciphers/cipher-rc6.c patching file crypto/ciphers/cipher-serpent.c patching file crypto/ciphers/cipher-twofish.c patching file crypto/ciphers/Config.help patching file crypto/ciphers/Config.in patching file crypto/ciphers/gen-cbc.h patching file crypto/ciphers/gen-cfb.h patching file crypto/ciphers/gen-cipher.h patching file crypto/ciphers/gen-ctr.h patching file crypto/ciphers/gen-ecb.h patching file crypto/ciphers/gen-rtc.h patching file crypto/ciphers/Makefile patching file crypto/Config.help patching file crypto/Config.in patching file crypto/cryptoapi.c patching file crypto/digests/Config.help patching file crypto/digests/Config.in patching file crypto/digests/digest-md5.c patching file crypto/digests/digest-ripemd160.c patching file crypto/digests/digest-sha1.c patching file crypto/digests/digest-sha256.c patching file crypto/digests/digest-sha384.c patching file crypto/digests/digest-sha512.c patching file crypto/digests/gen-hash.h patching file crypto/digests/gen-hmac.h patching file crypto/digests/Makefile patching file crypto/drivers/Config.help patching file crypto/drivers/Config.in patching file crypto/drivers/cryptoloop.c patching file crypto/drivers/ipsec_dev.c patching file crypto/drivers/ipsec_sa.c patching file crypto/drivers/ipsec_sa.h patching file crypto/drivers/Makefile patching file crypto/Makefile patching file Documentation/Configure.help patching file Documentation/cryptoapi/AUTHORS patching file Documentation/cryptoapi/cryptoapi.txt patching file Documentation/cryptoapi/cryptoloop.txt patching file Documentation/cryptoapi/README patching file include/linux/crypto.h patching file include/linux/wordops.h patching file Makefile
We will also need one of the loop-drivers. In this case we are choosing the loop-jari-patch:
[root@crypto linux]# patch -p1 <../loop-jari-2.4.20.0.patch patching file drivers/block/loop.c patching file include/linux/loop.h
Since we will be using the AES-cipher, we will use the optimized version cipher-aes-i586:
[root@crypto linux]# cd .. [root@crypto src]# tar -xvjf cipher-aes-i586-0.3.tar.bz2 [root@crypto src]# cd cipher-aes-i586-0.3 [root@crypto cipher-aes-i586-0.3]# make cipher-aes-i586.c
It now seems we are ready to configure the new kernel:
[root@crypto cipher-aes-i586-0.3]# cd /usr/src/linux [root@crypto linux]# make menuconfig
Select the following kernel options:
Code maturity level options ---> [*] Prompt for development and/or incomplete code/drivers Block devices ---> <*> Loopback device support Cryptography support (CryptoAPI) ---> [*] Cipher Algorithms <*> AES (aka Rijndael) cipher (NEW) <*> Serpent cipher (NEW) [*] Digest Algorithms <*> MD5 digest (NEW) <*> SHA512 digest (NEW) [*] Crypto Devices <*> Loop Crypto support (NEW)
Now, let's go on and compile the kernel:
[root@crypto linux]# make dep && make clean && make && \ make install modules && make modules_install
Go have a cup/glass of [Insert Your Favourite Caffeinated Beverage Here].
Now, let's reboot the system with the new kernel.
[root@crypto linux]# reboot
After bootin up, check that the cryptoapi was successfully loaded:
[root@crypto root]# dmesg | grep crypto cryptoapi: loaded cryptoapi: Registered aes-ecb (0) cryptoapi: Registered aes-cbc (65536) cryptoapi: Registered aes-cfb (131072) cryptoapi: Registered aes-ctr (262144) cryptoapi: Registered aes-rtc (524288) cryptoapi: Registered serpent-ecb (0) cryptoapi: Registered serpent-cbc (65536) cryptoapi: Registered serpent-cfb (131072) cryptoapi: Registered serpent-ctr (262144) cryptoapi: Registered serpent-rtc (524288) cryptoapi: Registered md5 (0) cryptoapi: Registered sha512 (0) cryptoloop: loaded
We will now need to recompile the util-linux-package with the appropriate patches applied. I will be downloading the source-RPM from redhat-updates, but since some extra options have been added to this package by redhat it is not easy to apply the official util-linux-patch. I have gone through the work of merging the two patches, so I have created the patch util-linux-2.11n-rh73-mho.patch, which I have used. This may very well not work with other versions of the utils-linux-package, but on the other hand, you could just get the official sources and use that.
NOTE: You should be aware that the util-linux-package contains utilities vital for the operation of a Linux-system. As Ryan puts it in the Loopback Encrypted Filesystem HOWTO: "If you don't carefully edit the MCONFIG file before compiling these sources have a boot disk and/or shotgun ready because your system will be quite confused".
So why would anyone want to use the RPM-approach? Well, it will simply ensure that the version I get installed will be appropriately configured for my system. This is nice if you don't own a shotgun. The downside is that you have to be more aware to avoid updating the RPM-package by accident at a later time.
[root@crypto root]# cd /usr/src/redhat/SRPMS/
Use the file-transfer-utility of your choice to get the source-rpm, either from your favourite RedHat mirror site, or perhaps from a redhat source CD-ROM.
Now we INSTALL the package, although it is a source-rpm. This will extract the sources:
[root@crypto SRPMS]# rpm -i util-linux-2.11n-12.7.3.src.rpm
Put the patch util-linux-2.11n-rh73-mho.patch in /usr/src/redhat/SOURCES/:
[root@crypto SRPMS]# cd ../SOURCES/ [root@crypto SOURCES]# wget http://symlink.dk/stuff/util-linux-2.11n-rh73-mho.patch
Now, we need to edit the spec-file:
[root@crypto SRPMS]# cd ../SPECS/ [root@crypto SPECS]# vi util-linux.spec
Add a line after the last "Patch???: ..."-line:
Then go to the last line in the %prep-stage (just before %build) and insert:
Now, we should be ready to compile the packages:
[root@crypto SPECS]# rpm -bb --target i686 util-linux.spec
If everything goes well, you will end up with binary packages in /usr/src/redhat/RPMS/i686/. Now install the new packages. Remember that we have already installed the "same" version of the packages, so we need to force rpm to update to the new ones:
[root@crypto SPECS]# cd /usr/src/redhat/RPMS/i686/ [root@crypto i686]# rpm -Uvh --force losetup-2.11n-12.7.3.i686.rpm \ util-linux-2.11n-12.7.3.i686.rpm mount-2.11n-12.7.3.i686.rpm
Now the installation is finished, and we go on to the configuration. First we create a mountpoint for the encrypted filesystem:
[root@crypto i686]# cd /mnt/ [root@crypto mnt]# mkdir crypt
Then we edit /etc/fstab, adding this line:
Of course you can use other filesystems on your encrypted device, but i prefer ext3.
Now, we will test the system. First we create a file for the encrypted filesystem to reside on.
[root@crypto mnt]# cd [root@crypto root]# dd bs=1M count=100 if=/dev/urandom of=crypto.dat
This will create a 100MB file of random data as /root/crypto.dat. If you are only testing you may want to use if=/dev/zero instead, which will be much faster. But for the final system it is important that the file contains random data so one cannot find out which parts of the filesystem is used by looking at the file.
Now, we set up the loop-back device:
[root@crypto root]# losetup -e aes -k 128 /dev/loop0 /root/crypto.dat Password:
The password entered here is critical for the encrypted filesystem. Remember that if you type a different password at some point, the underlying filesystem will not be visible. This will cause mount to complain when we try to mount the filesystem. In that case stop the loop-device (by issuing losetup -d /dev/loop0) and try again.
You can of course use other key-sizes. I will try to make some benchmarks with different ciphers and keysizes.
Now, back to our test. Create a filesystem on the newly created device:
[root@crypto root]# mke2fs -j -m 0 /dev/loop0
The option -j tells mke2fs to make a journalling node, which effectively makes the filesystem ext3.
Option -m 0 means that 0% of the space should be reserved for root. On this filesystem that is not needed.
Now, you should be able to mount the filesystem:
[root@crypto root]# mount /mnt/crypt/ [root@crypto root]# df | grep crypt /dev/loop0 99150 4127 95023 5% /mnt/crypt
Try creating a file on the new filesystem:
[root@crypto root]# touch /mnt/crypt/foo
For further testing, we can try to unmount:
[root@crypto root]# umount /mnt/crypt/
Stop the loop-device:
[root@crypto root]# losetup -d /dev/loop0
Then try to setup the loop again:
[root@crypto root]# losetup -e aes -k 128 /dev/loop0 /root/crypto.dat Password:
Try to supply a wrong password. Notice that you do not get any errors. This is because losetup does not know that anything is wrong.
Now try to mount the filesystem:
[root@crypto root]# mount /mnt/crypt/ mount: wrong fs type, bad option, bad superblock on /dev/loop1, or too many mounted file systems
NOTE: Could this be a bug in mount? We are trying to mount /dev/loop0, but mount complains about loop1. Oh, well.
Stop the crypto-loop again:
[root@crypto root]# losetup -d /dev/loop0
And restart, this time with the correct password:
[root@crypto root]# losetup -e aes -k 128 /dev/loop0 /root/crypto.dat
Now, mount should succeed.
[root@crypto root]# mount /mnt/crypt/
Congratulations. It seems everything is working.
Good. Now, we will try setting the system up on a harddisk partition instead of the file-based setup.
Since this disk has not been used before, we will create a new partition to hold our filesystem:
[root@crypto root]# fdisk /dev/hdh Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel Building a new DOS disklabel. Changes will remain in memory only, until you decide to write them. After that, of course, the previous content won't be recoverable. The number of cylinders for this disk is set to 14946. There is nothing wrong with that, but this is larger than 1024, and could in certain setups cause problems with: 1) software that runs at boot time (e.g., old versions of LILO) 2) booting and partitioning software from other OSs (e.g., DOS FDISK, OS/2 FDISK) Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First cylinder (1-14946, default 1): Using default value 1 Last cylinder or +size or +sizeM or +sizeK (1-14946, default 14946): Using default value 14946 Command (m for help): t Partition number (1-4): 1 Hex code (type L to list codes): 93 Changed system type of partition 1 to 93 (Amoeba) Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks. [root@crypto root]#
The partition-type is not important in this case, so I simply chose something that I will not confuse with other systems at a later time.
NOTE: For the final setup, you should run
# dd bs=16k if=/dev/urandom of=/dev/hdh1
in order to fill the partition with random data. But with the 120GB disk we are using here, this would take a very-long-time(tm), and since we are only interrested in testing the system performance (not the security), we will skip this step for now. During configuration some more light was shedded on this subject. First of all I calculated (based on dd'ing 2GB from /dev/urandom) that filling the entire disk with random data would take about 1.5 days on this system. Since I wanted to get on with it, I came up with the following reasoning: If what we are trying to avoid is the ability for people to see whether a particulair part of the disk has been used or not, another approach to achieving this would be simply to setup the device and then fill it up. Especially in cases where almost the entire filesystem is going to be occupied anyways, there is really no need to fill up the entire device with random date just to overwrite most of it with our encrypted data afterwards. We should still be aware that the penalty of doing this is that the part that has to be filled up will be going through the encryption layer and thus be far more CPU-intensive. I have also considered if one could then simply use dd if=/dev/zero on the encrypted device to create the randomness, but (without knowing a lot about AES), I suspect that having a large portion of known data on the encrypted volume, and in particulair at a known location would make it easier to brute-force the encryption key. On the other hand, one might argue that there are already several pieces of data with this property, such as the filesystem superblocks etc. However, compared to the amount of zeros we might put at the end of a filesystem, this should be fairly little. Furthermore the gigabytes of zeros would mean not only known data, but also repeated data in several blocks, giving an attacker better odds with brute-force encryption key-guessing. Still, I am not entirely sure how this should be done in the correct way. I guess one could use some kind of compromise: We dont want to fill the device with data from /dev/zero (weakens encryption), and we dont want to fill it with data from /dev/urandom (takes too long time), but as such, any data will do, so simply copying some of the files on the disk to another location should meet our requirements. Still these are just crazy speculations and I will leave it up to you to decide what level of security you want to implement.
Perhaps you don't even think /dev/urandom is "random" enough :-)
Now setup the loopback on the device /dev/hdh1:
[root@crypto root]# losetup -e aes -k 128 /dev/loop0 /dev/hdh1 Password:
Again we create a filesystem:
[root@crypto root]# mke2fs -j -m 0 /dev/loop0
This takes some time, but when it is done we can do:
[root@crypto root]# mount /mnt/crypt/ [root@crypto root]# cd /mnt/crypt/ [root@crypto crypt]# ls -l total 16 drwx------ 2 root root 16384 May 30 00:27 lost+found [root@crypto crypt]# df -h | grep crypt /dev/loop0 113G 33M 112G 1% /mnt/crypt
As you can see, we now have a 112GB encrypted filesystem.
Now, I will do some testing. All the tests in this section were carried out on a Celeron 333MHz, with 64MB RAM. The disk is a Maxtor 120GB connected to a Promise Ultra100 controller (PCI). For testing I used Bonnie v1.4. The test file was 1024MB in all cases. Be aware that the CPU-usage shown is for the bonnie-process. In other words you cannot expect a CPU-load of 15% when using a 256 bit AES. In fact the CPU-load is 100% in the test cases, since the bottleneck is the encryption layer. After the test results, I have a little comment regarding the CPU usage of the encryption layer.
Bare filesystem (ie. no loopback device):
---Sequential Output (nosync)--- ---Sequential Input-- --Rnd Seek- -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --04k (03)- Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec %CPU barefs 1*1024 4428 99.0 30177 66.4 14359 37.9 4372 98.8 35605 50.0 94.4 2.0
Loopback filesystem without encryption:
---Sequential Output (nosync)--- ---Sequential Input-- --Rnd Seek- -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --04k (03)- Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec %CPU none 1*1024 4423 98.9 29929 65.3 14448 38.4 4363 98.7 35613 51.7 94.6 1.9
Serpent, 128 bit keysize:
---Sequential Output (nosync)--- ---Sequential Input-- --Rnd Seek- -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --04k (03)- Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec %CPU Serpen 1*1024 2551 53.5 5463 7.4 2711 5.7 2463 56.1 5247 6.4 90.5 1.9
Old AES, 128 bit keysize:
---Sequential Output (nosync)--- ---Sequential Input-- --Rnd Seek- -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --04k (03)- Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec %CPU oldaes 1*1024 2872 62.0 7080 11.5 3502 8.2 2767 63.0 6757 9.5 93.7 2.1
New AES (cipher-aes-i586), 128 bit keysize:
---Sequential Output (nosync)--- ---Sequential Input-- --Rnd Seek- -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --04k (03)- Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec %CPU aes128 1*1024 3379 75.1 11163 19.0 5677 13.4 3292 75.0 10759 13.6 93.7 2.2
New AES (cipher-aes-i586), 256 bit keysize:
---Sequential Output (nosync)--- ---Sequential Input-- --Rnd Seek- -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --04k (03)- Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec %CPU aes256 1*1024 3162 70.3 9071 15.5 4663 10.4 3088 70.1 8967 11.4 92.3 1.7
The final test was made with a somewhat different setup. In this case we used cipher-aes-i586, a 256 bit keysize and the encrypted filesystem stored on a raid-5 consisting of 4 disks. The system was otherwise the same. In this case we see how the performance drops slightly.
---Sequential Output (nosync)--- ---Sequential Input-- --Rnd Seek- -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --04k (03)- Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec %CPU aes2md 1*1024 2907 63.7 7155 12.1 3935 9.5 3051 70.2 8568 12.4 132.9 3.9
Aparently the new AES-implementation (cipher-aes-i586) gives approximately 50% performance improvement. This is definetely worth considering. As you can see, the keysize can be changed from 128 to 256 while still maintaining a considerable performance increase if using the new AES-implementation as opposed to the old one.
Some further tests I carried out with cipher-aes-i586 and a keysize of 256 bits. When rewriting loop0 is using ~83% CPU, while Bonnie only gets ~10%. During block-functions (both read and write the figure is ~80%. There is no doubt that the encryption layer is the bottleneck in this test scenario, so we should end up with a CPU-usage of 100% when conducting the tests. Still, the benchmarks show that it is in fact possible to run a file-server on an encrypted filesystem without major performance problems (a 100mbit network interface will only sustain 6MB/s anyways), even on this somewhat old hardware.