Posts tagged ·

cryptsetup

·...

QNAP compatible encrypted disks

Comments Off

I’ve previously written about encryption on the QNAP TS-431P NAS and basic cryptsetup usage. Since then, encryption standards and defaults have changed, and it is now easy to create an external encrypted disk which cannot be mounted by a QNAP NAS. The following shows how to work around the issues with cipher and ext4 journaling settings.

The first issue has to do with the default cipher algorithms on Ubuntu and QNAP. As of Ubuntu 16.10, the default cipher is Advanced Encryption Standard (AES) in xts-plain64 mode with a SHA256 hash. The default and supported encryption on the QNAP NAS is also AES, using “128-bit block size, with key sizes of 128, 192 or 256 bits”. However, the supported mode is cbc-essiv:sha256 with hash spec SHA1, as with older Ubuntu and Debian distributions. When trying to decrypt the drive on the NAS, you might see errors like “Failed to setup dm-crypt key mapping for device /dev/sdc1. Check that kernel supports aes-xts-plain64 cipher (check syslog for more info)” and in /var/log/storage_lib.log an error like “crypt: IV mechanism required”.

It should be noted that this is most likely not an issue with volumes created by the NAS itself on its internal drives, unless you start moving drives from one NAS box to another, which is probably not recommended in the first place.

To see the supported ciphers, both on a normal GNU/Linux distribution and the QNAP, use the following commands:

cat /proc/crypto
 
cryptsetup --help

Also, to see the currently used cipher and LUKS formatted volume, use the luksDump command:

cryptsetup luksDump /dev/sdX1

Formatting

If you start from scratch, it’s easy to work around the incompatibility. Follow the instructions in the cryptsetup basics article, but add the following options for cipher and hash function to the luksFormat command. Note that this will format and erase all data on the partition.

cryptsetup luksFormat --cipher aes-cbc-essiv:sha256 --hash sha1 /dev/sdX1

You might also consider ext3 over ext4, since the former seems better supported by the QNAP NAS at the time of writing. See below for further details.

Changing the cipher

If on the other hand, you discover the incompatibility a bit too late, and have already filled up the external disk with a lot of content, you’re not all out of luck. You’ll just have to decide which way you’d like to waste your time: You can transfer it all over to another disk, reformat, and then transfer back. It will take a few hours, and a bit of work. Or, you can change the encryption cipher on the existing volume, using the cryptsetup-reencrypt tool. However, you’ll probably have to wait multiple days while the whole disk is re-encrypted. On a 2 TB external disk over USB 2.0, it took about 35 hours to complete.

cryptsetup-reencrypt --cipher aes-cbc-essiv:sha256 --hash sha1 --key-file /tmp/keyfile --key-slot 0 /dev/sdX1

Notice that the command uses the same cipher and hash arguments above. However, it adds arguments for the key file to unlock the volume, and which key slot that file is linked to. This is necessary to avoid being asked about each and every password for each of the key slots. Of course, if you have only added a single password based key slot, these arguments can be skipped, and you’ll have to type the password once.

ext4 journaling compatibility

Once the encrypted volume can be opened, there might still be hurdles. The default settings for the ext4 journaling might also not be compatible with the QNAP NAS. At this point, I have to admit I lost interest in researching exactly what the cause was, and fired off multiple changes at once. The error when using the mount command was “mount: wrong fs type, bad option, bad superblock on /dev/mapper/sdc1, missing codepage or other error”.

The default features set from the Ubuntu 16.10 created ext4 formatted partition was:

dumpe2fs /dev/mapper/sdc1
...
Filesystem features: has_journal ext_attr resize_inode dir_index filetype extent 64bit flex_bg sparse_super large_file huge_file dir_nlink extra_isize metadata_csum

The following commands removed a few of them, changed to 32 bits, finished with a check:

tune2fs -O ^huge_file /dev/mapper/sdc1
tune2fs -O ^dir_nlink /dev/mapper/sdc1
tune2fs -O ^extra_isize /dev/mapper/sdc1
tune2fs -O ^metadata_csum /dev/mapper/sdc1
 
e2fsck -f /dev/mapper/sdc1
 
tune2fs -O ^64bit /dev/mapper/sdc1
resize2fs -s /dev/mapper/sdc1

In the end, the following features remained, and the volume mounted.

Filesystem features: has_journal ext_attr resize_inode dir_index filetype needs_recovery extent flex_bg sparse_super large_file uninit_bg

Comments Off

cryptsetup basics

Comments Off

Talking about encryption in the previous post, I realized there are a few details I keep having to look up. This is a collection of the Frequently Asked Questions about cryptsetup formatting and mounting.

Note: For all the following examples, the example device /dev/sdX is used. It’s a device and file which doesn’t exist, on purpose. When replacing with your own e.g. /dev/sda or similar, be careful!

Formatting a new physical drive

Before working with a new drive, it’s recommended to check for bad blocks, to confirm it’s not a DOA (Dead on Arrival). If it is, you might want to claim it on the warranty immediately to avoid losing data in the future.

This command will check for bad blocks, as well as fill the disk with random data to better hide the encrypted volume later:

badblocks -c 10240 -s -w -t random -v /dev/sdX

Next is the partition setup, where all you need is a new cleared (similar to unformatted, but actually cleared) partition. In the gparted UI it’s simply “New -> Cleared -> Apply”, while on the CLI it would go something like this, to create an optimally aligned, primary partition.

parted /dev/sdX mklabel gpt
parted -a optimal /dev/sdX mkpart primary '0%' '100%'

Now, coming to the encrypted volume, you could just use a passphrase, and skip the first line, or store a salted hashed password in a key-file. The benefit of the latter, is that it will generally be a more secure key, and yet you could re-created the keyfile if you lost it, assuming you remember both the password and the salt.

mkpasswd --m=sha-256 --salt='SOME_SALT' | tr -d '\n' > /tmp/key-file

cryptsetup luksFormat /dev/sdX1 /tmp/key-file
cryptsetup open /dev/sdX1 unenc --key-file /tmp/key-file

Notice the mapping name “unenc“, which can be anything of your choosing.

Finally, format and mount the drive. Here, the ext4 file-system is used, with 1% reserved for system

mkfs.ext4 -m 1 -O dir_index,filetype /dev/mapper/unenc
mount /dev/mapper/unenc /mnt/tmp

Creating an encrypted file volume

In some cases, it is useful to encrypt only a small part of the disk, or even move the encrypted container around. A loop device can create a filesystem inside a file residing on any file system, be it USB stick, network mount or local disk.

First, you will have to create an empty file. The dd command will copy zeros to the specified filename. The total size is block size times count, or 500 MB in this example:

dd if=/dev/zero of=myfile bs=1M count=500

Then establish the loopback. It will become available on /dev/loop0, and can be formatted and mounted like any other block device.

losetup /dev/loop0 mycryptfile

Now repeat the luksFormat and filesystem format commands from above:

cryptsetup luksFormat /dev/loop0
cryptsetup open /dev/loop0 mycrypt
mkfs.ext4 -m 1 /dev/mapper/mycrypt
mount /dev/mapper/mycrypt /mnt/tmp

Key managment

Most of the cryptsetup commands above have at least two options when dealing with the keyslot: A passphrase and a key file. Typically, a passphrase is typed in on the prompt when unlocking the partition or modifying the other keys, while a key file is supplied using the –key-file argument. In terms of security, the first is “something you know”, while the latter is “something you have”.

To list the active keyslots use the following command. It will work both on an open and closed partition.

cryptsetup luksDump /dev/sdX

To add a new key with a prompted password:

cryptsetup luksAddKey /dev/sdX

or a randomly generated key-file:
dd bs=512 count=4 if=/dev/urandom of=~/keyfile_for_sdX iflag=fullblock

cryptsetup luksAddKey /dev/sdX ~/keyfile_for_sdX

To erease one of the existing key-slots, assuming you have more than one.

cryptsetup luksKillSlot /dev/sdX <key slot number>

You might also want to backup the LUKS header, which includes the key-slots, so in case you overwrite existing keys, you can restore the header and unlock with the old keys. It should be noted, that this header will then be able to unlock the partition given any password or keyfile in its keyslots. So, even if you change a password, the old header can be restored and an old password used to unlock. Therefore, it should be considered a secret file and stored securely just as the key file.

cryptsetup luksHeaderBackup /dev/sdX --header-backup-file ~/header_for_sdX

Finally, you might need to wipe the whole encrypted volume. You can do this with the luksKillSlot command, or manually remove all keys, and then change or add the remaining one with a password or keyfile you later remove or forget. E.g. by generating a key-file on the RAM disk /dev/shm, and then rebooting to lose it.

Comments Off

QNAP TS-431P NAS

Comments Off

Tasked with setting up another NAS solution, I went for the simple 4-bay QNAP TS-431P, since the previous QNAP gave a good impression. This one does not have HDMI; in fact the only external ports are three USB 3 ports and two RJ-45 Gigabit Ethernet – no eSATA. Compared to its previous version, TS-431P has double the amount of RAM (for a total of 1 GB), and a slightly faster CPU. Software is as expected from QNAP.

The following describes the standard disk layout when using a single / stand alone disk, which still gets formatted as RAID + LVM, and optionally an encrypted partition.

Windows shares setup is covered at the end.

 

RAID and LVM

The QNAP NAS OS supports encryption, and I wanted to evaluate how secure this is in terms of failure. That is, if a disk fails, or the NAS itself fails, can you recover the data from the remaining disks. You can, but there are a few steps to watch out for.

First of all, even if each disk in the NAS is set up as “Single Disk / Stand Alone”, using no RAID, the NAS will still configure each partition on the separate disks as RAID partitions and in a LVM2 single volume group. That means you’ll need the Linux RAID and LVM tools and commands to mount. (Some useful discussion here).

General install, scan and list commands:

apt-get install mdadm lvm2

mdadm --assemble --scan
cat /proc/mdstat
lsblk

vgscan
lvs
lvscan
lvmdiskscan
lvdisplay

And to mount, use the example commands below.

Note: The device names and volume names will most certainly be different. Use the commands above to understand the layout of the disk you’re working with.

Also note: if the mdadm scan command does not make all the RAID partitions available, it could be due to an existing /etc/mdadm/mdadm.conf file. You could try to rename it to mdadm.conf.old, or append the RAID details with mdadm –detail –scan >> /etc/mdadm/mdadm.conf. See here for more.

mdadm --assemble --scan
lsbkl

vgscan
vgchange -ay vg1
lsblk

mount /dev/vg1/lv1 /mnt/tmp

That should mount the drive, however, if you are working with an encrypted drive, you’ll need one more step before the mount command works, so ignore the last line and continue reading.

 

Encryption

If you have followed the steps above, and type lsblk, part of the output will look something like this. It shows the layers so far: from the physical partition (sdb3) to the raid1 partition (md126), which contains two LVM logical volumes. In this case, the second is the LUKS encrypted main partition.

├─sdb3              8:19   0   3.6T  0 part  
│ └─md126           9:126  0   3.6T  0 raid1 
│   ├─vg288-lv545 254:1    0  37.2G  0 lvm   
│   └─vg288-lv2   254:2    0   3.6T  0 lvm   

So, we continue to decrypt, and mount it. Using cryptsetup luksDump, you can confirm that there is only one keyslot on the encrypted volume, which uses the paraphrase you typed in when installing the drive. However, the password is salted and MD5 hashed, so you have to generate a key-file with the new key. The salt is YCCaQNAP when using the mkpasswd tool, but encoded as $1$YCCaQNAP$ when calling the crypt library. Also make sure the key-file does not contain a newline.

cryptsetup luksDump /dev/vg288/lv2

mkpasswd --hash=md5 --salt='YCCaQNAP' | tr -d '\n' > /tmp/key-file
cryptsetup luksOpen /dev/vg288/lv2 unenc_lv2 --key-file /tmp/key-file

mkdir /mnt/tmp
mount /dev/mapper/unenc_lv2 /mnt/tmp
lsblk

You now have access to the data files on the drive.

Coming back to the original question: Is this a resilient way of storing files? There are certainly a lot of layers, and although they each are well established technologies, they add complexity. Especially in the scenario when you would need to do the recovery it adds additional stress. Ideally, a single partition, no RAID, no LVM could be used. However, it seems that is not possible with the stock QNAP OS, since it will format any drive which is added to the NAS in its own way, including the RAID + LVM stack. In fact, this warning from the user manual is worthing taking careful note of:

Caution: Note that if you install a hard drive (new or used) which has never been installed on the NAS before, the hard drive will be formatted and partitioned automatically and all the disk data will be cleared.

 

Windows shared folders

The Windows sharing is easier to set up, but not without hurdles. On the local network, it typically will work out of the box when you point Windows Explorer to \\NAS_DOMAIN. If you need to connect across a firewall, you’ll have to open or forward at the minimum TCP 139,445, but possibly more ports on TCP and UDP.

The problem is that when sharing these ports cross the Internet, you will very likely run into other firewalls. ISP might block the default 139 or 445 ports. Although it is possible to port-forward to non-default ports, and this will work on Mac and Android, Windows will not accept it. A work-round if all else fails is therefore to set up a VPN or tunnel. Using SSH, this can easily be done with:

ssh -L 0.0.0.0:139:qnap:139 -L 0.0.0.0:445:qnap:445 admin@remotehost

Here it is assumed the NAS has DNS “qnap” on its local network, otherwise, replace with it’s IP. You might also want to forward 8080, forward SSH on a different port (e.g. 2222), as well as keep it running with autossh:

autossh -M 12340 -f -N -p 2222 -L 0.0.0.0:139:qnap:139 -L 0.0.0.0:445:qnap:445 -L 0.0.0.0:8080:qnap:8080 admin@remotehost

Finally, if using only Windows machines to connect to the shares, there is the option of combining multiple shares into one. However, if other OSes also connect, you probably want to skip that.

Comments Off