Posts tagged ·

NAS

·...

QNAP TS-431P NAS

Comments Off

Tasked with setting up another NAS solution, I went for the simple 4-bay QNAP TS-431P, since the previous QNAP gave a good impression. This one does not have HDMI; in fact the only external ports are three USB 3 ports and two RJ-45 Gigabit Ethernet – no eSATA. Compared to its previous version, TS-431P has double the amount of RAM (for a total of 1 GB), and a slightly faster CPU. Software is as expected from QNAP.

The following describes the standard disk layout when using a single / stand alone disk, which still gets formatted as RAID + LVM, and optionally an encrypted partition.

Windows shares setup is covered at the end.

 

RAID and LVM

The QNAP NAS OS supports encryption, and I wanted to evaluate how secure this is in terms of failure. That is, if a disk fails, or the NAS itself fails, can you recover the data from the remaining disks. You can, but there are a few steps to watch out for.

First of all, even if each disk in the NAS is set up as “Single Disk / Stand Alone”, using no RAID, the NAS will still configure each partition on the separate disks as RAID partitions and in a LVM2 single volume group. That means you’ll need the Linux RAID and LVM tools and commands to mount. (Some useful discussion here).

General install, scan and list commands:

apt-get install mdadm lvm2

mdadm --assemble --scan
cat /proc/mdstat
lsblk

vgscan
lvs
lvscan
lvmdiskscan
lvdisplay

And to mount, use the example commands below.

Note: The device names and volume names will most certainly be different. Use the commands above to understand the layout of the disk you’re working with.

Also note: if the mdadm scan command does not make all the RAID partitions available, it could be due to an existing /etc/mdadm/mdadm.conf file. You could try to rename it to mdadm.conf.old, or append the RAID details with mdadm –detail –scan >> /etc/mdadm/mdadm.conf. See here for more.

mdadm --assemble --scan
lsbkl

vgscan
vgchange -ay vg1
lsblk

mount /dev/vg1/lv1 /mnt/tmp

That should mount the drive, however, if you are working with an encrypted drive, you’ll need one more step before the mount command works, so ignore the last line and continue reading.

 

Encryption

If you have followed the steps above, and type lsblk, part of the output will look something like this. It shows the layers so far: from the physical partition (sdb3) to the raid1 partition (md126), which contains two LVM logical volumes. In this case, the second is the LUKS encrypted main partition.

├─sdb3              8:19   0   3.6T  0 part  
│ └─md126           9:126  0   3.6T  0 raid1 
│   ├─vg288-lv545 254:1    0  37.2G  0 lvm   
│   └─vg288-lv2   254:2    0   3.6T  0 lvm   

So, we continue to decrypt, and mount it. Using cryptsetup luksDump, you can confirm that there is only one keyslot on the encrypted volume, which uses the paraphrase you typed in when installing the drive. However, the password is salted and MD5 hashed, so you have to generate a key-file with the new key. The salt is YCCaQNAP when using the mkpasswd tool, but encoded as $1$YCCaQNAP$ when calling the crypt library. Also make sure the key-file does not contain a newline.

cryptsetup luksDump /dev/vg288/lv2

mkpasswd --hash=md5 --salt='YCCaQNAP' | tr -d '\n' > /tmp/key-file
cryptsetup luksOpen /dev/vg288/lv2 unenc_lv2 --key-file /tmp/key-file

mkdir /mnt/tmp
mount /dev/mapper/unenc_lv2 /mnt/tmp
lsblk

You now have access to the data files on the drive.

Coming back to the original question: Is this a resilient way of storing files? There are certainly a lot of layers, and although they each are well established technologies, they add complexity. Especially in the scenario when you would need to do the recovery it adds additional stress. Ideally, a single partition, no RAID, no LVM could be used. However, it seems that is not possible with the stock QNAP OS, since it will format any drive which is added to the NAS in its own way, including the RAID + LVM stack. In fact, this warning from the user manual is worthing taking careful note of:

Caution: Note that if you install a hard drive (new or used) which has never been installed on the NAS before, the hard drive will be formatted and partitioned automatically and all the disk data will be cleared.

 

Windows shared folders

The Windows sharing is easier to set up, but not without hurdles. On the local network, it typically will work out of the box when you point Windows Explorer to \\NAS_DOMAIN. If you need to connect across a firewall, you’ll have to open or forward at the minimum TCP 139,445, but possibly more ports on TCP and UDP.

The problem is that when sharing these ports cross the Internet, you will very likely run into other firewalls. ISP might block the default 139 or 445 ports. Although it is possible to port-forward to non-default ports, and this will work on Mac and Android, Windows will not accept it. A work-round if all else fails is therefore to set up a VPN or tunnel. Using SSH, this can easily be done with:

ssh -L 0.0.0.0:139:qnap:139 -L 0.0.0.0:445:qnap:445 admin@remotehost

Here it is assumed the NAS has DNS “qnap” on its local network, otherwise, replace with it’s IP. You might also want to forward 8080, forward SSH on a different port (e.g. 2222), as well as keep it running with autossh:

autossh -M 12340 -f -N -p 2222 -L 0.0.0.0:139:qnap:139 -L 0.0.0.0:445:qnap:445 -L 0.0.0.0:8080:qnap:8080 admin@remotehost

Finally, if using only Windows machines to connect to the shares, there is the option of combining multiple shares into one. However, if other OSes also connect, you probably want to skip that.

Comments Off

Add-on development for Kodi

Comments Off

On the heels of the QNAP NAS setup notes, here’s a fun integration with my home automation system for living room lights.

The idea is to send the same commands from the Kodi app as the custom Android app does to the Arduino controlled relays. Before the movie starts, the lights go off. I’ll skip the details of that code, but point to a few useful pages to get started. It’s simple.

The Kodi Add-on documentation is good. To get started, you need at least two files: The addon.xml configuration, and your Python script, e.g. myaddon.py. These have to be in a directory on the format script.name (more in the linked documentation) and zip-ed in a ZIP file which does not use use compression, as seen below. This zip-file can now be copied to the NAS, and installed from Kodi.
zip -0 -r myaddon.zip script.myaddon

One gottcha is that the addon.xml file cannot contain a final new-line. At least some people have reported that causing a install error.

For an easy way to get started, look at the Hello World add-on example, as well as its source code. It doesn’t get easier than that.

Comments Off

QNAP NAS and autofs auto mount

Comments Off

After considering multiple options to cover a HTPC and a NAS, I finally went with the combined “living room” QNAP HS-251+ NAS earlier this year. I’ll leave the reviews to other sites, and just summarize the main features:

  • 2 bay 3.5″ or 2.5″ HDD or SSD
  • Intel Celeron 2GHz Quad core; 2 GB DDR3 RAM
  • 2x 1Gb RJ-45 ports; 2x USB 2.0; 2x USB 3.0
  • 1x HDMI
  • Fan-less
  • Simple remote control
  • Multiple in-house and external apps
  • Good support for Kodi (aka. XBMC)
  • Linux based 32-bit OS, with most common tools and network services available, including SSHD, NFS, SMB, FTPS, rsync.

NFS

Setting up NFS shares on the NAS side is straight forward through the web based UI under “Control Panel”. You probably want to create one or more users which match your own client (e.g. laptop) user, and possibly also related group. All this can be achieved through the UI, however, for setting specific user IDs, SSH into the NAS (using the admin account) and edit /etc/passwd and /etc/group. If the IDs are changed, you’ll also have to update /mnt/HDA_ROOT/.config/nfssetting.

/etc/passwd
david:x:1001:8008:Linux User,,david,:/share/homes/david:/bin/sh
john:x:1000:8008:Linux User,,john,:/share/homes/john:/bin/sh

/etc/group
foobar:x:8008:david,john

The reason for changing the user or group IDs manually might be to match existing IDs on the client machines. In that case, you might also have to provide this option, to make those IDs are actually used by the NAS. This setting is not permanent, so if the NAS is restarted frequently, you might consider a start-up script solution.
echo N > /sys/module/nfs/parameters/nfs4_disable_idmapping

The two relevant configuration files for the NFS setup on the NAS are /etc/exports and /mnt/HDA_ROOT/.config/nfssetting. They will be automatically configured by the UI, however some manual tweaking might be needed. I ended up with something like this, for two machines (with DNS names”laptop”, “desktop” – you can also use IP address) and two shares (“pictures”, “video”). The user (UID) and group (GID) ids will match what’s seen in the /etc/passwd and /etc/group files above.

/etc/exports

"/share/CACHEDEV1_DATA/pictures" laptop(rw,async,no_subtree_check,insecure,no_root_squash) desktop(rw,async,no_subtree_check,insecure,no_root_squash)
"/share/CACHEDEV1_DATA/video" laptop(rw,async,no_subtree_check,insecure,no_root_squash) desktop(rw,async,no_subtree_check,insecure,no_root_squash)

/mnt/HDA_ROOT/.config/nfssetting
"/share/CACHEDEV1_DATA/Public" *(rw,async,no_root_squash,insecure)
[Global]
Version = 4.2.0
[Access]
/share/CACHEDEV1_DATA/Public = FALSE
/share/CACHEDEV1_DATA/pictures = TRUE
/share/CACHEDEV1_DATA/video = TRUE
[AllowIP]
/share/CACHEDEV1_DATA/Public = *
/share/CACHEDEV1_DATA/pictures = laptop,desktop
/share/CACHEDEV1_DATA/video = laptop,desktop
[Permission]
/share/CACHEDEV1_DATA/Public = rw
/share/CACHEDEV1_DATA/pictures = rw,rw
/share/CACHEDEV1_DATA/video = rw,rw
[SquashOption]
/share/CACHEDEV1_DATA/Public = no_root_squash
/share/CACHEDEV1_DATA/pictures = no_root_squash,no_root_squash
/share/CACHEDEV1_DATA/video = no_root_squash,no_root_squash
[AnonUID]
/share/CACHEDEV1_DATA/Public = 65534
/share/CACHEDEV1_DATA/pictures = 1001,1000
/share/CACHEDEV1_DATA/video = 1001,1000
[AnonGID]
/share/CACHEDEV1_DATA/Public = 65534
/share/CACHEDEV1_DATA/pictures = 8008,8008
/share/CACHEDEV1_DATA/video = 8008,8008

After making any changes to the NFS config, restart the service:
/etc/init.d/nfs restart

Client side and autofs

On the client, e.g. laptop or desktop, you’d want to point your NFS mount configuration to the shares created above. However, since either NAS or more likely personal machine will be rebooted, it is useful to configure this through autofs instead of the traditional /etc/fstab config. That way, the shares will be mounted and re-mounted on demand. It will also avoid long waits at boot and shutdown of the client machines.

First, make sure the NFS and autofs packages are installed:
apt-get install portmap nfs-common autofs cifs-utils

Edit /etc/auto.master and add the following line, which specify local mount point, and specific configuration files. Note that that has to match with your setup, so you might want to change the names here. As long as the /mnt directory and config file match, you can use whatever names you like.

/etc/auto.master
/mnt/qnap /etc/auto.qnap

The share specific configuration is then added in the file referenced above. It assumes you’ve named the shares on the NAS “pictures” and “video”. It also assumes the DNS name of the NAS is “qnap” (or you can use an IP here). Finally, it assumes that the shared group is called “foobar”, which should match the GID 8008 above. That GID should also be present on the client machine.

/etc/auto.qnap
pictures -fstype=nfs,rw,soft,tcp,nolock,gid=foobar qnap:/pictures
video -fstype=nfs,rw,soft,tcp,nolock,gid=foobar qnap:/video

Finally, after making changes to the NFS / autofs confg, restart the service:
/etc/init.d/autofs restart

Comments Off

Multi SATA support on Banana Pi

Comments Off

HTPC Guides has a fun post detailing how you can make the Banana Pi (and presumably the Banana Pro) work with multiple SATA drives. Using a SATA multiplier from AliExpress and a 2.5″ HDD enclosure with separate SATA ports more drives can be hooked up.

At the time of writing, a minor change and recompile of the kernel is required. However, if this catches on, it is sure to be supported out of the box.

Comments Off