Roland TB-03: Old school acid

Comments Off

The Roland TB-03 is a remake of the famous TB-303 bass synthesizer which defined acid techno since the late 1980s. The TB-03 recreates the squeaks and high pitches of original. The button and knob layout is mostly the same as the original, but with a few extra features: A four digit LCD display is added, which makes it easer to keep track of programmed patterns and tempo. Another modern feature is the micro USB port, which exposes a 24-bit/96kHz audio interface and MIDI. It can be powered by USB or 4 AA batteries.

I was lucky to be gifted one for Christmas, so have had only little time to try things out. So far I’ve gone through the brief but instructive videos from Roland’s own Youtube channel listed below; plugged it in over USB and seen the MIDI messages in midisnoop (packaged in both Debian and Ubuntu); and programmed a snippet inspired by Josh Wink’s “Higher State Of Consciousness”.

TB-03 Quick Start video

  1. Installation
  2. Using the Knobs to Adjust the Sound
  3. Pattern Playback
  4. Pattern Write (TB-303 Original Mode)
  5. Pattern Write (Step Recording Mode)
  6. Playing/Editing a Track

Also useful, this trick to copy patterns, using the Original 303 Mode.

Comments Off

Raspberry Pi headless install

Comments Off

The minimal “lite” image of Debian 8 (Jessie) is an excellent choice for a headless Raspberry Pi. After downloading to the SD card, these notes from Dmytro Bobkov covers the basic initial setup, while wifi setup from the command line is explained here. More details on CLI wifi on Debian in a previous post here.

If there is no screen or keyboard available, the SD card have to be prepared before the initial boot. Mainly to make sure SSH is running, so you can log in. This discussion covers the topic. However, if things are not working at once, a few debug statements can help. E.g., add as needed in the config file (change the IP as needed to your laptop or machine):

echo "$_IP" | nc 10100

echo "ssh has started" | nc 10100

On the other end, receive the messages by:

while true; do nc -l 10100; done

Finally, you might want to add a few extra packages, based on what you want to use the device for. These might come in handy:

apt-get update
apt-get upgrade

apt-get install htop itop atop git tig tree autossh nmap rsync lynx links emacs

Comments Off

cryptsetup basics

Comments Off

Talking about encryption in the previous post, I realized there are a few details I keep having to look up. This is a collection of the Frequently Asked Questions about cryptsetup formatting and mounting.

Note: For all the following examples, the example device /dev/sdX is used. It’s a device and file which doesn’t exist, on purpose. When replacing with your own e.g. /dev/sda or similar, be careful!

Formatting a new physical drive

Before working with a new drive, it’s recommended to check for bad blocks, to confirm it’s not a DOA (Dead on Arrival). If it is, you might want to claim it on the warranty immediately to avoid losing data in the future.

This command will check for bad blocks, as well as fill the disk with random data to better hide the encrypted volume later:

badblocks -c 10240 -s -w -t random -v /dev/sdX

Next is the partition setup, where all you need is a new cleared (similar to unformatted, but actually cleared) partition. In the gparted UI it’s simply “New -> Cleared -> Apply”, while on the CLI it would go something like this, to create an optimally aligned, primary partition.

parted /dev/sdX mklabel gpt
parted -a optimal /dev/sdX mkpart primary '0%' '100%'

Now, coming to the encrypted volume, you could just use a passphrase, and skip the first line, or store a salted hashed password in a key-file. The benefit of the latter, is that it will generally be a more secure key, and yet you could re-created the keyfile if you lost it, assuming you remember both the password and the salt.

mkpasswd --m=sha-256 --salt='SOME_SALT' | tr -d '\n' > /tmp/key-file

cryptsetup luksFormat /dev/sdX1 /tmp/key-file
cryptsetup open /dev/sdX1 unenc --key-file /tmp/key-file

Notice the mapping name “unenc“, which can be anything of your choosing.

Finally, format and mount the drive. Here, the ext4 file-system is used, with 1% reserved for system

mkfs.ext4 -m 1 -O dir_index,filetype /dev/mapper/unenc
mount /dev/mapper/unenc /mnt/tmp

Creating an encrypted file volume

In some cases, it is useful to encrypt only a small part of the disk, or even move the encrypted container around. A loop device can create a filesystem inside a file residing on any file system, be it USB stick, network mount or local disk.

First, you will have to create an empty file. The dd command will copy zeros to the specified filename. The total size is block size times count, or 500 MB in this example:

dd if=/dev/zero of=myfile bs=1M count=500

Then establish the loopback. It will become available on /dev/loop0, and can be formatted and mounted like any other block device.

losetup /dev/loop0 mycryptfile

Now repeat the luksFormat and filesystem format commands from above:

cryptsetup luksFormat /dev/loop0
cryptsetup open /dev/loop0 mycrypt
mkfs.ext4 -m 1 /dev/mapper/mycrypt
mount /dev/mapper/mycrypt /mnt/tmp

Key managment

Most of the cryptsetup commands above have at least two options when dealing with the keyslot: A passphrase and a key file. Typically, a passphrase is typed in on the prompt when unlocking the partition or modifying the other keys, while a key file is supplied using the –key-file argument. In terms of security, the first is “something you know”, while the latter is “something you have”.

To list the active keyslots use the following command. It will work both on an open and closed partition.

cryptsetup luksDump /dev/sdX

To add a new key with a prompted password:

cryptsetup luksAddKey /dev/sdX

or a randomly generated key-file:
dd bs=512 count=4 if=/dev/urandom of=~/keyfile_for_sdX iflag=fullblock

cryptsetup luksAddKey /dev/sdX ~/keyfile_for_sdX

To erease one of the existing key-slots, assuming you have more than one.

cryptsetup luksKillSlot /dev/sdX <key slot number>

You might also want to backup the LUKS header, which includes the key-slots, so in case you overwrite existing keys, you can restore the header and unlock with the old keys. It should be noted, that this header will then be able to unlock the partition given any password or keyfile in its keyslots. So, even if you change a password, the old header can be restored and an old password used to unlock. Therefore, it should be considered a secret file and stored securely just as the key file.

cryptsetup luksHeaderBackup /dev/sdX --header-backup-file ~/header_for_sdX

Finally, you might need to wipe the whole encrypted volume. You can do this with the luksKillSlot command, or manually remove all keys, and then change or add the remaining one with a password or keyfile you later remove or forget. E.g. by generating a key-file on the RAM disk /dev/shm, and then rebooting to lose it.

Comments Off


Comments Off

Tasked with setting up another NAS solution, I went for the simple 4-bay QNAP TS-431P, since the previous QNAP gave a good impression. This one does not have HDMI; in fact the only external ports are three USB 3 ports and two RJ-45 Gigabit Ethernet – no eSATA. Compared to its previous version, TS-431P has double the amount of RAM (for a total of 1 GB), and a slightly faster CPU. Software is as expected from QNAP.

The following describes the standard disk layout when using a single / stand alone disk, which still gets formatted as RAID + LVM, and optionally an encrypted partition.

Windows shares setup is covered at the end.



The QNAP NAS OS supports encryption, and I wanted to evaluate how secure this is in terms of failure. That is, if a disk fails, or the NAS itself fails, can you recover the data from the remaining disks. You can, but there are a few steps to watch out for.

First of all, even if each disk in the NAS is set up as “Single Disk / Stand Alone”, using no RAID, the NAS will still configure each partition on the separate disks as RAID partitions and in a LVM2 single volume group. That means you’ll need the Linux RAID and LVM tools and commands to mount. (Some useful discussion here).

General install, scan and list commands:

apt-get install mdadm lvm2

mdadm --assemble --scan
cat /proc/mdstat


And to mount, use the example commands below.

Note: The device names and volume names will most certainly be different. Use the commands above to understand the layout of the disk you’re working with.

Also note: if the mdadm scan command does not make all the RAID partitions available, it could be due to an existing /etc/mdadm/mdadm.conf file. You could try to rename it to mdadm.conf.old, or append the RAID details with mdadm –detail –scan >> /etc/mdadm/mdadm.conf. See here for more.

mdadm --assemble --scan

vgchange -ay vg1

mount /dev/vg1/lv1 /mnt/tmp

That should mount the drive, however, if you are working with an encrypted drive, you’ll need one more step before the mount command works, so ignore the last line and continue reading.



If you have followed the steps above, and type lsblk, part of the output will look something like this. It shows the layers so far: from the physical partition (sdb3) to the raid1 partition (md126), which contains two LVM logical volumes. In this case, the second is the LUKS encrypted main partition.

├─sdb3              8:19   0   3.6T  0 part  
│ └─md126           9:126  0   3.6T  0 raid1 
│   ├─vg288-lv545 254:1    0  37.2G  0 lvm   
│   └─vg288-lv2   254:2    0   3.6T  0 lvm   

So, we continue to decrypt, and mount it. Using cryptsetup luksDump, you can confirm that there is only one keyslot on the encrypted volume, which uses the paraphrase you typed in when installing the drive. However, the password is salted and MD5 hashed, so you have to generate a key-file with the new key. The salt is YCCaQNAP when using the mkpasswd tool, but encoded as $1$YCCaQNAP$ when calling the crypt library. Also make sure the key-file does not contain a newline.

cryptsetup luksDump /dev/vg288/lv2

mkpasswd --hash=md5 --salt='YCCaQNAP' | tr -d '\n' > /tmp/key-file
cryptsetup luksOpen /dev/vg288/lv2 unenc_lv2 --key-file /tmp/key-file

mkdir /mnt/tmp
mount /dev/mapper/unenc_lv2 /mnt/tmp

You now have access to the data files on the drive.

Coming back to the original question: Is this a resilient way of storing files? There are certainly a lot of layers, and although they each are well established technologies, they add complexity. Especially in the scenario when you would need to do the recovery it adds additional stress. Ideally, a single partition, no RAID, no LVM could be used. However, it seems that is not possible with the stock QNAP OS, since it will format any drive which is added to the NAS in its own way, including the RAID + LVM stack. In fact, this warning from the user manual is worthing taking careful note of:

Caution: Note that if you install a hard drive (new or used) which has never been installed on the NAS before, the hard drive will be formatted and partitioned automatically and all the disk data will be cleared.


Windows shared folders

The Windows sharing is easier to set up, but not without hurdles. On the local network, it typically will work out of the box when you point Windows Explorer to \\NAS_DOMAIN. If you need to connect across a firewall, you’ll have to open or forward at the minimum TCP 139,445, but possibly more ports on TCP and UDP.

The problem is that when sharing these ports cross the Internet, you will very likely run into other firewalls. ISP might block the default 139 or 445 ports. Although it is possible to port-forward to non-default ports, and this will work on Mac and Android, Windows will not accept it. A work-round if all else fails is therefore to set up a VPN or tunnel. Using SSH, this can easily be done with:

ssh -L -L admin@remotehost

Here it is assumed the NAS has DNS “qnap” on its local network, otherwise, replace with it’s IP. You might also want to forward 8080, forward SSH on a different port (e.g. 2222), as well as keep it running with autossh:

autossh -M 12340 -f -N -p 2222 -L -L -L admin@remotehost

Finally, if using only Windows machines to connect to the shares, there is the option of combining multiple shares into one. However, if other OSes also connect, you probably want to skip that.

Comments Off

Linux Credit Card

Comments Off

The Linux Foundation is offering a credit card as a way to donate to their cause. There’s an initial $50 price, and then the points which normally gather dust on other credit cards will automatically benefit them. And the card features Tux!

Comments Off

Facebook exit stage left

Comments Off

In an older article about social network trends, I discussed how Facebook in particular was far past its peak as measured by Google Trends. And that trend has not turned. There is a smaller and smaller fraction of web searches for the term “facebook”. It is now around a similar level as in 2009. Some of this can be explained by the fact that more people use the Facebook mobile app to access the social network, and therefore do not need to search Google to access the site. However, as pointed out by this Guardian article, despite claiming 1.79 billion users, 2016 was the year Facebook “became the bad guy”. In the Western world, interest was saturated several years ago, and new users come from other regions and possibly where Facebook have been blocked. They now plan to adapt their site and network to China’s censor and surveillance requirements.

Yet, if you are an investor in the social network, you might want to hold on a bit longer before you sell. In recent years, Facebook has snapped up both of the new popular social apps Whatsapp and Instagram (in 2014 and 2012 respectively). They continue their upwards trend, and will do so for many years to come. Compared to Facebook they both pale on Google Trends, meaning there’s a lot of potential.

Comments Off

12 TB helium and 14TB helium + SMR announced

Comments Off

HGST has just announced a new helium filled drive of 12 TB. The increase in capacity comes from an impressive and unexpected 8 platter design. Meanwhile, Western Digital forecasts that they will take the Ultrastar He12 based disk to 14 TB by combining it with Shingled Magnetic Recording (SMR). No prices are indicated yet.

Comments Off

Review: “The Internet Is Not the Answer”, Andrew Keen

Comments Off

Andrew Keen is a bitter man. He longs for the time when his family ran a tailor business in London and middle class people could afford to buy clothes from their store. He resents the fact that his own music Internet site failed, while seemingly similar sites like MySpace, Spotify came to be valued in billions. For some reason he mourns the demise of Kodak and its film roll processing centre in Rochester, NY. And most of all, he despises rich folk, but not any billionaire, just those who happen to have made their money through the Internet. Keen’s book “The Internet Is Not the Answer” reads a bit like a rant towards all these things, while blaming it all on Silicon Valley and The Internet. The solutions he favour are mostly based on government regulation: six strikes laws for copyright infringement; antitrust and monopoly busting; mixed with labor unions.

Even though Keen’s book has a bitter tone throughout, he does touch on important points regarding increasing wealth disparity, middle class jobs being replaced by automation and far fewer specialized jobs, monopolistic mega-cooperations, centralized services. He takes on Amazon, Google, Facebook, Instagram, Twitter, WhatsApp. He has done thorough research, and his book includes a substantial reference section. It is just that his conclusions doesn’t always align with actual causes and effects: Take the downfall of Kodak, where he spends a full chapter lamenting Instagram for the killing of film processing. Companies like Canon and Nikon which developed and sell high quality DSLR cameras and thus more directly caused the replacement of film are not mentioned.

Similarly, Keen reviews the history of the early days of the Internet, and its inventors and pioneers like Paul Baran, Bob Taylor, Bob Kahn, Vint Cerf, Tim Berners-Lee and points out how altruistic and anti-commercial they were. He contrasts this to “winner-take-all” companies in today’s economy. However, he does not discuss the seemingly obvious conclusion that what we’re lacking from Facebook, Instagram, Twitter are common open standards and protocols, which is what made early technology successful and long-lasting. Free and Open Source Software (FOSS) does not enter Keen’s field of view at all.

Maybe not worth it

If you already work in tech, and have good insight into these topics, Keen does not bring much new to the table. In fact, he disappoints in that regard. However, if you are interested in gentrification in San Francisco, Kodak in Rochester, or just want to hear a different point of view, give the book a try.

What Keen does have going for him, is that he is a very good writer. He writes almost poetically, albeit with great sarcasm, about topics like Internet economy, government regulation, and pretentious billionaires. Sound bites like the one below at least make the book entertaining.
While talking about the cult and praise of “failure” in tech-companies:

“Instagram actually represents the reverse side of Silicon Valley’s cult of failure. In the Valley, the rich and famous claim to be failures; on social networks like Instagram, millions of failures claim to be rich and famous”.

Finally, although it’s no way to be sure, it sometimes feels like the writes of Silicon Valley, the TV comedy series, have studied the book thoroughly and lifted several ideas from Keen onto the screen. What Keen scorns, like the double-speak; the feel-good big-company efforts and speeches; and the general Valley culture has made the TV series a hit. Keen’s book makes the series even more fun to watch.

Comments Off

Upgrading Debian Wheezy 7 to Jessie 8

Comments Off

Upgrading from Debian 7 to 8 is reasonably straight forward, following the official instructions. These shorter summaries are also useful references.

Very briefly then, make sure you have backup.
dpkg --get-selections "*" > dpkg_selections.txt
tar zcvf upgrade_backup.tar.gz /etc /var/lib/dpkg /var/lib/apt/extended_states /etc/mysql/my.cnf /etc/fuse.conf /etc/ssh/ssh_config

Update /etc/apt/sources.list, and replace all occurrences of wheezy with jessie.
sed -i 's/wheezy/jessie/g' /etc/apt/sources.list

If VirtualBox is installed, update to the new key:
wget -q -O - | sudo apt-key add -

Then comes the upgrade dance, with a few prompts, warnings, questions.

apt-get update
apt-get upgrade
apt-get dist-upgrade

After the upgrade, it is recommended to purge unused packages
apt-get purge $(dpkg -l | awk '/^rc/ { print $2 }')
apt-get autoremove

It is also recommended to install the linux-image-* metapackage, e.g. for AMD CPUs
apt-get install linux-image-amd64

Finally, cross your fingers and reboot.

Comments Off

Android Hacking

Comments Off

In a recent post on his blog, Thanassis Tsiodras has an impressive tale of his Android hacking adventures. Wanting to run Debian from chroot, he ends up customizing the boot image; attaching a serial logger to the headphone jack; and intercepting the over-the-air update image to control the boot process.

At last, it seems he achieved what wanted, but of course at an extremely high price. Most mobile devices are now very hostile towards any other use than was is dictated by the manufacturer and OS vendor. Thanassis ends his article on a sober note, saying that once even this hack is secured against “Android might as well be called iOS”.

Comments Off

Storage prices – end of 2016

Comments Off

The last half-year have seen few movements in spinning disk prices, and actually some have gotten a few coins more expensive. It is still the 8 TB SMR Seagate drive which give most storage for money, only with the exception of a special offer on an external 8 TB WD disk. In fact, several of the external offerings are now cheaper than their internal counterparts.

Also worth noting, is that Western Digital has deprecated their long time cheapest Green line, in favour of the Blue color. However, take care: The Blue line contains both 5400 and 7200 RPM drives. The ones listed here are all 5400 RPM.

On the SSD side, things have also not changed much. There were some promises, but not much in terms of concrete offerings this year.

Finally, many of the flash cards are getting more competitive. Indeed, the the Sandisk Ultra 256 GB key, as well as the 200 GB micro SD card are now pretty great offering for on-the-go storage. They’d fit very decent sized music collections, picture albums, etc.

Media Type Product Capacity Price CHF Price Euros Euros / GB GBs / Euro
External 3.5 Western Digital My Book 8TB, USB3 8000 GB 250.00 233.64 0.03 34.24
HDD-SMR Seagate ARCHIVE HDD 8TB 8000 GB 266.00 248.60 0.03 32.18
HDD Seagate Desktop 4TB 4000 GB 134.00 125.23 0.03 31.94
External 3.5 Western Digital My Book 6TB, USB3 6000 GB 202.00 188.79 0.03 31.78
External 3.5 Western Digital Elements Desktop 4TB, USB3 4000 GB 135.00 126.17 0.03 31.70
HDD Western Digital Blue, 5400 RPM 3TB 3000 GB 107.00 100.00 0.03 30.00
HDD Western Digital Blue, 5400 RPM 4TB 4000 GB 145.00 135.51 0.03 29.52
External 3.5 Western Digital My Book 4TB, USB3 4000 GB 149.00 139.25 0.03 28.72
SMR External 3.5 Seagate Backup Plus Desktop 8TB 8000 GB 300.00 280.37 0.04 28.53
External 3.5 Western Digital My Book 3TB, USB3 3000 GB 115.00 107.48 0.04 27.91
External 3.5 Western Digital Elements Desktop 3TB, USB3 3000 GB 118.00 110.28 0.04 27.20
HDD Western Digital Blue, 5400 RPM 6TB 6000 GB 244.00 228.04 0.04 26.31
HDD Western Digital Red 4TB 4000 GB 164.00 153.27 0.04 26.10
HDD Western Digital Blue, 5400 RPM 5TB 5000 GB 208.00 194.39 0.04 25.72
External 2.5 Western Digital My Passport Ultra 3TB, USB3 3000 GB 125.00 116.82 0.04 25.68
HDD Western Digital Red 3TB 3000 GB 126.00 117.76 0.04 25.48
HDD Western Digital Red 6TB 6000 GB 252.00 235.51 0.04 25.48
HDD Western Digital Blue, 5400 RPM 2TB 2000 GB 84.90 79.35 0.04 25.21
HDD-He Western Digital Red 8TB 8000 GB 345.00 322.43 0.04 24.81
External 2.5 Western Digital Elements Portable 2TB, USB3 2000 GB 88.00 82.24 0.04 24.32
HDD Western Digital Red 2TB 2000 GB 98.00 91.59 0.05 21.84
External 2.5 Western Digital My Passport Ultra 2TB, USB3 2000 GB 103.00 96.26 0.05 20.78
HDD-He Hitachi Ultrastar He6 6TB 6000 GB 361.00 337.38 0.06 17.78
HDD Western Digital Blue, 5400 RPM 1TB 1000 GB 61.70 57.66 0.06 17.34
External 2.5 Western Digital Elements Portable 1TB, USB3 1000 GB 62.00 57.94 0.06 17.26
HDD-He Hitachi Ultrastar He8 8TB 8000 GB 517.00 483.18 0.06 16.56
External 2.5 Western Digital My Passport Ultra 1TB, USB3 1000 GB 72.00 67.29 0.07 14.86
Blu-ray Verbatim BD-R SL 10 @ 25GB 250 GB 23.90 22.34 0.09 11.19
DVD-R Verbatim 16x DVD-R 100 @ 4,7GB 470 GB 46.00 42.99 0.09 10.93
Blu-ray Verbatim BD-R DL 10 @ 50GB 500 GB 49.00 45.79 0.09 10.92
DVD+R DL Verbatim 8x DVD+R DL 25 @ 8,5GB 213 GB 39.00 36.45 0.17 5.83
DVD+R DL Verbatim 8x DVD+R DL 50 @ 8,5GB 425 GB 95.40 89.16 0.21 4.77
SSD Crucial BX200 SSD, MLC, 480GB 480 GB 139.00 129.91 0.27 3.69
USB Flash SanDisk Ultra, USB 3.0, 256GB 256 GB 78.00 72.90 0.28 3.51
SSD Samsung SSD 850 EVO Basic, TLC, 500GB 500 GB 165.00 154.21 0.31 3.24
SSD Crucial BX200 SSD, MLC, 240GB 240 GB 81.00 75.70 0.32 3.17
SSD Samsung SSD 850 EVO Basic, TLC, 1TB 1000 GB 339.00 316.82 0.32 3.16
SSD Crucial MX200 SSD, MLC, 500GB 500 GB 175.00 163.55 0.33 3.06
SSD Crucial MX200 SSD, MLC, 1000GB 1000 GB 374.00 349.53 0.35 2.86
SSD Crucial MX200 SSD, MLC, 250GB 250 GB 95.30 89.07 0.36 2.81
SSD Crucial BX100 SSD, MLC, 500GB 500 GB 193.00 180.37 0.36 2.77
SSD Samsung SSD 850 EVO Basic, TLC, 250GB 250 GB 100.00 93.46 0.37 2.68
microSDXC SanDisk Ultra Premium microSDXC 90MB/s, 200GB 200 GB 85.00 79.44 0.40 2.52
USB Flash SanDisk Cruzer Edge Flash Drive 64GB 64 GB 27.80 25.98 0.41 2.46
SSD Samsung SSD 850 Pro, MLC, 1024GB 1024 GB 454.00 424.30 0.41 2.41
USB Flash SanDisk Ultra, USB 3.0, 64GB 64 GB 28.80 26.92 0.42 2.38
CD-R Verbatim CD-R 100 @ 700MB 70 GB 32.60 30.47 0.44 2.30
SSD Samsung SSD 850 Pro, MLC, 512GB 512 GB 239.00 223.36 0.44 2.29
SSD Samsung SSD 850 Pro, MLC, 256GB 256 GB 139.00 129.91 0.51 1.97
USB Flash SanDisk Cruzer Edge Flash Drive 32GB 32 GB 18.80 17.57 0.55 1.82
USB Flash SanDisk Ultra, USB 3.0, 32B 32 GB 19.50 18.22 0.57 1.76
SDXC SanDisk Extreme Pro SDXC UHS-I, Class 10/UHS 3, 95/90MB/s, 128GB 128 GB 82.80 77.38 0.60 1.65
SSD-NVM-M.2 Samsung SSD 950 Pro, M.2 2280, MLC, 2500/1500MB/s, 512GB 512 GB 361.00 337.38 0.66 1.52
SSD-NVM-M.2 Samsung SSD 950 Pro, M.2 2280, MLC, 2200/900MB/s, 256GB 256 GB 187.00 174.77 0.68 1.46
SDXC SanDisk Extreme Pro SDXC UHS-I, Class 10/UHS 3, 95/90MB/s, 512GB 512 GB 389.00 363.55 0.71 1.41
SSD Samsung SSD 850 Pro, MLC, 128GB 128 GB 99.70 93.18 0.73 1.37
SSD Samsung SSD 850 EVO Basic, TLC, 120GB 120 GB 95.70 89.44 0.75 1.34
SDXC SanDisk Extreme Pro SDXC UHS-I, Class 10/UHS 1, 95/90MB/s, 64GB 64 GB 55.00 51.40 0.80 1.25
USB Flash SanDisk Cruzer Edge Flash Drive 16GB 16 GB 14.90 13.93 0.87 1.15
SDHC SanDisk Extreme Pro SDHC UHS-I, Class 10/UHS 1, 95/90MB/s, 32GB 32 GB 33.00 30.84 0.96 1.04
Compact Flash SanDisk Extreme 120MB/s, UDMA 7, 64GB 64 GB 75.00 70.09 1.10 0.91
Compact Flash SanDisk Extreme Pro 160MB/s, UDMA 7, 256GB 256 GB 347.00 324.30 1.27 0.79
Compact Flash SanDisk Extreme Pro 160MB/s, UDMA 7, 128GB 128 GB 175.00 163.55 1.28 0.78
Compact Flash SanDisk Extreme Pro 160MB/s, UDMA 7, 64GB 64 GB 98.50 92.06 1.44 0.70
USB Flash SanDisk Cruzer Edge Flash Drive 8GB 8 GB 12.90 12.06 1.51 0.66
Compact Flash SanDisk Extreme 120MB/s, UDMA 7, 32GB 32 GB 52.70 49.25 1.54 0.65
SDHC SanDisk Extreme Pro SDHC UHS-I, Class 10/UHS 1, 95/90MB/s, 16GB 16 GB 27.00 25.23 1.58 0.63
SDXC SanDisk Extreme Pro SDXC UHS-II, UHS 3, 280/250MB/s, 64GB 64 GB 117.00 109.35 1.71 0.59
Compact Flash SanDisk Extreme Pro 160MB/s, UDMA 7, 32GB 32 GB 60.10 56.17 1.76 0.57
Compact Flash SanDisk Extreme 120MB/s, UDMA 7, 16GB 16 GB 36.40 34.02 2.13 0.47
SDHC SanDisk Extreme Pro SDHC UHS-II, UHS 3, 280/250MB/s, 16GB 16 GB 52.70 49.25 3.08 0.32

Exchange rate: 1 Euro = 1.070000 CHF.

Comments Off

Review: “Flashpoints – The Emerging Crisis in Europe”, George Friedman

Comments Off

In his latest book, “Flashpoints – The Emerging Crisis in Europe”, George Friedman gives a summary of various events through European history, focusing on the 20th century wars, inter-war years, and cold war eras. Although the book tries to sell itself as being about future predictions, he spends more than half of it looking at the past. In the second half, he tries to foresee various possibilities and outcomes for the regions of Russia / Eastern Europe; Germany; France; the Mediterranean region; Turkey and Britain. However, most of the attempt at analysis remains limited, shallow, and biased.

The problem with Friedman’s book, is that it seems he didn’t quite decide what it was supposed to be about: his own and his family’s escape from war in Europe; his travels across Europe in later years; a history of Europe; or as the title suggests, about future emerging crisis. It turns in to a bit of everything, and becomes colloquial and incoherent. Between the endless anecdotes, it seems what Friedman is missing is an editor who can cut away the cruft. Totally irrelevant stories covering taking a leak behind a border office in Ukraine or that some hotelier in Sarajevo reminds him of his aunt, are just some of what could have been left out.

Skip it

Frankly, this book is a waste of time. Although the history of Europe is important in order to understand the present and the future, this book is not a good summary. If any section of it had been part of Wikipedia, it would have been littered with “citation needed”. In fact, Friedman hasn’t included any references at all, and sometimes it seems he has not done his homework very well, with plenty of inaccurate facts.

This makes the future predictions in the book rather worthless. They are not based on any hard facts or research, but rather what Friedman could imagine. Furthermore, the book shows some age, despite being republished this year: In the chapter on Britain, he completely missed the risk of Britain voting to exit the EU, while it’s mentioned briefly in the after-word.

Overall, this is not a book worth spending time on.

Comments Off

Jekyll: Include partial snippets of code

Comments Off

The Jekyll include tag is useful when including files or templates on a page. Combined with the highlight tag, it makes including code snippets easy. However, it will include the complete file, and often it is desirable to include only a few lines, or maybe a method. That could of course be done by simply copy/pasting the code in question into the article, but then the code gets out of sync if the example file is changed.

The basic usecase is something like this:

{% highlight java %}
    {% include src/ %}
{% endhighlight %}

Ruby based plugin

A Jekyll tag to include only a section of a file would be great. As far as I can tell, that does not exist yet, so I started writing one. Unfortunately, Github Pages does not allow custom plugin for security reasons. There are work-arounds for that, but it also makes the deployment more complex, and loses the convenience of being able to edit the articles and code directly on

Sans error handling or caching, a simple implementation could look like this. It works outside Github Pages, so it’s a start.

module Jekyll
  class IncludeLines < Liquid::Tag
     Syntax = /(#{Liquid::QuotedFragment}+)\s(\d+)\s(\d+)\s\z/o
     def initialize(tag_name, markup, options)
       if markup =~ Syntax
         @file = $1
         @startline = $2.to_i
         @endline = $3.to_i
         raise "Syntax error in includelines: " + markup
     def render(context)
       lines = IO.readlines(context.evaluate(@file))
       part = lines.drop(@startline)
       part.take(@endline - @startline)
 Liquid::Template.register_tag('includelines', Jekyll::IncludeLines)

Liquid slice and split

Using the Liquid capture block, it's possible to read a file and store it as a string variable. It can then be processed by Liquid instead of the plugin, and works fine with Github pages. The Liquid syntax is certainly verbose, but it gets the job done.

An initial implementation cutting the file content as a single string looked like this. However, it is far from ideal, since the character index and count will shift with any source code modifications on the included file.

{% capture filecontent %}
    {% include src/ %}
{% endcapture %}

{% highlight java %}
    {{ filecontent | slice: 132, 57 }}
{% endhighlight %}

A slightly better solution uses the same idea, but operates on line numbers instead. It is almost as fragile when it comes to changes, but at least usable.

{% capture filecontent %}
    {% include src/ %}
{% endcapture %}

{% assign lines = filecontent | newline_to_br | split: '<br />' %}
{% highlight java %}
    {% for line in lines offset:10 limit:5 %}{{ line }}{% endfor %}
{% endhighlight %}

A helper include file implementing this idea can be found here. It can be used like this:

{% include includelines filename='src/' start=10 count=5 %}

Include method

Ideally, it would be possible to mark the start of a line to include, and then indicate how much should be included. Improving on the line based iterator above, this helper file does that. Usage goes like this:

{% highlight java %}
    {% include includemethod filename='src/' method='test()' before=5  after=1 %}
{% endhighlight %}

It also adds options to include lines before and after the specified method, for example for comment blocks or further methods below the first. There are of course some extensions which could be made, e.g. to include multiple split sections; support other non-C like languages, etc. The linked code is under the GPL 3 license, so feel free to improve.

Comments Off

Review: “ISIS: Inside the Army of Terror”, Michael Weiss and Hassan Hassan

Comments Off

In their book “ISIS: Inside the Army of Terror” from early 2016, Michael Weiss and Hassan Hassan give detailed insight into the Islamic State, its origin, key members, alliances, critical battles, and strategy of terror. The story begins with the early ties between Ayman Mohammed Rabie al-Zawahiri and Osama bin Laden in the 1990s, and the later split between them and the more radical and extreme Abu Musab al-Zarqawi. Their disagreement on who are their enemies, crucially whether it includes Muslims or not, has underlined the split between al-Qaeda and ISIS / Islamic State. In later chapters, the rise of the current leader, theology professor Abu Bakr al-Baghdad, is investigated. The book goes into great detail about several key battles in Iraq and Syria, and analyses positions and outcome. Finally, some of the terror attacks on civilians in Europe and the US is put in context.

Sunni vs. Shia

There are a few important take-aways from the book: The divide between Sunni and Shia Muslims is usually at the core of most of the conflicts. Crucially, the numbers goes a long way to explain the various positions: Word-wide, Sunnis are in majority at around 85–90% while 10–15% are Shia. However, in Iraq and Iran this is reversed, where 80% and 95% are Shia respectively. An important point is the fact that Saddam Hussein was Sunni, and his mostly Sunni minority Baath party ruled over the Shia majority. When US invaded and ended their rule and tried to create democracy, the stage was set for bitter conflict. Furthermore, Paul Bremer (presidential envoy to Iraq) fired the mostly Sunni Iraqi army, along with most other official positions. So around 2003 a large part of the previous Iraq elite was suddenly jobless, but with plenty of military experience and even weapons on their hands. al-Zarqawi exploited these fault lines to his fullest, and ignited the ensuing civil war.

A similar setup, but again reversed has been the background for the civil war in Syria. There, Bashar Hafez al-Assad and his party are Alawites, a branch of Shia, but in a minority at around 13%. When the spring revolutions in 2011 swept other Muslim countries, al-Assad pitted themselves as under attack by the Sunni (74%) majority. al-Assad’s regime has support from Iran and Hezbollah who are also Shia. The opposition in Syria has many factions, and ISIS has time and again proved that they are experts at driving a wedge between opposing forces to divide and conquer.

Enemy of my Enemy

From small tribes, to national organizations and rebel groups and all the way to international alliances, the relationships network is extremely complex. A graph like this hardly scratches the surface. Furthermore, alliances shift frequently, and often the short term strategy is “the enemy of mine enemy is my friend”. This can be seen going far back, and characterizes much of US and Russian involvement in the various conflicts: During the Cold War; the US backing of Iraq against Iran in 1979; later US attacks against Iraq; US backing of the Kurds. Iran and Russia have tended to back the opposite groups, and Iran in particular has now infiltrated much of the Shia resistance and politics in Iraq.

The book goes into detail on several of these fluid alliances, and looks at the decisive battles and opposing personalities. The point is made many times over that in order to understand the conflicts, one has to understand the tribal politics. At a higher level, the relationships are often more pragmatic: Although ISIS is waging war across Syria, and also against al-Assad forces, they have a business relationship in the oil trade, where ISIS is selling back oil to al-Assad’s regime from the oilfields they have captured. al-Assad benefits since ISIS also fights the rebels in Syria. The enemy of mine enemy is my friend.

Read it

Weiss and Hassan have done plenty of research and interviews for this book, and it shows through all the details revealed. They have also done a good job of explaining the background history, religious underpinnings, and political motivations for the parties involved in the conflict. However, it can get somewhat tedious to go through all the nitty-gritty, and the writing style can be trite with the occasional odd analogies.

Overall, the book is definitively worth a read if you are interested in the current conflict, want to understand the terror attacks. Regional and international politics become more clear with the information provided by this book.

Comments Off

Backblaze: SMART stats to predict hard drive failure

Comments Off

Over the years, Backblaze has published several interesting reports and statistics on their harddrive performance and failures. In their latest post, they look at which SMART metrics they monitor and react to. They say, “when the RAW value for one of these five attributes is greater than zero, we have a reason to investigate”.

SMART 5 Reallocated Sectors Count
SMART 187 Reported Uncorrectable Errors
SMART 188 Command Timeout
SMART 197 Current Pending Sector Count
SMART 198 Uncorrectable Sector Count

They go on to point out, that although a single one of those metrics might only indicate 30% to 40% probability of a failed drive, when combined the probability increases to 76%. Furthermore, looking at multiple failed values, and their rate of increase might also help determine if a drive is about to fail. A higher rate is usually worse. Finally, they note that the the SMART 189 – High Fly Writes is another good candidate to consider, and again when looking at the rate.

On a related note, here’s a note on Google’s data on their SSD failures.

Comments Off