SSDs last a long time

1 comment

At 24 GB/day, or 8.8 TB/year, it would take 114 years to reach 1 PB.
Several SSDs have gone past that point, and still work fine.

Last year, I wrote about estimates for durability of SSD drives. Looking at how long it would take to exhaust the theoretical limit of the storage cells, based on different write speeds and quantities. The assertion was: It would take a very long time, and other technical failures were more likely to render disk useless long before all cells were used up. In fact, the owner is not likely to outlive full cell exhaustion. This has now been confirmed through an experiment.

Over at Tech Report, they have done an endurance experiment lasting more than a year, with several consumer drives. For months, they have been writing data to the drives, while monitoring drive health, and verifying correctness. Some of the drives were rated for 20 GB / day for three days, which means about 22 TB. However, several of them made it past the 1000 TB, or 1 PB, mark. That’s about 50 times the advertised endurance rating. And two of the drives, a Samsung 840 Pro 256GB, and a Kingston HyperX 3K 240GB, have made to past 1.5 PB. It’s not all clear from the article exactly when they started, or how much they write per day. However, assuming a year to get to 1.5 PB, that’s 4.1 TB / day, or 47 MByte/s. The actual write speed is probably higher, but this leaves time for verification as well.

To put these numbers into perspective, I’ve extended the table from last time, and added a line for the speed of the Tech Report endurance test, as well as extra columns for multiple years total. As can be seen, the endurance test is running at 171 the write speed of what was identified as “heavy use”. Furthermore, the heavy use scenario is within what the typical consumer drives are rated for, i.e. about 20 GB/day.

MBit/s MByte/s MByte/hour GByte/day GByte/year TByte 3 years TByte 5 years
SATA3 max speed 6000 750 2700000 64800 23652000 70956 118260
Stress test 2000 250 900000 21600 7884000 23652 39420
Endurance test 376 47 171000 4109 1500000 4500 7500
Heavy use 2.2222 0.2778 1000 24 8760 26.280 43.800
Low/Medium use 0.0926 0.0116 41.67 1 365 1.095 1.825

What about the time to failure estimates; how do they compare to the empirical evidence? Last year, I noted that a 256 GB drive with cells of 10k write cycle life span (typical MLC memory), would take 256 years to reach 10% of failed cells, and about 350 years for full exhaustion, assuming 24 GB/day written. It turns out that was a bit optimistic. At 24 GB/day, or 8.76 TB/year, it would take “only” 172 years to reach 1.5 PB (where the Tech Report drives are now). If we go by 1 PB, it would take 114 years.

This is all for MLC memory. If we look at TLC, typically rated for 1000 write cycles, the endurance numbers are also a tenth. I.e. they would fail between 11 to 17 years of sustained 24 GB/day writes, again assuming a 256 GB drive.

If larger drives are used, the time to failure also increases. This is because there are more total space to level the writes across. In fact, doubling the size of the drive, will in theory double its lifespan. So an MLC 512 GB drive would last some 228 years, and a 1 TB drive 456 years. For TLC, the numbers are again a tenth, so 22 and 45 years respectively.

This is all well and good, and should at least put the final nail in the coffin regarding worries for SSD reliability. The only concern which the Tech Report experiment raises, is the way the drives fail when they do reach end of life. Of course there are plenty of relocated sector warnings in the SMART data beforehand. However, once they are past the point of no recovery, all data is lost. Several of them cannot be accessed at all. This is of course a bit different from spinning disks, which usually keep on reading some of the tracks, even if other parts are broken.

It highlights the fact that monitoring SMART data should be a standard procedure, and part of good data hygiene. Of course, a good backup strategy is required, regardless of drive type or usage pattern.

PC build: Silent yet powerful

no comments

It’s been a long time since I’ve had the chance to put together a machine. The one I’m typing on right now has a more than five years old AMD Athlon 64 X2 5050e, and one of its HDDs report 47220 Power_On_Hours, or 5.4 years. It was fun to look at some new hardware.

This build is not for me, though. My father’s current machine is from 2005, and the AMD Sempron 2600 1.6GHz has kept up well, however would not be a good fit for the new requirements: A silent build which can handle a modern Ubuntu distribution plus Windows 7 in a VM. After good advice from Redditors on r/buildapc, I got the following components.

Main

Storage

The rest

 
 

Requirements and reasoning

At € 1055 (in June 2014), it’s not a cheap build, and I could definitely have saved a bit here and there. However, that was not my main concern – my father deserved something top-notch. I wanted something powerful enough so that it would last many years to come without upgrading, yet silent for the living room. That’s why some of the components are somewhat over-provisioned: the fanless 460W PSU, while I expect the peak draw to be less than 150W; 16 GB RAM, 256 SDD, 4 TB HDD.

For the CPU, I went for the four core Intel Core i5 4570 (LGA 1150, 3.20GHz), based on redMarllboro’s advice. It is indeed more powerful than the AMD A10-6700 I had originally planned for, and furthermore, the virtual cores would not benefit the VM much.

With the CPU fixed, I narrowed down my search for an Asus motherboard to the ASUS Maximus VII Ranger (Z97). That was based on the following criteria: more than 4 SATA ports, Intel Ethernet controller (I try to keep away from Realtek based on this issue, even if that was WiFi related), 4 DIMM slots, an onboard DVI and/or VGA port. Turns out, that really narrows it down, and about the only contender was the ASUS Sabertooth Z97 Mark 2, however that only has HDMI and Displayport embedded.

Now, one could argue that both of those MBs are overkill for what I’m building. However, most of the boards I’d be looking at would be in the €100-150 range anyway, and as price was really not a main issue here, why not go for the latest chipset? Furthermore, the “Republic of Gamers (ROG)” marketing from Asus I find somewhat misleading. The Maximus board looks aggressive in black and red, but surely it is the hardware specifications which matter. For example, the 10K Black Metallic Capacitors are welcome when cooling is an issue. Also, some of the ROG “features” in the form of software are dubious at best: How is a RAM disk a feature of the MB? On most GNU/Linux distributions, it’s there by default under /dev/shm.

For storage, an SSD is a no-brainer these days, and the only questions are: How large? And is additional storage required? 128 GB might have been just enough, but with ~50 GB for the Ubuntu host OS, ~40 GB for the VM, and ~30 GB for swap it would have been very tight. (In fact, post install, only 70 GB is left on a 256 GB disk). Doubling to 256 GB is less than double the price. I will require more storage space, so added the 4 TB spinning disk. When it comes to WD Red over Green, it’s only about €10 difference, so another no-brainer.

As the VM will be running Windows, my plan is to back it up frequently, in the hope of recovering from certain problems of that OS. Now, several people on r/buildapc thread advised against this. I suppose they are mostly right; it might be possible to lock down a Windows installation to the point where malware and adware is not a problem. The first and second issues with that are I’d have to spend a lot of time learning about it, and I would not be very interested. And why should I? A restricted install with no direct user access to system binaries and most applications delivered from a trusted cryptographically signed source has been the norm on most GNU/Linux distributions for more than a decade. It takes no effort at all, so why go with something inferior? If this machine and setup can avoid my father spending hundreds of bucks at PC Repair shops every year, it will pay itself back quickly and be a success.

 

Silent and cool

The most important requirement for this build was to make it silent. The fanless Seasonic P-460 achieves that without breaking a sweat. At normal load, which is 35 to 50 W at the power socket (220 V; in EU), I’ve measured its temperature of the PSU at 31 C. Also, the modular cable system is very nice, as it means no lose cables hanging around. In fact, there are no cables crossing the motherboard at all, as seen in this picture.

For the CPU, I had wished for passive water cooling, however most solutions on the market today are downright ugly. If the Zalman Reserator tower was still around, I would have gotten that. The compromise was therefore the over-sized Noctua NH-U14S. Again it is probably a bit of an overkill, however the benefit is that it’s not pushing the limit of the cooling, so it remains silent and cold. CPU temperatures at load is around 30 C, and at peak 45 C when the case fans kick in. The part which gets warmest is the Z97 chipset heat-sink, at around 36 C.

One of the features I appreciated most with the ASUS Maximus VII Ranger motherboard was the fan-control. Five fans can be controlled individually based on temperature. Both PWM (Pulse Width Modulation) and DC (voltage) regulation is possible, based on fan type. As seen in the pictures below, the two case fans are off when they are not needed, and kick in slowly when it gets hot. On low to normal load the CPU fan spins at 350 RPM, and can barely be heard if you put your ear right next to the case.

Finally, the only other moving part in the machine is the Western Digital 4 TB Red HDD. At a maximum rotation speed of 5400 RPM it is not dead silent, but quiet enough.

 

Building

Building this machine came with a lot of fun! The Fractal Design case was pure joy to work with. All aspects were well thought out: Easy access to left and right side (back of MB), excellent cable management, easy disk mounting slots, two large (and quiet) fans. Gone are the days of scratched and bleeding hands because of sharp edges around the case. And the fact that there are no cables criss-crossing the motherboard not only looks good, but also makes for good airflow. If I were to say anything against the case, it would have to be that it is big heavy beast.

The other components were also top notch, and caused no problems. In particular the modular Seasonic PSU and cable system is very welcome. You only have to plug in the cables you actually need, so no lose ends hanging around. The fact that the PSU comes in a pouch which competes with expensive cologne is also a nice touch.

The Noctua NH-U14S is a massive cooler. And it was another reason why I ended up with the Define R4 case; it was one of the few cases which had enough clearing for the cooling block. With a 14 cm fan it keeps the CPU nice and cool. The initial boot was without the fan, and temperatures went up to about 45 C in the BIOS. With the fan at lowest speed (about 350 RPM), it sits at around 35 C (still without having applied thermal paste; will wait till it’s shipped). The only concern I had was with fan direction. Its default orientation was to blow air from the RAM side backwards over the cooler. Currently, I’ve put it on the other side, so it sucks air over the block, and blows it right out at the rear fan. I might experiment with the difference of direction and position.

Here are a few pictures while building, followed by a couple of BIOS screen shots.

(Click for larger images.)


(Click for larger images.)

 
 

Software

As mentioned above, the goal was to have an Ubuntu installation, with Windows 7 in a VM. I chose Ubuntu 14.04 (aka “Trusty Tahr”), since it is a Long Term Support (LTS) release, and figured this would be the right balance between stability, supported hardware and packages. Other distributions I am currently using include Fedora and Debian, but for this build I figured hitting the middle-ground would be OK, thus Ubuntu. Since my father is used to Windows, I went for the simple Xfce 4 desktop, with a familiar taskbar, window icons and SHIFT+TAB application switching. As seen in the screen-shots below, it blends nicely with the seamless VirtualBox integration.

I tried and installed both the alternative Xubuntu ISO and the main Ubuntu ISO. The main difference is the default desktop, which is Xfce in the former. However, that had boot problems with Secure Boot, even after I enabled “Other OS” in the BIOS. It would install fine, but not find the boot image afterwards. It was possible to repair that by refreshing Grub, however it gave me a bad feeling at the start. The main Ubuntu ISO had now boot issues, and changing the desktop is just a matter of installing a package and selecting a different option at log-in. (The Ubuntu variations are really a bit redundant in that regard. Especially when other basic functionality, like boot, fails).

Apart from the default ISO packages, I added the following. There you can see xfce4, the VirtualBox packages, various utilities, and a few benchmarking tools. Nothing much came out of the later. Instead, see the CPU graphs below, which shows calm and moderate load while running Windows in the VM.

apt-get install autossh bonnie++ conky cpuburn dbus dos2unix elementary-icon-theme emacs evince fancontrol feh geeqie gimp git gitk gnome-icon-theme-extras gnome-icon-theme-full gnome-icon-theme-symbolic gnome-terminal gnupg gparted gthumb htop iftop imagemagick iotop k3b kdiff3 libnss-myhostname lmbench mencoder mplayer mtr nmap openssh-server parcellite policykit-1 policykit-1-gnome policykit-desktop-privileges screen smart-notifier sysbench sysstat tango-icon-theme tor tree usbutils virtualbox virtualbox-guest-additions-iso vlc wireshark xfce4 xsensors xubuntu-icon-theme

The installation of Windows in the VM is very simple. One important option to notice, is the Intel Virtualization Technology (VT-x) setting in the BIOS, as seen here. Once that is enabled, the rest is a breeze. VirtualBox comes with a brief but useful “wizard” which guides you through creating the image. I opted for a 40 GB, 2 CPU cores, 8 GB setup. After that, add the install medium (physical CD or ISO), and boot. Windows 7 will reboot about ten times, just as in the old days, but eventually will leave you with a full fledged install. Right after installation, it’s useful to add the VirtualBox Guest Additions, which amongst other things enables the seamless mode. Also, a shared mount-point is useful, and can be easily enabled through the VirtualBox settings. It automatically appears in Windows.

The CD/DVD drives are passed through, and the physical drives were mapped to similar drives in the VM. For shared directories / drives, I wanted to makes sure the they were mounted to the same Windows drive all the time, regardless of other mount points. Thus, the VirtualBox setting does not use auto-mount, and instead the directory was manually mounted as seen in the Dropbox example below.

Installing Dropbox was a matter of downloading and installing this package, and start it as an unprivileged user. Then, in order to make that available in the Windows image as well, the top Dropbox directory was shared as a drive. (Note: The Windows VM is intentionally not connected to the network). Finally, a requirement was to have that fixed on C:\Dropbox, which was achieved with a symbolic link in Windows. The following lines has to be executed in a shell run “as Administrator”:

net use x: \\vboxsvr\Dropbox
mklink /d x:\ c:\Dropbox

One of the few special applications which requires Windows, was Corel Paint Shop Pro (PSP). The usage pattern for this is typically to download something from the web, and the process it. To make this easy and seamless, I added a Firefox plug-in so every image gets an extra right-click menu item which opens the image in PSP inside the VM. Details for this is explained here.

Finally, another special Windows only application was the genealogy program Aldfaer. The requirement here was that it could be updated, over the web. To make this work, the main install is on Ubuntu, with an option to run and update from Wine. However, it runs better inside the VM, so the application folder is mapped to Windows through another shared folder in VirtualBox. I will go into detail regarding this setup in a later post.

Writing this a few months after the machine was delivered, I’ll declare it a success. Raw performance is at a very different level from what my father was used to. The machine is silent, and in fact is turned on most of the time (as opposed to the old which he never used because of fan-noise). The split Ubuntu / VM setup is slightly complicated, but seems to work out well. As expected, the Windows install has already regressed, but it is easy to go back to a previous Snapshot, instead of re-installing everything again. This machine will definitely last a long time.


(Click for larger images.)

Storage prices

Comments Off

If you need massive storage, 6 TB disks are already very competitively priced. However, the 3 TB WD Green still gives the most GB per coin. Amongst SSD, several 1 TB disks rank towards the top, but still at ten times more than magnetic. Finally, USB flash drives and SD cards are coming down in price, with a large 64 GB option at the top.

Details of updates

There has been several announcements of products you cannot buy yet lately. On the following list, all are for sale and immediate delivery.

The list has seen significant updates since last time, in product ranges, prices: Western Digital now has five colour codes for their desktop disks, three of which are included here. There is the inexpensive large but slow Green, the Red for NAS and RAID with extra warranty, and the Purple in between. Also new since February are the 5 and 6 TB disks.

At 92 Euros, the 3 TB Green is at the top of the list at 32.4 GB per Euro. It is interesting to note that the 6 TB version is already towards the top, at just a bit more than double the price; €202, or 29.6 GB/€. For external disks, the 4 TB Elements Desktop is at the top, for €135, or 29 GB/€.

Amongst the SSD disks, the largest are now at the top, with Crucial 1 TB, 512 GB, and Samsung 1 TB all around 3 GB/€. Most interesting, is the fact that the Crucial claim to be MLC while the Samsung SSD 840 EVO Basic is a TLC based disk. The Samsung MLC disk are far more expensive, starting below 2 GB/€. It a sign of a very competitive market, and further decrease in prices ahead.

Finally, lower solid state prices also benefit memory cards and USB sticks. The Sandisk Cruzer Edge 64 GB drive now gives most storage for money. A bit further down, the Sandisk Ultra 64 GB and 128 GB SD cards are good large capacity options, although with slow 30 Mb/s read and write speeds. For top write speed, at an impressive 280/250 Mb/s read/write, see the new UHS-II based 64 GB SDXC card.

The List

Media Type Product Capacity Price CHF Price Euros Euros / GB GBs / Euro
HDD Western Digital Green 3TB 3000 GB 112.00 92.56 0.03 32.41
HDD Seagate Desktop 4TB 4000 GB 159.00 131.40 0.03 30.44
HDD Western Digital Green 4TB 4000 GB 161.00 133.06 0.03 30.06
HDD Western Digital Green 6TB 6000 GB 245.00 202.48 0.03 29.63
External 3.5 Western Digital Elements Desktop 4TB, USB3 4000 GB 164.00 135.54 0.03 29.51
HDD Western Digital Purple 3TB 3000 GB 125.00 103.31 0.03 29.04
HDD Western Digital Green 2TB 2000 GB 85.00 70.25 0.04 28.47
HDD Western Digital Red 3TB 3000 GB 130.00 107.44 0.04 27.92
External 3.5 Western Digital Elements Desktop 3TB, USB3 3000 GB 130.00 107.44 0.04 27.92
HDD Western Digital Purple 4TB 4000 GB 175.00 144.63 0.04 27.66
HDD Western Digital Red 4TB 4000 GB 177.00 146.28 0.04 27.34
External 3.5 Western Digital My Book 3TB, USB3 3000 GB 135.00 111.57 0.04 26.89
HDD Western Digital Red 5TB 5000 GB 226.00 186.78 0.04 26.77
External 3.5 Western Digital My Book 4TB, USB3 4000 GB 182.00 150.41 0.04 26.59
HDD Hitachi Deskstar 7K4000, 4TB 4000 GB 184.00 152.07 0.04 26.30
HDD Western Digital Red 6TB 6000 GB 279.00 230.58 0.04 26.02
HDD Western Digital Purple 2TB 2000 GB 95.40 78.84 0.04 25.37
HDD Western Digital Red 2TB 2000 GB 104.00 85.95 0.04 23.27
External 2.5 Western Digital Elements Portable 2TB, USB3 2000 GB 109.00 90.08 0.05 22.20
HDD Western Digital Green 1TB 1000 GB 62.20 51.40 0.05 19.45
External 2.5 Western Digital My Passport Ultra 2TB, USB3 2000 GB 127.00 104.96 0.05 19.06
HDD Western Digital Purple 1TB 1000 GB 69.60 57.52 0.06 17.39
HDD Western Digital Red 1TB 1000 GB 70.50 58.26 0.06 17.16
External 2.5 Western Digital My Passport Ultra 1TB, USB3 1000 GB 75.90 62.73 0.06 15.94
External 2.5 Western Digital Elements Portable 1TB, USB3 1000 GB 77.90 64.38 0.06 15.53
DVD-R Verbatim 16x DVD-R 100 @ 4,7GB 470 GB 40.40 33.39 0.07 14.08
Blu-ray Verbatim BD-R SL 10 @ 25GB 250 GB 25.00 20.66 0.08 12.10
Blu-ray Verbatim BD-R DL 10 @ 50GB 500 GB 57.30 47.36 0.09 10.56
DVD+R DL Verbatim 8x DVD+R DL 50 @ 8,5GB 425 GB 69.10 57.11 0.13 7.44
DVD+R DL Verbatim 8x DVD+R DL 25 @ 8,5GB 213 GB 36.40 30.08 0.14 7.06
SSD Crucial MX100 SSD, MLC, 512GB 512 GB 199.00 164.46 0.32 3.11
SSD Samsung SSD 840 EVO Basic, TLC, 1TB 1000 GB 399.00 329.75 0.33 3.03
SSD Crucial M550 SSD, MLC, 1024GB 1024 GB 424.00 350.41 0.34 2.92
SSD Crucial M550 SSD, MLC, 512GB 512 GB 224.00 185.12 0.36 2.77
SSD Samsung SSD 840 EVO Basic, TLC, 500GB 500 GB 226.00 186.78 0.37 2.68
SSD Crucial M500 SSD, MLC, 960GB 960 GB 445.00 367.77 0.38 2.61
SSD Crucial M500 SSD, MLC, 480GB 480 GB 237.00 195.87 0.41 2.45
USB Flash Sandisk Cruzer Edge Flash Drive 64GB 64 GB 33.50 27.69 0.43 2.31
CD-R Verbatim CD-R 100 @ 700MB 70 GB 36.80 30.41 0.43 2.30
SSD Samsung SSD 840 EVO Basic, TLC, 750GB 750 GB 404.00 333.88 0.45 2.25
SSD Samsung SSD 840 EVO Basic, TLC, 250GB 250 GB 137.00 113.22 0.45 2.21
SSD Samsung SSD 850 Pro, MLC, 1024GB 1024 GB 635.00 524.79 0.51 1.95
SSD Samsung SSD 840 Pro Basic, MLC, 512GB 512 GB 333.00 275.21 0.54 1.86
SSD Samsung SSD 840 Pro Basic, MLC, 256GB 256 GB 167.00 138.02 0.54 1.85
USB Flash Sandisk Cruzer Edge Flash Drive 32GB 32 GB 21.00 17.36 0.54 1.84
SSD Samsung SSD 850 Pro, MLC, 256GB 256 GB 179.00 147.93 0.58 1.73
SSD Samsung SSD 840 EVO Basic, TLC, 120GB 120 GB 84.20 69.59 0.58 1.72
SSD Samsung SSD 850 Pro, MLC, 512GB 512 GB 378.00 312.40 0.61 1.64
SDXC Sandisk Ultra, SDXC, 30MB/s, 64GB 64 GB 52.00 42.98 0.67 1.49
SDXC Sandisk Ultra, SDXC, 30MB/s, 128GB 128 GB 105.00 86.78 0.68 1.48
USB Flash Sandisk Cruzer Edge Flash Drive 16GB 16 GB 13.30 10.99 0.69 1.46
SSD Samsung SSD 840 Pro Basic, MLC, 128GB 128 GB 112.00 92.56 0.72 1.38
SSD Samsung SSD 850 Pro, MLC, 128GB 128 GB 128.00 105.79 0.83 1.21
SDXC Sandisk Extreme, SDXC, 45MB/s, 64GB 64 GB 65.10 53.80 0.84 1.19
USB Flash Sandisk Extreme Pro, USB 3.0, 128GB 128 GB 135.00 111.57 0.87 1.15
SDHC Sandisk Ultra, Class 10, 30MB/s, 32GB 32 GB 35.00 28.93 0.90 1.11
USB Flash Sandisk Cruzer Edge Flash Drive 8GB 8 GB 8.95 7.40 0.92 1.08
SDHC Sandisk Ultra, Class 10, 30MB/s, 16GB 16 GB 18.80 15.54 0.97 1.03
SDXC Sandisk Extreme SDXC, 80/60MB/s, 128GB 128 GB 162.00 133.88 1.05 0.96
SDXC Sandisk Extreme SDXC, 80/60MB/s, 64GB 64 GB 89.00 73.55 1.15 0.87
SDHC Sandisk Ultra, Class 10, 30MB/s, 8GB 8 GB 12.30 10.17 1.27 0.79
SDXC Sandisk Extreme Pro SDXC, 90/95MB/s, 64GB 64 GB 117.00 96.69 1.51 0.66
SDHC Sandisk Extreme Pro, Class UHS-I, 90/95MB/s, 32GB 32 GB 78.00 64.46 2.01 0.50
Compact Flash Sandisk Extreme 120MB/s, UDMA 7, 64GB 64 GB 167.00 138.02 2.16 0.46
Compact Flash Sandisk Extreme Pro 160MB/s, UDMA 7, 256GB 256 GB 709.00 585.95 2.29 0.44
SDHC Sandisk Extreme Pro, Class UHS-I, 90/95MB/s, 16GB 16 GB 46.10 38.10 2.38 0.42
Compact Flash Sandisk Extreme 120MB/s, 128GB 128 GB 384.00 317.36 2.48 0.40
Compact Flash Sandisk Extreme 60MB/s, 64GB 64 GB 228.00 188.43 2.94 0.34
Compact Flash Sandisk Extreme 120MB/s, UDMA 7, 32GB 32 GB 116.00 95.87 3.00 0.33
Compact Flash SanDisk Ultra 30MB/s, 16GB 16 GB 58.50 48.35 3.02 0.33
SDXC Sandisk Extreme SDXC, 80/30MB/s, 8GB 8 GB 29.30 24.21 3.03 0.33
SDHC Sandisk Extreme Pro, Class UHS-I, 90/95MB/s, 8GB 8 GB 30.30 25.04 3.13 0.32
SDXC Sandisk Extreme Pro SDXC, UHS-II, 280/250MB/s, 64GB 64 GB 247.00 204.13 3.19 0.31
Compact Flash Sandisk Extreme Pro 160MB/s, UDMA 7, 32GB 32 GB 149.00 123.14 3.85 0.26
Compact Flash Sandisk Extreme Pro 160MB/s, UDMA 7, 64GB 64 GB 333.00 275.21 4.30 0.23
Compact Flash SanDisk Extreme 60MB/s, 16GB 16 GB 84.30 69.67 4.35 0.23
Compact Flash Sandisk Extreme Pro 160MB/s, UDMA 7, 128GB 128 GB 708.00 585.12 4.57 0.22
Compact Flash SanDisk Extreme 120MB/s, UDMA 7, 16GB 16 GB 91.00 75.21 4.70 0.21
Compact Flash Sandisk Extreme Pro 100MB/s, 128GB 128 GB 776.00 641.32 5.01 0.20
Compact Flash Sandisk Extreme Pro 90MB/s, UDMA 6, 32GB 32 GB 224.00 185.12 5.79 0.17
Compact Flash Sandisk Extreme Pro 90MB/s, UDMA 6, 64GB 64 GB 452.00 373.55 5.84 0.17

Exchange rate: 1 Euro = 1.210000 CHF.

Comments Off

Why 10TB drives could be bad news

1 comment

In this week’s news, Hitachi GST (HGST) announced a 10 TB HDD. It ups the earlier announcement by Seagate of a 8 TB disk. Although both companies claim the disks are “shipped”, they are nowhere near consumers yet. Both are shipping samples to a select few partners. Tech writers are ecstatic, though, and are happily re-publishing most of the official press release.

10 TB in a 3.5″ HDD is an impressive feat in itself, however the technology used to achieve it might mark the end of general purpose magnetic disks: First, this is another helium based disk from HGST, which stacks seven platters. It remains to be seen how long the helium can be contained within the disk, but is probably a lot shorter than the shelf-life of an air-based magnetic disk. Regardless of lifespan, what makes the 10 TB milestone less impressive, and even slightly worrisome, is the seven platters. That is a one-time trick. We cannot expect disks with even more platters stacked on top of each other in the future, so growth will not continue that route. It has to come from increased areal density of the bits per platter.

The brings me to the second issue with this disk: Areal density is achieved by so called Shingled Magnetic Recording (SMR). As explained by Seagate, it means that each data track is stacked slightly above the other, just like shingles on a roof. To make this work, existing data has to be re-written if something is changed in neighboring tracks. This could result in delays when writing data on an almost full disk. Furthermore, it will require special handling by the OS (in case of the “host managed” 10 TB HGST disk). Implementing that right might not be trivial. SMR based disks are in other words not drop-in replacements for all applications. Although, for scenarios which are write once (or infrequently), and read often, this is not a problem.

Combined, these two technologies will most likely take us to 20 TB and beyond. Assuming the 8 TB Seagate disk used five 1.6 TB platters, a seven platter disk would give 11.2 TB. That is of course the advantage of more platters, it amplifies any advance in areal density. Between five and seven platters, the factor is 1.4, and between four and seven it’s 1.75.

The trend of ever decreasing HDD prices will continue, even though the products might not be for everyone.

Open a Firefox image in a VirtualBox guest OS

Comments Off

In my recent Ubuntu and Windows install for my father, a requirement was to open images shown in Firefox (which runs on the host Ubuntu OS) in Corel Paint Shop Pro which runs in the VirtualBox guest Windows 7 OS.

The command to execute an application from the host in the guest is vboxmanage guestcontrol [...] exec, documented here. It also needs the (guest OS) user credentials, the application to run and its arguments. Also note, that back-slashes have to be escaped, and since the path contains spaces it has to be in quotes.

The following assumes that the name of the VirtualBox machine is called “win7″, and application to run is PSP, and shows example values for username, password and filename.

Note that it also assumes that the Windows drive S:\ is a shared mount point between the host and guest OS, and that downloaded images is saved to that location.

vboxmanage guestcontrol win7 exec --image "C:\\Program Files (x86)\\Corel\\Corel PaintShop Pro X5\\Corel PaintShop Pro.exe" --username my_guest_user --password my_guest_password -- "s:\\image_file.png"

So far so good. However, to execute this for any image in Firefox proved a bit tricky. It does not seem to be possible to associate an Open action from the right-click menu with a given application in Firefox. The plugin “Open With Photoshop” came to the rescue. It gives an extra menu item when right-clicking on any image, and what most useful, it let’s you choose the executable to run.

I took the command above, and created my own script, e.g. “corel_psh.sh”, which looks like this:

#!/bin/bash
 
WINUSER=my_win_user
PASSWORD=my_win_password
 
f=$1
b=`basename $f`
 
vboxmanage guestcontrol win7 exec --image "C:\\Program Files (x86)\\Corel\\Corel PaintShop Pro X5\\Corel PaintShop Pro.exe" --username $WINUSER --password $PASSWORD -- "s:\\$b" &> /tmp/open_with.log

I suppose a valuable addition would be to move the downloaded file to the correct location if necessary.

Comments Off

Seagate brags about 8 TB disks

Comments Off

It was only a few days ago I mentioned Western Digital’s new 6 TB disks. It seems Seagate got nervous, and are now claiming to “ship” 8 TB disks. However, it is a misleading headline at best: The drives in question are not on sale anywhere, and only special “pre-production prototypes” have been given to few select partners. Furthermore, the initial batches will be aimed at enterprise customers who can afford them. As an early-adopter strategy, that makes sense, but it means you will not get your hands on one any time soon. So, the prediction from a few days ago holds: 8.4 TB is already within reach, with 7 platters * 1.2 TB, but needs special technology like helium filled drives. We will probably not have 8 TB drives before 2015.

The bit-tech article is also very light on technical detail, presumably because Seagate holds their cards close. There is no mentioned on how they have gotten to 8 TB, only that 10 TB might also be within reach soon. The later would be very interesting, since even with 7 platters it means an increase in areal density to 1.4 TB per platter. Or if we assume 5 platters, an impressive 2 TB. That will probably not reach the market before end of 2015, possibly 2016.

Comments Off

6 TB disks, 1.2 TB platters finally here

1 comment

Earlier this week, Anandtech could report that Wester Digital has finally taken the next incremental step in spinning HDD size. The WD Red series will now come in sizes of 5 and 6 TB, as will the Green series. 6 TB is accomplished using 5 platters, which is a new trend for Western Digital. As mentioned earlier, the five-platter technology was acquired from Hitachi, and we now start seeing them in NAS type WD drives. With five platters, it means each has 1.2 TB, which is shy of the 1.25 TB expected this year. It is important, since it means the 5 TB version also contains five platters, of 1 TB each, rather than 4 * 1.25 TB.

As the AnandTech article points out, WD is still using Perpendicular Magnetic Recording (PMR) technology, as in previous drives. It means that the 1.2 TB areal density should be easily compatible with the HGST seven platters helium drives. That should make 7 * 1.2 = 8.4 TB drives immediately possible. However, as marketing seems to be focused on whole integer increments only, expect 7 and 8 TB drives in this combination. That would be in line with the predictions seen here last year.

As for price, the WD Green is not yet announced. For the Red series, 5 TB will be $250, and 6 TB $300, which puts both in at 5 cents per GB. That is 39% more expensive than the cheapest Seagate 3 TB disk in the storage graphs. However, the newest and biggest drives always start at a premium for the early adopters. These new drives will probably mean that the previous step, 4 TB, will rise to the top as the best GB per buck.

Making an ARM Linux based computer from scratch

Comments Off

Over at Henrik Forstén’s blog, he has a write-up of his very impressive project where he designed, assembled, soldered and installed an BGA (Ball Grid Arry) ARM based board from scratch.

He discusses board design challenges with a four-layered PCB, considerations with traces for DDR2 RAM, CPU, and three voltage supplies. There are many pictures showing the soldering process. His summary is: “Many people say that soldering BGAs is hard but based on this experience I can’t agree. Maybe I just got lucky but I didn’t have any problems with them.”

Once the board is all put together, he goes on to boot Linux. That also proves somewhat tricky, and he ends up with a three-phase boot using an ARM bootloader, U-boot, and finally a custom built kernel.

He says, “I don’t really care about the usefulness of the board and this whole project is more of a learning experience”. Clearly it was a great success.

Comments Off

DealExtreme orders

Comments Off

DealExtrme, always useful stuff! Some of the things I’ve ordered, listed here so I can easily find it and order even more…

V117 Dual-Side SSOP16 / TSSOP16 / SOP16 SMD to DIP Adapter Boards Set - Green (2 PCS) E5YK Stainless Steel Aquarium Five Head Water Tank Regulating Valve - Black + Silver 5 x 5.5mm DC Power Extension Female Cable - Black (10 PCS) Male + Female DC Power Converter Connector Adapters w/ Terminal Blocks For CCTV Camera (Pair) Male + Female DC Power Converter Connector Adapters w/ Terminal Blocks for CCTV Camera (Pair) 1-to-2 Power Splitter Cable for CCTV Security System Camera (DC 12V) 5 x 5.5mm DC Power Extension Female Cable - Black + Silver (10 PCS) Aquarium Fish Tank Tubing Straight Connector T Splitter for 4mm Air Line (24 PCS) 5.5 x 2.5mm Plug AC Power Adapter - Black (AC 100~240V / EU Plug / 135cm-Cable) Ethernet Shield with Wiznet W5100 Ethernet Chip / TF Slot GM700 1.5 1298A 12.6V 9800mAh Rechargeable Li-ion Battery w/ US Plug Power Adapter Hubsan H107-A26 Body Shell for H107C R/C Quadcopter - Red + Black Hubsan H107-A21 Body Shell for H107C R/C Quadcopter - Red + Silver H107C-008 R/C Helicopter Replacement Blades for JD385, 310, 310B, YD928, F180, V252, H107, H107L Replacement 3.7V 500mAh 25C Lithium Polymer Battery for Hubsan H107 4-Axis - Blue + Black 0.9 XL7105-SY DIY 2.4GHz A7105 NRF24L01 Wireless Module for Arduino (2 PCS) 433Mhz RF Transmitter Module + Receiver Module Link Kit for Arduino / ARM /MCU WL - Green Hubsan H107C-A24 3.7V 380mAh Li-po Battery for H301C R/C Quadcopter - White 20906 6mm Silicone Hose - Translucent White (5.44m-Length) Hubsan H107-A02 Replacement Blades for X4 H107 Quadcopter - Black + White (4 PCS) E5YK Water Tank Adjusting / Regulating Valve - Black (10 PCS) E5YK Stainless Steel Aquarium Five Head Water Tank Regulating Valve - Black + Silver Desktop Wire Cord Cable Clip Organizer - Yellow + Green + Red (6-Piece) E5YK Aquarium Fish Tank 1-to-6 Air Splitter - White E5YK Water Tank Adjusting / Regulating Valve - Black (10 PCS) Aquarium Fish Tank Flexible Silicone Air Line Tube - Black (18m) 20903 Silicone Tube - Translucent White (5m) E5YK Inlet Air Pipe Air Control Valve for Fish Tank / Aquarium - Yellow (20 PCS) E5YK Aquarium Suction Cup Airline Tube Holders Clips - Transparent + White (10 PCS) DX 2014 Desk Calendar with 12 Months Hubsan X4 H107C 2.4G 4CH R/C Quadcopter w/ 0.3MP Camera - Black + Red (Mode 2) Protective PU Leather Case w/ Card Holder Slots for LG Nexus 5 - Deep Pink Protective PU Leather Flip-Open Case w/ Holder for Google Nexus 7 - Red Detachable 59-Key Bluetooth V3.0 Keyboard Case for Google Nexus 7 II - Blue M-013 Door Entrance Guard ID Card - White (10 PCS) 125K RFID Card Reader Module / RDM630 Series Non-Contact RF ID Card Module for Arduino - Green + Red 3.7V 350mAh 30C Li-ion Battery for Hubsan X4 RC Quadcopter + Walkera V120D06 - Silver Hubsan H107-A05 3.7V 240mAh Li-ion Polymer Battery - Black Hubsan R/C Spare Parts H107-A06 USB Charger for H107 / H107L R/C Quadcopter - Black Y-11 1 Female to 3 Male DC Power Splitter Adapter Cable - Black (35 cm) Male + Female DC Power Converter Connector Adapters w/ Terminal Blocks For CCTV Camera (Pair) Jtron Building Universal Terminal Block / Quick Connector 5 Holes / Wire Connector - Grey (5 PCS) 3P Universal PA6 Terminals - Grey + Red (10 PCS / 250V / 20A) 4-Channel 5V Optocoupler Isolation Relay Module w/ High Level Trigger - Blue Hubsan H107-A02 Replacement Blades for X4 H107 Quadcopter - Black + White (4 PCS) HSYY01 Micro Gear Water Pump Motor w/ Hose - White + Silver ENC28J60 Ethernet LAN / Network Module for 51 AVR STM32 LPC Ethernet Shield with Wiznet W5100 Ethernet Chip / TF Slot Jtron 0.36 HD2V04 HDMI to VGA + 3.5mm Audio Jack Converter Adapter Box - Black Universal AC Charger w/ Dual USB Output for Iphone / Ipad / Ipod - White (US Plug) USB 2.0 to Micro USB Charging Cable for Samsung / HTC / BlackBerry - White (200CM) Raspberry PI Acrylic Case - Transparent 5V 2A Wall Power Adapter for Scanner / Surveillance Camera + More (US Plug) W-1 E27 Automatic Rotating 3W 300lm Colorful RGB Light 3-LED Lamp for Decoration (85~265V) E14 6W 500lm 6500K White 15-SMD 5630 LED Light Bulb - White (220V) Lexin E14 4W 350lm 34-5050 SMD White Light Corn Lamp (220~240V) USB Terminal Power Adapter Voltage Current Tester - Grey + Black GH-10W 10W 430lm 9-LED Red + Blue Light Plant Grow Light Module - Silver + White (7.5~8V) Jtron 0.36 8-Channel 5V Relay Module Shield for Arduino (Works with Official Arduino Boards) JY-MCU 5V 3V IIC UART SPI Level 4-Way Converter Module Adapter HSYY01 Water Pump Motor w/ Hose - White + Silver 20906 6mm Silicone Hose - Translucent White (5.44m-Length) Aquarium Fish Tank Flexible Silicone Air Line Tube - Black (18m) E5HT Aquarium Air Tube - Transparent Blue (10m) Aquarium Fish Tank Tubing Straight Connector T Splitter for 4mm Air Line (24 PCS) E5YK Aquarium Fish Tank 1-to-6 Air Splitter - White USB Powered Flexible Neck 10-LED White Light Lamp - Blue (27cm) 4-Port High Speed USB 2.0 Hub - Black (60cm-Cable Length) Mini USB 4 Ports Hub 4-Port USB 2.0 HUB w/ Independent Switch - Black 5V 2A Universal Power Adapter Charger - Black (AC 100~240V / EU Plug / 3.5 x 1.35mm) Raspberry PI Acrylic Case - Transparent Protective Neoprene Bag Case for DSLR Camera Lens - Black (Size XL) Protective Neoprene Bag Case for DSLR Camera Lens - Black (Size L) HSYY01 Water Pump Motor w/ Hose - White + Silver W3-9 Immersible Water Pump for Miniature Garden - Off-white SZF280 PVC Mini Water Pump Motor - Beige Miniisw SW-015 1.5W Polysilicon Solar Panel - Black Miniisw SW-008 0.8W Solar Powered Battery Panel Board - Black Protective Jellyfish Pattern Silicone Back Case for LG E960 Nexus 4 - Multicolored Replacement Sound and Music Activated Spectrum VU Meter EL Visualizer - Smile Face (4*AAA) 5.5 x 2.5mm Plug AC Power Adapter - Black (AC 100~240V / EU Plug / 135cm-Cable) 1.5 5.5 x 2.5mm Plug AC Power Adapter - Black (AC 100~240V / EU Plug / 135cm-Cable) HDMI Female to Micro HDMI Male Adapter 40-Compartment Free Combination Plastic Storage Box for Hardware Tools / Gadgets - Translucent White 24-Compartment Free Combination Plastic Storage Box for Hardware Tools / Gadgets Panel Mount 10A 250V Fuse Holder - Black (5-Pack) Optical Triple Triangular Glass Prism Spectrum - White Dupont 4-Pin Female to Female Extension Wire Cable for Arduino (40cm / 10-Piece Pack) Dupont 4-Pin Male to Female Extension Wire Cable for Arduino (40cm / 10-Piece Pack) Universal DIY Bakelite Plate PCB Board - Brown (2-Piece Pack) Universal Glass Fiber PCB Board for DIY Project - Brown Prototype Universal Printed Circuit Board Breadboard - Brown (5-Piece Pack) Nano V3.0 AVR ATmega328 P-20AU Module Board + USB Cable for Arduino Nylon PP6 DC 12V 50mA Tact Switch - Black (100-Piece Pack) 1N4007 1000V 1A Unilateral Rectifier Diodes Set - Black + Silver (50 PCS) LM7805L 5V Voltage Regulator ICs (10 PCS) 2.54mm 1x40 Pin Breakaway Straight Male Header (10-Piece Pack) GP LR44 A76 1.5V Cell Button Batteries 10-Pack 5 x 20mm Glass Tube Fuse Set - Silver (100 PCS) LCD Keypad Shield for Arduino Duemilanove & LCD 1602 (Works with Official Arduino Boards) Protective Plastic Case for 3.5 635~645nm 800~1000MCD 5mm LED - Red (100-Piece Pack) 510~520nm 800~1000MCD 5mm LED - Green (100-Piece Pack) S1306 8-in-1 Gradual ABS Lens Filters + Lens Mount + Ring Set for 77mm Lens Camera - Black Unique Black 4 Series Armed Notebook - Rambo Knife (60-Page) Convenient Rectangle Sticky Note Memo Pads (4 x 100 Pieces) Stainless Steel 1/4 C2-07 Creative Inflatable Shoe Boot Support Spreader - Milk White (Pair) Double-Sided Glass Fiber Prototyping PCB Universal Board (12-Piece Pack) Double-Sided Glass Fiber Prototyping PCB Universal Board (3 x 7 / 5-Piece Pack) Prototype Universal Printed Circuit Board Breadboard - Green + Silver 3mm & 5mm Light-emitting Diode - Green + Red + Yellow (100-Piece Pack) Breadboard Jumper Wires for Electronic DIY (65-Cable Pack) 4 Channel 5V High Level Trigger Relay Module for Arduino (Works with Official Arduino Boards) 2-Channel Relay Shield Module for Arduino (Works with Official Arduino Boards) AMS1117 5V Power Supply Module Emolux 62mm Multi-Coated UV Lens Filter - Black Sound and Music Activated Multi-Mode Flashling EL Hearts T-shirt - M (3*AAA) Silica Gel Reusable Moisture-Proof Bead Desiccant - Blue Male + Female DC Power Converter Connector Adapters w/ Terminal Blocks For CCTV Camera (Pair) Universal Heavy Duty 6F22 9V Battery DSTE NB-7L Replacement 7.4V 1200mAh Battery for Canon G10 / G11 - G12 / SX30 IS - Grey 1/4 Universal Aluminum Alloy Straight Flash Bracket for Camera - Black Universal Aluminum Alloy Tripod Bracket for Speedlight / Camera- Black Universal Handheld Jar Opener White Magic Beans with Assorted Messages (10-Pack Growing Plant) Genuine Acecamp 2429 20L Outdoor Water Resistant Dry Bag - Yellow 1000mA Car Cigarette Powered USB Adapter/Charger (DC 12V/24V) DIY 433MHz Wireless Receiving Module for Arduino (Works with Official Arduino Boards) 433MHz Wireless Transmitter Module Superregeneration for Arduino DIY 16-Key AD Keypad Module - Blue 4 x 4 Matrix Switch Module - Green ES-71 II Lens Hood for Canon Mini Prototype Printed Circuit Board Breadboard for Arduino (5 PCS) Ceramic Capacitor for DIY Electronic Circuit - Red (270-Piece Pack) Solderless Breadboard with 400 Tie-Point (White) USB to RS232 Serial Port Adapter (Transparent Green) FreArduino Soil Humidity Sensor for Arduino (Works with Official Arduino Boards) Double-Sided Glass Fiber Prototyping PCB Universal Board (3 x 7 / 5-Piece Pack) DIY HR-202 Humidity Detection Sensor Module - Blue Aluminum Alloy Straight Hot Shoe Flash Bracket for Camera - Black Flash Diffuser for Canon 580 EX / EX II / YongNuo YN560 / YN565 Speedlite (3 PCS) HT RJ45 RJ11 Cable Tester Stainless Steel 1/4 Mini USB 2.4GHz 150Mbps 802.11b/g/n WiFi Wireless Network Card Adapter - Black USB 2.0 2.4GHz 802.11b/g/n 150Mbps WiFi/WLAN Wireless Network Adapter Ultra-Mini Nano USB 2.0 802.11n 150Mbps Wifi/WLAN Wireless Network Adapter 6.3V 3300uf Aluminum Motherboard Capacitors (20-Piece Pack) DIY ZIF DIP IC Socket Set - Green (8 PCS) Solder Tip Refresher 3-Pin Triode Transistor for DIY Project - Black (20 x 10-Piece Pack) Aluminum Electrolytic Capacitor for DIY Project (120-Piece Pack) DisplayPort DP Male to HDMI Female Adapter Cable - Black (15CM) Gold Plated 1080i HDMI V1.3 M-M Connection Cable (1M-Length) 11x12 132-Panel Brain Teaser Magic IQ Ball Micro USB To HDMI MHL Adapter - Black Gold Plated HDMI Male to DVI 24+1 Female Adapter 62mm Digital Camera Lens Cover Digital Camera Lens Cover/Cap with Strap for Canon (62mm) 7.4V 1200mAh Lithium Polymer Lipo Battery Pack for Lama or 4-CH R/C Helicopters HC-SR04 Ultrasonic Sensor Distance Measuring Module 4 x AA Battery Case Holder (3-Pack) Stainless Steel Triple Razor Blade Set (4-Pack) USB 2.0 Smart ID Card Reader - Silver 30cm Breadboard Wires for Electronic DIY (40-Cable Pack) Cree XR-E Q2 Emitter with Star 3W LED Emitter on Star (Multicolored RGB)

Comments Off

Historical Cost of Computer Memory and Storage

Comments Off

I’ve finally gotten around to update the graph and data at hblok.net/storage. Since it has been a while since last time, I thought I would be left with a gap in the data. However, John C. McCallum who collected the original data got my back. I’ve incorporated his newer data points, as well as automated my own collection. Hopefully, it will not go quit so long between updates in the future.
 
 

RAM prices up

 
Due to a fire at the Hynix Fab plants in 2012, RAM prices are significantly up, and still at 2012 prices. Joel Hruska at extremetech.com has an interesting analysis into its effects. In the chart below, he notes that prices doubled after the fire. The latest data I’ve collected show that it’s going in the right direction again. However, as Hruska also points out, the market for desktops is in decline, and laptops, tablets and phones will not need the same memory types.


 
 

HDD / Magnet disk prices go sideways

 
In 2011 flooding in Thailand caused major damage to HDD factories of Western Digital, Samsung, and Toshiba. Prices have almost recovered from that by now, however it means a set-back in the HDD trend by almost three years.

In another article by Joel Hruska, he includes the graph below from Backblaze who plotted real and estimated prices on different HDD types. Although he takes issue with Backblaze’s extrapolation, it did get the significant lag of the trend right.

Furthermore, larger drives have yet to materialize. The 6 TB HGST (owned by Western Digital) Ultrastar He6 has been out for a while. Paul Alcorn at tweaktown.com discussed the details in a review a few months back. The trick is to put a whopping seven platters into the drive (as opposed to the normal three to five). However, to make that work, the drive has to be sealed and filled helium; thus its name. The new technology comes at a higher prices: It retails at Amazon for $476.99, or $0.0794/GB. That’s more than double the Seagate 3TB at $109.99 or $0.0367/GB.

So although the helium technology has a long way to go, it could be the next step for higher capacity drives. At 6 TB over 7 platters, each hold only 857 GB. That is a bit off from the 1 TB platters which have been available for a long time now, and we were “promised” 1.25 TB platters by the end of this year. If the later are compatible with the “7Stac” technology, it could mean 1.25 * 7 = 8.75 TB drives in the future. If pricing also improves, it could be that the magnetic hard disk trend is back onto its 40 year track.

 
 

SSD and flash prices down

 
Flash memory and Solid State Drives have no accidents hampering their growth and price decline. The trend is linear (i.e. on the logarithmic scale) over the last ten years. Larger drives are also gradually becoming available, with SanDisk recently announcing their enterprise 4 TB Optimus MAX SAS, and predicting 8 TB SSDs by next year. In that Computer World article, they’ve included a Gartner graph, seen below, which predicts SSD price parity with HDD by 2017. It’s important to note that they compare enterprise drives here, which live in a completely different world than the cheapest and biggest consumer drives.

For the chart I’m tracking, it is mostly the USB sticks that make it to the top, with the best price / capacity ratio. At this point, it seems that is not because the USB sticks are getting larger, but rather that the smallest ones are getting cheaper. SanDisk is now “giving away” 8 GB drives for less than $3 (although that does not give you free shipping at NewEgg).

 
 

Updated overall graph

Finally, the updated graph. The permanent link is hblok.net/storage
 

(Click for larger image)

Comments Off

Microsoft-Nokia’s new phone: It’s an Android

Comments Off

In a surprise move, the result of the Nokia buyout by Microsoft is a new Android based phone, the X2. (Yes, I double checked that it was not a 1st April story). BBC reports that the mid-range smart-phone will be Android based, but that the UI will look like Microsoft’s Windows Phone.

At an estimated price of 100 Euros, the specs are not overwhelming, with 1 GB RAM, 5 MP camera. However, interestingly it’s a dual SIM phone. That suggests it’s targeting the Asian market, where people are shopping around for the best SMS and calling rates, and dual SIM phones are very popular.

As expected, the phone will not feature the common Google service apps, like Gmail, Calendar, Hangouts, Maps, and Youtube, but instead replace them with Microsoft equivalents like Outlook, Skype, and Bing. However, it also means that the Google Play store will not be available either, so Microsoft and Nokia will somehow have to back-fill their own market. Or perhaps developers will have to submit their Android apps to yet another market. Many open source apps are already dual-hosted on Google Play and the free software based app market F-Droid.

Comments Off

Default PDF viewer in Debian

Comments Off

By some strange logic, the primary and default application for viewing PDFs in Debian is Gimp. If you want to edit the PDF, that might make sense, but that is not the most common use case. There is a bug and discussion about this, but unfortunately, in somebody’s stubborn opinion, “it is not a bug”, and was closed many years ago.

Luckily it is easy to fix. The default setting can be found in the file /usr/share/applications/mimeinfo.cache which contains this line:
application/pdf=gimp.desktop;gimp.desktop;epdfview.desktop;evince.desktop;

Notice how Gimp is listed first, while the PDF viewers ePDFViewer and Evince are last in the list. You can edit that file (as root). Or if you prefer you can override the user local setting in /home/$USER/.local/share/applications/mimeinfo.cache, and insert something like

application/pdf=epdfview.desktop;evince.desktop;

The change should take effect immediately, across all applications and browsers, unless the default is overridden there. E.g. Firefox and Chrome have their own internal PDF viewers, however the default MIME applications will be available for selection when the file is downloaded.

Comments Off

Trends: Old social networks are past their peak – What’s next?

Comments Off

The doom and gloom of Facebook has become a common story. For example based on of a redesign of the page, or the fact that 14-year-olds wouldn’t want to be seen on the same social network as their parents. It’s usually speculation, however according to the last link, it is now also admitted in Facebook’s own earnings announcement. Furthermore, the search trends on Google also confirm the shift in interest.

Before jumping into these graphs, it is worth pointing out some caveats, even though they should be obvious: First, historical trends are no guarantee of future events. Secondly, the numbers are not absolutes, but capped fractions of Google’s keyword distribution. Thirdly, most of the searches will be “navigational”, that is typing “facebook” in the URL bar will invoke a Google search, and then immediately take the user to the top result. Therefore, part of the shift in the search trends might be that more people are using native mobile apps where such “navigation search” is not required. Fourthly, these trends are for Google searches, which of course exclude users of search engines like Bing, Yandex, or almost any Chinese user. Finally, statistics lie and cannot always be trusted. You have been warned.

Never the less, there are noticeable changes in the social and communication network landscape. Facebook, Twitter and Flickr seem to have reached a peak and might plateau or decline, while new-comers like Instagram, Reddit, WhatsApp, Snapchat all see steady growth. And then there are the solid stayers who full-fill basic communication needs and services: Gmail, Hotmail, Youtube, etc. They stay, but without much change in their search trends.
 

Past their peak

 
The following graphs show the search trends for the terms Facebook, Twitter, and finally Flicker and Picasa compared. (The reason more of the graphs are not compared is that the numbers are often on very different scales. E.g. compare Facebook to Google+, and the later hardly registers as a blip on a flat line a bit over zero). All the sites below show clear decline past their peak point. That peak was some time in 2013 for Facebook and Twitter, while Flickr and Picasa had a peak around 2010 and bumpy decrease in interest ever since.

What’s noteworthy about the Facebook graphs is first the significant and sudden increase around the beginning of 2013. It is unclear what caused this, however looking at the country specific graphs it is more present in Brazil than the others. From the Orkut graph below, it drops right down since mid-2012. That was about the same time that network was all but turned off by Google. It could explain some of the increase in searches for Facebook.

Secondly, what’s interesting about the country specific Facebook graphs are the relatively flat lines for UK and US, while Brazil and India come later to the party, but show rapid increase. Again, this is probably due to the Orkut effect, since that is where that network was most popular. Beyond that, we should be careful reading too much into that chart, since numbers are not normalized, nor adjusted for population, etc. Still, all trends are clearly down for Facebook in every country.

For the other networks, it is interesting that Twitter seems to follow the same trend as Facebook, albeit at a much lower scale. (Compare the two in the same chart, and they are some two multitudes apart). Although there is not much linking Facebook and Twitter, they do perhaps belong to the same era of sorts. Gaining widespread popularity around the same time in 2009, the same people might have been users of both, and now start to lose interest in both. It will be interesting to see if those comparative trends continue.

Finally, Flickr and Picasa are shown together, both declining over the last four to five years. What’s perhaps most interesting here is that they follow each other so closely. So how do people share photos if not on these services? Look no further than the graphs and section below: Instagram and Pinterest are the new Flickr and Picasa.


 

What does decline look like?

 

So how does a social network or popular website decline? The following graphs show some large networks through their rise, peak and fall. The common theme in all of them: There is no coming back. Once a trend has faded, it is gone, and the users have moved on. So even if they are not on the list of defunct networks yet, expect no return to greatness. At least in fashion and culture, there is the concept of retro. With ICQ and MySpace, probably not so much. Having said that, the Wikipedia article on ICQ claims there are still 100 million users, but also points out that those are mostly in Russia and Eastern Europe. Google Search might not have such a strong foothold there as in the West, thus that popularity is not reflected in this graph.

The other sites share a similar story: Although MSN is still in use by many, Facebook, Whatsapp, and plenty of built-in chat applications in everything from Gmail to World of Warcraft have taken its place in instant messaging. Orkut has more or less been merged with Google+, and I’m actually not sure if you are able to log in to what once was the Orkut specific site.

Digg lingers on, with a few attempts to stay relevant, but most users have moved on to other forums. Mostly to Reddit, it seems, from the discussion there. See the Reddit chart below.

Finally, nobody would want to be seen on MySpace anymore. Once the hip place for up-and-coming artists, it is now a has-been. Even so, it still claims some 30 million users, and in 2011 was bought by Specific Media Group and Justin Timberlake. It is not clear what they wanted to do with the site, but it might have been cheaper to launch a new service under a new domain. $30 M is after all a bit expensive for the old domain, especially since fewer and fewer searches link there.


 

Who wins? Who stays?

 
Now for the winners of the Internet based game of musical chairs. Some clear “hockey stick charts” can be seen: WhatsApp, SnapChat, Instagram, and Pinterest have all seen drastic increase in user interest over the last two to three years, all with no or little sign of decline. Furthermore, Reddit see continued growth. Then there are a group of services which continue to linger on, with no spectacular increase in interest, but overall very solid user numbers. These include the common email services Gmail, Hotmail and Yahoo, as well as Youtube for video sharing.

In the first graph below, it is interesting to note the significant down-tick in WhatsApp’s graph. According to the news annotations on Google Trends, this was exactly at the point where Facebook’s purchase of the company was announced. At the same time, we can see a stark increase in interest for the competing service SnapChat. For $19 billion dollars, Facebook might have bought the most expensive domain name of all time in an attempt to stay on an upwards trend. It will be interesting to watch how these trends develop over the coming year.

Next is the photo sharing site Instagram compared to Reddit, however maybe it would have been more natural to compare it to Pinterest. Either way, Instagram is clearly seeing strong and increasing interest. However, what will happen when it loses its “hipster” early adopter appeal? Many of its initial users seem to have been of the type which live to shun the mainstream. Other photo upload services like Tumblr, and Imgur are also popular. The former is flatting out a bit, while the later is mostly tied to Reddit, and see steady growth.

As for the discussion forum and online community site Reddit, they continue to see consistent growth. They are catering to pretty much anybody who care to make a group on any particular topic, and have shown very little sign of censuring content. It could turn out to be the Usenet or Yahoo groups of this decade.

The next chart compares Google+ to Pinterest, more as a reference point than those two being related in any way. What’s interesting to see though, is the extreme marketing power which comes with any new product Google launches. At its announcement in June 2011 everybody wanted to know what it was about, however that interest clearly did not keep up. In that regard it’s compelling to compare it to Pinterest, which saw a similar high spike, but managed to keep that momentum going, although with little increased growth.

The penultimate graph shows Youtube, with slow and bumpy growth. The site can handle videos which attract billions of views, so it is clear that there are few or no competitors out there for video sharing at such an enormous scale. However, as a social network and communication tool it is perhaps less successful. Once infamous for the most banal useless and outright hostile user comments on the Internet, they now face user revolt over Google+ integration. If all Youtube provide is infrastructure for large files, which happen to be video, they might be overtaken. If on the other hand, they manage to take the next step into the living room, and replace the TV, they could see growth in years to come.

The final graph compares the three large email providers Gmail, Hotmail, and Yahoo. Although there are some shifts in interest for all of them, it is nowhere the ups and downs seen in the other charts above. These providers will stay around for a long time, since email will continue to be relevant even if young Facebook users might deny it. Email will continue to be used for communication with businesses, registration to almost everything on the Internet, as well as conventional letter communication. Furthermore, both Hotmail and Gmail are gateways into other products from Microsoft and Google. For example, all Windows and Android users are now strongly compelled to sign up for an account with their respective companies. Gmail also contains an alternative to Skype (which is missing from the chart; although their trend is also rather flat) voice and video communication in the form of Hangouts. In other words, these services are not fads, but rather form the backbone of basic Internet based communication. The reason the don’t see further growth, is probably because they are coming close to saturating their current markets.

 

Federated services

 
What does the future of social networking and communication look like. It would be foolish to try to predict who will come out on top. There are too many factors and too many unknowns. However, it seems safe to bet on a few basic trends, founded on core ways of communicating. These would fall into the following groups:

  • Asynchronous letter based communication; e.g. mail, email.
  • Live voice and video communication; e.g. phone and video conferencing.
  • Informal messaging, both instant and asynchronous; e.g. chat, SMS.
  • Networking, self-promotion; e.g. social networks.

My prediction is that all these types of communication will stay and have strong presence in decades to come. It is maybe a boring prediction, since at its core it says that everything will stay the same. However, looking at very broad lines that has been the case for centuries. We tend to get blinded by new shiny technology and fashion, and forget that rounded corner on an iPhone does not change the basic form of communication which is voice over a telephone network.

Another example is letter or asynchronous one-to-one targeted message based communication. That has existed for millennia, with modern postal systems perhaps being one of few the significant changes before email arrived. We tend to take the fact that a letter or postcard can be sent from anywhere to anywhere in the world for granted, just as we take the Internet based electronic mail infrastructure for granted. What is important to highlight with both systems is that they are federated. That is, a message can be sent between different, possibly heterogeneous, countries or email systems / servers, without the users of these systems having to worry about infrastructure implementation. For the postal system, that is thanks to the Universal Postal Union, and for email is because of common APIs, where the SMTP (Simple Mail Transfer Protocol) is at the core. It is important, because this is a property which is missing from many modern web based communication tools.

Old style telephone networks also have this property: You can call from any wired or wireless phone in the world to any other, crossing multiple networks involving many different technologies. That has been the case for more than hundred years. “[S]ince the introduction of the public telephone system in the late 19th century, [it has stayed] mostly unchanged despite the introduction of Touch-Tone dialing, electronic telephone exchanges and fiber-optic communication into the public switched telephone network (PSTN).” (Wikipedia) The notable addition in recent times is video based calling, however that does not change the form and motivation for this kind of communication much. It is still mostly used to bring people closer together, to hear eachother’s voice, or see eachother’s face. It is instant synchronous and intimate. Which of course is why it is preferred by telemarketers; it is often harder to hang up on a call than to throw away flyers received in the mail box.

Compared to mail and phone, SMS and chat are all relatively new communication forms. Perhaps that is why it is not always clear how to relate to them. For example, is SMS and chat asynchronous or synchronous? From a technical point of view, they are both asynchronous: Both parties can send messages without waiting for the other, and message passing is starting and stopping independently. However, using it that way is typically not acceptable. It would be considered as rude to flood the other party with messages, just as it would be rude to interrupt while talking, or not let the other speak. However, how quickly are you expected to reply to a chat message or SMS? Is it instant, as the IM name suggests, or is it acceptable to linger? For how long? There are no set social protocols or rules here, and different communities will make them up as they go along.

Despite those uncertainties though, two things are clear: Instant messaging in the form of chat, SMS, WhatsApp or other implementations will stay as a separate communication form for the foreseeable future. Secondly, most of these services are currently not federated: That is, you cannot send a MSN message to a WhatsApp or Yahoo Chat user. That seems ripe for change. In fact, there is already a protocol for federated instant messaging: XMPP (Extensible Messaging and Presence Protocol) (aka. Jabber). It still has some way to go before it becomes the preferred way of exchanging chat messages, and there might in fact be other protocols which supersedes it. Regardless, the isolated short-message services we’ve seen over the last twenty years will be forced to standardize. Only then can they become stable back-bone services like the email providers seen in the chart above. The alternative is fads which come and go, which is what we can see in the graphs for the services of today.

The same is bound to happen for other web based communication as well, including video communication, VoIP (voice over IP), and social networking. Currently, you cannot call a Skype user from a Google Hangouts account. For VoIP, SIP (Session Initiation Protocol) is an important protocol, but there are many more often conflicting standards involved. For video, it gets even more complicated, with a long list of video formats, but few standards, and even fewer open standards which would be a necessity for a truly open federated system. (Email would not have gotten where it is today if the SMTP, POP and IMAP RFCs had been patented and closed).

Finally, something similar might happen for social networking, although that is the furthest out, and might not happen at all if the large “social networks” turn out to be mere promotion platforms and substitutes for self-hosted web sites. The feature that would make most sense to standardize on for social networks is the contacts or friends graph. That way, you could create a graph between users of different services like Facebook and Google+. Your contacts on your phone might also appear in your chat, and your social network, and all platforms would understand the links to other users across networks. Of course, that is exactly what many users might not want, as I touched on at the very beginning of this article. You could have something like the Google+ circles concept, where groups of friends and family can be segregated into different audiences. Is this something the causal user is ready to manage? Or will he just create different account for different purposes? Regardless, there would still be benefit in a federated social graph, as it would not matter where an account lived, and one would not have to chase the latest trend, just as email users stay relatively stable today.

On the other hand, if sites like Facebook and Google+ are mere content hosts similar to blogs and web sites, the contact graph might not be so important. It becomes a publishing platform for people who cannot or will not set up their own host and website. If that is the case, what is more important for the user, is that the content can be exported from one site and imported into another. Less he becomes a tenant or even serf locked to his host. Some services have been good at offering those features. However, it will leave them vulnerable to shifting trends and users moving on. Finally, for users who just use Facebook for the games that are hosted there, no message interchange or federation would be required. It is just a content provider and host which can be swapped out at any time. When Facebook dies, will the next FarmVille simply be hosted on Amazon EC2?

 

Conclusion

 

The search popularity of the major web sites and services above show the repeating cycles of culture and trends. Some sites and networks become popular, at the expense of others which fade away. Meanwhile, basic communication forms like letter writing and voice continue to be used in exactly the same way they have for centuries. The underlying technology might change, and modern objects will always seem more shiny and fashionable, but the mode of communication does not change.

The outstanding question is what will happen to the new forms of communication, like instant messaging and networked self-promotion. It seems unlikely they will continue to be hosted on separate technology islands, without any interchange. Instant Messaging might see a standard and federated service providers, like email, in the form of XMPP or other standard protocols.

For social networks and self promotion, there are some functionality to standardize on, like the social graph and contact list. However, as a self-promotion and publication tool, perhaps the standard is already there in the form of the HTML based web itself. If the user information, contacts and relations could be a separate federated service, independent of the web site, Facebook or similar platforms would no longer have such a stronghold. Some attempts at this are already in place: Diaspora, identi.ca, and GNU Social are interesting alternatives. Although, all have a long way to go before they reach mainstream.

Comments Off

Git branch in zsh prompt

Comments Off

When working in a git directory, I would like to see the current branch as part of the Zsh prompt. There are more advanced use cases out there, but I’ll stick with the branch name for now.

The following lines in ~/.zshrc takes care of the prompt. There are a few gotchas, though: The git command will fail if not in a git controlled directory, so we’ll have to hide that failure message. Then, for Zsh to execute the function, rather than printing its verbatim name, the prompt_subst option has to be set. Finally, it is important to use single quotes for the PROMPT line. If double quotes are used, the first output of the function is used, and never called again.

function get-git-branch {
  b="`git symbolic-ref --short HEAD 2> /dev/null`" && echo "($b)" || echo ""
}
 
setopt prompt_subst
 
PROMPT='%n@%2m %~ $(get-git-branch) %# '

Comments Off

Java 8: File handling becomes fun

Comments Off

With the introduction of functional-style syntax and several new Stream based methods in Java 8, file handling finally becomes fun and easy. In fact, the new helper methods and lambda syntax even gives Python a run for its money when it comes to compact code.

Here is how you could read all lines of a file, given as a Path p (since Java 7), and output to stdout.

Files.lines(p).forEach(System.out::println);

To make it a bit more clear what is going on, here a bit more is included and broken up.

Path p = Paths.get("myfile");
 
Stream lines = Files.lines(p);
lines.forEach(System.out::println);
lines.close();

A similarly neat helper function exists for recursively walking over the directory tree of the file system. Again, this prints to stdout.

Files.walk(p).forEach(System.out::println);

Comments Off