Archive for

December, 2012

...

The Do-It-Yourself Cloud

1 comment

“In the cloud”

The buzzword “cloud” seems to be here to stay for quite a lot longer. The problem is that it is rather ill-defined, and sometimes it is used to mean “on the Internet”, regardless of how or where a particular service or content is hosted.

It is not before we pick up further buzzwords that we can add some meaning to the term: Although there are even more terms used, I would like to focus on two of them: Infrastructure as a Service (IaaS), or what traditionally has been called “hosting”; virtual or dedicated machines which you can install and operate on OS root level with little or no oversight. Examples include your local hosting provider, and global businesses like Amazon EC2 and Rackspace.

Secondly, Software as a Service (SaaS), where you don’t write the software or maintain the system yourself. All it takes is to sign up for a service, and start using it. Think Google Apps, which includes GMail, Docs, Calendar, Sites and much more; or Salesforce, Microsoft Office 365, etc. Often these services are billed as “free”, with no financial cost to private users, and the development and operating costs of the provider is financed through various advertisement programs.

Black Clouds

The problem with the later model, Software as a Service, is that it can put many constraints on the user, including what you are allowed to do, say, or even make it difficult for you to move to another provider. In his 2011 essay “It’s the end of the web as we know it”, Adrian Short likens the later model to tenants: If you merely rent your home, there are many things you will not be allowed to do, or which you do not have control over. Short focuses on web hosting where using a service like Blogger will not let you control how links are redirected, or were you to move in the future, take those page-clicks with you onto your new site. The same goes for e-mail: If AOL decides that their e-mail service is not worth-while tomorrow, many people will lose e-mails with no chance to redirect. Or look at all the storage services which collapsed in the wake of the raid on MegaUpload. A lot of users are still waiting for FBI to return their files.

More recently, the security expert Bruce Schneier wrote about the same problem, but from a security perspective. We are not only tenants he claims, but serfs in a feudal system, where the service providers take care of all the issues around security for us, but in return our eye-balls are sold to the highest bidder, and again it is difficult to move out. For example, once you’ve invested in music or movies from Apple iTunes, it is not trivial to move to Amazon’s MP3 store; and if you’ve put all your contacts into Facebook, it is almost impossible to move to MySpace.

In early December, Julian Assange surfaced to warn about complete surveillance, and governments fighting to curb free speech. His style of writing is not always as straight to the point as one could wish for, but in between there is a clear message: Encrypt everything! This has spurred interesting discussion all over the Internet, with a common refrain: Move away from centralized services, build your own.

Finally, Karsten Gerloff, president of the Free Software Foundation Europe (FSFE), touced on the same theme in is talk at the LinuxCon Europe in Barcelona, in November 2012. He highlighted the same problems with centralised control as discussed above, and also mentioned a few examples of free software alternatives which distributes various services. More about those below.

Free Software

The stage is set then, and DIY is ready to become in vogue again. But where do you start, what do you need? If not GMail or Hotmail, who will host your e-mail, chat, and other services you’ve come to depend on? Well, it is tempting to cut the answer short, and say: “You”. However, that does not mean that every man, woman and child has to build their own stack. It makes sense to share, but within smaller groups and communities. For example, it is useful to have a family domain, which every family member can hinge their e-mail address off. A community could share the rent of a virtual machine, and use it for multiple domains for each individual group; think the local youth club, etc. The French Data Network (FDN), has a similar business model for their ISP service, where each customer is an owner of a local branch.

For the software to provide the services we need in our own stack, we find ourselves in the very fortunate situation that it is already all available for free. And it is not only gratis, it is free from control of any authority or corporation, free to be be distributed, modified, and developed. I’m of course talking about Free and Open Source Software (FOSS), which has much to thank Richard Stallman for its core values, defined in the GPL. (“There isn’t a lawyer on earth who would have drafted the GPL the way it is,” says Eben Moglen. [“Continuing the Fight“]). We may take it for granted now, however, we could very easily have ended up in a shareware world, where utilities of all kinds would still be available, but every function would come with a price tag, and only the original developers would have access to the source code, and be able to make modification. Many Windows users will probably recognize this world.

Assuming one of the popular GNU/Linux distributions, most of the software below should already be available in the main repositories. Thus it is a matter of a one-line command, or a few clicks to install. Again a major advantage of free software. Not only is it gratis, it usually refreshingly simple to install. The typical procedure of most proprietary software would include surfing around on an unknown web site for a download link, downloading a binary, and trusting (gambling really) that it has not been tempered with. Next, an “Install Wizard” of dubious usefulness and quality gives you a spectacular progress bar, sometimes complete with ads.

The DIY Cloud

The following is a list of some of the most common and widely used free and open source solutions to typical Internet services, including e-mail, web sites and blogging, chat and voice and video calls, online calendar, file sharing and social networks. There are of course many other alternatives, any this is not meant to be an exhaustive list. It should be plenty to get a good personal or community services started, though.

  • The Apache HTTP web server is the most widely used web server on the Internet, powering shy of 60% of web sites (October 2012). It usually comes as a standard package in most distributions, and is easy to start up and configure. For the multi-host use-case, it is trivial to use the same server for multiple domains.
  • If you are publishing through a blog like this one, the open source WordPress project is natural companion to the Apache web server. It too is available through standard repositories, however, you might want to download the latest source and do a custom install, both for the security updates, and to do custom tweaks.
  • For e-mail, Postfix is typical choice, and offers easy setup, multi-user and multi-domain features, and integrates well with other must-have tools. That includes SpamAssassin (another Apache Foundation project) and Postgrey to handle unwanted mail, and Dovecot for IMAP and POP3 login. For a web-frontend, SquirrelMail offers a no-frills fully featured e-mail client. All of these are available through repository install.
  • Moving into slightly less used software, but still very common services, we find the XMPP (aka Jabber) servers ejabberd and Apache Vysper, with more to choose from. Here, a clear best-of-breed has yet to emerge, and furthermore, it will require a bit more effort on the admin and user side to configure and use. As an alternative, there is of course always IRC, with plenty of software in place.
  • Taking instant chat one step further, a Voice-over-IP server like Asterix is worth considering. However, here setup and install might be tricky, and again, signing up / switching over users might require more effort. Once installed, though, there are plenty of FOSS clients to choice from, both on the desktop and mobile.
  • Moving on to more business oriented software, online calendar through the Apache caldav module is worth exploring. As an alternative the Radicale server is reported to be easy to install and use.
  • A closely related standard protocol, WebDav, offers file sharing and versioning (if plain old FTP is not an option). Again, there is an Apache module, mod_dav, which is relatively easy to set up, and access in various ways, including from OSX and Windows.
  • DIY Internet

    That list should cover the basics, and a bit more. To round it off, there are a number of experimental or niche services which is worth considering to their propitiatory and closed alternatives. For search, the distributed YaCy project looks promising. GNU Social and Diaspora aim to taken on heavy weights in social networking. Finally, GNUNet and ownCloud are peer-to-peer file-sharing alternatives.

    The future lies in distributed services, with content at the end-nodes, rather than the hubs. In other words, a random network, rather than scale-free. Taking that characteristic back to the physical layer (which traditionally always has been scale-free), there are “dark nets” or mesh nets, which aim to build an alternative physical infrastructure based on off-the-shelf WiFi equipment. Currently, this at a very early experimental state, but the trend is clear: Local, distributed and controlled by individuals rather than large corporations.

Ubuntu on the Nexus7

1 comment

I recently got my hands on an Asus Nexus 7 tablet. In it self, maybe not a groundbreaking device, if it wasn’t for the fact that Canonical will use it as their reference device for running Ubuntu on tablets and dual (or more) core mobile phones. Just to be clear, this is no dual boot, emulator, nor “chroot”-trick. The OS boots natively, and brings up the standard Ubuntu Unity desktop. The kernel is copied from (or based on) Google’s Android 4.1 kernel for the Nexus 7, which includes several non-committed changes, as well as binary drivers and firmware. See here fore more information.

A decent proof-of-concept build of Ubuntu 13.04 is already available, and it runs fine on the Nexus 7. If you’re running Ubuntu on your desktop, a pre-packaged installer is available from the a repository. Alternatively, download the boot and userdata images, and install using fastboot yourself. (All commands below need sudo).

fastboot devices
fastboot erase boot
fastboot erase userdata
fastboot flash boot raring-preinstalled-desktop-armhf+nexus7.bootimg
fastboot flash userdata raring-preinstalled-desktop-armhf+nexus7.img
fastboot reboot

Now, I said proof-of-concept, and what you get with this image is not really that handy on a tablet. So far, it just starts up a desktop Window Manager, which is not too comfortable with a touch screen. However, with a USB On-the-Go (OTG) adapter, you can plug in a USB hub, keyboard and mouse. Now it becomes usable like any other desktop. I got one of these compact ones from Deal Extreme, However, due to the rounded shape of the Asus Nexus 7, I had to chisel off a few millimetres to make it fit. The version with a wire would probably had worked better. Maybe also interesting to try would be a HDMI adapter (I’m not sure if that particular one works). Finally, the missing bit to have a fully functional docking setup would be charging while the OTG cable is connected. The Ubuntu FAQ mentions that this will be enabled, but you’ll probably need yet another special adapter cable to piece it all together.

What’s impressive about the current offering is that most, if not all, packages have already been compiled for the ARM architecture and are available in the Ubuntu repositories. This is very welcome, as it frees the tablet from the Android markets, and brings in an enormous selection of free and open source software. Not all of it is immediately suited for a small touch screen on a slow CPU, but that will change over time.

On a whim, I tried apt-get install emacs and eclipse. Both downloaded and worked fine, however, even with a four core CPU, ARM is not up to Eclipse quite yet. It should also be noted that the desktop UI has some unnecessary features which notably slows down the experience. For example, eye-candy like fading transitions when ALT-TABing between windows is enabled.

In conclusion, this is a very interesting first move from Canonical, and more GNU/Linux distributions will surely follow. With more alternatives and variety in this space, it will hopefully open people’s eyes up to the fact that the mobile phones and tablets they carry around are full-fledged computers in themselves, with no reason to remain restricted to a single OS from a single vendor. Maybe it will eventually turn stupid laws which makes it illegal to hack and experiment on these devices.

Storage Prices

1 comment

As Geoff Gasior pointed out in Tech Report last month, 3.5″ HDD prices are still significantly higher than pre-flod last year. Interestingly, some of the 2.5″ drivers are cheaper than last year, while some of the smaller 3.5″ drivers are up to 60% more expensive. Couch critics explain it with lack of demand, and HDD companies still recovering their loses from last year.

Despite this lag in price reductions, or perhaps due to the prevailing high prices of 2TB drives, both the WD Green 3 TB (internal and external) drives are now on top of the list. This is a first, and has been a long time coming. It is not much, but you now get 2.7% more bytes for your Euros on a 3TB drive, compared to 2TB internal.

Also interesting in this round, is that Western Digital has finally gotten around to release a 4 TB drive, more than a year after Hitachi showed off their Deskstar 7K4000 4 TB. What’s surprising about WD’s offering, is that this is, like the Deskstar, a five platter (5 x 800GB) drive. Surprising, since WD is already offering 1 TB platters in their Red line, but with max capacity of 3 TB. As discussed earlier, the 4 and 5 TB 1TB/platter drives are highly overdue. What’s more, WD has chosen to introduce the 4TB drives at the top of the range, in its RE 24/7 support, and Black lines. In terms of speed, it does makes sense use more platters in those high-end drives, however, they might also be less reliable, due to more moving parts. Five years factory warranty on the former line does mean they are serious about quality, though.

Finally, as a first, I have included Sandisk’s line of high-end SDHC cards. This is mostly with Raspberry Pi. It is tested and works on most SD cards, but performance might vary.

Media Type Product Capacity Price CHF Price Euros Euros / GB GBs / Euro
Harddisk Western Digital Caviar Green 3TB 3000 GB 159.00 131.62 0.04 22.79
External 3.5 Western Digital My Book Essential Edition 3TB, USB3 3000 GB 159.00 131.62 0.04 22.79
Harddisk Western Digital Caviar Green 2TB 2000 GB 109.00 90.23 0.05 22.17
External 3.5 Western Digital My Book Essential Edition 2TB, USB3 2000 GB 135.00 111.75 0.06 17.90
Harddisk Western Digital Caviar Black 4TB 4000 GB 322.00 266.56 0.07 15.01
Harddisk Hitachi Deskstar 7K4000, 4TB 4000 GB 329.00 272.35 0.07 14.69
Harddisk Western Digital Caviar Green 1TB 1000 GB 84.00 69.54 0.07 14.38
External 2.5 Western Digital My Passport 2TB, USB3 2000 GB 179.00 148.18 0.07 13.50
External 3.5 Western Digital My Book Essential Edition 1TB, USB3 1000 GB 105.00 86.92 0.09 11.50
Harddisk Western Digital RE 4TB 4000 GB 449.00 371.69 0.09 10.76
Harddisk Western Digital Caviar Green 500GB 500 GB 79.00 65.40 0.13 7.65
Blu-ray Verbatim BD-R SL 25 @ 50GB 1250 GB 238.00 197.02 0.16 6.34
DVD+R DL Verbatim 8x DVD+R DL 25 @ 8,5GB 213 GB 42.00 34.77 0.16 6.11
DVD-R Verbatim 16x DVD-R 100 @ 4,7GB 470 GB 96.00 79.47 0.17 5.91
CD-R Verbatim CD-R 100 @ 700MB 70 GB 35.00 28.97 0.41 2.42
SSD OCZ Agility 3 240GB 240 GB 169.00 139.90 0.58 1.72
SSD Corsair Force GT 240GB 240 GB 216.00 178.81 0.75 1.34
USB Flash Sandisk Cruzer Flash Drive 32GB 32 GB 29.00 24.01 0.75 1.33
SSD OCZ Vertex 3 120GB 120 GB 109.00 90.23 0.75 1.33
SSD Corsair Force3 240GB 240 GB 220.00 182.12 0.76 1.32
SSD OCZ Vertex 3 240GB 240 GB 229.00 189.57 0.79 1.27
SSD Corsair Force GT 120GB 120 GB 121.00 100.17 0.83 1.20
SSD OCZ Agility 3 120GB 120 GB 122.00 100.99 0.84 1.19
SSD Corsair Force3 120GB 120 GB 126.00 104.30 0.87 1.15
USB Flash Sandisk Ultra Cruzer BACKUP 64GB 64 GB 75.00 62.09 0.97 1.03
SDHC Sandisk Ultra, Class 10, 15/30MB/s, 32GB 32 GB 44.00 36.42 1.14 0.88
USB Flash Sandisk Cruzer Flash Drive 8GB 8 GB 13.00 10.76 1.35 0.74
USB Flash Sandisk Cruzer Flash Drive 16GB 16 GB 26.00 21.52 1.35 0.74
SDHC Sandisk Ultra, Class 10, 30MB/s, 16GB 16 GB 27.00 22.35 1.40 0.72
SDHC Sandisk Extreme, Class 10, 30MB/s, 16GB 16 GB 32.00 26.49 1.66 0.60
SDHC Sandisk Extreme, Class 10, 30MB/s, 8GB 8 GB 19.00 15.73 1.97 0.51
SDHC Sandisk Extreme Pro, Class 10, 90/95MB/s, 32GB 32 GB 84.00 69.54 2.17 0.46
SDHC Sandisk Ultra, Class 10, 30MB/s, 8GB 8 GB 22.00 18.21 2.28 0.44
SDHC Sandisk Extreme Pro, Class UHS-I, 90/95MB/s, 16GB 16 GB 52.00 43.05 2.69 0.37
SDHC Sandisk Extreme Pro, Class UHS-I, 90/95MB/s, 8GB 8 GB 28.00 23.18 2.90 0.35
Compact Flash SanDisk Ultra 200x, 16GB 16 GB 59.00 48.84 3.05 0.33
Compact Flash SanDisk Extreme 400x, 60MB/s, 32GB 32 GB 121.00 100.17 3.13 0.32
Compact Flash SanDisk Extreme 400x, 60MB/s, 64GB 64 GB 246.00 203.64 3.18 0.31
SDHC Sandisk Extreme, Class 6, 20MB/s, 4GB 4 GB 18.00 14.90 3.73 0.27
Compact Flash SanDisk Extreme 400x, 60MB/s, 16GB 16 GB 85.00 70.36 4.40 0.23
Compact Flash Sandisk Extreme Pro 100MB/s, 128GB 128 GB 819.00 677.98 5.30 0.19
Compact Flash Sandisk Extreme Pro 600x, 90MB/s, 64GB 64 GB 433.00 358.44 5.60 0.18
Compact Flash Sandisk Extreme Pro 600x, 90MB/s, 32GB 32 GB 218.00 180.46 5.64 0.18
Compact Flash SanDisk Extreme 400x, 60MB/s, 8GB 8 GB 55.00 45.53 5.69 0.18
Compact Flash Sandisk Extreme Pro 600x, 90MB/s, 16GB 16 GB 131.00 108.44 6.78 0.15

Exchange rate: 1 Euro = 1.208000 CHF.