Archive for the author ·

havard

·...

Linux and disk fragmentation

no comments

A wind of nostalgia blew past, and for some reason I remembered rainy nights in the early 90s, waiting for Norton Speed Disk to defragment a 30 MB FAT16 drive. But what happened to the defrag tools? Well, on Windows it seems they are all alive and well, with Windows 7 apparently doing automatic daily defrag in the background. In other words, on modern NTFS file systems it is still considered necessary. What I’d like to see benchmarks on is how much of a difference it makes. Is it really worth it?

On most Linux file systems the story is different. They typically do not require defrag, since they don’t suffer from fragmentation in the first place. In fact, ext based systems will intentionally scatter files so there is room for them to grow without splitting up. For a great and easy to understand explanation, see the OneAndOneIs2 blog.

For a comparison between file systems, see the Ubuntu forms, and Wikipedia for notes on defrag approaches and tools.

As for staring at those fancy looking progress and status screens of the defrag tools, it seems it’s a thing of the past across all OSes. It was a nice way to kill time; a bit like watching the washing machine tumble the clothes, I guess. Well, there’s always Bittorrent chunks. They actually look a bit similar when only part of the torrent is downloaded.

Tizen is out

no comments

The mobile operating system with the unfortunate name was released in version 1.0 about a week ago, and Samsung is now giving away phones which run the OS (to attendees of the Tizen Developer Conference). The 1.0 version comes with the somewhat ridiculous code name: “Larkspur”. I suggest they just call it T1, and stick to the Chrome and Firefox release model. By the end of the year, they’ll have reached T800, and it will sound cool. Anyway.

The Tizen OS is a merger between the LiMo OS (from the LiMo Foundation) and Intel’s (and previously Nokia’s) MeeGo, the later itself a merge between Maemo and Moblin. Their release announcement is sparse on details, but the front page for the source code section has more interesting details.

Here’s a few highlights from the list:

  • Peer to Peer connection with Wi-Fi Direct device.
  • Window system based on the X11 open source project
  • Composite window manager based on EFL open source project
  • Multimedia framework based on the GStreamer open source project
  • Audio decoder: AAC, MP3, WMA 7/8, WAV, Vorbis, AMR-NB / AMR-WB
  • Audio encoder: Vorbis, AMR-NB
  • Video decoder: MPEG-1, MPEG-4, H.263, H.264, On2 VP3, Theora
  • W3C/HTML5 specifications support

By including established FOSS projects and technologies like X11, GStreamer, OGG Vorbis, Theora, it is from the get-go a much stronger open source offering than Android ever was. Furthermore, in basing much of their core components on standard GNU/Linux projects, one would hope they follow the design of its predecessors and stricture around package repositories. Meego had a RPM repository, and it would be great to have the same upgrade mechanism on a phone, rather than the limited functionality of an “app market”.

The source code is on Git, and they have a bit of developer documentation. Guess it’s time to download and try.

Storage Prices

no comments

Writing for Softpedia, Constantin Murariu claimed last week that the “HDD crisis” has been greatly exaggerated by the big manufactures, Seagate and Western Digital. For a fact, they have posted very healthy profits over the last months. Furthermore, there is now a duopoly in the HDD market, after Seagate bought Samsung’s hard drive division in April 2011, and Western Digital bought Hitachi’s HDD division.

Murariu laments that we will never see prices as low as pre-flood, however, that would of course go against more than half a century in improved technology and value for money. After all, storage capacity has seen extreme growth and its own “Moore’s law”, but at a much more rapid phase than the transistor density. We already have Hitachi’s 4 TB Deskstar and Seagate’s 4 TB GoFlex Desk from last year. The later uses 5 platters of 800 GB, but the former is already at 1 TB / platter, which means that a 5 TB disk is already feasible if they wanted to. However, four platters usually makes for more stable and reliable disks, as they have less components, so that’s usually what the follow-up drives are made up from.

Looking at the current prices, they’ve come down a bit since right after the flood, but still have further to go before getting back to June 2011 levels. For example, the external 3 TB WD disk was ~140 Euros last year, went up to 203 Euros, and is now at 165 Euros (but with the faster USB 3 interface), or still 18% higher. Not a disaster for your wallet, but still a bump in the ever decreasing prices per byte.

Other than that, the relative ranking remains the same. The 2 TB disk still gives most bytes for money, optical disks don’t change much at all, SSD is coming down, but is still a lot more expensive than spinning disks, and CompactFlash is the most expensive medium around (but with increasing read/write speeds for ever larger sensors and pictures).

Media Type Product Capacity Price CHF Price Euros Euros / GB GBs / Euro
Harddisk Western Digital Caviar Green 2TB 2000 GB 125.00 104.05 0.05 19.22
Harddisk Western Digital Caviar Green 3TB 3000 GB 199.00 165.64 0.06 18.11
External 3.5 Western Digital My Book Essential Edition 3TB, USB3 3000 GB 199.00 165.64 0.06 18.11
External 3.5 Western Digital My Book Essential Edition 2TB, USB3 2000 GB 139.00 115.70 0.06 17.29
Harddisk Hitachi Deskstar 7K4000, 4TB 4000 GB 351.00 292.16 0.07 13.69
Harddisk Western Digital Caviar Green 1TB 1000 GB 99.00 82.41 0.08 12.14
External 2.5 Western Digital My Passport 2TB, USB3 2000 GB 237.00 197.27 0.10 10.14
External 3.5 Western Digital My Book Essential Edition 1TB, USB3 1000 GB 119.00 99.05 0.10 10.10
External 2.5 Western Digital My Passport Essential 1TB 1000 GB 124.00 103.21 0.10 9.69
Harddisk Western Digital Caviar Green 500GB 500 GB 87.00 72.42 0.14 6.90
Blu-ray Verbatim BD-R SL 25 @ 50GB 1250 GB 232.00 193.11 0.15 6.47
DVD+R DL Verbatim 8x DVD+R DL 25 @ 8,5GB 213 GB 43.00 35.79 0.17 5.94
DVD-R Verbatim 16x DVD-R 100 @ 4,7GB 470 GB 97.00 80.74 0.17 5.82
CD-R Verbatim CD-R 100 @ 700MB 70 GB 43.00 35.79 0.51 1.96
USB Flash Sandisk Cruzer Flash Drive 32GB 32 GB 31.00 25.80 0.81 1.24
SSD Corsair Force3 240GB 240 GB 237.00 197.27 0.82 1.22
USB Flash Sandisk Ultra Cruzer BACKUP 64GB 64 GB 67.00 55.77 0.87 1.15
SSD Corsair Force3 120GB 120 GB 128.00 106.54 0.89 1.13
SSD OCZ Agility 3 240GB 240 GB 269.00 223.91 0.93 1.07
SSD OCZ Agility 3 120GB 120 GB 135.00 112.37 0.94 1.07
SSD Corsair Force GT 240GB 240 GB 270.00 224.74 0.94 1.07
SSD Corsair Force GT 120GB 120 GB 145.00 120.69 1.01 0.99
SSD Intel SSD 330 Series 120GB, SATA-3 120 GB 159.00 132.35 1.10 0.91
SSD OCZ Vertex 3 240GB 240 GB 319.00 265.53 1.11 0.90
SSD OCZ Vertex 3 120GB 120 GB 169.00 140.67 1.17 0.85
SSD Intel SSD 520 Series 240GB, SATA-3 240 GB 359.00 298.82 1.25 0.80
SSD OCZ Vertex 3 Max IOPS 120GB 120 GB 199.00 165.64 1.38 0.72
SSD OCZ Vertex 3 Max IOPS 240GB 240 GB 399.00 332.12 1.38 0.72
USB Flash Sandisk Cruzer Flash Drive 16GB 16 GB 27.00 22.47 1.40 0.71
USB Flash Sandisk Cruzer Flash Drive 8GB 8 GB 14.00 11.65 1.46 0.69
Compact Flash SanDisk Ultra 200x, 16GB 16 GB 70.00 58.27 3.64 0.27
Compact Flash SanDisk Extreme 400x, 60MB/s, 64GB 64 GB 288.00 239.72 3.75 0.27
Compact Flash SanDisk Extreme 400x, 60MB/s, 32GB 32 GB 145.00 120.69 3.77 0.27
Compact Flash SanDisk Extreme 400x, 60MB/s, 16GB 16 GB 92.00 76.58 4.79 0.21
Compact Flash Sandisk Extreme Pro 100MB/s, 128GB 128 GB 845.00 703.36 5.49 0.18
Compact Flash Sandisk Extreme Pro 600x, 90MB/s, 64GB 64 GB 433.00 360.42 5.63 0.18
Compact Flash Sandisk Extreme Pro 600x, 90MB/s, 32GB 32 GB 218.00 181.46 5.67 0.18
Compact Flash SanDisk Extreme 400x, 60MB/s, 8GB 8 GB 64.00 53.27 6.66 0.15
Compact Flash Sandisk Extreme Pro 600x, 90MB/s, 16GB 16 GB 131.00 109.04 6.82 0.15

Exchange rate: 1 Euro = 1.201378 CHF.

SanDisk CompactFlash Speeds

no comments

Looking for more memory cards for my DSLR, I divided deep into the CF specification, history, and marketing. It’s a mess. Here I’ve focused on the SanDisk CF cards, as that’s what I usually get. More specifically, I wanted to know, which name or specification corresponded to which speed, and what I could expect from the that card when taking a lot of pictures, e.g. hundreds of pictures as part of a panorama.

Over at Digital Photography Insights, Rob Galbraith already has very extensive tests with different cards. Currently, only the EOS 5D Mark III list is covering most relevant CF cards, but tests with the Canon 50D is also planned. Looking forward to it. Also of interest from his page is the various card readers. This led me to the Addonics ADSACF which connects the CF reader directly to the internal SATA bus, and achieves read speeds of more than 90 MB/s. At ~$30, it looks like a very good buy.

Before digging into the SanDisk cards, a bit of background and relevant documents. The CF specification is guarded by the CompactFlash Association, and you have to buy the detailed docs, however they’ve started applying some cute icons to each of the more recent versions. Still, it does not tell you a lot about the read/write speed; e.g. the CF 5.0 and 6.0 specifications carry the same icon. Wikipedia has a bit more detail on the older CF specifications. It mentions that the first CF revisions were using PIO (Programmed input/output); PIO mode 2 since the beginning in 1995, and mode 4 in 2003, with DMA 33 in 2004. According to Rob Galbraith, several of the cards implement dual protocols, i.e. both PIO and UDMA. I’ve not been able to verify this anywhere else, but it seems to make sense, for backwards compatibility. Of course, when operating in PIO mode, one will not be able to take advantage of the speed the newer cards are rated for.

Current CF cards are based on PATA mode, so the UDMA (Ultra Direct Memory Access) modes apply. I’ve lifted he relevant table from the current Wikipedia article, as shown below. Of some relevance is also the speed rating used for SD cards. Here the “x” rating is based on the CD-ROM transfer speeds, e.g. 6x means 6 * 150 kB/s, or 900 kB/s. Some of the Lexar, Kingston, and other CF also use this in the name of the cards, but not SanDisk.

Mode Number Also called Maximum transfer rate (MB/s) Defining standard
Ultra DMA 0 16.7 ATA-4
1 25.0 ATA-4
2 Ultra ATA/33 33.3 ATA-4
3 44.4 ATA-5
4 Ultra ATA/66 66.7 ATA-5
5 Ultra ATA/100 100 ATA-6
6 Ultra ATA/133 133 ATA-7
7 Ultra ATA/ - - -

Back to the SanDisk cards, though. SanDisk itself is not very good at keeping old product pages around, so a lot of the references I found was at Digital Photography Review; they keep their articles around. The story starts in 1994, with SanDisk defining the CompactFlash standard, batteling it out with other companies and cards like Miniature Card, SmartMedia, and later MMC/SD, various Memory Stick formats, and xD-Picture Card.

The first article I can find on dpreview, is an updated product range of “larger capacity CF cards of 64MB, 80MB and 96MB”. No mention of speed, but another article from the same date talks about 1.4 MB/s as high-speed. Later on, the different product lines have been overlapping, upgraded, and renamed, and thus adding a lot of confusion. Add to that strange “marketing” products like the 2007 “Ducati Edition” (4 and 8 GB cards at 45 MB/s), and the terrain starts to get difficult to navigate. Also note, the “Extreme” line has two incarnations, its inital series from 2003, which was just called “SanDisk Extreme”, and the newer UDMA versions; not sure which year they were introduced. The same is the case for the initial “Ultra” line.

Below I’ve listed the SanDisk CF product lines I could find, with best effort searches for largest size, and guesses at standard / IO mode. The model prefixes where partly from Amazon, with hints from the Nikon approved cards page. Feel free to get in touch if you have further information.

Name Model number
prefix
Speed Largetst Size Modes Year Source
Standard SDCFB ?? ?? PIO 2 ??
Ultra ?? 2.8 MB/s 512 MB PIO 2 2001 dpreview
Extreme ?? 6 MB/s 1 GB PIO 2 2003 dpreview
Ultra II SDCFH 9 MB/s 8 GB ? PIO 4 2004 dpreview
Extreme III SDCFX3 20 MB/s 16 GB PIO 4 /
UDMA 2
2004 dpreview
Extreme IV SDCFX4 40 MB/s 8 GB PIO 4 /
UDMA 4
2006 dpreview
Extreme III SDCFX3 30 MB/s 32 GB PIO 4 /
UDMA 4
2008 dpreview
Extreme IV SDCFX4 45 MB/s 32 GB PIO 4 /
UDMA 4
2008 dpreview
Ultra SDCFH 30 MB/s 16 GB PIO 4 /
UDMA 4?
?? SanDisk
Extreme SDCFX 60 MB/s 32 GB PIO 4 /
UDMA 4?
?? SanDisk
Extreme Pro SDCFXP 90 MB/s 64 GB PIO 4 /
UDMA 6
2009 dpreview
SanDisk
Extreme Pro SDCFXP 100 MB/s 128 GB PIO 4 /
UDMA 7
2011 dpreview
SanDisk

Firefox plug-ins and fixes

no comments

Firefox versions continue to increase by one every week or so, but the more they change it, the more I’d just like it to stay the same. Here’s the list of plug-ins and fixes I usually go through on every new install.

Disable the “fancy” stuff. These settings can be reached from the special about:config page. I disable these

  • browser.tabs.animate
  • browser.tabs.insertRelatedAfterCurrent

Then some plug-ins, some of them just to fix stuff as well

Barcelona Panorama

no comments

I’ve added two new panoramas from Barcelona, one from high up in the towers of the Sagrada Familia, and one from the top of the roof of the Cathedral of the Holy Cross. The later could maybe just as well have been a wide-angle picture, as the most interesting element is the cross in the centre, but this time it turned out to be panorama.

CyanogenMod 7 on Nexus S

no comments

Building the Android OS from source has caused me a bit of pain, so it was therefore a pleasant surprise to see just how easy the CyanogenMod team has made their install and update process. For the Nexus S, it boiled down to: Follow the instructions, any everything just works. Therefore these are just some background notes; for full details, see the CyanogenMod install page.

What is a bit confusing in the Android world, are all the obscure and redundant code names from everything from hardware, models, OS, firmware, versions, regions. The Google Nexus binary pages gives some hints, as does the Android build instructions. For the Nexus S, it is crucial to know that the alternative codename is “crespo”. Furthermore, it is useful know that it has a “HDPI” display, and comes in three variations: “GT-I9020 (Super AMOLED) and GT-I9023 (Super Clear LCD), each aimed at different markets. The SPH-D720 is the newer 4G version of the phone available in the US.” (Wikipedia). Also, the radio binaries have different codes, e.g. “XXKB3″. This then forms the “baseband version” code on the form “I9023xxkd1″, which can be found under Settings -> About phone.

CyanogenMod is just a layer on top of the Android OS, thus it’s tracking the Android versions. Here we are talking about “Gingerbread” for the 2.3.x series, and “Ice Cream Sandwich” (aka. ICS) for the 4.x versions. (Who comes up with all these useless names??) Although Google pushed Android 4 to certain Nexus S devices in December 2011, and CM 9 (which based on Android 4) have nightly builds for the “crespo”, their stable CM 7 is still at Android 2.3.7. In addition to the basic OS, you might also want the Google Apps, although, some of them might have to be downloaded from the Market (now renamed to “Play”) anyway.

So, with that out of the way, and assuming the basic tools adb and fastboot in place (possibly from a previous build session), the upgrade can be summarized as this (with specific versions and URLs bound to change):

Download:
wget http://download.clockworkmod.com/recoveries/recovery-clockwork-5.0.2.0-crespo.img
wget http://download.cyanogenmod.com/get/update-cm-7.1.0.1-NS-signed.zip
wget http://cmw.22aaf3.com/gapps/gapps-gb-20110828-signed.zip

Flash the recovery image:
adb reboot-bootloader
sudo fastboot flash recovery recovery-clockwork-5.0.2.0-crespo.img
sudo fastboot reboot

Copy the CM files to the “sdcard” of the phone:
adb push update-cm-7.1.0.1-NS-signed.zip /sdcard/update.zip
adb push gapps-gb-20110828-signed.zip /sdcard/gapps.zip
adb reboot-bootloader

Follow the install instructions from CM for the rest. I was surprissed to learn that the data on the SD card was not wiped as well (I thought “Wipe data/factory reset” would take care of this). However, maybe I missed a step.

More Android Apps

no comments

Following the list from yesterday, here’s more useful Android apps, this time both from the F-Droid and Android Market (aka. Play). First from F-Droid (APK file):

And from the Android / Google Play Market

Finally, there’s a few which can, or have to, be downloaded from the creator’s web site:

Cell phone privacy guide for Android

1 comment

The Pirate Party of Canada has a nice list of applications and add-ons for Android phones which enhance security and privacy. It boils down to

Replicant – Free Android alternative

no comments

The Replicant project was covered by Computer World today, with an interview of one of the main developers Paul Kocialkowski. The article was also picked up by Slashdot, where the mod was more cynical. Kocialkowski talked about the importance of free hardware and software when communicating in an insecure environment, and the difficulty of implementing an alternative when many of the drivers and firmware for key hardware is proprietary and secret.

Currently, Replicant is only supported by a few phones, including HTC Dream, Nexus One and Nexus S. The last one is probably interesting, as it is still a reasonably modern phone, can run Android 4.0, and have “factory binaries” available so you can start from scratch if you get stuck.

However, as pointed out in the Slashdot comment above, this project will always trail Google’s releases. And as we’ve seen previously, the source codes does not always follow the release of the OS. Furthermore, Android in itself is not the most exciting OS around. The MeeGo (and presumably Tizen) initiatives are more standard GNU/Linux distributions from the get-go. Including features we’ve taken for granted over the last ten years or more: dependency resolving, updates over repositories, GNU applications. Of course, these will struggle with exactly the same problem: the proprietary drivers.

Update to HTML5 Canvas Panorama Viewer

no comments

I published the HTML5 Panorama Viewer last year, and have finally had the time to make some updates, including support on touch screen mobiles. I’ve tested on several browser for Android (bulti-in, Firefox, and Opera Mobile), as well as Safari on iPhone; it’s looking good, but please report any issues you might find. Known limitations include click and move lag on older phones, no support for history management in some browsers, nor multi-touch in most (thus no pinch-zoom yet).

Desktop controls:
Zoom in: double click
Zoom out: right click
Move: Hold and drag

Mobile controls:
Zoom in: double click (can be a bit tricky)
Zoom out: hold without moving for more than 2-3 seconds
Move: Hold and drag

Licenses: The JavaScript and HTML coded for the viewer is licenses under GPL3. The copyright of the panorama images are retained by the author.

Anonymous, Decentralized and Uncensored File-Sharing

no comments

The file-sharing landscape is slowly adjusting in response to the continued push for more anti-piracy tools, the final Pirate Bay verdict, and the raids and arrests in the Megaupload case“, starts a recent TorrentFreak article. It goes on to introduce Tribler, and RetroShare, which “[creates] a private space on the Internet. A social collaboration network where you can share anything you want. A space that is free from the prying eyes of governments, corporations and advertisers. This is vitally important as our freedom on the Internet is under increasing threat. RetroShare is free from censorship: like Facebook banning ‘obscene’ breast-feeding photographs. A network that allows you to use any pseudonym, without insisting on knowing your real name. A network where you will not face the threat of jail, or being banned from entry into a country for an innocent tweet“.

For RetroShare, there are pre-compiled installs for several OSes and GNU/Linux distributions, however not for Fedora. So I downloaded the source, and tried to compile with a bit of help from this page. First, some dependencies:

yum install qt-devel gpgme-devel libgpg-error-devel libupnp-devel libssl-devel libgnome-keyring-devel openssl-devel glib2-devel libXScrnSaver-devel

It turns out there is an old, widly published bug in the glib-2.0 include path dependencies. To work around it, edit the file libretroshare/src/libretroshare.pro, find these lines, and add the third line, marked in blue:

# These two lines fixe compilation on ubuntu natty. Probably a ubuntu packaging error.
INCLUDEPATH *= /usr/lib/x86_64-linux-gnu/glib-2.0/include/
INCLUDEPATH *= /usr/lib/i386-linux-gnu/glib-2.0/include/
INCLUDEPATH *= /usr/lib64/glib-2.0/include

Furthermore, there are a couple of missing library dependencies:

Edit retroshare-gui/src/RetroShare.pro, find the “## Linux ##” section, and add the last three libraries marked in blue:

LIBS += ../../libretroshare/src/lib/libretroshare.a
LIBS += -lssl -lgpgme -lupnp -lixml -lXss -lgnome-keyring -lcrypto -ldl -lX11

Then the make steps should work fine:

cd libbitdht/src
qmake-qt4 && make
cd libretroshare/src
qmake-qt4 && make
cd retroshare-gui/src
qmake-qt4 && make

And to start up:
retroshare-gui/src/RetroShare

DRM on harddisk and flash sticks

no comments

CNet is covering a press release from Western Digital which announced a new consortium called Secure Content Storage Association (SCSA) to create standards for transferring restricted / locked contented between storage and playback devices. From those two articles, it’s difficult to say what details might be, but the UltraViolet (UV) keyword gives a clue:

“UltraViolet (UV) is a digital rights authentication and cloud-based licensing system that allows consumers [...] to stream and download purchased content to multiple platforms and devices.

UltraViolet content is downloaded (or streamed) in the Common File Format, using the Common Encryption (CENC) system.

However, currently it seems UV is all cloud and streaming based, with no option to transfer locked contented off-line. Enter Western Digital. Presumably, they will provide special hard-drives which can store and restrict the content, and only allow read and copy operations under certain conditions. Possibly using TPM or similar technology, but embedded in the drive itself.

Further down in the Wikipedia article, this bit is interesting, and again, the storage aspect is the noticable missing piece:

“UltraViolet selected five DRM technologies allowing restrictions management on a broad range of devices: televisions, set-top-boxes, DVD & Blu-ray players, games consoles, PC, tablets and smartphones. The selected DRM technologies are:

  • Google Widevine DRM, chosen for its strong position on set-top boxes
  • Marlin DRM, chosen for its compatibility with many Connected TVs
  • OMA CMLA-OMA v2, chosen for its strong position on mobile devices
  • Microsoft PlayReady, chosen for its wide availability on PC and CE devices
  • Adobe Flash Access 2.0, chosen for its wide availability on PC”

It’s also interesting to see the rest of the list of companies supporting this, in the Digital Entertainment Content Ecosystem (DECE LLC) is a consortium. Nothing too surprising there, I guess, except for Tesco. (What will happen if your veggies are digitally restricted?)

Raspberry Pi launched

no comments

The $25/35 Raspberry Pi matchbox computer finally launched today, after much anticipation, impatience and extremely clever marketing. In the last few days, their web servers have been overwhelmed by the people hitting re-fresh to know when the device will go on sale. When it finally did, both distributor web sites melted. The 10k units produced were sold out before lunch.

So, if you’re like me, and did not get a device this time around, you might want to join the support group over at Slashdot. There you will find people crying like kids who missed Santa. If it was kids, that would be one thing, but some of these crying guys actually have jobs. Amazing. Then there was one guy who had missed the weekly, or almost daily articles and didn’t know what this stampede was about. I guess he didn’t get one either.

Well, there will be more of these devices, and then some bugs might even be ironed out.

Pipe Viewer – ETA for pipes

1 comment

DP has a nice write-up on the pv (for pipe viewer) command. One of his examples goes like this; also try it without the redirect to /dev/null.

for i in {1..21}; do date -d "`date +%m`/$i/`date +%Y`"; sleep 1; done | pv -leps 21 > /dev/null

To install on Feodra:

yum install pv

Bad Behavior has blocked 87 access attempts in the last 7 days.