At the end of last year, I wrote about DIY “cloud” services; how to get started with your own web server, blog, e-mail server, chat server, VOIP server. Add to that distributed services for social networking and micro-blogging like Dispora and Identi.ca; free software for file sharing like GNUNet; and even free and intendant dark / mesh nets. The trend is clear, there is strong momentum towards free software and alternative services.
Now there is also a public face to this, in the form of a new initiative: The CitizenWeb Project.
“The CitizenWeb Project is a mission to fight for a free, open, and above all a decentralized Internet. In order to achieve this, it aims to empower everyday internet users with the information and resources they need to take matters into their own hands. We seek to spread the word about how to secure yourself online and how to declare “digital independence” in this age of the Google hivemind and Facebook privacy nightmares. While these services may be convenient, they carry very dangerous implications for our freedoms. This is only getting worse with time, as the corporations behind these services become entangled and indiscernable from government services and real-life social obligation. And it is only getting worse for the most sensitive users: journalists, activists, muckrakers and whistleblowers.
There are viable alternatives to these invasive and ubiquitous services. The CitizenWeb Project is therefore focused on giving the tools to each individual user to become an independent “citizen” of the Web — to decentralize their social networks and platforms, to become the TRUE owners of their data, and to communicate and network in security.”
Next week, the “six strikes” system in the “war against downloaders” will take effect in the US. The music and movie industry has lobbied (and paid for) this law. However, it seems it might not have the desired effect, lest people are very compliant. After the first couple of times, you might get a warning, and have to complete courses on how to secure your Wifi. At the sixth strike, your Internet speed will be slowed down for a couple of days. However, after the 7th, 8th and 9th time, nothing else is in store for you. So, your duty as a freedom loving citizen is to pirate as much as possible, and break free from the whole system after six warnings you can happily ignore.
Last year I made the first progress towards a DIY remote for my home automation light switches (using the Everflourish RF based system). The goal was to use the 433.92 MHz radio transmitter and an Arduino to control the switches. Furthermore, the Arduino would receive control commands from my always-on computer, with Bash command line control (’cause that’s the way I like it). Finally, a custom Android app was developed as yet another option for remote control. The project is now fully operational, with a few adjustments here and there remaining. The figure below shows a high level overview.
From left to right, the mobile phones run Android apps which present a simple interface to control the lights. Each light can be turned on and off individually, or by some pre-sets (e.g. everything on or off, “good morning”, or “movie time”). The application has hard-wired the IP address of my always on server / PC on the local network. There is of course nothing stopping me from connection to a public IP (and port-forward), however, I don’t need away-from-home control at this point. The phone app simply sends a “light switch command” to the PC, which forwards it directly to the Arduino (see details below).
In this setup, the server is only a proxy, however, it could of course also be used to implement pre-programmed settings. E.g. daily rhythm commands when we’re away on holiday, or other automated functions. Since this box is also my PC, I can also send commands directly to the Arduino over serial /dev/tty; I have some convenience bash-scripts for that. For the phone app, it only does the forwarding on a hard-wired port:
while true; do nc -l 1234 > /dev/ttyUSB0; done
Maybe that bit could be improved in the future. Especially, since there are occasionally a few hiccups in the phone to PC communication. For example, the switch from 3G to local Wifi connection might be delayed as we enter the house, so commands will buffer up, and then suddenly sent en masse. Also, this would be a natural place to expand the system with pre-programmed actions. It should also be noted that to get this to work, I had to enable communication on the port 1234 both on the Wifi router, and the local server firewall.
Finally, the Arduino, connected via USB, accepts the incoming serial messages. The encoding is simple: A two digit code where the first digit is the index of the light, and the second is 0 or 1 for off or on (see the loop() method in the source listing below). So, “11″ will turn on the first light, while “10203040″ turns off all four lights. (Note that in the code below, to make things clearer for myself, index 0 is skipped).
The code below is all there is the the Arduino part. It hard-codes the Everflourish messages and timings. It should be noted that the messages as found in the previous article, are most likely some form of Manchester code, as there are always pairs of 01 or 10. Thus, the hard-coded messages could have been shortened, but this would have added (a bit) to the complexity of the code, so I left it.
The main loop waits for incoming bytes on the serial USB connection, and sends the according light switch command to the RF transmitter on pin 2. Here there is probably also room for some improvements, to avoid illegal numbers, stuck states, etc. Once a valid code is received, the transmitted signal for that button is repeated four times, just as seen with the original remote control.
If you don’t want to build yourself, there are some reasonably priced options from DealExtrme. (The two first look exactly the same, even if they are different names slapped on top)
My end goal is something like the Gigapanbot, mentioned earlier, so maybe some simple brackets might also do, plus a battery. More about that later.
DealExtreme / dx.com is a Hong Kong gadget site with thousands and thousands of China products of all kinds, often at very competitive prices. Every order is free of shipping charges, even if you buy a $1 item. It makes it easy to order and forget. The only downside is the shipping time, which sometimes can be around three to four weeks; some things might arrive earlier, though.
Here are the items I’ve bought for myself and others so far, followed my wish list, or list of items which might be interesting. The order list can be download in CSV form, while the wish list had to be scraped. For what it’s worth, here is a small Python script which does that, plus a bash script for HTML rendering of image links and product descriptions.
The Python script uses WebDriver / Selenium, and install for that goes: sudo apt-get install python-pip
sudo easy_install selenium
Computer storage, primary and secondary memory, has seen a tremendous phase of development over the last fifty years. As new technology has been brought to the market prices have continued to decline steadily at a logarithmic scale. For magnetic storage, the trend has been very stable over the last thirty years, with prices per MB going down around a third every year, or a ninety percent every five years. For primary storage, the trend has been more volatile, but overall we see a similar rate of decline all the way back to the first flip-flops in the 1950s.
John C. McCallum has done a good job collecting all the data over the years, and going back to computer magazines for reference. However, since the beginning of 2012 there have been no updates, so I’ve taken up the work where he left off. I’ve added a new page to my site, where I will collect the data and update the graphs over time: hblok.net/storage
(Click image for larger version)
In the first update, the harddisk prices are most interesting, and we can now clearly see the effect of the flood disaster in late 2011. It has interrupted a thirty year trend, and as a result prices are about the same per MB as they were one and a half years ago. Now the question is, will this have a lasting effect on the magnetic harddisk prices, or will it be just a blip in history, as technological improvements bring us cheaper storage at the same phase.
The two plots below extrapolate the trend over the last thirty years, with two different scenarios: 1) Improvements in technology will catch up with the delay over the last year, and thus the thirty year trend will continue unaffected (red line). Or 2) phase of improvments will not change, and thus the rate of decline in price will stay the same, but shift the line by about a year (blue line).
(Click image for larger version)
The price is 0.4 cents per GB today (4e-5 per MB). If we look two years ahead, with the uninterrupted scenario (red line), the price would be 0.05 cents per GB in 2015 (5e-6 per MB), or put in different ways: 3 TB of storage which costs $125 today would have to go down to about $15 in two years, or for the same $125 you’d have to get a whopping 25 TB (yes, twenty five!). Given the recent news from the major harddisk vendors, that seems rather unlikely to happen; they’re only planning for 5 TB drives at the end of this year. So, over two years time, prices will not catch up. Perhaps this will change looking even further ahead, however, extrapolating technological trends beyond a year or two is merely guessing.
If we look at the second scenario, where we assume that the prices will continue to decline at the same rate as they have done in the past, given today’s price we’re then looking at about 0.15 cents per GB (1.6e-5 per MB). That would mean that today’s 3 TB would go for around $50, while $125 would buy you about 8 TB. That seems more reasonable, and also in line with what products are being brought to market and in research right now. If the rumoured 5 TB Western Digital disk will be realised with four platters (4 * 1.25 TB) at the end of this year, it means five platter 6.25 TB (5 * 1.25) disks are already a possibility. Increasing storage density another 30% to reach 8 TB over the following year seems a reasonable assumption.
I recently found myself needing to compare bitmap images, to see if they were about the same. The images were from Gnuplot generate graphs, and I wanted to check whether subtle changes in the data had not introduced unexpected changed to the plot. A simple binary diff told me that there were indeed some differences, however, comparing them manually was not possible.
Enter the handy tool perceptualdiff, which lets you compare TIF and PNG images based on a perceptual metric. It actually goes beyond simple bitwise pixel diff, and tries to compare based on a model of the human visual system. Consider the examples below, and it is clear that it is a useful tool. It makes it easy to see where the small differences were introduced. (As it turns out, it is only the result of the two plots being generated on different machines, with different versions of Gnuplot and possibly also different available fonts).
The tool is in the Fedora repository, so a simple yum is enough: yum install perceptualdiff
To generate an output diff image, this command does the job: perceptualdiff -output diff.ppm image1.png image2.png
After struggling with fonts in Gnuplot 4.6 (on Fedora 17) (getting the not so useful error “gdImageStringFT: Could not find/open font while printing string”), I found tonicas post on debugging the issue. Although helpful, it did not give the full solution to my problem. It turns out, many of the old fonts are not available in Fedora 17 at all.
I wanted a sans-serif font, and in the end I went for the DejaVuSans. After installing the font packages, I specifically exported that path for use with Gnuplot: