Archive for

February, 2013

...

The CitizenWeb Project

Comments Off

At the end of last year, I wrote about DIY “cloud” services; how to get started with your own web server, blog, e-mail server, chat server, VOIP server. Add to that distributed services for social networking and micro-blogging like Dispora and Identi.ca; free software for file sharing like GNUNet; and even free and intendant dark / mesh nets. The trend is clear, there is strong momentum towards free software and alternative services.

Now there is also a public face to this, in the form of a new initiative: The CitizenWeb Project.

“The CitizenWeb Project is a mission to fight for a free, open, and above all a decentralized Internet. In order to achieve this, it aims to empower everyday internet users with the information and resources they need to take matters into their own hands. We seek to spread the word about how to secure yourself online and how to declare “digital independence” in this age of the Google hivemind and Facebook privacy nightmares. While these services may be convenient, they carry very dangerous implications for our freedoms. This is only getting worse with time, as the corporations behind these services become entangled and indiscernable from government services and real-life social obligation. And it is only getting worse for the most sensitive users: journalists, activists, muckrakers and whistleblowers.

There are viable alternatives to these invasive and ubiquitous services. The CitizenWeb Project is therefore focused on giving the tools to each individual user to become an independent “citizen” of the Web — to decentralize their social networks and platforms, to become the TRUE owners of their data, and to communicate and network in security.”

Comments Off

A few Friday Fractals

Comments Off

Comments Off

Pirating – it’s a civic duty

Comments Off

Next week, the “six strikes” system in the “war against downloaders” will take effect in the US. The music and movie industry has lobbied (and paid for) this law. However, it seems it might not have the desired effect, lest people are very compliant. After the first couple of times, you might get a warning, and have to complete courses on how to secure your Wifi. At the sixth strike, your Internet speed will be slowed down for a couple of days. However, after the 7th, 8th and 9th time, nothing else is in store for you. So, your duty as a freedom loving citizen is to pirate as much as possible, and break free from the whole system after six warnings you can happily ignore.

On an international scale it gets even more interesting. For their reluctance to comply with US law, Spain and Canada have been put on a special US “watch list”. Now, in Spain’s case, it is interesting that it seems it is actually legal to both upload and download copyrighted material, as long as it is not for commercial profit. Being on this “watch list” is in fact a sign of living in a country that care more about their citizens than old fat American corporations.

So, make sure you pirate to contribute to a free society, and get your country listed as a free nation.

Enjoy your weekend!

Comments Off

Home automation with Arduino and Android

Comments Off

Last year I made the first progress towards a DIY remote for my home automation light switches (using the Everflourish RF based system). The goal was to use the 433.92 MHz radio transmitter and an Arduino to control the switches. Furthermore, the Arduino would receive control commands from my always-on computer, with Bash command line control (’cause that’s the way I like it). Finally, a custom Android app was developed as yet another option for remote control. The project is now fully operational, with a few adjustments here and there remaining. The figure below shows a high level overview.

From left to right, the mobile phones run Android apps which present a simple interface to control the lights. Each light can be turned on and off individually, or by some pre-sets (e.g. everything on or off, “good morning”, or “movie time”). The application has hard-wired the IP address of my always on server / PC on the local network. There is of course nothing stopping me from connection to a public IP (and port-forward), however, I don’t need away-from-home control at this point. The phone app simply sends a “light switch command” to the PC, which forwards it directly to the Arduino (see details below).

In this setup, the server is only a proxy, however, it could of course also be used to implement pre-programmed settings. E.g. daily rhythm commands when we’re away on holiday, or other automated functions. Since this box is also my PC, I can also send commands directly to the Arduino over serial /dev/tty; I have some convenience bash-scripts for that. For the phone app, it only does the forwarding on a hard-wired port:

while true; do nc -l 1234 > /dev/ttyUSB0; done

Maybe that bit could be improved in the future. Especially, since there are occasionally a few hiccups in the phone to PC communication. For example, the switch from 3G to local Wifi connection might be delayed as we enter the house, so commands will buffer up, and then suddenly sent en masse. Also, this would be a natural place to expand the system with pre-programmed actions. It should also be noted that to get this to work, I had to enable communication on the port 1234 both on the Wifi router, and the local server firewall.

Finally, the Arduino, connected via USB, accepts the incoming serial messages. The encoding is simple: A two digit code where the first digit is the index of the light, and the second is 0 or 1 for off or on (see the loop() method in the source listing below). So, “11″ will turn on the first light, while “10203040″ turns off all four lights. (Note that in the code below, to make things clearer for myself, index 0 is skipped).

The code below is all there is the the Arduino part. It hard-codes the Everflourish messages and timings. It should be noted that the messages as found in the previous article, are most likely some form of Manchester code, as there are always pairs of 01 or 10. Thus, the hard-coded messages could have been shortened, but this would have added (a bit) to the complexity of the code, so I left it.

The main loop waits for incoming bytes on the serial USB connection, and sends the according light switch command to the RF transmitter on pin 2. Here there is probably also room for some improvements, to avoid illegal numbers, stuck states, etc. Once a valid code is received, the transmitted signal for that button is repeated four times, just as seen with the original remote control.

(Download source)

#define OUT_PIN 2

// Timings in micro seconds
#define SHORT_UP_0 567
#define LONG_UP_1 1134
#define DOWN_SPACE 680
#define UP_END 15000

const int PREAMBLE = {0x00};
const int OFF[] = {0x55};
const int ON[]  = {0xaa};

const int light_array[][5] = {
  {},
  // main
  {0x65, 0xa9, 0x46, 0x95, 0x67, },
  {0x65, 0xa9, 0x46, 0x96, 0x96, },
  {0x65, 0xa9, 0x46, 0x99, 0xa3, },
  {0x65, 0xa9, 0x46, 0x9a, 0x61, },

  // aux
  {0x65, 0xa9, 0x46, 0x95, 0xaa, },
  {0x65, 0xa9, 0x46, 0x96, 0x53, },
  {0x65, 0xa9, 0x46, 0x99, 0x62, },
  {0x65, 0xa9, 0x46, 0x9a, 0xa1, },
}; 

void setup() {
    Serial.begin(9600);
    Serial.println("setup");

    pinMode(OUT_PIN, OUTPUT);
}

void loop() {
  if (Serial.available() > 1) {
     int light_id = Serial.read() - '0';
     int on = Serial.read() - '0';
     Serial.print(light_id);
     Serial.println(on);
    
     for(int i = 0; i < 5; i++) {     
       send(light_array[light_id], on);
     }
     
     //delay(500);
  }
}
  
void send(const int *light_data, boolean on) {

  digitalWrite(OUT_PIN, LOW);
  delayMicroseconds(LONG_UP_1);
  
  send_bytes(PREAMBLE, 1, 4);
  send_bytes(light_data, 5, 8);
  send_bytes(on ? ON : OFF, 1, 8);

  digitalWrite(OUT_PIN, HIGH);
  delayMicroseconds(DOWN_SPACE);
  
  digitalWrite(OUT_PIN, LOW);
  delayMicroseconds(UP_END);

  digitalWrite(OUT_PIN, HIGH);
}

void send_bytes(const int *bytes, int len_bytes, int len_bits) {
  for(int b = 0; b < len_bytes; b++) {
    int data_byte = bytes[b];
    for(int i = len_bits - 1; i >= 0; i--) {
      digitalWrite(OUT_PIN, HIGH);
      delayMicroseconds(DOWN_SPACE);
    
      digitalWrite(OUT_PIN, LOW);
      int data_bit = ((data_byte >> i) & 1);
      if (data_bit == 0) {
        delayMicroseconds(SHORT_UP_0);
      } else {
        delayMicroseconds(LONG_UP_1);
      }
    }
  }
}

Comments Off

Panoramic Heads

Comments Off

Coming back to panorama photography again, I found some interesting DIY pano heads: First a wooden one by arodrix, then a just as cheap metal rod based by Peter Loud, and finally “bone” head by Matteo Cominetti. All of them interesting in that they full-fill what they set out to achieve by very simple means. At this point, it is probably worth nothing what that is: PanoHelp explains why the pivot point should be changed when taking pictures for stitching.

If you don’t want to build yourself, there are some reasonably priced options from DealExtrme. (The two first look exactly the same, even if they are different names slapped on top)

My end goal is something like the Gigapanbot, mentioned earlier, so maybe some simple brackets might also do, plus a battery. More about that later.

Comments Off

DealExtreme

Comments Off

DealExtreme / dx.com is a Hong Kong gadget site with thousands and thousands of China products of all kinds, often at very competitive prices. Every order is free of shipping charges, even if you buy a $1 item. It makes it easy to order and forget. The only downside is the shipping time, which sometimes can be around three to four weeks; some things might arrive earlier, though.

Here are the items I’ve bought for myself and others so far, followed my wish list, or list of items which might be interesting. The order list can be download in CSV form, while the wish list had to be scraped. For what it’s worth, here is a small Python script which does that, plus a bash script for HTML rendering of image links and product descriptions.

The Python script uses WebDriver / Selenium, and install for that goes:
sudo apt-get install python-pip
sudo easy_install selenium

Orders

Genuine Acecamp 2429 20L Outdoor Water Resistant Dry Bag - Yellow1000mA Car Cigarette Powered USB Adapter/Charger (DC 12V/24V)DIY 433MHz Wireless Receiving Module for Arduino - Green433MHz Wireless Transmitter Module Superregeneration for Arduino - GreenDIY 16-Key AD Keypad Module - Blue4 x 4 Matrix Switch Module - GreenES-71 II Lens Hood for CanonMini Prototype Printed Circuit Board Breadboard for Arduino (5 PCS)Ceramic Capacitor for DIY Electronic Circuit - Red (270-Piece Pack)Solderless Breadboard with 400 Tie-Point (White)USB to RS232 Serial Port Adapter (Transparent Green)Freaduino Soil Humidity Sensor for Arduino - WhiteDouble-Sided Glass Fiber Prototyping PCB Universal Board (3 x 7 / 5-Piece Pack)DIY HR-202 Humidity Detection Sensor Module - BlueAluminum Alloy Straight Hot Shoe Flash Bracket for Camera - BlackFlash Diffuser for Canon 580 EX / EX II / YongNuo YN560 / YN565 Speedlite (3 PCS)HT RJ45 RJ11 Cable TesterStainless Steel 1/4Mini USB 2.4GHz 150Mbps 802.11b/g/n WiFi Wireless Network Card Adapter - BlackUSB 2.0 2.4GHz 802.11b/g/n 150Mbps WiFi/WLAN Wireless Network AdapterUltra-Mini Nano USB 2.0 802.11n 150Mbps Wifi/WLAN Wireless Network Adapter6.3V 3300uf Aluminum Motherboard Capacitors (20-Piece Pack)DIY ZIF DIP IC Socket Set - Green (8 PCS)Solder Tip Refresher3-Pin Triode Transistor for DIY Project - Black (20 x 10-Piece Pack)Aluminum Electrolytic Capacitor for DIY Project (120-Piece Pack)DisplayPort DP Male to HDMI Female Adapter Cable - Black (15CM)Gold Plated 1080i HDMI V1.3 M-M Connection Cable (1M-Length)11x12 132-Panel Brain Teaser Magic IQ BallMicro USB To HDMI MHL Adapter - Black Gold Plated HDMI Male to DVI 24+1 Female Adapter62mm Digital Camera Lens Cover Digital Camera Lens Cover/Cap with Strap for Canon (62mm)

Interesting

Portable Dapper Stage Red/Green Laser Light Show - Blue + White (2 x AAA)Protective Plastic Case for 2.5Protective Plastic Case for 3.550mW Green + 100mW Red Laser Stage Lighting Projector with IR Remote + Tripod (100V~240V AC)Arduino Compatible Nano V3.0USB to RS232 Adapter Cable (1M-Length)RS232 Serial Port Express Card Adapter - Blue + Black + SilverPL2303HX Converter USB to TTL USB to COM Cable Module- Black (1m)Plastic Imine Membrane Screw Insulation Coat - White (1000 PCS)DIY 3mm Iron Hexagon Nut - Silver (200 PCS)2A Ceramic Tube Fuse Set (100-Piece Pack / 6 x 30mm)RGB LED Strip Control Box w/ IR 24-Key Remote Control (DC 12V)44-Key IR Remote Controller for LED Light Strip - White24-Compartment Free Combination Plastic Storage Box for Hardware Tools / GadgetsRemovable Dividers 40-Compartment Plastic Storage for Small Gadgets36-Compartment Free Combination Plastic Storage Box for Hardware Tools / GadgetsEKB-108 Plastic Battery Box for 10 x AA / AAA - Translucent WhiteStylish Anti-Slip Shoulder Strap for Canon DSLR Camera - RedMini Portable Metal Flash Holder Mount for Camera - BlackL-Shape Flash Bracket for Camera (2-Piece Set)Aluminum Alloy Straight Hot Shoe Flash Bracket for Camera - BlackFotomate 250mm Movable Range 2 Way Macro Focusing Rail Slider1/4433MHz 4-Channel Wireless Flash Trigger (1 x 12V 23A + 2 x AAA)433MHz 4-Channel Wireless Flash Trigger (1 x 12V 23A + 2 x AAA)Plastic Buckle Clip for High Power 220V SMD 3528 LED Flexible Strip - Transparent (10 PCS)DIY HR-202 Humidity Detection Sensor Module - Blue6490 DHT11 Humidity Temperature Sensor Module - Blue20 x 80mm Double-Sided PCB Prototype Boards (25 PCS)Double-Sided Glass Fiber Prototyping PCB Universal Board (10 x 16)Double-Sided Glass Fiber Prototyping PCB Universal Board (7 x 9)Electrical Bakelite Circuit Board (5x7cm/10-Piece)Double-Sided Glass Fiber Prototyping PCB Universal Board (3 x 7 / 5-Piece Pack)DS1307 I2C RTC DS1307 24C32 Real Time Clock Module for Arduino - BlueXL7105-SY DIY 2.4GHz A7105 NRF24L01 Wireless Module for Arduino - Green (2 PCS)2.4GHz NRF24L01+ Wireless Communication Module for Arduino - Green (2 PCS)433MHz Wireless Transmitter Module Superregeneration for Arduino - GreenDIY 433MHz Wireless Receiving Module for Arduino - GreenDIY DS18B20 Stainless Steel Temperature Sensor Probe - BlackDHT21/AM2301 Capacitive Digital Temperature & Humidity Sensor - BlackENC28J60 Ethernet Module for Arduino - WhiteHR31 Humidity Sensor Module for Arduino - BlueNRF905 Wireless Communication Transmission Module for Arduino2 Channel 4 Pins NC Type Shock Switch Sensor ModuleVibration Alarm Sensor Module for Arduino - BlackBMP085 Altimeter Atmospheric Pressure Module for ArduinoHuman Body Touch Sensor Module for Arduino - Black + BlueDS1302 Real Time Clock Module with Battery CR2032DS18B20 Temperature Sensor - Black + SilverMicro SD Card Storage Memory Board Module for ArduinoArduino Compatible Digital Vibration Switch Sensor ModuleArduino Tilt Switch Sensor Module - BlackArduino Compatible 3-Pin Light Sensor Module - RedArduino Compatible DHT22 2302 Digital Temperature Humidity SensorDS18B20 Digital Temperature Sensor Module for Arduino (-55~125IR Receiver Module Wireless Remote Control Kit for Arduino (1 x CR2025)Arduino 5V Relay Module - Blue + BlackDHT11 Arduino Compatible Digital Temperature Humidity Sensor ModuleFreaduino Soil Humidity Sensor for Arduino - WhiteHall Effect Magnetic Sensor Module for Arduino (DC 5V)MMA8451 3-Axial Triaxial Digital Accelerometer ModuleArduino 3-Axis Digital Gyroscope Sensor Module - Blue

Comments Off

Historical Cost of Computer Memory and Storage

1 comment

Computer storage, primary and secondary memory, has seen a tremendous phase of development over the last fifty years. As new technology has been brought to the market prices have continued to decline steadily at a logarithmic scale. For magnetic storage, the trend has been very stable over the last thirty years, with prices per MB going down around a third every year, or a ninety percent every five years. For primary storage, the trend has been more volatile, but overall we see a similar rate of decline all the way back to the first flip-flops in the 1950s.

John C. McCallum has done a good job collecting all the data over the years, and going back to computer magazines for reference. However, since the beginning of 2012 there have been no updates, so I’ve taken up the work where he left off. I’ve added a new page to my site, where I will collect the data and update the graphs over time: hblok.net/storage


(Click image for larger version)

In the first update, the harddisk prices are most interesting, and we can now clearly see the effect of the flood disaster in late 2011. It has interrupted a thirty year trend, and as a result prices are about the same per MB as they were one and a half years ago. Now the question is, will this have a lasting effect on the magnetic harddisk prices, or will it be just a blip in history, as technological improvements bring us cheaper storage at the same phase.

The two plots below extrapolate the trend over the last thirty years, with two different scenarios: 1) Improvements in technology will catch up with the delay over the last year, and thus the thirty year trend will continue unaffected (red line). Or 2) phase of improvments will not change, and thus the rate of decline in price will stay the same, but shift the line by about a year (blue line).


(Click image for larger version)

The price is 0.4 cents per GB today (4e-5 per MB). If we look two years ahead, with the uninterrupted scenario (red line), the price would be 0.05 cents per GB in 2015 (5e-6 per MB), or put in different ways: 3 TB of storage which costs $125 today would have to go down to about $15 in two years, or for the same $125 you’d have to get a whopping 25 TB (yes, twenty five!). Given the recent news from the major harddisk vendors, that seems rather unlikely to happen; they’re only planning for 5 TB drives at the end of this year. So, over two years time, prices will not catch up. Perhaps this will change looking even further ahead, however, extrapolating technological trends beyond a year or two is merely guessing.

If we look at the second scenario, where we assume that the prices will continue to decline at the same rate as they have done in the past, given today’s price we’re then looking at about 0.15 cents per GB (1.6e-5 per MB). That would mean that today’s 3 TB would go for around $50, while $125 would buy you about 8 TB. That seems more reasonable, and also in line with what products are being brought to market and in research right now. If the rumoured 5 TB Western Digital disk will be realised with four platters (4 * 1.25 TB) at the end of this year, it means five platter 6.25 TB (5 * 1.25) disks are already a possibility. Increasing storage density another 30% to reach 8 TB over the following year seems a reasonable assumption.

perceptualdiff – compare images perceptually

Comments Off

I recently found myself needing to compare bitmap images, to see if they were about the same. The images were from Gnuplot generate graphs, and I wanted to check whether subtle changes in the data had not introduced unexpected changed to the plot. A simple binary diff told me that there were indeed some differences, however, comparing them manually was not possible.

Enter the handy tool perceptualdiff, which lets you compare TIF and PNG images based on a perceptual metric. It actually goes beyond simple bitwise pixel diff, and tries to compare based on a model of the human visual system. Consider the examples below, and it is clear that it is a useful tool. It makes it easy to see where the small differences were introduced. (As it turns out, it is only the result of the two plots being generated on different machines, with different versions of Gnuplot and possibly also different available fonts).

The tool is in the Fedora repository, so a simple yum is enough:
yum install perceptualdiff

To generate an output diff image, this command does the job:
perceptualdiff -output diff.ppm image1.png image2.png

Comments Off

Fonts for Gnuplot

Comments Off

After struggling with fonts in Gnuplot 4.6 (on Fedora 17) (getting the not so useful error “gdImageStringFT: Could not find/open font while printing string”), I found tonicas post on debugging the issue. Although helpful, it did not give the full solution to my problem. It turns out, many of the old fonts are not available in Fedora 17 at all.

I wanted a sans-serif font, and in the end I went for the DejaVuSans. After installing the font packages, I specifically exported that path for use with Gnuplot:

sudo yum install dejavu-sans-fonts dejavu-fonts-common
 
export GDFONTPATH=/usr/share/fonts/dejavu

Then I can use that font in Gnuplot by addressing it specifically:

set title 'Test' font 'DejaVuSans,21'

Comments Off