Posts tagged ·

Everflourish

·...

Home automation with Arduino and Android

Comments Off

Last year I made the first progress towards a DIY remote for my home automation light switches (using the Everflourish RF based system). The goal was to use the 433.92 MHz radio transmitter and an Arduino to control the switches. Furthermore, the Arduino would receive control commands from my always-on computer, with Bash command line control (’cause that’s the way I like it). Finally, a custom Android app was developed as yet another option for remote control. The project is now fully operational, with a few adjustments here and there remaining. The figure below shows a high level overview.

From left to right, the mobile phones run Android apps which present a simple interface to control the lights. Each light can be turned on and off individually, or by some pre-sets (e.g. everything on or off, “good morning”, or “movie time”). The application has hard-wired the IP address of my always on server / PC on the local network. There is of course nothing stopping me from connection to a public IP (and port-forward), however, I don’t need away-from-home control at this point. The phone app simply sends a “light switch command” to the PC, which forwards it directly to the Arduino (see details below).

In this setup, the server is only a proxy, however, it could of course also be used to implement pre-programmed settings. E.g. daily rhythm commands when we’re away on holiday, or other automated functions. Since this box is also my PC, I can also send commands directly to the Arduino over serial /dev/tty; I have some convenience bash-scripts for that. For the phone app, it only does the forwarding on a hard-wired port:

while true; do nc -l 1234 > /dev/ttyUSB0; done

Maybe that bit could be improved in the future. Especially, since there are occasionally a few hiccups in the phone to PC communication. For example, the switch from 3G to local Wifi connection might be delayed as we enter the house, so commands will buffer up, and then suddenly sent en masse. Also, this would be a natural place to expand the system with pre-programmed actions. It should also be noted that to get this to work, I had to enable communication on the port 1234 both on the Wifi router, and the local server firewall.

Finally, the Arduino, connected via USB, accepts the incoming serial messages. The encoding is simple: A two digit code where the first digit is the index of the light, and the second is 0 or 1 for off or on (see the loop() method in the source listing below). So, “11″ will turn on the first light, while “10203040″ turns off all four lights. (Note that in the code below, to make things clearer for myself, index 0 is skipped).

The code below is all there is the the Arduino part. It hard-codes the Everflourish messages and timings. It should be noted that the messages as found in the previous article, are most likely some form of Manchester code, as there are always pairs of 01 or 10. Thus, the hard-coded messages could have been shortened, but this would have added (a bit) to the complexity of the code, so I left it.

The main loop waits for incoming bytes on the serial USB connection, and sends the according light switch command to the RF transmitter on pin 2. Here there is probably also room for some improvements, to avoid illegal numbers, stuck states, etc. Once a valid code is received, the transmitted signal for that button is repeated four times, just as seen with the original remote control.

(Download source)

#define OUT_PIN 2

// Timings in micro seconds
#define SHORT_UP_0 567
#define LONG_UP_1 1134
#define DOWN_SPACE 680
#define UP_END 15000

const int PREAMBLE = {0x00};
const int OFF[] = {0x55};
const int ON[]  = {0xaa};

const int light_array[][5] = {
  {},
  // main
  {0x65, 0xa9, 0x46, 0x95, 0x67, },
  {0x65, 0xa9, 0x46, 0x96, 0x96, },
  {0x65, 0xa9, 0x46, 0x99, 0xa3, },
  {0x65, 0xa9, 0x46, 0x9a, 0x61, },

  // aux
  {0x65, 0xa9, 0x46, 0x95, 0xaa, },
  {0x65, 0xa9, 0x46, 0x96, 0x53, },
  {0x65, 0xa9, 0x46, 0x99, 0x62, },
  {0x65, 0xa9, 0x46, 0x9a, 0xa1, },
}; 

void setup() {
    Serial.begin(9600);
    Serial.println("setup");

    pinMode(OUT_PIN, OUTPUT);
}

void loop() {
  if (Serial.available() > 1) {
     int light_id = Serial.read() - '0';
     int on = Serial.read() - '0';
     Serial.print(light_id);
     Serial.println(on);
    
     for(int i = 0; i < 5; i++) {     
       send(light_array[light_id], on);
     }
     
     //delay(500);
  }
}
  
void send(const int *light_data, boolean on) {

  digitalWrite(OUT_PIN, LOW);
  delayMicroseconds(LONG_UP_1);
  
  send_bytes(PREAMBLE, 1, 4);
  send_bytes(light_data, 5, 8);
  send_bytes(on ? ON : OFF, 1, 8);

  digitalWrite(OUT_PIN, HIGH);
  delayMicroseconds(DOWN_SPACE);
  
  digitalWrite(OUT_PIN, LOW);
  delayMicroseconds(UP_END);

  digitalWrite(OUT_PIN, HIGH);
}

void send_bytes(const int *bytes, int len_bytes, int len_bits) {
  for(int b = 0; b < len_bytes; b++) {
    int data_byte = bytes[b];
    for(int i = len_bits - 1; i >= 0; i--) {
      digitalWrite(OUT_PIN, HIGH);
      delayMicroseconds(DOWN_SPACE);
    
      digitalWrite(OUT_PIN, LOW);
      int data_bit = ((data_byte >> i) & 1);
      if (data_bit == 0) {
        delayMicroseconds(SHORT_UP_0);
      } else {
        delayMicroseconds(LONG_UP_1);
      }
    }
  }
}

Comments Off

Home Automation on 433.92 MHz with Arduino

1 comment

Last year I wrote about my plans with the RF Link Transmitter, Receiver, and Everflourish home automation system seen in the pictures below. The idea was to replace the remote with my own system, and control the switches using the RF transmitter and an Arduino. However, the problem was to first decode the bespoke message sent from the remote so I could reproduce it. Initially, I was concerned that the RF receiver was giving a too noisy signal; only reading its value from the Arduino and printing to Serial.out did not yield anything useful. It was not before I took the Arduino out of the loop, and connected the receiver directly to a oscilloscope that things started to look more promising.

On the oscilloscope, I could see the message very clearly. Now I just had to write it down, and analyse the code for all different buttons and remotes. Here the problem was that each button sends a rather long message, 52 bits in total, and the old oscilloscope I tried did not have much memory to store recordings to. Also, it wasn’t mine, so was not so convenient to access and work with. Enter the real break-through of this story: Dave Houston brilliant idea and post about a poor man’s oscilloscope – your sound card! He works with various X10 systems himself, and has more interesting information on his site. For the sound-card hook-up, any audio-wire will do; if using a stereo wire and mono recording, figure out which side is which, add some resistors (39k and 10k was suggested; I ended up with one 1 k Ohm and another at 100 Ohm, which worked just as well). David also points out that the line-in should be used, rather than the microphone plug.

The next bright idea came from Ray, who has done exactly the same project as I wanted to do, but with a different remote control system. He shows how the open source audio tool Audacity can be used to record and analyse the message. I found that Audacity also had a neat feature in that you could add a Label Track, that made it easy to write down the code, and later export it, as seen below.

The screenshots below show a recorded signal in Audacity, first at an overview, then zoomed in, labelled, and finally marking the length of each segment. Notice in the first picture how there is a lot of noise before the remote button is pushed, but that it is very clear where a signal comes in (repeating four times). Also, as seen from the second picture, the signal is strong and clear, without much noise as all, which speaks for the quality of the RF receiver, I guess. In the third picture, the segments has been labelled; see below for how to extract that, and further details about the code.

Click on the pictures for larger versions.

Each of the remote control systems have their own proprietary encoding for the messages sent, and the Everflourish system I have has a very different code from Ray’s Stanley equipment. Each bit in the message is expressed by a high with varying length followed by a low of common length. In the last picture above, these segments have been labelled, and as can be seen, the long high is twice the length of the short (50 vs. 25 samples) while the low space is at 30 samples. That’s using a sample rate of 44100 Hz. Ray points out that you can zoom all the way in, and count the individual samples. Another way is to use the unit selection in Audacity (at the bottom of the window), and select the highest precision unit “hh:mm:ss + samples”, and then select the area of one segment. The start and end timing is then shown at the bottom of the window, and it’s a matter of simple subtraction to find the length. Much easier than counting 50 samples.

For the Everflourish code, only the highs matter then, so I recorded one button, cut out the relevant message, copy/pasted it to a new Audacity project, added a Label Track as seen from the menu above (Tracks -> Add New -> Label Track) and went through and labelled the highs with 0 for short and 1 for long. That was just an arbitrary choice. Once done, the new project was saved with a descriptive name of the remote and button, and the labels could be extracted from the XML project file with something like this. (As a bonus, the digits are split into 4-bit nibbles.)

cat remote1_button2_off.aup | grep label.*title | cut -d '"' -f 6 | tr -d '\n' | sed "s/\([01]\{4\}\)/\1 /g"

Below, the messages from three different remotes (with 4 + 4 + 8 = 16 buttons) are listed, but in the interest of space and simplicity, only some are shown. From this I conclude that only a few parts of the message are consistent between the different remotes. First, it always starts with a 0-nibble. The ON and OFF buttons are consistent and reversed across all buttons. Beyond that, it’s hard to tell what is encoded. There is possibly a marker in the middle, at the seventh nibble, marked in gray, but it could be a coincidence. The third nibble is also the same for all three remotes. As can be seen, the two first remotes share data in the first part, and one could be led to believe that this was a common preamble, but this must be rejected when looking at the third advanced remote.

That seems to go against what is assumed in the the source code from TellDus. There they assume a “house” and “unit” (button) key on the last nibbles before the on/off code, but that does not match what I see across my remotes. There seems to be nothing identifying a specific remote, but rather all buttons are unique, and there is possibly some kind of general batch (as opposed to unique serial) number at the beginning of the message.

Simple 4 button remote

1 - OFF : 0000 0110 0101 1010 1001 0110 0110 1001 0101 1010 1010 0101 0101
1 - ON  : 0000 0110 0101 1010 1001 0110 0110 1001 0101 1010 1010 1010 1010

2 - OFF : 0000 0110 0101 1010 1001 0110 0110 1001 0110 0101 0101 0101 0101
2 - ON  : 0000 0110 0101 1010 1001 0110 0110 1001 0110 0101 0101 1010 1010

3 - OFF : 0000 0110 0101 1010 1001 0110 0110 1001 1001 0110 0110 0101 0101
3 - ON  : 0000 0110 0101 1010 1001 0110 0110 1001 1001 0110 0110 1010 1010

Simple 4 button remote

1 - OFF : 0000 0110 0101 1010 1001 0101 0110 1001 0101 0110 1001 0101 0101
1 - ON  : 0000 0110 0101 1010 1001 0101 0110 1001 0101 0110 1001 1010 1010

2 - OFF : 0000 0110 0101 1010 1001 0101 0110 1001 0110 1000 1000 0101 0101
2 - ON  : 0000 0110 0101 1010 1001 0101 0110 1001 0110 1000 1000 1010 1010

3 - OFF : 0000 0110 0101 1010 1001 0101 0110 1001 1001 1010 0101 0101 0101
3 - ON  : 0000 0110 0101 1010 1001 0101 0110 1001 1001 1010 0101 1010 1010

Advanced 8 channel remote

1 - OFF : 0000 1010 0101 0101 0101 1010 0110 0101 0101 0110 0110 0101 0101
1 - ON  : 0000 1010 0101 0101 0101 1010 0110 0101 0101 0110 0110 1010 1010

2 - OFF : 0000 1010 0101 0101 0101 1010 0110 0101 0110 1001 1001 0101 0101
2 - ON  : 0000 1010 0101 0101 0101 1010 0110 0101 0110 1001 1001 1010 1010

Determining the length of each segment to be sent is done by looking at the sample rate. I used 44100 Hz, so the duration in micro seconds for the 30 sample space segment is 30 / 44100 * 1000000 = 680 µs. For the short 25 sample high, it is 567 µs, and double or 1134 µs for the long. When sending the message, I did not need to subtract anything from that, as Ray mentioned in his post. It is worth noting the initial high before the start of the message, and also at the end. Furthermore, it both starts and finishes with a low space, so whatever for-loop you create, one of these will be before or after. However, what is intriguing, is that all this can be reversed. That is, switch HIGH and LOW in the code, and the switches will still react. I’m guessing this is a robustness measure designed into the Everflourish system, but I have to admit that goes beyond my RF knowledge.

Finally, it’s worth mentioning the All On or All Off buttons. They simply send a series of messages for each of the buttons, repeating each button message four times before moving on to the next. When using the single on/off buttons, the message is also repeated four times for a short click.

And that’s all for now. I expect to round this project up with a post on the complete system, including communication from the computer, and some kind of Android application as a remote. Other ideas include auto-triggering the lights when detecting a pre-defined Bluetooth device, and possibly switching off in the absence of any device. Possibly some programmed behaviour during holidays, and pre-set configuration for certain situations, e.g. “movie setting”, “dinner time”, etc.

RF Communication on 433.92 MHz

3 comments

I recently got a Sparkfun order on my door, so it’s time to play. In the box was a RF Link Transmitter, and Receiver. They are sold as 434 MHz radio wireless links. Others talk about 433 MHz. To be precise, it’s 433.92 MHz. That matters, because if you search for that number, you will find the so called home automation systems using that frequency, including X10, Everflourish, and many others. My plan then, is to build something similar to the TellStick from TellDus, which control these devices from my computer. Connect that up to an Android app, and I could control my lights and other appliances from any mobile phone.

First things first, though. Hooking up the bits was easy, following these two similar tutorials. Using the VirtualWire Arduino library v1.5 (1.6 released at the time of writing) by Mike McCauley (download version 1.5) transmitting data was a breeze. The library includes example code for transmitter and receiver, simply upload and go. Note that the transmitter data should be connected to pin 12 on the Arudino, and the receiver on pin 11 on the other. Data was received loud and clear, without errors. Mike’s library is well written, and covers several important aspects of RF communication, including a dedicated protocol, CRC handling, robust encoding over the air, baud rate, and to top it off, an easy to use API.

Now, of course when using my Everflourish nothing happened. Which was a good sign; it did not interfere with the Virtual Wire transmission. That remote is using a different protocol to talk to the light switches. Luckily, it has already been reversed engineered and the source code is available from TellDus. It will probably take some time to get this working. Meanwhile, some pictures.