Posts tagged ·

X10

·...

Nexa Home Automation – 433 MHz codes

1 comment

After successfully decoding the Everflourish home automation controls, and integrating that with a DIY Android app, I thought it was time to expand the system. I picked up a new set of switches and remote, however, this was a different brand, Swedish Nexa, and thus a slightly different code. So, time do go through the RF decoding again, and adjust my RF remote source code.

I found two people working with this system, taking different approaches to controlling it from the Arudino. The author of the arduinocoder seems to interface through the remote, while Sebastian Nilsson had success with the HomeEasy project. I was tempted to use that code, however, as I was also interested in analysing the raw data, I brought out my DIY “oscilloscope” instead.

Since this home-made RF-to-audio device is something I thought I’d use in the future as well, I took the RF receiver off the breadboard, and soldered it onto a mini PCB, as seen in the pictures above. While working with RF components, I always find myself coming back to winavr’s and Dave Houston’s blog on how to wire this together. Unfortunately, the receiver was a bit more sensitive to the voltage level than I had expected, so taking 4.5 or 6 V from my power brick did not work. Instead, I used an Arduino, or a small 5V converter.

After this was done and plugged in, I found myself in Linux audio problems. Between two HDMI video cards, one external USB audio source, and the onboard audio port, it has gotten rather difficult to find the correct knob to turn to make sure the correct input is not muted, or too low. Here I found the application pavucontrol to very useful, as it clearly displays which devices and ports are available, and which applications output sound. (yum install pavucontrol). In Audacity, the input source was simply “default”, while recording in mono.
Remember to NOT stick any home made equipment into the microphone port; use line-in instead.

Part of the wire message from one of the buttons on the remote can be seen above. Compared to the Everflourish system, it looks a bit more dragged out and the edges are not so clear cut. There’s probably a few reasons for that: The resistors I used between the RF receiver and audio line were slightly different (this time 47k and 100k, vs. 100 Ohm and 1k in the previous). Also, the power source played a small role. Finally, the message is different, and in particular, the timings of wave is shorter; see below for details.

wire_bits=`cat $file | grep label.*title | cut -d '"' -f 6 | tr -d '\n' | sed "s/\([01]\{2\}\)/\1 /g"`
data_bits=`echo $wire_bits | sed "s/01/0/g" | sed "s/10/1/g"`
data_bytes_bin=`echo $data_bits | tr -d ' ' | sed "s/\([01]\{8\}\)/\1 /g"`
data_bytes_hex=`echo $data_bytes_bin | tr ' ' '\n' | while read b; do echo "obase=16; ibase=2; $b" | bc; done | tr '\n' ' '`
echo -e " $wire_bits \n $data_bits \n $data_bytes_bin \n $data_bytes_hex"

The snippet above extracts the labels from the Audacity XML project file and groups every wire bit, so the correspond to one data bit. A 01 on the wire is a 0, and a 10 is 1. This is a typical Manchester code. Next, the data bits are grouped into bytes, and finally expressed in hex. It then becomes clear how the message is structured: The 26 first (most significant) bits are a unique “house” or “remote control” code; the last four bits are the button ID; the 27th bit is a group flag, and the following (28th) is the the on/off bit. See also the documentation from the HomeEasy project below for further details.


01 01 10 01 01 01 10 10 10 10 10 10 01 01 10 10 10 10 10 01 10 10 01 10 10 01 01 10 01 01 01 10
0 0 1 0 0 0 1 1 1 1 1 1 0 0 1 1 1 1 1 0 1 1 0 1 1 0 0 1 0 0 0 1
00100011 11110011 11101101 10010001
23 F3 ED 91

The data is encoded on the wire (aerial) as a Manchester code.

A latch of 275us high, 2675us low is sent before the data.
There is a gap of 10ms between each message.

0 = holding the line high for 275us then low for 275us.
1 = holding the line high for 275us then low for 1225us.

The timings seem to vary quite noticeably between devices. HE300 devices have a
low for about 1300us for a 1 whereas HE303 devices seem to have a low of about
1100us. If this script does not detect your signals try relaxing the timing
conditions.

Each actual bit of data is encoded as two bits on the wire as:
Data 0 = Wire 01
Data 1 = Wire 10

The actual message is 32 bits of data (64 wire bits):
bits 0-25: the group code – a 26bit number assigned to controllers.
bit 26: group flag
bit 27: on/off flag
bits 28-31: the device code – a 4bit number.

Once that was all clear, I just had to modify my transmitter code, which is connected over serial to my computer, and takes switch commands from my mobile phone. In the future, I will probably move to use an Ethernet shield, or possibly Bluetooth. There has been some problems with the connection to the Arudino, and also in sending the RF signals to the switches. In the new code, I’ve added quite a lot of extra redundancy, repeating the same command many times to makes sure it is received by the switch.

I’ve split the code into a library which can send commands to the Everflourish and Nexa systems, and the main sketch which takes commands from serial. There’s probably a few things to improve on, so feel free to send me comments or patches.

The code is released under GPL 3, or later version.

Home automation with Arduino and Android

Comments Off

Last year I made the first progress towards a DIY remote for my home automation light switches (using the Everflourish RF based system). The goal was to use the 433.92 MHz radio transmitter and an Arduino to control the switches. Furthermore, the Arduino would receive control commands from my always-on computer, with Bash command line control (’cause that’s the way I like it). Finally, a custom Android app was developed as yet another option for remote control. The project is now fully operational, with a few adjustments here and there remaining. The figure below shows a high level overview.

From left to right, the mobile phones run Android apps which present a simple interface to control the lights. Each light can be turned on and off individually, or by some pre-sets (e.g. everything on or off, “good morning”, or “movie time”). The application has hard-wired the IP address of my always on server / PC on the local network. There is of course nothing stopping me from connection to a public IP (and port-forward), however, I don’t need away-from-home control at this point. The phone app simply sends a “light switch command” to the PC, which forwards it directly to the Arduino (see details below).

In this setup, the server is only a proxy, however, it could of course also be used to implement pre-programmed settings. E.g. daily rhythm commands when we’re away on holiday, or other automated functions. Since this box is also my PC, I can also send commands directly to the Arduino over serial /dev/tty; I have some convenience bash-scripts for that. For the phone app, it only does the forwarding on a hard-wired port:

while true; do nc -l 1234 > /dev/ttyUSB0; done

Maybe that bit could be improved in the future. Especially, since there are occasionally a few hiccups in the phone to PC communication. For example, the switch from 3G to local Wifi connection might be delayed as we enter the house, so commands will buffer up, and then suddenly sent en masse. Also, this would be a natural place to expand the system with pre-programmed actions. It should also be noted that to get this to work, I had to enable communication on the port 1234 both on the Wifi router, and the local server firewall.

Finally, the Arduino, connected via USB, accepts the incoming serial messages. The encoding is simple: A two digit code where the first digit is the index of the light, and the second is 0 or 1 for off or on (see the loop() method in the source listing below). So, “11″ will turn on the first light, while “10203040″ turns off all four lights. (Note that in the code below, to make things clearer for myself, index 0 is skipped).

The code below is all there is the the Arduino part. It hard-codes the Everflourish messages and timings. It should be noted that the messages as found in the previous article, are most likely some form of Manchester code, as there are always pairs of 01 or 10. Thus, the hard-coded messages could have been shortened, but this would have added (a bit) to the complexity of the code, so I left it.

The main loop waits for incoming bytes on the serial USB connection, and sends the according light switch command to the RF transmitter on pin 2. Here there is probably also room for some improvements, to avoid illegal numbers, stuck states, etc. Once a valid code is received, the transmitted signal for that button is repeated four times, just as seen with the original remote control.

(Download source)

#define OUT_PIN 2

// Timings in micro seconds
#define SHORT_UP_0 567
#define LONG_UP_1 1134
#define DOWN_SPACE 680
#define UP_END 15000

const int PREAMBLE = {0x00};
const int OFF[] = {0x55};
const int ON[]  = {0xaa};

const int light_array[][5] = {
  {},
  // main
  {0x65, 0xa9, 0x46, 0x95, 0x67, },
  {0x65, 0xa9, 0x46, 0x96, 0x96, },
  {0x65, 0xa9, 0x46, 0x99, 0xa3, },
  {0x65, 0xa9, 0x46, 0x9a, 0x61, },

  // aux
  {0x65, 0xa9, 0x46, 0x95, 0xaa, },
  {0x65, 0xa9, 0x46, 0x96, 0x53, },
  {0x65, 0xa9, 0x46, 0x99, 0x62, },
  {0x65, 0xa9, 0x46, 0x9a, 0xa1, },
}; 

void setup() {
    Serial.begin(9600);
    Serial.println("setup");

    pinMode(OUT_PIN, OUTPUT);
}

void loop() {
  if (Serial.available() > 1) {
     int light_id = Serial.read() - '0';
     int on = Serial.read() - '0';
     Serial.print(light_id);
     Serial.println(on);
    
     for(int i = 0; i < 5; i++) {     
       send(light_array[light_id], on);
     }
     
     //delay(500);
  }
}
  
void send(const int *light_data, boolean on) {

  digitalWrite(OUT_PIN, LOW);
  delayMicroseconds(LONG_UP_1);
  
  send_bytes(PREAMBLE, 1, 4);
  send_bytes(light_data, 5, 8);
  send_bytes(on ? ON : OFF, 1, 8);

  digitalWrite(OUT_PIN, HIGH);
  delayMicroseconds(DOWN_SPACE);
  
  digitalWrite(OUT_PIN, LOW);
  delayMicroseconds(UP_END);

  digitalWrite(OUT_PIN, HIGH);
}

void send_bytes(const int *bytes, int len_bytes, int len_bits) {
  for(int b = 0; b < len_bytes; b++) {
    int data_byte = bytes[b];
    for(int i = len_bits - 1; i >= 0; i--) {
      digitalWrite(OUT_PIN, HIGH);
      delayMicroseconds(DOWN_SPACE);
    
      digitalWrite(OUT_PIN, LOW);
      int data_bit = ((data_byte >> i) & 1);
      if (data_bit == 0) {
        delayMicroseconds(SHORT_UP_0);
      } else {
        delayMicroseconds(LONG_UP_1);
      }
    }
  }
}

Comments Off

Home Automation on 433.92 MHz with Arduino

1 comment

Last year I wrote about my plans with the RF Link Transmitter, Receiver, and Everflourish home automation system seen in the pictures below. The idea was to replace the remote with my own system, and control the switches using the RF transmitter and an Arduino. However, the problem was to first decode the bespoke message sent from the remote so I could reproduce it. Initially, I was concerned that the RF receiver was giving a too noisy signal; only reading its value from the Arduino and printing to Serial.out did not yield anything useful. It was not before I took the Arduino out of the loop, and connected the receiver directly to a oscilloscope that things started to look more promising.

On the oscilloscope, I could see the message very clearly. Now I just had to write it down, and analyse the code for all different buttons and remotes. Here the problem was that each button sends a rather long message, 52 bits in total, and the old oscilloscope I tried did not have much memory to store recordings to. Also, it wasn’t mine, so was not so convenient to access and work with. Enter the real break-through of this story: Dave Houston brilliant idea and post about a poor man’s oscilloscope – your sound card! He works with various X10 systems himself, and has more interesting information on his site. For the sound-card hook-up, any audio-wire will do; if using a stereo wire and mono recording, figure out which side is which, add some resistors (39k and 10k was suggested; I ended up with one 1 k Ohm and another at 100 Ohm, which worked just as well). David also points out that the line-in should be used, rather than the microphone plug.

The next bright idea came from Ray, who has done exactly the same project as I wanted to do, but with a different remote control system. He shows how the open source audio tool Audacity can be used to record and analyse the message. I found that Audacity also had a neat feature in that you could add a Label Track, that made it easy to write down the code, and later export it, as seen below.

The screenshots below show a recorded signal in Audacity, first at an overview, then zoomed in, labelled, and finally marking the length of each segment. Notice in the first picture how there is a lot of noise before the remote button is pushed, but that it is very clear where a signal comes in (repeating four times). Also, as seen from the second picture, the signal is strong and clear, without much noise as all, which speaks for the quality of the RF receiver, I guess. In the third picture, the segments has been labelled; see below for how to extract that, and further details about the code.

Click on the pictures for larger versions.

Each of the remote control systems have their own proprietary encoding for the messages sent, and the Everflourish system I have has a very different code from Ray’s Stanley equipment. Each bit in the message is expressed by a high with varying length followed by a low of common length. In the last picture above, these segments have been labelled, and as can be seen, the long high is twice the length of the short (50 vs. 25 samples) while the low space is at 30 samples. That’s using a sample rate of 44100 Hz. Ray points out that you can zoom all the way in, and count the individual samples. Another way is to use the unit selection in Audacity (at the bottom of the window), and select the highest precision unit “hh:mm:ss + samples”, and then select the area of one segment. The start and end timing is then shown at the bottom of the window, and it’s a matter of simple subtraction to find the length. Much easier than counting 50 samples.

For the Everflourish code, only the highs matter then, so I recorded one button, cut out the relevant message, copy/pasted it to a new Audacity project, added a Label Track as seen from the menu above (Tracks -> Add New -> Label Track) and went through and labelled the highs with 0 for short and 1 for long. That was just an arbitrary choice. Once done, the new project was saved with a descriptive name of the remote and button, and the labels could be extracted from the XML project file with something like this. (As a bonus, the digits are split into 4-bit nibbles.)

cat remote1_button2_off.aup | grep label.*title | cut -d '"' -f 6 | tr -d '\n' | sed "s/\([01]\{4\}\)/\1 /g"

Below, the messages from three different remotes (with 4 + 4 + 8 = 16 buttons) are listed, but in the interest of space and simplicity, only some are shown. From this I conclude that only a few parts of the message are consistent between the different remotes. First, it always starts with a 0-nibble. The ON and OFF buttons are consistent and reversed across all buttons. Beyond that, it’s hard to tell what is encoded. There is possibly a marker in the middle, at the seventh nibble, marked in gray, but it could be a coincidence. The third nibble is also the same for all three remotes. As can be seen, the two first remotes share data in the first part, and one could be led to believe that this was a common preamble, but this must be rejected when looking at the third advanced remote.

That seems to go against what is assumed in the the source code from TellDus. There they assume a “house” and “unit” (button) key on the last nibbles before the on/off code, but that does not match what I see across my remotes. There seems to be nothing identifying a specific remote, but rather all buttons are unique, and there is possibly some kind of general batch (as opposed to unique serial) number at the beginning of the message.

Simple 4 button remote

1 - OFF : 0000 0110 0101 1010 1001 0110 0110 1001 0101 1010 1010 0101 0101
1 - ON  : 0000 0110 0101 1010 1001 0110 0110 1001 0101 1010 1010 1010 1010

2 - OFF : 0000 0110 0101 1010 1001 0110 0110 1001 0110 0101 0101 0101 0101
2 - ON  : 0000 0110 0101 1010 1001 0110 0110 1001 0110 0101 0101 1010 1010

3 - OFF : 0000 0110 0101 1010 1001 0110 0110 1001 1001 0110 0110 0101 0101
3 - ON  : 0000 0110 0101 1010 1001 0110 0110 1001 1001 0110 0110 1010 1010

Simple 4 button remote

1 - OFF : 0000 0110 0101 1010 1001 0101 0110 1001 0101 0110 1001 0101 0101
1 - ON  : 0000 0110 0101 1010 1001 0101 0110 1001 0101 0110 1001 1010 1010

2 - OFF : 0000 0110 0101 1010 1001 0101 0110 1001 0110 1000 1000 0101 0101
2 - ON  : 0000 0110 0101 1010 1001 0101 0110 1001 0110 1000 1000 1010 1010

3 - OFF : 0000 0110 0101 1010 1001 0101 0110 1001 1001 1010 0101 0101 0101
3 - ON  : 0000 0110 0101 1010 1001 0101 0110 1001 1001 1010 0101 1010 1010

Advanced 8 channel remote

1 - OFF : 0000 1010 0101 0101 0101 1010 0110 0101 0101 0110 0110 0101 0101
1 - ON  : 0000 1010 0101 0101 0101 1010 0110 0101 0101 0110 0110 1010 1010

2 - OFF : 0000 1010 0101 0101 0101 1010 0110 0101 0110 1001 1001 0101 0101
2 - ON  : 0000 1010 0101 0101 0101 1010 0110 0101 0110 1001 1001 1010 1010

Determining the length of each segment to be sent is done by looking at the sample rate. I used 44100 Hz, so the duration in micro seconds for the 30 sample space segment is 30 / 44100 * 1000000 = 680 µs. For the short 25 sample high, it is 567 µs, and double or 1134 µs for the long. When sending the message, I did not need to subtract anything from that, as Ray mentioned in his post. It is worth noting the initial high before the start of the message, and also at the end. Furthermore, it both starts and finishes with a low space, so whatever for-loop you create, one of these will be before or after. However, what is intriguing, is that all this can be reversed. That is, switch HIGH and LOW in the code, and the switches will still react. I’m guessing this is a robustness measure designed into the Everflourish system, but I have to admit that goes beyond my RF knowledge.

Finally, it’s worth mentioning the All On or All Off buttons. They simply send a series of messages for each of the buttons, repeating each button message four times before moving on to the next. When using the single on/off buttons, the message is also repeated four times for a short click.

And that’s all for now. I expect to round this project up with a post on the complete system, including communication from the computer, and some kind of Android application as a remote. Other ideas include auto-triggering the lights when detecting a pre-defined Bluetooth device, and possibly switching off in the absence of any device. Possibly some programmed behaviour during holidays, and pre-set configuration for certain situations, e.g. “movie setting”, “dinner time”, etc.

RF Communication on 433.92 MHz

3 comments

I recently got a Sparkfun order on my door, so it’s time to play. In the box was a RF Link Transmitter, and Receiver. They are sold as 434 MHz radio wireless links. Others talk about 433 MHz. To be precise, it’s 433.92 MHz. That matters, because if you search for that number, you will find the so called home automation systems using that frequency, including X10, Everflourish, and many others. My plan then, is to build something similar to the TellStick from TellDus, which control these devices from my computer. Connect that up to an Android app, and I could control my lights and other appliances from any mobile phone.

First things first, though. Hooking up the bits was easy, following these two similar tutorials. Using the VirtualWire Arduino library v1.5 (1.6 released at the time of writing) by Mike McCauley (download version 1.5) transmitting data was a breeze. The library includes example code for transmitter and receiver, simply upload and go. Note that the transmitter data should be connected to pin 12 on the Arudino, and the receiver on pin 11 on the other. Data was received loud and clear, without errors. Mike’s library is well written, and covers several important aspects of RF communication, including a dedicated protocol, CRC handling, robust encoding over the air, baud rate, and to top it off, an easy to use API.

Now, of course when using my Everflourish nothing happened. Which was a good sign; it did not interfere with the Virtual Wire transmission. That remote is using a different protocol to talk to the light switches. Luckily, it has already been reversed engineered and the source code is available from TellDus. It will probably take some time to get this working. Meanwhile, some pictures.