Propeller networking
Dr_Acula
Posts: 5,484
Ok, finally pulling together a project that has been in the works for years! I want to network Propeller chips together, both via wires, and also by radio, and possibly via the internet.
The ultimate goal is energy independence for the home. I put up lots of solar panels and they are working fine and the energy they make roughly balances the energy I use (around 70kWh a day) so the goal is not crazy. But the utility company doesn't pay as much back as they pay to use the energy. I've looked at batteries but the costs are too high.
So part of the answer is smart energy management. For instance, I can just put the pool pump on a timer. Or I can be smarter about it and turn it on when the sun is shining. Or I can be smarter again and only turn it on when more than a certain amount of energy is being exported, and skip running the pump on really cloudy days.
This needs some data communication. The pool pump is not near the meter box, so they have to be talking to each other somehow. And then I might want to use the excess energy to heat some water, or run a pump and every time something is added, it needs to know if any/all the other things are on or off. So this is a network that needs to grow in an organic way.
What I now have is a 3 phase kWh meter producing pulses, reading these with an Arduino and transmitting the data to another location via uart transceiver modules. It works fine for just that one link, but it is harder to grow and add more nodes. I've tried single wire multi-drop before but sooner or later data packets start clashing and the network clogs up.
The answer is a Propeller serial router!
At the simplest level, take a RS232 signal and split it into two. That just is two wires - easy. But take the data coming back from those two wires - if that data arrives at the same time there is a problem. The Arduino can't handle this - they suggest you poll each wire in turn but it is a single microprocessor and this really works much better with a multi core processor like the Propeller. Collect data from each source and splice them together.
This then needs consideration of data packets at the same time as hardware. It is no good having two serial lines each with data coming in and splicing each byte together - that won't make sense. And the old-skool answer of hardware lines for handshaking doesn't work well with wireless links. So we need to look at wrapping data up in a simple packet and then collecting packets in a buffer and sending them through one after the other. A perfect job for the propeller.
The world of packet protocols is confusing. Many website discussions end up with the suggestion to 'roll your own'. So in the next post I'll talk about some ideas. Attached is the schematic for a propeller router which has gone off to the PCB makers today.
The ultimate goal is energy independence for the home. I put up lots of solar panels and they are working fine and the energy they make roughly balances the energy I use (around 70kWh a day) so the goal is not crazy. But the utility company doesn't pay as much back as they pay to use the energy. I've looked at batteries but the costs are too high.
So part of the answer is smart energy management. For instance, I can just put the pool pump on a timer. Or I can be smarter about it and turn it on when the sun is shining. Or I can be smarter again and only turn it on when more than a certain amount of energy is being exported, and skip running the pump on really cloudy days.
This needs some data communication. The pool pump is not near the meter box, so they have to be talking to each other somehow. And then I might want to use the excess energy to heat some water, or run a pump and every time something is added, it needs to know if any/all the other things are on or off. So this is a network that needs to grow in an organic way.
What I now have is a 3 phase kWh meter producing pulses, reading these with an Arduino and transmitting the data to another location via uart transceiver modules. It works fine for just that one link, but it is harder to grow and add more nodes. I've tried single wire multi-drop before but sooner or later data packets start clashing and the network clogs up.
The answer is a Propeller serial router!
At the simplest level, take a RS232 signal and split it into two. That just is two wires - easy. But take the data coming back from those two wires - if that data arrives at the same time there is a problem. The Arduino can't handle this - they suggest you poll each wire in turn but it is a single microprocessor and this really works much better with a multi core processor like the Propeller. Collect data from each source and splice them together.
This then needs consideration of data packets at the same time as hardware. It is no good having two serial lines each with data coming in and splicing each byte together - that won't make sense. And the old-skool answer of hardware lines for handshaking doesn't work well with wireless links. So we need to look at wrapping data up in a simple packet and then collecting packets in a buffer and sending them through one after the other. A perfect job for the propeller.
The world of packet protocols is confusing. Many website discussions end up with the suggestion to 'roll your own'. So in the next post I'll talk about some ideas. Attached is the schematic for a propeller router which has gone off to the PCB makers today.
Comments
Really??? RS232??????????? Why didn't you just ask before Drac!!!
First off, there is no magic and perhaps even less usefulness having RS232 as you are at least better off with stock standard logic drive at 5V or so. The much lower impedance will run longer, faster, better than any RS232 anyway. RS232 has been too hyped for too long and it's all wrong.
Second, you are far better off running RS485 multidrop despite the fact that you may think you need some high level packet ack/nak/retry protocol, in fact I have implemented simple little techniques for this so your app just sends what it sends when it wants and everything gets to where it should be going.
Third, those ESP8266 chips are very nice and I am getting quantities of the bare chips from the factory for my own modules, so we will see how that goes.
BTW, I have a bank of 8 x 6V @160AH sub-station batteries to call upon at any time, that's a lot of power 48V @160AH could cause a rather large bang as that would peak at over 27MW if discharged in one second!!!
Some common themes gleaned from various websites. Keep the packets short. Use unique characters for start/end. Have a simple checksum. Wrap up the data with an 'envelope' which has the destination address and who sent the data.
So - here is what I am thinking. One byte for the location - eg pool shed. One byte for the node in that location. All data sent as hex (half the speed as sending as raw binary as two hex digits per byte, but then can use unique start/end characters. Base64 is another option)
Simple packet protocol
Start byte. Ascii 01 SOH as this is 'start of heading'
Source location 0-255 sent as two hex bytes.
Source node 0-255 sent as two hex bytes
Destination location 0-255 sent as two hex bytes
Destination node 0-255 sent as two hex bytes
Start data. Ascii 02 STX 'start of text'
Data bytes - each binary byte converted to 2 hex bytes, so ABC will be 414243. Can be no data at all. Or Ascii 06 ACK
End text. Ascii 03 ETX
Checksum. Two hex bytes eg A30F which is a simple sum of all the data in the packet
End of transmission. Ascii 04 EOT.
So the smallest packet with no data is 14 bytes. The data part of the packet can be as short or as long as needed - the transmitting and receiving nodes can work out their own protocols with this as the aim of this layer is to get the data there without errors, and give a path to send back a reply.
Nodes.
A node might sense an analog value, or turn on a led or a relay. Or it might be a robot scurrying around on the floor. Node addresses are fixed.
Local wired network.
Any signal injected into the RS232 board above will make its way to all other wired parts of the network, which would usually be close to each other - within a few metres. Baud rate is 9600 as this seems to be the fastest for a lot of radio links. The propeller makes sure no data packets clash.
Wireless links.
A simple setup would be two sheds - each has its own wired RS232 network, and they are linked by radio. So they might be location 01 and location 02. A simple solution is to send all packets via wireless as well, but many packets might be local, and lots of packets on a radio link clogs up the airwaves. So one can think of a packet filter. The hardware is the same as the propeller board above and it listens to all packets on the local wired network for location 01, but only sends on packets via wireless if they are for location 02. Packets coming back from 02 are not filtered.
For multiple wireless links, each radio uses a different frequency, and there are modules available eg in the 433Mhz band where you can program pairs of modules in up to 100 channels or more. For a bunch of robots in the house each would use a different channel.
Overview.
So with some simple components, 'real world nodes', serial router, wide area network filters and wireless links, it should be possible to build a flexible low cost network. Of course, the other way to do things is via wifi and indeed some of what I have built also involves nodes talking to wifi repeaters. Running wifi for a few weeks though, I'm not sure it is quite as reliable as good old RS232.
Has anyone built anything like this?
Thoughts and feedback would be most appreciated
RS485 and multidrop and TTL- yes I agree and maybe all of them are valid and one can think about replicating the same router with each of these? Certainly for one network in a shed TTL single wire with a 10k pullup or pulldown would probably do fine.
My wifi chips haven't arrived yet but I'm ready to go with testing these - I have antennas up with both 433Mhz serial and 2.4Ghz wifi repeaters so can do side by side reliability testing.
That is a nice battery bank. Battery banks may still be the answer too - who knows where the utilities will go next but I think yet another state is about to sell them off with a 99 year lease and the general trend seems to be they will charge more for what you buy and pay less for solar. (46c to buy, and down to 16c soon to sell). I think prices in other countries are less than this - less to buy, but then again many places don't pay anything to buy back solar power.
It's not even quite midnight here yet, the night is young and there are still many more hours to burn
Instead of slow pullups on single wire networks you could just simply put a 220R resistor in series with each node so that if they try to drive the line it won't damage anything. My bus method allows for a very short "hey Bob" which if not acknowledged by Bob is taken as a possible collision and backs off for a short random period before calling out to Bob again. If someone else calls out in the meantime then the holdoff period is extended until it hears a "goodbye" or times out. Once a connection is established then you just talk normally with Bob as if it were a full duplex connection. But there is no need for ASCII'fying the data, I just format the word as 9-bits as in 8051 style and that extra bit says it is either data or it's a special character such as an address or network control.
BTW, for any distance in a wired network it is just as easy to use RS485 which you can run for hundreds of metres at least, and I do. But never use the ancient 75176 chips or the MAX parts as there are cheaper 1/8 load fail-safe devices available that just work properly.
Think megabaud or at least 57.6k which is still fairly leisurely. You don't want the line being hogged for too long so get the conversation over and done with as quick as possible.
So what is the part number of the "cheaper 1/8 load fail-safe devices available that just work properly"?
Also there must be a pseudo standard simple packet format. Every body comes up with there own.
Were did you get you get those huge batteries?
cheers,
rich
I happen to prefer the RS-422 in full duplex to the restrictions of a RS-485 multi-drop.
And if you want to go wireless, you can use mini-routers to a direct TTL asynchronous serial with your existing code. That is what Erco, Heater and I have been hacking the WR703 and MR3020 mini-routers for. If you have a notebook or laptop computer with built-in wifi, it becomes a terminal entry and control point for your network with the wifi security already provided. And it can even download binaries and load them to the Propeller that is acting as youre gateway.
There are lots of good options here.. great fun.
-Phil
I started a project similar to this with a friend last year, but the project is idle for now... I wanted to make a really cheap module that could be used in sensor networks and in robotic/machine applications. We called it a "mote" and it is a little module that only contain an mcu (Propeller, PIC, Atmega, Cortex M3...) and also an RS485 transceiver. The goal of the module was to be able to push data and get data from the internet, be able to send SMS and emails, store data in a NoSQL database etc... The network would require a Raspberry PI as the master on the RS485 bus to give access to the internet. I was successfully able to make this network with 2 props and a Raspberry. Most of my code is on GitHub https://github.com/speccy88/Mote
I made the Raspberry Pi code in Python and the cool thing is that with your "mote" you can call Python function remotely and get result value just by sending the name of the function to the master with the communication library.
I also made a PyQt program to see whats happening in the mote network in real time and send data to the mote you want, and you can run this application anywhere, just give it the IP address of your raspberry PI.
I used the 75176 for my transceivers, I dont understand why everyone hate them, they are dirt cheap and just works great. You can get 10 of them for 2$ on ebay.
For the communication protocol, it is based a lot on the content of this page : http://www.bdmicro.com/code/robin/
An example of what I wanted to do with it is to automate some part of my home and make a simple security system.
A simple mote that only turn a light ON/OFF or read a sensor value (and may need an ADC) could work with a small low pin count mcu. More advanced stuff could done with a Prop mote. The goal of this was that all these devices could talk to each other with a common communication protocol.
In the picture below is a snapshot of the mote debugger program open with one connected mote, I send one fetch command (@) to get the value of cnt in the Propeller and I send a put command (!) to force the Prop to send an SMS to my phone.
-Phil
Yes, and even in full-duplex connections the communication is effectively half-duplex in nature due to the command/response method anyway. It is very easy to have a single master as that is very common but the advantage of a network that just talks is that all other nodes can eavesdrop on the conversation too and take appropriate action automatically. For instance a temp sensor may broadcast an over-temperature alarm which a shut-off valve node reacts to immediately, not being dependant upon a master which may be in the process of booting even!. So full-duplex or single master are only as good as the master which is also fine for domestic or non-critical systems.
BTW, if I run RS485 it is just as easy to use 4 core phone cable with one pair for the 485 and the other pair with bulk DC (12..24V) so that each node can be powered from the line easily. A little while ago I ran one such line from my place down to onto the sports ovals just to check it out, I had my little Pixie board which directly drives a VGA monitor and keypad. So even though you get a bit of voltage drop it doesn't matter if you have enough headroom and especially if you use switching regs.
For wired networks, I looked at using the CAN protocol (or subset) and the CAN driver chips (MCP2561 or older MCP2551) that work with failsafe. You require 4 wires, 2 for comms and 2 for power.
Just a word of warning about telephone cables...
Some of the cables are not solid and have significant resistive losses which may cause problems depending on the length. We had problems with these on modems even with short lengths (modems are indeed a different issue here). Just be aware of this for really long lengths.
Keep us informed with your progress Drac. This work really interests me for IoT devices and my boat electronics.
I've used a few different part numbers in various products but the reason I go for full fail-safe is that it gets around this problem in networks to do with biasing and termination. On long lines when no drivers are active the lines can pick up false signals or appear to be in an active break state to the receivers so typically bias resistors are used to pull the line back into an inactive state, but these can also cause problems especially if various nodes are all trying to do the same thing. Also if the line is over-terminated or shorted then your standard RS485 chip thinks this is an active break state too! The full fail-safe chips aren't affected by a floating or shorted line so the network is more reliable. Also they are 1/8 load too so you are not limited to 32 devices per line but you can have up to 256 (not to be confused with addressing).
So a couple of the part numbers for slew rate limited (115k baud) and cheaper are:
SN65HVD3082EDR (less than $2).
SP3082
These are all pin compatible with standard MAX485 type chips.
Faster chips instead of 3082 are simply 3085(1Mbit), 3088 (20Mbit) etc
About the phone cable, yes I'm not talking about the flexible stuff as that is not usually twisted pair anyway, just the good old Telecom grade solid copper of course and coupled with higher voltage DC supply and switch-mode regs it works a treat over long distances.
Are you making up your own ESP6288 boards from chips?
Altogether: a lot of work for years? or so?
ErNa
Radio links - some of mine are short and a 1/4 wave omni is fine. One link is longer - 500m, over a hill, through some trees and not quite line of sight. I think though that I can get it working with APC220 modules, up to 20mw programmable, multiple channels, -113db receiver sensitivity and 10 element yagi antennas at each end. But that is a bit of a specialist application - not many people would need a radio link like that. However, just for fun, I'd like to have a packet protocol that could let me log into a Z80 emulation computer running CP/M on the Propeller chip. I think it needs a small spin program to wrap and unwrap packets - eg send a data packet with a source and destination node, unwrap the data and send that to the CP/M machine as if it were typed in via a keyboard. Wait till bytes stop coming back, buffer them, wrap them up in a packet (simply swap the source and destination address to 'reply to sender', and send it back. So I think that simple packet protocol above should be enough for both querying a node for an analog value, turning a relay on, and also talk CP/M, so conceivably you could reprogram a node remotely, even recompile a program remotely. Old-skool C compilers for CP/M are not quite modern C syntax but they are close enough.
Somewhere in the link there needs to be an interface to the internet. This is where I have run into problems with the standard Arduino ethernet shields - they seem to crash after running for three days - maybe they overheat or maybe a buffer fills up or something. There are software patches on the internet, but it is not entirely clear they fix the problems, and certainly the standard software examples that come with the Arduino IDE are not reliable enough. The new wifi modules might be a way around that. Or use a Raspberry Pi. Or even a small netbook computer as the interface to the internet.
@FredBlais - checking your code now...
@ErNa, re solar power, I think solar panels charging batteries then drawing the power back has been around a long time. But the mass production of grid tie inverters has brought the price right down (10c a watt, or less). So if the grid is available, what I am doing is getting the solar power straight into the AC grid. That uses the MPPT tracking in the inverters, and so no power is wasted. Then the aim is to draw it back out of the grid and balance loads to match the solar input. The net result is similar to batteries, except you don't need the batteries. I guess much depends on what the utility is paying for energy you sell back.
I am trying to avoid mesh radio networks - they can get very complicated. So in simple terms - every physical location, eg a shed, has wired connections within it (RS232, 485, TTL, whatever). And links between distant locations is wireless, and there is a central hub which knows what each remote location address is, so it only sends packets out via wireless if they are for that remote location. I've become lost before in the complexities of mesh, and so I think this simpler concept is easier to grasp (for me at least!).
The other option I guess is to have everything talking to the internet, and upload and download to a site like Xively. Only trouble with that - if Xively goes offline, or starts charging too much, down goes the entire system.
Did you find something interesting? If there is some interest, I would like to continue with this project: add code, make it more robust, add examples and documentation.
scriptable WEBSERVER to it again, that worked with V2.3.
But Peter's IoT5500 module will be much smaller and a great Internet gateway to the local sensor/actor NW.
Tachyon based of course as well.
Peter,
I imagine that somehow, even with switch mode power supplies, you would connect a "common" between the two distant locations. Most (all ?) 485 devices, although differential on their signalling pairs, have a fairly low common mode voltage limit. Over longer distances, and especially if both ends have local ground connections, the ground potential between them can well be sufficient to blow the chips. In power grid substations hundreds or even thousands of volts during faults are not uncommon. So I always include a common line as part of my physical comm channel. It is good practice, even for short runs.
Cheers,
Peter (pjv)