Sending binary over the air?
Jay Kickliter
Posts: 446
What do you guys do when you need to send binary data, not ascii, asynchronously from one place to another? Ie, say you have a relatively slow link and want to reduce the latency. Ascii is simpler since, you can have a known packet header, i.e.. @, and have the data you're sending separated by commas. But with binary any data you're sending could wind up being the same value as your delimiting character.
The simplest way I could think of is (in pseudocode):
packet header of known value, @,
(sending new packet)
send @
repeat 8
if next byte == $40 {"@"}, send $3F
else send next byte
send final byte (one bit for for every byte sent after header byte, if that bit is 1, then that byte needs to have one added to it since it is actually $40, not $3F)
send checksum byte (doesn't need to be complicated, just add all the bytes between header and checksum, then & it with $FF
I think it may work, but I'm sure there's far more elegant solutions out there. If anyone has done something similar is spin and same some source code to share that would be awesome.
Jay
PS, I apologize guys for always asking questions and not answering many. But as I learn more I'll try pitch in my share.
The simplest way I could think of is (in pseudocode):
packet header of known value, @,
(sending new packet)
send @
repeat 8
if next byte == $40 {"@"}, send $3F
else send next byte
send final byte (one bit for for every byte sent after header byte, if that bit is 1, then that byte needs to have one added to it since it is actually $40, not $3F)
send checksum byte (doesn't need to be complicated, just add all the bytes between header and checksum, then & it with $FF
I think it may work, but I'm sure there's far more elegant solutions out there. If anyone has done something similar is spin and same some source code to share that would be awesome.
Jay
PS, I apologize guys for always asking questions and not answering many. But as I learn more I'll try pitch in my share.
Comments
Leon
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Amateur radio callsign: G1HSM
Suzuki SV1000S motorcycle
www.geocities.com/SiliconValley/Haven/4824/ethernet.html
Amateur packet radio uses the same system.
If you only have a small amount of data I'd convert it to ASCII hex and use the standard ASCII control codes.
Leon
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Amateur radio callsign: G1HSM
Suzuki SV1000S motorcycle
Post Edited (Leon) : 12/13/2008 3:26:08 PM GMT
Once you've finished initializing the xBee or 9xtend, all you need is a length byte or word (sent one byte at a time) for the length of your data, followed by the data itself.
If the data is a fixed length, just send the data itself.
In transparent mode you can send any of the 256 possible bit configurations of an 8-bit byte, receiving it at the other end as data.
In nontransparent mode some of the bit configurations are interpreted at the receiving end as control characters (commands for the receiver to obey), not as part of the data.
You would use nontransparent mode to send, for example, ASCII files that contain only a subset of the 256 possible bytes.· Most text files, for example, wouldn't contain any $00 bytes or any of several other possible bytes.· On the other hand, data consisting of binary numbers might contain any byte at all.
One way of transmitting transparent data would be to encode everything as ASCII characters.· That is, when you have a byte $00 to transmit, you would send two ASCII "0" characters, $3030.·· If you had the ASCII character "0" to transmit, it is $30 and you would send it as "30", or $3330.··Each byte of the original data would be encoded as two ASCII characters, and your transmitted data need contain only the characters "0" through "9" and "A" through "F".· This works but is very inefficient, because you must transport two bytes over your link for each byte of the original data.
One better way is to start out in nontransparent mode and use an escape sequence to switch to transparent mode.· An escape sequence is a sequence of bytes, usually only two bytes, the first of which is the ASCII "escape" or ESC character, $1B (decimal 27).· The receiving system would watch for escape sequences in the data, and would always remember whether it was in transparent mode or in nontransparent mode.
When in nontransparent mode, the receiver would watch for ESC characters.· You might define a sequence, say ESC CR or $1B0D, that would switch to transparent mode.·
Once you've switched to transparent mode, the receiver·would consider anything it receives as data, not as control codes that it must obey.
Well, great, you say, but if everything is data and nothing is commands, how can I get back to nontransparent mode?· I need a way to do that, because I still need to tell the receiver such things as "this is the end of the data" and "start a new file" and stuff like that.
The solution, again, is ESC sequences.· Even in transparent mode you have the receiver watch for ESC characters $1B.· If an ESC character is present in the data, you will send two of them instead of one, ESC ESC or $1B1B, and the receiver will interpret that as a single ESC character that is part of the data.· Then you can use any other two-character data starting with ESC as a command code.·
For example, you might define:
ESC "A" or $1B41 in nontransparent mode means "switch to transparent mode".· Any ESC character not followed by an "A" might have some other meaning, like "end of transmission" or "the next two bytes are a checksum" or whatever you like.· You could devise other ESC sequences for stuff like that.
ESC ESC in transparent mode is a single ESC character that happens to occur in the data.
ESC "B" or $1B42 in transparent mode means "switch back to nontransparent mode because I need to send you some commands".
In transparent mode, the only time you'd send (or receive) an ESC character would be in one of the two sequences ESC ESC and ESC "B".··Any other ESC·would be an error.
Note that the sequences ESC "A" and ESC "B" that I mention above are nonstandard.· There exist some sort-of-standard ESC sequences, but I can't remember them at the moment, so I made up my own for this discussion.· Use of ESC ESC for a single ESC that's part of the data is standard, though.· You need only three escape sequences for transparency in any case, one for "enter transparent mode" and one for "leave transparent mode" and one for "here's an ESC character that's part of the·transparent data".
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
· -- Carl, nn5i@arrl.net
The way I do this, and I've done it a lot, is with escapes. I use the datalink escape (DLE) with a value of $10, but you can use any character you want for an escape. Commands always begin with DLE, followed by a different character to indicate which command it is, then followed by any data the command requires. Command sequences don't need to be terminated if the amount of data following each command is fixed for that command. You could also define a terminator, if you need one for variable-length data sequences, as just another command. When sending data, if you need to send the DLE character in the datastream, you send two of them in sequence. The receiver, when it sees two in a sequence like this, knows it's not a command and buffers only the first one.
There's a document here that shows an example of this protocol in action.
That's really all there is to it, assuming you're transmitting over a reliable data link. If the data link is unreliable, you can embed this simple protocol in another layer that includes CRCs, FEC (forward error correction), or the like.
-Phil
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
'Just a few PropSTICK Kit bare PCBs left!
I realize my original idea isn't necessarily the best. I think Phil's suggestion gets at what I was trying to understand. Given no other reference but a stream of bytes coming in, any of which can have any value, how can you tell the beginning of a frame. Although I am applying this to RF modems, I was trying to figure out how to do this genericly.
Before the last two commnts came in, I sat down and tried to code my first Idea. It's the first time I've used any bitwise math, so I was supprised that it worked. Here's what I came up with:
PUB Main | index serialDebug.start(DEBUG_RX, DEBUG_TX, 0, DEBUG_SPEED) serialTransponder.start(-1, TRANSPONDER_PIN, 0, DEBUG_SPEED) cognew(startTransponder, @transponderStack) startReceiver PUB startTransponder | index packetToTransmit[noparse][[/noparse]0] := "A" packetToTransmit := "B" packetToTransmit := "C" packetToTransmit := "D" packetToTransmit := "E" packetToTransmit := "@" packetToTransmit[noparse][[/noparse]6] := "F" packetToTransmit[noparse][[/noparse]7] := "B" repeat waitcnt(clkfreq / 10 + cnt) index~ txCheckbyte~ serialTransponder.tx(CONTROL_CHARACTER) repeat PACKET_SIZE if packetToTransmit[noparse][[/noparse]index] == CONTROL_CHARACTER serialTransponder.tx(CONTROL_CHARACTER + 1) txCheckbyte += (%1000_0000 >> index) else serialTransponder.tx(packetToTransmit[noparse][[/noparse]index]) index++ serialTransponder.tx(txCheckbyte) PUB startReceiver | index repeat serialDebug.tx(1) serialDebug.str(string("Raw bytes before transmitting: ")) index~ repeat PACKET_SIZE serialDebug.tx(packetToTransmit[noparse][[/noparse]index++]) serialDebug.tx(CR) repeat until serialDebug.rx == CONTROL_CHARACTER ' Wait until beginning of packet index~ rxCheckbyte~ repeat PACKET_SIZE ' Fill packetBuffer with 8 packet bytes packetBuffer[noparse][[/noparse]index++] := serialDebug.rx rxCheckbyte := serialDebug.rx serialDebug.str(string("Packet as transmitted: ")) serialDebug.tx(CONTROL_CHARACTER) index~ repeat PACKET_SIZE ' Display 8 uncorrected bytes in the packet, without the check byte serialDebug.tx(packetBuffer[noparse][[/noparse]index++]) serialDebug.tx(CR) serialDebug.str(string("Check byte bits, a '1' represents a '@': ")) serialDebug.bin(rxCheckbyte, 8) ' Display the individual bits from the check byte, MSB first serialDebug.tx(CR) index~ repeat PACKET_SIZE ' Change any $41 bytes back to $40, if that was what they were supposed to be if packetBuffer[noparse][[/noparse]index] == constant(CONTROL_CHARACTER + 1) AND ((rxCheckbyte << index) & %1000_0000) packetBuffer[noparse][[/noparse]index] -= 1 index++ serialDebug.str(string("Bytes after processing: ")) index~ repeat PACKET_SIZE serialDebug.tx(packetBuffer[noparse][[/noparse]index++])Here's the output:
As mentioned above sending the data as hex works, but reduces the bandwidth by half. There are several ways around that.
1 - Encode your data as 6 or 7 bits per byte and use the high bit to indicate commands. This gives you 3/4 or 7/8 of the bandwidth.
2 - Use the parity bit to indicate data/command, odd for one, even for the other. This gives full speed but may not work if your xmit/rcv module does error correction/detection.
3 - Use a 9 bit transmission if available. Haven't seen hardware that supports that lately but if you are bit banging the data it is possible. 11/12 bandwidth.
4 - As others mentioned, send data and commands in packets. Unless your application has very simple requirements for data and commands this is probably the best way to go. Lots of standard protocols to choose from, and most have documentation and sample code available. Look at Profibus, ASI bus etc.
-Phil
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
'Just a few PropSTICK Kit bare PCBs left!
In an earlier post, it was asked, "how do I identify the start of a frame?" -- to which I wonder why you want frames at all, and to which I reply that the transparent data stream is simply a sequence of bytes, in which you can identify any structure you impose (such as frames) the same way you'd identify it in any nontransparent data.
That is, frames, if you wish them, are inherent in the structure you impose on the data -- they don't inhere in the way the data·are transmitted.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
· -- Carl, nn5i@arrl.net