Uart serial data packets!

QuadrtrFlyr · 2012-11-08 09:28

Good afternoon folks!

I have a question for you guys that pertains to reading a UART Packet. I have done my research but still seem to be missing the big picture and hopefully you guys can lend me a helping hand! At the moment I am working with a particular sensor, a Pololu IMU (http://www.pololu.com/catalog/product/1255). It is communicating with my PC and eventually Propeller chip over UART. However I have had a difficult time understanding how to read packets sent over UART. My communication settings at the moment are 115200 Baud (can decrease if need be but Full Duplex Serial seems more than capable of this speed), 8 bit no parity stop bit 1. So I have run a couple of programs such as "Realterm" which is similar to hyperterminal that lets me view the data coming in. Looking into the sensors datasheet (http://www.pololu.com/file/0J442/UM6_datasheet.pdf) you can find the serial packet structure on page 21 and I have posted a picture of it below.

So the packet starts of with the character 's' 'n' 'p' and then you get the packet type byte, address, etc. Using the software bit term, I got the following output reading from the sensor using a mix of ascii and hex.

s73 n6E p70  C8 d64 [5B  8A  EE  8B  CA  C6 ,2C  FB  07  92 
s73 n6E p70  C8 d64 _5F  B9  F3  D5  D2  AA .2E  A1  07  A8 
s73 n6E p70  C8 d64 c63 U55  FB ~7E  DC \5C 030  D2  06  E8 
s73 n6E p70  C8 d64 e65  C9  04 d64  E5 ^5E 131  84  06  0B 
s73 n6E p70  C8 d64 g67  00  0C  D4  EB  04 030 :3A  05  1D 
s73 n6E p70  C8 d64 f66  A5  14  D7  F0 >3E 030 131  06  02 
s73 n6E p70  C8 d64 e65 a61  1F  10  F5  89 .2E  C8  05  E6 
s73 n6E p70  C8 d64 c63 .2E (28  F7  FB  19 ,2C  BE  06 +2B 
s73 n6E p70  C8 d64 \5C }7D 838  BB  03 a61 )29  E9  05  BF 
s73 n6E p70  C8 d64 U55  FE D44 (28  08  1E %  E5  05 l6C 
s73 n6E p70  C8 d64 Q51 j6A J4A  B8  0B  EF "22 s73  05  C9 
s73 n6E p70  C8 d64 N4E  D1 M4D y79  0E y79 !21  8C  05  96 
s73 n6E p70  C8 d64 L4C  B9 N4E  EA  12  15 !21 H48  05 J4A

So I am getting the s, n, and p values followed by the C8 hex value which is equivalent to 11001000 (a.k.a the Packet Type) but after that I have no idea what the rest of the hex/ascii mean when pertaining to the address, data bytes, or checksum because maybe I am reading them incorrectly?? I have tried googling and reading up on this topic but have not found much. If I would like to implement this serial read/write on my propeller how would the prop know when to read a byte vs a character vs an address vs a checkum??

Looking at the Full Duplex Serial code its seems to be that it can only read bytes? so how do I read the s,n,p packets etc.?

I really appreciate the help! I am stuck on this one...

MagIO2 · 2012-11-08 10:29

There are several ways to solve this. One solution would be a ring-buffer. As you can read in the specs there is a maximum limit of number of bytes for a single package. So you could have a ring-buffer which has the double size of this package size. This way you always have at least one package in the buffer.
You'd simply search for snp in this buffer. The next byte (PT or package type) tells you how much bytes will be in the package. (Have a look at the package type bit descriptions in the datasheet (page 21).
So, C8 means %11000 1000 it contains data (bit 7=%1), it is batch (bit 6=%1) and it has 8 bytes of data (bits 2-5= %00010 = 2*4)

The next byte is the address, then you have 8 data bytes (according to PT) and 2 bytes of checksum.

So, your code should first generate the checksum and compare it with the transmitted checksum. If it's ok then you can decode the data. If it's not ok you discard the package and try to find the next snp starting from the point where you found the last snp (because by chance the last snp found could also be data of the package)

Another way is to implement a state-machine. So, you'd simply input each byte read into the state-machine and decide which would be the next state. This way you don't have to store the bytes for snp and you can decode the stream while reading it.

Hope this helps, otherwise just let us know when you don't know what a ring-buffer or a state-machine is.

Duane Degn · 2012-11-08 10:41

As MagIO2 said, it looks like it's outputing 8 bytes starting at register $64 (quaternion estimate), each component (of four transmitted) is two bytes.
The hex $645B, $8AEE, $8BCA, $C62C $5B8A, $EE8B, $CAC6, $2CFB works out (using the formula in the datasheet) to the quaternion
0.78666297, -0.1500212017, -0.4574152818, 0.3865504895

Was the device moving? I'd thing the output of the same register (it looks like it's only outputting $64) would be close to the same value when the device is stationary (though as I look at the data again, I guess the values aren't flucuating a whole lot).

QuadrtrFlyr · 2012-11-08 11:51

First off I'd like to thank both of you for your responses! I feel as though the sun is shining for the first time in a long time.

On to some questions:

Looking at the package type, what does the "is batch" bit mean? Googling batch operation I found that it is "A routine performed by the computer without the need for any user intervention.." So I am pretty sure it means that I am only reading and doing no writing.

The address byte shown is quaterion estimate! which is exactly what I set the sensor to output when I set it up using the pololu's interface.

So, your code should first generate the checksum and compare it with the transmitted checksum. If it's ok then you can decode the data.
If it's not ok you discard the package and try to find the next snp starting from the point where you found the last snp (because by chance the last
snp found could also be data of the package)

Looking into the manual on page 55 I found the command UM6_BAD_CHECKSUM (0xFD), which means a packet with this address is transmitted by the UM6 when a packet with an invalid checksum is received by the sensor. It goes on to say on page 53 that this is not technically a command, instead a packet with this address is transmitted by the UM6 when it receives a packet with a bad checksum. This address should only be used by the UM6. So should I still check the checksum and compare it?
How would I generate a checksum that I could compare with the recieved checksum?

Another way is to implement a state-machine. So, you'd simply input each byte read into the state-machine and decide
which would be the next state. This way you don't have to store the bytes for snp and you can decode the stream while reading it.

What would you reccommend MagIO2? I am not too familiar with either of them but reading into it I feel like the circular buffer (ring-buffer) would be more stable.. could I pick your brain a little with regards to both methods.

Duane Degn:
Looking into the datasheet, it stated that the quaterion estimates are stored as 16-bit 2's complement integers. Could you explain a little bit more as too how you got the decimal interpretation from the HEX? The step right before calculating the quaternion = register_data* 0.0000335693. Also why did you include $64 in the data bytes since it is the address?

Many thank!!

Duane Degn · 2012-11-08 12:19

QuadrtrFlyr wrote: »

Duane Degn:
Looking into the datasheet, it stated that the quaterion estimates are stored as 16-bit 2's complement integers. Could you explain a little bit more as too how you got the decimal interpretation from the HEX? The step right before calculating the angle estimate = register_data* 0.0109863. Also why did you include $64 in the data bytes since it is the address?

Oops, I typed the address by accident. I don't think I used the address in the conversion though. I'll fix my earlier post.

To convert to 16-bit 2's complement, I used windows calculator program in "programmer" mode and entered the hex values. I had set the size to "word" since it's 16-bits. I then pasted the dec equivelent to the "Scientific" calculator and used the equation you quoted.

QuadrtrFlyr · 2012-11-08 12:37

Duane Degn wrote: »

Oops, I typed the address by accident. I don't think I used the address in the conversion though. I'll fix my earlier post.

To convert to 16-bit 2's complement, I used windows calculator program in "programmer" mode and entered the hex values. I had set the size to "word" since it's 16-bits. I then pasted the dec equivelent to the "Scientific" calculator and used the equation you quoted.

\

Thank you Duane, I thought it was a bit more involved but it is just a 16 bit hex value converted to decimal and then multiplied by the datasheets scalar. Also to answer your previous question, the sensor was stationary on the table, therefore the consistent values.

MagIO2 · 2012-11-08 12:40

I don't know what batch means in this context because I just read the part of the datasheet needed to answer your first questions! But you don't need to ask google, just read the datasheet! It should tell you what they mean with batch!

The checksum should be used in both directions! Whenever you send a command to the device it expects a checksum. If the checksum is wrong it will assume that there was a transmission error and it won't execute the command! And that's when it will return the bad checksum package.
When the device sends periodic data you should also use the checksum to verify that the transmission was error-free. So you have to calculate the checksum of the package and compare it with the 2 checksum bytes of the package. The way how to calculate the checksum will also be described in the datasheet.

Well ... software development means that you start easy and try to write modules which later on can be used in different contexts. If a module is well tested you can go on with the next level.
What does that mean in this context:
For the first time I guess you should start as easy as possible!
I think you can set the rate of messages to be sent. For the beginning I'd set it to something like a second and start with a fixed buffer. You simply read 15 bytes. A message per second gives you enough time to process the package! Then you can easily start writing the modules which
a) verify that the message starts with snd
b) decode the packet type
c) calculate the checksum
d) decode the data

Ring buffer is only needed if you receive the next packet while decoding the actual one. So, if that's true later on you can add this later.

State machine might be to hard for a beginner.

QuadrtrFlyr · 2012-11-08 13:29

Alright, the checksum portion makes sense. Upon looking into the datasheet, there seemed to be no description for calculating this value. However, I checked the Pololu forums and found that it is indeed missing from the manual but can be calculated with these instructions.
The checksum is computed by summing the each unsigned character in the packet and storing the result in an unsigned 16-bit integer

So as an example (from a different reading):
73 6E 70 C8 62 F4 CE EF CB ED DB 00 00 07 BF
Using windows calculator in programming mode I got
73+6E+70+C8+62+F4+CE+EF+CB+ED+DB+00+00 = 07 BF !!! GREAT!!

Ease of use and simplicity is key, I agree! Looking into the specs, the pololu IMU can only transmit at a minimum of 20Hz. However I will start by reading only 1 package and letting the others drop and then progress to reading 20. I am assuming it won't be too difficult to jump to 20 or even 300 packages a second with the Full Duplex Serial Library. Excellent advice.

Looking into this specific Euler angle reading (pg.47 in datasheet - phi, theta, and psi ranging from -90 to 90 degrees): 73 6E 70 C8 62 F4 CE EF CB ED DB 00 00 07 BF, I was wondering if I was doing the calculation right because I am getting something strange. So the address of this reading is 0x62 where the angle estimates are stored as 16-bit 2's complement integers. However when I convert the HEX and calculate the first 2 bytes decimal value (F4 CE) I get 62670 which is then multiplied by the scalar given: angle estimate = register_data* 0.0109863
So angle estimate = 688.511. This is definitely not in the -90 to 90 degree range. Any ideas?

Mike G · 2012-11-08 13:43

2's complement means invert all the bits and add 1. It allows us to represent negative numbers in binary.
http://en.wikipedia.org/wiki/Two%27s_complement

MagIO2 · 2012-11-08 13:45

In win calculator you have to set it to hex and word-size, then you enter $F4CE and switch to decimal. Then you'll see, that the real value actually is -2866 which is somewhat around -30degrees.

QuadrtrFlyr · 2012-11-08 14:26

Excellent! -30 degrees was approximately the angle I was holding the sensor at. Once again, many thanks for all the help. Everything seems to have clicked and now I will be walking around with an extra skip to my step

Time to code..

Duane Degn · 2012-11-08 19:32

I've been thinking a bit about how to read in the 16-bit number while still maintaining the sign.

I don't think Spin handles negative numbers well with any variable type except longs so if you read the two bytes of a quaternion into a long variable (call it "longVariable" for now), storing a 16-bit two's compliment number in a 32-bit variable will change negative numbers to a positive number since the upper 16-bits will be incorrect (I think you saw this in your earlier conversion).

I think this little bit of code would fix the problem.

longVariable <<= 16
  longVariable ~>= 16

By shifting back to the right with "bitwise shift arithmetic right" (see page 158 of Prop Manual v1.2) after using a normal shift left, the negative value of the 16-bit number is preserved when changed to a 32-bit number. You could then use "longVariable" in normal calculations.

kuroneko · 2012-11-08 19:36

Alternatively use ~~longVariable instead (sign-extend bit 15).

Duane Degn · 2012-11-09 07:20

kuroneko wrote: »

Alternatively use ~~longVariable instead (sign-extend bit 15).

That's good to know.

Thanks.

QuadrtrFlyr · 2012-11-11 16:35

Excellent point, I will incorporate that into my code and see what results I get.

Regards,

Uart serial data packets!

Comments