Need help with parsing serial data

archerbrad700 · 2011-10-27 12:40

Hello again,

First off I have spend hours searching the forum looking but to no avail. I no experience with serial comms and am having trouble trying to parse data sent to my propeller. The data being sent to the prop are strings with an identifier and value like this for example (the values are always integers):

...
X112
Y2
S0
S1
...

What I am trying to do is sort out the data by the identifier and then store the value after it in a variable. For example, I have a program that will loop and increment or decrement a counter until the disparity between the values is eliminated. If the variable desiredX := 100 and Xcounter := 0 it will loop and increment XCounter until it is equal to desiredX. SO, the question is how do I take the string X112 from the serial port and store it as desiredX while differentiating it from data with different identifiers?

I realize this is a really basic question, one that has probably been asked a hundred times before, but I've gotten frustrated looking for the answer. Any help is greatly appreciated such as a link to a thread that has already discussed this, an article, example code, anything...

Thanks, Brad

MagIO2 · 2011-10-27 13:03

Well ... you have to find the rules of this communication (also called protocol) and then you can transfere those rules into a parsing program.
And these rules are simply given by the sender in this case!

So, for example there are protocols which send data via hex string, which has the advantage of fixed size. Some protocols send start, end and separator characters and so on.

For the example you gave us it looks like the different entities are separated by a newline, is this true? So is it in fact
...X112<NL>Y2<NL>S0<NL>S1<NL>...

Well then it's easy to brake down the problem in several steps:
1. read character into a buffer
2. if the character is not a <NL> go to 1.
3. if the character IS a <NL> the buffer contains an entity that can be parsed
3a. buffer[1] and following contain the number, so use a string library to convert the string representation of the number into a digital value (something like value := str.strtolong( @buffer[1] ) )
3b. buffer[0] is the character X, Y or S (maybe others you did not mention so far). So, based on buffer[0] you can decide what to do using a case statement.

case buffer[0]
'X': desiredX := value
'Y': desiredY := value
...

3c. clear the buffer and go back to 1.

Mike Green · 2011-10-27 13:10

Your data always starts with a single character that you use as an identifier, so you start by getting that single character and saving it to a variable for later. You use the 'get a character' routine in FullDuplexSerial for that. You can then use the decimal value input routine in FullDuplexSerialPlus which reads in all the characters it sees up to and including a carriage return (13) character, then converts the input to a numeric value. Now you have the identifier character and the numeric value. You then use a CASE statement using the identifier character variable. You can do whatever you need to for each known identifier value and use the OTHER: case to catch any invalid identifiers.

Sorry for not having the specific method names in FullDuplexSerial and FullDuplexSerialPlus, but I'm not where I have access to my source libraries. It should be obvious from the source.

archerbrad700 · 2011-10-27 13:14

@MagIO2 >Yes, thank you, you understood perfectly. That is very helpful.

It so happens that I am the one who is creating the protocol so I really could do whatever, this just seemed like an easy way to go. If there is a protocol that is even more simple I'm all for it.

What object do you recommend I use? FullDuplexSerial, FullDuplexSerialPlus, Simple_Serial, ... I'm looking for something that has all the features I need (which I'm not even sure of) and is WELL DOCUMENTED!

@Mike > Thank you, that is very helpful too.

archerbrad700 · 2011-10-27 13:24

For some reason, I am not finding FullDuplexSerialPlus on OBEX... Help?

MagIO2 · 2011-10-27 13:32

If you create the protocol by yourself you should ask yourself which needs do you have?

Is it important to convert it to strings at all? This might help when debugging, but it always needs more bytes to transfer the same amount of information.

If you can send digital data instead you not only save bandwidth. You also save the conversion efford. But you have other problems then. How can you separate values if the separator could also be a valid number? ;o)

But maybe the amount of data you want to send per second is so high that you have to get the best out of the available bandwidth.

I think simple serial is written in SPIN and is limited in the bitrate. And I think it also does not start a COG. So, it is helpfull when you can't efford a COG and you can live with not watching the port permanently.

FullDuplexSerial which met all my needs so far. There is a version which also allows to have up to 4 serial ports with only 1 COG. The FullDuplexSerialPlus I don't know, but as far as I remember discussions it's main point is to add some usefull functions like conversion functions ...

archerbrad700 · 2011-10-27 13:39

Bandwidth isn't going to be an issue, data is sent only occasionally and in small quantities, so I think I might stick with sending strings for now unless I can't find that string-to-long method Mike mentioned.

Can anyone show me source code that does what I'm asking? I think that's how I learn this stuff best....

Unsoundcode · 2011-10-27 14:56

Hi, you can get Extended Full Duplex Serial in the Obex at this link http://obex.parallax.com/objects/31/

Here is an example of what has been suggested

Pub main   | Identifier,var_1,var_2,var_3
 com.start(31,30,0,9600)

  repeat
     
        Identifier:=com.rx                       
        case  Identifier 
                          
          "X" : var_1:=com.rxdec
               'do something with var_1          
                           
          "Y" : var_2:=com.rxdec
               'do something with var_2           

          "S" : var_3:=com.rxdec
               'do something with var_3

In the start method of Extended_FDSerial there is a variable called Delimiter to which you can assign a string terminator of your choosing if you wish.

Jeff T.

Need help with parsing serial data

Comments