How to detect when a serial byte begins?

Erlend

I want to begin listening at a running asychronious transmission (GPS NMEA) at any time, then return as soon as a specific valid NMEA sentence is detected and decoded. So, when I write the code for a simple RX I begin wondering; how can the code 'know' when a byte transmission begins - the start bit looks like any other bit. I have not tested my code yet, but if it just works, I still need to know how.

     
PRI RxB(rxPin) | t, rxByte, isThere 

   isThere:= FALSE
   DIRA[rxPin]:= 0
   REPEAT 10000                                                                                  'Give it a fair amount of tries/time
     IF INA[rxPin]== 1
        isThere:= TRUE                                                                           'Test for bit input
        QUIT
   IF isThere                                                                                    'Quit trying when input detected
     t:= CNT + bitDur/2                                                                          'Set synch point half a bit duration late
     REPEAT 8                                                                                    'to miss the start bit, and then read 8 data bits
       WAITCNT(t += bitDur)                                                                      'by sampling each bit duration time interval
       rxByte:= INA[rxPin]<< 7 | rxByte>> 1                                                      'keep reading and setting latest bit as bit 7, OR'ing onto previous bits read
     WAITCNT(t += bitDur)                                                                        'let the stop bit pass
     RETURN rxByte & $FF                                                                         'allow only 8 lowest bits
   ELSE
     RETURN FALSE

Erlend

tomcrawford

The start bit is unique in that it is first.

Heater.

The start bit does not look like any other bit. The start bit is a LOW. Any other bit can be a LOW or a HIGH.

The idle state of the line is HIGH. So when the line goes LOW then you know you have a start bit.

Then, after clocking in 8 data bits one expects one or two stop bits. Which are HIGH. And the process begins again.

Having said all that. I do sometimes wonder how things get in sync when data is arriving continuously, back, to back, and the receiver happens to start listening halfway through the data.

Erlend

Heater. wrote: »

Having said all that. I do sometimes wonder how things get in sync when data is arriving continuously, back, to back, and the receiver happens to start listening halfway through the data.

Exactly! Mustn't there be some code to wait for a long idle (High) to signify a byte could be next? What happens if the first thing sensed is a High-Low transition in the middle of a byte transfer - who's to know if that is not and Idle-Start transition?

Erlend
Btw, Happy Christmas to the Forum!

localroger

If you walk in on a continuous transmission that never pauses, there is indeed an issue figuring out the start bit; ideally you wait for nine bits of logic zero, which means the line is idle, then the next logic one bit you see is a start bit. But if you have a device that transmits nonstop without ever pausing you never see such an idle period and there isn't really any unique indicator that any particular bit is a start bit and not a one in the middle of a byte. The usual solution to that was, and I am not kidding, to turn the transmitting device off and back on while the receiving device is listening.

Heater.

That's what I thought. Thing is in all these decades of using serial interfaces I have never known them to not sync on characters and require a restart of anything.

I can imagine that a continuous stream of data with one stop bit looks like:

LOW, D0, D1, D2, D3, D4, D5, D6, D7, HIGH, LOW, D0, D1, D2, D3, D4, D5, D6, D7, HIGH.....

In which case I can imagine any device listening to that being able detect the character frames. Even easier with two stop bits.

If you do happen to start in the middle, say at D3, D4, which happen to be HIGH, LOW, you will attempt to receive an incorrect character. But then the stop bit will likely be wrong. So you know to try again.

Do good old fashioned real UART chips do that?

Of course, often there is a higher level protocol working on the line. Which enforces request/response, uses NAK/ACK and CRC etc. So all the problems get taken care of.

Duane Degn

You might want to take a look at this thread.

Tracy Allen kindly showed me the various parts of an asynchronous transmission.

I don't know if you're aware many (most?) serial objects written in Spin/PASM for the Propeller use two stop bits.

Erlend

The code I posted above is based on one I lifted from the Simple_serial object, so should be tested and tried. I only replaced the waitpeq with a repeat loop to avoid lock-up. But I cannot see how it could be able to synchronize/ recover from the confusion of a mid transmission start?

Erlend

tomcrawford

So if you come in in the middle you will get a garbage sentence that does not begin with "$GPS". You throw that away and begin looking for the next start bit. All GPS devices I know of pause between sentences.

Edit: Actually, I parse the sentence as it comes in. So that means throwing away everything, one byte at a time, until I see the "$". Endedit

Erlend

Guess that is what was mentioned above - that the trouble will be taken care of at a higher level [of the protocol]. But I still do not see how - what prevents this 1+8+1 loop reading the bits from continue reading 10 signals that do not begin the correct place (from the start bit). Should there not be e.g. a detection of a long idle that forces the loop to start from count 1?

Erlend

tomcrawford

It isn't a simple 1+8+1 loop. It is a linear process that captures 8 bits after first detecting a start bit. It has to be called from a loop that repeatedly rejects garbage until it finds something it likes.

BTW, I'm pretty sure the first test should look for a "0" instead of "1".

kwinn

In 40+ years of working with equipment that uses async serial to communicate I have never seen any that transmits continuously. Almost all comms are 2 way using either hardware (RTS/CTS, DTR/DSR) or software (ACK/NAK), and may use a protocol with start and end bytes as well as parity and/or CRC checks.

Continuous async transmission without pauses would range from extremely rare to non-existent, and if it did exist would probably have some form of unique start and/or stop sequence such as the "GP" characters in a NEMA sentence.

Phil Pilgrim (PhiPi)

Synchronizing to a continuous asynchronous data stream is just a matter of statistics. If you start in the middle of a data byte, chances are good that you'll quickly encounter a framing error (zero stop bit) and have to start over from that point. Eventually, things will slide into place with consistent "zero" start bits and "one" stop bits. As pointed out above, this happens more quickly, the more stop bits that get added to the protocol.

-Phil

Erlend

tomcrawford wrote: »

It isn't a simple 1+8+1 loop. It is a linear process that captures 8 bits after first detecting a start bit. It has to be called from a loop that repeatedly rejects garbage until it finds something it likes.

BTW, I'm pretty sure the first test should look for a "0" instead of "1".

It should. When the signalling is non-inverted, anyway. Thanks, now fixed.

Erlend

Erlend

kwinn wrote: »

In 40+ years of working with equipment that uses async serial to communicate I have never seen any that transmits continuously. Almost all comms are 2 way using either hardware (RTS/CTS, DTR/DSR) or software (ACK/NAK), and may use a protocol with start and end bytes as well as parity and/or CRC checks.

Continuous async transmission without pauses would range from extremely rare to non-existent, and if it did exist would probably have some form of unique start and/or stop sequence such as the "GP" characters in a NEMA sentence.

I am looking forward to see what the GPS is ouputting. So far I have just hooked it up to PST, but I need to hook up the scope to see the idle periods. This is becoming interesting.

Erlend

Erlend

Phil Pilgrim (PhiPi) wrote: »

Synchronizing to a continuous asynchronous data stream is just a matter of statistics. If you start in the middle of a data byte, chances are good that you'll quickly encounter a framing error (zero stop bit) and have to start over from that point. Eventually, things will slide into place with consistent "zero" start bits and "one" stop bits. As pointed out above, this happens more quickly, the more stop bits that get added to the protocol.

-Phil

But I still fail to see how the Simple_serial code does this?

Erlend

jmg

Erlend wrote: »

But I still fail to see how the Simple_serial code does this?

It happens naturally from the general UART timing nature of 9-10 samples after start edge.
Any pauses or extra stop bits, shift the possible start bit, and eventually things sync-up.

FWIR, GPS NMEA does have gaps - you can check with the string length and repeat time.

xanadu

After you pour the milk and pickup the spoon...

Couldn't resist

Heater.

Strangely enough I have recently been dealing with a device that spits out data at 115200 baud, continuously.

I just listen on /dev/ttywhatever, and all works well.

jones

tomcrawford wrote: »

So if you come in in the middle you will get a garbage sentence that does not begin with "$GPS". You throw that away and begin looking for the next start bit. All GPS devices I know of pause between sentences.

Edit: Actually, I parse the sentence as it comes in. So that means throwing away everything, one byte at a time, until I see the "$". Endedit

I do the same thing with no issues. The sequence starts at the beginning of each second (sometimes more often, like 100 ms, but it's always a fixed interval), and since some of the strings are variable length, there has to be a pause. If you miss what you're looking for, you have to wait for the next sequence to start.

localroger

In 40+ years of working with equipment that uses async serial to communicate I have never seen any that transmits continuously.

The Toledo Model 8132 scale indicator, the second ever in the industry with a microprocessor at its heart, was infamous for this. Toledo uses the same protocol to this day but all their more modern indicators pause between frames.

kwinn

localroger wrote: »

In 40+ years of working with equipment that uses async serial to communicate I have never seen any that transmits continuously.

The Toledo Model 8132 scale indicator, the second ever in the industry with a microprocessor at its heart, was infamous for this. Toledo uses the same protocol to this day but all their more modern indicators pause between frames.

LOL, I knew there would be at least one. There are always exceptions to the norm. That's why I rarely say never.

localroger

I've seen a few older devices transmit continuously. In those days it was common for an unbuffered UART to generate an interrupt when it was ready for the next byte to send. In assembly language, the easiest thing to do when you get to the end of the buffer is check for an update, copy it into the send buffer if there is (you don't want other code changing the send buffer in the middle of a transmission), and reset the pointer to the beginning of the send buffer regardless. It's very program code efficient, which is a priority when you're trying to cram the firmware into a 2K EPROM, but you get a continuous blast of data without pauses.

Phil Pilgrim (PhiPi)

Some UARTS have the ability to send 1½ stop bits. Theoretically, this would make it possible to sync very quickly onto a continuous data stream, since every start bit begins in the middle of the previous character's bit boundaries. Given the right firmware, such an "out-of-phase" bit transition should be easy to detect.

-Phil

localroger

Detecting 1.5 stop bits as a start bit signal would require you to be sampling the data line at at least 8x the baud rate to do it reliably.

In practice, what was done was that the receiving UART powered up instantaneously, while the transmitting device had to boot even if it did so very quickly, so there was always a quiet period before the blast of data when the system was started.

Parts of the system being started one at a time was a failure mode they just didn't test or care about.

Erlend

For the sake of good order, this is how I decided to implement a function for occasional reading 'sentences' from a free-wheeling asynchronous transmission without stumbelling into mid-sentence. Maybe I have overdone it, but it works fine:

PRI RxSentence(strAddr, rxPin, idleT, endChar) | i, begin, t, wait, rxByte, isActive

   DIRA[rxPin]:= 0

   rxByte:= 32                                                                                    'some initial value
   i:= 0   
   begin:= CNT
   wait:= CNT
   
   REPEAT UNTIL (CNT> wait + idleT*mSec)                                                          'Test input for a period of at least 'idleT' with input High(idle), to get a clean start,
     IF INA[rxPin]== 0
       wait:= CNT
     IF CNT> begin + 5000*mSec                                                                    'or a max time (5sec) expired, in which case something is probably wrong
       RETURN FALSE                                                                                        
                                                                                         
   REPEAT UNTIL rxByte== endChar OR i> 127                                                        'Read characters and look for end of sentence character or too long
     isActive:= FALSE                                                                             'Is there an active link?         
     REPEAT 10000             
       IF INA[rxPin]== 0                                                                          
          isActive:= TRUE                                                                         'yes, 'low' means link is not idle 
          QUIT                 
     IF isActive                                                                                  '- go on read input
       t:= CNT + bitDur/2     
       REPEAT 8                              
         WAITCNT(t += bitDur)                
         rxByte:= INA[rxPin]<< 7 | rxByte>> 1
       WAITCNT(t += bitDur)                                                                       'and write back the byte read into the parent's string (array of bytes)
       BYTE[strAddr][i]:= rxByte & $FF
       i++            
   RETURN i
   

Erlend

Beau Schwabe

Traditionally with single threaded processors, it's not really continuous transmission, since there will be some overhead to deal with, but with multiple processor cores, true continuous transmission is possible. Framing error detection has been mentioned, but taking that a step further instead of discarding the entire BYTE upon error, treat each BIT within the BYTE as a FIFO buffer and just discard a single BIT at a time shifting one more BIT into the FIFO until the framing error goes away. Occasionally, there are circumstances where you can produce a false valid frame where the BITs align properly. Chances are however that the very next BYTE will not align properly. For this there are various techniques you can do to the data packet being sent to ensure that the receiving end will not process invalid data. Providing a SYNC header of at least 2-BYTES that are always the same value and that always follow the start bit is one method, A simple xor CRC BYTE at the end of the data is another, or some combination of the two. But sometimes adding checks like this can be half a dozen of one and six of the other. The time, and wasted BYTES, used to do the various checks, could simply be replaced by a pause between packet transmissions and be almost as effective and your back where you started with a non-continuous transmission scheme.

localroger

Traditionally with single threaded processors, it's not really continuous transmission, since there will be some overhead to deal with

You can also get continuous transmission if each byte is interrupt driven; the overhead to update the buffer is usually a trivial fraction of a byte frame duration, even on a circa 1980 device like the Toledo 8132. This was extremely common in 1980's era CPU-UART interfaces. In fact it wasn't until the late 1980's that UARTs emerged sporting multi-byte buffers, because those PC's were doing things like disc access that held them up for longer than the ever decreasing (because of increasing baud rate) serial frame lengths.