XBee input buffer size and CTS

Ron Czapala

According to the XBee manual, the DI (input) buffer is 202 bytes and
"When the DI buffer is 17 bytes away from being full; by default, the module de-asserts CTS (high) to signal to the host device to stop sending data [noparse][[/noparse]refer to D7 (DIO7 Configuration) parameter]. CTS is re-asserted after the DI Buffer has 34 bytes of memory available."

I wrote 2 little test programs to explore the flow control and CTS status.
The TX program prompts for the number of bytes to send (1-255).
The RX program waits for any key before using serin to read the buffered data.
Both XBee's are intialized to use flow control (ATD6 1).
(You effectively need two serial debug windows open to enter/view the data)

The buffer seems to only handle 132 bytes - not 202.· Also it does not appear that the CTS pin changes while the data is being transferred.

Has anyone made use of the XBee CTS pin to control data flow?

Wildatheart

ronczap, thanks for taking the time to write the test programs and for seeking to resolve yet another issue related to the Xbee. I didn’t run your programs because I have only 1 RS 232 to USB adapter at this time, so I can’t duplicate your findings.

Here’s another approach to evaluate buffer size.

Reading through your code, I noticed that you’re TX’ing data in a slightly different format from what I found works reliably. But then, you’re TX’ing byte sized data. From experience, TX’ing byte size data seemed to work reliably in several formats, including yours. However, when I TX’ed word values, certain anomalies showed up as we witnessed in the screenshots and as we previously discussed in another topic here. http://forums.parallax.com/showthread.php?p=901652

The reason I mentioned that is because of the single or double CR we’re using in the SEROUT statement. Perhaps you know the possible impact that CR(CR’s) have on the “size of the packet” being TX’ed. In other words, are the CR’s taking up space in the receiver Xbee’s buffer? And if they are TX’ed along with the data, should they be considered as we discuss buffer size?

My testing showed that the receiver’s buffer size was 56 words or (56 x 2 = 112 bytes) + (56 x 2 CR bytes = 112 bytes) = 224 bytes. When TX’ing byte values the results showed the buffer size to be (70 x 1 byte) + (70 x 2 CR bytes) = 210 bytes.

For this discussion, I’m defining “buffer size” as counting the displayed decimal values beginning at the first buffer read, and ending when program control (SERIN) experiences Timeout - then looking for the point of disconnected data (the point of obvious buffer overflow). This assumes a continuous flow of TX’ed data that is being sent at a rate just slow enough for the receiver to experience a Timeout.

Post Edited (Wildatheart) : 5/5/2010 8:24:37 PM GMT

Ron Czapala

I don't think the CR after each byte is being transmitted - if I omit it, I only receive every other byte. Also, the receiver buffer would not hold 132 x 2 bytes.

Using the DEC modifier causes considerably fewer values to be buffered since sending decimal values > 9 sends multiple bytes for each value
·· e.g.· sending the value 125 transmits one byte without using DEC and transmits four bytes using DEC (including the CR)

I tried different pack RO Packetization Timeout values with no change in the results.

wildatheart,

I have tweaked the code so you can see the results with one DEBUG window. The TX2 program sends 1 to 140 and ends (You can resend by reseting the stamp)

I question your result showing 224 bytes in the buffer. The XBee documentation from Digi states that it is 202 - it is unlikely to be greater than that.

If your test program used the DEC modifier - it would complicate the problem (due to the reason I mentioned above).

The stamp manual might explain it better:

Notice the decimal formatter in the SEROUT command. It is the “#” (for the BS1) or “DEC” (for all BS2 models) that appears just to the left of the number 65. This tells SEROUT to convert the number into separate ASCII characters which represent the value in decimal form. If the value 65 in· the code were changed to 123, the SEROUT command would send three bytes (49, 50 and 51) corresponding to the characters "1", "2" and "3".





Post Edited (ronczap) : 5/5/2010 10:25:22 PM GMT

Wildatheart

Although I haven't tried your revised programs yet (I will), further reading at the Digi site also talks about an input buffer ( DI holds data prior to transmission) at the transmitter. It too is limited in size. My thinking and obversations have been focused on what I thought was the receiver buffer. But because my testing gave different results for byte transmissions and word transmissions could it be that the transmitter buffer governs what we're seeing in our example programs? And if so, could this be the answer to your question about why CTS does not change states?

Post Edited (Wildatheart) : 5/5/2010 11:36:00 PM GMT

Ron Czapala

The buffering is explained in the product manual at http://ftp1.digi.com/support/documentation/90000982_B.pdf· on pages 11 and 12.

The data moves from the BS2 to the XBee thru the UART DIN pin (pin3)·into the DI buffer. It is packetized and transmitted.

Received data is stored in the DO buffer and sent out the serial port on the UART DOUT pin (pin 2) to the·BS2.

The packet overhead may be the reason the data·bytes appear to be limited to 132 of the 202 byte buffer.·The buffer overrun is most likely happening in the transmiiting XBee but I expected the CTS pin to change when the buffer·is close to being full.

I goofed trying to set the RO parameter!· I was·trying to set R0 (R ZERO) - not RO.

Now I'm confused because when I·set·the RO parameter to 10, it seems to work with 254 bytes!



Post Edited (ronczap) : 5/5/2010 11:53:37 PM GMT

Wildatheart

Packet overhead may very well be the reason for your observations. But, regarding CTS not changing state at the receiver, why would it - if the bottleneck is at the transmitter? Isn't the receiver always in the ready state?

Wildatheart

Referring to different TX and RX example programs...

Removing the PAUSE statement after SEROUT at the transmitter seemed to indicate that the BS2 was not fast enough to generate values and send them to the TX Xbee to cause a catastrophic DI buffer overflow. Although the BS2/receiver Xbee setup displayed those received values with a large degree of order, there were tiny gaps in the data, and SERIN did not Timeout.

This leads me to believe that the transmitter is not the bottleneck, and it takes me back to square one – the overflow can be caused by a purposefully slow keypress before reading the buffered data in the RX Xbee. A very orderly and catastrophic disconnect (and resumption) does appear in this case.

But… that would disprove the theory about CTS not changing state. Interesting.

Ron Czapala

wildatheart - I'm afraid you lost me - what "different programs" ??

The RO parameter is confusing -·the Digi forum didn't reveal any answers

http://www.digi.com/support/forum/listthreads?forum=7

The attached programs (using ATRO 10) seems to indicate that 350 bytes are being buffered

I wonder about the accuracy of the DIGI documentation...

Wildatheart

With… "ATD6 1",CR, ' Enable flow control
"ATD7 1",CR, ' Set pin for flow control

And with the CTS pin of the adapter board hardwired to a BS2 pin 12, shouldn’t we be seeing a break in program control when the DI buffer hits it’s limit using the following code additions?

INPUT 12
DO
X = X + 1
SEROUT TX,Baud, [noparse][[/noparse]DEC X,CR,CR] ' Send value of X as decimal to Xbee
PAUSE 300
DEBUG " ", DEC x
IF IN12 = 1 THEN GOTO CtsActive
LOOP

CtsActive:
Debug “DI buffer full”
END

Zoot

Your test might be more useful w/o the DEC formatter, which will split binary "numbers" into ASCII character "digits", i.e. the "number" 129 won't be sent as a byte, it will be sent as the characters "1","2","9". And so X won't give you a true count before CTS is theoretically released to high. This presumes that flow control has been enabled on the Xbee.

CTS PIN 12 ' input by default
ClearToSend CON 0
NotClearToSend CON 1
x VAR Word

DO
LOOP UNTIL CTS = ClearToSend ' wait for clear before trying

x = 1 ' init count and val to send
DO
IF CTS = NotClearToSend THEN GOTO CtsActive ' put this here to check if buffer full before sending next byte
SEROUT TX,Baud, [noparse][[/noparse]X,CR,CR] ' Send value of X as byte to Xbee
PAUSE 300
DEBUG " ", DEC3 X
X = X + 1
LOOP

CtsActive:
Debug CR, CR, “DI buffer almost full”, CR,
   "X: ", DEC3 X   
X = X+17
DEBUG CR, "X+17 (total buffer size):", DEC3 X
END

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
When the going gets weird, the weird turn pro. -- HST

1uffakind.com/robots/povBitMapBuilder.php
1uffakind.com/robots/resistorLadder.php

Zoot

ronzcap -- looking at your TX test, I'm not sure you would ever overrun the buffer. I think using \CTS in the serout might also cause you to "hang" during possible buffer overrun -- checking for the status of CTS manually and deciding whether or not to move on might be handy. Jacking the baud rate up would help force a de-assertion of CTS as well.

Are you getting good data transmitted?

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
When the going gets weird, the weird turn pro. -- HST

1uffakind.com/robots/povBitMapBuilder.php
1uffakind.com/robots/resistorLadder.php

Tracy Allen

Remember that the XBee will transmit a packet automatically when there are 100 characters in the buffer. The RF data rate is 250 kb/s, far faster than the BASIC Stamp's 9600, so the XBee's buffer under intended operation gets emptied faster than you can fill it.
There are three events that can trigger the XBee to send out an RF packet:
-- there are 100 chars in the buffer (maximum size of a RF packet)
-- the RO timeout delay has passed with no input to the transmit buffer
and if RO=0 chars are transmitted immediately
-- the command sequence CC framed with guard times is received.

Note that the XBee throws the data off into the radio cybervoid. In the default Digi mode MM=0 it will then listen for an ACK from the intended receiver, and make up to 3 retries with the same data. Suppose the intended receiver is offline. The transmitter waits up to 200 ms for an ACK that never comes, and then up to 48 ms for each retry. So maybe you would have better luck to hit capacity if you operate the XBee into a void. The RR command in Digi mode MM=0 can increase the number of retries up to 9.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Tracy Allen
www.emesystems.com

Ron Czapala

Zoot,

Checking the CTS pin doesn't have any effect - it never goes high.· The data received is accurate until the overflow...

The attached programs overflow if I send > 350 bytes before·triggering·the·serin loop·in the RX program (using ATRO 10 in the TX pgm).

---

Tracy,

Does·the following description sound accurate?· I am trying to get a handle on this whole process...

---

The flow control between an XBees (using default Transparent Operation mode) and a basic stamps can be described as:

From the transmitting stamp -> XBee(with flow control enabled)
Flow control prevents the stamp from putting data into the DI buffer faster than the the XBee can transmit it (CTS - Clear To Send).
(not really possible at 9600 BPS since the XBee sends a packet when 100 bytes are in the buffer and the XBee transmits at 250,000 bps)

At the receiving XBee(with flow control enabled) -> stamp
Flow control allows the stamp to inform the XBee that it is ready to receive data from the DO buffer, avoiding data loss between the XBee and stamp. (RTS - Ready To Send)

Flow control does not directly affect data loss between the transmitting XBee/stamp pair and the receiving Xbee/stamp pair.
If the transmitting pair sends data packets faster than the receiving pair processes them, data will be lost.

If an XBee uses API Operation mode, it can receive the success/failure status of each transmitted RF packet and
identify the source address of each received packet.
API mode examples: http://forums.parallax.com/showthread.php?p=885616

---

Also, with regard to transmission retries and ACK, I assume this only works if MY and DL addresses are used - correct?

- Ron


Post Edited (ronczap) : 5/6/2010 6:26:06 PM GMT

Tracy Allen

In transparent mode, you can read the number of ACK failures using the AE command.

>(not really possible at 9600 BPS since the XBee sends a packet when
>100 bytes are in the buffer and the XBee transmits at 250,000 bps)

That was what brought up the question of what happens if the receiver is offline and the buffer retains data for the retries. Under those conditions it should? be possible to see the CTS line go to the "don't send any more" state. That might also happen in situations where their is a crowded channel or interference. It is a question. I don't think I've ever tested it, so if you do I'd be interested to hear your results.

The XBee even with retries does not guarantee that packets will get through, because after the selected number of retries it will discard the packet. As you note, by checking for ACKs and taking advantage of flow control, a program should be able to improve the integrity.

The rules regarding retries and ACKs are pretty complicated, first having to do with the mode set by the MM command, and also by whether the XBee is operating in unicast mode (the default) or with a coordinator or in cyclic sleep mode. Not well documented.

In Digimesh firmware, the CTS flow control pin takes on a different role, as it signals back to the processor that the mesh is ready to accept data.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Tracy Allen
www.emesystems.com

Wildatheart

Although it is academically interesting to pursue factors relating to buffer overflow and the condition of the flow control signals, I’m wondering if it’s reasonable to think that we could implement CTS in a useful way on a slower and memory restricted micro, such as the BS2. It seems to me that if CTS was implemented, the intended outcome would be to halt processing when necessary - thus creating a bottleneck that would require substantial program resources.

Knowing that RTS enables contiguous receptions (until buffer overflow), and also knowing that we have a buffer of “some size” to hold those receptions, wouldn’t it be more efficient to receive a unique identifier along with the transmitted data that requires any amount of processing? i.e. - not necessarily data logging type activity that’s just read and pigeon holed.

At the end of the day, if we’re going to run these things bidirectionally based on real time events we’re going to have to do some kind of “send/receive/acknowledge” failsafe method of handshaking between the units. Then, doesn’t CTS become a moot point? Or am I missing a very useful feature of the Xbee?

Ron Czapala

It appears to me that if you want to make sure there is NO data loss, you need to· use API mode and check the status of each transmitted packet.

In my testing, a BS2 was not fast enough to read the Transmit Status packet after sending the 16-bit Transmit Packet, but a BS2p worked.

Tracy Allen

Wildatheart, with regard to the Stamp locking up on account of CTS, isn't that when a timeout on the SEROUT command could save the day.
SEROUT xbinpin\xbinflo,$54,xbtimeout,xbctserr,[noparse][[/noparse].......] ' timeout can rescue SEROUT from lockup
Same idea on SERIN--There has to be a timeout, to allow escape in case the expected data does not arrive.

I agree that API mode has better potential for error checking if the timing issues can be resolved. Nonetheless, burden of providing for data integrity above the automatic retry quotient would fall on the program code in either API or transparent mode. Another dimension comes when one XBee is assigned the job of coordinator. The coordinator can select a clear channel for the entire network, potentially improving reliability against RFI, but it puts a higher burden on the reliability of the coordinator itself.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Tracy Allen
www.emesystems.com

Wildatheart

Tracy, I wasn’t clear enough… yes, for sure we need to include a Timeout in the event the serial exchange fails. I was referring to a situation where a series of machine functions were dependent upon regular updating. Then in the event of an unanticipated request from CTS to halt regular processing, substantial micro resources could be required to maintain scaled back control of the system until the CTS problem is resolved.

Ron, if I understand your last comment, you’re now moving away from the original CTS discussion (timing) and moving toward the subject of data integrity. I’m not aware of our need to suspect the integrity of transmitted data. But if you’re extending the scope of this discussion its fine with me – the more tools we have in our toolbox, the better we can perform the work.

At some point, before we leave this discussion, I would like your ideas on the best way to accomplish bi-directional exchanges. I first thought a constantly changing CTS signal would help in some way. But now it seems some kind of “start character” attached to the received data seems to be an efficient way to accomplish that. (Again, I’m not talking about routinely attaching that “start character” in the case of data logging type exchanges.)

If a valve on my submersible happens to become nonresponsive, I wouldn’t want to get hung up in potentially lengthy and complex code while attending to a delay caused by a CTS signal as it relates to buffer overflow. I’d sooner dump the entire buffer while looking for a “start character” attached to a data that would permit me to move on. (I think.)

Ron Czapala

wildatheart,

· My original goal was to examine if flow control using CTS would prevent data loss (and to·verify the buffer size).·With 29 years of information systems·and programming experience, I am always concerned with data integrity. If data is lost, there is no data integrity.

With that said, I doubt that most basic stamp applications are critical and some data loss might be acceptable as long as the remaining data is accurate and timely.

I plan to continue exploring the XBee capabilities and two-way communication.