Effective Xon/Xoff flow control?

This discussion was created from comments split from: Prop2 FPGA files!!! - Updated 20 December 2017 - Version 30a.
Infernal Machine

Comments

  • 10 Comments sorted by Date Added Votes
  • Happy new year !
    I'm using - no I'm enjoying - tachyon with P1 and one thing I'm definitely missing is flow control. Although Peter stated above that there currently is no fiddling with newline delay and character delay on the P2, I'm sure in future there will be.
    From my point of view the lack of flow control would avoid unmanned builds which is inconvenient in times of robots ...
    Best regards,
    proplem
    I Am One With the Forth and the Forth Is With Me
  • I also use FL fast load once I have EASYFILE loaded so that everything is buffered into a temporary or named file and loaded from the file where it is not necessary for flow control. With TAQOZ in P2 ROM this should be possible since I have FAT32 already resident, all it needs is an SD card. One advantage of downloading to a named file is that it is also an easy way to transfer source code onto the card. Maybe some of the newer USB serial chips have proper flow control built into them because as we know you can't rely upon the O/S to do it.
    Of course flow control is only necessary for development purposes via serial communication - but (there) it is necessary!
    I think of an unattended build for different hardwares connected via UART or USB. Think of the situations where you @Peter plugged around different hardwares to reproduce problems we were asking for? If you could build for all hardwares in parallel without manual interference wouldn't this be a huge advantage? Or think of an automated build for users: download the files from dropbox/github and building automatically. I'd appreciate that very much.
    Would it be impossible to implement a driver for windows and/or linux which is flow controlling correctly?
    P2 should implement this - the pc side can follow if it is only software.
    Right?
    I Am One With the Forth and the Forth Is With Me
  • jmgjmg Posts: 10,760
    edited January 1 Vote Up0Vote Down
    ... Maybe some of the newer USB serial chips have proper flow control built into them because as we know you can't rely upon the O/S to do it.
    I tested the CP2102N, and that chip-level flow-control seems to work fine.
    Test was to loop back, set 4MBd and send 500k chars. With flow control disabled, this fails badly, with it enabled, 500k arrive fine. Averaged duplex baud is something over 2MBd, due to flow control working.

    Parts like FT232H/FT2232H have larger buffers and faster Max baud, so another way to manage flow control is to keep the chunks below the buffers and software ack.

  • jmg wrote: »
    I tested the CP2102N, and that chip-level flow-control seems to work fine.
    Test was to loop back, set 4MBd and send 500k chars. With flow control disabled, this fails badly, with it enabled, 500k arrive fine. Averaged duplex baud is something over 2MBd, due to flow control working.

    Parts like FT232H/FT2232H have larger buffers and faster Max baud, so another way to manage flow control is to keep the chunks below the buffers and software ack.
    It must be emphasized, that Tachyon receives commands serially and compiles or executes some of them immediately. So reading the buffer is synchronous to executing the commands and even Peter's code sometimes needed a few runtime which brakes reading the buffer. Flow control would care for deterministic behaviour in this matter.
    I Am One With the Forth and the Forth Is With Me
  • jmgjmg Posts: 10,760
    edited January 1 Vote Up0Vote Down
    proplem wrote: »
    jmg wrote: »
    I tested the CP2102N, and that chip-level flow-control seems to work fine.
    Test was to loop back, set 4MBd and send 500k chars. With flow control disabled, this fails badly, with it enabled, 500k arrive fine. Averaged duplex baud is something over 2MBd, due to flow control working.

    Parts like FT232H/FT2232H have larger buffers and faster Max baud, so another way to manage flow control is to keep the chunks below the buffers and software ack.
    It must be emphasized, that Tachyon receives commands serially and compiles or executes some of them immediately. So reading the buffer is synchronous to executing the commands and even Peter's code sometimes needed a few runtime which brakes reading the buffer. Flow control would care for deterministic behaviour in this matter.

    The Bridge device can support hardware flow control, however to operate as you describe as a byte-code system over a live serial link, I think Tachyon needs to include hardware handshake features.
    Does it do this ?
    Probably best to include both-way support, even if P1 is unlikely to saturate an up-link, a P2 certainly could.
    CP2102 specs say RTS Hi for > 448 bytes (of 512 buffer), Lo for < 384 Bytes, and CTS (in) says The CP2102N will not send more than two bytes of data once CTS is pulled high.

  • It's not practical to have hardware handshaking since it is only used during large serial downloads so I am talking about soft handshaking with Xon/Xoff. The trouble with most PC serial ports and USB serial chips is that the Xon/Xoff is pushed into the FIFO and has to wait its turn in the queue by which time the plane has taken off. On the P2 we can have larger bufffers and for the moment I choose to use a very large buffer so I don't have to worry about handshake but in a final system this would be extremely wasteful. However a minimum 1K buffer could probably tolerate these long queues so an Xon/Xoff might work.

    It is also possible to change the settings for the port to trim the length of the internal FIFO but a lot depends upon the O/S too. I will have to play with this to see what I can get away with but unlike the P2, the P1 does not have much memory to play with and even 1K for a seldom used buffer is an extravagance.I have had this P1 buffer dynamically resize into file buffer area but it gets messy.
    Tachyon Forth - compact, fast, forthwright and interactive
    useforthlogo-s.png
    Tachyon Forth News Blog
    TACHYON DEMONSTRATOR
    Brisbane, Australia
  • jmgjmg Posts: 10,760
    It's not practical to have hardware handshaking since it is only used during large serial downloads so I am talking about soft handshaking with Xon/Xoff. The trouble with most PC serial ports and USB serial chips is that the Xon/Xoff is pushed into the FIFO and has to wait its turn in the queue by which time the plane has taken off. ....

    Hmm, ok, I've not tested XON/XOFF, but the CP2102N info says this - hints 64 is the smallest ?

    "Software handshaking uses the same watermark levels as hardware handshaking and can be configured dynamically by host software.
    Watermark levels greater than 512 are converted to an XON limit of 448 bytes and an XOFF limit of 512 bytes. If the XON limit crosses over the XOFF limit, the XON limit will automatically be modified to not cross over the XOFF limit. An XOFF limit of 0 is converted to 64 to guarantee buffer space is available until the UART end device stops transmission. When setting the XON and XOFF limits, it's recommended to use values where the XON limit added to the XOFF limit is less than 512 bytes, like 192/192 or 128/128.
    CP2102N testing shows that the XON limit set to 192 and XOFF limit set to 192 provides optimal software flow control behavior."


  • Peter JakackiPeter Jakacki Posts: 6,776
    edited January 2 Vote Up0Vote Down
    I haven't yet found any software flow control that actually works either in Linux or in Windows. In Linux I can check and set soft handshake through the likes of stty and setserial, in minicom, or GTKterm, or even Cutecom although that is a poor excuse for a terminal. I can write test code that sends an XOFF after the first CR and count how many characters still come in. What do I get? Some 59,000, the whole file as if XOFF never worked.

    I can try the same thing in Windows with TeraTerm and I can even change the FT232R buffer levels to 256/256 vs 4096/4096, enable soft handshake, and try again. Not a sausage. Obviously none of the serial drivers seem to support it, but of course with such large buffers in the USB chips the handshake needs to be handled directly by the FT232R.

    BTW, I tested this on Tachyon V4r7 but with a special version where I could change the size of the receive buffers up to 2K. There is an XOFF sent after the first CR received during a TACHYON (no echo) block load and an XON is only sent once the buffer is exhausted etc.

    Now I will give the CP2102 a go and see what happens. I vaguely recall going through this exercise many years ago and giving up in disgust.

    EDIT: I think I will see how effective hardware handshake is as I really only need the one line back to the serial port. I'll test it out first with a direct I/O from the Prop and then I might add a tiny PIC12F1572 to intercept an XON/XOFF and have it handle the hardware handshake rather than the Prop, just so I don't need to change anything on my Prop boards.
    Tachyon Forth - compact, fast, forthwright and interactive
    useforthlogo-s.png
    Tachyon Forth News Blog
    TACHYON DEMONSTRATOR
    Brisbane, Australia
  • jmgjmg Posts: 10,760
    ..
    Now I will give the CP2102 a go and see what happens. I vaguely recall going through this exercise many years ago and giving up in disgust.

    EDIT: I think I will see how effective hardware handshake is as I really only need the one line back to the serial port. I'll test it out first with a direct I/O from the Prop and then I might add a tiny PIC12F1572 to intercept an XON/XOFF and have it handle the hardware handshake rather than the Prop, just so I don't need to change anything on my Prop boards.

    Note the part I quoted from & tested is the new CP2102N, (upgraded MCU version of CP2102, faster and (re)config is now flash, not OTP) - there is now a low cost CP2102N-MINIEK module.

    It also says this :

    If the CP2102N receives an XOFF request, it will stop transmission, even if the CP2102N receiver needs to transmit an XOFF over
    UART. This can potentially allow an overflow to occur or a deadlock condition if both the CP2102N and the connected UART device
    transmit XOFF at the same time. The XOFF_CONTINUE setting allows the CP2102N transmitter to send XOFF/XON requests even if it
    has received an XOFF request from the connected UART device. Once the connected UART device transmits XON, normal transmission
    from the CP2102N resumes
    .

    I think that means the CP2102N both manages XOFF internally, & can pass-thru the XOFF. (so you need to set P1 and CP2102N for the same XOFF )
    XON comes from the P1, and is acted on by the CP2102N, which feeds from the local buffer.
  • Peter JakackiPeter Jakacki Posts: 6,776
    edited January 2 Vote Up0Vote Down
    Yes, I had noticed that and had a look at the N version and I do remember that it had been discussed some time ago too. It's funny though that an Xoff blocks another Xoff because normally an Xoff is sent at the low (or high pov) watermark when there is still some buffer space available, but not much. Anyway, Xon/Xoff don't require buffer space, they are normally handled within the interrupt service routine on most micros or by the receive cog.

    The CP2102N at least discusses watermarks and conditions when it says it handles soft handshake whereas the FT232R says it handles it in hardware for "fast" response but says nothing further about it and neither does it seem to be effective. The FT232R chips I use are from an approved distributor so they're not fake chips either.

    However in spite of a lack of flow control I find P1 Tachyon quite snappy and it only seems to need a few milliseconds line delay. In contrast there is MeCrisp Forth that I have run on ARM chips and despite hardware and interrupts and stacks of RAM it still needs very generous delays, even character delays I find, on 200MHz+ ARM chips!. P2 Tachyon of course has plenty of RAM to spare and doesn't need any delays of any kind, even at 3M baud.
    Tachyon Forth - compact, fast, forthwright and interactive
    useforthlogo-s.png
    Tachyon Forth News Blog
    TACHYON DEMONSTRATOR
    Brisbane, Australia
Sign In or Register to comment.