USB Testing

1121315171820

Comments

  • Whew!
  • My apologies for being absent for so long, but I finally had a good stretch of days to make some progress in P2-ES evaluation.

    But, I've run into a bit of a snag that may include the smartpin USB transmitter state machine.

    When I did my initial quick/dirty tests with the P2-ES board I just modified the code to configure the sysclock to use the 20MHz crystal to match the 80MHz used by the FPGA. This gave the same results I had been getting on the FPGA -- most devices enumerated cleanly, some 3.x parts did not. Then I cranked the sysclock up to 180MHz and the IN transactions broke, which I was expecting because I had been cutting some timing corners due to the 80MHz constraints. Once those were cleaned up, device enumeration was working better than ever, including those pesky 3.x thumbdrives that were being so flaky at 80MHz!

    But then I ran my media read "stress" routine that scans the first FAT region and calculates the count of free cluster entries so the "DIR" output can show the volume bytes remaining -- and it broke.

    On USB, the SCSI command parameters are packaged into a CBW (Command Block Wrapper) that is sent from host to device using the bulk-out transfer protocol and if the CBW is "valid and meaningful" the device ACKs the CBW and waits for the host to issue consecutive bulk-in transfer requests, and the device responds with IN data packets (up to the full-speed data packet limit of 64 bytes) until the requested sector count is met. When/if the sector count is fulfilled, the device issues the SCSI command success/fail result in a CSW (Command Status Wrapper) using a final bulk-in transfer. To exercise the out/in process, I was only requesting one sector at a time so if things went smoothly the sequence was 1 OUT and 9 IN. But when the eventual fail happened, it was the OUT transfer which, given the 1:9 ratio, was an eyebrow-raising moment.

    The device was not responding to the OUT transfer because the out data was seen as being corrupt. The USB analyzer diagnosed the fail was due to a bad CRC (see image CRCFailRecover). At the time I had a transaction fail retry set to six, so I increased it 20 to see if it could recover and a pattern emerged. Sometimes the first error encountered would recover, but farther into the transfer it would fail again, and the retries increased exponentially until whatever retry cap I set was met.

    The bad packet byte number was always one of #6, 7 or 8. If the device deems the packet not valid due to corruption, it may ignore that packet and wait for the host to send another, or cancel the transfer by issuing and endpoint reset. The offending byte in this packet was #7 (0x91) in all these fails (but has differed in others) and 0xBF that makes the CRC work is in the recovery packet. The analyzer always shows the CRC that was transmitted and all failed packets AND the accepted packet had the same 0x3f29 CRC, so it was definitely a bus glitch. It's looking like just four bits or fewer are getting scrambled whenever there's a fail.

    My subroutine to output a byte to the USB has always been a clone of the example in the smartpin_usb_turnaround.spin2 program in the FPGA version drops:
    utx_byte
                    testp   #DP                     wc
            if_nc   jmp     #utx_byte
                    akpin   #DP
            _ret_   wypin   utx, #DM
    

    I added a RDQPIN after the return of the output sub to check status, but the error bit (#6) was never set. In the end, that doesn't really matter, because the only way to "recover" from a bad CRC is to re-transmit the packet. But I was stuck, because once the OUT CRC fail got locked in the transfer was doomed.

    I re-read the smartpin docs regarding WRPIN/WXPIN/WYPIN/RDPIN/AKPIN behavior and every thing looked good. The USB transmit looks to be somewhat unique in that the "feed me a byte" flag is raised on the D+ (upper/odd) pin, but the WRPIN is done on the D- (lower/even) pin, so an AKPIN is necessary on the upper pin. The doc mentioned that it took at least two clocks for the flag pin to drop following AKPIN, so I added a NOP, which didn't work, so I added another, which sometimes worked, so I added another and at 180MHz, there were still OUT CRC fails, but they would recover after two or three retries. So I kept adding NOPs until the OUT CRC fails vanished completely and I replaced the NOPs with a WAITX #10.

    Then I started testing at higher sysclocks, bumping it up +12MHz each time. But up around 216MHz the OUT CRC errors came back, so I increase the delay and they go away again. Eventually I ended up with these results, settling with a "mostly effective compromise tweak" setting :
    utx_byte
                    testp   #DP                     wc
            if_nc   jmp     #utx_byte
                    akpin   #DP
                    waitx   utx_tweak       ' Wait #3 if < 180MHz, wait #20 if 180MHz+
    '                waitx   #3              ' 108MHz..168MHz mostly OK, some 3.x parts flaky
    '                waitx   #13             ' clean to ~300MHz
    '                waitx   #20             ' clean 180MHz..360MHz 
            _ret_   wypin   utx, #DM
    

    But I have no answer as to why this is, other than it doesn't seem to me that a clock-based state machine should behave in this manner. All I know for sure is that the delay must be between AKPIN and WRPIN, or it all falls apart.

    I've got a little more cleanup to do, but hope to get some code posted sometime today...
    956 x 732 - 98K
    garryj
  • RaymanRayman Posts: 9,906
    edited 2019-01-24 - 21:27:02
    Glad to hear you figured out a way to make it work!
    I guess that's the important part.
    Seems we could just bit-bang the USB with the kind of clock speeds we have now, right?
    I've been afraid to go to 300 MHz... Wonder how safe that is...
    Can we just divide the clock freq by some number to get a good waitx value?

    I suppose "at least two clocks" is still technically accurate... Wonder if it's an amount of time, rather than fixed # of clocks...
    Prop Info and Apps: http://www.rayslogic.com/
  • Is it in any way possible that the PLL jitter is causing framing errors?
  • whicker wrote: »
    Is it in any way possible that the PLL jitter is causing framing errors?

    That has crossed my mind, but I don't have the engineering skills to adequately analyze it properly :confused:
    garryj
  • jmgjmg Posts: 14,088
    garryj wrote: »
    whicker wrote: »
    Is it in any way possible that the PLL jitter is causing framing errors?

    That has crossed my mind, but I don't have the engineering skills to adequately analyze it properly :confused:

    Just keep the PFD high, or the divider as low as possible.
    That will be easier with a 24Mhz xtal - do you have any you can drop in ?
  • RaymanRayman Posts: 9,906
    edited 2019-01-24 - 23:01:11
    I don't see how it could be jitter...

    But, maybe has to do with how the receiver is working... If the receiver is syncing it's clock to the packet itself, maybe it is forced to resync when there is a big enough space between packets...
    Prop Info and Apps: http://www.rayslogic.com/
  • jmgjmg Posts: 14,088
    edited 2019-01-24 - 23:20:18
    garryj wrote: »
    But I have no answer as to why this is, other than it doesn't seem to me that a clock-based state machine should behave in this manner. All I know for sure is that the delay must be between AKPIN and WRPIN, or it all falls apart.
    A state machine should be either correct, or not, and subsequent delays should, of course, not matter.
    However, I think P2 does not quite do a proper USB clock locking/extraction, instead you WAIT on some event, which gives you a lock reference, and then run a 12Mhz NCO to try and keep phase with following data, based on that initial sample. Waits there, may be shifting the following sample points ? - could that apply to your code ?

    How often do you resync to a USB edge ?

    As data packets get larger, that gets more fragile.
    I think you are saying, this is ok with smaller packets, but fails above some size ? (6,7,8?)
    Did you test sizes all the way up to the max 64 ?
    If you are going to lock on an edge, and then remain 1/4 USB time aligned, over ~64 bytes I get

    64*8*1.333*4 = one part in 2729.984
    or, in ppm = 366ppm

    Appx time is 64*8*1.333*(1/12M) = 56.87us, so those numbers are ballpark similar to VGA jitter.
    If you can hold one part in 2730, over 57us, you are likely to get a clean VGA display.
    Those are 64 byte estimates, as low as 6,7,8 should be proportionally more tolerant
    If you hope to resync to a frame marker only, that then needs ~21ppm timebase, but you can auto-cal to the USB frame marker.

    What you could also try, is capture a counter every frame marker, and check the variation in sysclks - that will show jitter over 1ms windows.

  • Rayman wrote: »
    I don't see how it could be jitter...

    But, maybe has to do with how the receiver is working... If the receiver is syncing it's clock to the packet itself, maybe it is forced to resync when there is a big enough space between packets...
    I think this particular issue may only be related to the transmitter. The smartpin tx/rx state machines are separate, but the baud generator is common to both. The receiver has been working spectacularly at all sysclocks I've tried. So far, if something has gone awry regarding the receiver, it has been due to stupid programmer tricks :blush:

    The start-of-packet byte is what the receiver syncs to and, on a larger scale, the 1ms frame counter must be 1ms +-500ns from SOF to SOF. In either of those cases, the analyzer will report them.

    Regarding the 1ms frame counter, in my testing of different sysclock rates, I tried 144.5MHz to see how the smartpin baud generator would do, and the analyzer flagged every 1ms frame as having jitter. I didn't go any deeper than this one quick test because I have a ton of other issues at a higher priority.
    garryj
  • garryjgarryj Posts: 283
    edited 2019-01-25 - 00:33:59
    jmg wrote: »
    As data packets get larger, that gets more fragile.
    I think you are saying, this is ok with smaller packets, but fails above some size ? (6,7,8?)
    Did you test sizes all the way up to the max 64 ?
    Yes.

    Edit: Next on my list is multi-sector writes, will should be the ultimate make/break for the transmitter, as that 1:9 OUT:IN ratio gets reversed. Should have some results fairly soon.
    garryj
  • I meant the receiver at other end of cable...
    Interesting puzzle
    Prop Info and Apps: http://www.rayslogic.com/
  • Rayman wrote: »
    I meant the receiver at other end of cable...
    Interesting puzzle

    Ah. If the device is having issues reading what's on the bus, you'll find out pretty quickly through the ACK/NAK/STALL/IGNORE and DATAx sync mechanisms.
    garryj
  • jmgjmg Posts: 14,088
    garryj wrote: »
    I think this particular issue may only be related to the transmitter. The smartpin tx/rx state machines are separate, but the baud generator is common to both. The receiver has been working spectacularly at all sysclocks I've tried. So far, if something has gone awry regarding the receiver, it has been due to stupid programmer tricks :blush:

    The start-of-packet byte is what the receiver syncs to and, on a larger scale, the 1ms frame counter must be 1ms +-500ns from SOF to SOF. In either of those cases, the analyzer will report them.
    That's good news, at least. Receiver issues are likely to be lower level, and harder to sort.

    Is this test a USB host, with flash drive slaves as test devices ?
    Maybe some slaves expect TX edges to be 12MHz aligned within the frame ?
    USB 3 devices likely have a fancier PLL, and more assumptions about timing that creep into their 12MHz side.

    12MHz only designs may use a simpler digital-locked loop edge extractor locking, which is more tolerant of unit to unit timing ?

  • Bob Lawrence (VE1RLL)Bob Lawrence (VE1RLL) Posts: 1,549
    edited 2019-01-25 - 01:59:15
    Here is the USB 3.0 Specs in case you need it:

    Universal Serial Bus 3.0 Specification - USB3.com

    Attached


  • garryjgarryj Posts: 283
    edited 2019-01-25 - 20:45:22
    jmg wrote: »
    Is this test a USB host, with flash drive slaves as test devices ?
    Yep. My test device pile is a dozen "thumb drives" of which four are 3.x parts, seven are 2.x and one is 1.1. I also have an old SDCard reader, a uSD reader and a uSD/SD reader (2.x). Capacities range from 64MB to 64MB 64GB.
    Maybe some slaves expect TX edges to be 12MHz aligned within the frame ?
    I don't think so. The host/hub must schedule the bus transfer(s) so they are guaranteed to complete within the 1ms frame window. At least that's the only rule I've been following and the analyzer hasn't complained, yet.
    USB 3 devices likely have a fancier PLL, and more assumptions about timing that creep into their 12MHz side.
    Yeah. It's a whole new world above 12MHz (micro frames and such).
    12MHz only designs may use a simpler digital-locked loop edge extractor locking, which is more tolerant of unit to unit timing ?
    In general, I've found 12MHz devices to be very tolerant. When I first started this project, the oldest devices I had were the first to talk to me :lol:
    garryj
  • garryjgarryj Posts: 283
    edited 2019-01-25 - 16:43:05
    USB Mass Storage Demo v0.05.

    It's currently a RAM pig -- ~15KB code, 74KB data and buffers.
    You must use PNut_v32j as USBMS.spin2 breaks the symbol limit of earlier versions. It's in the .zip file if you don't have it.

    Edit: The latest version of p2asm should work also.Doh! I forgot I'm working with my own patched version of p2asm.exe that adds support for the if_00, if_01, etc. instruction prefixes. I've submitted a pull request to @Dave Hein, but it hasn't been picked up, yet.

    Default sysclock is 180Mhz.
    Serial default is 2MBaud.
    On the P2-ES I recommend using the auxiliary "P2 USB" connector for power. If you only use power from the PC, connecting a USB device may trigger a brown-out detection reset.
    Remember that the USB smartpins have their own pull-up/pull-down resisters, according to the mode that is set (device or host). I use 27ohm terminating resistors on D-/D+.
    With one exception, all media accesses are read.
    See the Readme.txt file in the .zip for more information.

    Hope some of you can give it a spin.
    659 x 654 - 27K
    659 x 654 - 49K
    659 x 654 - 19K
    5312 x 2988 - 3M
    garryj
  • Garryj,

    Thanks for all your USB development work.

    Are you thinking there's anything in need of change on the P2 hardware side for USB? Is there anything that could be called a bug?
  • jmgjmg Posts: 14,088
    From what I understand of the above, it is working, but not at 100% on all test units, so there is still some subtle issues to resolve.
    The exact cause is still tbd.
  • jmgjmg Posts: 14,088
    garryj wrote: »
    Then I started testing at higher sysclocks, bumping it up +12MHz each time. But up around 216MHz the OUT CRC errors came back, so I increase the delay and they go away again. Eventually I ended up with these results, settling with a "mostly effective compromise tweak" setting :smile:
    Is that working from a 4Mhz PFD ? (20M/5)
    You could try 120MHz and 240MHz, as those allow 20MHz/1 into the PFD, which is known to be least-jitter case.
    If that makes little difference, then PLL/VCO jitter might not be an issue here.

    A P2D2 has a 12MHz TCXO, for another test point.


  • It might just be that some status signal in the smart pin needs to persist longer, in order to properly space transmit operations from receive operations. I need to review the design.
  • When I get a free moment in the next few days I will test this on the P2D2 since it has a 12MHz 5ppm oscillator and LDOs for I/O.

    Tachyon Forth - compact, fast, forthwright and interactive
    useforthlogo-s.png
    P2 --- The LOT --- TAQOZ INTRO & LINKS --- P2 SHORTFORM DATASHEET --- TAQOZ RELOADED - 64kB binary with room to spare
    P1 --- Latest Tachyon with EASYFILE --- Tachyon Forth News Blog --- More
    paypal.png PayPal me
    Brisbane, Australia
    phone.png
  • Hi garryj,

    Do you have any wiring diagram? It seems that you're using two resistors, but I can't figure out how they are connected.

    Kind regards, Samuel Lourenço
  • Have you tried using the internal pull-up/downs?
    Prop Info and Apps: http://www.rayslogic.com/
  • samuell wrote: »
    Hi garryj,

    Do you have any wiring diagram? It seems that you're using two resistors, but I can't figure out how they are connected.

    Kind regards, Samuel Lourenço

    Those are 27ohm terminating resistors on D+/D-. I looked at the P2-ES schematic for the USB PC port, but it will likely work without them (mine does).
    garryj
  • Rayman wrote: »
    Have you tried using the internal pull-up/downs?

    Those are automatically activated when the smartpin USB mode is set.
    garryj
  • garryjgarryj Posts: 283
    edited 2019-01-25 - 18:08:56
    jmg wrote: »
    Is that working from a 4Mhz PFD ? (20M/5)
    No, I was using a 20MHz PFD, (20M/20).

    Below are the frequency settings I've been testing at. Now, before you gasp in horror, remember that I am not an EE :crazy:
    I modified _FRQ_180 to use 20M/5. It does indeed look like it could be self-induced jitter.

    I would sure appreciate you reviewing the below and making changes as needed to what you think might be the best values, and I will test.

    And maybe you could put together a brief "PLL for dummies" that explains each values effect for us non-EEs? I know I would sure appreciate it.
    '------------------------------------------------------------------------------
    '        _XPPPP = ((_XDIVP >> 1) + 15) & $f ' (_XDIVP = 1)->15, 2->0, 4->1, 6->2...30->14
    '       _FRQ_*  = 1 << 24 | (_XDIV - 1) << 18 | (_XMUL - 1) << 8 | _XPPPP << 4 | _XOSC << 2
    '       _CLK    = _XTALFREQ / _XDIV * _XMUL / _XDIVP
    '------------------------------------------------------------------------------
    '        _FRQ_80    = 1<<24|(20-1)<<18|(160-1)<<8|((2>>1+15)&$f)<<4
    '        _FRQ_84    = 1<<24|(20-1)<<18|(168-1)<<8|((2>>1+15)&$f)<<4
    '        _FRQ_96    = 1<<24|(20-1)<<18|(192-1)<<8|((2>>1+15)&$f)<<4
            _FRQ_108   = 1<<24|(20-1)<<18|(108-1)<<8|((1>>1+15)&$f)<<4 
            _FRQ_120   = 1<<24|(20-1)<<18|(120-1)<<8|((1>>1+15)&$f)<<4 
            _FRQ_132   = 1<<24|(20-1)<<18|(132-1)<<8|((1>>1+15)&$f)<<4 
            _FRQ_144   = 1<<24|(20-1)<<18|(144-1)<<8|((1>>1+15)&$f)<<4 
    ' My USB analyzer flags the full-speed 1ms start-of-frame packets as having timing
    ' jitter if a fractional SYSCLOCK is set, e.g. 148.5Mhz. If the device type that's
    ' connected it tolerant to this jitter, it will still work.
            _FRQ_148_5 = 1<<24|(40-1)<<18|(297-1)<<8|((1>>1+15)&$f)<<4 ' Frame timing jitter
            _FRQ_150   = 1<<24|(20-1)<<18|(150-1)<<8|((1>>1+15)&$f)<<4
            _FRQ_156   = 1<<24|(20-1)<<18|(156-1)<<8|((1>>1+15)&$f)<<4
            _FRQ_160   = 1<<24|(20-1)<<18|(160-1)<<8|((1>>1+15)&$f)<<4
            _FRQ_168   = 1<<24|(20-1)<<18|(168-1)<<8|((1>>1+15)&$f)<<4
            _FRQ_180   = 1<<24|(5-1)<<18|(90-1)<<8|((2>>1+15)&$f)<<4
            _FRQ_192   = 1<<24|(20-1)<<18|(192-1)<<8|((1>>1+15)&$f)<<4
            _FRQ_200   = 1<<24|(20-1)<<18|(200-1)<<8|((1>>1+15)&$f)<<4
            _FRQ_216   = 1<<24|(10-1)<<18|(108-1)<<8|((1>>1+15)&$f)<<4
            _FRQ_240   = 1<<24|(10-1)<<18|(120-1)<<8|((1>>1+15)&$f)<<4
            _FRQ_250   = 1<<24|(10-1)<<18|(125-1)<<8|((1>>1+15)&$f)<<4
            _FRQ_260   = 1<<24|(20-1)<<18|(260-1)<<8|((1>>1+15)&$f)<<4
            _FRQ_300   = 1<<24|(10-1)<<18|(150-1)<<8|((1>>1+15)&$f)<<4
            _FRQ_360   = 1<<24|(10-1)<<18|(180-1)<<8|((1>>1+15)&$f)<<4
    '------------------------------------------------------------------------------
            _XOSC      = %10        ' OSC:    %00=OFF, %01=OSC, %10=15pF, %11=30pF
            _XSEL      = %11        ' XI+PLL: %00=rcfast(20+MHz), %01=rcslow(~20KHz), %10=XI(5ms), %11=XI+PLL(10ms)
            SYSCLOCK   = 180_000_000.0 ' Don't forget to set SYSCLOCK to match the above MHz rate being used!
    '------------------------------------------------------------------------------
    
    garryj
  • cgracey wrote: »
    Tubular wrote: »
    That'll be good, Chip

    Are you testing with an HID USB peripheral, or generated waveforms, or something else ?

    I'm just looking at waveforms on a scope and logic analyzer.

    @cgracey it looks like it might be self induced...
    garryj
  • The USB breakouts here might be handy for testing USB drivers:

    Limited stock !

    P2-ES Eval Board Accessory Set
    https://www.parallax.com/product/64006-es


    P2-ES Eval Protoboard Add-on
    https://www.parallax.com/product/64005-es
  • jmgjmg Posts: 14,088
    garryj wrote: »
    jmg wrote: »
    Is that working from a 4Mhz PFD ? (20M/5)
    No, I was using a 20MHz PFD, (20M/20).

    Below are the frequency settings I've been testing at. Now, before you gasp in horror, remember that I am not an EE :crazy:
    I modified _FRQ_180 to use 20M/5. It does indeed look like it could be self-induced jitter.

    I would sure appreciate you reviewing the below and making changes as needed to what you think might be the best values, and I will test.

    And maybe you could put together a brief "PLL for dummies" that explains each values effect for us non-EEs? I know I would sure appreciate it.
    That's a 1MHz PFD (20M/20 = 1), and that can certainly be easily improved to be 20M/5 (more flexible) or 20M/1 (best)


    The known issue with the P2 VCO/PLL, is that high values of _XDIV, are worse than lower ones.
    This can also have temperature zones, where jitter worsens, then improves as P2 warms.

    ie _XDIV = 1 is the best, giving the lowest jitter, and it should be used whenever possible, or when you may suspect jitter.
    _XIDV = 40, is very high, and that will contribute jitter.

    Hence my suggestion to use _XDIV = 1, and _XMUL = 6 for 120MHz or _XMUL = 12 for 240MHz least jitter test points.
    The P2D2 with a 12MHz Xi source, allows _XDIV = 1 for all 12MHz multiples.


    The PFD is the Phase Frequency Detector, which is Xi/_XDIV. That is the speed at which the PLL checks and corrects for lock.
    If you wanted to measure jitter, you could take the 1ms USB frame marker, and capture a counter difference.
    (that gives average time movement over a 1ms window, so is a little indirect, but it is easy to do)
    In an ideal world, that would be +/- 1 SysCLK, if we assume 0.25/12MHz is tolerable, that's 2.5 sysclks at 120MHz , or 5 sysclks at 240Mhz of variation.

  • Thank you @jmg, "the scales have been lifted from my eyes" :smile:

    I will be testing at 120, 180 and 240 shortly with:
            _FRQ_120   = 1<<24|(1-1)<<18|(6-1)<<8|((1>>1+15)&$f)<<4 
            _FRQ_180   = 1<<24|(5-1)<<18|(45-1)<<8|((1>>1+15)&$f)<<4
            _FRQ_240   = 1<<24|(1-1)<<18|(12-1)<<8|((1>>1+15)&$f)<<4
    
    garryj
Sign In or Register to comment.