W5200 Changing Buffer Size

JohnC · 2013-09-04 10:17

Hi,

I'm working through the W5200 quickstart files and need to make some changes.

It appears that the default buffers are 2K each for rx/tx, on each of 8 available sockets.

For this application, I'm receiving large (~550 byte) UDP packets which arrive in groups of 4-6, spaced only a few microseconds apart. These packet bursts are repeat every 20 mS or so.

So the 2K buffers are overrunning and I'm losing packets.

When the dust settles, I'd love to have something like this:

Socket 0 TX 2K
Socket 0 RX 8K
Socket 1 TX 2K
Socket 1 RX 2K

I've made these changes to the constant section of W5200.spin:

{{ Socket Buffer Defaults }}
  INTERNAL_RX_BUFFER_ADDRESS    = $C000
  INTERNAL_TX_BUFFER_ADDRESS    = $8000
  DEFAULT_RX_TX_BUFFER          = $1000' $800
  DEFAULT_RX_TX_BUFFER_MASK     = DEFAULT_RX_TX_BUFFER - 1

SOCKETS           = 2 '8

... but it seems like I've missed something else.

Ideas?

John

Mike G · 2013-09-04 18:45

I dreaded the day someone would ask how to configure buffers other than the default 8 x 2k.

IMO, the W5200 API is challenging when dealing with the socket buffers. It is not enough to simply assign a value of 8k to a buffer. The host controller must know the boundaries of the W5200s circular buffers. Otherwise the controller will not know when a buffer rolls over. This situation creates a dependency between the logic running in the controller and the W5200. Your case is special as there's only enough buffer resource to create 3 sockets from a potential 8.

The key is reading the W5200 manual related to buffers. Reading the whole thing does not hurt...

Next, tackle the W5200 SPIN code. the snippet below comes from the CON, VAR and program sections of W5200.spin

  {{ Socket Buffer Defaults }}
  INTERNAL_RX_BUFFER_ADDRESS    = $C000
  INTERNAL_TX_BUFFER_ADDRESS    = $8000
  DEFAULT_RX_TX_BUFFER          = $800
  DEFAULT_RX_TX_BUFFER_MASK     = DEFAULT_RX_TX_BUFFER - 1

'''

  sockRxMem       byte  $02[SOCKETS]
  sockTxMem       byte  $02[SOCKETS]
  sockRxBase      word  INTERNAL_RX_BUFFER_ADDRESS[SOCKETS]
  sockRxMask      word  DEFAULT_RX_TX_BUFFER_MASK[SOCKETS]
  sockTxBase      word  INTERNAL_TX_BUFFER_ADDRESS[SOCKETS]
  sockTxMask      word  DEFAULT_RX_TX_BUFFER_MASK[SOCKETS]

...

'----------------------------------------------------
' Set defaults
'----------------------------------------------------     
PRI SetDefault2kRxTxBuffers | i

  repeat i from 0 to 7
    sockRxMem[i] := $02
    sockTxMem[i] := $02
    
  repeat i from 0 to 7
    sockRxMask[i] := DEFAULT_RX_TX_BUFFER_MASK
    sockTxMask[i] := DEFAULT_RX_TX_BUFFER_MASK

  repeat i from 1 to 7
    sockRxBase[i] := sockRxBase[i-1] + DEFAULT_RX_TX_BUFFER
    sockTxBase[i] := sockTxBase[i-1] + DEFAULT_RX_TX_BUFFER

  repeat i from 0 to 7
    WriteByte(GetSocketRegister(i, S_RX_MEM_SIZE) , sockRxMem[i])
    WriteByte(GetSocketRegister(i, S_TX_MEM_SIZE) , sockTxMem[i])

Looking at the SetDefault2kRxTxBuffers method

The first REPEAT loop assigns 2k to every buffer, Tx and Rx
The second REPEAT saves the buffer mask in an array. The buffer mask is the buffer size minus one.
The third REPEAT saves the starting address of each buffer.
The fourth REPEAT writes the buffer sizes, set in the first REPEAT, loop to the W5200

Unroll the REPEATs to create a customer buffer scheme.

PUB SetCustomBuffers
  '2 = 2048
  '8 = 8192
  '1 = 1024
                                                        'Size
  sockRxMem[0] := 2                                     '2k
  sockTxMem[0] := 8                                     '8k
  sockRxMem[1] := 2                                     '2k
  sockTxMem[1] := 2                                     '2k
  sockRxMem[2] := 1                                     '1k 
  sockTxMem[2] := 1                                     '1k
  
                                                        'Mask
  sockRxMask[0] := 2048-1
  sockTxMask[0] := 8192-1
  sockRxMask[1] := 2048-1
  sockTxMask[1] := 2048-1
  sockRxMask[2] := 1024-1
  sockTxMask[2] := 1024-1

  sockRxBase[0] := $C000                                'Sock 0 Rx buffer start
  sockTxBase[0] := $8000                                'Sock 0 Tx buffer start
  sockRxBase[1] := sockRxBase[0] + 2048                'Sock 1 start
  sockTxBase[1] := sockTxBase[0] + 8192                'Sock 1 start
  sockRxBase[2] := sockRxBase[1] + 1024                'Sock 2 start | No more Memory for socks
  sockTxBase[2] := sockTxBase[1] + 1024                'Sock 2 start | No more Memory for socks

  'Send Config
  WriteByte(GetSocketRegister(0, S_RX_MEM_SIZE) , sockRxMem[0])
  WriteByte(GetSocketRegister(0, S_TX_MEM_SIZE) , sockTxMem[0])
  WriteByte(GetSocketRegister(1, S_RX_MEM_SIZE) , sockRxMem[1])
  WriteByte(GetSocketRegister(1, S_TX_MEM_SIZE) , sockTxMem[1])
  WriteByte(GetSocketRegister(2, S_RX_MEM_SIZE) , sockRxMem[2])
  WriteByte(GetSocketRegister(2, S_TX_MEM_SIZE) , sockTxMem[2])

JohnC · 2013-09-04 18:51

Thanks, Mike, for your direction. I'll play with this tonight.

Just to confirm - in your last |code| section should the Masks and Bases be in $hex or plain decimal? I'd think that something like

SockRxMask[0] := 2048 - 1
SockTxMask[0] := 8192 - 1

...would be better?

Mike G · 2013-09-05 03:30

Correct, should be decimal not hex. I fixed post 2.

JohnC · 2013-09-05 09:29

As an update, last night I got everything running, based on the template above. To start with, I defined only Sock[0] with 4K and 2K receive / transmit buffers. I'll add Sock[1] for TCP / web access later in the project.

Everything was running along nicely, and I was receiving at ~ 86 Kbytes per second, sustained, over about half an hour. So that made me happy. I was flipping a pin after a packet was received, then again after it was processed (basically, error checked, then copied to a different local buffer). Averaged out to .5 mS per packet. So the main SPIN loop seems to load the cog to about 15% of capacity. Lots of room to expand here.

Occasionally (maybe representing 1-2% of all packets received) when calling Sock.Available, returned would be a number representing a partial packet. And then, the next call to .Available would return the balance due. If the correct packet size was 550 bytes, I'd receive 518 and then 32, for example. Data isn't being lost, it's just packaged oddly.

Wireshark confirms that every packet being sent the Wiznet chip is the exact same length. So I need to do more poking around to see what might be causing this. For now, if .available returns an odd number, I just dump the current and subsequent packet, then continue on.

For reference, the RX buffer is set to 4K, and the maximum payload within one of the 'bursts' I ever receive is four packets comprising about 2400 bytes.

Is this sort of packet-splitting behavior something to be expected?

Mike G · 2013-09-05 10:10

Is this sort of packet-splitting behavior something to be expected?

Yes, as far as I remember. Be careful correlating packet sizes with the current state of the W5200 internal buffer pointers. I believe the W5200 manual has information, albeit brief, regarding this kind of situation. IIRC, the answer is keep grabbing data until done. Where "done" is up to the host program.

I had a similar issue downloading large files (20M) using TCP.

twc · 2013-09-05 11:04

...I encountered a similar situation. The fact sock.Available is non-zero does not mean an entire packet has been received. The issue becomes more apparent in an application that just receives data as fast as possible and doesn't spend much time processing it (like your test, and unlike most real apps). You can try introducing a 'processing' delay between checking for receive packets or checking sock.Available twice for a stable count.

JohnC · 2013-09-05 11:57

Thanks, TWC. Until now it didn't occur to me to hover for half a mS or so until .available resolves itself.

Mike G · 2013-09-06 04:16

Socket.Available reads the socket(n) receive size register. Socket.Available also blocks while waiting for data. The timeout is very short and derived through experimentation.

Socket.DataReady also reads the socket(n) receive size register but does NOT block.

...until .available resolves itself.

Socket.Available does not resolve itself.

Socket.Available and Socket.DataReady report the bytes in the Rx buffer. The methods do not guarantee an intact packet framed in the buffer. It's possible to read once and see a partial packet. Read again and see one full packet and the beginning of the next packet.

W5200 Changing Buffer Size

Comments