FBLOCK feature possibly flawed
evanh
Posts: 16,029
I've bumped into what looks like a hardware flaw in the FIFO mechanics. I setup FBLOCK for splitting off the 8 bytes of CRC at the end of each block read from SD cards and found it didn't behave well at all.
Checks for software bugs found nothing. So I setup a stand alone test to verify what I'm seeing - Confirming the odd behaviour. HubRAM writes are significantly more jarring than reads but both have what I consider as broken behaviour.
Here's the core proof routine:
wrfast #2, buf1 ' setup FIFO, 2 x 64 = 128 bytes (32 longwords)
fblock #0, buf2 ' new buffer after 32 longwords
mov pa, #1
rep @.rend, #40 ' longwords
wflong pa
add pa, #1
.rend
The proof loops 40 times, incrementing PA as a counter and writing it to the FIFO. The first 32 writes get written to buf1, the remaining 8 writes get written to buf2. And that works as expected as long as buf1 and buf2 are a multiple of 64 bytes apart. Here's a print of those two buffers after a correct run:
&buff1=258c &buff2=2bcc buff1: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 buff2: 33 34 35 36 37 38 39 40 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
But if the buffers are off the multiple of 64-byte difference, by say 4 bytes (1 longword), then we get this:
&buff1=258c &buff2=2bd0 buff1: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 buff2: 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
And for good measure, an off by 2 longwords example:
&buff1=258c &buff2=2bd4 buff1: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 buff2: 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Comments
All code is written in C and built using Flexspin. I haven't seen any reason to suspect a bug with the compiler. The same routine is called for each case. Only the buf2 buffer address changes.
If you think about it, it makes sense that FBLOCK would require this block alignment, but that really just isn't documented and the behaviour is very strange.
My guess is that FBLOCK is intended for / was only tested with sets of buffers tightly packed to eachother, so the problem never came up.
Chip is quite clear in the docs that alignment only has to be on longword boundaries. I'm gonna need convincing that the observed behaviour is as intended. I seriously doubt it.
Actually, I'd like to think I've just got a software bug, but I don't think so.
Is this beneficial sometimes?
That's about the alignment of a single wrapping buffer.
If you imagine the FIFO writing at full tilt, one long per cycle, it's easy to see why at least 32-alignment relative between the buffers is required: the first long of the new buffer has to be on the RAM slice following the last long of the old buffer, otherwise the data stream is interrupted. The FIFO really assumes the blocks are 64 bytes (as they would be on a 16-slice hub), so that's why it's actually 64.
Definitely not! It's explicitly not doing as instructed, and this depends address delta. It's as useful as putting a relative branch as the last instruction of a REP loop.
The logic probably is like this:
// on every write cycle: curr_ptr += 1; if (curr_ptr&63 == wrap_ptr&63) { block_count -= 1; if (block_count == 0) { curr_ptr = wrap_ptr; block_count = wrap_blocks; } } // then do the actual write on curr_ptrOkay, that is convincing.
Okay, a workaround for me then is to allocate a larger CRC buffer so that it extends over all combinations of alignment. It was only 8 bytes, I guess I add 64 to that and make it's size 72 bytes.
Or I could stop the SD clock at the beginning of the CRC and bit-bash the last 16 bitsnibbles. Or start a fresh WRFAST + streamer + clock gen.
Or I could use a temporary buffer for both data block and CRC then later copy the data block to the requested hubRAM address.
That might not actually work still. The FAT layer will give you a pointer into client memory when it can (avoiding memcpy) and that pointer may not be long-aligned.
Longword aligned could be made a requirement of block level read/write. That's not unreasonable.
Does it works if there's some nops between block writes?
Seems like it might....
What else needs to be longword aligned? Didn't we used to need "long long" for something?
Or, maybe that got fixed at some point?
@evanh You are pretty good at finding these undocumented "features"... Keep it up!
It'll be wired for worst case of sysclock/1. Keeps the state machine simple. That proof is already at sysclock/4 and shows no sign of variability based on time, just addressing. Not that adding NOPs would be helpful even if they did work because I'm wanting to use FBLOCK for seamless streamer ops.
But, in a more general case where want to transfer two or more very large blocks where a few nops (or something else) in between wouldn't hurt, adding in nops would probably work, right?
No, makes no diff. The FIFO state machine will be fixed parameter. Ada made that clear to me.
PS: I just tested sysclock/6 (added one NOP in the REP loop) and got same results as opening post.
Evan, my point was you've discovered a way to write varying amounts of data to two buffers based on their address difference and is this potentially a useful feature? In your example using
WFLONGthe first buffer can store 17-32 longs and the second buffer the remainder. How aboutWFWORDandWFBYTE? (I've not had time to test this behaviour yet.)It's the FIFO's ordered accesses of the hubRAM slices. Ada described it succinctly. It's tuned for handling worst case full speed longwords at sysclock/1. Using different instructions, or differently timed instructions, or using the streamer, don't make any difference to this mechanism.
It would be nice if it was more flexible but it isn't.
PS: I've PM'd Chip asking him to write up a full description for the docs. And I guess confirm all this.
Well, I've ditch using FBLOCK for now. And resetting the streamer+FIFO for the 16 nibbles of the CRC is less code than trying to bit-bash it. So I guess that's going to be the approach.
I don't imagine resetting the streamer temporarily for redirecting the CRC is going to waste that many clocks. Performance should be fine. I know you're trying to keep the clock sustained for as much as possible which is laudable but probably in reality has diminishing returns after some point. We might be talking around a percent or so for this, right? For each sector transferred of 1024 data nibbles, (or P2 clocks at the fastest sysclock/1 transfer rates), a handful of extra P2 clocks to restart the streamer for the CRC portion of the transfer shouldn't be a significant proportion of this total. At sysclock/2 or slower it will be even less again.
You're poo-poo'ing my poo-poo'ing!
EDIT: Just the extra streamer handling has added 2% to overheads. Example run from history - https://forums.parallax.com/discussion/comment/1560351/#Comment_1560351
Very surprising to me that buf1 gets messed up by the second block not being 64 bytes away...
Must be some interesting mechanics inside the fifo control...
I've tested this behaviour and the buffers do not need to be long-aligned.
FBLOCKaddress[1:0] are ignored and theFBLOCKdata start at the byte within a long given byWRFASTaddress[1:0].Are you sure the last 0..3 bytes of the first buffer are actually written in that case?
Yes, if unaligned the last 1..3 bytes of first buffer's data are written to first 1..3 bytes of second buffer.
I guess they can just be moved over manually then.
WRFASTcan write only one long, depending on the buffer address difference:wrfast #1, buf1 'writes 1 long fblock #0, buf2 'buf2 = buf1 + 64N + 4 ... wrfast #1, buf1 'writes 2 longs fblock #0, buf2 'buf2 = buf1 + 64N + 8 ... wrfast #1, buf1 'writes 3 longs fblock #0, buf2 'buf2 = buf1 + 64N + 12 ... wrfast #1, buf1 'writes 16 longs, i.e. complete block of 64 bytes fblock #0, buf2 'buf2 = buf1 + 64N 'N = positive or negative integer 'buf1 [1:0] and buf2[1:0] are don't care 'and these bits are zeroed in equations aboveWow, yeah, totally does that. Contradicting Chip's write-up. Could be handy in the future.
I guess it's a feature of misaligned writes that's used for a plain WRLONG and the likes. It'll be wired in below the FIFO, at the eggbeater level I presume.
If the two buffers overlap then data written by
WRFASTcan be overwritten byFBLOCK, with long granularity. EDIT: This is obvious but it allows unwanted data to be discarded.Also, all of the following write the same data starting at buf1:
'T = Total longs written wrfast #1, buf1 'writes 1 long fblock #0, buf1+4 'writes T-1 longs ... wrfast #1, buf1 'writes 2 longs fblock #0, buf1+8 'writes T-2 longs ... wrfast #1, buf1 'writes 3 longs fblock #0, buf1+12 'writes T-3 longs ... wrfast #1, buf1 'writes 16 longs, i.e. complete block of 64 bytes fblock #0, buf1+64 'writes T-16 longsWhat if buf1 and buf2 are the same?