@evanh said:
You brought up the read-modify-write topic as I was finishing up my demo code - That's something that is new because of the EC32MB wiring isn't it. Because it can only be written 32-bit at a time, right?
Yes. If it's wired as a 16 bit bus then with two cycles per write, only a complete 32 bit long should be written at a time. So to avoid data corruption in the memory both upper and lower 16 bits need to be written whenever a write is performed, and this necessitates a read first. You can't start writing half way into the long, you need to start from the beginning after CE is brought low, and there is no data qualifier brought out (unlike the OPI PSRAM).
As for any of the possible multiple PSRam configurations based on single (and commom to all devices) /CE and SCLK signals, the most atractive way to achieve Selective Write operations (either Overwrtie or Read-Modify_Write ones), would be to rely on the reservation of either the first or the last row, of each and every memory chip involved, in order to serve as a "dummy" scratchpad area.
Obviouslly, only the targeted chip within each group would receive the proper addressing information, while the other ones would be directed to proceed the "dummy" write operation at the scratchpad row.
Also the requested addressing information to be accessed would always need to be incremented by one (in case "Row Zero" was selected to be the "trash-blender"), or limited to the maximum available row number of each PSRam chip minus one (otherwise).
I did also considered the option of putting the non-targeted chips into Fast Read mode, but this would only complicate things, due to the forcefully interveining Wait-period, and the unnavoidable "noisy and power-hungry" other PSRam databuses, agressivelly-whipping into P2 inputs, that, sure, would need to be also reprogrammed to act as so.
@Yanomani The problem is not that we want to stop a write to a single chip, we want a way to stop the paired nibble in the same chip from needing to be re-written each time we update its partner nibble's contents. Without RWDS we can't do that and have to do a RMW. And a RMW of a data bus subset saves no time vs RMW of the full bus width (streamer has same performance in both cases), so no need to try to filter writes to individual chips.
Then I missed the point (as usual... ); PSRam R/W operations are byte-granular by definition. Similarly, Hyper's (and Xccela OPI's) R/W operations are word-granular.
@rogloh said:
Are you planning on a pair of OPI PSRAMs, or a pair of HyperRAMs?
I'm thinking in terms of a revB EC32MB. RevA was so close to getting DDR I was surprised disappointed it never happened at the time. Maybe it was the awkward FBGA package that was the deciding factor.
EDIT: OPI is the clear preference for Prop2. HyperRAMs are more suited to a hardware memory controller.
I could try hacking a capacitor in, it should work at low frequencies at least, ...
EDIT: 6.8 pF is not enough. It works 100% for only the best case config of last chip of the 48 MB add-on when plugged into P8..P15.
EDIT2: 8.2 pF improves all ... And just changed, from glob-top, to my newer production packaged ES revB Eval Board and it is a little better even. But still need better ...
@Yanomani said:
Then I missed the point (as usual... ); PSRam R/W operations are byte-granular by definition. Similarly, Hyper's (and Xccela OPI's) R/W operations are word-granular.
They are byte granular and use two 4 bit transfers to write them in their 4 bit bus arrangement. However when you combine them with multiple devices ganged together to widen the bus, then the second nibble in the pair (traditionally storing that single byte in a single chip) actually stores a different byte in the long now so you need a RMW operation if you want to preserve it whenever writing to either nibble portion independently for smaller than long sized elements (in 4x4 bit PSRAM widths), or for any unaligned longs at start and end of transfers.
Sometimes it seems necessary to perform many "head-shakes", before things can settle deep enough into my head (without having to resort to a hammer)...
Check this out - same Eval Board, production ES revB, same 48 MB 1x4-bit add-on board, same accessory header, same 22 pF capacitor on the clock pin, code the same except for the chip-select. Each of the three chips:
Basically want to avoid using a capacitor. It's a hack workaround that stops working above 10 pF.
Here's the reports I gathered for the worse case accessory header - P32..P39. And what I thought was the worst case chip, but may not have been correct on that count. Chip select P39 is breaking up much worse at larger capacitor values.
Forget the capacitor, it's useless and you are better off without it. The problem I think is the total load on the data bus with too many chips in parallel is distorting the data signals too much. Workaround is to clock at sysclk/3. A tighter layout might also be helpful to eliminate some skew differences between devices but is not guaranteed to resolve the underlying issue which is that we have too many devices in parallel (6 chips!).
There's possibly lots of reflections going on too. I imagine the bus voltage waveform is quite messy.
Oh, it's not as bad as that seemed, that particular test was for simulating sysclock/1 as if they were DDR parts. The CLK_ADV = 1 made the TX timing very tight. Ie: 1.0 ns tight. So, in one sense I was seeing if that 24 MB board layout could be tuned to fit. I wasn't surprised that it can't. There is likely a number of layout limitations for why ... but, yeah, reaching for the capacitor is poor fix. And certainly limited.
EDIT: Err, it's the 24 MB add-on. There is no 48 MB add-on. Duh, sorry, hadn't thought much on that detail.
I didn't bother testing the 96 MB add-on with this CLK_ADV = 1 partly because it needed two capacitors of each value fitted, and partly because the 24 MB was poorly performing anyway.
There's something to be said about series resistors on high speed transmission lines. I didn't believe it myself until in a hail Mary I added one on the CS signal on my rats nest hyperram a few years ago.
From time to time, the old gray beards say that adding capacitance never solves the timing issue, but I can't seem to get an explanation of why, other than "it just doesn't work."
Hmm, yeah, ground bounce is a likely candidate. Resistors would soften the switching edges. I'm not happy with the tiny track a single via for the header's ground pins of the 24 MB add-on. All the more reason to do a fresh layout with OPI parts.
That's a valid argument for having those header drill holes with pads. I think it was Vons that suggested doing that. It provides a substantial increase in the surface width of ground conductors. They act as an oversized via, which provides a lot more vertical surface through the PCB.
Thanks for these drivers. I have added your PSRAM driver as a Catalina plugin so I can use it from C. Still testing, but it appears to work very well, so I plan to include it in the next Catalina release.
As a real world demonstration of using it, I have also modified Lua to execute programs from PSRAM. I cache the PSRAM so the speed degradation is acceptable, and this allows larger Lua programs to be executed.
So far I have only added the PSRAM driver, but I would also like to add the SRAM and Hyper RAM drivers.
But I have a request - in your next release could you use local PASM labels instead of global ones where possible, and could you use different names for those names that must be visible outside the PASM (e.g. PASM_driver_start and SRAM_driver_start instead of just driver_start in both). This wouldn't matter to your Spin code, but it would make it easier to use more than one of your drivers in the same C program.
@RossH said:
But I have a request - in your next release could you use local PASM labels instead of global ones where possible, and could you use different names for those names that must be visible outside the PASM (e.g. PASM_driver_start and SRAM_driver_start instead of just driver_start in both). This wouldn't matter to your Spin code, but it would make it easier to use more than one of your drivers in the same C program.
Hi, yeah exactly how the next release proceeds will probably also depend on what #define/#ifdef and #include features Chip adds to the official Spin compiler. I will likely want to keep the shared names for entry points if #include files are used so we don't have different driver names and can drop in different files depending on which driver is selected for example.
That being said I might be able to address specific concerns if it's not a major overhaul to the code. Do you have a problem in Catalina because you are sharing label names from different PASM blocks (i.e. every PASM label is a from a shared global namespace) or because you want to merge multiple PASM blocks into the same source file? I'm not totally sure how your actual issue arises here as I don't use Catalina. Could you not also use the preprocessor with #if blocks in C to resolve your issue or is it much deeper than that?
@rogloh said:
That being said I might be able to address specific concerns if it's not a major overhaul to the code. Do you have a problem in Catalina because you are sharing label names from different PASM blocks (i.e. every PASM label is a from a shared global namespace) or because you want to merge multiple PASM blocks into the same source file? I'm not totally sure how your actual issue arises here as I don't use Catalina.
Catalina merges PASM drivers from multiple sources into a single PASM file, which it assembles to provide the runtime environment for the C program (which is compiled separately).
Ok, sounds like it's worse then. Common variable names would be problematic too (same label for those too as well as the code branch labels) and given I made up many driver variants from a common template, sharing variable names is going to be very common in the PASM code right now. Ideally they'd be compiled as different files/blocks with their own namespaces but if you copy into a common single PASM file and don't use #if to select only only one active driver for example I can see this will be an issue. You are sort of running into similar problems as I am with PNut, in trying to come up with a usable driver framework that lets different driver code be shared/selected without issues and with minimal maintenance needed. I've not solved it yet and have been waiting for this #if stuff to arrive before proceeding, hoping that will help out.
From memory I think 1024x768 can be achieved with a single chip in 8bpp over VGA. For DVI/HDMI you can do 800x600 at 60Hz with reduced blanking. Higher HDMI resolutions can be achieved but at lower refresh. For 4 bit PSRAM the total memory bandwidth is P2 clock rate /4 minus some overheads (roughly 10-15% or more is probably going to be lost). So if you run the P2 at 200MHz for example you'd probably sustain 40-45MB/s from a single PSRAM. This could do 800x600 at 8bpp. Performance depends on the burst size setting too and how that relates to the number of bytes on a scan line. Best performance is when you are able to transfer a full burst in the 8us CS low time and also if the total bytes per scan line is some integer number of complete bursts. This minimizes the data transfer setup overhead.
No. /CE has to go high between accesses to allow refresh to continue properly. Max is 8 microseconds low IIRC.
@rogloh said:
2. No. /CE has to go high between accesses to allow refresh to continue properly. Max is 8 microseconds low IIRC.
CE is also needed because there is no way to stop a transfer in-band. If you keep CE low it'll just keep reading/writing from incrementing addresses. A falling CE edge is needed to reset the state machine such that it accepts a new command.
Hey Roger, messing around with PSRAM as video framebuffer...
EDIT: Ignore, I made a silly typo
When running 640x480 RGB24, i'm getting glitches (and I think oddly low performance?) with the 16 bit driver, but it's fine with the 8 bit driver.
I'm using my custom spin code and set it up like this:
case MEMORY_TYPE
TYPE_PSRAM16:
exmem_size := 32*1024*1024 * PSRAM_BANKS
sizelen := encod(exmem_size-1)+1
repeat i from 0 to PSRAM_BANKS-1
exmem_banks[0+i*2] := 2048<<16 + PSRAM_DELAY<<12 + sizelen
exmem_banks[1+i*2] := 2048<<16 + PSRAM_DELAY<<12 + sizelen
exmem_banks[16+i*2] := PSRAM_SELECT+i + PSRAM_CLK<<8
exmem_banks[17+i*2] := PSRAM_SELECT+i + PSRAM_CLK<<8
i := drv16.getDriverAddr()
TYPE_PSRAM8:
exmem_size := 16*1024*1024 * PSRAM_BANKS
sizelen := encod(exmem_size-1)+1
repeat i from 0 to PSRAM_BANKS-1
exmem_banks[0+i] := 1024<<16 + PSRAM_DELAY<<12 + sizelen
exmem_banks[16+i] := PSRAM_SELECT+i + PSRAM_CLK<<8
i := drv8.getDriverAddr()
TYPE_PSRAM4:
exmem_size := 8*1024*1024 * PSRAM_BANKS
sizelen := encod(exmem_size-1)+1
repeat i from 0 to (PSRAM_BANKS-1)/2
exmem_banks[0+i] := 512<<16 + PSRAM_DELAY<<12 + sizelen
exmem_banks[16+i] := PSRAM_SELECT+i*2 + PSRAM_CLK<<8 + (PSRAM_SELECT+i*2+(((PSRAM_BANKS&1)&&(i>=PSRAM_BANKS/2)) ? 0 : 1))<<16
i := drv4.getDriverAddr()
I'm using the driver version with sys/3 support, but I'm running sys/2 (at least I'm supposed to)
EDIT: Ignore this also.
Also, as far as I can tell, getting the video driver to properly work in an interlaced mode is impossible because it ignores skew in external regions and also the alt-source doesn't seem to ever come from external memory.
@Rayman said:
@rogloh Is version in top post the latest?
It should be the latest officially released code, yes. Although there may also be some custom feature or other test variants provided to people like Wuerfel_21 for your 24/96MB boards floating about somewhere in these threads. Some of that may make it into another release at some point.
After accounting for other factors, maybe. But it still shifts with temperature and board design and we are dealing with very narrow bounds that it seems runtime retuning is inevitable.
Or we back off from the cliff and aim for lower bandwidth.
Comments
As for any of the possible multiple PSRam configurations based on single (and commom to all devices) /CE and SCLK signals, the most atractive way to achieve Selective Write operations (either Overwrtie or Read-Modify_Write ones), would be to rely on the reservation of either the first or the last row, of each and every memory chip involved, in order to serve as a "dummy" scratchpad area.
Obviouslly, only the targeted chip within each group would receive the proper addressing information, while the other ones would be directed to proceed the "dummy" write operation at the scratchpad row.
Also the requested addressing information to be accessed would always need to be incremented by one (in case "Row Zero" was selected to be the "trash-blender"), or limited to the maximum available row number of each PSRam chip minus one (otherwise).
I did also considered the option of putting the non-targeted chips into Fast Read mode, but this would only complicate things, due to the forcefully interveining Wait-period, and the unnavoidable "noisy and power-hungry" other PSRam databuses, agressivelly-whipping into P2 inputs, that, sure, would need to be also reprogrammed to act as so.
@Yanomani The problem is not that we want to stop a write to a single chip, we want a way to stop the paired nibble in the same chip from needing to be re-written each time we update its partner nibble's contents. Without RWDS we can't do that and have to do a RMW. And a RMW of a data bus subset saves no time vs RMW of the full bus width (streamer has same performance in both cases), so no need to try to filter writes to individual chips.
Then I missed the point (as usual...
); PSRam R/W operations are byte-granular by definition. Similarly, Hyper's (and Xccela OPI's) R/W operations are word-granular.
I'm thinking in terms of a revB EC32MB. RevA was so close to getting DDR I was surprised disappointed it never happened at the time. Maybe it was the awkward FBGA package that was the deciding factor.
EDIT: OPI is the clear preference for Prop2. HyperRAMs are more suited to a hardware memory controller.
@evanh I think I put a place for an extra capacitor on one of these boards to delay the clock signal (at your request ?).
Get a chance to try that?
Oops, remembered wrongly... It's the 8bit chip boards that have that...
I could try hacking a capacitor in, it should work at low frequencies at least, ...
EDIT: 6.8 pF is not enough. It works 100% for only the best case config of last chip of the 48 MB add-on when plugged into P8..P15.
EDIT2: 8.2 pF improves all ... And just changed, from glob-top, to my newer production packaged ES revB Eval Board and it is a little better even. But still need better ...
They are byte granular and use two 4 bit transfers to write them in their 4 bit bus arrangement. However when you combine them with multiple devices ganged together to widen the bus, then the second nibble in the pair (traditionally storing that single byte in a single chip) actually stores a different byte in the long now so you need a RMW operation if you want to preserve it whenever writing to either nibble portion independently for smaller than long sized elements (in 4x4 bit PSRAM widths), or for any unaligned longs at start and end of transfers.
Thanks @rogloh !
Sometimes it seems necessary to perform many "head-shakes", before things can settle deep enough into my head (without having to resort to a hammer)...
Check this out - same Eval Board, production ES revB, same 48 MB 1x4-bit add-on board, same accessory header, same 22 pF capacitor on the clock pin, code the same except for the chip-select. Each of the three chips:
What blows me away is the spread of percentages across single lines. It's just falling to pieces. The lower capacitor values were better.
Basically want to avoid using a capacitor. It's a hack workaround that stops working above 10 pF.
Here's the reports I gathered for the worse case accessory header - P32..P39. And what I thought was the worst case chip, but may not have been correct on that count. Chip select P39 is breaking up much worse at larger capacitor values.
Forget the capacitor, it's useless and you are better off without it. The problem I think is the total load on the data bus with too many chips in parallel is distorting the data signals too much. Workaround is to clock at sysclk/3. A tighter layout might also be helpful to eliminate some skew differences between devices but is not guaranteed to resolve the underlying issue which is that we have too many devices in parallel (6 chips!).
There's possibly lots of reflections going on too. I imagine the bus voltage waveform is quite messy.
Oh, it's not as bad as that seemed, that particular test was for simulating sysclock/1 as if they were DDR parts. The CLK_ADV = 1 made the TX timing very tight. Ie: 1.0 ns tight. So, in one sense I was seeing if that 24 MB board layout could be tuned to fit. I wasn't surprised that it can't. There is likely a number of layout limitations for why ... but, yeah, reaching for the capacitor is poor fix. And certainly limited.
EDIT: Err, it's the 24 MB add-on. There is no 48 MB add-on. Duh, sorry, hadn't thought much on that detail.
I didn't bother testing the 96 MB add-on with this CLK_ADV = 1 partly because it needed two capacitors of each value fitted, and partly because the 24 MB was poorly performing anyway.
There's something to be said about series resistors on high speed transmission lines. I didn't believe it myself until in a hail Mary I added one on the CS signal on my rats nest hyperram a few years ago.
From time to time, the old gray beards say that adding capacitance never solves the timing issue, but I can't seem to get an explanation of why, other than "it just doesn't work."
Hmm, yeah, ground bounce is a likely candidate. Resistors would soften the switching edges. I'm not happy with the tiny track a single via for the header's ground pins of the 24 MB add-on. All the more reason to do a fresh layout with OPI parts.
That's a valid argument for having those header drill holes with pads. I think it was Vons that suggested doing that. It provides a substantial increase in the surface width of ground conductors. They act as an oversized via, which provides a lot more vertical surface through the PCB.
EDIT: Ah, and Roger's add-ons did exactly that with great results - https://forums.parallax.com/discussion/comment/1540379/#Comment_1540379
Hi @rogloh
Thanks for these drivers. I have added your PSRAM driver as a Catalina plugin so I can use it from C. Still testing, but it appears to work very well, so I plan to include it in the next Catalina release.
As a real world demonstration of using it, I have also modified Lua to execute programs from PSRAM. I cache the PSRAM so the speed degradation is acceptable, and this allows larger Lua programs to be executed.
So far I have only added the PSRAM driver, but I would also like to add the SRAM and Hyper RAM drivers.
But I have a request - in your next release could you use local PASM labels instead of global ones where possible, and could you use different names for those names that must be visible outside the PASM (e.g. PASM_driver_start and SRAM_driver_start instead of just driver_start in both). This wouldn't matter to your Spin code, but it would make it easier to use more than one of your drivers in the same C program.
Ross.
Hi, yeah exactly how the next release proceeds will probably also depend on what #define/#ifdef and #include features Chip adds to the official Spin compiler. I will likely want to keep the shared names for entry points if #include files are used so we don't have different driver names and can drop in different files depending on which driver is selected for example.
That being said I might be able to address specific concerns if it's not a major overhaul to the code. Do you have a problem in Catalina because you are sharing label names from different PASM blocks (i.e. every PASM label is a from a shared global namespace) or because you want to merge multiple PASM blocks into the same source file? I'm not totally sure how your actual issue arises here as I don't use Catalina. Could you not also use the preprocessor with #if blocks in C to resolve your issue or is it much deeper than that?
Catalina merges PASM drivers from multiple sources into a single PASM file, which it assembles to provide the runtime environment for the C program (which is compiled separately).
Ross.
Ok, sounds like it's worse then. Common variable names would be problematic too (same label for those too as well as the code branch labels) and given I made up many driver variants from a common template, sharing variable names is going to be very common in the PASM code right now. Ideally they'd be compiled as different files/blocks with their own namespaces but if you copy into a common single PASM file and don't use #if to select only only one active driver for example I can see this will be an issue. You are sort of running into similar problems as I am with PNut, in trying to come up with a usable driver framework that lets different driver code be shared/selected without issues and with minimal maintenance needed. I've not solved it yet and have been waiting for this #if stuff to arrive before proceeding, hoping that will help out.
Using local names where possible solves most of the problem. The rest (duplicate names for program entry points) is easy enough to fix manually.
Locals are just names with a colon prefixed aren't they? Or is it a dot, I can't recall right now?
I think it is a dot. But that section of the PASM manual is still being written!
A couple of questions about the PSRAM chip that is used on P2 Edge Module with 32MB RAM:
From memory I think 1024x768 can be achieved with a single chip in 8bpp over VGA. For DVI/HDMI you can do 800x600 at 60Hz with reduced blanking. Higher HDMI resolutions can be achieved but at lower refresh. For 4 bit PSRAM the total memory bandwidth is P2 clock rate /4 minus some overheads (roughly 10-15% or more is probably going to be lost). So if you run the P2 at 200MHz for example you'd probably sustain 40-45MB/s from a single PSRAM. This could do 800x600 at 8bpp. Performance depends on the burst size setting too and how that relates to the number of bytes on a scan line. Best performance is when you are able to transfer a full burst in the 8us CS low time and also if the total bytes per scan line is some integer number of complete bursts. This minimizes the data transfer setup overhead.
No. /CE has to go high between accesses to allow refresh to continue properly. Max is 8 microseconds low IIRC.
CE is also needed because there is no way to stop a transfer in-band. If you keep CE low it'll just keep reading/writing from incrementing addresses. A falling CE edge is needed to reset the state machine such that it accepts a new command.
Hey Roger, messing around with PSRAM as video framebuffer...
EDIT: Ignore, I made a silly typo
When running 640x480 RGB24, i'm getting glitches (and I think oddly low performance?) with the 16 bit driver, but it's fine with the 8 bit driver.
I'm using my custom spin code and set it up like this:
case MEMORY_TYPE TYPE_PSRAM16: exmem_size := 32*1024*1024 * PSRAM_BANKS sizelen := encod(exmem_size-1)+1 repeat i from 0 to PSRAM_BANKS-1 exmem_banks[0+i*2] := 2048<<16 + PSRAM_DELAY<<12 + sizelen exmem_banks[1+i*2] := 2048<<16 + PSRAM_DELAY<<12 + sizelen exmem_banks[16+i*2] := PSRAM_SELECT+i + PSRAM_CLK<<8 exmem_banks[17+i*2] := PSRAM_SELECT+i + PSRAM_CLK<<8 i := drv16.getDriverAddr() TYPE_PSRAM8: exmem_size := 16*1024*1024 * PSRAM_BANKS sizelen := encod(exmem_size-1)+1 repeat i from 0 to PSRAM_BANKS-1 exmem_banks[0+i] := 1024<<16 + PSRAM_DELAY<<12 + sizelen exmem_banks[16+i] := PSRAM_SELECT+i + PSRAM_CLK<<8 i := drv8.getDriverAddr() TYPE_PSRAM4: exmem_size := 8*1024*1024 * PSRAM_BANKS sizelen := encod(exmem_size-1)+1 repeat i from 0 to (PSRAM_BANKS-1)/2 exmem_banks[0+i] := 512<<16 + PSRAM_DELAY<<12 + sizelen exmem_banks[16+i] := PSRAM_SELECT+i*2 + PSRAM_CLK<<8 + (PSRAM_SELECT+i*2+(((PSRAM_BANKS&1)&&(i>=PSRAM_BANKS/2)) ? 0 : 1))<<16 i := drv4.getDriverAddr()I'm using the driver version with sys/3 support, but I'm running sys/2 (at least I'm supposed to)
EDIT: Ignore this also.
Also, as far as I can tell, getting the video driver to properly work in an interlaced mode is impossible because it ignores skew in external regions and also the alt-source doesn't seem to ever come from external memory.
@rogloh Is version in top post the latest?
It should be the latest officially released code, yes. Although there may also be some custom feature or other test variants provided to people like Wuerfel_21 for your 24/96MB boards floating about somewhere in these threads. Some of that may make it into another release at some point.
Any chance of defining delay in terms of nanoseconds and having some equations doing the rest?
After accounting for other factors, maybe. But it still shifts with temperature and board design and we are dealing with very narrow bounds that it seems runtime retuning is inevitable.
Or we back off from the cliff and aim for lower bandwidth.