SPI ram

Dr_Acula · 2013-04-20 22:56

Attached is the config file for the 23LC1024 spi ram chip. (configured to run on propeller pins 4,5,6,7)

I understand there are some clever ways of telling GCC where to store data and it is even possible to store some data in hub and some in external ram. I think this is a split mode.

However, the only option that will compile and run is XMMC.

Is there a way of tweaking this file so it can run in split mode as well? Any help would be most appreciated

David Betz · 2013-04-21 04:07

Dr_Acula wrote: »

Attached is the config file for the 23LC1024 spi ram chip. (configured to run on propeller pins 4,5,6,7)

I understand there are some clever ways of telling GCC where to store data and it is even possible to store some data in hub and some in external ram. I think this is a split mode.

However, the only option that will compile and run is XMMC.

Is there a way of tweaking this file so it can run in split mode as well? Any help would be most appreciated

If you only have external RAM you want to use xmm-single not xmm-split. The xmm-split memory model is intended to be used where you have both flash and SRAM and want to put code in the flash and data in the SRAM. It won't work if you only have SRAM.

Dr_Acula · 2013-04-21 17:44

Ah, thanks David. I'll give it a go with xmm-single.

Dr_Acula · 2013-04-27 00:02

Hi David (and anyone else!),

Ok, I tried it with xmm-single and it doesn't do anything. Compiles and downloads fine but the program doesn't run. Same program in XMMC mode works fine.

Am I doing something wrong? Is there a setting that needs changing in the config file? Or is this a "hard" problem to solve?

(My little coding experiment - I think it might be possible to build a faster CP/M emulator but for that it needs to use the 24LC1024 chip to store both the C code and also set aside a 64k block for the 8080/Z80 ram. I want at least half the propeller ram devoted to a graphics buffer)

David Betz · 2013-04-27 04:36

Dr_Acula wrote: »

Hi David (and anyone else!),

Ok, I tried it with xmm-single and it doesn't do anything. Compiles and downloads fine but the program doesn't run. Same program in XMMC mode works fine.

Am I doing something wrong? Is there a setting that needs changing in the config file? Or is this a "hard" problem to solve?

(My little coding experiment - I think it might be possible to build a faster CP/M emulator but for that it needs to use the 24LC1024 chip to store both the C code and also set aside a 64k block for the 8080/Z80 ram. I want at least half the propeller ram devoted to a graphics buffer)

What program are you trying to run? Will all of the code and the data fit in the external SRAM? You might be running out of space in xmm-single mode. This is one place where supporting all of these memory models makes things difficult. The linker doesn't know how much external memory you have so it doesn't warn you if your program is too big. You have to use propeller-elf-objdump to figure out how much memory you're using if you think you're running out.

jazzed · 2013-04-27 08:43

Can your run a smaller single program like hello.c using xmm-single ?

Dr_Acula · 2013-04-27 17:08

Thanks guys!

Indeed I may be running out of space. It works fine for a Hello World in both XMMC and XMMC-Single. But the larger program is not running in XMMC-Single.

Only thing is that the program that isn't working is only a little bit bigger:

Project Directory: C:/Propeller/Centimanes/TestC/

propeller-elf-c++ -o a.out -Os -mxmmc -I . -m32bit-doubles -fno-exceptions -fno-rtti cog.h propeller.h tv_driver.cpp tv_text.cpp Test.cpp
propeller-elf-objdump -h a.out
Done. Build Succeeded!

propeller-load.exe -I C:/propgcc/propeller-load/ -b 23LC1024 -p COM6 a.out -rLoading the serial helper to hub memory

9528 bytes sent

Verifying RAM ... Loading cache driver 'spi_sram24_cache.dat'

1196 bytes sent             
Loading program image to RAM
3832 bytes sent             
Loading .xmmkernel
1720 bytes sent

The 'main' part of the program is just creating an array and printing a value from it

#include <propeller.h>
#include "Test.h"

#ifdef __GNUC__
#define INLINE__ static inline
#define Yield__() __asm__ volatile( "" ::: "memory" )
#define PostEffect__(X, Y) __extension__({ int32_t tmp__ = (X); (X) = (Y); tmp__; })
#else
#define INLINE__ static
static int32_t tmp__;
#define PostEffect__(X, Y) (tmp__ = (X), (X) = (Y), tmp__)
#define Yield__()
#endif

int32_t TestSpin::Main(void)
{
  Text.Start(16);
  Text.Clearscreen();
  Text.Str((int32_t)"Test memory");
  Text.Out(10);
  Text.Out(13);
  Myarray[5] = 20;
  Text.Dec(Myarray[5]);
  while (1) {
    Yield__();
  }
  return 0;
}


TestSpin MainObj__;

int main() {
  return MainObj__.Main();
}

It is a Spin program that has gone through spin2cpp, maybe that is an issue?

David Betz · 2013-04-27 19:30

Your example is building with -mxmmc but I tried it with -mxmm-single and the program certainly isn't too big. What happens when you try to run it in xmm-single mode? What kind of hardware would I need to run it? Should it work on the C3 with the appropriate cache driver?

Dr_Acula · 2013-04-27 22:34

Thanks David for your help here. Ok, xmmc and xmm-single both compile and download fine, but in xmm-single the program does not run.

I may need to strip it back even further to try to find the bug. Maybe there are things going on behind the scenes with the tv text driver after it runs through spin2cpp and it is putting the TV buffer into external ram instead of hub ram? Maybe it is actually running, but it is the TV output that is not working? So I need to simplify it more. A "Hello World" using the serial port and stdio works fine, so maybe I need to add that for debugging purposes, and then add in the TV part?

My hardware is very simple at the moment - 4 pins to talk to the sram, and 3 pins to a TV.

Re common hardware, do any of the commercial boards out there use the 23LC1024?

Rayman · 2013-04-28 04:16

My "FlashPoint Mini" board has the 23LC1024 plus a 32 Mbit SPI flash chip...
I'd like to have a driver for it, so I'll have to see what you have here...

BTW: If you're doing an 8080 emulator, I'd like to see that too.
I've tried getting the Toledo one going, but no luck so far...

David Betz · 2013-04-28 04:58

Rayman wrote: »

My "FlashPoint Mini" board has the 23LC1024 plus a 32 Mbit SPI flash chip...
I'd like to have a driver for it, so I'll have to see what you have here...

BTW: If you're doing an 8080 emulator, I'd like to see that too.
I've tried getting the Toledo one going, but no luck so far...

You should certainly be able to use the spi_sram24_cache.dat cache driver that Dr_Acula is using on your FlashPoint Mini board but it will, of course, only access the SRAM chip. The only thing you have to change are the pin settings in the .cfg file.

David Betz · 2013-04-28 05:01

Dr_Acula wrote: »

Thanks David for your help here. Ok, xmmc and xmm-single both compile and download fine, but in xmm-single the program does not run.

I may need to strip it back even further to try to find the bug. Maybe there are things going on behind the scenes with the tv text driver after it runs through spin2cpp and it is putting the TV buffer into external ram instead of hub ram? Maybe it is actually running, but it is the TV output that is not working? So I need to simplify it more. A "Hello World" using the serial port and stdio works fine, so maybe I need to add that for debugging purposes, and then add in the TV part?

My hardware is very simple at the moment - 4 pins to talk to the sram, and 3 pins to a TV.

Re common hardware, do any of the commercial boards out there use the 23LC1024?

You may have something with the TV driver. My guess is that the COG image for that driver gets put in the .data section which is in external memory in xmm-single mode. Since coginit can't load a driver from external memory the driver load will fail and you won't see any image on your TV. Try changing this line in tv_driver.cpp:

uint8_t tv_driver::dat[] = {

To this:

HUBDATA uint8_t tv_driver::dat[] = {

David Betz · 2013-04-28 05:47

David Betz wrote: »
You may have something with the TV driver. My guess is that the COG image for that driver gets put in the .data section which is in external memory in xmm-single mode. Since coginit can't load a driver from external memory the driver load will fail and you won't see any image on your TV. Try changing this line in tv_driver.cpp:
uint8_t tv_driver::dat[] = {
To this:
HUBDATA uint8_t tv_driver::dat[] = {

I just realized that this is insufficient. Since the TV driver knows nothing about external memory, everything that it has to access has to be in hub memory. That includes not only the COG image itself but also the parameters passed to it in PAR as well as its mailbox if it uses one and the frame buffer itself. All of this has to be in hub memory for the TV driver or any other driver to function properly.

Dr_Acula · 2013-04-28 07:02

That makes sense but it sounds complicated. It is a new way of thinking. With Spin, all cog data loads from hub. With some of the clever loaders we played with a few years back, we could load cog images off an SD card into the same 2k block of hub, then load them into the cogs and hence save a whole lot of hub ram.

C opens up more possibilities, but it may not be possible to mindlessly copy spin objects and run them through spin2cpp.

The vast majority of programs only load cogs once. So maybe there is a way of loading up all the cogs first, then overwrite all that hub ram with something else? Having said that, Spin wasn't really designed to do this.

I suppose where this is heading is a group of C objects, optimised for C. So that TV object, well maybe the cog part and the higher level language part need to be decoupled?

Thinking aloud, one needs to take advantages of C, but not lose the vast library of code that exists in the Spin Obex. But like you say, still maintain access to the hub parameters like the ones passed in PAR.

David Betz · 2013-04-28 11:45

Dr_Acula wrote: »

That makes sense but it sounds complicated. It is a new way of thinking. With Spin, all cog data loads from hub. With some of the clever loaders we played with a few years back, we could load cog images off an SD card into the same 2k block of hub, then load them into the cogs and hence save a whole lot of hub ram.

C opens up more possibilities, but it may not be possible to mindlessly copy spin objects and run them through spin2cpp.

The vast majority of programs only load cogs once. So maybe there is a way of loading up all the cogs first, then overwrite all that hub ram with something else? Having said that, Spin wasn't really designed to do this.

I suppose where this is heading is a group of C objects, optimised for C. So that TV object, well maybe the cog part and the higher level language part need to be decoupled?

Thinking aloud, one needs to take advantages of C, but not lose the vast library of code that exists in the Spin Obex. But like you say, still maintain access to the hub parameters like the ones passed in PAR.

I've already done something similar to what you're suggestion in my p2load loader for the P2. It can load stuff into hub memory, start COGs, and then load the full hub memory with a program thereby avoiding the loss of hub memory for COGs that only need to be loaded once. I've been thinking of adapting that to P1 as well. There is really no reason it couldn't work. That would help with LMM and CMM programs as well as freeing up hub memory for data in XMM modes.

Dr_Acula · 2013-04-28 21:21

Very interesting.

Thinking about this more, this does appear to be a fundamental problem in porting Spin objects over to C.

In Spin, all memory is in hub so this issue never came up. But take a typical Spin program and run it through Spin 2cpp and it ends up maybe twice as big, so many Spin program are going to run out of memory if they are ported to C. So - we add external memory.

But now we need some way of telling the program where to put data. In a simplistic sense, take that TV driver. Most of the code can live in external ram. But there are the PAR values that will need to live in hub ram.

Are variants on Spin allowed? In the purest sense, no, but there are a number of programs out there that will only compile on BST and won't compile on the Proptool because they use things like #ifdef. So if we allow new spin commands... could we think about a command, in Spin, that explicitly tells the compiler that this variable needs to be in hub? Or external memory for that matter. So maybe you might have a command #memory_hub and #end_memory that you wrap around the variable declaration.

Then the next thing is to add this to Spin2cpp.

Then it makes the job of translating all the obex easier. Go through a Spin program and note the variables that are definitely going to need to be in hub. It might only be the PAR list, so that could be quite easy to do. Then run that through spin2cpp which will (presumably) collect such variables from all the objects and they will probably end up as a group in hub.

I'm interested to see what you have done with your p2load program too. I've never really liked the way almost half of the hub in a typical spin program is used up with cog code, which gets loaded once and then the ram is never used again. With external memory, there is no need to do it this way and there ought to be enough information for a compiler to work out ways automatically to load data. So the cog data could be in external ram, if any cog code is detected the compiler reserves 2k of hub ram, then if a cogstart comes up, that runs a routine that transfers the 2k of data from external ram to hub ram and does a cogstart as per normal.

Such a system could be completely transparent to the user. Take a typical spin program with 7 cogs to load (14k), check a box that says "store cog data in external memory" and if the compiler is set to xmmc or other external memory models, suddenly 12k (14-2) becomes available in hub.

Thoughts would be most appreciated!

macca · 2013-04-28 23:58

Dr_Acula wrote: »

That makes sense but it sounds complicated. It is a new way of thinking. With Spin, all cog data loads from hub. With some of the clever loaders we played with a few years back, we could load cog images off an SD card into the same 2k block of hub, then load them into the cogs and hence save a whole lot of hub ram.

Maybe it could work with a buffer in hub memory that is overwritten with the cog data from external ram ?

declare a 2k buffer

HUBDATA uint8_t cogcode[2048];

copy the cog code from external ram:

memcpy(cogcode, _external_cog_code, 2048);
cogstart(cogcode, par)

wait the time needed to load the code so it won't get overwritten.

This will use 2k for all cog drivers. Maybe using dynamic allocation with malloc/free could allow to reuse the 2k block for other data (never tested).

Dr_Acula wrote: »

C opens up more possibilities, but it may not be possible to mindlessly copy spin objects and run them through spin2cpp.

My opinion, supported by my recent experience in porting Spin code to C, is that automated programs like spin2cpp should be used as an hint (o quick way) to port the code. Manual editing is always needed. Manual porting is the preferred way to make the code more compact and efficient.

Dr_Acula · 2013-04-29 01:49

Brilliant macca - that is exactly the sort of code that would work.

Re manual editing, I guess one place to start would be by taking the original spin code and collecting up all the variables that are going to be hub variables and trying to put them all in one group? Maybe with a VAR declaration at the beginning rather than having them declared throughout the code. I'll have a think about that some more.

Addit - several types of variables come to mind. There is the PAR group. Also there are large common buffers that the cog needs to access, eg the graphics buffer for a display, or the circular buffer for a serial port driver. There might be a third group hidden in some objects. Pulling apart the TV object will be a good place to start

macca · 2013-04-29 02:54

Basically yes, the concept is that you need to know how the spin code works to make an efficent (and working) C porting. spin2cpp doesn't have enough intelligence to know that a particular variable or block of memory need to reside in hub memory, if I'm not wrong it isn't even aware of lmm vs. xmmc (or other variants) differences. Also cog (or DAT) code is translated to uint8_t arrays which may cause alignment issues, they should be translated to uint32_t arrays, or my preferred method is to use objcopy to generate *_firmware.o objects from dat files and reference them from C code. The Vga and TV examples in the demos folder should do that.

David Betz · 2013-04-29 04:00

macca wrote: »

Maybe it could work with a buffer in hub memory that is overwritten with the cog data from external ram ?

declare a 2k buffer

HUBDATA uint8_t cogcode[2048];

copy the cog code from external ram:

memcpy(cogcode, _external_cog_code, 2048);
cogstart(cogcode, par)

wait the time needed to load the code so it won't get overwritten.

This will use 2k for all cog drivers. Maybe using dynamic allocation with malloc/free could allow to reuse the 2k block for other data (never tested).

My opinion, supported by my recent experience in porting Spin code to C, is that automated programs like spin2cpp should be used as an hint (o quick way) to port the code. Manual editing is always needed. Manual porting is the preferred way to make the code more compact and efficient.

Yes, we've done that in the past with PropGCC. In fact, that's the way the ecog support works.

jazzed · 2013-04-29 08:29

macca wrote: »

Maybe it could work with a buffer in hub memory that is overwritten with the cog data from external ram ?

declare a 2k buffer

HUBDATA uint8_t cogcode[2048];

copy the cog code from external ram:

memcpy(cogcode, _external_cog_code, 2048);
cogstart(cogcode, par)

wait the time needed to load the code so it won't get overwritten.

We actually do this now. The best example is in the gas_toggle demo: https://code.google.com/p/propgcc/source/browse/demos/toggle/gas_toggle/toggle.c

The ecog thing David mentions reads PASM binaries from EEPROM and loads them. For this to be beneficial, 2 or 3 PASM blobs need to be used. It is the easiest solution for a simple self-contained "embedded" program. We have a demo for this too although the latest SimpleIDE does not support it (0-8-5 does) : https://code.google.com/p/propgcc/source/browse/#hg%2Fdemos%2Fcog_loader

We have discussed 2 phase loader methods for starting all cogs before loading/running the main C program. My goal is to not carry the LMM/CMM kernel at all in HUB RAM. The problem is that the kernel code needs to live somewhere for multiple COG processes (gcc cogstart) to work like the Spin cognew command. Spin has an advantage because it's interpreter lives in ROM.

We have considered these solutions: 1) pre-load all unused cogs with kernels to be used as required with a small number of re-init bytes in HUB RAM - can't use cogstop, 2) COG cloning where a reserved COG will hold a version of the kernel, and 3) keep the kernel in EEPROM - since most boards have a 64KB EEPROM, this is more viable, but it's still a non-starter for things like the Propeller Demoboard.

For the 2 phase loader to be useful, there must be an agreement on PASM mailboxes. Ross uses a registry definition. Others in the P2 thread have ignored that and come up with another solution. Both of those seem inflated to me, but the P2 proposal seems simpler although it lacks any form of identification like a cookie word. Providing O/S (operating system) like features where one can run any number of programs one at a time (using registry connections between the application and the PASM driver services) is not my goal. I know others want this kind of O/S thing, but how important is it?

Dr_Acula · 2013-04-29 17:52

Providing O/S (operating system) like features where one can run any number of programs one at a time (using registry connections between the application and the PASM driver services) is not my goal. I know others want this kind of O/S thing, but how important is it?

Yes I agree. Defining mailboxes for things that are never used seems a waste. Plus, it isn't necessary anyway. Kyedos has no mailboxes or registry entries. Values can be passed between programs with standard command line parameters, and each program is fully independent and can use all of hub ram in whatever way it wants.

I'm working through the practicalities of rebuilding a typical Spin object for use with external memory. Attached is the standard TV object from the Obex.

It comes in three parts. The demo program, then the sub program TV_Text and that has a subprogram TV. In TV_text are two areas that will need to be in hub ram - the first is the VAR declaration which includes the screen buffer and all the parameters passed via PAR. Then at the end is a DAT section. So one needs a way of marking both VAR and DAT sections as being hub and not external ram.
Then there is the cog driver which is in tv.spin. So using the model we are proposing, this DAT section stays in external ram. But we will need a pointer to mark the beginning of that section, and somehow the value of that pointer needs to be passed out to the main program so we know how to do the memcopy(). I'm not sure about that bit.

Finally the program will need the common 2k block in hub for cogloads, and that would be part of the main program and it only needs to be defined once, no matter how many objects get added.

tv_text.spin and tv.spin are really one object so it may be best to run those two through spin2cpp and then start marking blocks of data for hub ram in the output file?

Dr_Acula · 2013-05-01 06:31

I am working through the spin example and porting it over to C

Re HUBDATA, does this put all the following data in hub up till the next }

so the cog driver for the tv is going to be this

HUBDATA uint8_t tv_driver::dat[] = {

I'm not sure what is happening to the screen buffer - I need to study the C code more as it seems to have disappeared. This is the spin code

CON

  cols = 40
  rows = 13

  screensize = cols * rows
  lastrow = screensize - cols

  tv_count = 14

  
VAR

  long  col, row, color, flag
  
  word  screen[screensize]
  long  colors[8 * 2]

  long  tv_status     '0/1/2 = off/invisible/visible              read-only   (14 longs)
  long  tv_enable     '0/non-0 = off/on                           write-only
  long  tv_pins       '%pppmmmm = pin group, pin group mode       write-only
  long  tv_mode       '%tccip = tile,chroma,interlace,ntsc/pal    write-only
  long  tv_screen     'pointer to screen (words)                  write-only      
  long  tv_colors     'pointer to colors (longs)                  write-only                            
  long  tv_ht         'horizontal tiles                           write-only                            
  long  tv_vt         'vertical tiles                             write-only                            
  long  tv_hx         'horizontal tile expansion                  write-only                            
  long  tv_vx         'vertical tile expansion                    write-only                            
  long  tv_ho         'horizontal offset                          write-only                            
  long  tv_vo         'vertical offset                            write-only                            
  long  tv_broadcast  'broadcast frequency (Hz)                   write-only                            
  long  tv_auralcog   'aural fm cog                               write-only                            


OBJ

  tv : "tv_driver"


PUB start(basepin) : okay

and this is (I think) the C

INLINE__ int32_t Rotl__(uint32_t a, uint32_t b) { return (a<<b) | (a>>(32-b)); }
INLINE__ int32_t Rotr__(uint32_t a, uint32_t b) { return (a>>b) | (a<<(32-b)); }
INLINE__ int32_t Lookup__(int32_t x, int32_t b, int32_t a[], int32_t n) { int32_t i = (x)-(b); return ((unsigned)i >= n) ? 0 : (a)[i]; }

uint8_t tv_text::dat[] = {
  0x00, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x12, 0x00, 0x00, 0x00, 
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x28, 0x00, 0x00, 0x00, 0x0d, 0x00, 0x00, 0x00, 
  0x04, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x07, 0x0a, 0x07, 0xbb, 0x9e, 0x9b, 0x04, 0x07, 
  0x3d, 0x3b, 0x6b, 0x6e, 0xbb, 0xce, 0x3c, 0x0a, 
};
int32_t tv_text::Start(int32_t Basepin)

The screensize is 40x13 so should have produced a bigger array than that. Or is it in the .h file?

jazzed · 2013-05-01 11:15

>>> Re HUBDATA, does this put all the following data in hub up till the next }

It should. Eric?

Wish I had time to help! This week is super hecktic.

David Betz · 2013-05-01 11:41

jazzed wrote: »

>>> Re HUBDATA, does this put all the following data in hub up till the next }

It should. Eric?

Wish I had time to help! This week is super hecktic.

Yes, the initializers included between the braces will be placed in hub memory.

David Betz · 2013-05-01 11:41

jazzed wrote: »

>>> Re HUBDATA, does this put all the following data in hub up till the next }

It should. Eric?

Wish I had time to help! This week is super hecktic.

Yes, the initializers included between the braces will be placed in hub memory.

SPI ram

Comments