Running large Propeller Programs from EEPROM or SDCARD
jazzed
Posts: 11,803
In another thread the topic of using EEPROM for running large Propeller programs came up.
Running programs from SD Card is interesting for better performance and would work the same way.
The idea is that an EEPROM is a cheap solution for large programs that doesn't use any more Propeller pins than necessary. Using a cache means the main "business logic" program performance is reasonable and COG PASM programs still run at normal Propeller speed.
Three software solutions are possible: C, SPIN, and xBasic.
Meanwhile, here are answers to some basic questions that came up.
Two ST 256KB EEPROMs can be used for 512KB. Running large C programs from EEPROM will be slower than SPIN. Also 8KB today is used for cache. Will talk about this in more detail in other posts.
Running programs from SD Card is interesting for better performance and would work the same way.
The idea is that an EEPROM is a cheap solution for large programs that doesn't use any more Propeller pins than necessary. Using a cache means the main "business logic" program performance is reasonable and COG PASM programs still run at normal Propeller speed.
Three software solutions are possible: C, SPIN, and xBasic.
- The C solution is available now in Propeller-GCC.
- A Spin solution can be made to work with compiler change support.
- The xBasic solution would need to be ported (uses same drivers as GCC).
Meanwhile, here are answers to some basic questions that came up.
potatohead wrote: »How large can EEPROM devices be? Can I get a mega byte in an EEPROM?
To me, it's very attractive, given the device can hold enough code, and it runs at SPIN speed or greater. The primary attraction would be to use the HUB more fully as a buffer or operating area, where little to no code exists. 2K for uploading to COGs, some amount for parallel code threads, and a master program that's quite large residing in EEPROM is a potent combination, IMHO.
Two ST 256KB EEPROMs can be used for 512KB. Running large C programs from EEPROM will be slower than SPIN. Also 8KB today is used for cache. Will talk about this in more detail in other posts.
Unfortunately "BigSpin" in any form requires compiler change support because it assumes a 16 bit memory space. Will also talk about this in more detail in other posts.Off-topic aside: As I was reading the above comments, I couldn't help but think: Why not take the Spin interpreter (now that it's been released) and extend it with some instructions to do LMM to EEPROM in a sane way (maybe with explicit swapping in/out of code) and drop the SPIN interpreter into a PASM section of an object, then do whatever the heck you want.
Comments
Today we can run larger programs from EEPROM or SDCARD using Propeller-GCC.
The EEPROM must be at least a single 64KB device as is available on several Parallax products and forum contributor boards.
Supported Parallax hardware:
- Parallax Propeller C3 (EEPROM, FLASH, & SDCARD)
- Parallax PropBOE (EEPROM & SDCARD)
- Parallax QuickStart + Human Interface
- Parallax PropProto Board 64K EEPROM
- Parallax QuickStart Board 64K EEPROM
- Parallax Hydra with 128K EEPROM
- Parallax Backpack with 64K EEPROM
- Parallax P8X32A-Q44 SchmartBoard Kit
- Parallax Spinneret Web Server
Possible EEPROM part numbers for DIY:- MCP 24FC512 (64Kx8) 8x = 512KB ($1.68 each)
- MCP 24FC1026 (128Kx8) 4x = 512KB ($3.67 each)
- Atmel AT24C1024B (128Kx8) 4x = 512KB ($3.45 each)
- ST M24M02-DRMN6TP (256Kx8) 2x = 512KB ($5.07 each)
One Propeller-GCC solution is an XMMC Memory model. What does that mean?- XMMC is eXternal Memory Model Code and means the external memory device holds program code only.
- XMMC programs run much faster than the XMM equivalent (code, data, and variables in external memory).
- C instruction width is 4 bytes (a long) and must be fetched from external memory for instruction execution.
- A cache is used for loading/keeping program code. Today cache is 8KB, but could be smaller if necessary.
- Constant data to be used by the program must be loaded from external memory (not part of the program).
- A small amount of EEPROM is used for program startup (about 5KB).
- C code functions can be tagged to run from HUB memory for critical speed tasks.
- Typically 24KB of HUB memory is free for variable memory.
Propeller-GCC supports 4 basic memory models: COG, LMM, XMM, and XMMC.XMM is the slowest but most flexible. Linker scripts allow XMM to be assigned to any type of memory.
XMMC can also run from SRAM or SDRAM.
Compiling and loading an XMMC program (assumes you have propgcc and a good test program).
$ propeller-elf-gcc -mxmmc testprogram.c -s
$ propeller-load -r -t a.out -b eeprom
The Big Spin thread mentions running programs from external memory.
The solution interprets Spin byte-codes and can run 32KB programs from external memory leaving most of HUB memory available for buffers and such.
There are several approaches to make Big Spin work, but without Spin compiler support they are just awkward.
Approaches:
- Modify the existing Spin interpreter for 32 bit address and change compiler for support.
- Rewrite the Spin interpreter in LMM mode for XMM/XMMC support.
- Modify the existing Spin interpreter and use a "virtual function stubs" stubs point to external memory.
Of these 3 approaches, the last is most likely to succeed, but will require some compiler helper tools. Item 2 is least likely to run into COG memory space issues, but it will still face the addressing problems as Item 1.All modes will run faster if external memory storage access is limited to program code.
There is a language called xBasic that supports external memory today with almost the same methods as GCC. The primary difference except for the language is that xBasic interprets byte-codes like SPIN and GCC uses LMM PASM instructions. This means that xBasic will be slower than GCC especially in LMM mode, but is actually a little faster than other solutions with XMMC programs. xBasic does not support EEPROM or SD card fetch/execution .... but dare I say it could?
Procedure for using SD card Propeller GCC XMMC programs (C3 specific example).
- Download and install the latest Propeller GCC package for your O/S.
- Follow the SD Card example here: http://code.google.com/p/propgcc/wiki/PropGccLoader#XMMC_SD_Example
Other programs like fft or dhrystone not having xmmc directories should follow this example.- cd c:\propgcc\demos\fft
- $ make clean
- $ make MODEL=xmmc
- $ propeller-load -x fft_bench.elf
- $ copy fft_bench.pex AUTORUN.PEX
- Insert SD card and copy the AUTORUN.PEX file generated in previous step.
- Eject SD card and insert to board.
- Boardtypes: C3, SDRAM, DRACBLADE, PROPBOE
- $ propeller-load -r -t -z -b <boardtype>
Simple user programs can be built and run like this:- $ propeller-elf-gcc -mxmmc program.c -o program.elf
- $ propeller-load -x program.elf
- $ copy program.pex AUTORUN.PEX
- Insert SD card and copy the AUTORUN.PEX file generated in previous step.
- Eject SD card and insert to board.
- Boardtypes: C3, SDRAM, DRACBLADE, PROPBOE
- $ propeller-load -r -t -z -b <boardtype>
You can define your own boardtype in:- c:\propgcc\propeller-load\propeller-load.cfg (windows)
- /usr/local/propeller/propeller-load/propeller-load.cfg (linux/mac)
Copy one of the existing boardtypes to "myboard" in the propeller-load.cfg, then change the parameters.Once you save propeller-load.cfg file, use this command to run the program $ propeller-load -r -t -z -b myboard
BTW: $ is a common linux command prompt token that says "enter this command".
@Jazzed, I know you are waist-deep in to the "C" project for the Propeller, but is there any chance you could move this up a little higher on the project priorities? (It was the bottom post)
OBC
I'm assuming this will also work on the Propeller Platform as well?
OBC
The original Propeller Platform board will require a 64KB EEPROM upgrade - just adding a second 32KB device will not work.
But meanwhile we have today's solution which is C and with caching it can run at a decent speed as well. My hat goes off to all the clever people writing these C compilers!
As a subtle point GCC is not really their main thrust. As I understand it, GCC is an enabling technology they will use to penetrate new markets.
I know that a binary library was released for Linux, that would mean there was at least some cross compatibility.
I just looked at BST and noted that they don't appear to use the Parallax compiler, what's the story on BST and source availability? It doesn't seem to be a commercial product, but it doesn't seem to be open source either, is there a backstory that I don't know about?
For whatever reason he has so far decided not to release the source to BST. That is his choice.
Then there is the HomeSpun Spin compiler my mpark. With the same motivations. This time written in C# so that it will run on any platform that supports it. Again not open source. And that is his choice to make.
Both of these are excellent tools. As a non-windows user I may have passed by the Prop if they had not existed.
I seem to remember that the Parallax Prop Tool could not be opensourced due it's use of some closed source GUI components. Not sure why that applies to the compiler itself though.
At various times I've corresponded with both Brad Campbell and Michael Park over their choices, and I respect their decisions not to release the source (they are not the same reasons, by the way - and neither one seems to be motivated by thought of commercial gain).
In my view (now that we have several C options) the closed nature of Spin is the biggest remaining commercial impediments to the wider success of the Propeller. Everyone knows that Spin is the "native" high-level language of the Propeller, despite now having the option of various other languages (including now 5 different options for C programming!). This will never change the fact that the Spin language was designed by the same persion who designed the chip itself, and it is therefore the language best positioned to exploit the unique features of that chip.
But for commercial use, Spin is an unmitigated disaster - both because it is closed, and (with both Brad and Michael having reduced their involvement) because it is essentially single sourced. And while I wouldn't go so far as to say it is unsupported, it may as well be, considering how little effort Parallax appears willing to put into improving it. In terms of functionality, both BST and Homespun left the Propeller Tool in the dust long ago - but in incompatible ways (and in a commercial application you can't really use either of them!).
I don't really understand Parallax's decision in this matter, but - assuming they consciously made one - again I guess I have to respect it.
However, I really do hope they change their mind. It is probably too late for the Prop 1 - history has passed it by - but this decision could make or break the commercial success of the Prop 2.
Ross.
I would disagree about Spin being a disaster. We've had these debates before about Spin vs. C mostly. There's a long history in this industry about using proprietary languages and compilers / assemblers for developing commercial products. The industry has used them before and I doubt there were more disasters than now with open source systems which can also suffer from lack of support. The fact that you can get the source code doesn't mean that you can maintain it yourself. If you have to pay someone else with the expertise to maintain it, that adds to your costs for its use. Keep in mind that the Stamps have been used for years in commercial products and the only way to program a Stamp is with Parallax's proprietary PBasic compiler. Sure, there are cheaper alternatives these days and there are commercial PBasic-variant compilers for them, but they're just as proprietary as the Propeller Tool.
With the widespread use of open source tools and GCC and its derivatives being used for embedded systems programming, there is certainly an expectation in the industry that newcomers should support that development model. It would be in Parallax's best interests to support Spin / assembly using this model going forward. That doesn't make it a disaster not to. It's much harder to overcome the "the Propeller is so different from what I'm used to" mindset. The use of C helps there, but doesn't cure it.
A Big Spin compiler could be done by using the current compilers to generate up to 32K-size objects, and then link them together into a much bigger Spin program that could be megabytes in size. The limitation with this method is that no single object could be greater than 32K in size. However, 32K of spin bytecodes represents hundreds of lines of Spin code, so I don't think that's a major problem. I looked at doing this a while ago, and it's certainly feasible to do it.
I've been critical of the Spin language in the past, so I won't restate my criticism of it. The only thing I would say at this point is that Spin is a propietary language, and it will probably never be used on other processors. This makes Spin code non-portable, and limits the usefulness of it to Propellers. Code written in Spin cannot be easily ported to other processors, and code written in other languages cannot be easily ported to Spin.