That is "exactly" the direction I'd like to go with this project. As I move into the 32K SRAM it should start to open all kinds of possibilities like this.
OBC
Consider a more object oriented sprite.
sprite1.Image[frame], Sprite1.x, Sprite1.Y
Thinking of extended memory - here is a thought. Most prop boards have 64K EEPROMS instead of 32K. How about storing the BASIC programs there which might free up HUB RAM for variables. This would be simpler than connecting 32K SRAM.
I'm not sure I like the EEPROM speed or limited write that comes with using the EEPROM as a place for BASIC code. Since I've established a project specifically for the this BASIC, I suspect I'll change BASIC to access SRAM directly as make it a standard feature on the Pocket Mini Computer Kit.
@cavelamb, Yes, as soon as RAM is freed up, this is what I have in mind.
Just a quick check on the schematic, there seem to be 10k pulldown resistors and I'm sure they are meant to be pullups to 3.3V?
Re jmg's comment, yes adding a serial sram would have to be a very simple addition and you get all that extra memory. Run your program from hub and store data in external sram, or run the program from sram and store data in hub. Either way has merit.
I'm taking a look at the code - I'd love to get this working on a touchscreen.
Addit: - I see there is an editor and there is Basic. Is this because there wasn't enough room to fit both in the one program?
Also, with the Basic program, where is the program stored? I can't seem to find an array where it would be. Is the program stored in hub or is it stored in external ram?
first, re the editor and the basic interpreter, are these two separate programs because there was not enough room to fit it all into one?
Also in the basic interpreter, is the program stored in hub or in external ram? I tried tracing through progsize hoping it would lead me to how the program is stored...
On a touchscreen, we have heaps of memory - a megabyte of external ram, plus lots more hub is free as there is no buffer needed for VGA. So maybe the editor and interpreter could be merged together?
I've got a touchscreen keyboard working with about half the screen devoted to the keyboard and about half for the text to display, so it is a sort of combo display and keyboard all in one.
Propeller color basic sure has grown since I last took a look - great work!
These new 128k sram chips will allow huge programs, but I think you already have drivers for the 32k sram chip and even using that as external ram would expand the program size.
Where is the program stored? I'm working through doline and processload, and both seem to suggest that the program is stored in an array called tline, but at the beginning that is defined as only having 256 bytes which doesn't seem right.
I think I can see some ways to get this onto the touchscreen
I think a serial sram is 4 pins. Up until recently that gave you 32k but with the new chips it is 128k. Parallel solutions work as well but take more pins. And there is always the slightly crazy solution of putting two I2C expander chips on the I2C bus and then adding a parallel ram to those pins. Technically, that takes no extra pins, but I am not sure anyone has done this as it would be a bit slow.
* Ability to send command line data to external binaries using BRUN (adopted DR_A's COMMAND.TXT format)
* Ability to HOLD a copy of the currently running BASIC program for re-run upon return from an external binary. (combining this with SRAM allows program to return to specific line after return.)
I've spent some time trying to decipher the source code but I seem to keep getting stuck trying to work out where the program is actually stored. As far as I can tell, programs have to fit in an array of 256 bytes, but I am sure that can't be right and the program is stored somewhere else.
I think you have an editor and the interpreter as separate programs, so for simulation purposes, you could have a DAT section with a program example in that DAT and that could be useful for testing things - eg maybe speed isn't a factor, but I have been playing around with binary tree searches - ie list all the commands in alphabetical order, then when searching for a match, start in the middle of the list, work out if it is in the top or bottom half, then divide that in half and keep going. For 256 items, that should find an item with 8 matches as against an average of 128. I was wanting to try this out with the code, if I could just work out where the code is actually stored
In all the FemtoBasic versions (including Propeller Color Basic ... until Jeff changes it), the program is stored in hub RAM in the area "beyond the end of the Spin stack". There's a long in a fixed location in high memory (userPtr) that contains the address of the start of the program storage area and this is initialized near the beginning of the "main" method. This area is initialized to contain an empty program. A program consists of a series of lines, each preceeded by two bytes containing the line number as a 16-bit value (0 - 65535) followed by the compressed text of the line, followed by a zero byte. These lines are in order by line number and the program ends with an empty line (just a zero byte) with a line number of 65535.
The compression scheme is simple. Each keyword in Basic is replaced by a single byte greater than 127. There's a table of these keywords in the Basic interpreter code near the beginning (tok0 to tokNN). The first keyword (tok0) is replaced by 128, the next by 129, etc. When you display a line in the program or save a program to a file, these bytes are converted back into text. If you save a program to EEPROM, the compressed program is saved as-is.
The interpreter includes a simple editor and the process goes as follows:
1) Read a line from the keyboard or serial port
2) If there's a line number, convert it to binary, and remove it from the text
3) Replace any Basic keywords with their special codes (> 127)
4) If there was a line number:
4a) Look for an existing line number in the saved program and delete it
4b) Store the compressed Basic line in the saved program area. An empty line (line number only) is not stored. Go back to step 1
5) If there was no line number, interpret the contents of the line
A RUN statement just sets the interpreter to read from the saved program in hub RAM rather than the (compressed) input buffer. A STOP or END statement effectively forces everything back to step 1.
I believe that what Jeff plans to do is to have the saved program stored in a serial SRAM and read it into a buffer one line at a time to be interpreted. There are some special cases that may benefit from some optimizations. For example, GOTO and GOSUB statements involve a linear search through the saved program in hub RAM for the line number. This could be slow in serial SRAM. It may help to keep the line numbers in a hub RAM table rather than the serial SRAM along with the length of the line in SRAM to allow for quick linear searches. FOR / NEXT loops also involve a linear search through the saved program for the matching NEXT statement. Flagging NEXT statements in the hub RAM table may help with the search. There are all sorts of other ways to efficiently store Basic programs in a serial SRAM including changing the whole system into separate compiler / interpreter phases, but that would be a rewrite rather than some minimal changes.
Thanks Mike! I knew you would answer that more accurately than I would. You also pointed out an issue or two that I hadn't considered yet with GOTO and GOSUB statements.
You could add a byte that specifies the length of the line in bytes after the two byte line number. This may speed up scanning in SRAM since all you have to do is take the current SRAM address + line Length byte + 3 to get to the next line. I believe the ZX81 used this method.
For the current memory scheme, that extra byte isn't necessary. For string lengths typical for Basic statements, the STRSIZE operation is very fast, very comparable to the time needed to handle an explicit length byte. It can be done in 16 clock cycles per byte plus some setup time. That's 200ns per byte with an 80MHz clock
Thanks Mike. That explains a lot of things, especially the userptr part, and brings up some interesting possibilities.
The system being used is not dissimilar to the way MBASIC works in CP/M, and I think there were two ways to save the file on the disk- the "small and fast way" which was the tokenized form, and the "slower and larger but human readable" way which was as normal code.
So ok, it is tokenized and you have a list of line numbers. Could you store those in a lookup table and make things even faster?
Also, storing the file in external memory one could think about caching like the C compilers are using. I'm thinking on the run here, but take a text program and read it in from an SD card and parse it through the tokenizer and move the tokenized code out to external memory. During that parsing, create a lookup table that links the line number to the location in external ram. Also, with any goto/gosub/next/wend etc statements that jump around in memory, create a cache lookup table. So in code, you might have GOTO 50 and that gets translated to a token for goto, and the 50 is now in a lookup table that says it is in external ram location 100 and cache block 2. Then when the interpreter finds this line, it asks the cache handler whether this block is in hub ram, and if not then it is loaded, and if it is then if it knows the block size and the ram location, it can work out where in the cache the next line of code is.
Ideally the parser does not split lines of code between cache blocks.
Thinking out loud, if the program is moved out to external ram, that frees up more hub space for code and maybe there is then room to put the interpreter and the editor in the same program?
The new 128k serial sram chips would be a super-simple addon for external memory, and with a cache driver, the slower speed of serial vs parallel ram would not be nearly such an issue.
Changed the COMMAND.TXT format a little (sorry Dr_A) and unloaded the filename from the file. (too restrictive for my taste)
This update changes the BASIC command line system. File names are no longer passed to the command-line file "COMMAND.TXT", meaning that the names of your binaries are no longer locked.
Replaced BASIC.BIN, replace your EEPROM copy.
Also, replace XMODEM.BIN, VIDEO.BIN, and PLAY.BIN if you are using any of these external tools. (Xmodem.bin and play.bin were also converted to work with FSRW, as a few SD cards were picky about Kye's Fat system.)
@Dr_Acula, All of the suggestions you've made are possible. My bias is that interpreting source code, even partially tokenized source code is slow. At this point, I would rather have a compiler / interpreter if I were going to put any significant effort into FemtoBasic or something similar. A compiler would make it easier to allow for integer, floating point, and string values and variables including arrays. Flow of control statements like GOTO, FOR / NEXT, etc. could have their "labels" resolved. By assuming the presence of serial SRAM of some kind plus an SD card or other flash memory, it would be straightforward to have a multipass compiler that could do all this and do it well. Older, smaller, slower computers with just some kind of mass storage device not fancier than an SD card have done it before. I think it's great that OBC is taking on moving the tokenized program out of hub memory. It's doable. It can be done without slowing down the already slow FemtoBasic interpreter too much more, and it will make it easier to handle larger programs with more language features than are currently possible.
I think one of the ideas Jeff is promoting is a standalone computer that does not need a PC to develop a program. For the C programs, the equivalent is a C IDE and a C compiler, written in C and running on the propeller. It might be doable too. I think C mainly exists in the world of compilers, but I think there are some C interpreters out there.
I don't know about the challenge of a Spin compiler running on the propeller.
Many early versions of Basic started off as interpreters and then changed to compilers. I always liked MBASIC as it gave you both and I wonder if that might be something to think about?. A tokenized interpreter, running from hub or external ram, and the ability to run it through a compiler if you want the speed. Maybe compile to "cached LMM" as the low level language?
I need to take another look at the Basic source code. Some interesting ideas here...
My bias is that interpreting source code, even partially tokenized source code is slow. At this point, I would rather have a compiler / interpreter if I were going to put any significant effort into FemtoBasic or something similar.
If you don't mind a program written in C rather than Spin you can try ebasic. It is one of the PropGCC demo programs (propgcc/demos/ebasic) and implements a compiler that produces bytecodes and an interpreter that executes them. I also have an interpreter written in PASM that is pretty fast. This could probably be used to replace the current ebasic interpreter which is written in C to produce a system that is much faster than ebasic. Unfortunately, the ebasic compiler/interpreter is too big to fit in hub memory so it would be necessary to use an external SPI flash chip and the PropGCC XMMC memory model.
Comments
Released version "J" to the Propellerpowered Library.
New support form for this project: http://propellerpowered.com/forum/index.php?board=5.0
Jeff
Consider a more object oriented sprite.
sprite1.Image[frame], Sprite1.x, Sprite1.Y
Your Thoughts?
I'm not sure I like the EEPROM speed or limited write that comes with using the EEPROM as a place for BASIC code. Since I've established a project specifically for the this BASIC, I suspect I'll change BASIC to access SRAM directly as make it a standard feature on the Pocket Mini Computer Kit.
@cavelamb, Yes, as soon as RAM is freed up, this is what I have in mind.
Jeff
Thought I'd write a respectable terminal program for COLOR BASIC with cursor and backspace support.
If you want faster access, no-write-limits, and 128K Bytes memory, have you seen the new 23LC1024 ?
http://www.microchip.com/wwwproducts/Devices.aspx?dDocName=en559066
Yeah, that pretty much takes the lid off this project. (mind is reeling from the possibilities this raises)
Jeff
http://www.instructables.com/id/Calling-Bulletin-Board-Systems-BBS/
Jeff
Just a quick check on the schematic, there seem to be 10k pulldown resistors and I'm sure they are meant to be pullups to 3.3V?
Re jmg's comment, yes adding a serial sram would have to be a very simple addition and you get all that extra memory. Run your program from hub and store data in external sram, or run the program from sram and store data in hub. Either way has merit.
I'm taking a look at the code - I'd love to get this working on a touchscreen.
Addit: - I see there is an editor and there is Basic. Is this because there wasn't enough room to fit both in the one program?
Also, with the Basic program, where is the program stored? I can't seem to find an array where it would be. Is the program stored in hub or is it stored in external ram?
I'll double check that schematic. Thanks!
Let us know how you make out on this running on the Touchscreen. That would be pretty cool.
Jeff
This is a clever piece of code.
first, re the editor and the basic interpreter, are these two separate programs because there was not enough room to fit it all into one?
Also in the basic interpreter, is the program stored in hub or in external ram? I tried tracing through progsize hoping it would lead me to how the program is stored...
On a touchscreen, we have heaps of memory - a megabyte of external ram, plus lots more hub is free as there is no buffer needed for VGA. So maybe the editor and interpreter could be merged together?
I've got a touchscreen keyboard working with about half the screen devoted to the keyboard and about half for the text to display, so it is a sort of combo display and keyboard all in one.
Propeller color basic sure has grown since I last took a look - great work!
Currently the program is stored in hub ram. I've started picking through the code in effort for getting it moved to external ram.
Thanks!
Jeff
Where is the program stored? I'm working through doline and processload, and both seem to suggest that the program is stored in an array called tline, but at the beginning that is defined as only having 256 bytes which doesn't seem right.
I think I can see some ways to get this onto the touchscreen
I think a serial sram is 4 pins. Up until recently that gave you 32k but with the new chips it is 128k. Parallel solutions work as well but take more pins. And there is always the slightly crazy solution of putting two I2C expander chips on the I2C bus and then adding a parallel ram to those pins. Technically, that takes no extra pins, but I am not sure anyone has done this as it would be a bit slow.
You made it to engadget: http://www.engadget.com/2012/08/21/back-in-the-bbs-days-how-to-helps-newbs-navigate-to-a-pre-inter/ :-) :-D
Added
* Ability to send command line data to external binaries using BRUN (adopted DR_A's COMMAND.TXT format)
* Ability to HOLD a copy of the currently running BASIC program for re-run upon return from an external binary.
(combining this with SRAM allows program to return to specific line after return.)
Latest version: https://www.dropbox.com/sh/qwhixzvtlrvp1u1/y-JshwklWj/PropellerBASIC
Examples of this method have also been added to this thread as well as this library.
Jeff
http://www.instructables.com/id/Pocket-Mini-Computer-Users-Guide/
A "User's Guide" to getting started with the COLOR BASIC as well as the complete project in general.
Jeff
I've spent some time trying to decipher the source code but I seem to keep getting stuck trying to work out where the program is actually stored. As far as I can tell, programs have to fit in an array of 256 bytes, but I am sure that can't be right and the program is stored somewhere else.
I think you have an editor and the interpreter as separate programs, so for simulation purposes, you could have a DAT section with a program example in that DAT and that could be useful for testing things - eg maybe speed isn't a factor, but I have been playing around with binary tree searches - ie list all the commands in alphabetical order, then when searching for a match, start in the middle of the list, work out if it is in the top or bottom half, then divide that in half and keep going. For 256 items, that should find an item with 8 matches as against an average of 128. I was wanting to try this out with the code, if I could just work out where the code is actually stored
The compression scheme is simple. Each keyword in Basic is replaced by a single byte greater than 127. There's a table of these keywords in the Basic interpreter code near the beginning (tok0 to tokNN). The first keyword (tok0) is replaced by 128, the next by 129, etc. When you display a line in the program or save a program to a file, these bytes are converted back into text. If you save a program to EEPROM, the compressed program is saved as-is.
The interpreter includes a simple editor and the process goes as follows:
1) Read a line from the keyboard or serial port
2) If there's a line number, convert it to binary, and remove it from the text
3) Replace any Basic keywords with their special codes (> 127)
4) If there was a line number:
4a) Look for an existing line number in the saved program and delete it
4b) Store the compressed Basic line in the saved program area. An empty line (line number only) is not stored. Go back to step 1
5) If there was no line number, interpret the contents of the line
A RUN statement just sets the interpreter to read from the saved program in hub RAM rather than the (compressed) input buffer. A STOP or END statement effectively forces everything back to step 1.
I believe that what Jeff plans to do is to have the saved program stored in a serial SRAM and read it into a buffer one line at a time to be interpreted. There are some special cases that may benefit from some optimizations. For example, GOTO and GOSUB statements involve a linear search through the saved program in hub RAM for the line number. This could be slow in serial SRAM. It may help to keep the line numbers in a hub RAM table rather than the serial SRAM along with the length of the line in SRAM to allow for quick linear searches. FOR / NEXT loops also involve a linear search through the saved program for the matching NEXT statement. Flagging NEXT statements in the hub RAM table may help with the search. There are all sorts of other ways to efficiently store Basic programs in a serial SRAM including changing the whole system into separate compiler / interpreter phases, but that would be a rewrite rather than some minimal changes.
Jeff
The system being used is not dissimilar to the way MBASIC works in CP/M, and I think there were two ways to save the file on the disk- the "small and fast way" which was the tokenized form, and the "slower and larger but human readable" way which was as normal code.
So ok, it is tokenized and you have a list of line numbers. Could you store those in a lookup table and make things even faster?
Also, storing the file in external memory one could think about caching like the C compilers are using. I'm thinking on the run here, but take a text program and read it in from an SD card and parse it through the tokenizer and move the tokenized code out to external memory. During that parsing, create a lookup table that links the line number to the location in external ram. Also, with any goto/gosub/next/wend etc statements that jump around in memory, create a cache lookup table. So in code, you might have GOTO 50 and that gets translated to a token for goto, and the 50 is now in a lookup table that says it is in external ram location 100 and cache block 2. Then when the interpreter finds this line, it asks the cache handler whether this block is in hub ram, and if not then it is loaded, and if it is then if it knows the block size and the ram location, it can work out where in the cache the next line of code is.
Ideally the parser does not split lines of code between cache blocks.
Thinking out loud, if the program is moved out to external ram, that frees up more hub space for code and maybe there is then room to put the interpreter and the editor in the same program?
The new 128k serial sram chips would be a super-simple addon for external memory, and with a cache driver, the slower speed of serial vs parallel ram would not be nearly such an issue.
Changed the COMMAND.TXT format a little (sorry Dr_A) and unloaded the filename from the file. (too restrictive for my taste)
This update changes the BASIC command line system. File names are no longer passed to the command-line file "COMMAND.TXT", meaning that the names of your binaries are no longer locked.
Replaced BASIC.BIN, replace your EEPROM copy.
Also, replace XMODEM.BIN, VIDEO.BIN, and PLAY.BIN if you are using any of these external tools.
(Xmodem.bin and play.bin were also converted to work with FSRW, as a few SD cards were picky about Kye's Fat system.)
Jeff
X10 control from BASIC
http://www.instructables.com/id/Take-over-your-world-using-BASIC/
Jeff
I think one of the ideas Jeff is promoting is a standalone computer that does not need a PC to develop a program. For the C programs, the equivalent is a C IDE and a C compiler, written in C and running on the propeller. It might be doable too. I think C mainly exists in the world of compilers, but I think there are some C interpreters out there.
I don't know about the challenge of a Spin compiler running on the propeller.
Many early versions of Basic started off as interpreters and then changed to compilers. I always liked MBASIC as it gave you both and I wonder if that might be something to think about?. A tokenized interpreter, running from hub or external ram, and the ability to run it through a compiler if you want the speed. Maybe compile to "cached LMM" as the low level language?
I need to take another look at the Basic source code. Some interesting ideas here...
http://www.instructables.com/id/TV-Out-for-the-Pocket-Mini-Computer/