Can SPIN Bytecode be Loaded from an SD Card after Initial Bootup?

JRetSapDoog

To spare readers the detail, the goal of my original post is summed up by the following:

"[noparse][[/noparse]Cause] a Spin program to be loaded into the Propeller's main memory
from a specified file on an attached SD card. This file should be a copy of
the binary form of a Spin program as saved from the Propeller Tool.
Once the Spin program has been successfully loaded, it begins execution.
The loaded Spin program completely replaces the running FemtoBasic."

······················· --excerpted from FemtoBasic/DongleBasic User Manuals

For gluttons for punishment, my original post is quoted in the first response that follows, but it may be good to skip it.

Post Edited (JRetSapDoog) : 11/8/2009 11:01:29 PM GMT

Bill Henning

Hi,

Take a look at FemtoBasic, PropDos, Sphinx... all reboot to different (loaded from SD) code.

JRetSapDoog said...
I'm still just learning to crawl in SPIN and PASM but I'm trying to determine the applicability of the Propeller to a long-term project I have in mind.



For this project, I'd like to "boot" normally and load up the cogs to get things rolling, particularly a cog with an SD driver from the Object Exchange. Then, I'd like to have either SPIN, PASM or a combination of both replace all or at least most of the 32KB of HUB memory with additional SPIN code/data off an SD card, with such code representing the main/final code to be run (i.e., the bulk of the application aside from any residual COG code or SPIN stub, if any). At this time, I'm not really looking to page code in-and-out, just seeking to replace the initial code (or bulk thereof).



In other words, I'd like a user to be able to conveniently reconfigure the system by just inserting an SD card before power-on, wherein the SD card would contain a file with a SPIN bytecode/data image that could be loaded into HUB ram and run, assuming one could have the interpreter in COG 0 "jump" somehow to a suitable continuation point in the new code. I don't want the user to have to connect to a PC at all (other than to possibly download and then save the file onto an SD card). I want the user to be able to run a variety of programs (though only one per power-on and probably just one per SD card), wherein such programs are collectively too large to fit in the boot EEPROM, including the 64KB EEPROM and so on. The file on the SD card could have a predetermined name such that the initial code could find it and load it (i.e., the initial code would always look for a file with the same name, regardles of what SD card was being used). Note that the user's system wouldn't necessarily even have H/W programming capability for the boot EEPROM (with the EEPROM being programmed in advance from "the factory").



This means that the contents of the initial 32KB would be replaced by an image from the SD card by either initial PASM or SPIN code, and if SPIN, perhaps a portion of the initial 32KB would have to be protected/kept so that the interpreter in COG 0 would have code to fetch. Alternately, perhaps a cog could replace all of HUB memory, but then there would need to be a way to hand off control to the interpreter or start a new one, but I don't know if a new version of the interpreter can be started after initial boot-up. I believe the interpreter in COG 0 can be stopped/wiped out, but I don't know if there's a way to have it or another one fire back up without power-cycling/resetting, which would only get me back to square one [noparse][[/noparse]unless the new HUB content could be retained on reset (Can it? It'd be nice if there was such an alternate reset option that would retain HUB and/or COG content)]. So, if that's not possible, then perhaps a small "stub" portion of the initial SPIN code could be retained during overwriting such that it could somehow "link" to the replacement code (see below).



What I'm not trying to do is build a compiler or interpreter. I'm not seeking to run another language other than SPIN/PASM. And I'm not seeking to use some kind of large memory model for PASM (I'll leave that to the Prop II), nor do continuous paging. More power to those of you that are, of course! However, I "just" want to replace the initial 32KB HUB image with another one, but one from an SD card, not the boot EEPROM. If, in the process of doing that, code in the cogs has to be killed, that's okay, though it would be an advantage space-wise if such code could be retained. But if cog content can't be retained, then any needed cog code (such as the SD driver to be used for accessing other non-SPIN data) could be reloaded by the replacement SPIN code.



Does anyone know whether this is possible or has been done, or have ideas on how to go about it, or suggestions for what keywords to search on? I searched and read for at least a few hours (most of which goes over my head at this time) and did come across this thread that seems to be relevant {Don't go just by the subject or initial post, as subsequent posts appear to go somewhat beyond those in scope}. From that thread, it appears that several folks here were working on or at least considereing something related (though apparently more general and powerful), but I'm not sure of the current status. I saw other theads about large memory models (LMM) and so on, but at least some of them seemed to be targeted at breaking the COG/PASM memory limit, as opposed to SPIN's. I'm not really even trying to break SPIN's memory limit, except for replacing the initial image and hooking the interpreter into it (which, admittedly, is kind of like breaking the limit, but only a little bit, and only for the purpose of one-time replacement).



It would seem that one can put any data they need into HUB memory, so why not bytecode loaded externally. Of course people here use self-modifying code in PASM, but the problem at hand involves HUB memory, and I'm just not sure about depositing (overwriting) bytecode on top of existing bytecode in HUB memory at runtime. In particular, I don't know about how it would be "lined up" so-to-speak, or whether the interpreter requires that the compiler-generated code be some kind of complete set from an addressing standpoint and so on. I don't think that there would be any checksums involved once the code is running, but I'm not sure about addressing matters. Does SPIN compile down to bytecode that uses only some kind of absolute addressing (as opposed to mixing in some relative addressing)? That is, is SPIN code relocatable, or, if it is not necessarily relocatable, can it be written in a way that makes it so (i.e., forced to be so)?



Whatever the case, it seems clear that any residual SPIN "stub" in the HUB from the initial image (assuming another method is not available) would need to know what address to go to on calling a procedure. So perhaps the address of at least one such procedure would need to be "pre-selected" in advance (i.e., common to the old and new code), but I'm not at all sure. Upon compiling the replacement code, perhaps the compiler could be "tricked" into generating a procedure at the same address as one in the initial code by including identical code for the stub before the replacement procedure code (or otherwise). Lots of questions there.... I've read here on the forum that people have studied the compiler and its generated bytecode, as well as the boot procedure, and thus it's likely someone knows (not that they're obligated to respond, of course). Incidentally, it'd be nice if the Prop II would support booting directly off an SD card, although I guess that would imply an SD driver being provided [noparse][[/noparse]either in firmware (unlikely since it's OTP) or on the SD card with perhaps a known name (similar to the above), which would seem a fine solution].



Anyway, any help or comments would be appreciated. Apologies in advance if this is a rehash of already covered ground (which is likely the case), but another newcomer like me might find it useful. Also, apologies for speculating too much about a solution (as opposed to just asking a question and/or indicating my goal), but hopefully doing so will allow any responders to help me clear up misunderstanding that I may have. Thanks! --Jim

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
www.mikronauts.com Please use mikronauts _at_ gmail _dot_ com to contact me off-forum, my PM is almost totally full
Morpheusdual Prop SBC w/ 512KB kit $119.95, Mem+2MB memory IO board kit $89.95, both kits $189.95
Propteus and Proteus for Propeller prototyping 6.250MHz custom Crystals run Propellers at 100MHz
Las - Large model assembler for the Propeller Largos - a feature full nano operating system for the Propeller

Mike Green

FemtoBasic and DongleBasic (from the Object Exchange) have a built-in Spin program loader that loads a file from an SD card into hub memory and executes it just like it was loaded from the EEPROM. FemtoBasic was used to create a kiosk with a bunch of Propeller demo programs that could be selected with a keyboard from a menu. All you need to do is compile the programs using the Propeller Tool, but save them as binary files (.bin) to the SD card. FemtoBasic and DongleBasic will automatically run a Basic program called "autoexec.bas" if it is on the SD card. This program can load a new program from the SD card. FemtoBasic uses a TV or VGA display and a PS/2 keyboard while DongleBasic uses the programming serial port for control. FemtoBasic requires a PS/2 keyboard to be present while DongleBasic will execute the "autoexec.bas" program even if the programming port is not connected.

JRetSapDoog

Gee...responses already?· You guys are great!· I didn't think anyone would even skim-read such a lengthy post, let alone reply so quickly.

Thanks for those comments, Bill and Mike.· That sounds like exactly what I want to do.· I'll take a look at those objects.· Much appreciated!

Maybe a general-purpose object for this kind of "rebooting" as you call it would be nice.· The menu functionality sounds great, as well.

Hmm...I want the Spin program loader to also load Spin code (guess that's what a Spin program loader should load, but I'm not sure).·

Or does that mean to [noparse][[/noparse]1] use a spin program to [noparse][[/noparse]2] load a Basic or other interpreter to [noparse][[/noparse]3] finally load and run the target Spin program?·

Well...it's too early for me to be clear on the full scenario or·underlying mechanisms, but I know where to start digging.· Double thanks!!

Mike Green

There are two pieces to the loader in FemtoBasic. There's "fsrwFemto.spin" which is the file system level code written in Spin that looks up a file name in the SD card root directory and "sdspiFemto.spin" which is the low-level I/O code that uses block numbers on the SD card and also does EEPROM I/O. The Basic interpreter passes a string containing the file name to the fsrw open routine which looks it up and stores the starting block address of the file in a variable. If it's found, the Basic interpreter passes the starting block address to the sdspi boot routine. This routine reads the length of the program from the file, stops all running cogs except what's needed for loading, then loads the file from the SD card and starts a new copy of the Spin interpreter on the just loaded code as if it were loaded from the boot EEPROM.

You can include the fsrwFemto and sdspiFemto objects in your own Spin program and call the various routines yourself using the FemtoBasic code as a model or you can use FemtoBasic itself (or DongleBasic) to do the work for you. Your choice.

The low-level code in sdspiFemto can also load a Spin program from any address in any EEPROM attached to the Prop. For example, the Protoboard comes with a 64K byte EEPROM. The sdspiFemto EEPROM loader can load a Spin program from the 2nd 32K of the EEPROM and can reload the program from the 1st 32K.

JRetSapDoog

@ Mike:· Thanks for the additional information with the precisely worded explanation and a list of my choices.· I've downloaded both FemtoBasic and DongleBasic to have a look.· I haven't tried them yet...as I want to study them more and check on pin usage and so on.·The PSM board I'm currently using uses P0..P3 for the SD Card (and I'm not sure if pins are readily configurable in your code, though I imagine so).·

Interesting, at least one of the sample·programs available for·the PSM·uses your sdspiFemto·driver that you mentioned in your last response here (the 2009/04/04 modified version), but·another program appears to use one called mb_spi (from·"lonesock").· Also, I think I've seen a couple different SD drivers used in the sample programs for the PSM, but I believe they either are a version of the fsrw driver·or derived therefrom.· So, perhaps that's good news for the equipment on hand.··But I can put together a different system, if need be,·as I intend to·do at some point anyway for other usage.·

Anyway, from the FemtoBasic manual, the SPIN "<file>" instruction appears to be the key.· There--described in 4 sentences what took me about 40--is exactly what I'm trying to do:

"This causes a Spin program to be loaded into the Propeller's main memory
from a specified file on an attached SD card. This file should be a copy of
the binary form of a Spin program as saved from the Propeller Tool.
Once the Spin program has been successfully loaded, it begins execution.
The loaded Spin program completely replaces the running FemtoBasic."

If I understand you correctly, it appears that·I can just create the autoexec.bas Basic·file (using a standard plain text editor, I believe) and have it issue the SPIN "<FILE>" instruction on the Spin file of my choice, which will then take over from thereon.

BTW, it is this case that the as-yet-to-be-designed system I'm targeting won't have a PS/2 keyboard attached.· So, on the one hand, if being lazy and using the pre-built objects, perhaps I'd need to go with DongleBasic...since FemtoBasic may need a keyboard attached·in order to run.· On the other hand, perhaps using "my" own Spin program that uses the fsrwFemto.spin and your·sdspiFemto.spin objects would be the way to go, in that those probably don't reference a keyboard.· Or, perhaps I could just slightly modify FemtoBasic·to supply a file name.· I did·notice that DongleBasic uses objects of different names, but perhaps they are·similar (since, overall,·similar functionality is involved).

Well, although I'm still somewhat in the fog (being a total newbie), the sun is beginning to burn off that fog.· Thanks again for the help.

Mike Green

1) Yes. You can put DongleBasic in your EEPROM and an "autoexec.bas" program on the SD card that just has a SPIN statement in it.

2) I recommend using the fsrwFemto and sdspiFemto objects from FemtoBasic if you're going to write your own Spin program. They're better designed to work separate from FemtoBasic than the corresponding routines in DongleBasic.

3) All of the high-level file system code is based on Rokicki's FSRW object which has recently been updated to V2.4. This newer version uses a new low-level driver called mb_spi... All of the FemtoBasic derivatives use a different low-level driver written in assembly that combines an I2C driver and the low-level SD card driver and includes boot loader functionality.

JRetSapDoog

@Mike:· Geepers!· I think I received a response to my last comment before I pressed the send button, or at least in Google search response time (nnn·results in 0.125 seconds).· You must have had the response precompiled!· Well, I noticed that you were online when I last posted, but I'll try not to get used to such quick responses (and hand-holding).· Honestly, I don't "expect" any answers, per say, but I know that we've got a great community here (with vastly mixed experience, from yours on one end to mine somewhere·on the other).· But I do really appreciate the help!· Actually, I'm not so used to "collaborating" with others or seeking assistance (especially if I think I can help myself).· However, it sure is an advantage, provided that·one doesn't depend on it too much.· As for help being a give-and-take thing on the forum, so far, I've only been of help once (and very marginally at that), but I'm new to embedded.· Of course, just asking questions and commenting can be of help to others.· Anyway, this is the first thread that I've personally created seeking assistance (yeah, I could have picked something somewhat easier (maybe next thread)).· But in order to not take advantage too much, I have tried to do my "homework" prior to thread creation and seeking follow-up info on the responses so far.· In reading thousands of comments here, I've noticed that that's something you and others appreciate.· The 'ole "God helps those that help themselves" thing, I guess (well, not to compare "Mr. roy_G_biv" (big 'G') with the Creator, cryptically-speaking).··</Sucking_Up>

Regarding DongleBasic, if I'm not mistaken, it appears that the SPI pin configuration is set in the Constant section of the BB_definitions spin file [noparse][[/noparse]spiDO ('O' as in "Out," not zero/0), spiClk, spiDI, spiCS].· And I believe that we're using SPI to do MMC driving on MMC/SD cards.· So, from the DongleBasic perspective, would that mean that those are the pins that control an SD card?· It would seem so.· And if so, are they only set once (well...I could dig deeper to check that, admittedly)?· My Propeller System Module (PSM) from Raymond Allen uses Propeller pins P0..P3 to connect to pins on an SD card socket labeled Dat3, Cmd, Clk and Dat0, respectively (for reference, a pic with a partial schematic is attached from the PSM manual).· The PSM schematic appears to use the SD naming scheme (since·the PSM·uses·an SD socket).· I found a mapping from SD usage to SPI usage online.· For the mapping in this case (DongleBasic to PSM), I believe the mapping is as follows:·

spiDO ==>·Dat0,·· spiClk·==> Clk,·· spiDI·==> Cmd,·· spiCS·==> DAT3

But if so, functionally speaking, is there anything in the DongleBasic driver that requires the pins to be in the same order and/or be contiguous?· While my PSM SD socket pins are contiguous, they are in reverse order compared to the usage given in the DongleBasic caller.· I'm guessing (or hoping) that it's not a problem and that I can merely change the pins accordingly in BB_definitions (from 8..11 to 3..0).· Being contiguous is probably more important.· This is more of a status report than anything; I'm not expecting you to verify the pin usage, per se; just mostly talking out loud and hoping that someone will give a shoutout if they just happen to overhear something that souds a bit odd while they are working on something else (Huh?· Did he just say the world will end in 2012?·--or, since we're talking about terminals-- Did you say Gate 67?).·

BTW, I'm wondering, just for initial testing purposes, if the DongleBasic would need to be in EEPROM, or if it could merely be loaded into HUB RAM from the PC (in that the first 32KB of EEPROM content is merely copied to RAM on booting).· I'm guessing either way is fine, just as with other programs, but it's best to check.· I'm guessing you said "EEPROM" because that is my ultimate·target application.· Edit 1:· Hmm...on the other hand, maybe those objects have to load from EEPROM.· Yeah, guess that makes more sense.··And perhaps something involving reset is involved.· Edit 2:· Whoops!· Turns out I was wrong about being wrong; first casual guess was right, oddly enough.· See next 2 posts.

Anyway, at some point, I'll probably get the courage to try the above (or something close to it) and see what happens.· I thought I'd start with a DongleBasic test, as that seems like the quickest way to get started.· And if that works, maybe I'll have more confidence to look into FemtoBasic.· And if I want to "roll my own," I'll take your advice and use the fsrwFemto and sdspiFemto objects.· Thanks again.·

Post Edited (JRetSapDoog) : 11/8/2009 9:33:13 PM GMT

Mike Green

1) Yes, the pin definitions are in BB_definitions.spin and the SD card pins can be in any order. One thing to be careful about is that DI (Data Input) and DO (Data Output) are relative to the SD card, not the Propeller. If it doesn't work ("Can't mount SD card"), you may want to try swapping DI/DO. As far as I can tell from the diagram you posted, the I/O pin definitions (P0-P3) are backwards from the ones used in DongleBasic (P8-P11). It should work if you just define them backwards (8->3, 9->2, 10->1, 11->0).

2) DongleBasic runs fine if just downloaded to RAM. In fact, that's used as a technique to load programs into 64K byte or 128K byte EEPROMs. You load your program into EEPROM, then load DongleBasic into RAM and do a "COPY [noparse][[/noparse]0],[noparse][[/noparse]<destination>]" where <destination> is the address where you want the program copied (in 32K multiples). Hitt Consulting's Prop-Dongle has a 64K EEPROM like Parallax's Protoboard and I would do a "COPY [noparse][[/noparse]0],[noparse][[/noparse]$8000]" to copy the downloaded program to the 2nd 32K of EEPROM, then re-download DongleBasic to the 1st 32K. I can then run the new program by doing "SPIN [noparse][[/noparse]$8000]". The same idea can work with the SD card but you can also use your PC to load up the SD card.

Post Edited (Mike Green) : 11/8/2009 8:18:02 PM GMT

JRetSapDoog

@Mike: It works as advertised! Thanks so very much! Happy camper, here. Thanks for confirming the pins, too. That's above-and-beyond the call of "duty" even more than normal. But I appreciate it, as I'm an American currently over in Taiwan and it's not so easy to get Propeller-related parts here. As such, I was at least making some effort not to fry things (though the Prop and the PSM seem pretty robust).

Initially, it couldn't find my Spin program because it was named .binary from the spin tool, but I showed the hidden extensions and remedied that (renamed as myfile.bin not myfile.binary or myfile.binary.bin). I tried running things out of RAM first, and that worked. Then I burned to EEPROM, which also worked. Then, I reset, and that worked. Then, I took the card out, and, as expected, that did not work. In summary, it works from RAM or EEPROM, as you said (but that's not news to you).

Currently, I am confused, though, as to why it works out of RAM. Initially, I casually guessed that it would work out of RAM, like a typcial program. Then, after thinking about it for a bit, I concluded that it probably wouldn't because I thought it would specifically look for the EEPROM. Well, apparently, I over-thunk it. But in terms of why it does work out of RAM, I've under-thunk it. But I do know that it works. I also know you've got a nice explanation in the response before this one; it just hasn't sunk in yet. I don't as yet have a good idea about what things are fixed in unchangeable ROM in terms of the booting sequence or options thereof. And, equally important, I don't know how those various DongleBasic routines work. For that matter, at this point, I'm not clear on whether this technique is doing a reset or whether the routines just start a Spin interpreter in such a way that it equates to a fresh boot (interpreter in COG 0 and 7 clear ones (with no idea about the internal counter) and the SD Spin program in RAM). Actually, I'm such a newbie, that I only know how to start a Spin interpreter from one that up and already running. Well, for all I know, perhaps that's what this technique is doing but just killing itself in the process (I guess I mean, loading the SD Spin code into RAM, starting COG 0 on it, and then "falling on its sword" (COGSTOP) in the control cog and support cogs immediately afterwards).

Hey, sorry for the stream-of-consciousness (though hopefully it's instructive), but after writing the foregoing paragraph, I just looked in the Propeller manual to see if there's a command called RESET. There's not, but there's one called REBOOT (as well as a bit in the clock register that I guess can bring about the same result). So, this would be a so-called software reset. However, that wouldn't help us, would it! That would just result in loading DongleBasic once again, not loading the Spin program on the SD card. So, I guess that means that the "reset" that DongleBasic accomplishes is not done with smoke-and-mirrors (though well over my head) but rather through pretty much ordinary cog-control commands. I mean, of course it doesn't have any special access to Prop hardware/firmware that the rest of us don't, but, originally, I was open to the idea that some hardware of software reset was being done, or some special kind of linkage with the firmware, but it would seem not. DongleBasic can't get to the hardware reset pin (unless someone's circuit board was designed to connect a Prop output pin to it), and even if it could, it wouldn't work, just like a software reset wouldn't work. Or have I overthunk it again! Please feel free to correct me, anyone. And even if that's right, there's nuances that I'm not getting or misconceptions.

Having said all that, for now, I'm just glad it's working. There's many things that work that I don't understand but can use (my foot, especially while inserting into my mouth). So, we don't have to figure this out all today. I've learned a lot, already. And thanks again for the responses.

JRetSapDoog

@Bill:··Prior to calling it a day (well, actually a night), I just wanted to say that I read through most of the posts on Sphinx.· I watched a video presentation about it, as well, and came across one of your videos somewhere in the process.· While most of that is over my head at this point, it does give me a feel for lingo and a better idea of the environment.· Although I'm still trying to digest what I read/watched, thanks again.

Also, I used the Forum Search, search.parallax.com (based on a tip in one of Mike's posts somewhere) and good 'ole Google to search for PropDOS and that also looks quite interesting.· For anyone interested,·a couple "starter" PropDOS links are as follows:·

PropDOS website:· http://www.orrtech.us/propdos/get.html
Recent Forum Post:· http://forums.parallax.com/showthread.php?p=683157

Thanks again, Bill, for those useful pointers.· Greatly appreciated!

Bill Henning

You are most welcome!

JRetSapDoog said...
@Bill: Prior to calling it a day (well, actually a night), I just wanted to say that I read through most of the posts on Sphinx. I watched a video presentation about it, as well, and came across one of your videos somewhere in the process. While most of that is over my head at this point, it does give me a feel for lingo and a better idea of the environment. Although I'm still trying to digest what I read/watched, thanks again.


Also, I used the Forum Search, search.parallax.com (based on a tip in one of Mike's posts somewhere) and good 'ole Google to search for PropDOS and that also looks quite interesting. For anyone interested, a couple "starter" PropDOS links are as follows:


PropDOS website: http://www.orrtech.us/propdos/get.html

Recent Forum Post: http://forums.parallax.com/showthread.php?p=683157


Thanks again, Bill, for those useful pointers. Greatly appreciated!

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
www.mikronauts.com Please use mikronauts _at_ gmail _dot_ com to contact me off-forum, my PM is almost totally full
Morpheusdual Prop SBC w/ 512KB kit $119.95, Mem+2MB memory IO board kit $89.95, both kits $189.95
Propteus and Proteus for Propeller prototyping 6.250MHz custom Crystals run Propellers at 100MHz
Las - Large model assembler for the Propeller Largos - a feature full nano operating system for the Propeller

JRetSapDoog

Just in case anyone stumbles upon this thread in the future, I'd like to add that Nick McClick's Propeller Platform SD available from Gadget Ganster·(GG) encompasses the functionality for which I was striving, both in terms of software and hardware.· Nice work bringing those together, Nick!· Smart-looking (and functioning) board.

On Propeller Forum:· http://forums.parallax.com/showthread.php?p=892438

On Gadget Gangster: ·http://www.gadgetgangster.com/find-a-project/56?projectnum=281

Perhaps a version of that board with a prototyping (soldering) area right on the board might also be good.

P.S.:· Here in Taiwan, I ordered a Propeller System Module from GG and received good service from Nick.