Are fixed PUB locations possible in Spin?
Dr_Acula
Posts: 5,484
I've got another idea for Big Spin. Is this possible?
Maybe using the @ symbol to work out the location and then adding in a dummy array or something?
I've got this idea that you have a main program running in the lower half of hub ram and then in the upper half you can pull in groups of spin functions. Probably grouped in blocks of 16k and in groups where there is not too much 'churn'.
Up until now, two good reasons not to do this. First is that the SD driver code takes 1/4 to 1/3 of the memory. And second, most of the rest of the memory is taken up usually with a graphics buffer. So not much point.
But - if you have external memory, you can move all the files you need into external ram, and then the SD driver is only needed infrequently at most so you can unload it. And if you have a display with its own memory (touchscreen etc) then hub ram is not needed for a screen buffer.
So in that scenario, once some cogs are loaded up, the whole 32k of hub ram becomes a blank canvas.
One of the problems writing big programs (eg in C) is that the download times get longer and longer and debugging becomes a pain. 1 megabyte programs take ages to download. So maybe there is a way to precompile things (call them objects, or classes, or (shock!) DLLs) and then only have to compile them once. An example might be some string routines and put them all in a class called string.bin.
So the main program sits in the lower half of hub and handles moving 16k binary blobs in and out of the upper 16k of hub. Probably from external ram rather than an SD card because then you don't need the SD driver in ram any more.
I guess the hard part is the linking. If this works, the main program would have a dummy PUB located half way through ram. Pass some longs - or maybe a pointer to a string with the name of the pub you want to run in the class you have loaded. Then for the classes, compile those with a dummy array filling the first 16k so that the first pub appears half way through ram. Have one common entry and exit point for all called PUBs. Then all the jumps/calls don't need relocating.
Could this be possible?
Addit - I just thought of something. How about running two spin programs in parallel.
The "main" program runs in the lower 16k. The loadable one runs in the upper 16k and so all you do is have a 16k dummy array at the beginning. Maybe there is a bit of jump code that might overwrite the main program so perhaps the main program has a few bytes reserved at the beginning in an array that get overwritten. So the first program loads up a binary into upper 16k and then sets a second cog running it. Set some rendevous points in hub for inter-spin comms. Send a message - run this PUB in your class. Pass as a string. Pass some variables too. In the calling main program, call a PUB RunPub(stringptr,var1,var2,var3) where stringptr is the name of the pub to run. Set a flag when the pub is finished.
Reading this thread at the moment http://forums.parallax.com/showthread.php?104847-Spin-Object-Binary-Structure
Not sure if able to run two spin programs at once if their headers are going to overlap??
More research - Cluso asked exactly the same question a few months back http://forums.parallax.com/showthread.php?138534-Spin-modules-as-separate-relocatable-modules-(in-OS)&highlight=object+loader
More research - I think what I want is this http://forums.parallax.com/showthread.php?100406-DOL-Dynamic-Object-Loader BUT - load from external ram to make it faster (and to save on the code needed to run an SD card which can be unloaded once the ram has been filled with files).
PUB Main() MyFunction() ' instruction to put the next PUB at fixed location $8000 PUB Myfunction()
Maybe using the @ symbol to work out the location and then adding in a dummy array or something?
I've got this idea that you have a main program running in the lower half of hub ram and then in the upper half you can pull in groups of spin functions. Probably grouped in blocks of 16k and in groups where there is not too much 'churn'.
Up until now, two good reasons not to do this. First is that the SD driver code takes 1/4 to 1/3 of the memory. And second, most of the rest of the memory is taken up usually with a graphics buffer. So not much point.
But - if you have external memory, you can move all the files you need into external ram, and then the SD driver is only needed infrequently at most so you can unload it. And if you have a display with its own memory (touchscreen etc) then hub ram is not needed for a screen buffer.
So in that scenario, once some cogs are loaded up, the whole 32k of hub ram becomes a blank canvas.
One of the problems writing big programs (eg in C) is that the download times get longer and longer and debugging becomes a pain. 1 megabyte programs take ages to download. So maybe there is a way to precompile things (call them objects, or classes, or (shock!) DLLs) and then only have to compile them once. An example might be some string routines and put them all in a class called string.bin.
So the main program sits in the lower half of hub and handles moving 16k binary blobs in and out of the upper 16k of hub. Probably from external ram rather than an SD card because then you don't need the SD driver in ram any more.
I guess the hard part is the linking. If this works, the main program would have a dummy PUB located half way through ram. Pass some longs - or maybe a pointer to a string with the name of the pub you want to run in the class you have loaded. Then for the classes, compile those with a dummy array filling the first 16k so that the first pub appears half way through ram. Have one common entry and exit point for all called PUBs. Then all the jumps/calls don't need relocating.
Could this be possible?
Addit - I just thought of something. How about running two spin programs in parallel.
The "main" program runs in the lower 16k. The loadable one runs in the upper 16k and so all you do is have a 16k dummy array at the beginning. Maybe there is a bit of jump code that might overwrite the main program so perhaps the main program has a few bytes reserved at the beginning in an array that get overwritten. So the first program loads up a binary into upper 16k and then sets a second cog running it. Set some rendevous points in hub for inter-spin comms. Send a message - run this PUB in your class. Pass as a string. Pass some variables too. In the calling main program, call a PUB RunPub(stringptr,var1,var2,var3) where stringptr is the name of the pub to run. Set a flag when the pub is finished.
Reading this thread at the moment http://forums.parallax.com/showthread.php?104847-Spin-Object-Binary-Structure
Not sure if able to run two spin programs at once if their headers are going to overlap??
More research - Cluso asked exactly the same question a few months back http://forums.parallax.com/showthread.php?138534-Spin-modules-as-separate-relocatable-modules-(in-OS)&highlight=object+loader
More research - I think what I want is this http://forums.parallax.com/showthread.php?100406-DOL-Dynamic-Object-Loader BUT - load from external ram to make it faster (and to save on the code needed to run an SD card which can be unloaded once the ram has been filled with files).
Comments
I have another idea. I am going the reverse route and minimising the EEPROM size to 8KB (24LC64 33cents).
With a cog running the low level SD driver, we can issue a read sector instruction with sector address (like we do in CPM/ZiCog) to run a program object. So I am thinking a small stub program started in a cog could load itself directly from SD. So a binary blob to be loaded into a cog would take up to 4 sectors. With a size (long) we could determine the length and only load what was required which would make things faster.
One really simple place to start would be two spin programs running at the same time. One in the lower 16k, one in the upper 16k. We have the source for the spin interpreter so it should be possible to load it into another cog. The harder part might be the PUB list which seems to reside at the beginning of the program, before any VAR ie any opportunity to add in a dummy 16k array to push things higher up. So any 'loader' for the second spin program might need to edit it a bit. I'm thinking that if the program running in the upper part of ram were compiled to run in that part of ram, no relocation is needed.
I'm not entirely sure how you write programs using this structure. In a simplistic way, you take all the existing obex objects and recompile them so they run starting at 16k instead of 0k. Turn them all into binaries and put them on the SD card. Some have internal pasm, some do not but it does not matter.
Step 2 would be to move them all into external ram but we can leave that out for the moment and reload slower from an SD card.
So say you want to run the TV object with tv.start(param1, param2)
You don't really know that the TV object has the .start method. I guess you could load the original spin into a separate tab in the spin tool. You don't reference it in your main program though and it is not compiled when you hit the F10 button. But it is there so you know what all the PUBs are called. That way we don't need to reinvent any new spin tool or editor.
How do you reference the PUBs? By name or by number? Name would be easier to read. Call a routine and pass it a string "tv.start" and then it looks similar to existing spin? But if the file is already compiled, all you have is the number? Hmm - tricky. Maybe copy and paste a large comment section over - here are all the PUBs and here are their numbers. Don't change the order!
Somewhere along the line something has to convert that string into the jump to the correct PUB. Maybe a precompiler that notes that "tv.start" is the third pub in the list and then you can work out where it is in the binary file?
One of the original barriers in the DOL thread might have been the lack of many boards with SD cards. Now that most boards have an SD card, maybe we can revisit this?
More thoughts.
Would it be possible to use existing obex objects without modifying them in any way?
How about the "main" program goes in high ram, and the loaded object goes in low ram.
Ok, so the TV object has been compiled and is on the SD card as a binary. Of course, the compilation process has lost the names of the PUBs. But what you could do is take that object and remove everything except the PUBs, and turn that into a text file. (No need for the PRIs as they are internal to the object). That can produce a list of numbers which can be used to get the locations of where the PUBs are by comparing the text file with the compiled binary. That could even be a text file on the SD card so when you load a binary object, you also load a text file with the names of its PUB methods and their numbers. Then to use the object, you can pass the original names of the PUBs and then a processing routine can read the text file, determine the number and do the jump.
If that works, it makes it very easy to do all obex objects (so long as they are under 16k, which most would be).
Take Jazzed et al decoding the binary structure http://forums.parallax.com/showthread.php?104847-Spin-Object-Binary-Structure
Take a typical object. Run it once through a little program that searches for the string "PUB"
and also takes the .bin file and as per the structure in that link, produces a simple text file with the name and number
Maybe for reference later, include the variables passed (for human readers)
For the object loader, that should have enough information to extract the pointers to the pub locations in the binary.
For human readers, that text file is a useful quick reference to what the object can do (maybe include comments as well?)
To load a binary object, load both the binary into address 0 and the .txt file and there should be enough information to jump around in that binary as if it was part of the program. If it worked, why not process the entire obex and put all the binaries and .txt files on an SD card?