Integrating Spin with C
RossH
Posts: 5,519
Hi all,
Today I had a few hours spare, so I added the ability to launch Spin programs from Catalina (previously, Catalina could only run PASM programs). These Spin programs can be run in any available cogs, and they will execute in parallel with the executing C programs. Multiple Spin programs can be launched, or they can be started and stopped dynamically.
One of the big advantages of this is that the Spin programs can either load their own drivers, or share the Catalina Hardware Abstraction Layer, making use of all the various Catalina plugins instead. For instance, here is a simple interactive "hello world" type program written in Spin that uses the Catalina HMI plugin:
Another advantage is that you can either use the built-in Spin Interpreter (i.e. the one in ROM) or provide your own - e.g. you could use Cluso's Spin Interpreter that is supposed to offer improved Spin execution speed.
While this new capability is not "Big SPIN", it does allow you to launch Spin programs from a C program which is itself executing from XMM RAM. You can execute multiple Spin programs from XMM RAM at once, provided all the executing Spin programs fit in the 32k of available Hub RAM (when running).
Interacting between C programs and the Spin programs is also easy - each Spin program's code, data and stack blocks are dynamically allocated by the C program as required, and their address is therefore known to the C program. In particular, the address of each VAR block is known, so both Spin and C can access the Spin variables if required. Writing a C function that launches a Spin program to do something, then returns the results to the C program is now quite simple.
Preparing the Spin program is trivial - you just compile it using any Spin compiler (I use homespun, but bst or the Parallax Propeller Tool would work as well), then run a new enhanced spinc utility on it. This utility generates a C file suitable for inclusion within a Catalina C program, which can then launch the original Spin program at any time.
The Spin programs will share the Hub RAM with the stack of the C program that launched them - but if launching Spin programs is all the C program does then it won't need much stack space. Also, Hub RAM space will be required for any plugins that have been loaded - but these plugins will be shared between all Spin and C programs - i.e. it is no longer necessary for each Spin program to be compiled to include all the drivers it requires, and compiled Spin programs can now be trivially ported between different Propeller platforms - in fully compiled form!
All the building blocks for this have been available in Catalina for some time - I just decided it was time I pulled them all together.
I will release this functionality shortly as part of Catalina 3.1.
Ross.
Today I had a few hours spare, so I added the ability to launch Spin programs from Catalina (previously, Catalina could only run PASM programs). These Spin programs can be run in any available cogs, and they will execute in parallel with the executing C programs. Multiple Spin programs can be launched, or they can be started and stopped dynamically.
One of the big advantages of this is that the Spin programs can either load their own drivers, or share the Catalina Hardware Abstraction Layer, making use of all the various Catalina plugins instead. For instance, here is a simple interactive "hello world" type program written in Spin that uses the Catalina HMI plugin:
' ' A simple Spin program that demonstrates the use of the Catalina HMI plugin. ' ' This program is intended to be run by a Catalina C program. ' OBJ HMI : "Catalina_HMI" DAT str1 byte "hello, world!",13,10,0 str2 byte "goodbye, world!",13,10,0 PUB Hello | key HMI.find_hmi HMI.t_string(1, @str1) key := HMI.k_wait HMI.t_string(1, @str2)The Catalina_HMI object this program uses is just a thin "wrapper" object - when this Spin program is run under Catalina on any propeller platform, it will use whatever HMI plugin has already been loaded by the Catalina program that launched it. This HMI plugin could use TV, VGA or serial drivers - or even proxy drivers that get their input and display their output on a screen and keyboard connected to another Propeller chip! Of course, any C programs executing will also be able to use the plugin - i.e. the HMI plugin will be shared between all the executing C and Spin programs. In some cases, any contention will have to be resolved using locks - the same way this is currently done for some of the existing Catalina multi-cog and multi-thread demo programs.
Another advantage is that you can either use the built-in Spin Interpreter (i.e. the one in ROM) or provide your own - e.g. you could use Cluso's Spin Interpreter that is supposed to offer improved Spin execution speed.
While this new capability is not "Big SPIN", it does allow you to launch Spin programs from a C program which is itself executing from XMM RAM. You can execute multiple Spin programs from XMM RAM at once, provided all the executing Spin programs fit in the 32k of available Hub RAM (when running).
Interacting between C programs and the Spin programs is also easy - each Spin program's code, data and stack blocks are dynamically allocated by the C program as required, and their address is therefore known to the C program. In particular, the address of each VAR block is known, so both Spin and C can access the Spin variables if required. Writing a C function that launches a Spin program to do something, then returns the results to the C program is now quite simple.
Preparing the Spin program is trivial - you just compile it using any Spin compiler (I use homespun, but bst or the Parallax Propeller Tool would work as well), then run a new enhanced spinc utility on it. This utility generates a C file suitable for inclusion within a Catalina C program, which can then launch the original Spin program at any time.
The Spin programs will share the Hub RAM with the stack of the C program that launched them - but if launching Spin programs is all the C program does then it won't need much stack space. Also, Hub RAM space will be required for any plugins that have been loaded - but these plugins will be shared between all Spin and C programs - i.e. it is no longer necessary for each Spin program to be compiled to include all the drivers it requires, and compiled Spin programs can now be trivially ported between different Propeller platforms - in fully compiled form!
All the building blocks for this have been available in Catalina for some time - I just decided it was time I pulled them all together.
I will release this functionality shortly as part of Catalina 3.1.
Ross.
Comments
Is this the understatement of the week?
Very impressive, and I'm guessing it took more than a few hours.
So this launches a new spin interpreter with its associated spin code?
I see some intriguing possibilities here, especially with the ability to add existing obex code into C programs.
In particular, I see the use of the word "parallel" and I think this is something that really leverages the unique architecture of the propeller. I'm running my C program, it is happily doing what it needs to do, and in parallel, a Spin program is running as well.
Fantastic!
Ross.
It should work great on the design I am working on, as well as my new Game Platform + RamBlade II which is all in a 2" sq box.
I'm a very poor programmer, so obex code reuse is a key factor..
Massimo
This is a nice addition to Catalina. It seems like this could be easily extended to running Spin programs from the file system as well. The boot program in spinix starts up other Spin cogs that are located RAM, but it loads other Spin cogs from an SD card once the kernel is up. Have you documented how the HMI plugins work? Maybe this could become a standard for sharing I/O drivers.
Dave
I love the way Cayalina and Codeblocks is going. Great work!!!
@Dr_Acula You have also very interesting ideas and I enjoy reading them.
Nice feature! I assume this also means that the rules for writing PASM objects to use with C programs have been relaxed, and that it's okay now to initialize cog data from Spin?
-Phil
You spawn Spin cogs using the normal Catalina _coginit() function - but the new spinc utility automatically builds a function to do this for you.
@Dave,
The spinc program processes compiled spin binaries and embeds them in a C wrapper, along with a function to start them. It would be very easy to load the spin binaries at run-time from a binary file on SD card and launch them manually instead.
@Phil,
Yes - there are no longer any restrictions. To demonstrate, I have attached a copy of Bagger's 40*30 TV driver taken straight from the OBEX (here) - this version was initially compiled with the Parallax Propeller tool, then turned into a Catalina C function using the spinc utility. The only change I made to the original program was to set the pins for the C3. If you have a C3, you can just run it. It will execute the TV text demo program on the TV output. But if you also connect a serial terminal to the C3, the program will also spit out text from the C program that is still executing concurrently with the demo.
One interesting thing about this is that Bagger's demo program loads and runs its own TV drivers - but this is ok because when I compile it, I tell the C program to use the PC (i.e. serial) drivers for its HMI functions. Of course I could also use Bagger's drivers from C if I wanted to, simply by writing to the appropriate location in the var array.
I could use the same technique to load and use any ofher OBEX driver that I wanted to use from C - but in general it is better to use a plugin (where available) since it is more efficient and also provides a standardized interface to all drivers on all platforms.
To build the program, I used the following commands after (compiling the Spin demo using the Parallax Propeller Tool): And here is the code of the main C program (run_demo.c):
// include some standard header files #include <stdlib.h> #include <catalina_hmi.h> // include the utilities required to launch SPIN programs #include "spin_utilities.h" // include the Spin program (as generated by the SpinC utility) #include "demo.c" void main() { // to be accessible to the Spin interpreter the program, var and stack must // all be in Hub RAM - so we declare them locally in case this program is // running as an XMM program with the actual Spin program in XMM RAM: char prog[DEMO_PROG_SIZE]; char var[DEMO_VAR_SIZE]; char stack[DEMO_STACK_SIZE]; // start the Spin program (using the function built by the SpinC utility): start_DEMO(prog, var, stack); // now just do something to show we are still alive: while (1) { wait(1000); t_string(1, "This is Catalina C!\n\r"); } }The same technique could be used to include and run multiple Spin programs. Also, if you know the offset of the Spin variables in the var segment, you can access them by just using the var array. I'd really like to automate this process so that the spinc utility automatically builds a structure to access the var array - but since Spin compilers output no symbol data, to do that I would have to process the source file itself.Ross.
Thanks,
David
Hi David,
Yes, you can just coginit() the ROM Spin interpreter, passing it the appropriate parameters. Or you can coginit() your own interpreter if you prefer. Originally, I was embedding a copy of the Parallax Spin interpreter in the C program - until I realized I could just use the one already in ROM and save myself 2kb. But I have left that option in the code in case anyone wants to substitute their own interpreter.
From memory, the ROM Spin interpreter is at address $F004, and Chip's source code for the interpreter tells you what parameters you need to pass to it.
Too easy!
Ross.
True - now fixed!
Ross.
Here's another interesting example of integrating Catalina C with Spin. In this example, I will use both the standard tv_text and keyboard driver objects (both straight from the OBEX) - but access them from a Catalina C program.
There are actually many ways to do this - this example just demonstrates one method.
First, I write a simple program (in Spin) that starts the OBEX drivers, and then waits for commands to be sent from the C program (I call this program HMI.spin, since it does essentially what the normal Catalina HMI drivers do):
'' '' HMI - a simple Spin-based HMI for demonstrating C integration '' CON ' define the HMI commands KEY_WAIT = 1 KEY_READY = 2 PUT_CHR = 3 PUT_STR = 4 DAT welcome_msg BYTE "Welcome to Spin",13,0 VAR long command ' command will be written here long data ' data for command will be written/returned here OBJ kbd : "keyboard" scr : "tv_text" PUB Start kbd.Start (26, 27) scr.Start (12) data := 0 command := 0 scr.str(@welcome_msg) repeat case command KEY_WAIT: data := kbd.newkey command := 0 KEY_READY: if kbd.gotkey data := 1 else data := -1 command := 0 PUT_CHR: scr.out(data) command := 0 PUT_STR: scr.str(data) command := 0 until command < 0 scr.stop kbd.stopThen here is an example C prgram that uses the Spin HMI (I call this run_hmi.c) All this program does is echo any characters typed on the keyboard to the tv display (as upper case) - but just to add interest it translates any number keys into their equivalent text strings:// include some standard header files #include <stdlib.h> // include the utilities required to launch SPIN programs #include "spin_utilities.h" // include the Spin HMI program (as generated by the SpinC utility) #include "hmi.c" // define the HMI commands (must match those in the Spin HMI) #define KEY_WAIT 1 #define KEY_READY 2 #define PUT_CHR 3 #define PUT_STR 4 // define a macro to simplify execution of HMI commands #define HMI_CMD(c, d) *data = (long)(d); *command = (c); while (*command) ; void main() { // declare some local variables char welcome_msg[] = "Welcome to Catalina\r"; long *command; long *data; // declare local storage for the Spin HMI program char prog[HMI_PROG_SIZE]; char var[HMI_VAR_SIZE]; char stack[HMI_STACK_SIZE]; // point to the locations of the Spin variables // which we will use to communicate with the HMI command = (long *)(&var[0]); // 'command' is the first long in VAR data = (long *)(&var[4]); // 'data' is the second long in VAR // start the Spin HMI (which will in turn start the tv and keyboard) start_HMI(prog, var, stack); // print a welcome message HMI_CMD(PUT_STR, welcome_msg) // now read and process characters using the HMI while (1) { // wait for a key HMI_CMD(KEY_WAIT, *data); *data = *data & 0xFF; // interpret keys '0' .. '9' specially, otherwise // just echo each key we read (as upper case) switch (*data) { case '1': HMI_CMD(PUT_STR, " one "); break; case '2': HMI_CMD(PUT_STR, " two "); break; case '3': HMI_CMD(PUT_STR, " three "); break; case '4': HMI_CMD(PUT_STR, " four "); break; case '5': HMI_CMD(PUT_STR, " five "); break; case '6': HMI_CMD(PUT_STR, " six "); break; case '7': HMI_CMD(PUT_STR, " seven "); break; case '8': HMI_CMD(PUT_STR, " eight "); break; case '9': HMI_CMD(PUT_STR, " nine "); break; case '0': HMI_CMD(PUT_STR, " zero "); break; default: HMI_CMD(PUT_CHR, toupper(*data)); break; } } }Here are the commands I used to build this program - in this case I used homespun to compile the Spin program, but I could have used any Spin compiler:Note that I compiled the C program with the NO_HMI option because I don't want any of Catalina's HMI drivers included with this program - instead, this program uses the OBEX drivers!
I have attached the resulting binary which can be run on a C3 (i.e. the Spin TV and keyboard drivers have been set up to use the C3 pin numbers).
In theory, I could now replace Catalina's HMI drivers entirely with SPIN equivalents that use drivers taken straight from the OBEX. I could even use the registry (instead of the VAR mechanism) to communicate with them, which means they would appear via the standard C stdin and stdout mechanism instead of requiring special function calls and macros to access. So why wouldn't I do this? Well, for one thing these drivers would be much slower than the current Catalina HMI drivers, and for another thing the Spin HMI drivers would occupy precious Hub RAM, whereas Catalina's HMI drivers do not - once loaded and executing, they run entirely within cog RAM.
But it shows you what can be accomplished by using Catalina and Spin together!
Ross.
Massimo
Zog has been making use of Spin for I/O since day one. Any Spin objects can be used with C in a similar manner with Zog.
Strangely enough I've always been wanting to eliminate Spin from Zog systems. As you say, they waste space.
Hi Massimo,
Yes - the PASM part of each driver requires a cog, and the Spin part of all the drivers combined requires another cog (to execute the Spin interpreter). I could instead just load and start the PASM drivers from Spin and then terminate the Spin interpreter - but it would be much more complex to use the drivers because you would then have to replicate (in C) what the Spin part of each of the drivers does.
This is in fact pretty much what Catalina already does via its 'plugin' drivers - but Catalina does it in much less space and with much better performance.
However, people have repeatedly asked for this capability, so I thought I'd demonstrate how to do it even though there is little point in using this technique to replace the standard Catalina drivers (although I suppose if you had a platform that required a very non-standard video or keyboard driver there might be some point in it). But in fact the example illustrates a technique you can use to add any OBEX driver to a Catalina C program. HMI drivers are just the easiest ones to demonstrate.
Ross.
I prefer to think of it that Catalina has briefly descended to Zog's level - but only in order to demonstrate why you should not do things this way unless you have no alternative
Ross.
The version of ZOG that I checked into Google Code does not do its I/O through a Spin server. It loads the PASM portion of Obex drivers from C and talks to them through their mailbox interfaces. Most of these are drivers that were ported by jazzed. Once the ZOG C program is started, the entire of hub memory is used by ZOG. Of course, Ross's technique could be used to get some Spin code running again launched from the C code.
There is a reason you might want to do it this way.
My direction with Zog is to get the code out into external memory, serial RAM/FLASH say, with perhaps the stack and/or data in HUB. I've always hoped the byte code nature of the ZPU engine would make that quite efficient. This allows for quite large C programs. But then you have most of the HUB free for using any interesting Spin objects, like a FAT file system say. And they will be running at normal Spin/PASM speeds.
Think of it as being able to add a bunch of C code to an existing Spin/PASM application with out major changes to what you have already. All you need is a few free pins for the external memory and a couple of free COGS, one for Zog and one for the memory driver.
Yes, as you know that is also what I did in the run_zog program which eliminates all Spin from the Prop.
I kind of lean towards that Spin free and C/PASM only purity.
I think all "Gold Standard" PASM drivers should be usable in this way with no Spin dependency. Stand alone and language neutral that is. Presumably this will come to pass as Parallax gets GCC for the Prop working.
Yes, I know all about run_zog.spin! Where do you think I got the code I started with to build my zogload system? :-)
Thanks for providing such a clear example!
Ah yes, so you said once. Glad that run_zog code did not go to waste.
Hmm... did Zog hijack RossH's thread? Not really, after all the title does not specify which variety of C we are integrating with Spin:)
Sorry Ross.
No problem - that's why I started this as an independent thread. I'm interested to see that some of Zog's recent design directions will bring it in line with Catalina (driver architecture, memory models). I'm sure GCC will end up adopting the same solutions. One day we may be able to standardize drivers between the various C offerings.
Ross.
Well, good point I suppose - I just assumed the GCC team would want to be at least competitive against the existing solutions in the same space.
Ross.
If you were willing to join the effort, you could have some influence.