A proposal to develop a standard for communicating with cogs from any language!
RossH
Posts: 5,512
All,
Over in the thread about new drivers for C/C++, the discussion has moved to the idea of developing a standard interface to allow high level languages like Spin, C, Forth or BASIC (whether interpreted or LMM-based) to interact with code running in cogs. This could be driver code or library code (such as the various floating point librarys).
I think this is a good idea, and is something that has been missing from the Propeller for far too long, leading everyone (including me) to come up with their own solution.
Catalina uses the concept of a registry, which is 8 longs in a fixed high Hub RAM location which allows you to specify what programs are loaded in each cog, and point to a communication block which the cog will monitor and use to accept commands (including configuration commands, or requests to perform a function) and return results. Synchronization and handshaking are built in, but multi-cog access must be added over the top.
Catalina calls cog programs plugins, and the Catalina Registry/plugin mechanism is described in the attached document. This is just to kick off discussions - I'm not proposing this as the final solution. It does work quite well, but may not be ideal for all types of drivers/programs. If anyone has any ideas about a better solution, please post them here and we'll see if we can come up with a new scheme that everyone can use.
Ross.
Over in the thread about new drivers for C/C++, the discussion has moved to the idea of developing a standard interface to allow high level languages like Spin, C, Forth or BASIC (whether interpreted or LMM-based) to interact with code running in cogs. This could be driver code or library code (such as the various floating point librarys).
I think this is a good idea, and is something that has been missing from the Propeller for far too long, leading everyone (including me) to come up with their own solution.
Catalina uses the concept of a registry, which is 8 longs in a fixed high Hub RAM location which allows you to specify what programs are loaded in each cog, and point to a communication block which the cog will monitor and use to accept commands (including configuration commands, or requests to perform a function) and return results. Synchronization and handshaking are built in, but multi-cog access must be added over the top.
Catalina calls cog programs plugins, and the Catalina Registry/plugin mechanism is described in the attached document. This is just to kick off discussions - I'm not proposing this as the final solution. It does work quite well, but may not be ideal for all types of drivers/programs. If anyone has any ideas about a better solution, please post them here and we'll see if we can come up with a new scheme that everyone can use.
Ross.
Comments
I think the best solution is Chip's solution. You pass n longs, which have to be in the same order and which may include variables and arrays. These are the only variables that can be shared between the cog and the higher level language, but you can have as many or as few as you want.
What?
Ross.
Ross.
Start with the registry stuff on page 6 - the rest of the document describes how a typical plugin would be written to use it.
Ross.
I could have posted this in the other thread, but I really don't want to antagonize Chris, since he's one of the Good Guys.
-Phil
-Phil
Hi Phil - Spin/PASM programs can be used "in situ" by other languages provided the other language makes special (and somewhat complex) provisions for the Spin program data areas and stack space (I believe Catalina and Zog can both do this, but this is precisely the bit that Heater describes as "yucky!"). Spin was never designed with interoperability in mind, and so (for example) Spin programs cannot use C stack space and vice-versa. Spin also has no mechanism for sharing symbol information with other languages, which makes things very difficult!
But this thread is not really about general purpose intregation of Spin with other programs, it is specifically about integrating cog programs (generally written in PASM) with other languages. However, in the world of Spin it is often the case that some functionality is implemented in Spin - but that's because the program was originally expected to be used only from Spin. In many cases that Spin code can simply be "stripped off" and replaced with equivalent C code. This is the approach I first took with Catalina, but even so there was still a demand to be able to use Spin programs unmodified - so now I support that as well. But it is still difficult to interact with such programs, which is what this thread is really about - if there was a common mechanism to interact that was independent of the language used, it would make life easier for everyone.
Ross.
Slightly off-topic, but does Parallax intend to support Spin on the Prop 2 at all? Originally they said they did, but my feeling is they may have changed their minds.
Ross.
Parallax wants to ensure that anyone from K-12 to University curricula/programs, hobbyists, and large scale commercial enterprises can use their products.
-Phil
I'm open to suggestions, but I find that most things can be built on top of this simple structure fairly easily. Adding a few "conventions", like the one you mention - i.e. publishing a list of standard driver types - is also required. Perhaps specifying a standard "initialization" command would be useful? Also "fixing" the location of the registry in Hub RAM - e.g. always use the last 8 longs?.
Another thing missing at this level is synchronization across multiple cogs - I currently add that over the top (e.g. by using a lock around specific driver accesses - but perhaps there is a simpler mechanism we could use? One problem is knowing which lock to use, so one idea I have had since I developed this in the first place is to use a few of the extra registry bits to specify which lock is allocated to which cog - we don't need all 24 bits as address (even on the Prop 2 hub addresses will only need 18 bits) - so we could use 3 of the spare bits to specify that a specific lock has been allocated to this cog - I didn't want to just assume that lock 3 (for example) would always be allocated for use with cog 3. But perhaps this would add too much complexity?
Ross.
Ross.
Having a driver "publish it's usage" means the API is well defined. In C we have header files where we can define structs for I/O.
I suppose classifying a driver is not a bad idea, but to me that's just another part of the API. I wouldn't want to be restricted by some intermediary.
Locks can be passed in the mailbox struct. Is it a requirement? Maybe, but again that should be defined in the API.
I will never agree to fixing the start address of an 8 long registry or any mailbox. That is an implementation detail.
-Phil
-Phil
Perhaps someone can think of a clever mechanism for passing such fixed information between separately compiled processes?
Ross.
I've been thinking about this some more. I think we are in more agreement than I initially thought.
My big ask in any communications protocol is that all the communication longs are grouped together. I don't really care what those longs are or whether they are arrays or longs or even how many there are. I just feel that if they are grouped together then it makes code a lot more portable between languages, and it also greatly decreases the risk of code overwriting variables and introducing strange bugs, and it means that the cog pasm code can be compiled at another time (you can even distribute precompiled cog binaries on an SD card eg mouse.cog, keyboard.cog, display.cog).
Now what you are saying (I think) is that you have a registry that points to each of these blocks.
For Catalina, use it as it is. For backwards compatability with existing obex code, I think that could be changed fairly easily. Define your registry in spin as an 8 long array. Then instead of passing the location of a block with PAR, pass the location of the registry.
Hmm - and the cog number?
One problem - what if your obex code fills the cog with only one or two longs free? (not uncommon!). You are going to need an extra long or two for the code that reads the registry location, adds the cognumber and then extracts the pointer to the data block.
So just to check, in the cog code, you extract par which instead of pointing to the comms data block, it points to the registry which then points to the comms data block. And for the higher level code, you some code to work out where the comms data block is, and then incorporate this into the long that goes in the registry (with lock bits and other things?).
Yes, I'm certainly interested - but I'd encourage you to raise this in a separate thread. Even though I prefer C, Spin offers the easiest and best way for most users to take advantage of the unique capabilities of the Propeller - i.e. the ability to use multiple cores as "soft" periperals, or to do parallel processing. Without Spin, the Propeller is no match for the cheaper and/or faster alternatives.
Ross.
What would sync primitives look like?
Ross.
Ross.