Ross, I think we are missing context. We need pictures. A registry is not necessary in most pictures.
Two Basic Pictures
Embedded Hosted O/S
+---------------+ +---------------+
| User | | User |
+---------------+ +---------------+
| Application | | Application |
+---------------+ +---------------+
+---------------+ +---------------+
| Hardware | | O / S |
+---------------+ +---------------+
| Hardware |
+---------------+
No Registry Regsitry
In the first picture a registry has zero value. In the second it has value.
What causes us to need a registry in your view? O/S or other?
I guess I don't understand your question, because your pictures don't seem to clarity anything for me. Why is the O/S or lack thereof relevant?
Let me try another tack ...
A registry is not required if your cog programs do not need to interact. If they do, a registry is also not required if all of the following are true:
All your cog usage is static - i.e. you start each cog once and never stop them
All your comms blocks are static (and all the cogs that need them know where to find them)
You will never want to swap out or replace a cog program with a functional equivalent (e.g. TV vs VGA driver, FLOAT32 vs FLOAT_A & FLOAT_B)
You only ever start one instance of each cog program
You don't need to know which cog program is running in which cog
You don't need to know which cogs are currently free
A registry starts to be beneficial if any of these are true, but there are of course other means of satisfying each of these individual needs. But the more of these needs that your program has, the more beneficial a registry solution becomes since it solves all of these needs in one simple and uniform mechanism.
I guess I don't understand your question, because your pictures don't seem to clarity anything for me. Why is the O/S or lack thereof relevant?
Let me try another tack ...
A registry is not required if your cog programs do not need to interact. If they do, a registry is also not required if all of the following are true:
All your cog usage is static - i.e. you start each cog once and never stop them
All your comms blocks are static (and all the cogs that need them know where to find them)
You will never want to swap out or replace a cog program with a functional equivalent (e.g. TV vs VGA driver, FLOAT32 vs FLOAT_A & FLOAT_B)
You only ever start one instance of each cog program
You don't need to know which cog program is running in which cog
You don't need to know which cogs are currently free
A registry starts to be beneficial if any of these are true, but there are of course other means of satisfying each of these individual needs. But the more of these needs that your program has, the more beneficial a registry solution becomes since it solves all of these needs in one simple and uniform mechanism.
Sorry Ross, but I don't see how a registry is necessary for any of those items.
Obviously it isn't necessary - thousands of programs have been written without one. But it would have simplified many of those programs if they had one!
Because we don't have any common techniques (and I'm not stuck on a registry as the only possible one) everyone uses ad-hoc mechanisms for cog initialization and management, and especially for cog-to-cog communication - with the end result that calling a cog program from a high level language (which is one place where you typically need to do all these things) is quite difficult. Every one of the Propeller's high-level languages (as far as I know) uses a different mechanism, and every one of them has had to re-develop all their own basic drivers (or other cog programs) as a result.
Wouldn't you like Parallax customers to be able (as Heater suggested) to pick and choose amongst any of the appropriate "gold standard" OBEX drivers and know they will work with their chosen Spin, PASM, C, C++, Basic, Forth or Java compiler?
We have a common technique for cog initialization and management:
cognew, coginit, cogstop, and a communication block passed in the par register.
Every language for the Prop I know, use these functions, because they are given by the Propeller chip hardware and PASM instructions.
With Spin you can start, stop, restart cogs, change the par pointer when you restart a cog, get the cognumber when you start a cog, and you can have several instances of an object which starts a cog.
I just don't see the need of a registry for that.
If you need that two Assembly cogs can communicate directly, then pass the same communication block to both cogs when you start them.
Regarding Spin objects called from C:
Say you need an I2C driver with 400kHz bitclock. You go to the obex and find one in Spin.
This Spin driver has implemented the I2C communications in a PASM cog, because Spin is too slow.
So if you include this object into C, you will start 2 additional cogs (1 for the Spin interpreter, 1 for the PASM part).
Now if you call a I2C_wite in C this passes the command to Spin (which waits for commands and do nothing else), then the C part waits until the Spin part has finished. The Spin code passes the command to it's PASM cog (which waits for such commands and do nothing else), then the PASM part send the IC byte, signals the end to the spin code which signals the end to the C code.
If you make the same I2C driver in C, then the LMM code is fast enough to send the byte with 400 kHz with no additional cog.
We have a common technique for cog initialization and management:
cognew, coginit, cogstop, and a communication block passed in the par register.
Every language for the Prop I know, use these functions, because they are given by the Propeller chip hardware and PASM instructions.
That's true, Ariba - all cog functions are based on these primitives. But there is no way to tell whether the cog is running, or what program is running in the cog, or where the communications block for the cog is, or how to use it, or what services it offers, or what state it's in, or what lock to use to coordinate access from multiple cogs, or which cog to terminate if you want to stop the program ... (etc etc)
With Spin you can start, stop, restart cogs, change the par pointer when you restart a cog, get the cognumber when you start a cog, and you can have several instances of an object which starts a cog.
I just don't see the need of a registry for that.
There is no need for a registry if you don't need to allow other programs to interact with that cog.
If you need that two Assembly cogs can communicate directly, then pass the same communication block to both cogs when you start them.
And if you want to be able to communicate with them from a third cog? Or terminate one or both of the cogs later? To do these things you have to store (somewhere) the comms block and the cog numbers - i.e. you have to maintain a little "ad hoc" registry of your own - which is what many programs do.
Say you need an I2C driver with 400kHz bitclock. You go to the obex and find one in Spin.
This Spin driver has implemented the I2C communications in a PASM cog, because Spin is too slow.
So if you include this object into C, you will start 2 additional cogs (1 for the Spin interpreter, 1 for the PASM part).
Now if you call a I2C_wite in C this passes the command to Spin (which waits for commands and do nothing else), then the C part waits until the Spin part has finished. The Spin code passes the command to it's PASM cog (which waits for such commands and do nothing else), then the PASM part send the IC byte, signals the end to the spin code which signals the end to the C code.
If you make the same I2C driver in C, then the LMM code is fast enough to send the byte with 400 kHz with no additional cog.
But how? When you pull an object out of the OBEX, you have no idea what mechanism it uses to communicate, what parameters it must be passed on startup, how many cogs it needs, or how to request its services. Some of this information is encoded in the Spin parts of the object, but the Spin methods cannot be invoked from within a high level lanuage like C, since you can't easily pass a Spin program parameters without ... wait for it ... some standard mechanism not currently defined anywhere!. It also takes a large amount of messing about - and it is also ruinously inefficient.
I disagree. My main reason is that with your proposal (which is essentially the existing mechanism), you have to decide in advance whether you are going to need the registry. If you decide to change your program later and require registry access, you have to change your parameter block and all your initialization code - nasty!.
If you change your program and require registry access, you have to change your program -- but that is a tautology :-). It's no different from changing any of the other parameters to your program -- there will always be cases where programs change and their interfaces change as a result. I don't think it will be very common for programs that didn't need the registry to be changed to require it. But we could certainly adopt a standard whereby the registry pointer is always the first parameter (as I proposed later in my post) or where a pointer to the registry is kept in some fixed place.
The advantage of passing the registry as one of the parameters rather than passing the registry itself is that the COG can just skip over the registry pointer if it doesn't care about it. It also means that we can decouple the registry of services and the assignment of cogs. If it's the registry that's passed, the COG code has to know how to parse the registry and find itself, which is difficult if registry entries are not in 1-1 correspondence with COG numbers.
It better to always pass the registry and let programs get their parameter block from that. There is no overhead to this, and it means every program uses the same initialization code.
There's a conceptual overhead. As I think we agree, the most common case is that a driver just needs one COG, and often does not need registry services. So everyone can easily know how to write their driver code -- the parameter block is received in PAR, the first thing in the parameter block is a registry (which can often be ignored), the rest of the parameters are accessed the same way most programs do it today.
I do like the idea of a variable length registry, where services are registered rather than cogs. My main concern is that it would be much more complex to manage than a simple fixed length registry. Even the Catalina model is too complex for many - do you think the additional complexity is worthwhile, and will not just drive people away?
Most of the management will be done in the high level code, and presumably can be put into libraries that only need to be written once. The COG side is different (many drivers will be written in PASM). However, not many PASM COGs will need access to the registry, and the ones that do could be helped to some degree by the high level language. For example, a service that commonly needs floating point support could get pointers both to the registry and to the floating point service entry in the registry. We can also write some standard registry parsing macros that can be included in PASM code.
As I mentioned above, by far the simplest method is just to pass the registry, and let the cogs figure out everything else for themselves. For cogs that don't really need the registry, the overhead can be zero if you really need it to be, and the advantage is that it makes all cogs potentially equal.
The overhead of parsing the registry is not zero -- at least, it's not zero in programmer time. It may be that the code size overhead can be hidden in variable space, but that depends on the format of the registry. But I don't think the code size is the biggest hurdle for adoption of the registry -- it's just that it's perceived as unnecessary overhead for most drivers. Using PAR to point to a parameter block instead of to point to the registry is much easier to get people to comply with.
Moreover, I do feel strongly that having a registry of services rather than a registry of cogs is both more useful and more future-proof. Granted, Prop2 seems set to have 8 cogs as well, but will Prop3? How about dual Propeller systems -- there are some of those out there now, I believe. They can easily have more than 8 services running. For that matter, with multi-threaded LMM code or with cleverly written PASM code it's pretty easy to put more than one service on a cog.
This model model is also the simplest to implement, to explain and also - importantly! - to get people to comply with. If you want something more complex, I think you need to be able to justify the potential benefits.
I think having PAR point to a parameter block is simpler to explain and implement, and many drivers already comply with it. Changing existing drivers to accept the convention that the first parameter is a registry pointer is pretty simple -- it's just adding an "add par_ptr,#4" before they start parsing the parameters.
But there is no way to tell whether the cog is running, or what program is running in the cog, or where the communications block for the cog is, or how to use it, or what services it offers, or what state it's in, or what lock to use to coordinate access from multiple cogs, or which cog to terminate if you want to stop the program ... (etc etc)
To be fair, though, there's absolutely no need for anyone to know whether a cog is running, or what's running in a cog. The only thing anybody can do with a cog is to stop it, and that should be done only by the program that started the cog. There's no other way to talk directly to a cog, and no need to know a cog number to use a service. All you need to know is the address and format of the communication block (and potentially of a lock). Which cog or cogs the service is running on, or even whether it is running on a cog or on dedicated hardware attached to the propeller, is an implementation detail that users of a service don't need to know.
When you pull an object out of the OBEX, you have no idea what mechanism it uses to communicate, what parameters it must be passed on startup, how many cogs it needs, or how to request its services.
Unfortunately a registry on its own does not solve this problem. We also need a set of common communication protocols and standard object types, so that we know that a "video driver" always takes certain parameters, and a "floating point" service provides certain basic functions in a common way. That's a little beyond the scope of the registry or of this thread, I believe (although it's something that we should do!)
Regardless of whether one needs a registry or not, it is useful to develop a standard that would allow any language to use a COG driver.
Here is a minimalist proposal starting point:
COG drivers should be self contained where possible.
If extra data is needed for the cog, it should be passed as parameter(s).
All external control data should be passed as parameter(s).
The COG driver API must be described so that any interface code can use it.
Anything else?
I think that's a pretty good starting point. It might also be useful to have a way for the COG to signal the high level language that it's ready to start running (for example the first long word pointed to by PAR should be non-zero to start, and the COG should write 0 there when it is ready to receive and process requests).
In #5, I don't think the "non-zero before COGNEW" is necessary. A command sent prematurely will just wait to be executed until the driver is ready to start accepting commands. This also precludes the driver having to write a zero to the PAR variable to get the ball rolling. After that, the exchange is: caller writes non-zero to issue a command; driver writes zero to signal done.
1 COG drivers should be self contained where possible.
2 If extra data is needed for the cog, it should be passed as parameter(s).
3 All external control data should be passed as parameter(s).
4 The COG driver API must be described so that any interface code can use it.
5 COG should write zero back to *PAR value when ready for commands (non-zero before cognew).
That sounds brilliant and gets my vote.
I especially like #4 - the cog needs a good writeup describing how to use it. Maybe even with a Spin and C example.
And #5 is intriguing - it will save things like "add a 1 second delay here to make sure the driver has started".
The great thing about jazzed's proposal is that this is the minimum starting point. You can layer on a registry if you want, but you don't have to.
jazzed's proposal also allows intercog comms - note the location of one cog's parameter list and then pass this as a parameter to the second cog.
Another thought: how does this paradigm fit with the usual way that queuing is done in the hub for serial I/O drivers? It would be a real PITA to have to send each character to a cog-based driver, one at a time as a command, and wait for an acknowledgement, rather than just queuing it and bumping a hub pointer.
No problem Phil. I adapted Full Duplex Serial for use with Zog.
Only had to remove some initial set up that was done by Spin prior to launching the COG. After that the C code writes reads/writes from the cyclic buffers and tweaks the head/tail pointers in the same way the original Spin wrapper did. This all fits within Jazzed's proposal and could be used by wrappers in any other language.
In #5, I don't think the "non-zero before COGNEW" is necessary.
I disagree. A long initialization in the PASM can mean a meta-stable startup.
Here's what I mean in simple terms:
var
long command
pub start
command := -1 ' non-zero so repeat works
cognew(@pasm, @command)
repeat while command ' wait for cog to say it's ready
Heater is right of course:
The minimalist proposal does not preclude using queues because all queue pointers must be passed to the driver according to 2 and 3.
I disagree. A long initialization in the PASM can mean a meta-stable startup.
I don't see how. Heck, you could even post a command before calling cognew, and it wouldn't matter. The PASM cog won't bother reading it until it's ready to process it. Here's what I see happening without the startup handshaking:
1. PASM module finishes initializing.
2. PASM module reads command buffer. Non-zero?
2a. No: Loop back to #2.
3. Yes: Decode and process command.
4. Write results, if any.
5. Write a zero to command.
6. Jmp to #2.
Maybe I'm missing something, though. Can you show me an example where omitting the startup handshaking produces a metastable condition?
And if you want to be able to communicate with them from a third cog? Or terminate one or both of the cogs later? Ross.
IMHO: this implies a kind of pathological coupling between program/drivers that seems like it would be rather insidious and very hard to debug. While this is an interesting mental exercise -- I wonder if a real world example can be stated that would show why all these cogs would need to know so much about each other and their surrounding neighbors. Further unless this registry uses some kind of FIFO buffer for inter-cog communications then it also seems that messages/commands could be dropped and this in itself sounds like a doorway into chaos manor.
I prefer the simplist possible solution...but that is just me....your mileage may vary.
Another way we might do all of this more efficiently is with the LOCKxxx functions. If we want to maximize throughput (the prop can do 2 memory transfers per hub window, right?) we should LOCK signalling to avoid the repeat/wait loops that are inefficient. At least with LOCK signalling we can do all transfers in 1 shot instead of longs at a time.
To elaborate further, there are 8 locks and 8 cogs. We could use the set/test and clear/test to signal whether there is data waiting in the PAR location. Then a cog simply issues a COGID to get it's lock number and uses that to test for messages. You could simply pass commands, or get more elaborate with a formal message passing system. I have always liked the locks, though they are way underutilized because most programmers don't know how to use semaphores.
Further unless this registry uses some kind of FIFO buffer for inter-cog communications then it also seems that messages/commands could be dropped and this in itself sounds like a doorway into chaos manor.
I prefer the simplist possible solution...but that is just me....your mileage may vary.
Hey, does the Prop 1 have a PortB internally, or are those just wasted memory locations? If there was a PortB, we could use it for quick message passing.
What are you trying to show with your example? That's not typically the way commands and data are passed to a PASM routine. This is how it's usually done (for one data long and one result long):
SendCommand(cmd, data)
long[parptr+1] := data
long[parptr] := cmd
repeat while long[parptr]
return long[parptr+1]
COG drivers should be self contained where possible.
If extra data is needed for the cog, it should be passed as parameter(s).
All external control data should be passed as parameter(s).
The COG driver API must be described so that any interface code can use it.
COG should write zero back to *PAR value when ready for commands (non-zero before cognew).
Added 5 suggested by Eric and found useful by many others. Anything else?
Drat! my original reply to this post seems to have disappeared - how odd! Trying again ... (apologies if it suddenly appears again and this turns out to be a redundant post!).
Jazzed, I have no problem with you developing a proposal based on this scheme and seeing what other think, but it doesn't address the point of this thread, which is to develop a standard mechanism for communication with cogs.
For instance, with your proposal I don't know if a cog requires a comms block, or its size - so I don't know what I should pass in the par parameter. If it does require a comms block, I don't know what needs to be in it. I also don't know how to let other cogs know about the comms block so they can use it. I don't know what services the cog provides, and there is also no way for me to tell if a cog providing the services I need to use has been started yet or not.
But of course the main thing is that this proposal offers no standard for actually passing information to the cog once it is running, or receiving information from it. Assuming the cog does offer some interactive services, there is also no standard for requesting a service, indicating when the service is complete, or what the result is. Finally, there is no standard for coordinating access from multiple cogs that may want to use the same service, which means I have to provide my own mechanism and then communicate that information to all the cogs that may need to use it.
Apart from point 5 (added by Eric) your proposal seems to be just a restatement of the ad-hoc mechanisms we use now. I don't understand what the benefits of your proposal might be, or how it addresses the subject of this thread.
IMHO: this implies a kind of pathological coupling between program/drivers that seems like it would be rather insidious and very hard to debug. While this is an interesting mental exercise -- I wonder if a real world example can be stated that would show why all these cogs would need to know so much about each other and their surrounding neighbors. Further unless this registry uses some kind of FIFO buffer for inter-cog communications then it also seems that messages/commands could be dropped and this in itself sounds like a doorway into chaos manor.
I prefer the simplist possible solution...but that is just me....your mileage may vary.
Well, yes - my mileage does vary. I already do all these things, and I find them both interesting and useful. This is in fact precisely the kind of thing that the unique design of the Propeller facilitates, and where it manages to differentiate itself from other micro. These are fairly simple requirements of any multiprocessing system, or any concurrent language. If you don't want to use facilities such as these in your applications then that's fine - but then why use a Propeller in the first place?
Hey, does the Prop 1 have a PortB internally, or are those just wasted memory locations? If there was a PortB, we could use it for quick message passing.
Hi pedward,
My understanding is that port B is not wired internally, so it cannot be used to communicate between cogs.
I didn't say I didn't want to use these kinds of facilities. Instead I asked if anyone could provide a real world example of why something like a registry would be useful. There are several of us here that are well heeled in the hazards of concurrent programming (and last I checked plain-old-c was not all that well versed in it so that is why we all have to invent so much stuff to deal with that shortcoming). I've also had the misfortune to debug very complex multi-threaded programs that failed because of synchronization errors and out of order message passing -- which I think is the more meaningful level of a discussion like this. It is one thing to say that a bunch of programs can jabber freely with each other -- it is quite another to be able to prove that the concurrent behavior is also coherent and reliable.
Not trying to pick a war open here -- as I said this is an interesting topic -- but very very abstract. I think a pretty real world example may bring some additional understanding to the exact nature of the problem that fits the solution...that's all.
What are you trying to show with your example? That's not typically the way commands and data are passed to a PASM routine. This is how it's usually done (for one data long and one result long):
SendCommand(cmd, data)
long[parptr+1] := data
long[parptr] := cmd
repeat while long[parptr]
return long[parptr+1]
Same thing, just that one long was being used for bidirectional comms. I was trying to follow you steps with a pseudo code. There is some value in sending a data value aside form parptr+1, sending a ptr to some other buffer of data, like when implementing read() or write(). It's really 6-in-1, half dozen of the other.
Maybe I'm missing something, though. Can you show me an example where omitting the startup handshaking produces a metastable condition?
It's always possible for a driver to miss the first or even second command in a sequence. Propeller windowing does not queue anything. It is the user's responsibility to do the right thing. If you don't want to pre-initialize the command to a value and then wait for zero result, that is your choice. So, the requirement to pre-initialize may be unnecessary like other things and may even cause some driver writer to want to rely on that piece and see a failure. I guess the requirement is to wait for zero after cognew. Then it doesn't matter and those of us that want to know for sure that the driver is done before continuing can have it.
I didn't say I didn't want to use these kinds of facilities. Instead I asked if anyone could provide a real world example of why something like a registry would be useful. There are several of us here that are well heeled in the hazards of concurrent programming (and last I checked plain-old-c was not all that well versed in it so that is why we all have to invent so much stuff to deal with that shortcoming). I've also had the misfortune to debug very complex multi-threaded programs that failed because of synchronization errors and out of order message passing -- which I think is the more meaningful level of a discussion like this. It is one thing to say that a bunch of programs can jabber freely with each other -- it is quite another to be able to prove that the concurrent behavior is also coherent and reliable.
Not trying to pick a war open here -- as I said this is an interesting topic -- but very very abstract. I think a pretty real world example may bring some additional understanding to the exact nature of the problem that fits the solution...that's all.
Ok - I agree this is very interesting, but it is really a subject for another thread. To bring this back to the subject of this thread, and also to some simple real world cases, let's suppose someone has generated a new text-based VGA driver. Everyone thinks the new driver is the bees knees - but no-one can use it without rewriting all their programs (sometimes from the ground up!)
Of course, this new driver will most likely provide all sorts of neato functionality that I may or may not want to use, but even if I only want to use it as a basic text driver, I cannot currently do so without major programming effort. I don't know how to start the program, how to pass it the resources it may need, or how to interact with it once it is running.
Or let's consider another example (again a real world case, but this time not a driver). Someone invents a wonderful new floating point library cog program. But because we have no standard for communications, they use a different means of invoking the services it provides, passing parameters, and returning results. If I want to use it I again have to rewrite my program.
The current OBEX is full of cases like this - wonderful programs that you can only use by rewriting your application! Of course (as hobbyists) many of us are quite happy to "pull the bonnet" on our programs and rewire the guts of them each time - but not everyone wants to do this just to get a simple project working.
Of course, a registry is not the only solution to this - but it is one solution - or more correctly it is part of one solution. It is part of the solution because it provides a way of high-level languages discovering the services they need, and then it also gives you the basic information you need to communicate with them - i.e. via a comms block. Another part of the solution is a set of standards for how to communicate using the comms block.
However, a registry is not the only solution, and may not be the best solution. But so far in this thread it appears to be the only well-specified and practical solution.
Jazzed, I have no problem with you developing a proposal based on this scheme and seeing what other think, but it doesn't address the point of this thread, which is to develop a standard mechanism for communication with cogs.
But it is the point of this thread! It is as you say "what most of us use anyway", and that by convention (and wide-spread usage) is a standard.
Yes, it is what we already do to a point, but it should be formalized if people want to rely on it. It is a method "for communicating with cogs from any language!"
If you think it's off topic or that I'm being unreasonable some way, you are welcome to hit that little yellow warning button.
If it does require a comms block, I don't know what needs to be in it. I also don't know how to let other cogs know about the comms block so they can use it. I don't know what services the cog provides, and there is also no way for me to tell if a cog providing the services I need to use has been started yet or not.
It provides the services stated by the driver. Why is it important to have a typeof(COG)?
Okay. I see where you are going with this then. And these are a pair of very diverse examples. So, is this what you are trying to promote here is something along the lines of:
1. We have a common means for ICC (inter-cog-communications) that is "the registry" or something like that.
2. Each chunk of pogram code can elect to implement ICC as its means of interacting with other modules
3. Then the provider of that program "chunk" just has to document how to interact with it via the specific way it deals with its ICC interface.
4. Then adopters of the "chunk" just have to follow the documentation in order to integrate it.
Is that what you are after here or am I way off the mark still? --- and if I still don't get it -- then I do not expect that you correct me (possibly) errant thinking. I'll just continue to follow along and maybe my lightbulb will just suddenly go on.
Comments
I guess I don't understand your question, because your pictures don't seem to clarity anything for me. Why is the O/S or lack thereof relevant?
Let me try another tack ...
A registry is not required if your cog programs do not need to interact. If they do, a registry is also not required if all of the following are true:
- All your cog usage is static - i.e. you start each cog once and never stop them
- All your comms blocks are static (and all the cogs that need them know where to find them)
- You will never want to swap out or replace a cog program with a functional equivalent (e.g. TV vs VGA driver, FLOAT32 vs FLOAT_A & FLOAT_B)
- You only ever start one instance of each cog program
- You don't need to know which cog program is running in which cog
- You don't need to know which cogs are currently free
A registry starts to be beneficial if any of these are true, but there are of course other means of satisfying each of these individual needs. But the more of these needs that your program has, the more beneficial a registry solution becomes since it solves all of these needs in one simple and uniform mechanism.Ross.
Obviously it isn't necessary - thousands of programs have been written without one. But it would have simplified many of those programs if they had one!
Because we don't have any common techniques (and I'm not stuck on a registry as the only possible one) everyone uses ad-hoc mechanisms for cog initialization and management, and especially for cog-to-cog communication - with the end result that calling a cog program from a high level language (which is one place where you typically need to do all these things) is quite difficult. Every one of the Propeller's high-level languages (as far as I know) uses a different mechanism, and every one of them has had to re-develop all their own basic drivers (or other cog programs) as a result.
Wouldn't you like Parallax customers to be able (as Heater suggested) to pick and choose amongst any of the appropriate "gold standard" OBEX drivers and know they will work with their chosen Spin, PASM, C, C++, Basic, Forth or Java compiler?
Ross.
cognew, coginit, cogstop, and a communication block passed in the par register.
Every language for the Prop I know, use these functions, because they are given by the Propeller chip hardware and PASM instructions.
With Spin you can start, stop, restart cogs, change the par pointer when you restart a cog, get the cognumber when you start a cog, and you can have several instances of an object which starts a cog.
I just don't see the need of a registry for that.
If you need that two Assembly cogs can communicate directly, then pass the same communication block to both cogs when you start them.
Regarding Spin objects called from C:
Say you need an I2C driver with 400kHz bitclock. You go to the obex and find one in Spin.
This Spin driver has implemented the I2C communications in a PASM cog, because Spin is too slow.
So if you include this object into C, you will start 2 additional cogs (1 for the Spin interpreter, 1 for the PASM part).
Now if you call a I2C_wite in C this passes the command to Spin (which waits for commands and do nothing else), then the C part waits until the Spin part has finished. The Spin code passes the command to it's PASM cog (which waits for such commands and do nothing else), then the PASM part send the IC byte, signals the end to the spin code which signals the end to the C code.
If you make the same I2C driver in C, then the LMM code is fast enough to send the byte with 400 kHz with no additional cog.
Andy
So why not have a common mechanism instead?
But how? When you pull an object out of the OBEX, you have no idea what mechanism it uses to communicate, what parameters it must be passed on startup, how many cogs it needs, or how to request its services. Some of this information is encoded in the Spin parts of the object, but the Spin methods cannot be invoked from within a high level lanuage like C, since you can't easily pass a Spin program parameters without ... wait for it ... some standard mechanism not currently defined anywhere!. It also takes a large amount of messing about - and it is also ruinously inefficient.
Ross.
Here is a minimalist proposal starting point:
- COG drivers should be self contained where possible.
- If extra data is needed for the cog, it should be passed as parameter(s).
- All external control data should be passed as parameter(s).
- The COG driver API must be described so that any interface code can use it.
Anything else?If you change your program and require registry access, you have to change your program -- but that is a tautology :-). It's no different from changing any of the other parameters to your program -- there will always be cases where programs change and their interfaces change as a result. I don't think it will be very common for programs that didn't need the registry to be changed to require it. But we could certainly adopt a standard whereby the registry pointer is always the first parameter (as I proposed later in my post) or where a pointer to the registry is kept in some fixed place.
The advantage of passing the registry as one of the parameters rather than passing the registry itself is that the COG can just skip over the registry pointer if it doesn't care about it. It also means that we can decouple the registry of services and the assignment of cogs. If it's the registry that's passed, the COG code has to know how to parse the registry and find itself, which is difficult if registry entries are not in 1-1 correspondence with COG numbers.
There's a conceptual overhead. As I think we agree, the most common case is that a driver just needs one COG, and often does not need registry services. So everyone can easily know how to write their driver code -- the parameter block is received in PAR, the first thing in the parameter block is a registry (which can often be ignored), the rest of the parameters are accessed the same way most programs do it today.
Most of the management will be done in the high level code, and presumably can be put into libraries that only need to be written once. The COG side is different (many drivers will be written in PASM). However, not many PASM COGs will need access to the registry, and the ones that do could be helped to some degree by the high level language. For example, a service that commonly needs floating point support could get pointers both to the registry and to the floating point service entry in the registry. We can also write some standard registry parsing macros that can be included in PASM code.
The overhead of parsing the registry is not zero -- at least, it's not zero in programmer time. It may be that the code size overhead can be hidden in variable space, but that depends on the format of the registry. But I don't think the code size is the biggest hurdle for adoption of the registry -- it's just that it's perceived as unnecessary overhead for most drivers. Using PAR to point to a parameter block instead of to point to the registry is much easier to get people to comply with.
Moreover, I do feel strongly that having a registry of services rather than a registry of cogs is both more useful and more future-proof. Granted, Prop2 seems set to have 8 cogs as well, but will Prop3? How about dual Propeller systems -- there are some of those out there now, I believe. They can easily have more than 8 services running. For that matter, with multi-threaded LMM code or with cleverly written PASM code it's pretty easy to put more than one service on a cog.
I think having PAR point to a parameter block is simpler to explain and implement, and many drivers already comply with it. Changing existing drivers to accept the convention that the first parameter is a registry pointer is pretty simple -- it's just adding an "add par_ptr,#4" before they start parsing the parameters.
Eric
Unfortunately a registry on its own does not solve this problem. We also need a set of common communication protocols and standard object types, so that we know that a "video driver" always takes certain parameters, and a "floating point" service provides certain basic functions in a common way. That's a little beyond the scope of the registry or of this thread, I believe (although it's something that we should do!)
Eric
I think that's a pretty good starting point. It might also be useful to have a way for the COG to signal the high level language that it's ready to start running (for example the first long word pointed to by PAR should be non-zero to start, and the COG should write 0 there when it is ready to receive and process requests).
Eric
- COG drivers should be self contained where possible.
- If extra data is needed for the cog, it should be passed as parameter(s).
- All external control data should be passed as parameter(s).
- The COG driver API must be described so that any interface code can use it.
- COG should write zero back to *PAR value when ready for commands (non-zero before cognew).
Added 5 suggested by Eric and found useful by many others. Anything else?-Phil
That sounds brilliant and gets my vote.
I especially like #4 - the cog needs a good writeup describing how to use it. Maybe even with a Spin and C example.
And #5 is intriguing - it will save things like "add a 1 second delay here to make sure the driver has started".
The great thing about jazzed's proposal is that this is the minimum starting point. You can layer on a registry if you want, but you don't have to.
jazzed's proposal also allows intercog comms - note the location of one cog's parameter list and then pass this as a parameter to the second cog.
-Phil
Only had to remove some initial set up that was done by Spin prior to launching the COG. After that the C code writes reads/writes from the cyclic buffers and tweaks the head/tail pointers in the same way the original Spin wrapper did. This all fits within Jazzed's proposal and could be used by wrappers in any other language.
I disagree. A long initialization in the PASM can mean a meta-stable startup.
Here's what I mean in simple terms:
Heater is right of course:
The minimalist proposal does not preclude using queues because all queue pointers must be passed to the driver according to 2 and 3.
2. PASM module reads command buffer. Non-zero?
2a. No: Loop back to #2.
3. Yes: Decode and process command.
4. Write results, if any.
5. Write a zero to command.
6. Jmp to #2.
Maybe I'm missing something, though. Can you show me an example where omitting the startup handshaking produces a metastable condition?
-Phil
repeat while long[parptr] 'wait for par to become clear
long[parptr] := cmd 'send cmd
repeat while long[parptr] 'wait for ack
PUB sendCommandData(parptr,cmd,data)
sendCommand(parptr,cmd)
sendCommand(parptr,data)
PUB getResult(parptr)
repeat while long[parptr] == 0 'wait for result to appear
result:=long[parptr] 'copy result
long[parptr] := 0 'ack
IMHO: this implies a kind of pathological coupling between program/drivers that seems like it would be rather insidious and very hard to debug. While this is an interesting mental exercise -- I wonder if a real world example can be stated that would show why all these cogs would need to know so much about each other and their surrounding neighbors. Further unless this registry uses some kind of FIFO buffer for inter-cog communications then it also seems that messages/commands could be dropped and this in itself sounds like a doorway into chaos manor.
I prefer the simplist possible solution...but that is just me....your mileage may vary.
To elaborate further, there are 8 locks and 8 cogs. We could use the set/test and clear/test to signal whether there is data waiting in the PAR location. Then a cog simply issues a COGID to get it's lock number and uses that to test for messages. You could simply pass commands, or get more elaborate with a formal message passing system. I have always liked the locks, though they are way underutilized because most programmers don't know how to use semaphores.
Hey, does the Prop 1 have a PortB internally, or are those just wasted memory locations? If there was a PortB, we could use it for quick message passing.
What are you trying to show with your example? That's not typically the way commands and data are passed to a PASM routine. This is how it's usually done (for one data long and one result long):
Drat! my original reply to this post seems to have disappeared - how odd! Trying again ... (apologies if it suddenly appears again and this turns out to be a redundant post!).
Jazzed, I have no problem with you developing a proposal based on this scheme and seeing what other think, but it doesn't address the point of this thread, which is to develop a standard mechanism for communication with cogs.
For instance, with your proposal I don't know if a cog requires a comms block, or its size - so I don't know what I should pass in the par parameter. If it does require a comms block, I don't know what needs to be in it. I also don't know how to let other cogs know about the comms block so they can use it. I don't know what services the cog provides, and there is also no way for me to tell if a cog providing the services I need to use has been started yet or not.
But of course the main thing is that this proposal offers no standard for actually passing information to the cog once it is running, or receiving information from it. Assuming the cog does offer some interactive services, there is also no standard for requesting a service, indicating when the service is complete, or what the result is. Finally, there is no standard for coordinating access from multiple cogs that may want to use the same service, which means I have to provide my own mechanism and then communicate that information to all the cogs that may need to use it.
Apart from point 5 (added by Eric) your proposal seems to be just a restatement of the ad-hoc mechanisms we use now. I don't understand what the benefits of your proposal might be, or how it addresses the subject of this thread.
Perhaps if you could provide some examples?
Ross.
Well, yes - my mileage does vary. I already do all these things, and I find them both interesting and useful. This is in fact precisely the kind of thing that the unique design of the Propeller facilitates, and where it manages to differentiate itself from other micro. These are fairly simple requirements of any multiprocessing system, or any concurrent language. If you don't want to use facilities such as these in your applications then that's fine - but then why use a Propeller in the first place?
Ross.
Hi pedward,
My understanding is that port B is not wired internally, so it cannot be used to communicate between cogs.
Ross.
Not trying to pick a war open here -- as I said this is an interesting topic -- but very very abstract. I think a pretty real world example may bring some additional understanding to the exact nature of the problem that fits the solution...that's all.
Same thing, just that one long was being used for bidirectional comms. I was trying to follow you steps with a pseudo code. There is some value in sending a data value aside form parptr+1, sending a ptr to some other buffer of data, like when implementing read() or write(). It's really 6-in-1, half dozen of the other.
Ok - I agree this is very interesting, but it is really a subject for another thread. To bring this back to the subject of this thread, and also to some simple real world cases, let's suppose someone has generated a new text-based VGA driver. Everyone thinks the new driver is the bees knees - but no-one can use it without rewriting all their programs (sometimes from the ground up!)
Of course, this new driver will most likely provide all sorts of neato functionality that I may or may not want to use, but even if I only want to use it as a basic text driver, I cannot currently do so without major programming effort. I don't know how to start the program, how to pass it the resources it may need, or how to interact with it once it is running.
Or let's consider another example (again a real world case, but this time not a driver). Someone invents a wonderful new floating point library cog program. But because we have no standard for communications, they use a different means of invoking the services it provides, passing parameters, and returning results. If I want to use it I again have to rewrite my program.
The current OBEX is full of cases like this - wonderful programs that you can only use by rewriting your application! Of course (as hobbyists) many of us are quite happy to "pull the bonnet" on our programs and rewire the guts of them each time - but not everyone wants to do this just to get a simple project working.
Of course, a registry is not the only solution to this - but it is one solution - or more correctly it is part of one solution. It is part of the solution because it provides a way of high-level languages discovering the services they need, and then it also gives you the basic information you need to communicate with them - i.e. via a comms block. Another part of the solution is a set of standards for how to communicate using the comms block.
However, a registry is not the only solution, and may not be the best solution. But so far in this thread it appears to be the only well-specified and practical solution.
Ross.
Yes, it is what we already do to a point, but it should be formalized if people want to rely on it. It is a method "for communicating with cogs from any language!"
If you think it's off topic or that I'm being unreasonable some way, you are welcome to hit that little yellow warning button.
It is part of the API. Why do you have a problem with that?
1. We have a common means for ICC (inter-cog-communications) that is "the registry" or something like that.
2. Each chunk of pogram code can elect to implement ICC as its means of interacting with other modules
3. Then the provider of that program "chunk" just has to document how to interact with it via the specific way it deals with its ICC interface.
4. Then adopters of the "chunk" just have to follow the documentation in order to integrate it.
Is that what you are after here or am I way off the mark still? --- and if I still don't get it -- then I do not expect that you correct me (possibly) errant thinking. I'll just continue to follow along and maybe my lightbulb will just suddenly go on.