we would probably "reserve" some numbers (or number ranges) specifically for this purpose
That sounds great. I think this could be quite a simple thing to explain and to code.
If we define a service request number as a long, we have an awful lot of service requests (2^32 = over 4 billion) so I doubt we would run out.
You could reserve some numbers. Keyboard could be 0-10. Mouse could be 11-20. You might not use all the range. Actually, because there are so many numbers, you could have 255 sub numbers for each service request which means more than enough for all the objects I can think of. Keyboard is 0-255. Mouse 256-511. You can have numbers that do other abstract things, like reserve ram in hub. You could number these like IP addresses aaa.bbb.ccc.ddd so Keyboard would be 000.000.000.xxx and Mouse would be 000.000.001.xxx
This could save a lot of cog code too - I'm thinking of external memory drivers that only use a small part of a cog codespace. Why not combine them with the keyboard driver? Or display, and do the I/O in the blanking period? Many possibilities.
Some interactions could be flexible, eg if mouse request 000.000.001.000 was 'give me mouse x' and 000.000.001.001 was 'give me mouse y' then 000.000.001.002 could be 'give me x and y combined in a single long'.
Cluso has something similar with his original memory driver except that the command codes were single letters instead of numbers - "W" was "write a byte and "R" was "read a byte". The code that jumped around at the beginning to match the service request was really quite simple so I don't think this would add much more complexity as we already have working example(s).
You could then have some simple code that works out which cog the keyboard code is in and sends the request to the appropriate cog.
I can already see how this could free up some cogs in code I have written by combining code into one cog.
Ok, so I'm thinking video cogs and sound cogs, specialized things, and we have a LOT of different options already, suffering from the can't use it, until... syndrome described above.
Part of this effort would be breaking things into pieces to see where the common elements are.
With a video cog there almost always is:
Start pin, pointer to video buffer, display blanking state, mode or command (varies some, but doesn't have to in a lot of cases)
There can be:
Scan line buffer pointer, number of scan line buffers present (often depends on graphics service cogs), scan line handshake flag
resolution, number of characters, etc..
On a graphics service cog:
pointers to buffers, scan, video, command, arguments,
Common to both is
start, stop, standby, setup
For a sound cog, it could be:
Start, stop, frequency, pins, duration, etc...
Then we have things like SIDcog, which actually just implements a memory mapped SID.
Seems to me, some basic things are common:
start, stop, standby, pin, buffer pointer, setup
It almost seems to me like we need to break this task down by type, then have the method of communication operate for that type, or what should be quick, precise comms ends up being somewhat less precise and thicker comms, maybe more than one where there could be one. Something like the SIDcog, or a scan line, multi-cog video driver is going to be damn tough to sort out this way.
Common elements are:
States: Start, stop(die), standby
Buffers: They have sizes, alignments, addresses
Hardware: There are pins, clock, masks
Commands: These will vary, as do the arguments.
Edit: So are we headed toward a buffer always is specified this way, a pin and mask set this other way, commands, states, etc.... their way?
Another edit: Now the NyanCat demo uses a video driver that initalizes on it's own, requiring only a few parameters. That's a big step in this direction compared to older efforts that did anything and everything, including Chip's, which has a lot more options than are typically needed. I can see some video and sound code going this way, but not all. The full color tile driver with sprites code is going to need to do what it does, how it does it. Optimizations in that one really don't allow for any "thicker" operation, without a significant development effort, potentially incurring limitations.
That said, maybe this is only appropriate for CORE drivers, or a nice subset of cog code things, where we get most of the value most of the time. The rest of the time, somebody is pushing the edge, requiring others to push the edge and deal.
Ok, so I'm thinking video cogs and sound cogs, specialized things, and we have a LOT of different options already, suffering from the can't use it, until... syndrome described above.
Part of this effort would be breaking things into pieces to see where the common elements are.
<... lots of good and useful stuff here ...>
That said, maybe this is only appropriate for CORE drivers, or a nice subset of cog code things, where we get most of the value most of the time. The rest of the time, somebody is pushing the edge, requiring others to push the edge and deal.
Yikes!
I hadn't antipicated this (but I guess I probably should have). The initialization of cog programs - specificially, how to take into account and/or manage the hardware specific resources (like memory space, pins and clock speed and stuff) - is a whole 'nother ball game. There have been several attempts to come up with a common technique for doing this, and to be honest, I'm not even sure what the current "state of play" is - like cog-to-cog communcations, there seemed to be no "standard", so I just came up with my own techniques that seem to work - as (no doubt) have many others!
However, there is one point that needs to be made that may be new - the last time I saw this particular issue discussed, people were still talking only about how to do it in Spin programs. How to do this in a way that suits other languages as well opens an even bigger can of worms!
And I also seem to remember the same kind of arguments against developing a standard were being raised back then (i.e. along the lines of "why would we ever need this?" and "surely this is only relevant if you want to build an operating system?" and "I think everything is fine the way it is?" and "surely all we need is to document it sometwhere?" and "how about we just agree on a set of common conventions and move on?")
Like many complex issues discussed in these forums, this one has probably since either died a natural death, or descended into a seething morass of resentment, arguments and irreconcilable differences.
However, since we seem to have arrived in the same general neighbourhood already, I'm happy for this discussion to take place in this thread, especially since it is a related topic - but in my view it is definitely peripheral to the main topic, and will probably just use the same mechanism we use for cog to cog communication - i.e. we either specify these parameters in something like a registry (where the cog program can look them up) or we pass this information to the plugin as part of a common initialization structure at startup time, or we define an "initialization" service to be invoked at run time - in other words, we use the same comms mechanism for this as we use for everything else - if we can ever agree what that mechanism should be!
You could reserve some numbers. Keyboard could be 0-10. Mouse could be 11-20. You might not use all the range. Actually, because there are so many numbers, you could have 255 sub numbers for each service request which means more than enough for all the objects I can think of. Keyboard is 0-255. Mouse 256-511. You can have numbers that do other abstract things, like reserve ram in hub. You could number these like IP addresses aaa.bbb.ccc.ddd so Keyboard would be 000.000.000.xxx and Mouse would be 000.000.001.xxx
This sounds a little bit like Unix device identification, where you reserve a major number for a type of device, and a minor number for the actual device (and then Unix also uses a mechanism which translates file names to those numbers - so when applications access /dev/ttyS0 what really happens is that a driver for the 4,64 device is activated.)
Intriguing how that second article in wikipedia mentions CP/M. This stuff goes back a long way.
So - it would appear that Unix uses the idea that every physical device is represented as if it was a file. You write data to the file. You read data from the file. The "file" might be a keyboard driver, or serial port, or servo controller, or a sector on an SD card.
I like this. There is a beautiful simplicity about such a system.
And, as you say, the 'name' of a file like tty becomes a number.
People may laugh at this, but I wonder if such a system could be the beginnings of moving from a CP/M type operating system to a Unix type operating system?
Let's say we create some code to translate the text to numbers. I might prefer COM1 and you might prefer /dev/ttyS0. No matter - both get translated to a number which is the major and minor device number, and in reality might be a Long. Then call a lower level routine that takes the number and work out whether a cog has a serial port plugin running. Then write the data to a registry and the appropriate cog collects it and processes it.
The cog is running our standard serial pasm driver. But what this has done is translated the interaction with our driver into a language that Unix can understand. Unix is written in C (right?) and I understand it even predates CP/M. If we can run CP/M and we can run C, we must be able to run one of those early versions of Unix, right?
This is not without precedence. Juergen wrote the CP/M driver into pasm so he could do the interface between CP/M and Pasm. CP/M uses a program called PIP to move data between 'devices' such as a disk drive and a serial port, or a serial port and a display. This is the same thing, but using C and writing device drivers in pasm using a common standard.
Ross wants intercog comms using a registry. He wants every bit of code running in a cog to be registered (or to register itself) in the registry. This is what you need to do if you want a higher level program to know what is actually available. And cogs can write data to other cogs - just take your message (a data byte) and add a destination device number and send to the registry.
And if every 'device' now becomes a number, we can mix and match objects into cogs in all sorts of combinations. Why not put a keyboard and mouse into the same cog? Or change your mind later and split them up again - it does not matter because we are addressing a device number, not actual pasm code. If the device number is registered, it exists in the registry and you can talk to it. If it does not, return a -1 to the calling routine, which could then decide if it wanted to load a spare cog with that device number. Or even tell the user, "no sorry, we don't have com port 4, but if you want to tell us the pin numbers to use we can load it off an SD card and run it".
We have many objects already split into Spin and Pasm (I have done 9). We can start to think about a standard naming convention (why not use the one Unix uses?) . All you then need is a little bit of glue code to work out where a message is going and which cog to send it to and to return the result (or return an error to say the device does not exist, or load a cog and run it).
The 'secret' is to consider every device as a file that you can read and write to. You can write data to a file. You can write a startup command block to a file. You can read data from a file to see if the cog has started yet. You can read a stream of data from a file. This becomes the common standard.
Dr_A, I admire your ability to grab what the rest of us are mentioning in passing and getting to the core of it and make it into something useful! I'm not experienced enough with the Propeller to really judge what's possible or not, but to me what you describe looks great.
Intriguing how that second article in wikipedia mentions CP/M. This stuff goes back a long way.
Yes, good ideas tend to survive in some form or another. Some kind of computer science Darwinism I guess.. the device file concept has managed to stand the test of time, the only problem it has sometimes ran into is when a too limited range of possible major and/or major numbers were defined for an architecture to start with. So sometimes the minor numbers would 'overflow' and a second major number had to be acquired for the same type of device. Often solved by e.g. increasing the values from their original byte-sized (0-255) range.
People may laugh at this, but I wonder if such a system could be the beginnings of moving from a CP/M type operating system to a Unix type operating system?
Before people start arguing that 'Unix needs hardware with memory management!'.. well, yes, that's true.. nowadays. But, to be fair, there did exist a very old variant of Unix that ran on hardware which didn't have memory management. As far as I know that variant still exists, in some form, as the base for a version which runs on a microcontroller board somewhere.
But maybe 'Unix-type' is what we should concentrate on, because if anyone gets the idea that you're suggesting to port Unix (or get Linux running) then they'll just shoot you down with 'impossible!'.
I can see where the idea of the registry has appeal if you want to have the propeller setup to run "general purpose" programs from an SD card, where the program on the SD card looks to see if the peripherals needed are available, but does that use case happen often enough to burden every driver and program with needing to deal with a registry?
I did not mean to expand the discussion. Frankly, I am still having trouble internalizing what you are calling for exactly. I'm thinking, "ok, why not refactor a device cog or two", and I'm just not sure about what the registry really gets me in terms of using that device cog code in tandem with a program. Maybe it would help to put some limits out there, or some more boundaries about what is needed.
I can get from PAR being the direct reference to the shared memory block to PAR referencing something that contains the reference to the shared memory block. As soon as something like type gets mentioned, or the idea of files above, then I think it's not a shared memory block, but a stream, or structure. Breaks down for me there, because there are a lot of specific cases, or a lot of code needed to move the data, or time required to move the data, where it could just be a single HUB OP.
When I think "comms" between COGS, what is communicated?
A reference to memory?
Buffer?
State data?
Command?
How do we currently communicate? Sometimes it's just a value. Some people have made those values equal to characters. (that's cool)
For simple things, like the states, or some address, it could be a stream of data, but for the more complicated things, it can't, meaning a structure is needed, or am I missing something?
SIDcog, for example acts like a SID, so it, got registers, state bits, etc... A block of memory is needed, or one needs to maintain a stream of data.
Maybe that's the common path?
Either a stream or a block.
A stream of communication has a protocol. A block has a size.
1) "Registry" - Windows has had one of those for ages and pretty much everyone hates it. We don't want any registry in our Propeller code.
2) "Plugins" - Those are the things that I always need for my Firefox web browser. They either, can't be found, don't "plug in" if they are found or crash the browser when they are plugged in. We don't want any plugins on the Prop.
3) "Service discovery" - That's what CORBA is all about. CORBA is big, bloated, complex, slow. No one likes it. We don't want any service discovery on the Propeller.
OK, Tongue out of cheek. Whilst I was sleeping 3 or 4 pages of comments were added here. I have to catch up a bit.
Regardless of whether one needs a registry or not, it is useful to develop a standard that would allow any language to use a COG driver.
Here is a minimalist proposal starting point:
COG drivers should be self contained where possible.
If extra data is needed for the cog, it should be passed as parameter(s).
All external control data should be passed as parameter(s).
The COG driver API must be described so that any interface code can use it.
Anything else?
I was kind of hoping for a more standard way of passing data between objects. I don't know if it should be enforced by language or hardware but a clear standard for the obex would be nice.
A lot of us old time embedded systems guys just hate any kind of abstraction. Abstractions are:
1) Big, well bigger than not having them.
2) Slow, well slower than not having them.
3) Complex.Drat now I have to understand all that code between me and the hardware and spent time fixing bugs in there. It's hard enough stamping out the bugs in my own code.
4) Non portable. (Yes that's right, choose an abstraction and then you are stuck to those few platforms that support it or you have to recreate the abstraction for unsupported platforms.)
I was kind of hoping for a more standard way of passing data between objects. I don't know if it should be enforced by language or hardware but a clear standard for the obex would be nice.
What did you have in mind?
It may be best to start another thread if it has nothing to do with a registry.
It seems like a registry is only needed for an OS. Normal apps don't need it, and it would just complicate things to have to support it. An OS can provide system calls to completely hide the details of the registry. A non-OS app just needs to know the API for the drivers it uses.
Conventions for passing data to/from cog drivers would be useful, though.
I'm not sure we really need a one-size fits all convention for passing data to drivers, given that the drivers have different parameters and ways of operating anyway. But reducing it to just a few common cases would be handy.
The Catalina "request word/result word" model would work well for simple drivers.
An obvious alternative for more complicated drivers would be to use queues (as in the serial driver case): an input queue and an output queue. Requests for service go into the input queue; results are placed by the cog in the output queue. The queue size would vary from driver to driver, but the structure would always be the same. The advantage of this is that asynchronous requests are handled very easily, and the model is easily adapted to the Unix "stream of data" model (it looks just like a terminal).
Eric,
I was about to make the same suggestion. All comms between COGs go through queues or "channels". As in the good old Transputer or the modern day MCU that shall remain unnamed here. That is "Communicating Sequential Processes", CSP.
With a potential bonus that processes can be moved out to COGs on other Propeller chips with those channel ends being conected by some serial comms link. No changes to the processes at either end required.
With a potential bonus that processes can be moved out to COGs on other Propeller chips with those channel ends being conected by some serial comms link. No changes to the processes at either end required.
That sounds similar to what's apparently available in PropFORTH. I'd love to see the ability to run code in COGs on other Propellers.
I was kind of hoping for a more standard way of passing data between objects. I don't know if it should be enforced by language or hardware but a clear standard for the obex would be nice.
What did you have in mind?
It may be best to start another thread if it has nothing to do with a registry.
Why start another thread? Read the title of this thread again! Any solution we come up with would also be usable by Spin.
A registry is just one part of one proposed solution - one that I proposed as an example to get the ball rolling. Non-registry solutions are certainly possible, although I can't quite see how they might work since no-one has proposed one yet.
But I don't really understand why people are obsessing about the "registry" idea itself - that's kind of missing the main point.
However (for those who do want to obsess about these things) Catalina's "registry" is trivially simple - it is nothing more than 8 longs of Hub RAM that store the type of plugin running in each cog, plus (optionally) a pointer to a comms block allocated for that plugin. This allows a high-level language to find the plugin it needs and then communicate with it - without having to have anything hard-coded, or compiled or linked in (other than the address of the registry). This represents the simplest solution I could think of that works. But there may be others (and they may be better).
That idea of coming up with the simplest possible solution that will work is important, because I would like us to have a language-independent solution (again, refer to the title of this thread!). Different languages use different mechanisms for compilation and linkage (and may use different PASM assemblers, or even offer no PASM capability at all) so I don't think a language-independent option should depend on capabilities provided only by some languages.
With a registry solution you don't even need to be able to compile new plugins to use them - as long as you have a binary blob, you can load it into a cog and it will work as advertised. How often have people complained that you cannot distribute binary programs with Spin? Well, if we can come up with a solution here, at least you could distribute binary versions of plugins! But then if you can do that, you can also do the opposite - i.e. distribute your application as a binary - sans plugins - and allow people to add whatever platform-dependent plugins they need to make that application work on their Propeller.
If someone can come up with a better solution than the one proposed, then let's hear about it!
No, it's not. If I've not made it perfectly clear yet, I will say so now - I will change Catalina if we can agree on a standard solution for communicating between different high level language programs and cog programs. If the standard solution doesn't include a registry, then so what? - if I still need one, I can always add one on top of whatever we come up with. So could anyone else.
But so far, concrete proposals have been made (as far as I can recall) only by Dave and Eric. No-one else (apologies if I have missed a post) has proposed anything that could be implemented to achieve the intended objective. Perhaps because people didn't understand the intended objective of this thread originally. I'll accept responsibility for that.
Conventions for passing data to/from cog drivers would be useful, though.
I'm not sure we really need a one-size fits all convention for passing data to drivers, given that the drivers have different parameters and ways of operating anyway. But reducing it to just a few common cases would be handy.
The Catalina "request word/result word" model would work well for simple drivers.
An obvious alternative for more complicated drivers would be to use queues (as in the serial driver case): an input queue and an output queue. Requests for service go into the input queue; results are placed by the cog in the output queue. The queue size would vary from driver to driver, but the structure would always be the same. The advantage of this is that asynchronous requests are handled very easily, and the model is easily adapted to the Unix "stream of data" model (it looks just like a terminal).
Eric
Hi Eric,
Absolutely! I agree we may end up with (or want the design to be able to encompass) more than one "standard" mechanism. C "streams" are very byte-oriented, and this works well for most simple drivers - but other models would work better in other cases - e.g. inter-process communications may want to have a standard mechanism for defining and using blocks of shared memory.
Eric,
I was about to make the same suggestion. All comms between COGs go through queues or "channels". As in the good old Transputer or the modern day MCU that shall remain unnamed here. That is "Communicating Sequential Processes", CSP.
With a potential bonus that processes can be moved out to COGs on other Propeller chips with those channel ends being conected by some serial comms link. No changes to the processes at either end required.
I agree - I'd love to be able to communicate to a process running on a remote Prop as easily as I can communicate with one running on another cog on the same Prop!
I have no problem with the CSP model other than that it has proven to be virtually unusable in practice. As a theoretical model it actually works very well.
Another take on this whole landscape (bumpy though it appears to be for some). Define a CSP solution enough so that it works and is demonstrable for inter-operation between coglets. Then once that is in place add this CSP "in front of" some existing obex favorites and show how it can work. Nice bridging solution and a working sample is worth a thousand posts
What did you have in mind?
It may be best to start another thread if it has nothing to do with a registry.
I'm not opposed to a registry of some type. If I understand the concept of the registry as it would be used here, each cog would have a slot or long in a table which would point to the data that other objects/cogs would be accessing.
I thought the purpose of this thread was to discuss a standard for obj/cogs to communicate in a language independent way not about the implementation of a registry. If that's not the case then I apologize and will ignore this thread from here on out.
I'm not opposed to a registry of some type. If I understand the concept of the registry as it would be used here, each cog would have a slot or long in a table which would point to the data that other objects/cogs would be accessing.
I thought the purpose of this thread was to discuss a standard for obj/cogs to communicate in a language independent way not about the implementation of a registry. If that's not the case then I apologize and will ignore this thread from here on out.
Hi 4x5n,
You are quite right - this thread is about a standard for cog-to-cog communications.
This thread is not about a registry, except as one possible mechanism to help achieving that.
After reading the last 4 pages of this thread, I think I have misunderstood the subject of this thread (as so many others).
I was thinking you want a standard way to communicate between the API part and the assembly-cog part of a driver object. But now I think you want cog drivers, which are totally independent of an API and do all the functions in the cog itself. And there is a standardized way to connect with the services in the cog from any language (even PASM).
If this is true, then almost all of the Spin drivers must written new, because a big part of the functionality of the objects are in the Spin part, and the Assembly cog only handles some low level stuff. It would make no sense to connect to such a cog from another language without rewriting the Spin part in the new language.
There are some drivers for Sphinx, that are written so, that the whole functions are in the cog, you only communicate through the "Rendevous-area" with it. The Sphinx-TV driver for example gets only the characters to show and does all what is necessary in the cog, even the screen buffer is in the cog.
In contrast the TV_Text object in Spin starts only a TV driver cog, and does not communicate with that cog any further. All the functions to draw characters are in the Spin part, which just puts the characters at the right position into the screenbuffer. The TV-cog does nothing than output the screen buffer as a TV signal.
So am I correct with that? And should the thread title then not be something like:
"Let us make a new class of cog drivers, which have all the functionality in the cog and commuinicate only through standardized services, so they are independant of a high level programming language."
But I don't really understand why people are obsessing about the "registry" idea itself - that's kind of missing the main point.
It may be a case of shell shock from dealing with the windows registry.
However (for those who do want to obsess about these things) Catalina's "registry" is trivially simple - it is nothing more than 8 longs of Hub RAM that store the type of plugin running in each cog, plus (optionally) a pointer to a comms block allocated for that plugin. This allows a high-level language to find the plugin it needs and then communicate with it - without having to have anything hard-coded, or compiled or linked in (other than the address of the registry).
Reminds me of the 8080/Z80 interrupt vectors or the jump tables/address pointers I used for simple monitor programs and more complex menu systems. I think something simple and light weight like this is what we need. No reason CSP could not be used as part of this for cog-cog communications within and between props as well.
No, it's not. If I've not made it perfectly clear yet, I will say so now - I will change Catalina if we can agree on a standard solution for communicating between different high level language programs and cog programs. If the standard solution doesn't include a registry, then so what? - if I still need one, I can always add one on top of whatever we come up with. So could anyone else.
But so far, concrete proposals have been made (as far as I can recall) only by Dave and Eric. No-one else (apologies if I have missed a post) has proposed anything that could be implemented to achieve the intended objective. Perhaps because people didn't understand the intended objective of this thread originally. I'll accept responsibility for that.
But I think we've got that sorted now.
Ross.
@Ross,
Ok, I agree. Sorted. Registry as defined by your document would be used only if necessary.
So, reset. From the top down:
What are the functional requirements for a standard for communicating with cogs from any language?
I would like to see your concise point by point definition. People have talked about all kinds of stuff in this thread.
Please compose a short point by point list of statements that define the requirements that can be objectively considered.
If you've done this already, please post a pointer. I do not consider your .pdf a list of requirements.
Comments
That sounds great. I think this could be quite a simple thing to explain and to code.
If we define a service request number as a long, we have an awful lot of service requests (2^32 = over 4 billion) so I doubt we would run out.
You could reserve some numbers. Keyboard could be 0-10. Mouse could be 11-20. You might not use all the range. Actually, because there are so many numbers, you could have 255 sub numbers for each service request which means more than enough for all the objects I can think of. Keyboard is 0-255. Mouse 256-511. You can have numbers that do other abstract things, like reserve ram in hub. You could number these like IP addresses aaa.bbb.ccc.ddd so Keyboard would be 000.000.000.xxx and Mouse would be 000.000.001.xxx
This could save a lot of cog code too - I'm thinking of external memory drivers that only use a small part of a cog codespace. Why not combine them with the keyboard driver? Or display, and do the I/O in the blanking period? Many possibilities.
Some interactions could be flexible, eg if mouse request 000.000.001.000 was 'give me mouse x' and 000.000.001.001 was 'give me mouse y' then 000.000.001.002 could be 'give me x and y combined in a single long'.
Cluso has something similar with his original memory driver except that the command codes were single letters instead of numbers - "W" was "write a byte and "R" was "read a byte". The code that jumped around at the beginning to match the service request was really quite simple so I don't think this would add much more complexity as we already have working example(s).
You could then have some simple code that works out which cog the keyboard code is in and sends the request to the appropriate cog.
I can already see how this could free up some cogs in code I have written by combining code into one cog.
Part of this effort would be breaking things into pieces to see where the common elements are.
With a video cog there almost always is:
Start pin, pointer to video buffer, display blanking state, mode or command (varies some, but doesn't have to in a lot of cases)
There can be:
Scan line buffer pointer, number of scan line buffers present (often depends on graphics service cogs), scan line handshake flag
resolution, number of characters, etc..
On a graphics service cog:
pointers to buffers, scan, video, command, arguments,
Common to both is
start, stop, standby, setup
For a sound cog, it could be:
Start, stop, frequency, pins, duration, etc...
Then we have things like SIDcog, which actually just implements a memory mapped SID.
Seems to me, some basic things are common:
start, stop, standby, pin, buffer pointer, setup
It almost seems to me like we need to break this task down by type, then have the method of communication operate for that type, or what should be quick, precise comms ends up being somewhat less precise and thicker comms, maybe more than one where there could be one. Something like the SIDcog, or a scan line, multi-cog video driver is going to be damn tough to sort out this way.
Common elements are:
States: Start, stop(die), standby
Buffers: They have sizes, alignments, addresses
Hardware: There are pins, clock, masks
Commands: These will vary, as do the arguments.
Edit: So are we headed toward a buffer always is specified this way, a pin and mask set this other way, commands, states, etc.... their way?
Another edit: Now the NyanCat demo uses a video driver that initalizes on it's own, requiring only a few parameters. That's a big step in this direction compared to older efforts that did anything and everything, including Chip's, which has a lot more options than are typically needed. I can see some video and sound code going this way, but not all. The full color tile driver with sprites code is going to need to do what it does, how it does it. Optimizations in that one really don't allow for any "thicker" operation, without a significant development effort, potentially incurring limitations.
That said, maybe this is only appropriate for CORE drivers, or a nice subset of cog code things, where we get most of the value most of the time. The rest of the time, somebody is pushing the edge, requiring others to push the edge and deal.
Yikes!
I hadn't antipicated this (but I guess I probably should have). The initialization of cog programs - specificially, how to take into account and/or manage the hardware specific resources (like memory space, pins and clock speed and stuff) - is a whole 'nother ball game. There have been several attempts to come up with a common technique for doing this, and to be honest, I'm not even sure what the current "state of play" is - like cog-to-cog communcations, there seemed to be no "standard", so I just came up with my own techniques that seem to work - as (no doubt) have many others!
However, there is one point that needs to be made that may be new - the last time I saw this particular issue discussed, people were still talking only about how to do it in Spin programs. How to do this in a way that suits other languages as well opens an even bigger can of worms!
And I also seem to remember the same kind of arguments against developing a standard were being raised back then (i.e. along the lines of "why would we ever need this?" and "surely this is only relevant if you want to build an operating system?" and "I think everything is fine the way it is?" and "surely all we need is to document it sometwhere?" and "how about we just agree on a set of common conventions and move on?")
Like many complex issues discussed in these forums, this one has probably since either died a natural death, or descended into a seething morass of resentment, arguments and irreconcilable differences.
However, since we seem to have arrived in the same general neighbourhood already, I'm happy for this discussion to take place in this thread, especially since it is a related topic - but in my view it is definitely peripheral to the main topic, and will probably just use the same mechanism we use for cog to cog communication - i.e. we either specify these parameters in something like a registry (where the cog program can look them up) or we pass this information to the plugin as part of a common initialization structure at startup time, or we define an "initialization" service to be invoked at run time - in other words, we use the same comms mechanism for this as we use for everything else - if we can ever agree what that mechanism should be!
Ross.
-Tor
Intriguing how that second article in wikipedia mentions CP/M. This stuff goes back a long way.
So - it would appear that Unix uses the idea that every physical device is represented as if it was a file. You write data to the file. You read data from the file. The "file" might be a keyboard driver, or serial port, or servo controller, or a sector on an SD card.
I like this. There is a beautiful simplicity about such a system.
And, as you say, the 'name' of a file like tty becomes a number.
People may laugh at this, but I wonder if such a system could be the beginnings of moving from a CP/M type operating system to a Unix type operating system?
Let's say we create some code to translate the text to numbers. I might prefer COM1 and you might prefer /dev/ttyS0. No matter - both get translated to a number which is the major and minor device number, and in reality might be a Long. Then call a lower level routine that takes the number and work out whether a cog has a serial port plugin running. Then write the data to a registry and the appropriate cog collects it and processes it.
The cog is running our standard serial pasm driver. But what this has done is translated the interaction with our driver into a language that Unix can understand. Unix is written in C (right?) and I understand it even predates CP/M. If we can run CP/M and we can run C, we must be able to run one of those early versions of Unix, right?
This is not without precedence. Juergen wrote the CP/M driver into pasm so he could do the interface between CP/M and Pasm. CP/M uses a program called PIP to move data between 'devices' such as a disk drive and a serial port, or a serial port and a display. This is the same thing, but using C and writing device drivers in pasm using a common standard.
Ross wants intercog comms using a registry. He wants every bit of code running in a cog to be registered (or to register itself) in the registry. This is what you need to do if you want a higher level program to know what is actually available. And cogs can write data to other cogs - just take your message (a data byte) and add a destination device number and send to the registry.
And if every 'device' now becomes a number, we can mix and match objects into cogs in all sorts of combinations. Why not put a keyboard and mouse into the same cog? Or change your mind later and split them up again - it does not matter because we are addressing a device number, not actual pasm code. If the device number is registered, it exists in the registry and you can talk to it. If it does not, return a -1 to the calling routine, which could then decide if it wanted to load a spare cog with that device number. Or even tell the user, "no sorry, we don't have com port 4, but if you want to tell us the pin numbers to use we can load it off an SD card and run it".
We have many objects already split into Spin and Pasm (I have done 9). We can start to think about a standard naming convention (why not use the one Unix uses?) . All you then need is a little bit of glue code to work out where a message is going and which cog to send it to and to return the result (or return an error to say the device does not exist, or load a cog and run it).
The 'secret' is to consider every device as a file that you can read and write to. You can write data to a file. You can write a startup command block to a file. You can read data from a file to see if the cog has started yet. You can read a stream of data from a file. This becomes the common standard.
Tor - this is brilliant!
Yes, good ideas tend to survive in some form or another. Some kind of computer science Darwinism I guess.. the device file concept has managed to stand the test of time, the only problem it has sometimes ran into is when a too limited range of possible major and/or major numbers were defined for an architecture to start with. So sometimes the minor numbers would 'overflow' and a second major number had to be acquired for the same type of device. Often solved by e.g. increasing the values from their original byte-sized (0-255) range.
Before people start arguing that 'Unix needs hardware with memory management!'.. well, yes, that's true.. nowadays. But, to be fair, there did exist a very old variant of Unix that ran on hardware which didn't have memory management. As far as I know that variant still exists, in some form, as the base for a version which runs on a microcontroller board somewhere.
But maybe 'Unix-type' is what we should concentrate on, because if anyone gets the idea that you're suggesting to port Unix (or get Linux running) then they'll just shoot you down with 'impossible!'.
-Tor
C.W.
I did not mean to expand the discussion. Frankly, I am still having trouble internalizing what you are calling for exactly. I'm thinking, "ok, why not refactor a device cog or two", and I'm just not sure about what the registry really gets me in terms of using that device cog code in tandem with a program. Maybe it would help to put some limits out there, or some more boundaries about what is needed.
I can get from PAR being the direct reference to the shared memory block to PAR referencing something that contains the reference to the shared memory block. As soon as something like type gets mentioned, or the idea of files above, then I think it's not a shared memory block, but a stream, or structure. Breaks down for me there, because there are a lot of specific cases, or a lot of code needed to move the data, or time required to move the data, where it could just be a single HUB OP.
When I think "comms" between COGS, what is communicated?
A reference to memory?
Buffer?
State data?
Command?
How do we currently communicate? Sometimes it's just a value. Some people have made those values equal to characters. (that's cool)
For simple things, like the states, or some address, it could be a stream of data, but for the more complicated things, it can't, meaning a structure is needed, or am I missing something?
SIDcog, for example acts like a SID, so it, got registers, state bits, etc... A block of memory is needed, or one needs to maintain a stream of data.
Maybe that's the common path?
Either a stream or a block.
A stream of communication has a protocol. A block has a size.
This whole idea sucks because:
1) "Registry" - Windows has had one of those for ages and pretty much everyone hates it. We don't want any registry in our Propeller code.
2) "Plugins" - Those are the things that I always need for my Firefox web browser. They either, can't be found, don't "plug in" if they are found or crash the browser when they are plugged in. We don't want any plugins on the Prop.
3) "Service discovery" - That's what CORBA is all about. CORBA is big, bloated, complex, slow. No one likes it. We don't want any service discovery on the Propeller.
OK, Tongue out of cheek. Whilst I was sleeping 3 or 4 pages of comments were added here. I have to catch up a bit.
[runs, screaming frantically]
I was kind of hoping for a more standard way of passing data between objects. I don't know if it should be enforced by language or hardware but a clear standard for the obex would be nice.
A lot of us old time embedded systems guys just hate any kind of abstraction. Abstractions are:
1) Big, well bigger than not having them.
2) Slow, well slower than not having them.
3) Complex.Drat now I have to understand all that code between me and the hardware and spent time fixing bugs in there. It's hard enough stamping out the bugs in my own code.
4) Non portable. (Yes that's right, choose an abstraction and then you are stuck to those few platforms that support it or you have to recreate the abstraction for unsupported platforms.)
Oops... tongue got stuck again:)
It may be best to start another thread if it has nothing to do with a registry.
Conclusion: A registry standard is not needed.
Donning flame-retardant suit now.
I'm not sure we really need a one-size fits all convention for passing data to drivers, given that the drivers have different parameters and ways of operating anyway. But reducing it to just a few common cases would be handy.
The Catalina "request word/result word" model would work well for simple drivers.
An obvious alternative for more complicated drivers would be to use queues (as in the serial driver case): an input queue and an output queue. Requests for service go into the input queue; results are placed by the cog in the output queue. The queue size would vary from driver to driver, but the structure would always be the same. The advantage of this is that asynchronous requests are handled very easily, and the model is easily adapted to the Unix "stream of data" model (it looks just like a terminal).
Eric
I was about to make the same suggestion. All comms between COGs go through queues or "channels". As in the good old Transputer or the modern day MCU that shall remain unnamed here. That is "Communicating Sequential Processes", CSP.
With a potential bonus that processes can be moved out to COGs on other Propeller chips with those channel ends being conected by some serial comms link. No changes to the processes at either end required.
Why start another thread? Read the title of this thread again! Any solution we come up with would also be usable by Spin.
A registry is just one part of one proposed solution - one that I proposed as an example to get the ball rolling. Non-registry solutions are certainly possible, although I can't quite see how they might work since no-one has proposed one yet.
But I don't really understand why people are obsessing about the "registry" idea itself - that's kind of missing the main point.
However (for those who do want to obsess about these things) Catalina's "registry" is trivially simple - it is nothing more than 8 longs of Hub RAM that store the type of plugin running in each cog, plus (optionally) a pointer to a comms block allocated for that plugin. This allows a high-level language to find the plugin it needs and then communicate with it - without having to have anything hard-coded, or compiled or linked in (other than the address of the registry). This represents the simplest solution I could think of that works. But there may be others (and they may be better).
That idea of coming up with the simplest possible solution that will work is important, because I would like us to have a language-independent solution (again, refer to the title of this thread!). Different languages use different mechanisms for compilation and linkage (and may use different PASM assemblers, or even offer no PASM capability at all) so I don't think a language-independent option should depend on capabilities provided only by some languages.
With a registry solution you don't even need to be able to compile new plugins to use them - as long as you have a binary blob, you can load it into a cog and it will work as advertised. How often have people complained that you cannot distribute binary programs with Spin? Well, if we can come up with a solution here, at least you could distribute binary versions of plugins! But then if you can do that, you can also do the opposite - i.e. distribute your application as a binary - sans plugins - and allow people to add whatever platform-dependent plugins they need to make that application work on their Propeller.
If someone can come up with a better solution than the one proposed, then let's hear about it!
Ross.
No, it's not. If I've not made it perfectly clear yet, I will say so now - I will change Catalina if we can agree on a standard solution for communicating between different high level language programs and cog programs. If the standard solution doesn't include a registry, then so what? - if I still need one, I can always add one on top of whatever we come up with. So could anyone else.
But so far, concrete proposals have been made (as far as I can recall) only by Dave and Eric. No-one else (apologies if I have missed a post) has proposed anything that could be implemented to achieve the intended objective. Perhaps because people didn't understand the intended objective of this thread originally. I'll accept responsibility for that.
But I think we've got that sorted now.
Ross.
Hi Eric,
Absolutely! I agree we may end up with (or want the design to be able to encompass) more than one "standard" mechanism. C "streams" are very byte-oriented, and this works well for most simple drivers - but other models would work better in other cases - e.g. inter-process communications may want to have a standard mechanism for defining and using blocks of shared memory.
Ross.
I agree - I'd love to be able to communicate to a process running on a remote Prop as easily as I can communicate with one running on another cog on the same Prop!
I have no problem with the CSP model other than that it has proven to be virtually unusable in practice. As a theoretical model it actually works very well.
Ross.
Sounds like you are volunteering?
Ross.
I'm not opposed to a registry of some type. If I understand the concept of the registry as it would be used here, each cog would have a slot or long in a table which would point to the data that other objects/cogs would be accessing.
I thought the purpose of this thread was to discuss a standard for obj/cogs to communicate in a language independent way not about the implementation of a registry. If that's not the case then I apologize and will ignore this thread from here on out.
oops....did I do that? not till after Christmas
HO HO HO
Hi 4x5n,
You are quite right - this thread is about a standard for cog-to-cog communications.
This thread is not about a registry, except as one possible mechanism to help achieving that.
Ross.
I was thinking you want a standard way to communicate between the API part and the assembly-cog part of a driver object. But now I think you want cog drivers, which are totally independent of an API and do all the functions in the cog itself. And there is a standardized way to connect with the services in the cog from any language (even PASM).
If this is true, then almost all of the Spin drivers must written new, because a big part of the functionality of the objects are in the Spin part, and the Assembly cog only handles some low level stuff. It would make no sense to connect to such a cog from another language without rewriting the Spin part in the new language.
There are some drivers for Sphinx, that are written so, that the whole functions are in the cog, you only communicate through the "Rendevous-area" with it. The Sphinx-TV driver for example gets only the characters to show and does all what is necessary in the cog, even the screen buffer is in the cog.
In contrast the TV_Text object in Spin starts only a TV driver cog, and does not communicate with that cog any further. All the functions to draw characters are in the Spin part, which just puts the characters at the right position into the screenbuffer. The TV-cog does nothing than output the screen buffer as a TV signal.
So am I correct with that? And should the thread title then not be something like:
"Let us make a new class of cog drivers, which have all the functionality in the cog and commuinicate only through standardized services, so they are independant of a high level programming language."
Andy
It may be a case of shell shock from dealing with the windows registry.
Reminds me of the 8080/Z80 interrupt vectors or the jump tables/address pointers I used for simple monitor programs and more complex menu systems. I think something simple and light weight like this is what we need. No reason CSP could not be used as part of this for cog-cog communications within and between props as well.
@Ross,
Ok, I agree. Sorted. Registry as defined by your document would be used only if necessary.
So, reset. From the top down:
What are the functional requirements for a standard for communicating with cogs from any language?
I would like to see your concise point by point definition. People have talked about all kinds of stuff in this thread.
Please compose a short point by point list of statements that define the requirements that can be objectively considered.
If you've done this already, please post a pointer. I do not consider your .pdf a list of requirements.
@Ariba, your post echos many of my own thoughts.