At 496 longs, cog RAM is a very dear commodity. I can't point to a particular example, but it's not hard to imagine a PASM cog so packed to the rafters with essential code, that the only way to initialize its parameters is to poke them into its hub space before a cognew. This is one example that illustrates the compiler writer's responsibility to make life easier for programmers and enhance their productivity, rather than vice-versa. I can well understand the temptation to establish programming standards that make a compiler writer's job easier -- especially if one is a compiler writer. But, in the end, what programmers really want is accommodation for the practices they already use to advantage. In the case I cited, what would be wrong with a C program invoking the Spin byte-code interpreter to handle a Spin start method? Assuming the start method is the first public method in the object, I believe this could be done without needing a symbol table or any relocation fix-ups. (I might be wrong about that, since I haven't paid much attention to Spin's object code structure lately.)
I guess what I'm getting at is that the truly hard work should be done in the compiler so that programmers are free to use whatever techniques are suitable for the job at hand. And that includes accommodating boundary cases that entail programming practices considered less than ideal -- that is, until you really need them.
-Phil
I agree! As I've mentioned more then once in this thread if done right most of us won't need to know the standards even exist. I'm almost finished with my first "largish" (not sure if that's even a word! :-) ) program for the propeller and so far all of the code is mine. The only standards I'm using are my own coding standards. Since the standards being discussed are unlikely to make into the spin or pasm definition or even worse hardware having standards won't have any effect on the software I've written.
I still remember the early days of the PC and there was a lot of fuss over compilers, libraries, etc being less efficient that "pure" assembly. The problem of course was that if you wanted to build on the libraries or work of others you needed standards. That's still the case with network traffic. Would it be possible to write code in assembly that would use less hardware and be faster then tcp/ip or udp/ip if you want to connect to individual machines together? Of course. It would also be possible to write code that's faster then nfs to transfer a given file from one machine to another machine? Again the answer is yes. There are times however that you need to solve for the general case and not a single case. Almost by definition a compiler has to solve for the general case. That requires some type of standard.
I see the standards being discussed here as being more for the compilers to follow and to a lesser extent objects submitted to the obex. I've already suggested that it's time to start a discussion on what the standards should be. I also think that it's possible to have multiple standards that are more or less compatible. Something to keep in mind is that as far as I know no one has suggested that these standards be enforced in hardware or the "spin tool". Meaning that if you want to write all of your own code you could do so anyway you want.
I agree a lot of the discussion is aimed at the compilers, however loading, initializing, and communicating with the cog is a layer 0 & 1 function that is very much tied to the propeller architecture. This is the foundation for everything above that so it needs to be done well. I don't think these standards should be enforced in hardware, the spin tool, or by any other means. Using the standards should be voluntary, and if they simplify using cog objects in writing software I'm sure they will be used.
I was reading the plugin pdf Ross posted earlier and it looks like a very good approach to accomplishing the lowest levels of the standard. Then I went and looked at the modified version of Tim Moores quad serial object (I use it quite often) to see how that could be made to work as a plugin. That definitely does not look like a simple task, but I do believe it is possible.
In the summary I posted, "injecting" values into the COG was frowned upon. Later, here in this discussion, it's suggested as a good idea, so long as it's done at the low addresses.
Thoughts on that? I personally do not favor modifying the COG image prior to running it. Can I hear some advocacy on this technique? I want to add it to the best practices, but want to make sure it actually is a best practice. I know, that's starting a pie fight, but maybe just a small one.
Then I went and looked at the modified version of Tim Moores quad serial object (I use it quite often) to see how that could be made to work as a plugin. That definitely does not look like a simple task, but I do believe it is possible.
Hi kwinn,
Turning the quad serial object into a Catalina plugin is on my "to do" list. There are several ways to go about it - by far the simplest way is to use another cog as an interface. I have done this in other cases, but only when I could also add enough "value" to the original object to justify the overhead (i.e. the loss of a cog). For instance, Catalina's HMI plugins add a one cog overhead to the underlying screen/keyboard/mouse driver - but the benefit is that I can add functionality to these drivers in the "overhead" cog that I would need to implement in C anyway - such as providing a uniform interface even when using completely different drivers, providing some necessary screen management functions, providing a visible (and flashing) cursor on any type of screen, interpreting control characters read from the keyboard etc, etc. If I didn't do this in the extra "overhead" cog, I would have to implement the same functionality in Hub RAM - and in some cases (e.g. the flashing cursor) I am not even sure I could do so. So in this case the tradeoff makes sense.
In the case of the quad serial object, I don't think I could justify the overhead of another cog per serial cog, so it becomes a little harder. But not impossible - so maybe what we should do is use this particular object as a practical example of whatever "standard" we end up with. This object is quite "pathological" in several ways (e.g. the way it is initialized), but it is also a very useful object that lots of people use - so it would make a really good "test case".
In the summary I posted, "injecting" values into the COG was frowned upon. Later, here in this discussion, it's suggested as a good idea, so long as it's done at the low addresses.
Thoughts on that? I personally do not favor modifying the COG image prior to running it. Can I hear some advocacy on this technique? I want to add it to the best practices, but want to make sure it actually is a best practice. I know, that's starting a pie fight, but maybe just a small one.
Hi potatohead,
I don't know whether it is "best practice" but injecting stuff into the cog image is a useful technique, and is used a lot in Spin (where you have direct access to all the PASM symbol information) - but it is a bit more difficult to implement from a high level language if you want to re-use the PASM code "as is" since you won't know where to inject stuff (at least you won't if you use any of the current PASM compilers, since they do not generate any symbol information that a higher level language could use). All the low memory mechanism does is standardize things so that from the high-level language you can inject stuff at a known (fixed) offset - that way you don't need the actual symbol table. This fixed offset doesn't need to change even if you have to change the PASM code, so it also gives you a layer of insulation against future changes.
This is one of those "layer 0" issues that we could argue about all day, but which doesn't really affect you once you move up to "layer 1" where the actual cog-to-cog communications takes place.
In my view, any "layer 0" technique that works and doesn't interfere with the higher layers is ok - but I can see the sense in the "low memory" idea.
Gee Ross, I take this long hard look at the quad serial object, do a lengthy writeup on it, and you go and spoil it by answering before I can post it ; - ) Oh well.
I do think it is possible to modify the quad serial object so the cog can load it's own parameters and use the two (or maybe 4) longs from your “registry” to communicate. Looks like the changes would be minor as well. Mainly rearranging the longs in the dat section. I will double check things in the morning. Getting a bit tired now.
Gee Ross, I take this long hard look at the quad serial object, do a lengthy writeup on it, and you go and spoil it by answering before I can post it ; - ) Oh well.
I do think it is possible to modify the quad serial object so the cog can load it's own parameters and use the two (or maybe 4) longs from your registry to communicate. Looks like the changes would be minor as well. Mainly rearranging the longs in the dat section. I will double check things in the morning. Getting a bit tired now.
Hi Kwinn,
Don't worry - your work won't be wasted - if you want to work on this object, go right ahead - I'll choose another OBEX object to use as an example.
Had a bit of a disrupted day today, so my work on summarizing this thread is not ready to post yet. However, in the meantime, I thought I would have a go at summarizing the purpose and principles for the new thread.
In them, I've tried to address as many of the comments and criticisms raised in this thread as I can so that they do not have to be argued all over again in the next thread. That makes them a bit longer and more complex than I'd like - perhaps longer than the final "standard" might be - but I think it may prove worthwhile in the end.
Please feel free to comment or offer any feedback.
Purpose
The purpose of the new thread will be to standardize (and document by example) a set of techniques concerning how cog programs can communicate with other cog programs in order to simplify the process of re-using them from different languages - including Spin and PASM, but mainly intended for non-Spin languages like C, C++, Basic, Forth, Java etc.
Spin has some specific language features that have encouraged techniques that are easy to use in Spin (such as modifying arbitrary locations in the PASM cog program before loading it into a cog) but which can be difficult to emulate in other languages. This is not a criticism of Spin, but using these techniques can make it harder to use the resulting cog programs from other languages - and (increasingly) the Propleller is now being programmed in languages other than Spin.
Also, some mechanisms used to perform cog to cog communication rely on knowing or sharing symbolic information - e.g. the name of a shared variable in Hub RAM - that is not easily accessible outside Spin.
Even when programming in Spin (or directly in PASM) this standardization may assist in reducing the complexity of programs, and increasing both their comprehensibility and their maintainability.
If a suitable standard can be agreed, it will be suggested that new "gold standard" OBEX entries adopt it (where feasible) so as to make the cog programs (which are usually embedded within thin Spin wrappers) also suitable for re-use in other langauge contexts. However, there is no intent (and probably no benefit) in going back and modifying all the existing OBEX entries.
Principles
The standard will be a permissive techical standard, not an enforcable regulatory standard. There may be other different standards adopted by particular users, or for particular purposes, and many programs may choose to adopt no standard at all. But from a language that adopts the standard, it will be much easier to "plug and play" using cog programs that also adopt the standard, even if those cog programs were developed within a different language context.
.
The standard will not assume there is a central "intermediary" responsible for doing any of the work. While it may fall to a particular cog to do some of the initial setup (e.g. the cog running the initial Spin interpreter), once that is completed any cog program should be able to talk to any other cog program without the need of an intermediate "controller" or "kernel".
.
The standard is intended to be language independent, and should not assume or impose a particular programming paradigm. While the most common paradigm is expected to be "one client running in a single cog, using services provided by one or more programs running in other cogs" (e.g. a main C or Spin program using some drivers) the standard should be flexible enough to support paradigms such as a "multiple clients sharing services provided by multiple cogs", or "multiple cooperating processes".
.
The standard will aim for "minimum overhead", both in code space and performance. It may offer multiple mechanisms to achieve the same functionality, allowing the best "fit" to be selected on the basis of code space available and performance required. The overhead will be as close to zero as possible, consistent with achieving the necessary functionality.
.
The standard will be service-based. A cog programs does not need to offer any services - but if it does, the basic services provided will be determined by the nominated "class" of the cog program. The class is just an integer value from a maintained list of "public" classes - a range of "private" class values will be provided for custom cog programs to use. However, note that just because the standard is service-based, this does not preclude cog programs using other mechanisms for communication. For instance, a cog program that wishes to use a block of shared memory as a buffer or a queue may do so, using a block of memory whose address is passed as part of the initialization process or using an initialization service. Other cog programs will determine the communication mechanism to use based on the class.
.
The standard will be "layered" with as much independence between the layers as possible. The services supplied by lower layers and used by upper layers will define the interface between the layers. All layers can also provide services to cog programs, and if the services provided by a layer are not required by any cog programs or by another layer, then the whole layer can simply be omitted. Layers can be replaced with functional equivalents, and additional services can be implemented in any layer. As few restrictions as possible will be placed on the techniques used in each layer, and these will generally only be those necessary to ensure each layer works correctly with other layers. Conventions and "recommended practice" may be documented for each layer, but they will not be mandatory.
.
The standard will be primarily a binary standard. It will be able to be implemented entirely in PASM so that cog programs can be compiled as stand-alone objects, with the resulting binary cog programs being able to be re-used from any high-level language that adopts the standard. Note that some languages provide their own assembler, which may not be PASM compatible - but this is irrelevant provided the binary results are the same as if the cog program was written and assembled in standard PASM.
.
The standard is primarily intended to facilitate the re-use of cog program binaries on the same platform. Supporting the re-use of binary objects on different platforms is not a goal of the standard, but it can be accomodated by adopting additional conventions - e.g. that all platform-dependent information (pins etc) be written to a specific location in the cog image prior to initialization (based on the classs of the cog program) or passed at run-time via an initialization service.
.
The standard will minimize as far as possible the information that needs to be provided about cog programs to the high-level languages that want to use them (such as C or C++). This information will be limited to the class of the cog program (which defines the basic interface) and a fixed set of configuration constants that need to be made visible to any high-level language. This information may be represented in different formats for different languages (e.g. in C it would probably be a header file containing a list of #defines). The standard requires these constants be defined and documented, but does not specify how they will be represented, interpreted or used. An example of such configuration constants could be the number of rows or columns supported by a video driver class (assuming these are constant - if not then the cog program should instead make some other provision for making their value available - e.g. via a service).
.
The standard will recommend that cog programs providing substantially similar functionality belong to the same class, and hence present the same basic interface. However, cog programs may also "extend" the basic interface by the additon of new services. If an instance of the class wants to offer additional services, the cog program must support a service that allows these extensions to be discovered. The technique used to identify the extensions supported is class dependent. For instance, for a video driver class an extension service might return a bitmap that indentifies the additional services or capabilities the driver supports.
.
Cog programs should make no assumptions about the layout of Hub RAM, or the address of any communications blocks they are passed at initialization or run time. Any memory locations not permanently "fixed", and not specifically passed to the cog program, should be derived from the single par parameter passed to the cog at initialization time.
.
Any fixed locations in Hub RAM should be allocated at the highest available Hub RAM location. Any areas of Hub RAM that need to be dynamically allocated should be allocated below these fixed locations.
.
Addresses will always be represented using a minimum of 18 bits, to allow for up to 256kb of Hub RAM expected to be available on the next generation of Propeller chip.
Don't know if this fits, but we talked about something similar to this in propforth, and since there wasn't any standard to follow, Sal made one internal to forth. Maybe some part of this might be useful, or at least a point of reference whether you use it or not.
The interface to each forth cog is always via a location defined in the kernel. Each forth application talks through this location. The cog that handles the user interface listens to the keyboard, sends the data to the cog's input area, recieves data from thecog's output area, and displays it on the output device (screen).
To talk to a differenet cog, the interface just poiints to the new cog's IO area. It does not matter if the cog is on the same physical chip or on a different chip. The synchronous serial comms use one or two pins to send a continuous stream of 96 bit packets at clock speed. The apllication just drops the data for I/O into the queue, and the comms sends and recieves.
Sal said this could be used as the lowest level support for interpropcommunications, and can just as easily be used as an inter-application scheme.
It is the minimum to get the job done, and does not include error reporting etc. Each synchronous channel has an error count that can be queried, but we have not encountered errors yet (aside from pull the wire etc during testing). The interprop scheme needs no eror checking reporting etc, as you would notice these during use, and we have not. So this could be built upon, but its all very low level and minimal.
If anyone is interested in seeing this in action, it is in the multiprop comms section of the prop forth download on google code. The new (final) version of propforth with assembler optimization will be out in the next couple weeks, (I just got the release candidate today). This might be something to take a look at, as a reference on one way to do a part the cog comms standard effort.
The interface to each forth cog is always via a location defined in the kernel. Each forth application talks through this location. The cog that handles the user interface listens to the keyboard, sends the data to the cog's input area, recieves data from thecog's output area, and displays it on the output device (screen).
To talk to a differenet cog, the interface just poiints to the new cog's IO area. It does not matter if the cog is on the same physical chip or on a different chip. The synchronous serial comms use one or two pins to send a continuous stream of 96 bit packets at clock speed. The apllication just drops the data for I/O into the queue, and the comms sends and recieves.
Hi prof_braino,
Certainly does sound relevant. I particularly like the simple concept of the multi-prop stuff. Is there a document you can point me to? Ron Sutcliffe posted some information about PropForth - I presume that's not related?
Don't worry - your work won't be wasted - if you want to work on this object, go right ahead - I'll choose another OBEX object to use as an example.
Ross.
No, no, please go ahead and use this object. If it can be made compliant with the standard then pretty much any other object could be made compliant.
What got my attention with the pasm code was that the initialization is divided into 2 distinct types of data.
The 8 data items for the AddPort function could be loaded by the cog using a small loop. All that would need to be passed to the cog is the start address of the data items. Of course the order of these parameters in the cog would have to be rearranged to match as well.
The other 6 items that need to be initialized for each port are the pointers that control the tx and rx data buffers. If an extra 6 longs were allocated when the tx/rx buffers were allocated these pointers could be stored in the first 6 longs of the buffer. The start address of the buffer would be passed to the cog and the cog would add that to the offsets already stored there. As before, the pointers in the cog would have to be arranged in matching order.
Certainly does sound relevant. I particularly like the simple concept of the multi-prop stuff. Is there a document you can point me to? Ron Sutcliffe posted some information about PropForth - I presume that's not related?
I'm still reading with interest, Ross. Only wish I were a good enough coder to be able to help. I'd love to be able to more easily use OBEX objects in PropBASIC.
I'm still reading with interest, Ross. Only wish I were a good enough coder to be able to help. I'd love to be able to more easily use OBEX objects in PropBASIC.
Hi simonl,
Sometimes things get lost in the noise. I'm not very literate in either PropForth or PropBASIC, but I will try and accommodate what I can understand of their needs.
What we really need is for people literate in these languages to participate in the process, and review the proposals as they are generated, or suggest changes they may need. Otherwise they will inevitably end up with a solution focused primarily on the "C-like" languages - Spin, C, C++, Java etc
Ron's material might be related, I've been concentrating on the next release (and unfortunately been ignoring the current release, but Ron and Caskas are more knowledgeable than me anyway).
This is Sal's tutorial and functional demonstration of the implementation. It shows the location of the source code, and the instructions for using it. To run the examples, you need to load propforth and run the set up steps. The instructions are not as clear as they will be so it can take a couple tries to get it to run. (The code is fine, the instructions are weak).
I don't know much more about the internals aside from general description I've given, but if you have deeper questions I can relay them to Sal.
EDIT - in response to simon's question, I started a page on MCS in the propforth wiki
so hopefully we can get the information without you having to become and expert in propforth. I will log the details there so we don't sidetrack your discussion here. The page is just a stub for now but I will fill it as questions get asked and answered.
Ron's material might be related, I've been concentrating on the next release (and unfortunately been ignoring the current release, but Ron and Caskas are more knowledgeable than me anyway).
This is Sal's tutorial and functional demonstration of the implementation. It shows the location of the source code, and the instructions for using it. To run the examples, you need to load propforth and run the set up steps. The instructions are not as clear as they will be so it can take a couple tries to get it to run. (The code is fine, the instructions are weak).
I don't know much more about the internals aside from general description I've given, but if you have deeper questions I can relay them to Sal.
EDIT - in response to simon's question, I started a page on MCS in the propforth wiki
so hopefully we can get the information without you having to become and expert in propforth. I will log the details there so we don't sidetrack your discussion here. The page is just a stub for now but I will fill it as questions get asked and answered.
Thanks, prof_braino - I'll check it out and see if I can come up with some intelligent questions.
Addresses will always be represented using a minimum of 18 bits, to allow for up to 256kb of Hub RAM expected to be available on the next generation of Propeller chip.
This smacks of the (mythical?) Bill Gates syndrome "640K is all anyone will ever need".
What about that Prop III with megabytes of RAM on chip. What about when Chip implements an external HUB RAM interface accessing gigabytes. What about that 64 bit Prop? ...
Personally, I'd allow 24 bits for an address. It's greater than 18 bits and represents an even number of bytes. Packing and unpacking a byte value and a 3 byte value to make a long is pretty easy.
I'm not sure what is in mind with that 18 bits. I can imagine commands/responses and parameters/results being packed into a 32 bit words along with some buffer addresses or other pointers. And then oops we don't have enough bits in the future.
Why not use the same method the COG uses, addresses are to units of LONG, not BYTE. Since LONG is the minimum addressable unit, this saves at least 2 bits.
Well, 18 bits gets you 1MB at least. I'm sure there will be a revision in the future.
This may have been brought up already, but:
IIRC the Prop 1 can fetch 2 LONGs per hub window.
The Prop 2 can fetch 4 LONGs per hub window.
Design the parameter data around 2 and 4 LONGs, that way the functionality expands with the architecture.
The other solution, which is a bit more involved, is to have a minimum unit greater than 1 LONG, so you would
address in 2 LONG or 4 LONG chunks. Though this requires some trickery to get the right alignments in code.
I'm ok to standardize on 24 bits for addresses in Hub RAM - that's what I tend to use anyway. I was really just responding to those who thought 16 bits would be enough - we already know this is not the case.
Sorry - been a bit busy the last couple of days. I hope to have new stuff to post later this week.
One question for all - I think I will end up proposing we "standardize" on using a fixed block of memory in high Hub RAM (e.g. the last "n" longs). Does anyone know of any reason why this would NOT work?
[QUOTE=RossH;1058984
One question for all - I think I will end up proposing we "standardize" on using a fixed block of memory in high Hub RAM (e.g. the last "n" longs). Does anyone know of any reason why this would NOT work?
[/QUOTE]
One awkwardness is that as the hub RAM size changes the address of the fixed block of memory is going to change, too. The size of the fixed block may need changing too, if there are more cogs (or propellers) in the system, and/or if more services than cogs are available.
An alternative might be to put a pointer to the block somewhere in low memory, similar to the clkfreq variable.
One awkwardness is that as the hub RAM size changes the address of the fixed block of memory is going to change, too. The size of the fixed block may need changing too, if there are more cogs (or propellers) in the system, and/or if more services than cogs are available.
An alternative might be to put a pointer to the block somewhere in low memory, similar to the clkfreq variable.
Eric
Hi Eric,
I don't think the change in memory size between Props is going to be a problem - I don't think code is not going to be binary compatible between them anyway (does anyone know?).
The main problem with low memory is that I think it conflicts with the way Spin compilers work. I'd have to do some testing, but I don't think you can count on reserving low memory locations from Spin - and Spin is always what's started first on the Prop 1. Perhaps someone who knows more Spin than I do (which wouldn't be difficult!) can comment?
Ross
I appologise for not dealing with the coms issue that you raised. I am also aware that you wish to move the debate along, so treat this post as notes for your File!
I can't answer you query because, as a result of some discussions between Sal and Kuroneko, Sal has made some changes which have resulted in a significant increased in the transaction rate. I am not sure how this impacts on the 5.0 kernel. This discussions in part can be found in issues on the Wilki site.
Prof has pointed you to coms in the PF4.6 supplementary folder. Whilst the Proforth documentation is fine for forthers, it may seem a bit alien to everybody else.
In summary the main points from a my perspective are:-
1.Par points to an area in hub allocated by the kernel for each (128bytes)cog.
Par cannot be changed back to zero.
2.Hub ram is fully allocated. There is no free space in Hub.
Forth is a standalone interactive system. Once configured (and there are many options) it does not require the support of other tools and services. I can't remember the last time I used Spin.
Users can move from cog to cog at will (3 >con) make cog3 the current cog
(except for cog 7, which is normally I/O , but can be reconfigured by boot.f)
Users can stop and start any services from any cog, because the dictionary is common to all cogs.
All cogs have equal access to all resources, including locks,
Sdsys is an integral part on the propforth kernel, it uses a Propforth file system, it's primitive but fast. The Fat file system is considered to be too slow. It also limits the ability to use the sdcard for spi ram in conjunction with a cog cache. There are 512 bytes available above the kernel. You could create a file spi_ram of fixed file length of n blocks and use it for spi-ram. (maybe not a good idea)
Propforth boot.f can be configured to run scripts etc. A script can close itself down after rewinding the dictionary. New words/definitions which are introduced by the script and added to the dictionary will be dropped when the script shuts down. Thats assuming you build *forget* into the end of the script.
Most of the above does not impact on what you are trying to achieve, but items 1 and 2 do not fit well with Forth.
I don't think the change in memory size between Props is going to be a problem - I don't think code is not going to be binary compatible between them anyway (does anyone know?).
The main problem with low memory is that I think it conflicts with the way Spin compilers work. I'd have to do some testing, but I don't think you can count on reserving low memory locations from Spin - and Spin is always what's started first on the Prop 1. Perhaps someone who knows more Spin than I do (which wouldn't be difficult!) can comment?
Many of Prop 1's instructions will use the same binary opcodes on Prop 2, but there are several Prop 1 instructions that are different on Prop 2, so a Prop 1 binary will not run on a Prop 2.
The first 16 byte locations in hub RAM are used to boot up a Spin program. Spin programs continue to use locations $0000 to $0004 for storing CLKFREQ and CLKMODE after startup, but locations $0005 to $000F are not use by the Spin program, and could be used for other purposes.
I don't think the change in memory size between Props is going to be a problem - I don't think code is not going to be binary compatible between them anyway (does anyone know?).
I was thinking more of source code compatibility. I would imagine that there will be some kind of PASM compatibility at least at the source level (and possibly the binary level).
Just in case anyone is interested, the conclusion to this thread is here. I thought I'd better post it here in case anyone finds this thread in future and wonders what happened to it.
Comments
I agree! As I've mentioned more then once in this thread if done right most of us won't need to know the standards even exist. I'm almost finished with my first "largish" (not sure if that's even a word! :-) ) program for the propeller and so far all of the code is mine. The only standards I'm using are my own coding standards. Since the standards being discussed are unlikely to make into the spin or pasm definition or even worse hardware having standards won't have any effect on the software I've written.
I still remember the early days of the PC and there was a lot of fuss over compilers, libraries, etc being less efficient that "pure" assembly. The problem of course was that if you wanted to build on the libraries or work of others you needed standards. That's still the case with network traffic. Would it be possible to write code in assembly that would use less hardware and be faster then tcp/ip or udp/ip if you want to connect to individual machines together? Of course. It would also be possible to write code that's faster then nfs to transfer a given file from one machine to another machine? Again the answer is yes. There are times however that you need to solve for the general case and not a single case. Almost by definition a compiler has to solve for the general case. That requires some type of standard.
I agree a lot of the discussion is aimed at the compilers, however loading, initializing, and communicating with the cog is a layer 0 & 1 function that is very much tied to the propeller architecture. This is the foundation for everything above that so it needs to be done well. I don't think these standards should be enforced in hardware, the spin tool, or by any other means. Using the standards should be voluntary, and if they simplify using cog objects in writing software I'm sure they will be used.
I was reading the plugin pdf Ross posted earlier and it looks like a very good approach to accomplishing the lowest levels of the standard. Then I went and looked at the modified version of Tim Moores quad serial object (I use it quite often) to see how that could be made to work as a plugin. That definitely does not look like a simple task, but I do believe it is possible.
In the summary I posted, "injecting" values into the COG was frowned upon. Later, here in this discussion, it's suggested as a good idea, so long as it's done at the low addresses.
Thoughts on that? I personally do not favor modifying the COG image prior to running it. Can I hear some advocacy on this technique? I want to add it to the best practices, but want to make sure it actually is a best practice. I know, that's starting a pie fight, but maybe just a small one.
Hi kwinn,
Turning the quad serial object into a Catalina plugin is on my "to do" list. There are several ways to go about it - by far the simplest way is to use another cog as an interface. I have done this in other cases, but only when I could also add enough "value" to the original object to justify the overhead (i.e. the loss of a cog). For instance, Catalina's HMI plugins add a one cog overhead to the underlying screen/keyboard/mouse driver - but the benefit is that I can add functionality to these drivers in the "overhead" cog that I would need to implement in C anyway - such as providing a uniform interface even when using completely different drivers, providing some necessary screen management functions, providing a visible (and flashing) cursor on any type of screen, interpreting control characters read from the keyboard etc, etc. If I didn't do this in the extra "overhead" cog, I would have to implement the same functionality in Hub RAM - and in some cases (e.g. the flashing cursor) I am not even sure I could do so. So in this case the tradeoff makes sense.
In the case of the quad serial object, I don't think I could justify the overhead of another cog per serial cog, so it becomes a little harder. But not impossible - so maybe what we should do is use this particular object as a practical example of whatever "standard" we end up with. This object is quite "pathological" in several ways (e.g. the way it is initialized), but it is also a very useful object that lots of people use - so it would make a really good "test case".
Ross.
Hi potatohead,
I don't know whether it is "best practice" but injecting stuff into the cog image is a useful technique, and is used a lot in Spin (where you have direct access to all the PASM symbol information) - but it is a bit more difficult to implement from a high level language if you want to re-use the PASM code "as is" since you won't know where to inject stuff (at least you won't if you use any of the current PASM compilers, since they do not generate any symbol information that a higher level language could use). All the low memory mechanism does is standardize things so that from the high-level language you can inject stuff at a known (fixed) offset - that way you don't need the actual symbol table. This fixed offset doesn't need to change even if you have to change the PASM code, so it also gives you a layer of insulation against future changes.
This is one of those "layer 0" issues that we could argue about all day, but which doesn't really affect you once you move up to "layer 1" where the actual cog-to-cog communications takes place.
In my view, any "layer 0" technique that works and doesn't interfere with the higher layers is ok - but I can see the sense in the "low memory" idea.
Ross.
I do think it is possible to modify the quad serial object so the cog can load it's own parameters and use the two (or maybe 4) longs from your “registry” to communicate. Looks like the changes would be minor as well. Mainly rearranging the longs in the dat section. I will double check things in the morning. Getting a bit tired now.
Hi Kwinn,
Don't worry - your work won't be wasted - if you want to work on this object, go right ahead - I'll choose another OBEX object to use as an example.
Ross.
Had a bit of a disrupted day today, so my work on summarizing this thread is not ready to post yet. However, in the meantime, I thought I would have a go at summarizing the purpose and principles for the new thread.
In them, I've tried to address as many of the comments and criticisms raised in this thread as I can so that they do not have to be argued all over again in the next thread. That makes them a bit longer and more complex than I'd like - perhaps longer than the final "standard" might be
Please feel free to comment or offer any feedback.
Purpose
Also, some mechanisms used to perform cog to cog communication rely on knowing or sharing symbolic information - e.g. the name of a shared variable in Hub RAM - that is not easily accessible outside Spin.
Principles
- The standard will be a permissive techical standard, not an enforcable regulatory standard. There may be other different standards adopted by particular users, or for particular purposes, and many programs may choose to adopt no standard at all. But from a language that adopts the standard, it will be much easier to "plug and play" using cog programs that also adopt the standard, even if those cog programs were developed within a different language context.
- The standard will not assume there is a central "intermediary" responsible for doing any of the work. While it may fall to a particular cog to do some of the initial setup (e.g. the cog running the initial Spin interpreter), once that is completed any cog program should be able to talk to any other cog program without the need of an intermediate "controller" or "kernel".
- The standard is intended to be language independent, and should not assume or impose a particular programming paradigm. While the most common paradigm is expected to be "one client running in a single cog, using services provided by one or more programs running in other cogs" (e.g. a main C or Spin program using some drivers) the standard should be flexible enough to support paradigms such as a "multiple clients sharing services provided by multiple cogs", or "multiple cooperating processes".
- The standard will aim for "minimum overhead", both in code space and performance. It may offer multiple mechanisms to achieve the same functionality, allowing the best "fit" to be selected on the basis of code space available and performance required. The overhead will be as close to zero as possible, consistent with achieving the necessary functionality.
- The standard will be service-based. A cog programs does not need to offer any services - but if it does, the basic services provided will be determined by the nominated "class" of the cog program. The class is just an integer value from a maintained list of "public" classes - a range of "private" class values will be provided for custom cog programs to use. However, note that just because the standard is service-based, this does not preclude cog programs using other mechanisms for communication. For instance, a cog program that wishes to use a block of shared memory as a buffer or a queue may do so, using a block of memory whose address is passed as part of the initialization process or using an initialization service. Other cog programs will determine the communication mechanism to use based on the class.
- The standard will be "layered" with as much independence between the layers as possible. The services supplied by lower layers and used by upper layers will define the interface between the layers. All layers can also provide services to cog programs, and if the services provided by a layer are not required by any cog programs or by another layer, then the whole layer can simply be omitted. Layers can be replaced with functional equivalents, and additional services can be implemented in any layer. As few restrictions as possible will be placed on the techniques used in each layer, and these will generally only be those necessary to ensure each layer works correctly with other layers. Conventions and "recommended practice" may be documented for each layer, but they will not be mandatory.
- The standard will be primarily a binary standard. It will be able to be implemented entirely in PASM so that cog programs can be compiled as stand-alone objects, with the resulting binary cog programs being able to be re-used from any high-level language that adopts the standard. Note that some languages provide their own assembler, which may not be PASM compatible - but this is irrelevant provided the binary results are the same as if the cog program was written and assembled in standard PASM.
- The standard is primarily intended to facilitate the re-use of cog program binaries on the same platform. Supporting the re-use of binary objects on different platforms is not a goal of the standard, but it can be accomodated by adopting additional conventions - e.g. that all platform-dependent information (pins etc) be written to a specific location in the cog image prior to initialization (based on the classs of the cog program) or passed at run-time via an initialization service.
- The standard will minimize as far as possible the information that needs to be provided about cog programs to the high-level languages that want to use them (such as C or C++). This information will be limited to the class of the cog program (which defines the basic interface) and a fixed set of configuration constants that need to be made visible to any high-level language. This information may be represented in different formats for different languages (e.g. in C it would probably be a header file containing a list of #defines). The standard requires these constants be defined and documented, but does not specify how they will be represented, interpreted or used. An example of such configuration constants could be the number of rows or columns supported by a video driver class (assuming these are constant - if not then the cog program should instead make some other provision for making their value available - e.g. via a service).
- The standard will recommend that cog programs providing substantially similar functionality belong to the same class, and hence present the same basic interface. However, cog programs may also "extend" the basic interface by the additon of new services. If an instance of the class wants to offer additional services, the cog program must support a service that allows these extensions to be discovered. The technique used to identify the extensions supported is class dependent. For instance, for a video driver class an extension service might return a bitmap that indentifies the additional services or capabilities the driver supports.
- Cog programs should make no assumptions about the layout of Hub RAM, or the address of any communications blocks they are passed at initialization or run time. Any memory locations not permanently "fixed", and not specifically passed to the cog program, should be derived from the single par parameter passed to the cog at initialization time.
- Any fixed locations in Hub RAM should be allocated at the highest available Hub RAM location. Any areas of Hub RAM that need to be dynamically allocated should be allocated below these fixed locations.
- Addresses will always be represented using a minimum of 18 bits, to allow for up to 256kb of Hub RAM expected to be available on the next generation of Propeller chip.
.Ross..
.
.
.
.
.
.
.
.
.
.
.
The interface to each forth cog is always via a location defined in the kernel. Each forth application talks through this location. The cog that handles the user interface listens to the keyboard, sends the data to the cog's input area, recieves data from thecog's output area, and displays it on the output device (screen).
To talk to a differenet cog, the interface just poiints to the new cog's IO area. It does not matter if the cog is on the same physical chip or on a different chip. The synchronous serial comms use one or two pins to send a continuous stream of 96 bit packets at clock speed. The apllication just drops the data for I/O into the queue, and the comms sends and recieves.
Sal said this could be used as the lowest level support for interpropcommunications, and can just as easily be used as an inter-application scheme.
It is the minimum to get the job done, and does not include error reporting etc. Each synchronous channel has an error count that can be queried, but we have not encountered errors yet (aside from pull the wire etc during testing). The interprop scheme needs no eror checking reporting etc, as you would notice these during use, and we have not. So this could be built upon, but its all very low level and minimal.
If anyone is interested in seeing this in action, it is in the multiprop comms section of the prop forth download on google code. The new (final) version of propforth with assembler optimization will be out in the next couple weeks, (I just got the release candidate today). This might be something to take a look at, as a reference on one way to do a part the cog comms standard effort.
Hi prof_braino,
Certainly does sound relevant. I particularly like the simple concept of the multi-prop stuff. Is there a document you can point me to? Ron Sutcliffe posted some information about PropForth - I presume that's not related?
Ross.
No, no, please go ahead and use this object. If it can be made compliant with the standard then pretty much any other object could be made compliant.
What got my attention with the pasm code was that the initialization is divided into 2 distinct types of data.
The 8 data items for the AddPort function could be loaded by the cog using a small loop. All that would need to be passed to the cog is the start address of the data items. Of course the order of these parameters in the cog would have to be rearranged to match as well.
The other 6 items that need to be initialized for each port are the pointers that control the tx and rx data buffers. If an extra 6 longs were allocated when the tx/rx buffers were allocated these pointers could be stored in the first 6 longs of the buffer. The start address of the buffer would be passed to the cog and the cog would add that to the offsets already stored there. As before, the pointers in the cog would have to be arranged in matching order.
I'm still reading with interest, Ross. Only wish I were a good enough coder to be able to help. I'd love to be able to more easily use OBEX objects in PropBASIC.
Hi simonl,
Sometimes things get lost in the noise. I'm not very literate in either PropForth or PropBASIC, but I will try and accommodate what I can understand of their needs.
What we really need is for people literate in these languages to participate in the process, and review the proposals as they are generated, or suggest changes they may need. Otherwise they will inevitably end up with a solution focused primarily on the "C-like" languages - Spin, C, C++, Java etc
Ross.
But the real information is in the google code archive http://propforth.googlecode.com/files/PropForth4.6-20110725.zip
under docs/test-4.3.1 MCSLoopack.txt
This is Sal's tutorial and functional demonstration of the implementation. It shows the location of the source code, and the instructions for using it. To run the examples, you need to load propforth and run the set up steps. The instructions are not as clear as they will be so it can take a couple tries to get it to run. (The code is fine, the instructions are weak).
I don't know much more about the internals aside from general description I've given, but if you have deeper questions I can relay them to Sal.
EDIT - in response to simon's question, I started a page on MCS in the propforth wiki
http://code.google.com/p/propforth/wiki/MutliChannelSerialInterPropComm
so hopefully we can get the information without you having to become and expert in propforth. I will log the details there so we don't sidetrack your discussion here. The page is just a stub for now but I will fill it as questions get asked and answered.
Thanks, prof_braino - I'll check it out and see if I can come up with some intelligent questions.
Ross.
I have a bit of a problem with principle #13.
This smacks of the (mythical?) Bill Gates syndrome "640K is all anyone will ever need".
What about that Prop III with megabytes of RAM on chip. What about when Chip implements an external HUB RAM interface accessing gigabytes. What about that 64 bit Prop? ...
Personally, I'd allow 24 bits for an address. It's greater than 18 bits and represents an even number of bytes. Packing and unpacking a byte value and a 3 byte value to make a long is pretty easy.
This may have been brought up already, but:
IIRC the Prop 1 can fetch 2 LONGs per hub window.
The Prop 2 can fetch 4 LONGs per hub window.
Design the parameter data around 2 and 4 LONGs, that way the functionality expands with the architecture.
The other solution, which is a bit more involved, is to have a minimum unit greater than 1 LONG, so you would
address in 2 LONG or 4 LONG chunks. Though this requires some trickery to get the right alignments in code.
I'm ok to standardize on 24 bits for addresses in Hub RAM - that's what I tend to use anyway. I was really just responding to those who thought 16 bits would be enough - we already know this is not the case.
Sorry - been a bit busy the last couple of days. I hope to have new stuff to post later this week.
One question for all - I think I will end up proposing we "standardize" on using a fixed block of memory in high Hub RAM (e.g. the last "n" longs). Does anyone know of any reason why this would NOT work?
Ross.
One question for all - I think I will end up proposing we "standardize" on using a fixed block of memory in high Hub RAM (e.g. the last "n" longs). Does anyone know of any reason why this would NOT work?
[/QUOTE]
One awkwardness is that as the hub RAM size changes the address of the fixed block of memory is going to change, too. The size of the fixed block may need changing too, if there are more cogs (or propellers) in the system, and/or if more services than cogs are available.
An alternative might be to put a pointer to the block somewhere in low memory, similar to the clkfreq variable.
Eric
Hi Eric,
I don't think the change in memory size between Props is going to be a problem - I don't think code is not going to be binary compatible between them anyway (does anyone know?).
The main problem with low memory is that I think it conflicts with the way Spin compilers work. I'd have to do some testing, but I don't think you can count on reserving low memory locations from Spin - and Spin is always what's started first on the Prop 1. Perhaps someone who knows more Spin than I do (which wouldn't be difficult!) can comment?
Ross.
Don't under estimate yourself
Ross
I appologise for not dealing with the coms issue that you raised. I am also aware that you wish to move the debate along, so treat this post as notes for your File!
I can't answer you query because, as a result of some discussions between Sal and Kuroneko, Sal has made some changes which have resulted in a significant increased in the transaction rate. I am not sure how this impacts on the 5.0 kernel. This discussions in part can be found in issues on the Wilki site.
Prof has pointed you to coms in the PF4.6 supplementary folder. Whilst the Proforth documentation is fine for forthers, it may seem a bit alien to everybody else.
In summary the main points from a my perspective are:-
1.Par points to an area in hub allocated by the kernel for each (128bytes)cog.
Par cannot be changed back to zero.
2.Hub ram is fully allocated. There is no free space in Hub.
Forth is a standalone interactive system. Once configured (and there are many options) it does not require the support of other tools and services. I can't remember the last time I used Spin.
Users can move from cog to cog at will (3 >con) make cog3 the current cog
(except for cog 7, which is normally I/O , but can be reconfigured by boot.f)
Users can stop and start any services from any cog, because the dictionary is common to all cogs.
All cogs have equal access to all resources, including locks,
Sdsys is an integral part on the propforth kernel, it uses a Propforth file system, it's primitive but fast. The Fat file system is considered to be too slow. It also limits the ability to use the sdcard for spi ram in conjunction with a cog cache. There are 512 bytes available above the kernel. You could create a file spi_ram of fixed file length of n blocks and use it for spi-ram. (maybe not a good idea)
Propforth boot.f can be configured to run scripts etc. A script can close itself down after rewinding the dictionary. New words/definitions which are introduced by the script and added to the dictionary will be dropped when the script shuts down. Thats assuming you build *forget* into the end of the script.
Most of the above does not impact on what you are trying to achieve, but items 1 and 2 do not fit well with Forth.
Ron
The first 16 byte locations in hub RAM are used to boot up a Spin program. Spin programs continue to use locations $0000 to $0004 for storing CLKFREQ and CLKMODE after startup, but locations $0005 to $000F are not use by the Spin program, and could be used for other purposes.
Just in case anyone is interested, the conclusion to this thread is here. I thought I'd better post it here in case anyone finds this thread in future and wonders what happened to it.
Ross.