P2 Reserved hub ram $00E80-$00FFF for system & mailboxes (discussion)
Cluso99
Posts: 18,069
There has been some preliminary discussion on the p2load thread (and previously elsewhere). It is now time to start the discussion in a separate thread.
Let me say I am quite concerned that the naysayers will derail this thread as has been done in the P1 threads many times before. This has prevented consensus, and thus many have reinvented the wheel with variations, making the use of standard drivers impossible between the various implementations. So, the latest proposition below covers the naysayers (I hope) in that their bare metal version can ignore the rest of us. They will have to write their own drivers or modify ours to make them work with what I hope can be our standard drivers using standard interfaces, as agreed.
Current discussions center around whether this is desired or not, and in what format. General consensus seems to be to start user programs at $01000 and reserve $00E80..$00FFF for system parameters and mailboxes.
I proposed using 8*4 longs, one for each cog ($E80-$EFF) followed by CLKFREQ & CLKMODE, lower and upper hub allocation pointers, and then 15*4 longs available.
Bill Henning modified the 15*4 longs to be 15 mailboxes and defined a prospective use for them. I felt and Bill agreed that the cog table fitted nicely in that gap. David Betz advocated that CLKFERQ & CLKMODE start at $E80.. for a bare metal solution.
Here is the latest combined propositions...
[*]$F00 be the "system constants"
[*]$F10 be the "console input/output" mailbox
[*]$F20 be the FLASH mailbox
[*]$F30 be the uSD mailbox
[/LIST]A few standad messages come to mind (Bill):
So, I am torn between the two choices...
Using a cog table and using an I/O table where mailboxes are used for stdin, stdout, filesystem, low level SD access, maybe auxin & auxout, etc.
So, lets get the discussion going. We have time, but not a lot of it.
Let me say I am quite concerned that the naysayers will derail this thread as has been done in the P1 threads many times before. This has prevented consensus, and thus many have reinvented the wheel with variations, making the use of standard drivers impossible between the various implementations. So, the latest proposition below covers the naysayers (I hope) in that their bare metal version can ignore the rest of us. They will have to write their own drivers or modify ours to make them work with what I hope can be our standard drivers using standard interfaces, as agreed.
Current discussions center around whether this is desired or not, and in what format. General consensus seems to be to start user programs at $01000 and reserve $00E80..$00FFF for system parameters and mailboxes.
I proposed using 8*4 longs, one for each cog ($E80-$EFF) followed by CLKFREQ & CLKMODE, lower and upper hub allocation pointers, and then 15*4 longs available.
Bill Henning modified the 15*4 longs to be 15 mailboxes and defined a prospective use for them. I felt and Bill agreed that the cog table fitted nicely in that gap. David Betz advocated that CLKFERQ & CLKMODE start at $E80.. for a bare metal solution.
Here is the latest combined propositions...
- $00E80..$00E8F: System Globals (to accommodate bare metal code)
- $00E80: CLKFREQ
- $00E84:
- bits b8..0: CLKMODE
- bits b31..b9:
- All 0's for bare metal code (no mailboxes available - used for code/etc)
- All 1's for 23 bits, one for each mailbox ($00E90..$00FFF)
- When a mailbox is allocated, the bit is cleared = claimed/used
- $00E88: reserved
- $00E8C: reserved
- $00E90..$00FFF: 23* 4 long mailboxes
- $01000: Start of user code (bare metal can start at $00E90 or $00E88 if required)
- In general, a mailbox format: (Bill)
- long 0 - function/status
- long 1 - from pointer
- long 2 - to pointer
- long 3 - length/size
- Variation for using cyclic buffers: (Cluso99/Bill)
- long 1 - from fifo base pointer (rx fif0)
long 2 - to fifo base pointer (tx fifo)
long 3 - lower word - index within rx fifo / or head & tail
long 3 - upper word - index within tx fifo / or head & tail
- long 1 - from fifo base pointer (rx fif0)
- Single char mailbox (Cluso99)
- long 0 - function/status
- long 1 - tx char
- long 2 - rx char
- long 3 - pins/parameters/etc
- Small buffers format: (Cluso99)
- long0 - function/status
- long 1 - 4 byte tx buffer
- long 2 - 4 byte rx buffer
- long 3 -
- byte 0 & 1 tx buffer head & tail
- byte 2 & 3 rx buffer head & tail
- Medium single buffer format (variant A): (Cluso99)
- bytes 0..13: 14 byte cyclic buffer
- bytes 14 & 15: buffer head & tail
- Medium single buffer format (variant
: (Cluso99)
- bytes 0..15: 16 byte cyclic buffer
- pointer & head & tail reside in another mailbox
[*]$F00 be the "system constants"
[*]$F10 be the "console input/output" mailbox
[*]$F20 be the FLASH mailbox
[*]$F30 be the uSD mailbox
[/LIST]A few standad messages come to mind (Bill):
- open
- status
- ioctl
- read
- write
- close
So, I am torn between the two choices...
Using a cog table and using an I/O table where mailboxes are used for stdin, stdout, filesystem, low level SD access, maybe auxin & auxout, etc.
So, lets get the discussion going. We have time, but not a lot of it.
Comments
Thanks for starting this thread!
Actually with David wanting CLKFREQ/CLKMODE at $E80-$E83, I'd consider that the "system" box, and mail boxes 1..23 would just be at $E80+16*n where n is the mailbox number.
Variant 1 - maps easily onto Unix style drivers, it is language-neutral and does not require applications to know anything about the internals of drivers.
Variants 2, 4, 5, 6 - are great for directly manipulating various buffers, but add additional complexity to clients - I am not convinced client apps should directly extract data from such buffers, or insert directly into them - however Ray might convince me :-)
Variant 3 - can map onto 1 easily, and the 'putchar' and 'getchar' messages could certainly use long 1 and 2 respectively.
For pins/parameters etc. I was planning on am "ioctl" message, which would use longs 1-3 to pass such parameters, or a memory block pointed to by the "from".
A "stat" (or status) message could return information from the driver either in longs 1-3, or areas pointed to by the "to" long pointer.
Basically, every C compiler's libraries map nicely onto a Unix-like paradigm, and ideally we want all languages to be able to share language-neutral drivers.
Flash and uSD low level calls also map nicely onto such a scheme.
If we use such a scheme, we can drop in new drivers with a fairly high confidence level that it will work with all code.
For language neutral standardized drivers, maybe, depends on what he does for Spin 2.
- What does this do for "discovery" of drivers?
- What if more than 23 mailboxes are needed?
C.W.
- mailboxes can be located anywhere in the hub, we are just making the first 23 easy to find
- perhaps one of the spare longs in the "system" box (that hold CLKFREQ, CLKMODE) can be used to point to an extended mailbox table
- with eight cogs, and a potential maximum of 32 hardware tasks in total, it is unlikely we will need more than 23 mailboxes
- David is right, potentially one driver could have several mailboxes, but I have not seen that yet either
Current long 0 encoding:
[cogid:3, driverid:6, reserved:7][status:7,command/data:9]
For more details about a driver, a driver could respond to a DRIVERINFO status message
For driver configuration, a driver should use IOCTL messages, or if simple enough, the OPEN message.
This way, drivers will be language-neutral, as any prop language will easily interface to such message based drivers. This will also allow for binary driver distribution.
I was thinking of cases of multiple channel serial drivers using multiple mailboxes.
C.W.
We also need a protocol for installing drivers; perhaps in order to modify the mailbox area a COG should first acquire lock 0 and release it when its finished?
It is for Universal drivers -- I think
Drivers that can loaded by Spin and used by C else Forth, Basic else like
Think You -- You write driver but will not give out code to it only BIN file.with any of languages
I can attach that and use with that possibility.
It is I think meaning
Anyway, dynamic discovery is probably required for an OS or BIOS like platform so it is certainly worth considering.
Granted -- I'm not the expert that some of you are here -- but these thoughts come to mind:
I would add another pointer to this struct that could point to an "extra info" struct (could catch everything that was overlooked at the early stages). This is where the tags etc could be stored.
I am wondering how this construct would work for buffers that are written-out more slowly than they are filled (think of a CAN or some other 'lower speed' network) that might get periodically swamped with data from a task. Should there a "driver busy" message for this kind of circumstance. I suppose the larger question here is: Are there a standard set of returns possible from the standard messages outlined above?
Is asynch I/O possible with these proposals? Or would the tasks be dogged by having to poll for completion to their prior driver messages?
Having a standardized driver model (which I think is what you all are digging at here) would be very helpful.
This standard I/O driver can be:
* A serial port
* A PS2 style keyboard and Monitor which can be one of...
* A VGA (various resolutions/sizes/colours/mono)
* A TV (composite video)
* An HD driver (not sure what this is likely to be)
* An LCD driver (char or pixel, mono or color)
* A 2pin combined Keyboard & TV
* Anything else compatible
Now, you can either select the driver, and compile your code. Or you can load a binary driver (before or after your code) and it just runs. You can dynamically change the driver while your code is running - you may have to do this in code (by another provided driver) or you may be using an OS and it can do it.
In other words, while an OS may be present, there is ABSOLUTELY NO demand for an OS to be present.
Some of us have had experience with PropDos, PropCmd, Sphinx, Spinix, PropOS, Propteus/Morpheus? (Bill's OS), Catalina, CPM/ZiCog, and perhaps PropGCC?, in addition to other OS'es on other micros, minis and mainframes.
The multiport serial driver on P1 was mentioned. how would this work?
Well, there are a few possibilities... The P2 can multitask, so it is possible that buffering can actually be done internally in the cog (or using it's own hub), and then just use 4 standard I/O mailboxes
Alternately, a mailbox might be used this way...
long 0,1,2,3 - configured as 8* words being 2 for each port
Each word would use the lower 9 bits (easy access using MOVS) where the 9th bit is used as available/free.
The upper 7 bits might be used in a different way to identify the driver.
We have to at least define a few uses of the mailboxes, and then a define a Driver ID for their use. From what a group of us have done, the overwhelming consensus is that 4 longs is an ideal standard size, encompassing what Ross did on Catalina in 2 stages.
This does not preclude 2 contiguous (or more) mailboxes from being concatenated, but there are only 23 available in the reserved area, presuming of course we start user programs at $01000 hub, which again seems to be the consensus.
For the naysayers, they can do their own thing. Nothing is binding, and the compilers/loaders should not preclude anything, though the defaults ought to be carefully considered.
I'll be spending most of the Easter long weekend with family, sneaking some forum time in when I can; so I will make a consolidated reply...
ctwardwell:
ok, four port serial driver is a good example of using four mailboxes
ersmith:
Sorry, four longs, quad long aligned is ideal (RDQUAD/WRQUAD are atomic); and I don't like embedding the driver name. DRIVERID's could be looked up.
For my boards and software I am using LOCK 0 for avoiding contention to memory, perhaps we should use LOCK 1 for drivers?
David:
There is room for a 6 bit DRIVERID (think SERIAL, SPI, VGA, HDTV etc), and seven reserved bits; also a stat message could get more info
Yes, the idea is to have the drivers be independent of the application (and language), however for bare metal they could be statically linked into the image.
Dynamic discovery was not a goal, that is why there was no provision for a name. I suppose DRIVERID could be used for limited discovery, but the intent was more for checking that needed drivers were linked or loaded.
If desired, full dynamic discovery could be implemented by doing an extended stat call on each mailbox.
To keep the design clean, easy for new users to use and understand, if a single cog provides more than one service, it should probably use more than one mailbox.
Obvious exceptions: generic i2c cog (can be used to talk to multiple devices), flash cog (more than one chip) etc
Tinkersalot:
ioctl messages are for control, stat messages for getting more info - no need for another pointer
both sync and async IO are possible with this scheme, a stat call can poll to see if the io is completed, or the call can block waiting for a status update in the mailbox.
Ray (Cluso99):
Excellent explanation!
(fyi, my OS is/was Largos)
This thread seems to discuss means of sharing services for a "hosted application" as the basis for some kind of Propeller Computer. Of course none of it is necessary for a single embedded program that performs a specific task saved in boot Flash.
It seems hosted applications are started by some kind of on-chip program loader (possibly with program parameters), and when the application terminates, the host can possibly get some status from a program. A host should be able to kill a program that is stuck for example and clean up any residual effects.
Whatever happens the requirements to me are:
1) optional. that is, the whole idea can be ignored - this appears to be a stated goal.
2) simple. if too many fields are added to a mailbox for example, then it is propbably wrong.
3) expandable to a point. a list of pointers should be good enough to find everything else.
4) small footprint. if the idea is too big, it won't get used.
If you want to provide services for an application to use, there must be a defined way to interface with the services - this is apparently what needs to be solved. Whether you need a service directory or not depends on how much flexibility is wanted or available in the footprint (tiny as possible or full service?).
Is it possible to define a header that describes a type or version? This is trying to address what always seems to be the sticking point in these discussions. Defining or interpreting a type in a header, you could use some simple classification, a full service classification (if ever), or some other for that matter to determine the interface compatibility type - a single byte or less would be sufficient for this.
I'd like to think that drivers and mailboxes could be done easily. If any code can access it without unreasonable over-head I'm all for it.
As for drivers, two obvious types come to mind. Character and buffer types. The ioctl as Bill mentioned can be added to any type, though the details of that interface must be also defined, and ioctl is a greek word to some. One can argue that a socket interface is mostly a buffer type, but it is not a file strictly speaking (although it can be made to look like a file). Sockets, for those who don't know, are the notification (and transfer to a point) mechanism for network packets.
The simple serial and keyboard types are character types. There is an argument for FullDuplex to be buffered, these can be read/written with standard fread/fwrite, but are most likely too complicated for doing seek like in a file.
Multi-port drivers or any other for that matter can use an index number in specifying a file. Just because it's a different physical port doesn't mean it needs a different mailbox, etc...
That's all I have time for just now. Most of it would probably be ignored anyway. LOL.
The "naysayers" are the ones who say we don't need this, or antagonists to any ideas proposed but who do not propose any alternatives. The P1 has proven we DO need something! Many have tried to discuss this on the P1 and totally wasted their time.
I don't have time right now to fully respond to your post at length. But yes, there are character drivers and buffered drivers. And again yes, we don't want to make it too complex that noone will use it, or too simple that it doesn't cater for the variations. There will always be some things that just don't fit, and that is fine too.
David: Ross allocated IIRC 1 long for each cog, then allocated another 2 longs that the first cog pointed to. Then if you required more than 2 longs you had to allocate more yourself. Therefore, upon open/start catalina and your program required to establish where the 2 long mailbox was, using the first as a basis, (This is sort of how it is done from my memory anyway). So the argument was why not allocate 4 longs in the first place, and this way you gained an extra usable long. It didn't actually waste hub because the instructions to vcalculate the location took up at least an extra instruction and hence long. Now Ross allocated his mailboxes for each cog. Others (Sphinx and my PropOS) allocate long(s) for services (more like drivers) such as stdin, stdout, auxin, auxout, and of course FAT32 and SD.
[cogid:3, driverid:6, reserved:7][status:7,command/data:9]
It looks like the intent of "driverid" is to indicate the type of driver, SERIAL, I2C, etc.
Maybe it should called "drivertype" and part of what is now marked as reserved, say maybe 3 bits, could be "driverid" which would used to identify specific instances when more than one driver of a given "drivertype" is present.
What about removing the identification type data, cogid, drivertype, driverid, etc., from the first long and making them part of a second level?
It seems like in the bare metal scenario and even in an OS scenario once discovery has been completed that data is not really needed.
The second level can hold identification info as well as anything else useful for discovery and then just have a pointer to the corresponding mailbox.
C.W.
A lot of good points and questions. We are trying to keep it small, simple - yet flexible.
You are right, a different channel/file number allows one mailbox to be used for multiple serial ports, however that does make waiting for a character more difficult - espetially if there was one client per serial port.
ctwardwell,
Correct, driverid is to identify the type of driver. The seven reserved bits could be used to identify a specific version.
Having minimalistic identification embedded into the mailbox allows the user application to quickly check for needed capabilities without having to do stat calls. Only apps that need a lot of detail from drivers that provide them need to do that.
all,
the basic idea is to have a small, simple scheme that handles most needs easily, we can always complicate it later for outrider cases :-)
The four long mailbox, as described, maps rather nicely onto the standard unix-like driver interface, which is well understood and well tested - and does not require many layers of abstraction that would chew up cog and hub resources.
I thought the mailbox allowed for a pointer that could be used to access the different channels or other necessary resources for the driver. If the mailbox doesn't allow for such things, then the model is broken.
Ok, then no problem using multiple ports with one mailbox right?
Flexibility is key.
Why exactly do you need two pointers in format one though or exactly four longs for each "type"? Is it to align space for RDQUAD?
Honestly I think there should be some global key/variables in the topic memory space somewhere that can indicate the "type" methodology being used or what the table means. Then the market of ideas can decide what makes sense rather than being required to make exactly one "type" or some other contraption work for all solutions.
A global header could start with a key like 0xFEEDFACE, then a long with 0xUUUUTTTT where U is a user code and TTTT is a type code. The next 2 longs through the end of the space could then be reserved according to type. If the 0xFEEDFACE key is not found, then the table has no established meaning.