Shop OBEX P1 Docs P2 Docs Learn Events
"Spinless" objects, "coglet's", etc. — Parallax Forums

"Spinless" objects, "coglet's", etc.

ctwardellctwardell Posts: 1,716
edited 2012-04-23 07:06 in Propeller 1
I recall seeing discussion in the past, about objects with no dependency on spin.

The general idea is to have objects that can be loaded and used by any language.

Did the group, as in forum members, ever come to any consensus on best practices?

I'm working on an 1802 emulator and plan to use cogs as various sub-systems, video, keyboard driver, etc.

I'd like to not have any dependency on spin.

I can just roll my own, but if there is some agreed to "standard" I'd like to use it if possible.

Thanks,

C.W.
«1

Comments

  • Dr_AculaDr_Acula Posts: 5,484
    edited 2012-04-12 16:21
    I wrote a whole lot for C at one stage.

    The 'standard' is kind of dictated by the hardware, and basically you can't reference any hub locations (rdlong, wrlong) within the cog code unless you have explicitly passed that location in the startup parameters.

    This means that some code in the obex won't work as it declares "myvariable" in the spin and then references "myvariable" in pasm. Of course, the compiler in this situation knows where "myvariable" is so the pasm will still work.

    In a general sense, and it may be slightly politically incorrect to say this, all the code written by people employed by parallax seemed to work out of the box, and followed a standard of sending a list of variables at startup. It is the third party code that seemed to break the rule more often.

    So I did the low hanging fruit - the keyboard driver, mouse driver, the standard video drivers.

    The ones I didn't do were because they were quite hard to find those particular variables in the pasm. What you have to do is take a typical object, compile the pasm separately and then find where it fails, and then go through and put back in the variables passed at startup and see if it works.

    There is another problem you may come across, and that is objects where there is a huge amount of supporting spin and only a bit of pasm. So if there is lots of cog space spare, and the spin looks like it could be translated easily into pasm, the temptation is to then start on that translation. Which can take a bit of time (but is well worth it).

    The huge advantage of 'coglets' is you can put them all in eeprom or on an SD card and reclaim up to 14k of the 32k hub space.
  • kuronekokuroneko Posts: 3,623
    edited 2012-04-12 16:40
    @ctwardell: FWIW, all my more recent drivers use the concept of cog binaries. This includes plain cog images (coginit/cognew) and overlays. The image itself features a header which contains a magic number and some flags. You can also include information about label mapping tables (e.g. the caller can extract function entry points by name). For an example have a look at the [thread=139112]most recent VGA driver[/thread], files of interest are the DDK (HowTo) and the source itself (128xP, specifically the 2048 files).

    Note, the support code (caller context) is currently written in SPIN, this would have to be rewritten in the host language.
  • ctwardellctwardell Posts: 1,716
    edited 2012-04-12 17:57
    Thanks for the input.

    It looks RossH's plan is still evolving and it may be more complex than what I need, looks promising for an "OS" based approach.

    Not breaking the 'standard' of only accessing hub locations that are passed in PAR as mentioned by Dr_Acula is definately something to keep in mind.

    kuroneko, your methods look like what I'm after. I may have a few questions for you after I spend a little more time reviewing your code.

    Thanks again,

    C.W.
  • prof_brainoprof_braino Posts: 4,313
    edited 2012-04-12 18:30
    Can you interface using CSP channels? Then it doesn't matter what language, or what processor.
  • ctwardellctwardell Posts: 1,716
    edited 2012-04-12 18:58
    Can you interface using CSP channels? Then it doesn't matter what language, or what processor.

    I think that's above my paygrade. ;-)

    For what I'm doing I'm looking for a "conventional" way to pass params to a cog during initialization and to not depend on any spin "helper" code.

    At this point it looks like it will be having PAR point to a structure that holds the config info for each cog.

    My plan is for the "initialization structure" to be temporary, once the cog loads and reads the info from the structure it will set the first long of the structure to $0000 to indicated that the memory is free for other use.

    This of course means the first long will need to be some value that is never $0000 on entry.

    The reason for the temporary structures is I want to free as much hub ram as possible to be used as "RAM" for the emulated system.

    I seem to recall seeing someone use a similar idea of clearing the memory @PAR after initialization, there are so many ideas on the forum that it is very hard to keep them straight.


    C.W.
  • Cluso99Cluso99 Posts: 18,066
    edited 2012-04-12 19:51
    I am also working on this for an OS. I found spinix to be overcomplicated for what I wanted/required as it more like *nix and I wanted a more DOS like interface. My aim is th make ZiCog totally independant from the hardware by loading "objects" that are for StdIn and StdOut and can be any driver (pc/keyboard/tv/vga/lcd/serial and my 1pin tv and keyborad drivers). I am taking kyedos as the base, and then using some sphinx concepts. I am trying to use as much as there is already done as possible, and just do the minimum mods to make this work.

    Sphinx provided the base to separate the spin interface from the driver. My thread is here
    http://forums.parallax.com/showthread.php?138251-A-Propeller-OS-that-can-run-on-multiple-hardware...&highlight=propeller+os
  • RossHRossH Posts: 5,336
    edited 2012-04-12 21:38
    ctwardell wrote: »

    It looks RossH's plan is still evolving and it may be more complex than what I need, looks promising for an "OS" based approach.

    Hi C W

    The current version is pretty much "final" as far as I'm concerned, and is already implemented in Catalina 3.5. I believe this model is actually pretty close to the "minimum" we will eventually need (although it may take a while yet before others reach the same conclusion :smile:). Personally, I would have liked to see it go a bit further - but this is as much as I was willing to implement without help.

    Ross.
  • potatoheadpotatohead Posts: 10,253
    edited 2012-04-12 21:43
    I don't think that's politically incorrect at all Ross. There was "make it work" and "do it well" in play, and when one is at, "can I make it work?" stage, it ends up being what it is. Anyone want something of mine updated? PM me.
  • Cluso99Cluso99 Posts: 18,066
    edited 2012-04-12 23:52
    potatohead wrote: »
    I don't think that's politically incorrect at all Ross. There was "make it work" and "do it well" in play, and when one is at, "can I make it work?" stage, it ends up being what it is. Anyone want something of mine updated? PM me.

    I don't understand???

    Ross sought comments before he implemented it in 3.5. Catalina 3.4 was using an older mechanism and Ross put it all out there for comment but there was really no consensus. The one Ross implemented seems to me to be middle ground with the option for more complexity. Currently I am implementing a simpler fixed allocation scheme following on from the work of mpark in sphinx. But I have had to dive in to use a plugin in catalina and have now seen the 3.4 implementation first hand. While it was a bit difficult for me to grasp, I think this is because I am not a C programmer and some of C concepts are really foreign to me.

    Dave has implemented another complex scheme with different results.

    Unfortunately, I still think it is early days for some to realise what is required. Some of us require an implementation now. So I am trying where possible to permit my method to be modified later easily, perhaps when consensus has been obtained. After all, CPM took some time to reach its standards, with a lot of intervening OSes out there first.
  • ctwardellctwardell Posts: 1,716
    edited 2012-04-13 04:57
    potatohead wrote: »
    I don't think that's politically incorrect at all Ross. There was "make it work" and "do it well" in play, and when one is at, "can I make it work?" stage, it ends up being what it is.

    I think you are referring to Dr. Acula's comment.

    C.W.
  • ctwardellctwardell Posts: 1,716
    edited 2012-04-13 05:00
    RossH wrote: »
    The current version is pretty much "final" as far as I'm concerned.

    I'll try to go back this evening an read your documentation. I based my comment on the amount of discussion still going on in the thread.

    C.W.
  • RossHRossH Posts: 5,336
    edited 2012-04-13 05:18
    ctwardell wrote: »
    I'll try to go back this evening an read your documentation. I based my comment on the amount of discussion still going on in the thread.

    C.W.

    The last discussion in this thread was in Jan/Feb, which was probably the last time I spent any substantial amount of time in these forums. Or are you referring to some other thread?

    Ross.
  • ctwardellctwardell Posts: 1,716
    edited 2012-04-13 05:29
    RossH wrote: »
    The last discussion in this thread was in Jan/Feb, which was probably the last time I spent any substantial amount of time in these forums. Or are you referring to some other thread?

    Ross.

    Ross,

    I was referring to that thread, mostly post 21 and 22 where your reply to ersmith "There's some stuff in there that I will have to re-read and think about" lead me to think there may still be some changes.

    C.W.
  • Dave HeinDave Hein Posts: 6,347
    edited 2012-04-13 10:08
    Cluso99 wrote: »
    Dave has implemented another complex scheme with different results.
    I agree that my approach in Spinix "looks" complex, but I think that perception is because I haven't documented it. It's actually pretty simple. My PASM cog drivers include the Spin code that starts them. I basically split a standard object into two files, where one file contains the PASM code plus the Spin start routine, and the other file contains the rest of the Spin code.

    As an example, the the safe_spi.spin object from FSRW is split into two files. safe_spi_p.spin contains the start routine plus the PASM code. The original file was modified to contain only the Spin driver code. I should probably rename this to something else, such as safe_spi_d.spin. The boot program runs the start method in safe_spi_p.spin, and sets up the mailbox at a known location defined in sysdefs.spin. It also adds an entry in the process table so that Spinix knows which cog it is using, and that it is a PASM driver.

    A better example of a self-contained driver is the PASM I2C driver. A program called loadi2c is used to load and unload the driver. It is self-contained, which means that it will check whether the I2C driver is already loaded, and start or stop a PASM cog with the driver. It also adds or removes the driver in the process list. That's the model that I plan on using for all Spinix drivers in the future.

    My view is that a single driver standard is not necessary, or even practical. I see no problem with maintaining the status quo, and allowing various approaches to donig drivers. I think it promotes more creativity that way.
  • Cluso99Cluso99 Posts: 18,066
    edited 2012-04-13 11:52
    Dave: Perhaps we can discuss this again. I wont have time this week and there will be a lot of things coming out of UPEW. Some of the complexities to me were IIRC more on the *nix base than the simpler dos base. But I DID like your concept of the fixed cog table but I thought 6 longs was not ideal (not a power of 2) so I thought that 8 longs would be better and this obviously needs a full preferred definition. Ross has two tables (catalina 3.4) of 1 & 2 longs which is more complex. I cant recall what he decided for 3.5.

    I agree that the drivers should remain as close to standard as possible, with a simple start spin method. If we can agree on a minimum base hub format and a couple of other OS type things we can pool our resources rather than the split atm. For example, if we combine the efforts, we only need 1 DIR module/program, etc. The one thing I see as necessary is the ability to change any program running to use a StdIn & StdOut simple interface so that the driver can be changed without the program knowing or caring. In this respect, I dont want it to be complex (not a *nix style file i/o scheme, but a character scheme) and for this reason I wanted a hub definition for this.

    BTW: Your xtal concept is fantastic. I have modified Kyes driver to keep the xtal params (hub $0000-$0005) and not to load/clear above the hubtop (the allocation table). The one standard I really would like to see is that the allocation pointer to the top of usable hub be located at the long $7FFC (or whatever it will be in P2) i.e. the last hub sram long. I think we all agree that an alloc is required, a fixed cog table of some size, and some other os parameters.

    I want space for my faster spin interpreter (the vector table) and it must be on a boundary (1KB on a 1KB boundary eg $7800-$7BFF ).
    Anyway, I need to see what is happening in UPEW and will revisit this later in the week.

    Postedit: Fixed some errors (in a rush to hear UPEW)
  • potatoheadpotatohead Posts: 10,253
    edited 2012-04-13 14:16
    Yes, referring to Dr_A. Nice catch, no worries.
  • RossHRossH Posts: 5,336
    edited 2012-04-13 16:41
    ctwardell wrote: »
    Ross,

    I was referring to that thread, mostly post 21 and 22 where your reply to ersmith "There's some stuff in there that I will have to re-read and think about" lead me to think there may still be some changes.

    C.W.

    Fair point.I did in fact end up adopting one of Eric's suggestions (to do with memory allocation), but at the time I figured I would soon have Catalina 3.5 to release with a fully working implementation, and I haven't been back to the thread.

    I hope to have Catalina 3.5 out "Real Soon Now"TM

    Ross.
  • Cluso99Cluso99 Posts: 18,066
    edited 2012-04-13 19:22
    RossH wrote: »
    I hope to have Catalina 3.5 out "Real Soon Now"TM
    Ross.

    No way... ms must have beaten you to that TM.

    I find it strange that ms hasn't prosecuted builders from using "windows" in houses!
  • Dr_AculaDr_Acula Posts: 5,484
    edited 2012-04-13 19:32
    I wonder if one could think about a proptool that could handle coglets?

    First, don't define any protocols or standards for cog comms. That is up to the user to do.

    Let's say I have a program and it has a single pasm section, and I want that pasm to be loaded from somewhere externally rather than included as part of the program.

    You can put the pasm code on an sd card but the overhead of the SD code (maybe 1/4 to 1/3 of hub ram) kind of negates using SD. So I'm wondering about using an eeprom, and simply use a 64k instead of a 32k eeprom.

    You may not need to always load the data to eeprom. What you need is a smart editor program and if you hit F10, it knows whether the pasm code was changed since last time you did a download, and it only sends the pasm part if there is a change. So most of the time the coglets are not updated.

    It probably would need a tiny bootloader program on the prop. Send that first as part of the F10 download and it handles transferring any new coglets into eeprom. Then send the main program.

    In the editor program, you might need a checkbox to select or deselect sending pasm separately. It might get a little complex if some pasm code you want included and some you want separate.

    This could almost double the size of a spin program you could fit in a propeller.
  • Cluso99Cluso99 Posts: 18,066
    edited 2012-04-13 21:08
    Drac: The problem with the SD code is FAT compatability. If you forgo that you could have a small footprint. So eeprom does not get you anything really. Much better for an os to have and use only SD after initial boot. Otherwise, the os will require both SPI and I2C and the I2C will require some type of file structure.

    But we need to keep the hub to a minimum. If you don't need FAT after you load up everything then that can be unloaded. Currently in my os (derived from your kyedos) the sd driver of kye's is not resident at all.

    I have reserved a 2KB block of hub ram ($7800-$7FFF):
    • a 1KB vector table for my faster spin interpreter (can be optional)
    • a 512B sector buffer
    • 8 x 32B cog table (undefined for now)
    • 3 x 64B buffers (which can be joined as 1 x 192B buffer) eg for string buffers, etc
    • leaving 64B (16 longs) for miscellaneous including the last long as a pointer to the start of protected hub sram (eg $7800)
    • ..
    Dave has a 24B cog table. I prefer this to be a block of longs to the power of 2 so that a simple shift of the cogid is all that is required to access your cog table. The first B should be a "type" flag indicating the cog is loaded, and what type of service (input, output, I/O, etc it provides). Something akin to Catalina seems perfect. The rest can depend on the useage, but we should define a couple of standard ones for keyboard, tv, vga, pc serial. If the next 11B is not required, then it should be used to store the cogs' filename it was loaded from. Now, presuming we allocated 32B (8 longs) per cog, that still leaves 20B (5 longs) free for buffer/whatever. The alternative is to trim to 16B (4 longs) per cog.

    I allocated a long to be used for: 1B with a bit for each cog to indicate it should remain loaded between each os/program load. 2B stores the screen size (8 bits columns, 7 bits row, and 1 bit for autoLF). 1B stores the clock mode.

    1L stores the clockfreq; 1L stores the SD pins (1B per pin); 1L stores the serial pins (SI, SO, mode and the cogid); 1L stores the baud;

    I used 4L to store rendezvous locations for: StdIn, StdOut, AuxIn, AuxOut. Perhaps with the Cog Tables, these could be reduced to a nibble each and just store the cog# to be used. I really liked the Sphinx idea of a single buffer for stdin and another single buffer for stdout, etc. But, this is a problem that means serial needs to internally buffer, or another buffer needs to be added to the mix (particularly for input). We have a few drivers (serial, tv, vga 1pin tv, keyboard, 1pin keyboard) already done with 1 character buffers courtesy of Sphinx. This method gives spinless objects. I would like to see some more discussion here.
  • ctwardellctwardell Posts: 1,716
    edited 2012-04-15 19:23
    It looks like there are two use cases here:

    1) Treating the prop as a general purpose computer running some type of OS with an SD card based file system.

    2) More traditional "microcontroller type" applications running a fixed set of firmware to do one specific task.

    Most of the ideas presented here seem to be more focused on case 1 than case 2.

    My primary interest is case 2.

    C.W.
  • jazzedjazzed Posts: 11,803
    edited 2012-04-15 23:48
    ctwardell wrote: »
    It looks like there are two use cases here:

    1) Treating the prop as a general purpose computer running some type of OS with an SD card based file system.

    2) More traditional "microcontroller type" applications running a fixed set of firmware to do one specific task.

    Most of the ideas presented here seem to be more focused on case 1 than case 2.

    My primary interest is case 2.

    C.W.

    Hi C.W.

    Case 2 is a simple and practical approach as first presented here - resetting the propeller really complicates things and will be avoided. In the near future we will be adding a loader feature that put's COG code into upper 32KB EEPROM locations for loading in Propeller-GCC according to the object linking. This eliminates complexity required otherwise. That is, you always know before hand what will be loaded and can let the compiler and tools do the work for you with no overhead other than PAR mailbox info. The start-up I2C COG PASM buffer in the main program will be recycled for the code array space - the I2C PASM will read the code into the array and then we can do cognew with that. That is the general approach we will take.

    Does that sound about right for your interest?

    Thanks,
    --Steve
  • Dr_AculaDr_Acula Posts: 5,484
    edited 2012-04-16 01:23
    Thanks jazzed for that discussion thread link. All very interesting to read through that.

    re Cluso99, I am thinking simpler than a FAT. If you are going to put 7 coglets into high eeprom, you don't really need a FAT. Make them all the same length (2048 bytes) and put them one after the other in order. The compiler and the program then knows where they are.

    I think this is the same as what jazzed is saying.

    I'm also thinking you have a smart compiler that only reloads eeprom code that has changed. So when you first turn on your PC and start the compiler (GCC or whatever), maybe the first download it either sends all the coglets to eeprom, or if you want to be smart, reads them off the eeprom, compares with what might be downloaded, and only downloads if things have changed since before (this latter system avoids any unnecessary eeprom writes).

    Then if the compiler program detected a change when you do an F10, it would send new code. Most of the time, coglets would be fairly stable code, eg keyboard, mouse, video driver, so they never need change. This could even speed up a download compared with the current proptool as you would not be sending all the pasm code each F10. I am thinking the way I often write programs, I start by grabbling the core bits out of the obex. Mouse. Keyboard. A video driver. So the first time you download all that, all the coglets go out to eeprom. But from that point on, as you write and rewrite your main program and do repeated F10 downloads to test it, no further coglets ever get written to eeprom. They are already there.

    re
    I have reserved a 2KB block of hub ram ($7800-$7FFF):

    I'm not sure you need any rules about where things are in hub ram. That will change for each program you write, and you might not even need a buffer for keyboard or serial port (eg the GUI I'm writing with a touchscreen, where inputs and outputs are pictures, not text, and the pictures are too big to fit in hub anyway). So the only rule really is that all parameters must be passed to a cog via a par section at the beginning. That keeps it very simple and easy to explain.

    It sounds like the GCC team is making great progress.
  • ctwardellctwardell Posts: 1,716
    edited 2012-04-16 03:58
    jazzed wrote: »
    Hi C.W.

    Case 2 is a simple and practical approach as first presented here - resetting the propeller really complicates things and will be avoided. In the near future we will be adding a loader feature that put's COG code into upper 32KB EEPROM locations for loading in Propeller-GCC according to the object linking. This eliminates complexity required otherwise. That is, you always know before hand what will be loaded and can let the compiler and tools do the work for you with no overhead other than PAR mailbox info. The start-up I2C COG PASM buffer in the main program will be recycled for the code array space - the I2C PASM will read the code into the array and then we can do cognew with that. That is the general approach we will take.

    Does that sound about right for your interest?

    Thanks,
    --Steve

    Steve,

    It sounds very close. Will the COG code be able to be imported as a PASM binary?

    By that I mean if I've already created a PASM binary using "normal" propeller assembly in the propeller tool will I be able to use that binary versus needing to code it using GCC or GAS.

    It sounds like the only requirement of the COG PASM is that it gets all setup info via the address passed to PAR, is this correct.

    Thanks,

    C.W.
  • Dave HeinDave Hein Posts: 6,347
    edited 2012-04-16 06:16
    There's been some great ideas proposed in this thread about Spinless objects, but to me it seems like it's better to include a little bit of Spin code to set up the PASM cog. This alleviates having to do the setup in PASM or having the calling program do the setup. As far as storing the drivers in EEPROM, you could implement a very simple linear file system in EEPROM. It would basically consist of a file name, file size and file contents stored one after another. Another possibility is to have a small directory consisting of file name, size and starting locations. The EEPROM file system would be read-only, so it doesn't have to be very complicated.
  • Heater.Heater. Posts: 21,230
    edited 2012-04-16 06:41
    Dave Hein,
    There's been some great ideas proposed in this thread about Spinless objects, but to me it seems like it's better to include a little bit of Spin code to set up the PASM cog

    Perhaps so but what if you want to borrow such a "coglet" for use in language other than Spin? Arranging to get binary blobs of "firmware" started in a COG is hard enough with having to also start up a Spin interpreter to configure the thing and get it going.
  • Dr_AculaDr_Acula Posts: 5,484
    edited 2012-04-16 07:02
    How about keeping the spin code to such a minimum that it is easy enough to write the equivalent code in any language? eg

    To start
      cog := 1 + cognew(@mycode, @parameters)  
    

    To stop
       if cog
          cogstop(cog~ - 1)    
    

    and to pass parameters and do clever things like pass different commands to a cog to run various routines, just designate one of the longs in the parameter list as a "command" long and change it and the cog code changes it to something else to say it has finished. It will all be very similar in different languages - you talk to a cog by changing a long in hub, and wait for the long to change to signal the routine has finished. All languages can write to and read longs in hub.

    If the cog is a "start and run forever" cog, you just run the startup routine.

    I suppose the key here is good documentation. For a particular coglet, you might define it has n longs as the parameter list, and they are in a fixed order. So you just need to explain that in a verbose way that is easy to understand.

    re ctwardell
    By that I mean if I've already created a PASM binary using "normal" propeller assembly in the propeller tool will I be able to use that binary versus needing to code it using GCC or GAS.

    That is exactly the idea. You can compile these using the proptool. Save them as a .binary file. Or a .cog file, or whatever you want. Store them on an sd card, or in high eeprom, it does not matter. Once that binary file exists, any language can use it, Spin, C, Basic etc.

    This is some code I was using for Catalina that is a skeleton pasm routine that compiles independently using the proptool and which passes a group of variables:
    CON
      _clkfreq = 80_000_000
      _clkmode = xtal1 + pll16x
    
    PUB Main
        coginit(1,@entry,0)                     ' cog 1, entry, dummy value
    
    DAT
                      org 0
    entry                                          ' start of the cog code
                      mov arraypointer,par         ' arraypointer is the address of the array
                      rdlong array0,arraypointer   ' array0 = array[0]
                      mov array1,array0            ' array[1]=array[0] - long is 4 bytes so add 4
                      add array1,#4                ' array[1] += 1 change the value so see cog works
                      add arraypointer,#1          ' pointer += 1
                      wrlong array1,arraypointer   ' write array[1] back to hub
                      jmp #entry
    
    arraypointer      long    $00000000            ' pointer to array[0]
    array0            long    $00000000            ' first array location
    array1            long    $00000000            ' second array location
                      fit 496
    
  • jazzedjazzed Posts: 11,803
    edited 2012-04-16 07:53
    ctwardell wrote: »
    It sounds very close. Will the COG code be able to be imported as a PASM binary?

    Yes, we support PASM, GAS, or GCC native COG C drivers. We have a few different ways of extracting PASM from the SPIN code.

    Going the extra step to loading the COG code after boot just requires a loader enhancement and including the I2C driver.
    ctwardell wrote: »
    It sounds like the only requirement of the COG PASM is that it gets all setup info via the address passed to PAR, is this correct.

    Yes, that's correct.

    Using SPIN methods to poke values into the PASM at SPIN run time before cognew should of course be avoided. Some obex code does this and would have to be ported. The Graphics.spin file is an example that had to be ported. Of course if you know the address of things to poke it doesn't matter except for portability and keeping up with whatever changes may happen.

    I like the idea of a COG letting the main program know when it's ready, but it is not necessary to do that. When you have control over things at compile time, you can do whatever you like within reason.
  • ctwardellctwardell Posts: 1,716
    edited 2012-04-16 08:05
    Jazzed,

    Excellent. Do you have an ETA for when the loader will be available?

    I'll move ahead making my objects having them get setup via PAR and can migrate to the new loader and GCC when it is ready.

    Thanks,

    C.W.
Sign In or Register to comment.