Cluso's Propeller OS V1.14 - now with SPIN/PASM Compiler, EEPROM Read/Write :)

Cluso99 · 2012-02-26 22:08

An OS (operating system) for the P8X32A Propeller and SD card using FAT16/32...

Should work on all propeller boards using 5MHz PLL16X and with SD/microSD on P0..3, plus a serial terminal program (eg PST) on P30/31.
The serial terminal program (on your PC) is usually connected to P30/31 via a USB-Serial adapter such as a PropPlug.
Automatically detects and supports the RamBlade, TriBlade(2), RamBlade3 (CpuBlade+MemBlade), P8XBlade2 directly, else defaults to 5MHz PLL16X and SD/microSD on P0..3 (without recompiling).
Other boards are supported by simply changing _hwdef.spin and recompiling _os.spin.

Version 1.14 08 October 2016...
Better DUMPFIL & DUMPHUB display both hex and ascii.
DUMPTOK & DUMPSOB to display intermediate steps of the modified Sphinx Spin/Pasm Compiler.
EEPROM now supports Read & Write <file> from/to Lower/Upper 32KB EEPROM.
https://forums.parallax.com/discussion/comment/1389683/#Comment_1389683

_BOOT_OS v1.11: P8X2, SD mount FAT32   "V110_OS    ", boot _BOOT_P1.BIN, boot _OS.CMD

*** Cluso's Propeller Operating System v1.13?***
------------------------------------------------
 (see 'VER' for acknowledgements)
 Hardware: Cluso's P8XBlade2
 ClockFreq 096000000Hz, ClockMode $6E, Cog 0
 Serial on pins 30,31, Mode 00, Baud 00115200, Cog 1
 Screen size: 080 x 040
 <LF> is OFF
 SD card on pins 12,13,14,15
 SD Driver Cog = 2
 Resident Cogs : 1 2  This Cog: 0
 Cogs available: 3 4 5 6 7 (=5)
 Booting from SD card...
SD:>help
*** Cluso's Propeller SD card Operating System v1.13 ***         (26 July 2016)
-------------------------------------------------------------------------------
clear                                     ' Clear screen
cmd     ---                               ' Internal use (command interpreter)
copy    <source_file> <destination_file>  ' Copy a file
del     <file>                            ' Delete a file
diff    <file1> <file2>                   ' Display file differences
dir     [mask]                            ' Display directory of files
dircpm  <cpm_disk>                        ' Display directory of files on CPM
dnload                                    ' Download code Ctl-F10/F11 w/o reset
dumpfil <file>                            ' Dump xxx file in hex+ascii
dumphub <addr> <bytes>                    ' Dump hub memory addr(hex) bytes(hex)
dumpsob <file>                            ' Dump SOB file (compiler file)
dumptok <file>                            ' Dump TOK file (compiler file)
echo    <text>                            ' Echo the line of text
eeprom  <file> [{-WL}{-WU}{-RL}{-RU}]     ' EEPROM Read/Write Lower/Upper 32KB
free                                      ' Display FAT16/32 used/free space
getcpm  <cpm_disk> <cpm_file> <fat_file> -T/B ' Get (copy) CPM file to FAT16/32
getfat  <fat_file>                        ' Get (copy) FAT16/32 file to PC
help                                      ' Display help info
lf                                        ' Toggles LF on/off
ls      [mask] [A]                        ' Display directory of files
mapcpm  <cpm_disk> [<cpm_file> [-d]]      ' Maps info about the CPM filesystem
os      ---                               ' PropOS binary file (runs PropOS)
putcpm  <fat_file> <cpm_disk> <cpm_file>  ' Put (copy) FAT16/32 file to CPM
putfat  <fat_file>                        ' Put (copy) FAT16/32 file from PC
reboot                                    ' Reboot the Propeller Chip
ren     <source_file> <destination_file>  ' Rename a file
run     <file>                            ' Run a ".BIN" file (kills PropOS)
testsd                                    ' Tests the SD card (writes to SD)
type    <file> [-Hn]                      ' Display the contents of a file
used                                      ' Display FAT16/32 used/free space
ver                                       ' Display info about PropOS
-------------------------------------------------------------------------------
*** Compiler (spin/pasm) in 3 parts...
lex     <file> [-Vn]                      ' Compiles xxx.spn to xxx.tok
codegen <file> [-Vn]                      ' Compiles xxx.spn/tok to xxx.sob
link    <file> [-Vn]                      ' Compiles xxx.spn/tok/sob to xxx.bin
-------------------------------------------------------------------------------
_OS     .CMD    ' Boots the OS
BOOTxxxx.EEP    ' May be programmed into EEPROM to boot PropOS (v1.10)
                '  - runs SD program "PROP_P1.BIN" else "_OS.CMD"
    xxxx = board/xtal/sdpin specific (refer to forum/website)
-------------------------------------------------------------------------------
To RUN any Propeller Binary (replaces the OS completely)...
SD:>RUN z3_174fx                          ' Runs ZiCog RamBlade3 V174 binary                                                         
        ^^^^^^^^ this binary requires the RamBlade3 hardware
A:>HALT                                   ' Reboots the prop (from CPM prompt)
-------------------------------------------------------------------------------
SD:>

Version 1.10 04 March 2016...
Now includes Propeller based Spin/PASM Compiler (based on Michael Park's Sphinx Compiler)
Latest version is here...
forums.parallax.com/discussion/comment/1367946/#Comment_1367946

Version 1.00 19 Feb 2016...
Latest version is here...
forums.parallax.com/discussion/comment/1366323/#Comment_1366323

Update v0.90 21 Jan 2016...
forums.parallax.com/discussion/comment/1362806/#Comment_1362806
If the code cannot detect any of my boards it defaults to 5MHz PLL16X and SD on P0..3 so it should work on many boards if an SD is added to P0..3. No recompiling required!

Update 15 Dec 2015...
It now automatically detects and supports the RamBlade, TriBlade(2), RamBlade3 (CpuBlade+MemBlade), P8XBlade2 directly.
Support for other boards is simple... Just edit the _hwdef.spin file, with PropTool select the top file as _os.spin and compile and program your eeprom with Ctl-F11.
Then copy all *.cmd files and *.txt files to the root directory on a FAT16/32 microSD card.

Full instructions here (thanks to pidzero for this): http://forums.parallax.com/discussion/comment/1358370/#Comment_1358370

Files are here: http://forums.parallax.com/discussion/comment/1358412/#Comment_1358412

Update 3 Aug 2013...
I thought it was time to update the first post with the current progress. Some objectives have changed, and some additional features have been added.
No-one seemed interested in assisting, and there was no consensus about mailboxes (apart from a very small few, thanks Bill) so I have done it alone.

I started with KyeDos (developed by Dr_Acula using Kye's FatEngine) as the base. I made each command a separate file. Next I separated out the I/O and used mailboxes. Then I rewrote most of the commands, and added many new ones. These commands are similar to the CPM & PC DOS days, with the standard DIR, COPY, REN (rename), DEL (delete), TYPE (display), etc.

My OS can currently detect my 3 hardware pcbs automatically (RamBlade, TriBlade2, RamBlade3). If they are not found, then a default hardware of Serial on P30/31 and SD card on P0-3. The xtal & PLL setting is in one file only and requires only a recompile of the OS program only (plus RUN temporarily) after the _hwdef file is changed. Currently I post both 5MHz and 6.5MHz (overclocked) compiled binaries.

My OS can now reprogram the eeprom (lower or upper 32KB) from a file.

In ZiCog/CPM2.2 the HDDs (disks) are stored as files in the FAT16/32 file system on the SD card. Additional OS commands can access these pseudo disk files, including file transfers between these file systems.

Prop Binary files may be run from the SD card using the RUN command. In this case, the OS is replaced. For ZiCog/CPM, it is possible to RUN ZiCog/CPM and at the end, return to the OS via reboot.

Today, the latest code is V0.75 at post # 59
http://forums.parallax.com/showthread.php/138251-A-Propeller-OS-that-can-run-on-multiple-hardware...?p=1198890&viewfull=1#post1198890

This OS should be compatible with the other OSes available (such as Sphin, Spinix, Catalina C) so they can all co-exist on the one SD card, and can be switched between them easily, just as I can switch to ZiCog/CPM.

BTW ZiCog/CPM allows us to run most CPM2.2 programs (ZiCog does not quite emulate all the additional Z80 instructions faithfully). In particular, there is MSBASIC and a number of file editors, etc. WordStar, TurboPascal, SuperCalc are also available but I have not tried them.

Enjoy
[/COLOR]

Original Post...
Quite some time ago I was working on a version of Sphinx as a propeller OS. I have also done some work using PropDos and PropCmd.

I also have ZiCog and CPM2.2 running on my props as well.

Over the past year or so, I have been quite busy with other things and the prop time suffered, apart from some spurts where I sneaked some time off.
Anyway, I am seeing some more free time coming up and its time to dig back into where I was up to.

Here, I have 3+ of my own pcbs...
* TriBlade
* RamBlade
* RamBlade3
* Some other normal type prop pcbs of mine

The problem I have found is that I regularly shift between the TriBlade (cpu#2), RamBlade and RamBlade3. All have 512KB of SRAM and microSD cards. My desire is to shift the same microSD card between these boards and have them all work. But, that does not work under my current regime.

It is my intention (and this is what I have done) is to only ever program the eeprom once. Programs can be copied onto the SD card by inserting the microSD into a USB adapter and I often do this. OK, some of my testing is downloaded to hub, but generally not to the eeprom. In fact, on both RamBlades I would require a WE link to write to the eeprom.

Now, I currently just program an extremely primitive bootloader into the eeprom. This loader only knows how to address the SD card and nothing else. This is the way I want it because I do not want to ever replace it. However, I recompile a different version for each pcb. That is fine. BUT, this loader then locates a file AUTOEXEC.BAT and if it exists, it runs the program file in the first line entry. If it is not found, then "BOOTPROP.BIN" is run. If not found, then the program terminates and the prop hibernates.

The problem with this philosophy is that the program "BOOTPROP" has to be different for each pcb.

So, I am going to change the name of this file in the eeprom of each hardware to be "BOOT_xxx.BIN" where xxx = TB2, RB1 and RB3 for TriBlade#2, RamBlade and RamBlade3. While I could (and may) detect these hardware differences, I think it better not to do this. To bebackward compatible, I will also look for "BOOTPROP.BIN".

Now, this will solve the first level problem. Onto the next...

I can configure the RamBlade and RamBlade3 (by plugging in a propplug or 1pin TV and 1pin Keyboard). The TriBlade#2 could also do this.

One of my dreams with Sphinx was to follow on from Michael Park, and make the I/O drivers independant from the OS. I achieved this with my 1pin versions of SphinxOS, as Michael had previously achieved. I intend to extend this to ZiCog so that I only need to release 1 version for each of my pcbs. Ideally I would like to release just 1 version to suit all my pcbs, but lets crawl first.

To make this work, I have to be able to make each (cog) driver self-contained and interface via a standard interface. That is why I posted in RossH's thread about the registry ideas. However, the registry is for cogs and the redirection is for I/O devices. I think a combination of these is in order.

I am interested in anyone who wants to comment or offer ideas or be involved in the development of this. Please, I am not trying to make a *nix platform, so I don't want the *nix overheads. The CPM and MSDOS were fine as far as I can see. So, I am after the I/O drivers to be character based, not file based.

StdOut (and AuxOut) could be any one of ...
* PC with PST or similar terminal program
* external serial device such as a VT100 terminal
* TV (composite video)
* VGA
* TV (1pin monochrome)
* LCD (various)
* Sound out (mono or stereo)

StdIn (and AuxIn) could be any one of...
* PC with PST or similar terminal program
* external serial device such as a VT100 terminal
* Keyboard (PS2 style)
* Pushbuttons
* Mouse
* Touchscreen (not sure how this would work yet)

Now, I want to be able to load, unload, and replace these drivers on the fly by reverting back to the os and changing the driver. So, I may be using the 1pin TV and 1pin Keyboard for editting source, and then I may want to switch to the pc to perform something else. I do not want to change my other "user" code. This is all possible. The hardest part is to come up with a simple arrangement that most everyone would like to adopt. It is really no use each of us going off on our own tangent, which is what has happened over the past few years.

The other I/O that I see as being required is the SD card interface using some form of Kye's FAT16/32 driver(s).

Now, the next part is where Ross's registry comes in because it is also necessary to know what is loaded into the 8 cogs. But, I think I will discuss this later.

So, I am open to comments...
Or do I just go and do my own thing??? Postedit: Did my own thing

Just a little warning:
This FAT system only supports the old filenaming system of 8.3 ie long filenames are not supported.
I believe this may be the reason that Windows complains about the format errors when inserting a card into a PC - just ignore the message, don't fix it!

Dr_Acula · 2012-02-27 00:13

Hmm. Tricky. Reloadable drivers/objects.

Given past experience, the fastest way to get something happening on the propeller is to declare it impossible. So here goes - I declare that it is impossible to have reloadable objects.

Sure, it is quite possible to have reloadable cog code, but that is only half the story. There is always the glue code in Spin or C or whatever that handles the interface.

I don't know how you reload that sort of code within the framework of a compiled language.

Precompiled DLL type files? An interpreted language? Script based languages? Compiled programs that can be relocated in memory as they only contain relative jumps and calls?

I say it is impossible to do!

Cluso99 · 2012-02-27 00:53

The way Sphinx addressed this problem was to separate the cog driver from the spin code by making the interface a simplistic character interface using the rendezvous long. It works extremely well because I have used it in ZiCog, just by changing the underlying object, and recompiling. Next is to reload the cog driver without having to recompile the code.

pjv · 2012-02-27 10:26

Dr_Acula;

You are not quite right! .....

Impossible no; at least not in Propeller-land. Last year I was able to perform reloading / relocating Cog binary blobs at runtime, simultaneously while the Cog contiued to execute other threads. The Cog itself was not shut down and reloaded.... the OS running inside the Cog performed the driver reload and relocate.

Cheers,

Peter (pjv)

Mike Green · 2012-02-27 11:06

Here are a couple of links to a Sphinx-like OS using EEPROM rather than an SD card. There are older versions that included both a TV and VGA driver that were compatible and either could be launched during bootup and the programming interface would adjust itself based on parameters left in a shared area of RAM.

http://forums.parallax.com/showthread.php?87243-Propellor-quot-Operating-System-quot-(work-in-progress)&p=598350#post598350

http://forums.parallax.com/showthread.php?88311-Propeller-OS-VGA-Version&p=605819#post605819

http://forums.parallax.com/showthread.php?88186-Propeller-OS-TV-version&p=605096#post605096

http://forums.parallax.com/showthread.php?90457-Propeller-OS-for-Demo-Board-and-Hydra&p=620700#post620700

Dr_Acula · 2012-02-27 14:14

Last year I was able to perform reloading / relocating Cog binary blobs at runtime, simultaneously while the Cog contiued to execute other threads. The Cog itself was not shut down and reloaded.... the OS running inside the Cog performed the driver reload and relocate.

Reloading cog binary blobs is certainly possible. The hard part is the high level language glue that goes with the binary blob.

You can attempt to move all the Spin or C into Pasm. Some objects will be easier than others. But many times you will run out of cog space. I tried a few of the standard objects but the task is daunting. Consider as an example a propeller object that has no pasm in it. How would you load and reload that?

I don't know how to have reloadable Spin. Or reloadable C. In a vague way, I think part of the solution is to think about code that can be relocated in memory such that jumps are relative rather than absolute. Or possibly an interpreted rather than compiled language. Or possibly some compilation down to tokens but then interpreted.

The solution to this problem also breaks out of the 32k limit enabling code to run in external memory, plus it leads to a way of writing and developing code on the propeller itself, so it is a concept worth exploring.

Cluso99 · 2012-02-27 16:53

The secret is in the I/O being character based. Take the serial out for example. If the serial driver only supports the fdx.tx command, then you just place each character into a fixed mailbox/rendezvous location. It is cleared when the pasm serial driver takes it. Any higher level outputs such as hex, dec, string, etc just place each character in this single byte buffer. Now, if TV and VGA are written the same. i.e. they also take a single character at a time via a mailbox.rendezvous location, they can be substituted at any time. That is how Sphinx works and all I am doing is extending this to be able to substitute the pasm driver at any time.

There is nothing special about this. CPM does this too.

Mike: Tahnks for the links - I will take a look.

Dr_Acula · 2012-02-27 17:42

It probably is possible to get things working if you concentrate on simple objects that output and input characters. I'm not so sure about dynamically loading in spin code. Pondering this question has just taken me down a long and winding road looking at Basic and C interpreters, all written in various languages. I'm looking at a C interpreter written in C which is an intriguing idea.

In general terms this problem comes up http://en.wikipedia.org/wiki/Position-independent_code

Objects that are almost all pasm are easy to think about as pure pasm as they are then reloadable. So solving the easy ones may be easy, and perhaps it is worth thinking about solving the hardest ones first and that solves the easy ones automatically. How about a pure Spin object like Kye's string driver. Load that as you need it, then unload it.

Unless you can get right inside all the inner workings of Spin, maybe this is easier in other languages?

I'm probably taking this discussion off in the wrong direction.

Consider the abstract idea of a file system. CP/M gives you access to opening, closing, reading and writing blocks of data. The O/S handles the hardware interface. You would need a number of SD driver objects all with different code. And it isn't even a matter of changing the CON block to passed variables, because some hardware platforms might need multiplexers to even select which pins.

This is not a trivial problem to solve for all platforms.

pjv · 2012-02-27 18:00

@ Dr_Acula
Sorry, my misunderstanding. All my musings are in regard to ASM running in a Cog...... I don't yet know that much about Spin, so not sure what all the issues are, other than speed obviously, and I have no interest in the "C" stuff or other high level languages. I'm pretty much an assembler guy exclusively.

@ Cluso;
I may be pusuing this thread improperly, but in case anyone is interested, I would like expand a little on my Spin interface, although you probably already were aware of this by explanation in the other thread.

The drivers in the Cog indeed CAN take single bytes in their mailbox, but they also can be handed the Hub starting address of a list which is then processed to its end according to rules. For example, my serial_out driver can be invoked via a single long in its mailbox as a SerByte (direct single byte) or SerAsc (converts the byte to two Ascii chars), or ZTS (zero terminated string) or a ZTZ (zero terminated string with_zero) FLL (fixed length list), or a PLL (prefixed length list where the length is the first byte in the string).

The single interface long encodes the particular invocation desired as 1 of 16 choices, plus embodies the Hub address of the string / list.

Cheers,

Peter (pjv)

Dr_Acula · 2012-02-27 19:21

@pjv, your serial idea sounds interesting. So you are passing parameters to the cog. In a general sense, this could lead to replacing spin code in an object with pasm code. No reason it can't be done for many objects. Even the string driver I mentioned above. We just need to write the code.

If every object were in pasm only, it would make reloading *much* simpler!

pjv · 2012-02-27 21:38

Dr_Acula;

Each Cog owns a fixed location near the high end of HubRam that they use as a mailbox. The OS that runs in each ASM based Cog checks its mailbox every one uSec for a command. The command is placed there by either, a Spin application, OR an ASM program. The format of the command is application dependant, but generally consists of a single long comprising a Group nibble to select 1 of 16 application groups such as SerialTx, SerialRx, OneWire, I2C, etc. Another Function nibble selects 1 of 16 sub functions within the group such as Init to set baud rate and port bits or, TxByte or TxAscii or TxFLL or TxZTS etc. Then in the mailbox long there is also a byte that can be used as a quantity specifier to control a transmit, or receive quantiy, and lasly there is a word that is typically used to set an address in Hub where further data or a selected string resides.

The intent is to keep the Cog programs as small and lean as possible, and for greatest flexibility the associated programs to manipulate the data such as formatting or decimal conversion, is left out in the Spin domain. The Cog internals are treated as high speed "set and forget" engines that self terminate (but stay residentt) on concluding their respective tasks, and through the mailbox signal interested Spin programs that they are finished. And of course start checking the mailbox for a signal to be triggered again.

I chose this path because "data push" feeding Cogs by Spin one byte-at-a-time is much too slow. A typical Method call consumes 100 or so microseconds. Whereas operating a Cog in a "data pull" mode is very fast and extremely responsive. Typically a cog is executing a thread at full speed in 2 microseconds from Spin putting the appropriate long in the Cog's mailbox.

Cheers,

Peter (pjv)

potatohead · 2012-02-27 21:51

Seems like LMM PASM code is a potential answer here. Compile the object, or rewrite it. Might even be worth developing something to do that in a standard way, so the connects to and from the object, and it's LMM kernel needs are consistent.

Cluso99 · 2012-02-27 22:52

pjv: Your interface sounds nice. For sure, a long can be used in a lot of ways depending is what is on each end. It is exactly this interface that I am currently interested in.

I have solved the problem of having cogs running pasm code for various character style devices. MY 1pin TV is a good example of this because it also contains a subset of VT100 code to move the cursor, scroll, etc. IIRC the font is also in hub and the screen buffer (2K) actually overlays the cog code in hub after the launch (the code is extended in hub to take the full 2,000 bytes.

potatohead: One problem of LMM is that is does take quite a lot of space compared to spin. However, for that you get speed. Provided the code is only short by a small amount in cog, then allocating hub space would be ideal for LMM. I think these requirements will fall out as I/we progress.

I am no longer concerned about loading the cogs on the fly with different objects when I want to change the output from say, pc serial, to say tv or vga.

I am actually more concerned with how to carve up the upper hub ram to contain these malibox/rendezvous locations and any larger buffers required.

Dr_Acula · 2012-02-27 22:57

pjv - that sounds a very similar protocol to the one driving the dracblade and the more recent touchscreens. Only difference is that there is no mailbox - each cog has its own mailbox created on startup when that cog is started. Hmm - originally, that code started off life as a ram driver written by .... Cluso99!

Adding a mailbox makes intercog comms possible.

potatohead - LMM probably is a part of the solution. For instance, if you start off with an "off the obex" object, and start converting the spin to pasm and find it doesn't fit in the cog, but you want the flexibility and speed of assembly, LMM would be the answer.

As I understand it, LMM has 'special' code to handle jumps. And if there is code to handle jumps, then that code could ? be tweaked so all jumps are relative rather than absolute. Is that all it would take to make the code fully relocatable??? Anywhere in hub. Anywhere in external ram too. You could even think about crazy things like putting all your objects in external ram and then pulling them into hub ram for faster execution.

potatohead · 2012-02-28 07:46

Well, don't have coffee too close to laptops. Spent most of last night taking mine apart... I've got spares, and a few hours reassembling it again before it will work. Smile.

@Dr_A That is precisely where I was headed with that. Yes, the code size is bigger, but a kernel setup for relocatable LMM code would make for modules that could be fetched and ran from anywhere, and they would be fast.

The other thought I had was to use a COG to do an interpreter for byte code that is explicitly relocatable, then compile to that, for a kind of pseudo-spin. Smaller, much slower, though not any slower than SPIN is most likely. Maybe just modify the SPIN one.

Dave Hein · 2012-02-28 08:26

It might be possible to write an LMM kernel that adds an offset to all memory accesses and jumps. It would run slightly slower than the normal kernel, but it would allow for relocation. The main issue would be distinguishing between absolute addresses and relative addresses that would need to be relocated. An LMM jump can be made relocatable to adding an offset to the PC register instead of loading it with the target address.

Dr_Acula · 2012-02-28 13:59

@Cluso, carving up upper ram should not be too hard. I guess that is the same problem as the way strings were handled in MBASIC - if you delete a value/block of values, make a note in a lookup table this is deleted. When adding a block of values, if it fits in an existing gap, fit it. If not, put it at the end. If it does not fit but there is room but it is fragmented, 'defragment' the space.

Or you can define a rigid set of rules that everyone adheres too. The only problem with rigid rules (eg all objects must have n bytes) is invariably you need an exception to the rules.

Dynamic allocation is harder to code but much more flexible - eg object 1 needs just one long for rendevous. Object 2 needs 5000 longs. It would be the same algorithm as the one used for hard disk storage.

The main issue would be distinguishing between absolute addresses and relative addresses that would need to be relocated.

Good point. If an object is self contained, then all references would be internal - eg references to a local array. If you want the object to reference outside values, pass the location of those at startup. So this way everything becomes self referenced. Jumps are relative. Arrays are relative to the start of the object. The only jump that would not be relative would be the jump at the end to exit the object. Unless one was a messy programmer who had multiple exit points...

Dave Hein · 2012-02-28 14:51

Spin is a nice implementation of a relocatable object. All the addressing is done relative to PBASE, VBASE and DBASE. The only absolute address is the final return address of $FFF9, which is located in the ROM and performs a COGSTOP(COGID). Of course, the use of the BST @@@ operator will generate an absolute address, and render the object non-relocatable.

I wonder if the Catalina and PropGCC compilers produce code that could be relocated with a modified LMM kernel that added a program offest.

Cluso99 · 2012-03-03 14:23

Dave: I would presume it would just be a change in the LMM execution code to add (signed add) to the pc (program counter) rather than to replace it. In fact it should have no penalty.

Cluso99 · 2012-03-03 14:32

I am up to the next phase. I have a version of PropCmd running that determines the hardware after booting.

Now, what I need to do, is to load 2+ cogs as follows:

To handle the StdIn routine in a cog (e.g. Keyboard, or Serial PC)
To handle the StdOut routine in a cog(s) (e.g. TV, VGA, LCD, or Serial PC)

I need to allocate a buffer for each routine, and I need to mark somewhere what cogs are used, so that when a program terminates, not all cogs are restarted, and the boot file is reloaded.

What I am thinking is that the reloaded program is a short stub program (a cut down PropCmd equiv) in spin that is resident in hub. This program is started in the existing main programs cog, and it shuts down all other cogs excepting those for the StdIn and StdOut routines. All functions of the OS (i.e. were in PropCmd, such as DIR, SPIN/RUN, etc) are all separate mini-programs.

Later, I would also want the SD access programs to remain resident in cog(s).

Any thoughts, suggestions, etc???

MagIO2 · 2012-03-03 15:25

Ok ... I know it's not done yet and maybe will not be done ever ... but maybe something is interesing for you.

The idea is to have
1. All hardware relevant drivers in upper part of EEPROM. To keep it easy each PASM driver there has a slot of 512 longs even if these are not used. One slot is reserved for a table where the content of the slots are registered by name. It also contains the number of longs needed for a mailbox.

2. A core system will load the basic drivers. First one part of the core system reserves the buffer needed for mailbox. Then the driver is loaded and registered in upper RAM. My current newCogOS contains some commands to list the running drivers and list the drivers installed in EEPROM and for starting other programs. In the end the goal would be to support an autorun.bat or a command which starts any kind of program. The program would have an *.ini - file which lists the drivers needed. The OS starts any additional driver needed and then starts the program.

3. In this case the program would simply read the list of drivers already running - especially the buffer-addresses - and simply use them.

The only thing which does not really match with what you want is, that the non PASM-part of all drivers are still proprietary and the PASM driver expects a PASM-driver with the one and only driver name. In a general standard in/standart out way the non PASM-part of the standard IO should work together with all kind of streaming device drivers. This part is where I stopped - the DriverStdCommands.spin. I think it would be nice to have a standard list of commands for the drivers like set the pin-base, set input-buffer ... and maybe read_byte, read bytes, read block, write block .....
Drivers work as usual - waiting for a command in the communication-buffer. If command is known it's executed and after execution the command is set back to 0 (=NOP) by the driver. If the command is not known, it's set to ERROR.

Cluso99 · 2012-03-03 15:49

Here is my proposed hub layout.

con
'Hub Rendezvous locations...
' _HubFree: stores the total of hub available (i.e. $7000 means $0000-$6FFF is available)
'           the top byte us set to $5A if its valid
  _HubFree      = $8000         - 1 * 4    'stores total hub avalable (see note above)
  _StdIn        =               - 1 * 4    'standard input  rendezvous
  _StdOut       =               - 1 * 4    'standard output rendezvous
  _AuxIn        =               - 1 * 4    'auxilary input  rendezvous
  _AuxOut       =               - 1 * 4    'auxilary output rendezvous
  _User1        =               - 1 * 4    'user 1          rendezvous
  _User2        =               - 1 * 4    'user 2          rendezvous
  _User3        =               - 1 * 4    'user 3          rendezvous
  _SIbuf        =               - 16       'StdIn  buffer
  _SObuf        =               - 16       'StdOut buffer
  _OS           =               ???        'Possible resident spin OS stub             
  _SDbuffer     =               - 512      'buffer                 /
  _Interpreter  =               - 2048     ' \ Cluso's Interpreter
  _I_Vector     =               - 1024     ' /

I have taken the top 4KB of hub ram. 3 large blocks are taken for...

512 byte SD card buffer (in an OS, I am accepting the fact that all programs will use the SD card in some fashion)
2KB + 1KB for my faster spin interpreter (yes, I am planning on using it)

That leaves 512 bytes for...

The top long at $7FFF is a pointer to the free hub (in my case it will point to $7000 which means $0000-6FFF hub is free for the user. If space is required to be allocated for something resident (e.g. a screen buffer) then this will be allocated from $6FFF down, and this location will be reduced by the size of the allocation. The top byte of this long will store $5A to indicate it is valid, leaving 24 bits for the hub address.
The StdIn and StdOut routines and buffers (1 long + 16 bytes each but could be more)
Some of the remaining bytes may be used for a spin OS routine (issues the command prompt, etc) to be loaded after each users program terminates. It also permits some LMM expansion possibilities for the spin interpreter.

All the above would be contained/defined in an external _hubdef.spin file so that it could be simply changed.

Cluso99 · 2012-03-03 22:32

MagIO: Thanks for the ideas. Since I have an SD card, using eeprom makes no sense to me. All this can be done quite simply on an SD card.

AUTOEXEC.BAT has been working for a looong time.

ALL:

Here is some code based on Dracs modified KeyDos3, and now with my mods. I have a hardware detection object that detects my TriBlade#2, RamBlade & RamBlade3. This should be easy for others to add into this. There is some small checking and setting done in the main program atm.

What I have atm is KyeDos3.07 which has a new command <togglelf> so that <lf> is optional so we dont get a blank line in between on PST. I added a <bin> command to display the binary files.

Now I can load between my version of PropCmd and KyeDos and ZiCog/CPM as well as run other prop programs such as Catalina C programs (yes that works too). I need to get FTF running to/from the PC and FTF from KyeDos/PropCmd to CPM (CPM to KeyDos/PropCmd is working).

Next is to split out the commands from KyeDos/PropCmd to standalone programs and then use the StdIn/Out routines and resident drivers in the cogs.

For anyone interested, here is my KyeDos3.07 version. It runs on RamBlade3, but should also run on RamBlade and TriBlade#2 hardware without change (except change the xtalfreq as I use 6.5MHz)

KyeDOS3_07 - Archive [Date 2012.03.04 Time 16.58].zip

Cluso99 · 2012-03-04 04:28

I now have Kyedos split into 3 modules.
_DOS.CMD sets up all the hub and then calls _CMD.CMD
_CMD.CMD inputs the command and then calls _xxx.CMD to execute the command.
_DIR.CMD does the DIR command and then calls _CMD.CMD

Currently the boot command wipes the top hub so I still have to fix this.

Postedit: now passing the parameters via upper hub when booting the cmd modules

MagIO2 · 2012-03-04 09:00

Cluso99 wrote: »

MagIO: Thanks for the ideas. Since I have an SD card, using eeprom makes no sense to me. All this can be done quite simply on an SD card.

I think I was already having this kind of discussion in another thread a while ago ...

You say you want to have an OS running on several hardware setups. So, for me it's simply logical to have the hardware-dependend stuff in EEPROM which simply belongs to the hardware. This way you do not even need to change an *.ini file for running the OS on your hardware. Whatever driver it finds in EEPROM which is compatible to stdout will be used ... whatever driver it finds in EEPROM which is compatible to stdin will be used.
Move the SD-card (or a ZIP-file of your OS/applications) to another system and it will run.

What happens to your OS when there is no SD-card? What happens if you allow dynamic driver reload and the SD-card is worn out?

Cluso99 · 2012-03-04 16:13

MagIO2: Good questions..

If the card fails, not present, or wears out... This is a normal problem. The user should have a backup on PC. I am preparing an FTF program ATM.

I wanted the eeprom program to be minimalistic. Perhaps now is the time to add extra config parameters for the hardware. Currently I am testing for my versions of the hardware - it is easy for me to check where the pullups are to determine my hardware. I think others can be added easily.

I wanted to be able to change my hardware on the fly. This also means changing the same hardware by removing the PC (via PropPlug) and replacing with my 1pin keyboard and TV drivers. This is a problem ATM because I cannot figure out how to determine this. I also change xtals but I don't think this is a normal user problem.

Certainly most of these items could be held in the eeprom. In fact, they could be compiled into the eeprom boot program. I guess as I progress I will encounter some of these problems and solutions.

For the main part, I did not want to have eeprom routines as well in my code. I share these pins, so it does make accessing the eeprom an issue if say a catalina xmm program is running in external sram and you need to access the eeprom.

Anyway, I have the makings to separate all the DOS commands such as DIR, BOOT, RUN, etc into separate reloadable modules (like Sphinx did) and maintain the drivers in cogs and upper hub.

MagIO2 · 2012-03-04 23:53

If the card fails, not present, or wears out... This is a normal problem. The user should have a backup on PC.

I'm not concerned about loosing the content of the SD-card. Let's say you have an application which needs to swap some drivers while running and success is mission critical. With an SD card you have more reasons for failure than with EEPROM. Missing the SD-card, corroding connector, bad contact because of vibration, waering out faster.
Some of these problems occur from startup, some might occur during runtime. But how to report the problem when you only have the OS running without any drivers? I guess it makes sense to also have a STDERR-driver which is setup-independend.
Ok ... let's say it that way ... for me it makes sense to have some drivers in EEPROM and some can also be stored on SD-card.

Yes ... I think it HAS to be that flexible ... what if you want to build some hardware having no SD card at all? Your OS should not run on those?

Hmm ... changing XTAL ... nice ...

So, if you have a look at DriverStdCommands.spin in my newCogOS.zip. This was an idea to have at least some standard commands that drivers might implement. GET_CMD_MASK would return a bitmask that shows which of the commands are implemented in a driver. SUSPEND/PAUSE and CONTINUE are meant for allowing dynamic driver exchange during runtime of an application. For example if you have a menu-driven system you might not need to have all drivers running at the same time and would rather load them on demand. SUSPEND for example would store the variables which define the inner state of a driver in HUB-RAM. The OS/application then can replace the PASM-code in the COG and come back whenever needed by loading + CONTINUE with a pointer to the saved state.
SET_PIN_BASE allows to change the pins the driver uses. Of course each driver has it's pin setting defaults, but I think it could be nice to use one serial driver for several serial interfaces ....

Replacing a full duplex with a 1pin keyboard and TV would be easy. Simply replace the full duplex PASM code with that driver and use SET_I_BUFFER/SET_O_BUFFER or SET_IO_BUFFER to attach it to the same buffers used before by full duplex.

According to your requirement it surely makes sense to have a SET_XTAL as well.

Hmm ... for the drivers ... I started to dream of an application where you can have parametrized driver code. You simply select a driver and an input-mask opens which allows you to set the default parameters for your current hardware setup. Clicking a button would compile the driver with these settings and send it to the OS. The OS stores it in EEPROM or on SD-card. With just a few clicks you can prepare the setup of a board. For standard boards there will be ready to run configurations... Maybe I should add this to my PropellerHub ;o)

Cluso99 · 2012-03-05 00:43

This OS is too complex for eeprom IMHO. But you have some good ideas that are relevant and can be used by both systems.

The xtal cannot be changed on the fly of course. But, it would require that the OS be made aware that on the next boot the xtal would change. I think this is best forgotten because I don't really see a normal user doing this. So, it would make sense for this to be a stored parameter. The same applies to pinouts. I would not expect an SD card to change pins from day to day, although when I plug my sd card into different hardware it does. But I can sense this anyway.

I am interested in some file method (could be on eeprom or sd) where the hardware configuration is specified. For instance, I could see where the TV or VGA could share the same pins. While once again you can detect this, with different hardware this becomes a bigger problem.

Another point I have conceeded is that the boot program in eeprom only knew about the sd pins and would load a boot program from there. I now have added conditional compilation (using bst) to send a message to the serial port (P30/31) if it fails. I think this is a reasonable assumption although my hardware could have 1pin tv and kbd on these pins - but I believe I need to know this congfiguration as I don't see how I can readily test for this (or other options I am planning for these pins).

I agree that it would be good for drivers to be able to be loaded multiple times and just set with new buffer(s) and pins. I am against pin_bases unless there is also a method to set the individual pins too. For say an SD driver, while both are already provided, a simple way is to pack 4 bytes with each pin number. If the long <32 (or <128 for P2 or P1B) then it is a pin base. Otherwise, it is 4 8bit pin numbers. Easy to pass, simple to code, and only 1 call.

Your Set_i_buffer /o/io commands are exactly what I am after, and is similar to how mpark did in Sphinx and I extended in my 1pin drivers. There is an input, output and inputoutput spin driver that can interface to any other pasm character device driver inclusing serial. This makes sense because the actual user program no longer needs to know about the physical device in most circumstances.

So, my parttime progression is to work out a standard hub definition that I can live with. Next is to work out a file definition which can hold the drivers, pinouts, etc like a windoze ini file but simpler of course.

BTW mpark has written a compiler for the prop hosted in Sphinx. It is my dream to get this fully functional within this os.

MagIO2 · 2012-03-05 01:35

The XTAL can be changed ... well ... not the piece of hardware, but you could switch from XTAL to internal clock which means that you might want to change some of the drivers timings so that they still run with the max. possible speed.

Cluso99 · 2012-03-05 02:29

The problem with the xtal being specified incorrectly is that any serial device will not work correctly. It is possible to time the input baud but that requires a character to be sent and it must be a known character(s) like the AT in the modem command set. BTW my xtals are all socketed.

MagIO2 · 2012-03-05 03:00

I understand what you mean. What I mean is, that you might have an application running which want's to switch the clock from XTAL to internal clock on the fly to save energy, but also wants to keep this or that device operational. So, the standard command SET_XTAL (or name it CHANGE_CLOCK) could be used and the driver re-calculates the timings.

Cluso's Propeller OS V1.14 - now with SPIN/PASM Compiler, EEPROM Read/Write :)

Comments