A very essential and useful function is missing: a system timer
Due to the fact that the ZOG speed is not so clear, there has to be a possibility to the C-program to synchronize on a timer.
Therefore a function like
#define _getSysTime 0x-- // gets the system time in uSeconds
Bill: Yep, a little program with sprintf ("Something\n"); is only about 4KBytes changing that to sprintf ("Something %d\n", some_int); blows it up to 43KBytes !!
Bill: Silly me, the answer has been staring in the face all the time. The ZyLin ZPU emulator in Java defines a SYSCALL instruction (0x3C) which as you might guess does some kind of syscall.something in Java. I have allowed for SYSCALL in Zog and was wondering what to do with it !
SYSCALL is not defined as an instruction in the documentation it just appears in the simulator. As does CONGIG (0x3A).
Now the Java syscall thing is used to notify the simulator of important things like: syscall.halted(), syscall.doneContinue(), etc etc. Strangely enough it is also used for syscall.writeUART(val) and syscall.readUART(val) after some code has intercepted the memory access addresses.
So there we have it, we can borrow SYSCALL. It may not be defined but it exists in the simulator so I guess the ZPU guys are unlikely to define it for something else in the future. Here is a whole world of possibilities.
Jazzed: The soft interrupt idea from BIOS and DOS only came about because Intel provided software interrupts in the 8086 processors. This meant you could use interrupt numbers to get into OS functions rather than relying on a jump table at some fixed address or whatever. ZPU has no such equivalent unfortunately.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.
Chris Micro: Fear not. The ZPU C run time system out of the box supports very minimal hardware, UART in, UART out and a TIMER. Enough for debugging and getting results out f the Dhrystone bench mark [noparse]:)[/noparse]
For sure I'd like to expand on that. All features of the Prop, timers, counters, pins, locks, HUB RAM should be accessible from Zog. Not sure if there is much point in having access to the video hardware but still.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.
This would allow access from ZOG to the special cog registers, without requiring C/Fortran/etc extensions.
C pushes arguments right to left, so the system vector would always be in TOS when the SYSTEM instruction was invoked, with arguments handily stacked up.
Chris Micro said...
Wow, unbelivable how fast the work is going on
A very essential and useful function is missing: a system timer
Due to the fact that the ZOG speed is not so clear, there has to be a possibility to the C-program to synchronize on a timer.
Therefore a function like
#define _getSysTime 0x-- // gets the system time in uSeconds
see the additional examples I added. Very handy, very simple interface. The convention should be that SYSCALL (aka SYSTEM) always pushes a return value onto the stack. Glad to see the Java implementation was using basically this method, did not know that.
re/ software interrupts... rename CONGIG (0x3A) as INT, have it take arguments like SYSTEM.
What is the difference then?
System goes "behind the curtain", and causes Pasm code or messaging.
INT goes to a function dispatch table, and calls a ZOG function.
I'd suggest something like the following prototype be standard for software interrupt handlers:
int int_xx(argc, **argv)
and in ZOG assembly:
<load argv pointer>
<load argc>
IM intnum
INT
Yes, it does mean that a minimum int call is 4 bytes
IM 0
IM 0
IM intnum
INT
BUT it allows a nice generic interface, with an arbitrary number of pointers passed in.
SYSTEM would still have a variable number of arguments, but it goes to PASM code or sends messages, where INT just dispatches through a table of 128 entries (0..127)
A single SYSTEM call is needed for C to set up handlers:
setint(vector, &inthandler)
The beauty of this approach is that it is possible to change the interrupt handlers at will, during run time!
Theoretically it could be used to support DLL's as well.
I think the combination of SYS and INT is a winner [noparse]:)[/noparse]
Also... removing support for floating point would probably shrink formatted output routines big time.
heater said...
Bill: Yep, a little program with sprintf ("Something\n"); is only about 4KBytes changing that to sprintf ("Something %d\n", some_int); blows it up to 43KBytes !!
Bill: Silly me, the answer has been staring in the face all the time. The ZyLin ZPU emulator in Java defines a SYSCALL instruction (0x3C) which as you might guess does some kind of syscall.something in Java. I have allowed for SYSCALL in Zog and was wondering what to do with it !
SYSCALL is not defined as an instruction in the documentation it just appears in the simulator. As does CONGIG (0x3A).
Now the Java syscall thing is used to notify the simulator of important things like: syscall.halted(), syscall.doneContinue(), etc etc. Strangely enough it is also used for syscall.writeUART(val) and syscall.readUART(val) after some code has intercepted the memory access addresses.
So there we have it, we can borrow SYSCALL. It may not be defined but it exists in the simulator so I guess the ZPU guys are unlikely to define it for something else in the future. Here is a whole world of possibilities.
Jazzed: The soft interrupt idea from BIOS and DOS only came about because Intel provided software interrupts in the 8086 processors. This meant you could use interrupt numbers to get into OS functions rather than relying on a jump table at some fixed address or whatever. ZPU has no such equivalent unfortunately.
Bill: You are racing away with ideas there[noparse]:)[/noparse]
I think I'd like whatever happens with SYSCALL to remain compatible with whatever the ZPU currently does with it. For example in the Java ZPU simulator. It should be flexible enough already that we can "piggyback" more functionality onto it.
I'm a bit lost as to how/if SYSCALL is used in ZPU and it's simulator. In the provided GCC setup there is a thing called libgloss providing I/O stuff. In there is a comment that syscalls are only used in the so called "Zeta" platform for ZPU. Thing is the Zeta platform is supposed to be obsolete! Without syscalls the libgloss ignores most file I/O stuff and sends standard I/O straight to the UART.
Perhaps the SYSCALL instruction is a totally free opcode nowadays.
I'm asking on the ZPU mailing list. They seem quite keen to advise there.
Yep, adding a single floating point multiply to the test program blows it up by 4000 bytes withou any I/O.
Lots of room for pruning there I'm sure.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.
In my opinion the interface routines should be kept as minimalistic as possible.
If there are to many propeller specific things inside it is hardly possible to transfer the whole system to other miicrocontrollers.
Therefore I vote for more abstract "Bios" functions.
like:
writeByte serial
readByte serial
write32bitPortpins <- abstract, because on every processor architecture this pins will be mapped their IOs
read32bitPortpins
readByteMassMemory
writeByteMassMemory
...
Chris, I'm all for simple. We should support standard C interfaces as much as possible stdin, stdout, file I/O etc.
After that things get a bit more tricky.
For example when you say "write32bitPortpins" that actually comes down to writing to a register. OUTA in our case. There are many such registers in the Prop (and other micros). It comes down to providing a mechanism to enable reading/writing those registers from Zog Code.
Now if you or anyone else has abstracted that into write32bitPortpins() and read32bitPortpins() functions in your application code it becomes a simple matter to implement them when moving your code to the Prop, using the supplied mechanism.
In general these things get abstracted away by operating systems, even very simple ones. So serial I/O, for example may become a case of opening a device, say "/dev/tty2", then using read() and write() on it as if it was a file. When dealing with an VM like Zog the important thing is to expose the underlying hardware in such a way that any such abstractions can be used on them. Including the Chris Micro Abstraction Layer.
Then we might want to think about things the Prop can do that other micros cannot, like cogstart.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.
@heater, as requested, attached is a variation of the OBEX C VGA package.
It may not be immediately clear, but cognew_native is used to start a PASM COG on Propeller accessible via HUB variable interface and the code being used is the VGA.spin PASM binary.
As I mentioned, there are function stubs for cognew/cogstop. The wordfill/wordmove methods are kernel services in ICC for better performance; some macro versions using memset/memcpy are included, but untested. Non-standard library functions wait and itoa need separate implementation (propeller.h provides msleep using waitcnt for example).
Ok Jazzed. Thanks. Well it does compile for me at least with a couple of warnings. Glad to see that Makefile came in useful.
Very interesting, let's see if I understand this correctly:
1) This C code runs in HUB (ICC or Catalina?) perhaps with it's own address space i.e. C addresses are not HUB addresses.
2) VGA_Text_Array is a binary blob of PASM code that is the VGA driver. It sits in C address space.
3) gVgaText is an array of parameters for the VGA drive to pass through PAR when it is started.
4) cognew_native has to some how cause the kernel or Zog VM to execute a COGINIT with those as parameters.
5) gscreen and gcolors are the screen buffer, referenced from gVgaText.
6) When everything is up and running the C code interacts with the driver through gscreen and gcolors only.
All of this seems eminently doable from the Zog VM. Lets say we used the SYSCALL instruction. The VM could trap the SYSCALL, determine that a COGINIT is required, grab the VGA_Text_Array and gVgaText address parameters off the stack and use them in COGINIT.
A little complication is that the address space the Zog VM is working in is not the HUB address space so the ZPU memory base address would have to be added to the addresses to translate them to HUB addresses for COGINIT. No problem. (Hmm what about the gscreen and gcolors pointers? How do they can adjusted? Zog knows nothing of them or any other content of the two COGINIT parameters it has been given)
This all breaks down when we move Zog to external memory. Or at least gets a lot more complicated.
Now the SYSCALL has to:
1) Copy the binary PASM blob to some place in HUB that's free so that COGINIT can find it.
2) Copy the gVgaText parameter array to some place in HUB that's free so that COGINIT can find it.
3) Now it needs a way to have direct access to HUB RAM, which is not in it's normal external address space, so that it can write to gscreen and gcolors. How does it know where they live in HUB now?
4) Even less sure how the pointers to gscreen and gcolors get fixed up in this case.
Hmm... I'll ponder this some more.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.
Look at my proposed SYSCALL extensions... which included hub read/write for byte/word/long, coginit etc
Currently VMCOG is passed a top address, and number of 256 byte pages it can use, growing down from there. Easy to use hub memory below that for a "cog loader buffer" (2k) and the frame/color buffers.
Note the VMCOG API includes a call that when given a virtual address, returns the hub address that the page containing that virtual address is in.
vm.GetPhysVirt(vaddr) returns the base of the hub page containing the page vaddr is in.
I am considering adding two new calls:
vm.Lock(v_base_page,num_pages) ' allocate a contigous buffer for hardware use - disk/screen/cog buffer, may not be swapped
vm.Unlock(v_base_page,num_pages) ' release locked pages back into the general working set pool, may be swapped
To use this to make a cog loader buffer (2k)
rc := vm.Lock($F000,8) ' page in eight contigous 256 byte pages, and lock them in-core starting at $F000 virtual address
hubaddr := vm.GetPhysVirt($F000) ' return the hub address of the continous locked in memory 2K block
Now an i2c or spi function could be used to load the driver
i2c.read(eepromaddr,hubaddr,2048) ' load a driver from eeprom
Then, using SYSCALL and the extensions I suggested,
cognew(hubaddr,hubpar)
After the cognew completes (I recommend waiting 1ms)
vm.Unlock($F000,8) ' release the cog buffer
If I add Lock and Unlock, it would be possible to manage basically all the hub memory via VMCOG, allocating and de-allocating it in 256 byte pages on the fly as needed, so allocating screen buffers also becomes easy. So does allocating/deallocating disk buffers.
Lock would fail (and return -1) if it could not lock the requested number of pages (ie there are only 16 pages available in the working set, and the program wants to lock 32 pages - can't do it. I would have lock fail if at least 8 pages were not left unlocked)
This also allows for changing drivers on the fly for TV or VGA, and resizing screen buffers, file buffers etc...
I am now leaning to having VMCOG manage all hub memory, except for the SphinxOS top pages, and the Largos required first two pages ($0000-$001FF), allowing VMCOG to manage 123 of the possible 128 pages (of 256 bytes).
vm.IsLocked(vpage) could be added to test if a page is locked or not.
heater said...
Ok Jazzed. Thanks. Well it does compile for me at least with a couple of warnings. Glad to see that Makefile came in useful.
Very interesting, let's see if I understand this correctly:
1) This C code runs in HUB (ICC or Catalina?) perhaps with it's own address space i.e. C addresses are not HUB addresses.
2) VGA_Text_Array is a binary blob of PASM code that is the VGA driver. It sits in C address space.
3) gVgaText is an array of parameters for the VGA drive to pass through PAR when it is started.
4) cognew_native has to some how cause the kernel or Zog VM to execute a COGINIT with those as parameters.
5) gscreen and gcolors are the screen buffer, referenced from gVgaText.
6) When everything is up and running the C code interacts with the driver through gscreen and gcolors only.
All of this seems eminently doable from the Zog VM. Lets say we used the SYSCALL instruction. The VM could trap the SYSCALL, determine that a COGINIT is required, grab the VGA_Text_Array and gVgaText address parameters off the stack and use them in COGINIT.
A little complication is that the address space the Zog VM is working in is not the HUB address space so the ZPU memory base address would have to be added to the addresses to translate them to HUB addresses for COGINIT. No problem. (Hmm what about the gscreen and gcolors pointers? How do they can adjusted? Zog knows nothing of them or any other content of the two COGINIT parameters it has been given)
This all breaks down when we move Zog to external memory. Or at least gets a lot more complicated.
Now the SYSCALL has to:
1) Copy the binary PASM blob to some place in HUB that's free so that COGINIT can find it.
2) Copy the gVgaText parameter array to some place in HUB that's free so that COGINIT can find it.
3) Now it needs a way to have direct access to HUB RAM, which is not in it's normal external address space, so that it can write to gscreen and gcolors. How does it know where they live in HUB now?
4) Even less sure how the pointers to gscreen and gcolors get fixed up in this case.
Bill, I was just looking over the syscall code in the GCC libraries that come with Zog. Suddenly I realized I was confusing two issues.
Firstly an operating system, like Linux, defines a syscall interface as a way of getting in and out of kernel space but ultimately as a way of providing services like file I/O etc. One could imagine getting ucLinux running with Zog and uCLinux would provide the syscalls in "normal " C code.
Secondly, there is the ZPU SYSCALL instruction that hyperspaces you out of the virtual machine into whatever system is running it.
So lets say we build a FAT file system from the C FAT code we have available for Zog.
At the first level the application makes a syscall into the "operating system" (Perhaps just the C library)
That in turn calls on the FAT code modules to open/close/read/write files.
That, in Zog, may well use the SYSCALL instruction to get something outside the virtual machine to do the actual block reads from SD card or whatever.
Looks like Virtual Memory is the key to the HUB/External RAM space problems.
What you are basically saying is that with VM all memory is the same, HUB or EXT. It's all on the other side of the VM as far as Zog is concerned. Only thing is to be able to lock certain pages to HUB so that the rest of the Prop can see them. Cool.
So can we get Zog to "malloc" a bunch of RAM. Pull one of those PASM drivers of an SD card into that RAM, lock it into HUB space and the start the driver?
Also looks like we are not going to need Linux on the Prop when you have finished[noparse]:)[/noparse]
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.
Ok, you are officially unconfused! Yep, my idea is to make all memory look the same to virtual machines. Really VM is not my idea - I am just moving a "mainframe" (although it is in PC's now) style virtual memory manager onto the Prop. My idea just moves it to the prop (VMCOG), accessible through messaging [noparse]:)[/noparse]
Heck, the old mainframes (and other newer OS's) had similar mechanisms for locking pages for DMA.
Which is why I was proposing, using SYSTEM (which is why i named it different from SYSCALL) as an escape hatch for running PASM/LMM/SPIN code, either directly, or by sending messages to another cog!
If you re-read my long emails, it will now make sense!!!!
And my last long email adds Lock and Unlock, allowing for creating (and re-sizing!) video buffers, disk buffers, cog buffers on the fly, and releasing them when not needed!
Yep, you re-stated how I would load a driver in the previous message, except I used an i2c example.
LOL
actually Largos is intended to be (at the user level) very similar to v7 Unix.
I am still toying with what names to call locking / unlocking pages....
I'd lock it first, but certainly it would also work to lock it after loading, but exactly right.
Lock a page range big enough, load driver, launch cog, release range.
For video buffers, just lock a big enough range, and get its physical address. Ditto for extra file buffers (but file buffers would actually work without locking, and then could potentially swap out when infrequently used)
heater said...
That, in Zog, may well use the SYSCALL instruction to get something outside the virtual machine to do the actual block reads from SD card or whatever.
Looks like Virtual Memory is the key to the HUB/External RAM space problems.
What you are basically saying is that with VM all memory is the same, HUB or EXT. It's all on the other side of the VM as far as Zog is concerned. Only thing is to be able to lock certain pages to HUB so that the rest of the Prop can see them. Cool.
So can we get Zog to "malloc" a bunch of RAM. Pull one of those PASM drivers of an SD card into that RAM, lock it into HUB space and the start the driver?
Also looks like we are not going to need Linux on the Prop when you have finished[noparse]:)[/noparse]
heater said...
Well it does compile for me at least with a couple of warnings. Glad to see that Makefile came in useful.
Yes, it is a fine and very familiar Makefile example.
You understand the ideas mostly. Some clarifications are in order though.
The ICC C code is LMM interpreted. C .text, .data, .bss, etc... addresses are all HUB addresses in ICC.
I have used the same code with slight compiler/kernel modifications from Richard to do the same methodology fetching from and in XMM.
In ICC, cognew_native executes the PASM coginit instruction normally (no special sauce). The HUB address of the C code BLOB and the parameters are given to coginit. PASM coginit grabs the COG code from HUB regardless of where coginit is called (spin interpreter in a cog or straight PASM in a cog) not a problem.
Since all pointers defined in C space are used by the ICC LMM kerenel, there is nothing complicated about it there ... I'm not sure what would be different for ZOG since I have not examined your design carefully enough (and would rather leave that to you).
heater said...
All of this seems eminently doable from the Zog VM. Lets say we used the SYSCALL instruction. The VM could trap the SYSCALL, determine that a COGINIT is required, grab the VGA_Text_Array and gVgaText address parameters off the stack and use them in COGINIT.
...
Clearly, this is why I posted the idea to the thread [noparse]:)[/noparse] If ICC can do it, ZOG should have no problem.
I figured I would share since I've been down this road before, it is a quick and mostly appreciated solution, and the code is in the OBEX (some enhancements will be required obviously to make the code ZOG compatible unless ZOG implements ICC interfaces which probably won't happen). Ross has provided a compatibility header for ICC and Catalina so that it is easier to share/migrate code between the two compilers. Having a good ANSI compatible, cross platform library with a HAL would be best though (POSIX and ioctl would be even better, but all that would probably require tons of overhead which puts it out of budget).
Guess I've known about virtual memory for a long time now. Just took it for granted without thinking it through much.
I think the next step for Zog is to get it to work with VMCog. Just 16K (or more) of HUB RAM initially and then my 512K external RAM. Think I said this before. We have crude single stepping and console output, enough to test it works.
Meanwhile think about all this I/O, SYSCALL stuff some more. I want to find out more about how the ZPU C libs are put together and how they interact with real ZPU hardware platforms and the simulator. There are some little tricks going on with the later.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.
Jazzed. Being able to make use of C objects in OBEX is a worthy goal. Especially if they are already usable in both ICC and Catalina.
What is different in Zog is that it works (currently) within a 16Kb area of HUB RAM. The C code only has access to memory from 0 to 3FFF. The VM will actually roll over addresses if the C code tries to go out of range.
Well that 16K of space is actually sitting in HUB at whatever address BST put it at when the Spin code was compiled.
So address 0000 in C is not zero HUB.
Hence passing those addresses directly to coginit will not work, they at least need translating back to HUB addresses.
When we move into external memory this all gets more complicated. But it looks like Bill is many steps ahead of us there [noparse]:)[/noparse]
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.
Of course, I'm not pushing any solution as preferred and have no bias affected by NIH syndrome. I'm just offering possibilities in one working model.
Catalina uses a different model with built-in drivers and would probably never want to use the OBEX C driver code. There are some advantages to this as I said before. If ZOG C memory is not adjustable to match HUB addresses by some means, the whole idea of using the current OBEX ICC driver model is a waste of time for ZOG.
Bill's efforts implementing VM ideas that have been discussed several ways in the forum will of course be very useful to many of us one way or another, and I look forward to the result. Of course I also patiently look forward to Largos ... maybe this summer's Parallax nerd-fest will be a good podium for an update on that.
heater said...
Jazzed. Being able to make use of C objects in OBEX is a worthy goal. Especially if they are already usable in both ICC and Catalina.
What is different in Zog is that it works (currently) within a 16Kb area of HUB RAM. The C code only has access to memory from 0 to 3FFF. The VM will actually roll over addresses if the C code tries to go out of range.
Well that 16K of space is actually sitting in HUB at whatever address BST put it at when the Spin code was compiled.
So address 0000 in C is not zero HUB.
Hence passing those addresses directly to coginit will not work, they at least need translating back to HUB addresses.
When we move into external memory this all gets more complicated. But it looks like Bill is many steps ahead of us there [noparse]:)[/noparse]
I've been looking at the ZPU SYSCALL instruction some more. SYSCALL is not documented in the ZPU instruction set so it took a bit of poking around in the ZPU Java Simulator and the ZPU C libs to find out what happens.
Don't forget what we were originally looking for is a way for Zog to do I/O. The ZPU architecture uses memory mapped I/O devices which we don't like much in the Zog Virtual Machine as it requires checking all memory accesses for I/O addresses which is a bit of a performance hit.
Turns out the ZPU SYSCALL instruction, intended to be used only with ZPU simulators I guess, probably allows everything we want and is very simple to use.
So basically it works like this:
1) In C you might want to call some standard I/O function like "write (fd, buf, nbytes)".
2) In the C library that call comes to a system call for the OS to handle the write: "result=_syscall(&t, SYS_write, fd, buf, nbytes);"
Here the &t is used for returning errno, SYS_write is just a number the means "write" then we have the "write" parameters.
3) In the C lib for ZPU the _syscall function is handled in assembler and simply executes a SYSCALL ZPU instruction.
(Provided you are not running on real hardware FPGA that has no SYSCALL)
All of a sudden the all the info required to perform the write is on the stack and the ZPU virtual machine can take care of it in PASM or Spin and the proceed to the next instruction.
So there it is, we can do all normal stdio, and file handling stuff easily via SYSCALL.
So what about all those other Prop specific goodies like access to timers, pins, cogint, cogstop etc?
Well quick and dirty is that we can always add some SYS_**** function numbers and define whatever other syscall parameters we like. So for example:
Initially I think I like the "quick and dirty approach".
Yep, Bill has said all this previously, I'm just pointing out how well it fits into the existing ZPU and the supplied C libs for GCC. We can live without any IO instructions in Zog and without any memory mapped virtual peripherals.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.
Jazzed, "You have room for open, read, write, close, and ioctl?"
I don't need any room in Zog itself to do most of this.
For example, a simple approach would be to have open, close, read, write calls pass through the described SYSCALL mechanism at the end of which they just get passed to fsrw. There we have our file system established with very little "glue" code in Zog.
Now we could inspect the path strings used for open and look for things that are not files but devices. So for example our "Virtual File System" may have: "/dev/uart". Opening that gets you a handle for the UART and all subsequent read/writes for that handle get directed to FullDuplexSerial or whatever.
Not sure what we would do with ioctl yet.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.
I have been pondering the future direction of Zog. The question comes down to:
Zog OS or not Zog OS?
Let me elaborate:
Catalina and ICC not only provide the capability of running C code they also carry along with them a whole run time environment complete with drivers, "plug-ins", loaders etc etc. Pretty much they are their own operating systems.
Currently Zog enables one to run C code but is pretty much a regular standalone Propeller object, like fswr or FDX etc, that can be dropped into any normal Propeller project. without any impact on the existing project.
Now any C program needs a fairly well defined environment to run in, to supply standard IO, file handling etc. So adding that environment naturally starts to grow the system into an operating system.
Then we have the issue of enabling C code to make use of Prop specific feature like pins, timers, coginit etc.
You start to see why ICC and Catalina have grown up to be mini-operating systems that take over the whole Propeller.
Now I'm inclined to think Zog should not do that. It should remain a small, simple, self contained object.
So for example if you feel that your latest all signing all dancing Propeller creation would be all the better if it had a big blob of C code to manage it or provide a user interface logic or whatever. Then you can throw in the Zog object and point it at some C binary in SPI RAM or SPI FLASH or such, add a few lines of "glue" Spin to connect Zog to your app and there it is. With no impact on the rest of your existing creation.
The reasons I say this are:
1) Catalina and ICC already exist and do the total environment thing very well. I don't want to re-invent the wheel.
2) Seems to me Zog should keep itself small and simple and adaptable to existing applications and operating systems.
I have not really looked at any of the Propeller OS or DOS systems. But it seems to me that Zog should be applicable and adaptable to all of them.
3) My itch right now is to get Zog to run large C codes from the external RAM on TriBlade/RamBlade/DracBlade etc. The fascination for me is the simple task of getting GNU compiled code to work nicely on the Prop.
4) I'm lazy [noparse]:)[/noparse] If people have already built operating system infra-structure for the Prop I'd rather just bolt onto that than recreate it myself.
5) Time, as always, is short.
What does everybody think about this?
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.
I am going to reply to a bunch of messages here...
1) Looks like the Java SYSCALL is basically identical to what I proposed, only difference being returning the status to a location pointed to by a pointer vs. leaving it on the top of the stack. It would work great [noparse]:)[/noparse] and I believe it is the right way to go.
2) re/ standard unix-like open/read/write/close/ioctl ... like heater, I feel they should be implemented via SYSCALL, if needed, for an application
3) re/ ZogOS vs. ZogObject
I think both will live long and prosper, because they address two separate markets and while I can see why Heater is leaning towards Zog/Object, I can definitely see the advantages to Zog/OS
Right now, I think what is needed is to get Zog/Object running, with VMCOG and XMM solutions. I think VMCOG will be the "common denominator" low-cost version, and that over time, people will implement higher performance XMM for different platforms (like Triblade, Morpheus etc) - which I think is a VERY good thing, because it will allow spanning the spectrum from extremely inexpensive two SPI RAM solutions to Ramblade/Triblade with up to 1MB on one blade, to Morpheus with up to 16Mb... or even 128MB with mctrivia's modules - not to mention other upcoming platforms.
If Zog works half as well as I think it will, I may very well decide to make a Largos/Zog at a later date, as it would save me a ton of work... stringutils, binutils etc should just compile and run!
I know Largos is taking a long time... but frankly, I need to spend most of my time on revenue generating projects, or potentially decent revenue generating projects as I don't have a "day job".
heater said...
Jazzed, "You have room for open, read, write, close, and ioctl?"
I don't need any room in Zog itself to do most of this.
...
Not sure what we would do with ioctl yet.
I was referring to the overhead however small in using the driver model's functions rather than the implementation of them in the ZOG ... still a driver has to define some functions based on the device needs and could be done in either code domain. Function ioctl can get very ugly, but I've seen it used basically to abstract a "SYSCALL" mechanism (*) ... not that you want to do that [noparse]:)[/noparse]
I'm not sure ICC is a mini-operating system ... perhaps this requires a little study for a good understanding. In short however, you build, download, and run. The code does what you tell it to do with the libraries you choose. Catalina has a slightly different, slightly more complex, and slightly smaller resultant library model where the devices are chosen by the compiler/linker rather than by specifying big globs that the user provides.
Added: * i.e. details not handled by open/read/write such as modes for char/buffer devices that may need to be changed after open.
Bill: "I can definitely see the advantages to Zog/OS"
Well yes, but if you happen to have an OS lying around (largos) then I'd rather make use of that[noparse]:)[/noparse]
There is no need for yet another operating system.
I guess the syscall is designed to return a status to let you know if the syscall fails, unknown sys ID parameter say. Then it also returns a status from whatever the particular operation is, read, write etc, through the first pointer parameter.
VMCOG and XMM for Zog is next up. Well, after I've fixed up the SYSCALL console I/O.
Hmm... For some reason printf() does not work. print(), putnum(), snprintf() all work. Odd.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.
I do have a prototype Largos... and I am considering re-targeting it to Zog instead of LMM, in order to run larger programs and have all the GCC toolchain and goodies readily available [noparse]:)[/noparse]
I'd suggest that all SYSCALL's return a value on the stack as follows:
0 = OK
anything else, depending on the function called, could be interpreted as an integer, or a pointer, in case an elaborate error result needs to be returned
Very odd about printf.
heater said...
Bill: "I can definitely see the advantages to Zog/OS"
Well yes, but if you happen to have an OS lying around (largos) then I'd rather make use of that[noparse]:)[/noparse]
There is no need for yet another operating system.
I guess the syscall is designed to return a status to let you know if the syscall fails, unknown sys ID parameter say. Then it also returns a status from whatever the particular operation is, read, write etc, through the first pointer parameter.
VMCOG and XMM for Zog is next up. Well, after I've fixed up the SYSCALL console I/O.
Hmm... For some reason printf() does not work. print(), putnum(), snprintf() all work. Odd.
I think Largos/LMM2 will be very very happy on Prop2
I think Largos/Zog would be happier on Prop1, and would show up earlier than Largos/LMM on Prop1 [noparse]:)[/noparse] due to having to re-invent, re-code, and re-test far less wheels...
I don't intend to abandon Largos/LMM by any means, but I am thinking of going for the "low hanging fruit" first to get it out faster.
Also... Largos is VM agnostic, each file is tagged by a file type - LMM and ZOG programs could happily co-exist on the file system [noparse]:)[/noparse] allowing for LMM speed for more critical programs, and memory savings with ZOG on less critical programs.
Basically, I was inspired by the early PDP-11 Unix work, and what Tannenbaum got running on a small model x86, combined with advances from Plan9 and Hurd; resulting in a message passing extremely light weight design. Realistically, Largos (LMM version) needs at least 256KB of XMM, however I could see a Largos (ZOG... hmm... Zargos?) running fairly decently with 2 SPI Ram's and an SD card, using VMCOG. Certainly running as well (and probably better) than the early PDP-7 and PDP-11 versions of Unix.
Largos will really shine on Prop2, and conceptually, could be grown to support the GCC toolchain "natively" with LMM2, with a 32 bit virtual address space.
But just in case... I've started a paper design for a VLMM that might give Prop1 a fighting chance with SPI RAM's and an SD card... and Catalina would allow for a lot of Unix code to be just recompiled (given sufficient libc and system infrastructure support)
Largos sounds totally audacious, cat cd du df ls rm rmdir mv mkdir mkfs pwd touch, can't wait. So if Zog gets us there quicker that's excellent. Many things will be very happy on Prop II, I'm not very good at waiting though.
Don't forget:
chmod +x:ZiCog CPM
[noparse]:)[/noparse]
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.
Comments
A very essential and useful function is missing: a system timer
Due to the fact that the ZOG speed is not so clear, there has to be a possibility to the C-program to synchronize on a timer.
Therefore a function like
#define _getSysTime 0x-- // gets the system time in uSeconds
is strongly recommended.
Bill: Silly me, the answer has been staring in the face all the time. The ZyLin ZPU emulator in Java defines a SYSCALL instruction (0x3C) which as you might guess does some kind of syscall.something in Java. I have allowed for SYSCALL in Zog and was wondering what to do with it !
SYSCALL is not defined as an instruction in the documentation it just appears in the simulator. As does CONGIG (0x3A).
Now the Java syscall thing is used to notify the simulator of important things like: syscall.halted(), syscall.doneContinue(), etc etc. Strangely enough it is also used for syscall.writeUART(val) and syscall.readUART(val) after some code has intercepted the memory access addresses.
So there we have it, we can borrow SYSCALL. It may not be defined but it exists in the simulator so I guess the ZPU guys are unlikely to define it for something else in the future. Here is a whole world of possibilities.
Jazzed: The soft interrupt idea from BIOS and DOS only came about because Intel provided software interrupts in the 8086 processors. This meant you could use interrupt numbers to get into OS functions rather than relying on a jump table at some fixed address or whatever. ZPU has no such equivalent unfortunately.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.
For sure I'd like to expand on that. All features of the Prop, timers, counters, pins, locks, HUB RAM should be accessible from Zog. Not sure if there is much point in having access to the video hardware but still.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.
That was not meant to be an exhaustive list, just a starting point [noparse]:)[/noparse]
Since then, I think it would also be very useful to add
#define _cogread 0x07
#define _cogwrite 0x08
for raw reading/writing of cog registers.
Dangerous? YES
Useful? Incredibly!
#define cnt() system(_cogread,0x1F1)
#define getdira() system(_cogread,0x1F6)
#define setdira(bits) system(_cogwrite,0x1F6,bits)
This would allow access from ZOG to the special cog registers, without requiring C/Fortran/etc extensions.
C pushes arguments right to left, so the system vector would always be in TOS when the SYSTEM instruction was invoked, with arguments handily stacked up.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
www.mikronauts.com E-mail: mikronauts _at_ gmail _dot_ com 5.0" VGA LCD in stock!
Morpheus dual Prop SBC w/ 512KB kit $119.95, Mem+2MB memory/IO kit $89.95, both kits $189.95 SerPlug $9.95
Propteus and Proteus for Propeller prototyping 6.250MHz custom Crystals run Propellers at 100MHz
Las - Large model assembler Largos - upcoming nano operating system
re/ software interrupts... rename CONGIG (0x3A) as INT, have it take arguments like SYSTEM.
What is the difference then?
System goes "behind the curtain", and causes Pasm code or messaging.
INT goes to a function dispatch table, and calls a ZOG function.
I'd suggest something like the following prototype be standard for software interrupt handlers:
int int_xx(argc, **argv)
and in ZOG assembly:
<load argv pointer>
<load argc>
IM intnum
INT
Yes, it does mean that a minimum int call is 4 bytes
IM 0
IM 0
IM intnum
INT
BUT it allows a nice generic interface, with an arbitrary number of pointers passed in.
SYSTEM would still have a variable number of arguments, but it goes to PASM code or sends messages, where INT just dispatches through a table of 128 entries (0..127)
A single SYSTEM call is needed for C to set up handlers:
setint(vector, &inthandler)
The beauty of this approach is that it is possible to change the interrupt handlers at will, during run time!
Theoretically it could be used to support DLL's as well.
I think the combination of SYS and INT is a winner [noparse]:)[/noparse]
Also... removing support for floating point would probably shrink formatted output routines big time.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
www.mikronauts.com E-mail: mikronauts _at_ gmail _dot_ com 5.0" VGA LCD in stock!
Morpheus dual Prop SBC w/ 512KB kit $119.95, Mem+2MB memory/IO kit $89.95, both kits $189.95 SerPlug $9.95
Propteus and Proteus for Propeller prototyping 6.250MHz custom Crystals run Propellers at 100MHz
Las - Large model assembler Largos - upcoming nano operating system
Post Edited (Bill Henning) : 2/21/2010 7:31:16 AM GMT
I think I'd like whatever happens with SYSCALL to remain compatible with whatever the ZPU currently does with it. For example in the Java ZPU simulator. It should be flexible enough already that we can "piggyback" more functionality onto it.
I'm a bit lost as to how/if SYSCALL is used in ZPU and it's simulator. In the provided GCC setup there is a thing called libgloss providing I/O stuff. In there is a comment that syscalls are only used in the so called "Zeta" platform for ZPU. Thing is the Zeta platform is supposed to be obsolete! Without syscalls the libgloss ignores most file I/O stuff and sends standard I/O straight to the UART.
Perhaps the SYSCALL instruction is a totally free opcode nowadays.
I'm asking on the ZPU mailing list. They seem quite keen to advise there.
Yep, adding a single floating point multiply to the test program blows it up by 4000 bytes withou any I/O.
Lots of room for pruning there I'm sure.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.
If there are to many propeller specific things inside it is hardly possible to transfer the whole system to other miicrocontrollers.
Therefore I vote for more abstract "Bios" functions.
like:
writeByte serial
readByte serial
write32bitPortpins <- abstract, because on every processor architecture this pins will be mapped their IOs
read32bitPortpins
readByteMassMemory
writeByteMassMemory
...
After that things get a bit more tricky.
For example when you say "write32bitPortpins" that actually comes down to writing to a register. OUTA in our case. There are many such registers in the Prop (and other micros). It comes down to providing a mechanism to enable reading/writing those registers from Zog Code.
Now if you or anyone else has abstracted that into write32bitPortpins() and read32bitPortpins() functions in your application code it becomes a simple matter to implement them when moving your code to the Prop, using the supplied mechanism.
In general these things get abstracted away by operating systems, even very simple ones. So serial I/O, for example may become a case of opening a device, say "/dev/tty2", then using read() and write() on it as if it was a file. When dealing with an VM like Zog the important thing is to expose the underlying hardware in such a way that any such abstractions can be used on them. Including the Chris Micro Abstraction Layer.
Then we might want to think about things the Prop can do that other micros cannot, like cogstart.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.
It may not be immediately clear, but cognew_native is used to start a PASM COG on Propeller accessible via HUB variable interface and the code being used is the VGA.spin PASM binary.
As I mentioned, there are function stubs for cognew/cogstop. The wordfill/wordmove methods are kernel services in ICC for better performance; some macro versions using memset/memcpy are included, but untested. Non-standard library functions wait and itoa need separate implementation (propeller.h provides msleep using waitcnt for example).
Very interesting, let's see if I understand this correctly:
1) This C code runs in HUB (ICC or Catalina?) perhaps with it's own address space i.e. C addresses are not HUB addresses.
2) VGA_Text_Array is a binary blob of PASM code that is the VGA driver. It sits in C address space.
3) gVgaText is an array of parameters for the VGA drive to pass through PAR when it is started.
4) cognew_native has to some how cause the kernel or Zog VM to execute a COGINIT with those as parameters.
5) gscreen and gcolors are the screen buffer, referenced from gVgaText.
6) When everything is up and running the C code interacts with the driver through gscreen and gcolors only.
All of this seems eminently doable from the Zog VM. Lets say we used the SYSCALL instruction. The VM could trap the SYSCALL, determine that a COGINIT is required, grab the VGA_Text_Array and gVgaText address parameters off the stack and use them in COGINIT.
A little complication is that the address space the Zog VM is working in is not the HUB address space so the ZPU memory base address would have to be added to the addresses to translate them to HUB addresses for COGINIT. No problem. (Hmm what about the gscreen and gcolors pointers? How do they can adjusted? Zog knows nothing of them or any other content of the two COGINIT parameters it has been given)
This all breaks down when we move Zog to external memory. Or at least gets a lot more complicated.
Now the SYSCALL has to:
1) Copy the binary PASM blob to some place in HUB that's free so that COGINIT can find it.
2) Copy the gVgaText parameter array to some place in HUB that's free so that COGINIT can find it.
3) Now it needs a way to have direct access to HUB RAM, which is not in it's normal external address space, so that it can write to gscreen and gcolors. How does it know where they live in HUB now?
4) Even less sure how the pointers to gscreen and gcolors get fixed up in this case.
Hmm... I'll ponder this some more.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.
Look at my proposed SYSCALL extensions... which included hub read/write for byte/word/long, coginit etc
Currently VMCOG is passed a top address, and number of 256 byte pages it can use, growing down from there. Easy to use hub memory below that for a "cog loader buffer" (2k) and the frame/color buffers.
Note the VMCOG API includes a call that when given a virtual address, returns the hub address that the page containing that virtual address is in.
vm.GetPhysVirt(vaddr) returns the base of the hub page containing the page vaddr is in.
I am considering adding two new calls:
vm.Lock(v_base_page,num_pages) ' allocate a contigous buffer for hardware use - disk/screen/cog buffer, may not be swapped
vm.Unlock(v_base_page,num_pages) ' release locked pages back into the general working set pool, may be swapped
To use this to make a cog loader buffer (2k)
rc := vm.Lock($F000,8) ' page in eight contigous 256 byte pages, and lock them in-core starting at $F000 virtual address
hubaddr := vm.GetPhysVirt($F000) ' return the hub address of the continous locked in memory 2K block
Now an i2c or spi function could be used to load the driver
i2c.read(eepromaddr,hubaddr,2048) ' load a driver from eeprom
Then, using SYSCALL and the extensions I suggested,
cognew(hubaddr,hubpar)
After the cognew completes (I recommend waiting 1ms)
vm.Unlock($F000,8) ' release the cog buffer
If I add Lock and Unlock, it would be possible to manage basically all the hub memory via VMCOG, allocating and de-allocating it in 256 byte pages on the fly as needed, so allocating screen buffers also becomes easy. So does allocating/deallocating disk buffers.
Lock would fail (and return -1) if it could not lock the requested number of pages (ie there are only 16 pages available in the working set, and the program wants to lock 32 pages - can't do it. I would have lock fail if at least 8 pages were not left unlocked)
This also allows for changing drivers on the fly for TV or VGA, and resizing screen buffers, file buffers etc...
I am now leaning to having VMCOG manage all hub memory, except for the SphinxOS top pages, and the Largos required first two pages ($0000-$001FF), allowing VMCOG to manage 123 of the possible 128 pages (of 256 bytes).
vm.IsLocked(vpage) could be added to test if a page is locked or not.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
www.mikronauts.com E-mail: mikronauts _at_ gmail _dot_ com 5.0" VGA LCD in stock!
Morpheus dual Prop SBC w/ 512KB kit $119.95, Mem+2MB memory/IO kit $89.95, both kits $189.95 SerPlug $9.95
Propteus and Proteus for Propeller prototyping 6.250MHz custom Crystals run Propellers at 100MHz
Las - Large model assembler Largos - upcoming nano operating system
Post Edited (Bill Henning) : 2/21/2010 5:45:16 PM GMT
Firstly an operating system, like Linux, defines a syscall interface as a way of getting in and out of kernel space but ultimately as a way of providing services like file I/O etc. One could imagine getting ucLinux running with Zog and uCLinux would provide the syscalls in "normal " C code.
Secondly, there is the ZPU SYSCALL instruction that hyperspaces you out of the virtual machine into whatever system is running it.
So lets say we build a FAT file system from the C FAT code we have available for Zog.
At the first level the application makes a syscall into the "operating system" (Perhaps just the C library)
That in turn calls on the FAT code modules to open/close/read/write files.
That, in Zog, may well use the SYSCALL instruction to get something outside the virtual machine to do the actual block reads from SD card or whatever.
Looks like Virtual Memory is the key to the HUB/External RAM space problems.
What you are basically saying is that with VM all memory is the same, HUB or EXT. It's all on the other side of the VM as far as Zog is concerned. Only thing is to be able to lock certain pages to HUB so that the rest of the Prop can see them. Cool.
So can we get Zog to "malloc" a bunch of RAM. Pull one of those PASM drivers of an SD card into that RAM, lock it into HUB space and the start the driver?
Also looks like we are not going to need Linux on the Prop when you have finished[noparse]:)[/noparse]
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.
Heck, the old mainframes (and other newer OS's) had similar mechanisms for locking pages for DMA.
Which is why I was proposing, using SYSTEM (which is why i named it different from SYSCALL) as an escape hatch for running PASM/LMM/SPIN code, either directly, or by sending messages to another cog!
If you re-read my long emails, it will now make sense!!!!
And my last long email adds Lock and Unlock, allowing for creating (and re-sizing!) video buffers, disk buffers, cog buffers on the fly, and releasing them when not needed!
Yep, you re-stated how I would load a driver in the previous message, except I used an i2c example.
LOL
actually Largos is intended to be (at the user level) very similar to v7 Unix.
I am still toying with what names to call locking / unlocking pages....
vm.Lock(vmaddr,numpages)
vm.Unlock(vmaddr,numpages)
-OR-
vm.AllocPhys(vmaddr,numpages)
vm.FreePhys(vmaddr,numpages)
-OR-
vm.Pin(vmaddr,numpages)
vm.Rel(vmaddr,numpages)
I'd lock it first, but certainly it would also work to lock it after loading, but exactly right.
Lock a page range big enough, load driver, launch cog, release range.
For video buffers, just lock a big enough range, and get its physical address. Ditto for extra file buffers (but file buffers would actually work without locking, and then could potentially swap out when infrequently used)
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
www.mikronauts.com E-mail: mikronauts _at_ gmail _dot_ com 5.0" VGA LCD in stock!
Morpheus dual Prop SBC w/ 512KB kit $119.95, Mem+2MB memory/IO kit $89.95, both kits $189.95 SerPlug $9.95
Propteus and Proteus for Propeller prototyping 6.250MHz custom Crystals run Propellers at 100MHz
Las - Large model assembler Largos - upcoming nano operating system
Post Edited (Bill Henning) : 2/21/2010 6:19:56 PM GMT
You understand the ideas mostly. Some clarifications are in order though.
The ICC C code is LMM interpreted. C .text, .data, .bss, etc... addresses are all HUB addresses in ICC.
I have used the same code with slight compiler/kernel modifications from Richard to do the same methodology fetching from and in XMM.
In ICC, cognew_native executes the PASM coginit instruction normally (no special sauce). The HUB address of the C code BLOB and the parameters are given to coginit. PASM coginit grabs the COG code from HUB regardless of where coginit is called (spin interpreter in a cog or straight PASM in a cog) not a problem.
Since all pointers defined in C space are used by the ICC LMM kerenel, there is nothing complicated about it there ... I'm not sure what would be different for ZOG since I have not examined your design carefully enough (and would rather leave that to you).
Clearly, this is why I posted the idea to the thread [noparse]:)[/noparse] If ICC can do it, ZOG should have no problem.
I figured I would share since I've been down this road before, it is a quick and mostly appreciated solution, and the code is in the OBEX (some enhancements will be required obviously to make the code ZOG compatible unless ZOG implements ICC interfaces which probably won't happen). Ross has provided a compatibility header for ICC and Catalina so that it is easier to share/migrate code between the two compilers. Having a good ANSI compatible, cross platform library with a HAL would be best though (POSIX and ioctl would be even better, but all that would probably require tons of overhead which puts it out of budget).
Good luck.
--Steve
I think the next step for Zog is to get it to work with VMCog. Just 16K (or more) of HUB RAM initially and then my 512K external RAM. Think I said this before. We have crude single stepping and console output, enough to test it works.
Meanwhile think about all this I/O, SYSCALL stuff some more. I want to find out more about how the ZPU C libs are put together and how they interact with real ZPU hardware platforms and the simulator. There are some little tricks going on with the later.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.
What is different in Zog is that it works (currently) within a 16Kb area of HUB RAM. The C code only has access to memory from 0 to 3FFF. The VM will actually roll over addresses if the C code tries to go out of range.
Well that 16K of space is actually sitting in HUB at whatever address BST put it at when the Spin code was compiled.
So address 0000 in C is not zero HUB.
Hence passing those addresses directly to coginit will not work, they at least need translating back to HUB addresses.
When we move into external memory this all gets more complicated. But it looks like Bill is many steps ahead of us there [noparse]:)[/noparse]
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.
Catalina uses a different model with built-in drivers and would probably never want to use the OBEX C driver code. There are some advantages to this as I said before. If ZOG C memory is not adjustable to match HUB addresses by some means, the whole idea of using the current OBEX ICC driver model is a waste of time for ZOG.
Bill's efforts implementing VM ideas that have been discussed several ways in the forum will of course be very useful to many of us one way or another, and I look forward to the result. Of course I also patiently look forward to Largos ... maybe this summer's Parallax nerd-fest will be a good podium for an update on that.
Don't forget what we were originally looking for is a way for Zog to do I/O. The ZPU architecture uses memory mapped I/O devices which we don't like much in the Zog Virtual Machine as it requires checking all memory accesses for I/O addresses which is a bit of a performance hit.
Turns out the ZPU SYSCALL instruction, intended to be used only with ZPU simulators I guess, probably allows everything we want and is very simple to use.
So basically it works like this:
1) In C you might want to call some standard I/O function like "write (fd, buf, nbytes)".
2) In the C library that call comes to a system call for the OS to handle the write: "result=_syscall(&t, SYS_write, fd, buf, nbytes);"
Here the &t is used for returning errno, SYS_write is just a number the means "write" then we have the "write" parameters.
3) In the C lib for ZPU the _syscall function is handled in assembler and simply executes a SYSCALL ZPU instruction.
(Provided you are not running on real hardware FPGA that has no SYSCALL)
All of a sudden the all the info required to perform the write is on the stack and the ZPU virtual machine can take care of it in PASM or Spin and the proceed to the next instruction.
So there it is, we can do all normal stdio, and file handling stuff easily via SYSCALL.
So what about all those other Prop specific goodies like access to timers, pins, cogint, cogstop etc?
Well quick and dirty is that we can always add some SYS_**** function numbers and define whatever other syscall parameters we like. So for example:
"result=_syscall(&t, SYS_cogint, cog_blob, par, parbytes);"
Could be used to get the VM to start a COG with PASM code in cog_blob and parameters in par.
The more Unixy way to do it might be to abstract all this into some device which is operated on like a file:
fd = open ("/dev/prop/coginit", "r");
write(fd, cog_blob, nbytes);
write(fd, cog_par, nbytes);
close(fd);
Or do it via ioctl calls.
Initially I think I like the "quick and dirty approach".
Yep, Bill has said all this previously, I'm just pointing out how well it fits into the existing ZPU and the supplied C libs for GCC. We can live without any IO instructions in Zog and without any memory mapped virtual peripherals.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.
You have room for open, read, write, close, and ioctl?
Would be nice to write device drivers a familiar way.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Links to other interesting threads:
· Home of the MultiBladeProps: TriBlade,·RamBlade,·SixBlade, website
· Single Board Computer:·3 Propeller ICs·and a·TriBladeProp board (ZiCog Z80 Emulator)
· Prop Tools under Development or Completed (Index)
· Emulators: CPUs Z80 etc; Micros Altair etc;· Terminals·VT100 etc; (Index) ZiCog (Z80) , MoCog (6809)·
· Prop OS: SphinxOS·, PropDos , PropCmd··· Search the Propeller forums·(uses advanced Google search)
My cruising website is: ·www.bluemagic.biz·· MultiBlade Props: www.cluso.bluemagic.biz
I don't need any room in Zog itself to do most of this.
For example, a simple approach would be to have open, close, read, write calls pass through the described SYSCALL mechanism at the end of which they just get passed to fsrw. There we have our file system established with very little "glue" code in Zog.
Now we could inspect the path strings used for open and look for things that are not files but devices. So for example our "Virtual File System" may have: "/dev/uart". Opening that gets you a handle for the UART and all subsequent read/writes for that handle get directed to FullDuplexSerial or whatever.
Not sure what we would do with ioctl yet.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.
Post Edited (heater) : 2/24/2010 9:17:01 AM GMT
Zog OS or not Zog OS?
Let me elaborate:
Catalina and ICC not only provide the capability of running C code they also carry along with them a whole run time environment complete with drivers, "plug-ins", loaders etc etc. Pretty much they are their own operating systems.
Currently Zog enables one to run C code but is pretty much a regular standalone Propeller object, like fswr or FDX etc, that can be dropped into any normal Propeller project. without any impact on the existing project.
Now any C program needs a fairly well defined environment to run in, to supply standard IO, file handling etc. So adding that environment naturally starts to grow the system into an operating system.
Then we have the issue of enabling C code to make use of Prop specific feature like pins, timers, coginit etc.
You start to see why ICC and Catalina have grown up to be mini-operating systems that take over the whole Propeller.
Now I'm inclined to think Zog should not do that. It should remain a small, simple, self contained object.
So for example if you feel that your latest all signing all dancing Propeller creation would be all the better if it had a big blob of C code to manage it or provide a user interface logic or whatever. Then you can throw in the Zog object and point it at some C binary in SPI RAM or SPI FLASH or such, add a few lines of "glue" Spin to connect Zog to your app and there it is. With no impact on the rest of your existing creation.
The reasons I say this are:
1) Catalina and ICC already exist and do the total environment thing very well. I don't want to re-invent the wheel.
2) Seems to me Zog should keep itself small and simple and adaptable to existing applications and operating systems.
I have not really looked at any of the Propeller OS or DOS systems. But it seems to me that Zog should be applicable and adaptable to all of them.
3) My itch right now is to get Zog to run large C codes from the external RAM on TriBlade/RamBlade/DracBlade etc. The fascination for me is the simple task of getting GNU compiled code to work nicely on the Prop.
4) I'm lazy [noparse]:)[/noparse] If people have already built operating system infra-structure for the Prop I'd rather just bolt onto that than recreate it myself.
5) Time, as always, is short.
What does everybody think about this?
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.
1) Looks like the Java SYSCALL is basically identical to what I proposed, only difference being returning the status to a location pointed to by a pointer vs. leaving it on the top of the stack. It would work great [noparse]:)[/noparse] and I believe it is the right way to go.
2) re/ standard unix-like open/read/write/close/ioctl ... like heater, I feel they should be implemented via SYSCALL, if needed, for an application
3) re/ ZogOS vs. ZogObject
I think both will live long and prosper, because they address two separate markets and while I can see why Heater is leaning towards Zog/Object, I can definitely see the advantages to Zog/OS
Right now, I think what is needed is to get Zog/Object running, with VMCOG and XMM solutions. I think VMCOG will be the "common denominator" low-cost version, and that over time, people will implement higher performance XMM for different platforms (like Triblade, Morpheus etc) - which I think is a VERY good thing, because it will allow spanning the spectrum from extremely inexpensive two SPI RAM solutions to Ramblade/Triblade with up to 1MB on one blade, to Morpheus with up to 16Mb... or even 128MB with mctrivia's modules - not to mention other upcoming platforms.
If Zog works half as well as I think it will, I may very well decide to make a Largos/Zog at a later date, as it would save me a ton of work... stringutils, binutils etc should just compile and run!
I know Largos is taking a long time... but frankly, I need to spend most of my time on revenue generating projects, or potentially decent revenue generating projects as I don't have a "day job".
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
www.mikronauts.com E-mail: mikronauts _at_ gmail _dot_ com 5.0" VGA LCD in stock!
Morpheus dual Prop SBC w/ 512KB kit $119.95, Mem+2MB memory/IO kit $89.95, both kits $189.95 SerPlug $9.95
Propteus and Proteus for Propeller prototyping 6.250MHz custom Crystals run Propellers at 100MHz
Las - Large model assembler Largos - upcoming nano operating system
Post Edited (Bill Henning) : 2/24/2010 8:32:53 PM GMT
I'm not sure ICC is a mini-operating system ... perhaps this requires a little study for a good understanding. In short however, you build, download, and run. The code does what you tell it to do with the libraries you choose. Catalina has a slightly different, slightly more complex, and slightly smaller resultant library model where the devices are chosen by the compiler/linker rather than by specifying big globs that the user provides.
Added: * i.e. details not handled by open/read/write such as modes for char/buffer devices that may need to be changed after open.
Well yes, but if you happen to have an OS lying around (largos) then I'd rather make use of that[noparse]:)[/noparse]
There is no need for yet another operating system.
I guess the syscall is designed to return a status to let you know if the syscall fails, unknown sys ID parameter say. Then it also returns a status from whatever the particular operation is, read, write etc, through the first pointer parameter.
VMCOG and XMM for Zog is next up. Well, after I've fixed up the SYSCALL console I/O.
Hmm... For some reason printf() does not work. print(), putnum(), snprintf() all work. Odd.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.
I'd suggest that all SYSCALL's return a value on the stack as follows:
0 = OK
anything else, depending on the function called, could be interpreted as an integer, or a pointer, in case an elaborate error result needs to be returned
Very odd about printf.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
www.mikronauts.com E-mail: mikronauts _at_ gmail _dot_ com 5.0" VGA LCD in stock!
Morpheus dual Prop SBC w/ 512KB kit $119.95, Mem+2MB memory/IO kit $89.95, both kits $189.95 SerPlug $9.95
Propteus and Proteus for Propeller prototyping 6.250MHz custom Crystals run Propellers at 100MHz
Las - Large model assembler Largos - upcoming nano operating system
Bill Henning the inventor of the LMM technique is considering forgoing LMM for his own operating system!
Cannot be.
Being able to use the GCC toolchain on the Prop is the raison d'
I think Largos/Zog would be happier on Prop1, and would show up earlier than Largos/LMM on Prop1 [noparse]:)[/noparse] due to having to re-invent, re-code, and re-test far less wheels...
I don't intend to abandon Largos/LMM by any means, but I am thinking of going for the "low hanging fruit" first to get it out faster.
Also... Largos is VM agnostic, each file is tagged by a file type - LMM and ZOG programs could happily co-exist on the file system [noparse]:)[/noparse] allowing for LMM speed for more critical programs, and memory savings with ZOG on less critical programs.
Yep, Largos is meant to be very Unixy.. it already is... see mikronauts.com/software-products/largos/ for the details I have released so far.
Basically, I was inspired by the early PDP-11 Unix work, and what Tannenbaum got running on a small model x86, combined with advances from Plan9 and Hurd; resulting in a message passing extremely light weight design. Realistically, Largos (LMM version) needs at least 256KB of XMM, however I could see a Largos (ZOG... hmm... Zargos?) running fairly decently with 2 SPI Ram's and an SD card, using VMCOG. Certainly running as well (and probably better) than the early PDP-7 and PDP-11 versions of Unix.
Largos will really shine on Prop2, and conceptually, could be grown to support the GCC toolchain "natively" with LMM2, with a 32 bit virtual address space.
But just in case... I've started a paper design for a VLMM that might give Prop1 a fighting chance with SPI RAM's and an SD card... and Catalina would allow for a lot of Unix code to be just recompiled (given sufficient libc and system infrastructure support)
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
www.mikronauts.com E-mail: mikronauts _at_ gmail _dot_ com 5.0" VGA LCD in stock!
Morpheus dual Prop SBC w/ 512KB kit $119.95, Mem+2MB memory/IO kit $89.95, both kits $189.95 SerPlug $9.95
Propteus and Proteus for Propeller prototyping 6.250MHz custom Crystals run Propellers at 100MHz
Las - Large model assembler Largos - upcoming nano operating system
Post Edited (Bill Henning) : 2/25/2010 8:03:31 PM GMT
chmod +x:Zog vi
and to set a file to be an LMM/Catalina executable
chmod +x:Catalina grep
and a PropellerBasic program:
chmod +x:PropellerBasic startrek
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
www.mikronauts.com E-mail: mikronauts _at_ gmail _dot_ com 5.0" VGA LCD in stock!
Morpheus dual Prop SBC w/ 512KB kit $119.95, Mem+2MB memory/IO kit $89.95, both kits $189.95 SerPlug $9.95
Propteus and Proteus for Propeller prototyping 6.250MHz custom Crystals run Propellers at 100MHz
Las - Large model assembler Largos - upcoming nano operating system
Largos sounds totally audacious, cat cd du df ls rm rmdir mv mkdir mkfs pwd touch, can't wait. So if Zog gets us there quicker that's excellent. Many things will be very happy on Prop II, I'm not very good at waiting though.
Don't forget:
chmod +x:ZiCog CPM
[noparse]:)[/noparse]
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.