Has anyone tried using syscall with the USE_JCACHED_MEMORY build of ZOG? I can run fibo.bin just fine with the JCACHE build but I'm having trouble with terminal I/O when I enable syscalls so that I can do SD card sector I/O through the read/write syscall interface. It seems that the sector I/O works fine but the terminal I/O starts behaving strangely. At first I thought it was because I needed to invalidate the ZOG cache line pointer on return from a syscall but when I added "mov cacheaddr, #1" after coming back from a syscall, that didn't help my problem. Any ideas?
More information: I forgot to mention that this same code seems to work fine when I build ZOG with USE_VIRTUAL_MEMORY. In both cases, I'm using the two 32k SPI SRAM chips on the C3 for external memory.
Has anyone tried using syscall with the USE_JCACHED_MEMORY build of ZOG?
Does the hello.c test program work for you? That uses syscall read/write and works for me. I am using an updated version of SdramCache.spin but I don't believe the zog interface has changed. I'll be adding a block interface for TV using syscall read/write today. Will let you know if I have trouble.
Guess I should post the ZOG I'm using today when I have time.
Does the hello.c test program work for you? That uses syscall read/write and works for me. I am using an updated version of SdramCache.spin but I don't believe the zog interface has changed. I'll be adding a block interface for TV using syscall read/write today. Will let you know if I have trouble.
Guess I should post the ZOG I'm using today when I have time.
No, hello.c doesn't work on my version of the cache code. In fact, it fails even without _use_syscall enabled. I guess I must have a bug in my implementation of your cache API. Time to start reading over my code...
I've been thinking a little about how to access the buffer area of a video driver from ZOG. To me it really makes sense to use a string to specify the driver the same way we use /dev/* in *nix. As things are now, there is only one IO device in ZOG and that's UART.
Can we come to a consensus on how ZOG can access devices?
Is the file association method satisfactory? Is there a better alternative?
Next thing you know we'll want kernel loadable modules
In my ideal there would be no Spin.
A loader, which can be in Spin, would get all required Cogs running for hardware support, and then load in ZPU code which could occupy all of HUB except for mailboxes and buffers used by the COG interfaces.
I made a little start on this with run_zog.spin.
Adding new PASM drivers would be a case of:
a) Probably tweak the PASM to run without Spin and use mail boxes only.
c) Modify the loader to start up those PASM drivers.
d) Arrange that the driver mail boxes and buffers can be found by C code.
I guess it's that last part you want to do via device strings with parameters. Presumably kept at some high location in HUB.
Of course there should also be the option of letting the spin code continue in case anyone wants to us Spin and C together.
Next thing you know we'll want kernel loadable modules
Yep, I think Catalina has that with it's plug-in mechanism.
The kernel loadable module "KLM" I was referring to is a more advanced concept like a dynamic library. Catalina's modules are part of the kernel and must be specified at compile time. A KLM can be loaded and installed from media like the SDCARD.
A device string is a way to communicate with the "kernel" by name for getting a "file handle" and performing operations until the application closes the connection. The user should not care what the implementation is in language or otherwise, only that access is available, and changes to the kernel don't affect application code which can be in any language that understands the interface.
With a handle provided by open, read, write, ioctl, and close operations would be possible. This is standard legacy functionality that most C programmers understand. Why is it so difficult?
Catalina's modules are part of the kernel and must be specified at compile time.
Hi Jazzed,
Catalina's plugins are not part of the kernel. They are included compile time only because it simplifies the initialization process - i.e. most plugins are permanently loaded device drivers which use a fixed allocation of Hub RAM (such as a video driver). In such cases it makes sense to load them once during initialization - and then discard them since they will never be needed again.
Plugins that require a Hub memory buffer accept the location of the buffer as a run-time parameter - the actual address can be figured out either at compile time (for permanently loaded plugins), or at run time (for dynamically loaded plugins).
The only interaction the kernel(s) have with any plugin is via the registry, so Catalina plugins can be loaded and unloaded dynamically as required.
Sorry, I mis-spoke. They are part of the "build" with an un-detachable interface to the kernel. Show me a module that can be loaded and re-instantiated after boot-up. The dynamic part of the KLM is what I was describing. Still, it's not a handicap for the feature to be missing. But you certainly don't have dynamic kernel loadable modules in Catalina.
Adding new PASM drivers would be a case of:
a) Probably tweak the PASM to run without Spin and use mail boxes only.
c) Modify the loader to start up those PASM drivers.
d) Arrange that the driver mail boxes and buffers can be found by C code.
I guess it's that last part you want to do via device strings with parameters. Presumably kept at some high location in HUB.
Well, today's been a weird day for my household so posts here have not received attention they deserve. In addition to that, it appears the rainy season is upon us now and we have that silly bi-annual time change.
Back to business. Your items a) and b) make sense for key infrastructure drivers. Optional device drivers could be added statically at first with dynamic KLM function added later.
As far as c) goes, some of that has been discussed in other threads, but the hookup between device identifiers and the mailboxes are TBD. One could use enums for device id's, but maintaining the enumeration table definitions over generations of code seems unlikely. I really believe Kernighan, Thompson, and Richie had it right when they decided to use strings to identify devices. Of course /dev/ is just a prefix. I know ZOG is not *nix, but using *nix methodology is not unreasonable.
I look forward to whatever you decide to implement. Until then, I have to get a demo running and see if I can help David with this potential JCACHE bug in zog.spin.
Sorry, I mis-spoke. They are part of the "build" with an un-detachable interface to the kernel. Show me a module that can be loaded and re-instantiated after boot-up. The dynamic part of the KLM is what I was describing. Still, it's not a handicap for the feature to be missing. But you certainly don't have dynamic kernel loadable modules in Catalina.
The interface of all plugins to the kernel is identical. Perhaps the problem is that I'm not sure what you mean by "dynamic kernel loadable modules". But have a look at the various programs in the Catalina\demos\multicog directory. This contains examples of code that loads and unloads LMM kernels dynamically at run time, and also code to kill a plugin at run time (it never reloads it - but it could). The code for loading or unloading a plugin would be almost identical to some of this code in these examples. A couple of additional function calls would be required, to unregister the old plugin and register the new one - these calls are defined in catalina_plugin.h. The pseudo code to terminate one plugin and then load another in its place would look something like:
cog = _locate_plugin(old_plugin_type); // if we don't already know the cog
_unregister_plugin(cog); // make the old plugin unavailable for more requests
_cogstop(cog); // terminate the old plugin
_coginit(start_up_parameters, new_plugin, cog); // start the new plugin - pass the buffers to use in the start up parameters
_register_plugin(cog, new_plugin_type); // indicate the new plugin is available
Of course if you wanted to load a plugin from SD card, you would first have to read the code from a file into Hub RAM - but the principle is the same.
Am I misunderstanding what you mean?
Ross.
P.S. Maybe we should move this discussion over to the Catalina thread?
Ahh. Well it looks like you're only a few steps away from fully re-loadable KLMs.
Seems like it would be a nice improvement in boot-up configuration to allow for a boot config script (think /sbin and /etc/init.d) for "products" as an alternative to the Catalina command line incantations. Of course there is nothing is wrong with a VB GUI to manage those options.
Using a config script could greatly reduce your maintenance burden (in the feature expansion sense where software tends to get broken) and make it easier for new developers to navigate your ocean of dependencies.
But all that's up to you of course
I plan to get back to porting SDRAM to Catalina after I have a reasonable ZOG demo. This week we are on manufacturing pilot runs for hardware ... so much to do, so little time.
With a handle provided by open, read, write, ioctl, and close operations would be possible. This is standard legacy functionality that most C programmers understand. Why is it so difficult?
Indeed. But, syscalls, handles, open, close, read, write, iocltls etc etc are abstractions provided by an operating system. We don't have an OS. Or even much of a boot loader at this point.
At this time Zog is one level down, at the OS or more like boot loader level. Or say the level of a typical C program running on a micro-controller with no OS just naked hardware.
It runs and it expects to find some IO ports for a console, a block storage device, perhaps a VMCog for external memory, perhaps even a float32 coprocessor.
Currently we have run_zog that loads all that stuff and provides a syscall interface. We could continue with expanding that. run_zog could start load and start up whatever you like and then provide access to it via syscall as open, close, read, write, etc.
This leads to a management nightmare, every platform is different, every application has different requirements. All combinations of platform/application may require a different configuration of debug_zog.
Alternatively we could build on run_zog. Currently run_zog provides no hardware interfaces at all. Just loads Zog and sets it running optionally killing off all Spin and giving all HUB to ZPU code.
As a minumum run_zog could load up all basic hardware as required such as:
a) FullDuplexSerial
b) SD card,
c) VMCog
d) Float32
At that point it can start Zog giving it all of HUB RAM. All space used by driver PASM code being recycled.
Problem is, at that point, how does the C code know where all the device I/O registers (mailboxes) are. And where is the video buffer etc.
These have to be built into both run_zog spin and the C code as constants. Or run_zog has to leave this information somewhere in HUB for the C code to find.
heater At this point I have no idea which way to move
I wonder if I could make a tiny suggestion? How about you think about borrowing the rendevous points from catalina?
If these became a standard of sorts, then the zog people and the catalina people could work together on building cogjects. [insert big group hug]
Consider, as a hypothetical example, Kye's new 640x480 4 color driver. Right now this exists for spin, but if we took the pasm code, stripped out the supporting spin code and instead interfaced with this via known mailbox locations (in high hub ram), we could potentially use this driver for both zog and catalina.
Indeed. But, syscalls, handles, open, close, read, write, iocltls etc etc are abstractions provided by an operating system. We don't have an OS. Or even much of a boot loader at this point.
At this time Zog is one level down, at the OS or more like boot loader level.
Ya, sure. So why does debug_zog have all those sys calls now? Perhaps that's where you hook up your O/S or whatever we happen to have at the moment
Look, at this point I can't write a TV block based driver without syscalls at all because I can't even cheat and read/write HUB ram directly. Character based drivers are easy.
How do you expect to access rendezvous memory without direct access to any HUB RAM from C code?
You decide what you can support and we'll go that direction. Until then I'll produce what I can the easiest way I know to see what ZOG is capable of doing to demo possibilities. At the moment nothing is possible beyond console IO via serial port or keyboard/TV.
Also, How fast can your Triblade with VMCOG finish fibo(20) as measured after downloading from BST? I see 13.9 seconds total and fibo(20) takes 2190ms today at 80MHz with SdramCache. Just curious.
How about you think about borrowing the rendevous points from catalina?
You have a good point. Bill Henning, I and others have been suggesting the idea of stand alone, Spin free, PASM driver modules for some time. Modules that can be used solely through a memory interface with no odd set up steps in Spin first. Bill has his "mailbox" approach. RossH has his "plugins". I have something less formal going on, for example Zog has a modified FullDuplexSerial with it's cyclic buffers exposed in the memory interface for use by C.
In this way PASM drivers can then be used by Catalina, Largos, Zog, or any other system that comes along.
The idea of filtching Catalina's drivers wholesale has made me nervous. Seems like cheating. However the way you put it, including the "group hug" makes me feel better. If there were "one way" to do this, from Catalina or not, it benefits everyone perhaps even encourages systems from others in the future that have not been invented yet.
So an issue becomes, how to define those rendevous interfaces. How to make their location known to C code running under Zog or whatever other code running under something else.
A solution could be Catalina's plugin approach but as yet I have no idea how that works and no idea if it is generally useful in other, as yet unknown, systems.
I was quite prepared to take a simplistic approach:
1) All drivers are loaded at start up. There is no support for dynamic loading/unloading at run time. After all you don't generally unload a UART on a normal micro-controller application. Such dynamic activity can be done in the application code if ever required.
2) All such drivers present a memory interface, registers, buffers etc, in high HUB space.
3) There is a memory map defined for those interfaces. Just like the special function registers in an MCU.
4) The selection of drivers loaded and the resulting memory map is application defined.
5) This allows for all PASM space in HUB to be recycled as data, stack, video buffer, whatever when the C code is up.
The idea of filtching Catalina's drivers wholesale has made me nervous. Seems like cheating.
Feel free to filch! I have filched code from others as part of Catalina (although I always acknowledge where I filched it from).
Catalina's registry concept (which is the interface to all plugins) is not particularly complex and it could probably be improved a bit - but it has been good enough for everything I have needed to do with it to date.
If you come up with a better approach, you can be sure I will certainly feel free to filch it and use it in Catalina!
So why does debug_zog have all those sys calls now? Perhaps that's where you hook up your O/S or whatever we happen to have at the moment.
Ah, good question.
The ZPU includes a SYSCALL instruction. As far as I can tell before Zog this was only ever used to get console I/O out of ZPU code whilst running under the Java ZPU simulator. For sure the provided C libs only support read/write on stdio using SYSCALL.
I came along and implemented the SYCALL instruction and the console I/O using it. Why?. Because.
Then I started adding those hooks for open, close, read, write, seek etc etc because they are all the SYSCAlL IDs defined in the ZPUs libs. Given that we have an SD card driver and a file system in Spin there is no reason not to continue down this path and a provide file system (and other devices) to C via sys calls. This may require a little work on the C library so as to take all the syscalls into use.
In this scenario we have C code in ZPU and "OS" in Spin. As we are using Syscalls the OS could be replaced at any time with something else.
This sounds like quite a reasonable approach, especially if the ZPU code is running from external RAM. and it has the advantage that a lot of application code can be run and tested in my C version of ZPU under Linux or Windows.
If you are keen on this approach I'm quite happy to pursue it.
If we go that way I'm going to drop all thoughts of running ZPU code from HUB only systems, at least for the time being.
...at this point I can't write a TV block based driver without syscalls at all because I can't even cheat and read/write HUB ram directly. Character based drivers are easy.
How do you expect to access rendezvous memory without direct access to any HUB RAM from C code?
Ah but you can:) As of Zog v1.6 ZPU code running from external RAM has been able to access HUB RAM.
The memory map is like so:
256MB ZPU RAM, followed by 32KB HUB RAM window, followed by 2KB COG RAM window.
Have a look for the definitions in ZOG:
zpu_hub_start long $10000000 'Start of HUB access window in ZPU memory space
zpu_cog_start long $10008000 'Start of COG access window in ZPU memory space
zpu_io_start long $10008800 'Start of IO access window
Warnings:
1) I probably have not tested this sufficiently.
2) The HUB window should be bigger, we may want to access the ROM, and what about Prop II?
3) It may change, due to 1) and 2) above or any suggestions you may have.
4) It only works for read/write of LONGS from HUB. I did not want to test the addresses on all possible memory accesses.
As I said, if you are keen on this approach I'm quite happy to pursue it.
All attributions, copyright notices etc will remain in place. It's more useful to me and others to know were things come from than to try and hide the origin.
It occurs to me that one would probably want read/write BYTE access for any video RAM. As Zog has a separate function for read BYTE as opposed to instruction fetch this can be done with minimal performance impact.
If you are keen on this approach I'm quite happy to pursue it.
The syscall approach you laid out and the reasons for using it sound fine except for missing ioctl. I do believe HUB only support should be available some time, some way. Allowing *any* driver is important of course, but some framework should be defined.
Does ZOG/ZPU not have an ioctl syscall? Seems that we need it. As you know, ioctl allows for abstraction of non-buffer properties in a driver. For example, ioctl could be used to access to row, col, color, and screen in TV_Text.spin and friends by reference or value depending on what makes sense.
Let me know the approach you plan to use for associating file/device descriptor IDs with devices. I'll hard-code something for the time being. Console uses 0,1,2 ... I'll try using 3 or something for screen block driver.
I'm *not* advocating hard-coding file/device IDs as a permanent solution.
Ah but you can As of Zog v1.6 ZPU code running from external RAM has been able to access HUB RAM.
The memory map is like so:
256MB ZPU RAM, followed by 32KB HUB RAM window, followed by 2KB COG RAM window.
Well that helps a lot! Thanks for reminding me.
Increasing the HUB RAM virtual window to 128KB or even 1MB is not unreasonable. Hard-coding things bothers me, but do we have a choice? At least a common .h file could carry this stuff for C build.
It occurs to me that one would probably want read/write BYTE access for any video RAM. As Zog has a separate function for read BYTE as opposed to instruction fetch this can be done with minimal performance impact.
Beware of ZPU/Prop endianness issues.
Where is this separate function for read BYTE and how do I use it from C? Read/modify/write using long is probably sufficient for byte only data elements.
I guess no one is using on_syscall from with external memory.
Consider this:
PRI on_syscall | p
'Handle CPU SYSCALL instruction
' Example C call:
' result=_syscall(&t, SYS_write, fd, buf, nbytes);
'Set a frame pointer into the ZPU stack (but HUB realtive)
framep := zog_mbox_sp + @zpu_memory
'printStackFrame
'Perform system call function given by the ID param
case long[framep + 8]
SYS_exit:
_exit
SYS_open:
_open
SYS_close:
_close
SYS_read:
_read
SYS_write:
_write
What's wrong with that? Well, zog_mbox_sp will contain an external
memory address which we can't access using HUB memory syntax.
Instead something like this needs to be done:
PRI on_syscall | p
'Handle CPU SYSCALL instruction
' Example C call:
' result=_syscall(&t, SYS_write, fd, buf, nbytes);
#ifdef USE_JCACHED_MEMORY
'Set a frame pointer into the ZPU stack (but CACHE realtive)
p := cm.readlong(zog_mbox_sp + 4) ' syscall ID offset at + 4
#endif
#ifdef USE_VIRTUAL_MEMORY
'Set a frame pointer into the ZPU stack (but VMCOG realtive)
p := vm.rdvlong(zog_mbox_sp + 4) ' syscall ID offset at + 4
#endif
#ifdef USE_HUB_MEMORY
'Set a frame pointer into the ZPU stack (but HUB realtive)
framep := zog_mbox_sp + @zpu_memory
' printStackFrame
p := long[framep + 8]
#endif
'Perform system call function given by the ID param
case p
SYS_exit:
_exit
SYS_open:
But then there's the implementation of all the other syscall stuff. :<
Still plugging away at it. I can run all of your tests sucessfully once. I just get in trouble if I alternate tests 3 and 4. I may have introduced bugs in your test code when I adapted it to my cache code though. I'm still working on it.
@Heater,
I guess no one is using on_syscall from with external memory.
I am. However, I hacked debug_zog.spin significantly. I merged the JCACHED and VIRTUAL code and modified the syscall stuff to work with either or hub RAM. This also could be a source of my trouble. I may have introduced a bug in the process of doing this simplification. I've attached my new debug_zog.spin so you can point out the errors of my ways! :-)
I hacked debug_zog.spin significantly. I merged the JCACHED and VIRTUAL code and modified the syscall stuff to work with either or hub RAM.
This looks pretty good so far. I'm working on interfaces to a DISPLAY_FILENO descriptor now. Some day I would like to try your SDCARD code. We should probably consider wrapper objects for implementing the file operation function code to keep debug_zog.spin simpler.
Here's a new SdramTest.spin file that automatically adjusts the walking test size. Hopefully there is little else to change to support SPI RAM beyond the "mem" object. Do you get errors if you use the loop test all command "T a" ?
Do you get errors if you use the loop test all command "T a"?
Yes, I get occasional errors with the "T 3" or "T 4" tests whether I run them as part of "T a" or by themselves. I'm beginning to think I may have SPI bus timing issues. I removed a NOP instruction that I didn't think was necessary but maybe it was!
We should probably consider wrapper objects for implementing the file operation function code to keep debug_zog.spin simpler.
I'm not sure what you mean by "simpler". I'm only using the read/write calls now to handle raw sector I/O to the SD card. The actual filesystem code is written in C and linked with the ZOG image. The only thing that fsrw is used for is sector I/O once the ZOG program is started.
Here's a new SdramTest.spin file that automatically adjusts the walking test size. Hopefully there is little else to change to support SPI RAM beyond the "mem" object.
I mean less code clutter. I expect that *any* SPIN device driver could be implemented using syscalls.
Can you imagine how many lines of code would be necessary in _read to support 144 block devices for example? Of course with that many devices, just the long chain of #ifdef would be unbearable.
Too bad SPIN doesn't support inheritance and virtual methods. This of course is primarily why I mentioned the possibility of using kernel loadable modules "KLM" on the previous page.
Howdy! I'm happy to report the results of some efforts this week
I've been optimizing again while working on a ZOG TV_Text_Half_Height + Keyboard demo. I found that zog.spin cache_read routine was always using the WRITE_CMD parameter. While there's nothing wrong with that really, it just means all cache_reads were always writing back data unnecessarily. So, that was a drag on performance, and it is fixed in my private ZOG copy.
Another optimization is to use COG based tags instead of HUB tags. While this limits the burst size of cache lines, it also reduces the cache swap decision time by 2 more instructions than using HUB tags.
What's the impact of all this?
Fibo test platform: ZOG HUB*1 ZOG SdramCache*2 ZOG*VMCOG*3
(System Clock) 80 MHz 80 MHz 100 MHz
fibo(20) 684ms 2060ms 2782ms
fibo(26) 12288ms 36980ms unknown
fibo test time 43.9s 133.1s unknown
I had to use old numbers since people refuse to post anything new for some reason.
ZOG running from HUB could finish FIBO(26) in about 41 seconds according to an old Heater post.
The VMCOG performance I found was for 100MHz Triblade is fibo(20) at 2782ms.
Conclusions:
As far as I can tell without more up to date information ...
SdramCache is 25% faster than VMCOG for fibo(20) on a 20% slower Propeller
SdramCache is 3x slower than HUB on the same machine.
The equivalent SDRAM performance with VMCOG as the front end is impossible to achieve because 1) VMCOG's HUB interface is slower, and 2) there is not enough room for the 32 byte 10MB/s SDRAM read/write code.
Standardizing on VMCOG for external memory interface would be a mistake even if VMCOG could support 32MB+ of memory.
Integrating with Catalina should yield interesting results.
Comments
More information: I forgot to mention that this same code seems to work fine when I build ZOG with USE_VIRTUAL_MEMORY. In both cases, I'm using the two 32k SPI SRAM chips on the C3 for external memory.
No. I have only run the ZPU VHDL on the GHDL simulator.
Now that there is the Zpuino by
Guess I should post the ZOG I'm using today when I have time.
Can we come to a consensus on how ZOG can access devices?
Is the file association method satisfactory? Is there a better alternative?
Next thing you know we'll want kernel loadable modules
Yes this needs a little thinking about.
In my ideal there would be no Spin.
A loader, which can be in Spin, would get all required Cogs running for hardware support, and then load in ZPU code which could occupy all of HUB except for mailboxes and buffers used by the COG interfaces.
I made a little start on this with run_zog.spin.
Adding new PASM drivers would be a case of:
a) Probably tweak the PASM to run without Spin and use mail boxes only.
c) Modify the loader to start up those PASM drivers.
d) Arrange that the driver mail boxes and buffers can be found by C code.
I guess it's that last part you want to do via device strings with parameters. Presumably kept at some high location in HUB.
Of course there should also be the option of letting the spin code continue in case anyone wants to us Spin and C together.
Yep, I think Catalina has that with it's plug-in mechanism.
A device string is a way to communicate with the "kernel" by name for getting a "file handle" and performing operations until the application closes the connection. The user should not care what the implementation is in language or otherwise, only that access is available, and changes to the kernel don't affect application code which can be in any language that understands the interface.
With a handle provided by open, read, write, ioctl, and close operations would be possible. This is standard legacy functionality that most C programmers understand. Why is it so difficult?
Hi Jazzed,
Catalina's plugins are not part of the kernel. They are included compile time only because it simplifies the initialization process - i.e. most plugins are permanently loaded device drivers which use a fixed allocation of Hub RAM (such as a video driver). In such cases it makes sense to load them once during initialization - and then discard them since they will never be needed again.
Plugins that require a Hub memory buffer accept the location of the buffer as a run-time parameter - the actual address can be figured out either at compile time (for permanently loaded plugins), or at run time (for dynamically loaded plugins).
The only interaction the kernel(s) have with any plugin is via the registry, so Catalina plugins can be loaded and unloaded dynamically as required.
Ross.
Back to business. Your items a) and b) make sense for key infrastructure drivers. Optional device drivers could be added statically at first with dynamic KLM function added later.
As far as c) goes, some of that has been discussed in other threads, but the hookup between device identifiers and the mailboxes are TBD. One could use enums for device id's, but maintaining the enumeration table definitions over generations of code seems unlikely. I really believe Kernighan, Thompson, and Richie had it right when they decided to use strings to identify devices. Of course /dev/ is just a prefix. I know ZOG is not *nix, but using *nix methodology is not unreasonable.
I look forward to whatever you decide to implement. Until then, I have to get a demo running and see if I can help David with this potential JCACHE bug in zog.spin.
Thanks.
--Steve
Of course if you wanted to load a plugin from SD card, you would first have to read the code from a file into Hub RAM - but the principle is the same.
Am I misunderstanding what you mean?
Ross.
P.S. Maybe we should move this discussion over to the Catalina thread?
Seems like it would be a nice improvement in boot-up configuration to allow for a boot config script (think /sbin and /etc/init.d) for "products" as an alternative to the Catalina command line incantations. Of course there is nothing is wrong with a VB GUI to manage those options.
Using a config script could greatly reduce your maintenance burden (in the feature expansion sense where software tends to get broken) and make it easier for new developers to navigate your ocean of dependencies.
But all that's up to you of course
I plan to get back to porting SDRAM to Catalina after I have a reasonable ZOG demo. This week we are on manufacturing pilot runs for hardware ... so much to do, so little time.
Indeed. But, syscalls, handles, open, close, read, write, iocltls etc etc are abstractions provided by an operating system. We don't have an OS. Or even much of a boot loader at this point.
At this time Zog is one level down, at the OS or more like boot loader level. Or say the level of a typical C program running on a micro-controller with no OS just naked hardware.
It runs and it expects to find some IO ports for a console, a block storage device, perhaps a VMCog for external memory, perhaps even a float32 coprocessor.
Currently we have run_zog that loads all that stuff and provides a syscall interface. We could continue with expanding that. run_zog could start load and start up whatever you like and then provide access to it via syscall as open, close, read, write, etc.
This leads to a management nightmare, every platform is different, every application has different requirements. All combinations of platform/application may require a different configuration of debug_zog.
Alternatively we could build on run_zog. Currently run_zog provides no hardware interfaces at all. Just loads Zog and sets it running optionally killing off all Spin and giving all HUB to ZPU code.
As a minumum run_zog could load up all basic hardware as required such as:
a) FullDuplexSerial
b) SD card,
c) VMCog
d) Float32
At that point it can start Zog giving it all of HUB RAM. All space used by driver PASM code being recycled.
Problem is, at that point, how does the C code know where all the device I/O registers (mailboxes) are. And where is the video buffer etc.
These have to be built into both run_zog spin and the C code as constants. Or run_zog has to leave this information somewhere in HUB for the C code to find.
At this point I have no idea which way to move:)
https://mail.google.com/mail/?hl=fi&shva=1#inbox/12c28405562a1a7b
I wonder if I could make a tiny suggestion? How about you think about borrowing the rendevous points from catalina?
If these became a standard of sorts, then the zog people and the catalina people could work together on building cogjects. [insert big group hug]
Consider, as a hypothetical example, Kye's new 640x480 4 color driver. Right now this exists for spin, but if we took the pasm code, stripped out the supporting spin code and instead interfaced with this via known mailbox locations (in high hub ram), we could potentially use this driver for both zog and catalina.
Look, at this point I can't write a TV block based driver without syscalls at all because I can't even cheat and read/write HUB ram directly. Character based drivers are easy.
How do you expect to access rendezvous memory without direct access to any HUB RAM from C code?
You decide what you can support and we'll go that direction. Until then I'll produce what I can the easiest way I know to see what ZOG is capable of doing to demo possibilities. At the moment nothing is possible beyond console IO via serial port or keyboard/TV.
Also, How fast can your Triblade with VMCOG finish fibo(20) as measured after downloading from BST? I see 13.9 seconds total and fibo(20) takes 2190ms today at 80MHz with SdramCache. Just curious.
You have a good point. Bill Henning, I and others have been suggesting the idea of stand alone, Spin free, PASM driver modules for some time. Modules that can be used solely through a memory interface with no odd set up steps in Spin first. Bill has his "mailbox" approach. RossH has his "plugins". I have something less formal going on, for example Zog has a modified FullDuplexSerial with it's cyclic buffers exposed in the memory interface for use by C.
In this way PASM drivers can then be used by Catalina, Largos, Zog, or any other system that comes along.
The idea of filtching Catalina's drivers wholesale has made me nervous. Seems like cheating. However the way you put it, including the "group hug" makes me feel better. If there were "one way" to do this, from Catalina or not, it benefits everyone perhaps even encourages systems from others in the future that have not been invented yet.
So an issue becomes, how to define those rendevous interfaces. How to make their location known to C code running under Zog or whatever other code running under something else.
A solution could be Catalina's plugin approach but as yet I have no idea how that works and no idea if it is generally useful in other, as yet unknown, systems.
I was quite prepared to take a simplistic approach:
1) All drivers are loaded at start up. There is no support for dynamic loading/unloading at run time. After all you don't generally unload a UART on a normal micro-controller application. Such dynamic activity can be done in the application code if ever required.
2) All such drivers present a memory interface, registers, buffers etc, in high HUB space.
3) There is a memory map defined for those interfaces. Just like the special function registers in an MCU.
4) The selection of drivers loaded and the resulting memory map is application defined.
5) This allows for all PASM space in HUB to be recycled as data, stack, video buffer, whatever when the C code is up.
Feel free to filch! I have filched code from others as part of Catalina (although I always acknowledge where I filched it from).
Catalina's registry concept (which is the interface to all plugins) is not particularly complex and it could probably be improved a bit - but it has been good enough for everything I have needed to do with it to date.
If you come up with a better approach, you can be sure I will certainly feel free to filch it and use it in Catalina!
Ross.
Ah, good question.
The ZPU includes a SYSCALL instruction. As far as I can tell before Zog this was only ever used to get console I/O out of ZPU code whilst running under the Java ZPU simulator. For sure the provided C libs only support read/write on stdio using SYSCALL.
I came along and implemented the SYCALL instruction and the console I/O using it. Why?. Because.
Then I started adding those hooks for open, close, read, write, seek etc etc because they are all the SYSCAlL IDs defined in the ZPUs libs. Given that we have an SD card driver and a file system in Spin there is no reason not to continue down this path and a provide file system (and other devices) to C via sys calls. This may require a little work on the C library so as to take all the syscalls into use.
In this scenario we have C code in ZPU and "OS" in Spin. As we are using Syscalls the OS could be replaced at any time with something else.
This sounds like quite a reasonable approach, especially if the ZPU code is running from external RAM. and it has the advantage that a lot of application code can be run and tested in my C version of ZPU under Linux or Windows.
If you are keen on this approach I'm quite happy to pursue it.
If we go that way I'm going to drop all thoughts of running ZPU code from HUB only systems, at least for the time being.
Ah but you can:) As of Zog v1.6 ZPU code running from external RAM has been able to access HUB RAM.
The memory map is like so:
256MB ZPU RAM, followed by 32KB HUB RAM window, followed by 2KB COG RAM window.
Have a look for the definitions in ZOG:
zpu_hub_start long $10000000 'Start of HUB access window in ZPU memory space
zpu_cog_start long $10008000 'Start of COG access window in ZPU memory space
zpu_io_start long $10008800 'Start of IO access window
Warnings:
1) I probably have not tested this sufficiently.
2) The HUB window should be bigger, we may want to access the ROM, and what about Prop II?
3) It may change, due to 1) and 2) above or any suggestions you may have.
4) It only works for read/write of LONGS from HUB. I did not want to test the addresses on all possible memory accesses.
As I said, if you are keen on this approach I'm quite happy to pursue it.
Cheers. Consider it filched:)
All attributions, copyright notices etc will remain in place. It's more useful to me and others to know were things come from than to try and hide the origin.
It occurs to me that one would probably want read/write BYTE access for any video RAM. As Zog has a separate function for read BYTE as opposed to instruction fetch this can be done with minimal performance impact.
Beware of ZPU/Prop endianness issues.
Does ZOG/ZPU not have an ioctl syscall? Seems that we need it. As you know, ioctl allows for abstraction of non-buffer properties in a driver. For example, ioctl could be used to access to row, col, color, and screen in TV_Text.spin and friends by reference or value depending on what makes sense.
Let me know the approach you plan to use for associating file/device descriptor IDs with devices. I'll hard-code something for the time being. Console uses 0,1,2 ... I'll try using 3 or something for screen block driver.
I'm *not* advocating hard-coding file/device IDs as a permanent solution.
Well that helps a lot! Thanks for reminding me.
Increasing the HUB RAM virtual window to 128KB or even 1MB is not unreasonable. Hard-coding things bothers me, but do we have a choice? At least a common .h file could carry this stuff for C build.
Where is this separate function for read BYTE and how do I use it from C? Read/modify/write using long is probably sufficient for byte only data elements.
Are you having any luck with your cache problem?
@Heater,
I guess no one is using on_syscall from with external memory.
Consider this:
PRI on_syscall | p 'Handle CPU SYSCALL instruction ' Example C call: ' result=_syscall(&t, SYS_write, fd, buf, nbytes); 'Set a frame pointer into the ZPU stack (but HUB realtive) framep := zog_mbox_sp + @zpu_memory 'printStackFrame 'Perform system call function given by the ID param case long[framep + 8] SYS_exit: _exit SYS_open: _open SYS_close: _close SYS_read: _read SYS_write: _writeWhat's wrong with that? Well, zog_mbox_sp will contain an external
memory address which we can't access using HUB memory syntax.
Instead something like this needs to be done:
PRI on_syscall | p 'Handle CPU SYSCALL instruction ' Example C call: ' result=_syscall(&t, SYS_write, fd, buf, nbytes); #ifdef USE_JCACHED_MEMORY 'Set a frame pointer into the ZPU stack (but CACHE realtive) p := cm.readlong(zog_mbox_sp + 4) ' syscall ID offset at + 4 #endif #ifdef USE_VIRTUAL_MEMORY 'Set a frame pointer into the ZPU stack (but VMCOG realtive) p := vm.rdvlong(zog_mbox_sp + 4) ' syscall ID offset at + 4 #endif #ifdef USE_HUB_MEMORY 'Set a frame pointer into the ZPU stack (but HUB realtive) framep := zog_mbox_sp + @zpu_memory ' printStackFrame p := long[framep + 8] #endif 'Perform system call function given by the ID param case p SYS_exit: _exit SYS_open:But then there's the implementation of all the other syscall stuff. :<
Here's a new SdramTest.spin file that automatically adjusts the walking test size. Hopefully there is little else to change to support SPI RAM beyond the "mem" object. Do you get errors if you use the loop test all command "T a" ?
--Steve
Can you imagine how many lines of code would be necessary in _read to support 144 block devices for example? Of course with that many devices, just the long chain of #ifdef would be unbearable.
Too bad SPIN doesn't support inheritance and virtual methods. This of course is primarily why I mentioned the possibility of using kernel loadable modules "KLM" on the previous page.
I've been optimizing again while working on a ZOG TV_Text_Half_Height + Keyboard demo. I found that zog.spin cache_read routine was always using the WRITE_CMD parameter. While there's nothing wrong with that really, it just means all cache_reads were always writing back data unnecessarily. So, that was a drag on performance, and it is fixed in my private ZOG copy.
Another optimization is to use COG based tags instead of HUB tags. While this limits the burst size of cache lines, it also reduces the cache swap decision time by 2 more instructions than using HUB tags.
What's the impact of all this?
I had to use old numbers since people refuse to post anything new for some reason.
ZOG running from HUB could finish FIBO(26) in about 41 seconds according to an old Heater post.
The VMCOG performance I found was for 100MHz Triblade is fibo(20) at 2782ms.
Conclusions:
As far as I can tell without more up to date information ...
SdramCache is 25% faster than VMCOG for fibo(20) on a 20% slower Propeller
SdramCache is 3x slower than HUB on the same machine.
The equivalent SDRAM performance with VMCOG as the front end is impossible to achieve because 1) VMCOG's HUB interface is slower, and 2) there is not enough room for the 32 byte 10MB/s SDRAM read/write code.
Standardizing on VMCOG for external memory interface would be a mistake even if VMCOG could support 32MB+ of memory.
Integrating with Catalina should yield interesting results.
Cheers
--Steve