malloc/free works fine. But I guess I only tested it when running from HUB.
read/write_long need to functions, they are used to much to in-line, that eats all the COG.
read_byte is in-lined for speed as it is on the "critical path" as opcode fetch. Perhaps only of noticeable benefit hen running from HUB.
Therefore loadb/storeb got in-lined byte access as well.
read/write_word are functions because....well I don't know why, they are only used once each so it makes little difference.
Thanks for the explanation. I guess I understood why opcode fetch was inlined. I suppose the loadb and loadh stuff could both be inlined if neither function is needed elsewhere in the interpreter. Maybe I'll work on that.
jazzed: I'm trying to get your SdSerial driver working with my C3 ZOG environment and am having a bit of trouble. First, you load 520 longs into the COG in the TVText code you sent me but the COG can only really hold 496 longs, right? What are the extra longs for? Is there some sort of header on the COG image? Also, is any byte swapping necessary before calling SYS_cognew? I can't seem to be able to get the SdSerial driver to work and I'm wondering if I'm loading the COG image correctly.
The COG loader takes what it needs from the PASM blob. Having the extra longs is a mistake. Byte swapping is not necessary with the BSTC PASM extraction utility. Rather than using the blob that comes with FDSerial, It would be better to extract the blob using BSTC. I'll look at this in the morning.
What is the format of the data in FdSerial_Array? It looks like it is mostly just a dump of the COG memory image but there is an entry at the end that I don't understand. All of the other entries look like 32 bit hex numbers that are the data to load into the COG but the last entry is "0000554" which is interpreted as an octal number in C since it begins with a zero. What is that entry for?
Here is a working FdSerial library and test package.
Also here is an updated TvText and test package.
The files propeller.c/h and zog.spin have changed.
The problem with FdSerial is you can't have the ZOG console and FdSerial running simultaneously on the same pins. Of course you can use FdSerial on different pins.
I have not tested FdSerial_rx yet and FdSerial_rxtime needs a rewrite with a real system delay.
Here is a working FdSerial library and test package.
Also here is an updated TvText and test package.
The files propeller.c/h and zog.spin have changed.
The problem with FdSerial is you can't have the ZOG console and FdSerial running simultaneously on the same pins. Of course you can use FdSerial on different pins.
I have not tested FdSerial_rx yet and FdSerial_rxtime needs a rewrite with a real system delay.
Thanks for the updated files! Is there any reason I can't just shut down the ZOG console before starring ZOG? Once I get FdSerial working I'll use that for console I/O and abandon the syscall interface entirely. In fact, I'd like to stop the Debug_zog.spin COG right after I start ZOG.
Hmmm... I must have made some mistake in my mods to your FdSerial code. I tried using your new binary blob and my C3 version still doesn't work. I'm trying to set the baud rate to 115200. That should work, right? All I really did to your code is move the buffers into the FdSerial_t structure and I put that structure into a fixed area of hub RAM. I guess I also added an extra FdSerial_t parameter to every function so I could have multiple instances of the serial driver if necessary. What I get is lots of question marks instead of the correct characters.
Okay, I solved my own problem. I had forgotten to merge in your changes to ZOG.spin to add support for loadb/storeb to hub memory. I added that and then ran into another problem. My ZOG.spin was too big by seven longs. I temporarily fixed that by commenting out the code to handle external syscalls since I don't need them for either console or file I/O anymore and now my test program runs. I can now use open/close/read/write to do either SD card file I/O or console I/O and all of the code is written in C running under ZOG. Maybe I can try porting XLISP to the C3 now! :-)
I'm not sure what you mean by a fixed area of hub ram. It's better in this scheme if the data structure is used to dereference or "frame" the allocated memory. The buffer area is two contiguous parts and I suppose having two pointers at the end of the data structure is fine, but I decided to dereference them separately as a mirror of what is done in the PASM. Having a descriptor passed by pointer as the first argument to all functions is a good enhancement for multiple instances. The 115200 bps connection should work fine.
What I mean is that instead of using your more general huballoc/hubfree solution, I allocated a space the size of a FdSerial_t structure plus the buffers at the top of hub memory using some funky defines. I will probably try to port your hub memory allocator to C now that I have this simpler solution working. For some perverse reason I wanted to get all of the code running under ZOG without relying on any Spin code after bootup. If nothing else, it frees up a COG for something more interesting. :-)
I ran into the 7 long overrun problem too. As you can see, I just removed an SdramCache inline optimization to make room.
Hopefully you can release your source so I can see what you're doing and participate. Those #defines scare me though.
Of course, if the entire propeller memory is available to C, you can allocate it any way you like. Video graphics can eat HUB quickly, and I'll be finding out more about that very soon
Just in case you're interested in what I'm doing, here is my C3 test program. It doesn't do much but does verify that SD file I/O and console I/O is working okay.
I ran into the 7 long overrun problem too. As you can see, I just removed an SdramCache inline optimization to make room.
Hopefully you can release your source so I can see what you're doing and participate. Those #defines scare me though.
Of course, if the entire propeller memory is available to C, you can allocate it any way you like. Video graphics can eat HUB quickly, and I'll be finding out more about that very soon
Yeah, the defines are kind of a hack. I could get rid of most of them if I just port your huballoc/hubfree but some will still remain. I can't use huballoc to allocate the cache memory since ZOG can't even be started until that is allocated. So, at a minimum I'd have to allocate vm_mbox, vm_params, and vm_cache statically and then allocate the rest with huballoc.
All of hub memory is available once I start ZOG except for what is used for the cache. Unfortunately, that's a little over 8k and if you want to use the SD card filesystem another 512 byte sector buffer is needed. The serial driver takes another 72 bytes. I ended up hard allocating a 2k buffer that I use as both an SD card sector buffer and a staging area for loading code into COGs. That adds up to almost a third of hub memory already. :-(
Where is your FdSerial test program? How do you get the data structure pointed to HUB memory?
I was hoping you could post a ZOG.zip so I could run it with SDRAM.
Where is your FdSerial test program? How do you get the data structure pointed to HUB memory?
I was hoping you could post a ZOG.zip so I could run it with SDRAM.
The serial device is initialized by _initIO() that is in fileio.c. That is called during the initialization of the C runtime library indirectly by crt0.S. It just passes the address of one of the pointers I defined at the start of propeller.h (my funky hub memory map macros). I'll attach my ZOG.spin but I think the only thing I added to it was a new syscall to spin on a mailbox. I don't think you'll need that since your cache driver doesn't also handle the SD card. I posted my test program in a previous message. It's just standard C code calling the C runtime library for console and file I/O. What I haven't tried yet is using fopen/fclose/fread/fwrite for SD card files but I suspect it will work as long as I haven't broken malloc/free. My other changes also won't apply in your environment. I wrote a new linker script that allows me to place code in flash and I modified debug_zog.spin to be able to load the flash memory. That won't be needed in your environment.
Anyway, I'll zip up my entire work directory and post it so you can see everything. The only other change I made was to libgloss.a but that was only to support the code-in-flash and data-in-SRAM memory model.
What I'd like to work on next is a way to merge this stuff that supports stdio into libgloss.a so it doesn't have to be linked explicitly with every program. Maybe we need to find a way to use custom runtime libraries selected by a command line switch. I think one of the big contributions that RossH made was writing a driver program for LCC that bundles together all of the complexity of building a C program to run on the Propeller. We could benefit from that too.
Anyway, here's my code. Don't laugh too loud when you read it. The good parts are what I stole from you and others. The bad parts are my own invention! :-)
Also, I need to add back in all of the license stuff before this gets released officially. Obviously I will use the MIT license that Parallax requires. I just haven't had time to add that to all of the files.
Okay, to partially atone for my sins of inventing an arcane block of random variables permanently allocated to the top of hub memory I've developed a new cheezy way to handle variables that have to be in hub memory. First, I've eliminated all of the fixed hub memory allocations except for vm_mbox and vm_cache. Those can't easily be eliminated because they implement ZOG's external memory that must be in place before ZOG is started. The only way around that would be to load some code into hub memory at startup time and have that initialize the external memory interface. I may look at that when I get a chance.
The reason I say that my solution is cheezy is that I got lazy and didn't implement jazzed's huballoc/hubfree code in C. Instead I did the following:
From propeller.h:
#define HUB __attribute__ ((section(".hub")))
From fileio.c:
static HUB FdSerial_t serialData;
static HUB uint32_t vm_params[3];
HUB uint8_t scratch_buffer[2048]; // big enough for a COG image or a sector
This statically places variables in hub memory at compile time so there is no need for huballoc/hubfree. Well, that's not entirely true of course. Some allocations may be temporary. In my case, I wanted these structures statically allocated so just placing them in the ".hub" linker section works fine.
Oops. I just realized that by placing things in the ".hub" section, I've allowed the linker to allocate hub memory starting at location zero. That isn't really legal since many drivers expect location zero to contain the clock frequency. How many more longs should I reserve at the start of hub memory to avoid colliding with variables that PASM programs expect to find in low hub memory?
I guess it's possible to also have a hub heap since the linker addresses are visible. Are the HUB addresses assigned from the top down?
No, the linker assigns addresses starting at the base of the section. I placed my vm_mbox and vm_cache variables at the top because they have to co-exist with the Spin loader code from debug_zog.spin until the ZOG interpreter overwrites COG 0 and starts the C code. You're right though, we can have a hub heap. It will just start at _hub_end and go up to my fixed cache stuff at the top of hub memory. I'll look into porting your huballoc/hubfree stuff soon.
Guess I should have refreshed before hitting the reply button. Isn't there a reverse attribute for linker sections? One can always read the current stack pointer to find the end of spin code in use. As far as throwing anything else away it's guesswork unless the listing is used some way. I'll have to think about all this when I'm fresh in the morning. My allocator is simple and may not be what is used in the end product; t allocates from top down rather than bottom up..
Guess I should have refreshed before hitting the reply button. Isn't there a reverse attribute for linker sections?
I'm not sure about that. I don't really know that much about linker scripts. I only learned enough to put code into flash and allow access to hub and cog space from C code.
One can always read the current stack pointer to find the end of spin code in use. As far as throwing anything else away it's guesswork unless the listing is used some way. I'll have to think about all this when I'm fresh in the morning. My allocator is simple and may not be what is used in the end product; t allocates from top down rather than bottom up..
I'm not worried about throwing anything away since there is no Spin code running once ZOG has started. All of hub memory is available for the C code and any COGs that might be loaded by the C code. I think allocating from the bottom up would be fine. I looked at your allocator a little but don't really understand it. There are comments about garbage collection. Do you really do that?
Oops. I just realized that by placing things in the ".hub" section, I've allowed the linker to allocate hub memory starting at location zero. That isn't really legal since many drivers expect location zero to contain the clock frequency. How many more longs should I reserve at the start of hub memory to avoid colliding with variables that PASM programs expect to find in low hub memory?
Here I am talking to myself again. :-)
I decided to just allocate a single variable CLKFREQ at the start of hub memory. Turns out to be pretty easy to do using the linker script. Andre' is wanting me to port his tile graphics driver to C now so I guess I will need to get huballoc/hubfree written.
Does ZOG.spin use the parameter passed to coginit once it has started? I just realized that I assume that ZOG has total control of hub memory after starting but that won't be true if it still needs access to its own parameter block. Does it ever access the memory pointed to by par after initialization?
Does it[ zog ] ever access the memory pointed to by par after initialization?
No, not the actual PAR block. All parameters and pointers should be held internal to the COG after start up.
The only HUB it should not touch is is the dispatch table. run_zog moves that to the very top of HUB prior to starting the COG in order to keep it out of the way when taking over the whole Prop.
If external memory is in use the memory mail box needs to be not trampled on .
Hmmm... I didn't know about the dispatch table in hub memory. I guess I'm in danger of overwriting it unless I place it at the top of hub memory. I guess it will have to join vm_mbox and vm_cache as fixed locations at the top of hub memory. Thanks for pointing that out. I guess I also have to worry about what happens if a breakpoint instruction is executed since I don't have any COG waiting for those events once I overwrite debug_zog.spin.
Ah ya, I've always considered a breakpoint without any debug going on as a fatal crash. Perhaps not if you have other things going on.
Can't the breakpoint instruction be handled as an interrupt like it is on conventional processors? That way we could write an interrupt handler that would at least display the PC to the serial port so we could tell where the breakpoint occurred.
I think I'm right in saying that Zylin have not defined BREAKPOINT for the ZPU apart from how it is used in their Java ZPU simulator. So we can do what we like with it.
Comments
Thanks,
David
Here is a working FdSerial library and test package.
Also here is an updated TvText and test package.
The files propeller.c/h and zog.spin have changed.
The problem with FdSerial is you can't have the ZOG console and FdSerial running simultaneously on the same pins. Of course you can use FdSerial on different pins.
I have not tested FdSerial_rx yet and FdSerial_rxtime needs a rewrite with a real system delay.
Thanks for the updated files! Is there any reason I can't just shut down the ZOG console before starring ZOG? Once I get FdSerial working I'll use that for console I/O and abandon the syscall interface entirely. In fact, I'd like to stop the Debug_zog.spin COG right after I start ZOG.
Here is output from test/FdSerial/main.bin
Here is my funky fixed hub memory map:
Here is the data structure I ended up with:
typedef struct FdSerial_struct { int rx_head; // receive queue head int rx_tail; // receive queue tail int tx_head; // transmit queue head int tx_tail; // transmit queue tail int rx_pin; // recieve pin int tx_pin; // transmit pin int mode; // interface mode int ticks; // clkfreq / baud int buffptr; // pointer to rx buffer char rxbuff[FDSERIAL_BUFF_MASK+1]; // receive buffer char txbuff[FDSERIAL_BUFF_MASK+1]; // transmit buffer int cogId; //cog flag/id } FdSerial_t;I ran into the 7 long overrun problem too. As you can see, I just removed an SdramCache inline optimization to make room.
Hopefully you can release your source so I can see what you're doing and participate. Those #defines scare me though.
Of course, if the entire propeller memory is available to C, you can allocate it any way you like. Video graphics can eat HUB quickly, and I'll be finding out more about that very soon
#include <unistd.h> #include <stdio.h> #include <fcntl.h> int main(void) { int ifd, ofd, count; char buf[1024]; iprintf("OpenFile input:\r\n"); ifd = open("in.dat", O_RDONLY); iprintf(" -->%ld\r\n", ifd); iprintf("ReadFile:\r\n"); count = read(ifd, buf, sizeof(buf)); iprintf(" -->%ld\r\n", count); iprintf("contents: '"); fwrite(buf, count, 1, stdout); iprintf("'\r\n"); iprintf("OpenFile output:\r\n"); ofd = open("out.dat", O_WRONLY | O_CREAT); iprintf(" -->%ld\r\n", ofd); iprintf("WriteFile:\r\n"); count = write(ofd, buf, count); iprintf(" -->%ld\r\n", count); iprintf("Done\r\n"); return 0; }All of hub memory is available once I start ZOG except for what is used for the cache. Unfortunately, that's a little over 8k and if you want to use the SD card filesystem another 512 byte sector buffer is needed. The serial driver takes another 72 bytes. I ended up hard allocating a 2k buffer that I use as both an SD card sector buffer and a staging area for loading code into COGs. That adds up to almost a third of hub memory already. :-(
Here are all of my files.
I was hoping you could post a ZOG.zip so I could run it with SDRAM.
Anyway, I'll zip up my entire work directory and post it so you can see everything. The only other change I made was to libgloss.a but that was only to support the code-in-flash and data-in-SRAM memory model.
What I'd like to work on next is a way to merge this stuff that supports stdio into libgloss.a so it doesn't have to be linked explicitly with every program. Maybe we need to find a way to use custom runtime libraries selected by a command line switch. I think one of the big contributions that RossH made was writing a driver program for LCC that bundles together all of the complexity of building a C program to run on the Propeller. We could benefit from that too.
Anyway, here's my code. Don't laugh too loud when you read it. The good parts are what I stole from you and others. The bad parts are my own invention! :-)
Also, I need to add back in all of the license stuff before this gets released officially. Obviously I will use the MIT license that Parallax requires. I just haven't had time to add that to all of the files.
The reason I say that my solution is cheezy is that I got lazy and didn't implement jazzed's huballoc/hubfree code in C. Instead I did the following:
From propeller.h:
#define HUB __attribute__ ((section(".hub")))From fileio.c:
This statically places variables in hub memory at compile time so there is no need for huballoc/hubfree. Well, that's not entirely true of course. Some allocations may be temporary. In my case, I wanted these structures statically allocated so just placing them in the ".hub" linker section works fine.
I decided to just allocate a single variable CLKFREQ at the start of hub memory. Turns out to be pretty easy to do using the linker script. Andre' is wanting me to port his tile graphics driver to C now so I guess I will need to get huballoc/hubfree written.
The only HUB it should not touch is is the dispatch table. run_zog moves that to the very top of HUB prior to starting the COG in order to keep it out of the way when taking over the whole Prop.
If external memory is in use the memory mail box needs to be not trampled on .
Ah ya, I've always considered a breakpoint without any debug going on as a fatal crash. Perhaps not if you have other things going on.
I think I'm right in saying that Zylin have not defined BREAKPOINT for the ZPU apart from how it is used in their Java ZPU simulator. So we can do what we like with it.