PDA

View Full Version : Post Boot PASM COG Loader?



jazzed
01-15-2009, 03:58 AM
Anyone created boot loader that would not include PASM until after startup?

The PASM code would have well defined interfaces and be compiled independent
of Spin. Spin would have small API wrappers as is typically used already for
PASM interface. The difference would be not having to carry it all in the binary.

It seems that using such an approach would let most of 32K Hub memory be
dedicated to Spin only except for maybe SDcard code. Having all the 32 bit per
PASM instruction code·in the binary seems quite wasteful.


▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
--Steve

heater
01-15-2009, 04:53 AM
Funny you should ask that. I was just pondering ways of reclaiming the 2K byte space taken up by the PASM 8080 emulator code. Currently this loaded once from the binary and hence from the RAM and thereafter constitutes a huge waste of HUB RAM given that I never want to stop and restart it. Same goes for all the other bits of PASM.

Now I can fiddle with my code and arrange for the PASM emulator code to sit where the emulated 8080 expects it's RAM to be when it runs thus reclaiming the space. And I probably will one day, but it messes up the code structure and makes it harder to maintain.

Some way of keeping the PASM part of the binary on SD card or other EEPROM would be great. But how to compile it? The spin tool needs some kind of "memory segments" idea so that you can tell it where to put things and still have them linked nicely against the SPIN parts.

In the worst case we have 8 * 2 = 16K of our HUB RAM wasted in "dead" PASM code. 50% !!!

Ideally the boot EEPROM would have been twice as big (is/was there such a thing?) and the SPIN tool would have put the PASM parts in the extra space when told to.

Perhaps mpark or BradC could add such a "segment" feature like this to their already wonderfull compilers to allow us to pull out the PASM blobs into SD card or wherever we like.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.

Carl Jacobs
01-15-2009, 05:35 AM
I've put a fair bit of thought into this, but not yet found the time to write the specific loader code.

I agree with heater that possibly the best place to put the PASM parts is in the top 32K of a 64K EEPROM (you could even use the top section of the bottom 32K).

Consideration needs to be given to how the PASM code is written. All parameters passed to the assembly COG need to be loaded via the PAR register, otherwise the code relocation becomes more of a challenge. The following example is bad for relocatable code:



PUB Start(baudrate)
...
aBaud := CLKFREQ / baudrate
...

DAT
...
aBaud long 0
...



The following is much better:



VAR
LONG baud_rate

PUB Start(baudrate)
...
baud_rate := CLKFREQ / baudrate
cognew(@entry, @baud_rate)
...

DAT
...
entry rdlong aBaud, PAR
...
aBaud res 0
...




All of my recently written code has been written in this way for when I eventually write a PASM loader/linker/mapper tool. My recently submitted serial port object discussed in another post follows these principles for this reason. In that object I was trying to save even *more* RAM by having the FIFO buffers in the COG.

If the 64K of EEPROM is divided into 2K segments, then there are a total of 32 segments, of which probably at most 6 will be used in the typical large application - this still represents a saving of up to 14K of RAM!! although in reality none of the PASM COGs are ever all that full, so the real saving will be more like half of that.

The whole idea of having segments, is that it is probably easier to manage the linker if you know that you'll be using segments 0 to 6 which then get hardcoded into the spin file. It would probably be worth including a simple checksum as part of the PASM image to prevent the loading and execution of non-existent code.

Regards,

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Carl Jacobs


JDForth - Forth to Spin Compiler http://www.jacobsdesign.com.au/software/jdforth/jdforth.php
Includes: FAT16 support for SD cards. Bit-bash Serial at 2M baud. 32-bit floating point maths.·Fib(28) in 0.86 seconds. ~3x faster than spin, ~40% larger than spin.

jazzed
01-15-2009, 05:36 AM
Ya, I figured I wasn't the only one to consider this :)

Compile the file the ordinary way and exctract the PASM. After that it's simple. I've done this often with C.
Load the binary into a Propeller buffer and cognew with the address and the user parameter buffer.
I guess one needs a generic 2K byte buffer to get started; reuse space from the load device DAT section ....
Some way to check the interfaces would be nice, but we have to do that by hand today anyway.

I was thinking that just about any function that uses local·data could be made in PASM and called by Spin.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
--Steve

Post Edited (jazzed) : 1/14/2009 9:45:57 PM GMT

Phil Pilgrim (PhiPi)
01-15-2009, 05:57 AM
You can always reuse the hub's DAT space for data, buffers, stacks, etc., once the ASM code has been "encogged". For example, a serial object's FIFO buffer could easily share the hub RAM occupied by its assembly code. Basically, you just pass the hub address of the code itself via the PAR register when issuing a COGNEW, and let the assembly startup routine initialize it as a buffer area. Of course, this works only if the driver is started once, from a single object.

If this is not the case, the driver could act as its own bootloader, reloading itself directly out of EEPROM. For this method, you'd pass a the address of the long just before the code. This long is initialized to zero and points to the last assigned buffer space. In the assembly initialization code, if it reads a zero, it assigns it a value of PAR + 4 * BootloaderLongs + 4. Otherwise, it just increments it by the size of the buffer(s). It can always find itself in EEPROM at PAR + $8004 and begin loading from PAR + $8004 + 4 * BootloaderLongs.

-Phil

heater
01-15-2009, 06:19 AM
Phil, I know what you mean but doesn't that start to make spaghetti out of your code. Objects that need RAM space then need to know where other, totally unrelated objects PASM may be. Mixing and matching objects becomes a mess.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.

jazzed
01-15-2009, 06:29 AM
Phil, the approaches you describe are fine (and i've used the first), but niether one buy more code space for Spin interpretable code. Edit: actually, your second approach would buy spin space if the driver for the load target can be fully enclosed in one generic PASM loader so that all drivers or other support code could be loaded to different cogs. Then, you would have the luxury of loading 7 cogs and could reclaim the original loader area for other purposes if you don't need it. Sweet :)

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
--Steve

Post Edited (jazzed) : 1/14/2009 10:40:21 PM GMT

heater
01-15-2009, 06:42 AM
Hmm, so what we want is:

a) bootloader that is the only code that ever gets blown into EEPROM.
b) boot loader is probably written in PASM and loads PASM blobs to cogs and SPIN code to HUB.
c) Loader can be customized to use something other than SD card.
d) PASM blobs have no linkage to SPIN VAR and DAT variables except via a pointer passed in PAR
e) The bootloader should be able to fetch new SPIN/blob packages from a PC to put into SD card as per the Prop loader.

Some how in b) the loader needs to be able to load it's own COG with new code as a last step.

Some how COGS don't want to start their processing until all COGS and SPIN has been loaded. Perhaps they could wait on their PAR registers becoming non-zero.

This requires that PASM blobs are written with loader awareness i.e. only PAR linkage and start up procedure.
And that the SPIN parts provide the other end of that loader awareness to set PAR when the COGs should RUN.

Now all we need is a compiler that will spit out separate PASM binary blobs and SPIN for the loader to fetch from the PC.

Anything else ?

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.

jazzed
01-15-2009, 06:56 AM
Items b - e make sense. I was thinking a small snippet of Spin could manage the loading to simplify
things, etc in less than 20 lines + data.

I don't think we have to go as far as a) to have access to more executable spin byte space.
The bootloader only needs to be in one pasm section.
One loader variant per project specified at compile time: EEloader, SDloader, SerialLoader, etc...).
Bootloader does not have to send spin to HUB and start it, though that is an interesting feature that could boot "anything".
We don't need a compiler, I have a tool that automatically extracts PASM. I'll find it ... it's posted here somewhere.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
--Steve

Carl Jacobs
01-15-2009, 06:59 AM
heater, a couple of comments:

Extraction of PASM images from EEPROM (as opposed to SD) is probably going to be a sufficient start. Not that SD is hard, but it does require extra pin configurations. EEPROM access is pretty much stock-standard.

The only way to load PAR is by using cognew - so I guess what you're saying is wait for the address pointed to by PAR to become non-zero.

My normal method for a sychronised start up is exactly that, i.e. start when the time is ??? implemented quite simply with WaitCnt. I'm sure that most loader processing is going to take less than 53 seconds.

Regards,

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Carl Jacobs


JDForth - Forth to Spin Compiler http://www.jacobsdesign.com.au/software/jdforth/jdforth.php
Includes: FAT16 support for SD cards. Bit-bash Serial at 2M baud. 32-bit floating point maths.·Fib(28) in 0.86 seconds. ~3x faster than spin, ~40% larger than spin.

Mike Green
01-15-2009, 07:04 AM
The Propeller OS that I wrote a while ago was designed just this way. The "main" program included all the I/O drivers and the command interpreter. If the I/O drivers were not running, the program loaded and initialized them. "User" programs included only the Spin API for the drivers (keyboard, either TV or VGA display, and I2C driver). The drivers and the Spin programs communicated through common areas in the high end of hub ram. The low level SPI driver was added to the I2C driver later. It should be possible for the "main" program to include Rokicki's SD card routines that would be used to look up (and possibly create) some specific files and stash their absolute SD card addresses in a work area so that the low level routines could access them with minimal memory impact.

heater
01-15-2009, 07:08 AM
Well what I'm imagining is that we basically replace the Props native boot loader with a new one that fetches a file containing concatenated 8 * 2K PASM blobs and 1 * 32K SPIN blob from your PC. This file is the output of the compilation process by whatever means. It can use some nice high speed serial protocol to do this. This downloaded file is placed in SD card. The loader can be customized to use different storage, baud rates, serial protocol, clock settings etc. etc.

Now if there is no file to be downloaded from the PC it starts to fetch the file off the SD card, loading everything into the right place. and then starting the main SPIN method. Not sure how the SPIN objects then tell their COGs they can start now.

Problem is that all those nice OBEX objects would have to be modified to work with this scheme....

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.

jazzed
01-15-2009, 07:20 AM
Mike can you post a link to your work so it can be evaluated ?

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
--Steve

Kye
01-15-2009, 07:22 AM
You guys should be care not to overcomplicate things. Rewriting clever code is good... but the track you guys plan on getting on will defeat the purpose of using the wonderful propeller chip.

If you think your running out of ram then triming your code of all unessential elements will help, not hurt.

It's suprising how much space is wasted in some code...

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Nyamekye,

heater
01-15-2009, 07:29 AM
OK my solution may be a bit over the top and complicated. It's would be of use in some some extreme cases where one has a lot of SPIN code, nearly 32K, and little data. In which case the EEPROM would then be stuffed with PASM code and no place for the SPIN you want.

In a case like my emulator really I have as little SPIN as possible, I want lots of free RAM for the emulators 8080 RAM space.

As it stands with the Prop tool I have a module/object containing the emulator PASM which is wasting space. I have another object/module that defines the RAM content for the programs emulator is to run and free RAM space. Basically the emulated CPUs memory map.

To reclaim the PASM space I would have to put the emulator PASM into the memory module somewhere. Then that space would be reused when the emulator runs. This is horribly messy.

Now. It occurs to me that if I could extract the emulator PASM from the Prop tools binary I could then put that binary into a file. And then in a separate simulator build I could place a "file" statement into the memory map object to include my emulator PASM code. Bingo I have reused the memory in a very neat way. I only have to get the PASM blobs address out of the memory module and use it to start the emulator COG.

So whoever has the PASM extractor please let me know how to do this.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.

heater
01-15-2009, 07:34 AM
Kye, I agree with you. I like simple and elegant. And I'm not into rewriting a lot of existing stuff that works.

But in normal use of the Propeller tool it is wasting space with every piece of PASM code you have. Unless you take steps to place your PASM code in places that will later be used as variable/buffer storage.

The Prop tool sadly does not support such placement of code or overlaying of code/data areas.

So we have to come up with some tricks or end up with spaghetti code.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.

Kye
01-15-2009, 08:02 AM
Hmm, maybe it would be possible to use a 64k eeprom with a 512 long buffer in memory and an I2C object to squentially grab pasm code. Put into the buffer, and then load it up, and then repeat for other cogs.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Nyamekye,

Cluso99
01-15-2009, 08:55 AM
Its all doable, and not that complex. http://forums.parallax.com/images/smilies/smile.gif

Need to find the best method to load from anywhere and create a stub that exists in hub. Just on the way out, so will comment later.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Prop Tools under Development or Completed (Index)
http://forums.parallax.com/showthread.php?p=753439

My cruising website http://www.bluemagic.biz

jazzed
01-15-2009, 09:34 AM
Good or bad almost all my projects need more memory :) My current Prop project has 380 longs left ...
barely enough for stack I think, and I cringe at the moment when I start hitting stack problems ... again.
It's hard to do anything truly useful with a 32K Byte limit. I only need two PASM drivers, so at least
12KB is wasted. Being able to use any extra on-chip space for instructions in any way at all helps.

@Heater, The program I mentioned produces a C array, but I'm sure you could easily make a PASM
binary with some source changes ... be my guest, just post back. Source is here:
http://forums.parallax.com/forums/default.aspx?f=25&m=287818&g=294044#m294044

@Mike, I looked and it looks like I'm forced use your O/S to harvest extra memory. Not what I wanted.
I want a stand-alone solution. It was kind of you to bring it up though. Please explain if I misunderstand.
Thanks.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
--Steve

Cluso99
01-15-2009, 12:02 PM
Has anyone looked at the PropLoader and code in the Prop that burns to EEPROM. What I am thinking is to just have the two compilers modified to allow compilation up to 64KB (an option). If required, modify the loader so that the EEPROM is loaded with this 64K, instead of 32K at present.

Now, the writers can place the pasm code above 32KB, and the compiler will accommodate this. There will be some restrictions in that any addresses referred to which are >32KB will be errors, so only pasm DAT would be able to be placed here (for now anyway).

So now all that is needed is a pasm stub that can load the pasm code from EEPROM above 32KB. Par will remain as always, below 32KB.

This method is simple and can be expanded later to load from other places, but lets get this going first.

What do we need...

1. Ask BradC (bst)·and MPark (Homespun) to allow option to compile above 32KB.

2. Check if the Prop Loader and burn code will burn 64KB, else create new object to do this.

3. Write stub code for pasm loading above 32KB - I can do this with zero cog footprint, so all cog memory available. For this to work, I will need about $30 longs of lower hub ram (below 2KB) and probably a 2KB buffer (or maybe much smaller) to load from EEPROM.

Any thoughts???


▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Prop Tools under Development or Completed (Index)
http://forums.parallax.com/showthread.php?p=753439

My cruising website http://www.bluemagic.biz

Post Edited (Cluso99) : 1/15/2009 8:14:07 AM GMT

heater
01-15-2009, 01:09 PM
@kyw, just to reiterate briefly. The real problem is that from the 32K EEPROM all the way up to the PropTool we cannot get 32K + 8 *2K = 48K of code into a Prop. This is one third of the chips cabability and really needs fixing or working around.

@jazzed, That tool sounds useful, I will check. Becomes all the more urgent for me when I move to using 4 COGS for emulation,

@Cluso99, Your proposal appears to be quite sound. I hope BradC and mpark see this:)

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.

jazzed
01-15-2009, 01:49 PM
@Cluso99. It would be nice to have a fully integrated solution in the compiler. I hope there are not constraints other than compiler talent or time :)

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
--Steve

heater
01-15-2009, 01:51 PM
Looks like Cluso99 had exactly that in mind, where is BradC and mpark ?

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.

Cluso99
01-15-2009, 04:46 PM
Ok, I have checked the Booter code...

The PropLoader can only load 32KB from the pc to hub ram. Anything less than 32KB will be zero filled.·If eeprom is selected,·it will·burn the full 32KB from hub ram to eeprom.

So, the compiler will need to output a seperate file for the code above 32KB and a special Loader will need to be used to load this from the pc to eeprom. There is nothing complex about this, just that it has to be done.

I will have a think about the prop side code.

Michael and Brad - any comments?

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Prop Tools under Development or Completed (Index)
http://forums.parallax.com/showthread.php?p=753439

My cruising website http://www.bluemagic.biz

BradC
01-15-2009, 05:01 PM
Cluso99 said...

Michael and Brad - any comments?


Certainly none from me. Aside from some special code in the prop to load the second half of the eeprom I don't see anything that requires assistance from the compiler really.
From what I see you are effectively loading the top half of the eeprom with PASM that you will need to page in 512 longs at a time to load cogs. Is there something else I'm missing?

If you can come up with a concrete proposal and requirement I don't have any problems at all modifying the compiler to suit and working with the loader to do the job.

I already have the ability to load the second half of the eeprom with my USB HID bootloader (which requires the top 16 longs of ram, 2 cogs and 3 port pins), so it's not that hard to do. You could probably modify chips boot code to load ram / burn low /verify low / load ram / butn high / load high, and modifying the loader to do this would not be difficult at all.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Cardinal Fang! Fetch the comfy chair.

heater
01-15-2009, 06:14 PM
BradC,

Isn't it just that a compiler should be directed to place one or more or all of the PASM blocks from objects into the high end of the binary file each in their own 512 long space. Rather than have them lying around at random within the file.

Of course the NOT PASM code i.e. actual DAT data still needs to be in the low 32K as normal so that Spin can find it. How do we know which bits are code and which bits are data ? Especially when writing LMM.

The user should some how specify what goes where.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.

Carl Jacobs
01-15-2009, 06:41 PM
heater said...
The user should some how specify what goes where.

This was my thought on it which is why I suggested numbering the memory blocks. Since we can't use the bottom 32K it makes sense that 32K is block 0, and block n is 32K + 2K x n.

The way I would see this as working is that block 0 would always get a copy of (or a variation of) FullDuplexSerial. Everytime I load my code I wouldn't bother with the top 32K - it's debugged code that just needs to be used.

Development of new PASM COGs would occur in the usual manner that we are all accustomed to without the requirment for any special tools. All thats needed then is a file something along the lines of:


CONFIG.SPIN
CON
FullDuplex = 0
VGA = 1
PS2 = 2

FullDuplexSerial.spin
OBJ
cfg : "config.spin"
ee : "eepromloader.spin"

VAR
long cog 'cog flag/id
long rx_head '9 contiguous longs
long rx_tail
long tx_head

PUB Start(rx, tx, mode baud) : cognum
...
okay := cog := ee.LoadCog(cfg#FullDuplex, @rx_head) + 1
...




Something like that would do me just fine. Debug and develop as normal, then burn to an upper page for production use.

Regards,

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Carl Jacobs


JDForth - Forth to Spin Compiler http://www.jacobsdesign.com.au/software/jdforth/jdforth.php
Includes: FAT16 support for SD cards. Bit-bash Serial at 2M baud. 32-bit floating point maths.·Fib(28) in 0.86 seconds. ~3x faster than spin, ~40% larger than spin.

Cluso99
01-15-2009, 08:22 PM
Brad: The compiler needs a new option "HubObj $8000" to move the hub pointer up to the next 32KB. It also needs to compile code up to $FFFF (64KB). It would look as though the Prop now has 64KB of ram to the compiler. The programmer would be responsible for ensuring that only cog pasm code was placed above the 32KB limit. http://forums.parallax.com/images/smilies/cool.gif· I think this may only be a simple mod to the compiler.

There will need to be a new PropLoader which will load > 32KB programs, and of course there will need to be code in the prop, both for loading and storing in eeprom, and for loading cogs from the upper 32KB eeprom bank. I am happy to do the Prop code (download and write to eprom and run loader to load from upper 32KB to cog). It will also require the zero footprint code I use in my debugger to load the cog code from the upper 32KB eeprom. http://forums.parallax.com/images/smilies/cool.gif

Is anyone interested in writing the new PropLoader?··http://forums.parallax.com/images/smilies/eyes.gif I can write the spec if required. It would be preferable to be windoze/linux/mac compatable (I'm windoze only).

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Prop Tools under Development or Completed (Index)
http://forums.parallax.com/showthread.php?p=753439

My cruising website http://www.bluemagic.biz

BradC
01-15-2009, 08:36 PM
Cluso99 said...

Is anyone interested in writing the new PropLoader? http://forums.parallax.com/images/smilies/eyes.gif I can write the spec if required. It would be preferable to be windoze/linux/mac compatable (I'm windoze only).


Umm hell yeah, I'll write it no problem at all. You give me a spec and prototype and I'll make sure it works across the platforms..

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Cardinal Fang! Fetch the comfy chair.

Cluso99
01-15-2009, 08:50 PM
Brad - did you write the proploader for linux and mac?

Spec by PM. I'll get started my end asap.

Do you want to zero fill up to the 32KB ($7FFF) if your compiler gets the "HubObj $8000" option/line? Would make the loading program easier - I am not sure of the binary or hex formats.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Prop Tools under Development or Completed (Index)
http://forums.parallax.com/showthread.php?p=753439

My cruising website http://www.bluemagic.biz

Post Edited (Cluso99) : 1/15/2009 1:13:50 PM GMT

heater
01-15-2009, 09:25 PM
Shouldn't we make that "huborg $8XXX" ?

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.

BradC
01-15-2009, 09:30 PM
Cluso99 said...
Brad: The compiler needs a new option "HubObj $8000" to move the hub pointer up to the next 32KB. It also needs to compile code up to $FFFF (64KB). It would look as though the Prop now has 64KB of ram to the compiler. The programmer would be responsible for ensuring that only cog pasm code was placed above the 32KB limit. http://forums.parallax.com/images/smilies/cool.gif I think this may only be a simple mod to the compiler.



I'm not entirely sure this is going to be as easy as you think its going to be.

I'm assuming that you are talking about moving PASM code (and ONLY PASM code) that is exclusively loaded into cogs up above $7FFF. Remember that the DAT section in each object starts at the beginning of the object, so any data or variables you have in the DAT section *must* be below $7FFF.
If you use a "HubObj $8000", what precisely do you want to occur?
Compiling hub code is all well and good, and your code will start at $8000, but how do you launch that cog from Spin then ?

cognew ($8000, @par) is going to try and do funky things, unless you plan on subverting coginit/cognew in the interpreter. Of course then you can't simply use cognew/coginit from PASM either.

Alternatives do exist. Carl wrote a serial object that loads the cog and then uses the code space for buffers. I have written a TV object that loads a cog then uses the code space for line buffers.. you just need to be a little clever. I get the idea you want to store your cog binary code above $7FFF and then swap it in 512 longs at a time, boot a cog from it and then move on to the next one. How do you actually plan on using it ?

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Cardinal Fang! Fetch the comfy chair.

Cluso99
01-15-2009, 09:51 PM
Brad: Yes you raise something I missed - the headers around the dat object. Will have to think that one out a bit more. http://forums.parallax.com/images/smilies/confused.gif

Basically what you said (offline) about the load sequence makes sense - I think we are both on the same page, just expressing it slightly different·with a slight variation.

I thought a fixed program (issued in binary form file) which would be loaded into ram and executed to perform the replaced booter code in the prop. Then the first 32KB would be loaded to hub and stored in eeprom, then the final 32KB would be loaded into the upper 32KB eeprom, then the prop would be reset and the prop would boot the new eeprom code which includes a facility to load upper 32KB sections into cog. If the feature was not used, then the code would be loaded as normal.

Heater: I couldn't see any reason not to make it exactly $8000. Do you have a reason? The current booter actually fills the full 32KB with zeros, so it is probably easier for the compiler to zero fill to $8000 if it finds this line. But I guess it doesn't have to be $8000 either but somehow we are going to need to overcome the issue of the object headers (which I forgot).

There is no need for the cog code to be exactly 2KB, it can still be overlapped. I was not thinking of a modified ram interpreter, but rather a call to a stub loader passing par and the actual cog eeprom address. You see, it is necessary to get the cog's par to be set correctly, and it cannot be referring to code >32KB.

Hope this makes sense.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Prop Tools under Development or Completed (Index)
http://forums.parallax.com/showthread.php?p=753439

My cruising website http://www.bluemagic.biz

heater
01-15-2009, 10:05 PM
I'm imagining this, add a DATH section type (instead of DAT) for PASM code to be in the high memory or call it whatever.
Then in there use ORG xxx to specify where in high memory it goes

DAT ' Normal DAT section
org 0 'Normal org
bla
bla

DATH 'DAT section to be loaded high

org $8xxx 'Where to put it in high

mov bla,bla

fit
.
.

Now whenever the compiler sees this it places the code in the correct user specified high location. Thus making otherwise used HUB space free for more SPIN code/data.

User code then is responsible for loading that PASM from the high end of the EEPROM and getting a COG started.

With a bit of simple post extraction users could take those PASM parts out of the binary for use in SD card or whatever as well.
This could also be used for blobs of constant data, like tables and strings, it is a kind of DAT section after all.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.

Cluso99
01-15-2009, 10:12 PM
I am just wondering what the object headers are being used for in the pasm code, other than to step through the spin and var objects.

I seem to recall the compiler places local variables at the end of the code. Brad, is this correct? This will still need to be done below the 32KB limit, so it complicates the compiler.

However, I cannot see any use for the headers by the DAT COG objects. Certainly, the Interpreter does not use these for loading cogs with pasm code, only for spin code. For pasm, the par register is loaded with a constant that can be anything and the pasm code knows what to do with it (i.e. if it is an address for parameters, etc then it must be in lower 32KB, otherwise it is a constant which does not matter)

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Prop Tools under Development or Completed (Index)
http://forums.parallax.com/showthread.php?p=753439

My cruising website http://www.bluemagic.biz

jazzed
01-15-2009, 10:27 PM
The headers are just spin packaging that are easily removed. Object vars go in the section before the stack and are specified in the headers.

Added: ·@Heater, Having any luck extracting·PASM to a binary?· I'll do it if you like; give the word ....

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
--Steve

Post Edited (jazzed) : 1/15/2009 5:35:30 PM GMT

Cluso99
01-15-2009, 10:33 PM
Steve, I was trying to simplify the compiler extensions and make the user responsible for where (s)he places the dat code rather than the compiler having to determine this.
This way, the user can choose which cog pasm code is loaded in lower and which is loaded in upper.

However, Brad raised the point about the object headers and these cannot go past the 32K boundary. These are the headers that the spin interpreter uses to locate global and local variables, pub and pri calls, etc. It steps through these headers. I do not have a much of an understanding about these. Best place to look at them is a homespun listing.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Prop Tools under Development or Completed (Index)
http://forums.parallax.com/showthread.php?p=753439

My cruising website http://www.bluemagic.biz

jazzed
01-15-2009, 10:38 PM
Ah spin boundary issue ... of course. Never ending story :) Which development boards have 64K eeprom?

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
--Steve

Cluso99
01-15-2009, 11:15 PM
It's not a show stopper, just something to be overcome as simply as possible.

My Prop Proto Boards (non USB) have 64KB eeproms. I believe so has the USB version. Don't know about the others.

Heater said... User code then is responsible for loading that PASM from the high end of the EEPROM and getting a COG started.
This needs to be done for the user automatically, and yes it could be expanded later to get it from anywhere. It need to be kept simple for the user.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Prop Tools under Development or Completed (Index)
http://forums.parallax.com/showthread.php?p=753439

My cruising website http://www.bluemagic.biz

Phil Pilgrim (PhiPi)
01-16-2009, 01:26 AM
heater said...
DATH 'DAT section to be loaded high

org $8xxx 'Where to put it in high

Don't forget that the real purpose of org is to set the PC for the assembler. You don't want to lose that.

-Phil

Mike Green
01-16-2009, 02:20 AM
As I see it, there are two things involved:

Some compiler changes so that special DAT sections, rather than being compiled into the 32K spin binary, get compiled into a separate 32K binary file with some kind of directory containing an identifier (like a long .. possibly with a 4 letter identifier), a starting location in the 32K image, and a length. The compiler would start with an object (in a file set by a compiler option) containing a short program that just copies these DAT sections to the 2nd half of a 64K EEPROM without the short program (just the directory and the DAT sections). The program would use a simple I2C driver in Spin, something like Basic_I2C_Driver. Typically the user would download this 2nd program to RAM and execute it thus preparing the EEPROM.

The other compiler change would involve another object specified by an option along with some "syntactic sugar" to make the call look as much like a COGNEW or COGINIT as possible. It would use a temporary 2K buffer at the high end of RAM to load the assembly program from the EEPROM given the identifier, again using a short Spin routine to do the reading. A COGINIT or COGNEW could be done and the buffer then reused for something else. It would be possible to write a cog loader that doesn't need a buffer by starting a short program in the cog that only waits for a flag to be set, then transfers a long from hub memory to cog memory, increments the cog address, and clears the flag. This small routine could probably run from "shadow memory" in the cog. Its last transfer would be of a JMP to location zero in the cog to start the newly loaded program. All of the EEPROM I/O would be done by Spin code in the "main" cog.

heater
01-16-2009, 03:23 AM
Perhaps I was misusing "org"

1. We need to indicate to the compiler that the following PASM or data that would normally be in DAT should actually be placed outside the normal 32K HUB/EEPROM space and into some higher EEPROM space. Hence the idea of DATH (DAT High) whatever name you wan to give it.
As I said this could be used for data tables and strings as well as actual PASM code,

2. We need to know where in the high EEPROM space the compiler has put this so that it can be loaded eventually. Isuggested using "org" which may be an incorrect use. What about DATH $8XXX ? Who says sections can't have addesses? With this one could still use "org" within DAT with it's original meaning

So far this seems like a relatively simple thing for the compiler to do. It would require some user code to fetch those PASM/data blobs from high EEPROM and use them as data or something to load a COG with.

I'm not sure I would go down the road of fixing the compiler to load "high PASM blobs" to cogs automatically. Who says they are in EEPROM? maybe I've put extracted them to SD card. How can the compiler, in general, possibly know what is actually to be loaded and run anyway?

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.

heater
01-16-2009, 03:29 AM
@jazzed: No time to play with that today, got some good company and Chardonnay to keep me busy:)

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.

jazzed
01-16-2009, 03:52 AM
@heater: Boy you sure know how to hurt me http://forums.parallax.com/images/smilies/smile.gif
Chadonnay and old Cabernet are my favorites.
I also enjoy New Zeland Sauvignon Blanc.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
--Steve

Paul Baker
01-16-2009, 08:35 AM
Honestly I think this is being made more complicated than it needs to be, to implement this the way some people are envisioning requires a special compiler, a special program loader and and a special indexing method to be able to access addresses beyond 32K. If you guys are intent on following this direction, I would recommend swaping the two banks. For the most part assembly programs are a permanently run in cog. It would be far easier to have them loaded into the Propeller first (lower 32K) get them loaded into thier cogs with forced pointers (the pointer into upper 32K they will eventually point to), then load the entire upper 32K into the propeller which has the spin code and data and bootstrap into the code once it's loaded. This is much simpler than trying to piecemeal portions of the upper 32K into the Propeller.

But I don't think that this is really necessary to begin with. I helped a customer develop a method for reclaiming assembly areas for data by simply placing a function in the object that contains the assembly code to be reused to return the address of the assembly routine. Then the calling function parcels out the space by generating new addresses by using base + offset. This is basically treating the memory area as a heap. While he wanted to keep things as simple as possible, it wouldn't be much trouble to create a heap manager that takes a size argument and returns an base address of allocated memory, then it would move it's internal base pointer for the next call, when it runs out of space from one heap it moves to the next heap. The only time this wouldn't be much help is when you need a very large block of consecutive memory, such as a video buffer.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Paul Baker (mailto:pbaker@parallax.com)


Post Edited (Paul Baker) : 1/16/2009 12:40:21 AM GMT

Cluso99
01-16-2009, 11:28 AM
The org must still function as normally. We just need a new command/option that tells the compiler to place the following code above 32KB boundary.

Loading to eeprom is not an issue. There are many simple ways. We have to overcome the object format problems first.

@Mike: Yes along the lines you suggest. But we need to keep it simple with as few changes to the compiler(s) as possible so as to not introduce any bugs, and keep it simple for the user. One binary/hex makes more sense.

@Paul: Need to keep it simple and that will require a few compiler mods. It could be done as 2 seperate compiles, but then the source becomes extremely complicated for the user. Your method does not allow for loading cogs later. It may be necessary to determine which routines are to be loaded depending on what is happening. It also allows for on-the-fly cog loading. This way, the maximum memory will be available in hub for whatever the user decides. I realise that Parallax would like to keep control of the compiler, but the Jeanie is out of the box and it will only help Parallax anyways. After all, only advanced users will use the extensions.

I see the following:

There needs to be a small table and stub pasm code resident in the hub. When a cognew/init is issued, the stub would be called with the table pointed to so that the correct high eeprom section can be loaded to the cog. I see most of the code loaded directly to the cog, with only the last bit loaded into hub to be copied over the stub (which does the eeprom I2C), and a goto $000 cog at the end of the 496 longs being placed in cog.

It is only necessary to load the length required (reduces loading time).
Here is my take on the table and stub (which would be provided as an include file)




..... user code here..... (note @ maybe incorrect for labels - not debugged of course)

'-----------------------------------------------------------------------------------------------------
'===== HIGH LOAD TABLE AND ROUTINES RESIDENT IN HUB (and lower 32KB eeprom) =====
DAT
org $000

'===== DAT TABLE OF HIGH MEMORY PASM ROUTINES =====
' format: size << 23 | start-location
e_pasm01 long (@pasm01_end - @pasm01) << 23 | (@pasm01) '\\ set by user
e_pasm02 long (@pasm02_end - @pasm02) << 23 | (@pasm02) '||
...more table entries for more routines... '//

'===== STUB COG PASM CODE GOES HERE ======
'Note this code has to allow for the stub code to actually be offset inside the cog.
'The first table at cog $000 is the table for the pasm code to be loaded from high eeprom and it
' contains the length in the top 9 bits and the location in the eeprom in the lower 18 bits.
' They execute as nop's.

stub .... '\\ provided by include file
...pasm stub loader code goes here '||
'||

'===== BUFFER REQUIRED FOR MOVING BETWEEN EEPROM AND COG ===== '||
'length to be determined but should be much less than 2KB '||
tempbuf long 0[256] 'length probably can be less '||
'//


'================================================= =============
'-----------------------------------------------------------------------------------------------------

..... more user code here ......

'-----------------------------------------------------------------------------------------------------
HUBORG $8000 'for code above 32KB eeprom
'-----------------------------------------------------------------------------------------------------
DAT
org $000
pasm01 ..... user cog pasm code (loaded in high eeprom)
.....
pasm01_end 'end of code to be loaded for this cog
'-----------------------------------------------------------------------------------------------------

DAT
org $000
pasm01 ..... user cog pasm code (loaded in high eeprom)
.....
pasm01_end 'end of code to be loaded for this cog
'-----------------------------------------------------------------------------------------------------
...etc






The call would be like this:



cognew @e_pasm02,@parval 'loads cog from upper 32KB eeprom "pasm02" via the stub "e_pasm02" passing the PAR as required by user


The eeprom does not have to be limited to 64KB. In the method above, there is no reason it could not be up to 256KB. By changing the stub include file (the actual cog loader), the cog code could be located anywhere (SD card, etc).

Also, the code in high eeprom could also be tables, overlays, LMM or anything. Once the basic method is there, anything is possible. Note however, it will be slower.

Does this make sense?

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Prop Tools under Development or Completed (Index)
http://forums.parallax.com/showthread.php?p=753439

My cruising website http://www.bluemagic.biz

Post Edited (Cluso99) : 1/16/2009 3:50:08 AM GMT

jazzed
01-16-2009, 12:36 PM
Looks reasonable. Having some reference methodology pays dividends.

This is all achievable without compiler mods of course; as long as it's documented well enough for the target audience, and can be automated, that's fine. Target audience definition can be illusive though. If I wasn't busy on something else, I would just do it ... and I will at some point anyway I'm sure regardless of what shows here. Funny I have 4.5 protoboards and I never thought about them having 64K eeprom http://forums.parallax.com/images/smilies/rolleyes.gif

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
--Steve

Cluso99
01-16-2009, 03:18 PM
What is the 0.5 Protoboard :-(

And yes, it CAN all be done without compiler mods but it is more difficult and harder to document. I liked the fact it can be expanded to 256KB eeproms simply http://forums.parallax.com/images/smilies/smile.gif

Brad & Michael: Do you have an include file function? I know it was discussed at one point.
INCLUDE <filename> 'where <filename> may include or omit partial or full drive/path, optional xxxx.spin extension
The compiler just inserts the lines from the file into the source stream. No restrictions.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Prop Tools under Development or Completed (Index)
http://forums.parallax.com/showthread.php?p=753439

My cruising website http://www.bluemagic.biz

BradC
01-16-2009, 05:00 PM
Cluso99 said...
Brad & Michael: Do you have an include file function? I know it was discussed at one point.
INCLUDE <filename> 'where <filename> may include or omit partial or full drive/path, optional xxxx.spin extension
The compiler just inserts the lines from the file into the source stream. No restrictions.


I've played with it, but I want to make sure that it plays properly with the list file first before I go any further.

Its certainly on my todo list, I've been thinking about it on and off for a while.

You could always use the command line compiler and a pre-processor like cpp in the mean time I guess.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Cardinal Fang! Fetch the comfy chair.

Phil Pilgrim (PhiPi)
01-16-2009, 05:02 PM
Questions for those who want this capability:

1. Would you be willing to accept a diminished cog RAM capacity due to the necessity of including a loader in each cog?

2. Or is it more reasonable to set aside a 2K hub RAM area into which the asm code is loaded from high EEPROM before starting the cog?

3. Sometimes it's handy to preload cog variables into the hub's DAT space during a Start routine before doing the COGNEW. Is sacrificing this capability a reasonable price to pay for increased asm space?

-Phil

heater
01-16-2009, 05:08 PM
1. For the 8080 emulator, no, it's overflowing already.

2. For the 8080 emulator, no, I want all the HUBRAM for 8080

3. Not worried.

8080 is perhaps a freak program and I'm going to reclaim RAM "manually" at some point. Otherwise I feel option 2 makes more sense generally.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.

Carl Jacobs
01-16-2009, 06:23 PM
1. No way. I'd rather have a slow loader taking up a couple of longs of spin - probably as part of a more generic I2C driver.

2. No problems on this one. That RAM area is only going to turn into Ethernet buffers or SD card buffers once the COG loading process has completed.

3. Definitely, I think gaining the extra 2K back from HUBRAM is worth it. In the case of JDCogSerial I thought about moving the variable initialisation code to the end of the PASM section; allowing it to become part of the FIFO buffers once the working variables had been loaded. In the end I didn't bother with trying to reclaim those 11 longs! (I wrote some really cryptic PASM, Instead).

Regards,

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Carl Jacobs


JDForth - Forth to Spin Compiler http://www.jacobsdesign.com.au/software/jdforth/jdforth.php
Includes: FAT16 support for SD cards. Bit-bash Serial at 2M baud. 32-bit floating point maths.·Fib(28) in 0.86 seconds. ~3x faster than spin, ~40% larger than spin.

Cluso99
01-16-2009, 08:23 PM
1. There will be no cog ram used - period! I will use the shadow ram.

2. I think about·500 Bytes may be required from hub memory and possibly may not be reclaimable, but ALL your cog code could now be in higher eeprom, so you are reclaiming all cog space.

3. If·you·need to preload·hub DAT space, you will have to have your code in·hub space. You can try·passing variables via hub, but you will not be able modify hub ram.

So, do you want a simple way to place cog code in upper eeprom and reclaim most of hub or not???

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Prop Tools under Development or Completed (Index)
http://forums.parallax.com/showthread.php?p=753439

My cruising website http://www.bluemagic.biz

heater
01-16-2009, 08:47 PM
Yep!

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.

jazzed
01-16-2009, 10:25 PM
Here are some choices and comparisons (choice descriptions follow):
Spin Code Spin Data Implementation/ Reclaimed PASM Waste/ COGs
Choice Usable HUB For Loader Use Difficulty Spin Code KB Rewrites Wasted

A.SL ~30KB-EE&Loader 2KB Easy Up to ~14K None None
B.PHL 30KB 2KB Moderate Up to ~14K None 1
B2.PHL 31.5KB 0.5KB Moderate Up to ~15K None None
C.PL 32KB-EE-loader Loader size Hard Up to ~15K All None
D.APL 32KB-EE/4-loader Loader size-EE Hardest Up to ~15K Some? None
E.BF 32KB None Moderate Up to 16K None None
F.POS 32KB None Done Up to 16K None None
G.NIL 32KB-PASM None Done None None None


A. Spin-loader (SL)
· 1. Allocate 2K of HUB DAT for COG load space.
· 2. Spin includes EEPROM read (or other) code.
· 3. Spin loads HUB and starts each COG.
· 4. Spin uses 2K for Heap or other purposes.

B. PASM-HUB-loader (PHL)
· 1. Allocate 2K of HUB DAT for COG load swap space.
· 2. 2K HUB initially holds PASM loader.
· 3. PASM loader includes EEPROM read code.
· 4. PASM loads HUB and starts each COG.
· 5. Spin uses 2K for Heap or other purposes.

B2. PASM-HUB-loader (PHL) @Cluso99
· 1. PASM stub loader and COG load swap space in hub (target total <0.5KB)
· 2. PASM loader includes EEPROM read code.
· 3. PASM loads COG and starts each COG.

No other cogs used.
C. PASM-loader (PL)
· 1. Spin includes PASM loader.
· 2. PASM loader includes all loader/EEPROM read code.
· 3. PASM loads its cog-enumerated block.
· 4. PASM restarts itself.

D. Alternate PASM-loader (APL)
· 1. Spin includes Primary/Secondary role PASM loader.
· 2. Spin launches PASM loader on all COGs.
· 3. COG number (1 or 7?) determines PASM role.
· 4. Spin includes EEPROM read (or other) code. Spin = PASM/4.
· 5. PASM Primary manages Spin load of Secondary cog-enumerated blocks.
· 6. PASM Secondaries contains mem-copy and re-start command code.
· 7. PASM Primary loads and restarts itself.

E. Baker-Flush (BF)
· 1. Target program with API wrappers live in upper 32K.
· 2. Loader and all target PASM code boot normally.
· 3. One PASM section loads upper 32K to HUB and reboots Spin.
· 4. Target boots and can load one PASM driver.

F. Prop-O/S (POS)
· 1. Use Mike's Prop-O/S -OR-
· 2. Use another loader feature O/S

G. Don't bother with any of this. (NIL)

@Cluso99
My 0.5 Propeller Protoboard is the one where the Prop PLL was destroyed by an over-current accident.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
--Steve

Post Edited (jazzed) : 1/18/2009 6:56:31 AM GMT

Cluso99
01-16-2009, 11:06 PM
Steve:
My idea for loading the upper eeprom into cog was to load a table followed by a stub loader into cog. This would be a short program to read blocks ($40 bytes I think) from eeprom direct into the cog. When the cog stub is about to overwrite itself, the pasm would shift into LMM zero footprint mode (requiring some hub space for the LMM) to read the final blocks into cog, then jump to cog $000 to execute the newly loaded code. The only reason it is not all in LMM is to speed the load up. If the load is shorter than 2KB then the LMM may not need to be executed. So, I think that we may only require about 0.5KB of hub space, but certainly not the whole 2K (including heap space).

I have only worked on starting a cog in pasm whereby the code resides in eeprom above 32KB. The cog may be started by a cognew or coginit instruction as per normal (but the pasm start is the table, par is normal) from spin or pasm. Existing methods of starting cogs and spin residing in hub start (and code) as normal.

In getting this running I would propose to do it in 2 stages. Firstly place all the cog pasm stub code in hub and have a 2K buffer in cog. Once working, shrink the code to reduce hub footprint and speedup loading.

So, currently I propose no changes to the Interpreter or loader. And it can be done without changing the compiler, but is just more complex (have to manually edit the lower hub table)

:-( Pitty about the other 0.5 Prop.

PS: I don't see any of this as being complex. The bits of code required for reading and writing eeprom exist already, just not the upper section, although I know others are already doing this. I already have zero footprint modified LMM running with my debugger, so I understand how to do it.

All memory must be available in the cog (all 496 longs) otherwise there is no point in this exercise. And we want to minimise the hub memory. If cogs do not need to be re-loaded, all space used in the hub table and stub·can be reclaimed. Otherwise I hope 0.5KB will be all that will be required.

Oh, and did anyone notice? I said this method will work with up to 256KB of eeprom. Because the table must have bits 18-21 zero to force a nop. Bits 0-17 give an 18 bit cog start address in eeprom. Bits 23-31 are used for the 9 bit·load length - if zero then the full 512 (well 496) longs will be loaded.

Is this making sense yet?·

Guess I better unpack my prop and get to work...

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Prop Tools under Development or Completed (Index)
http://forums.parallax.com/showthread.php?p=753439

My cruising website http://www.bluemagic.biz

Post Edited (Cluso99) : 1/16/2009 3:26:02 PM GMT

jazzed
01-16-2009, 11:50 PM
@Cluso99:
Sounds like a minimum-impact / maximum reclaimation scheme :)
Can you please summarize the steps for your method and qualify the impact for comparison categories?
I'll add it to the post above.

How do you propose getting code into EEPROM? Looks like a two step process in any method.
Paul's approach (the Baker Flush) can have a downloader stuff the upper 32K for EEPROM or HUB for RAM testing.
I am very attracted to not writing EEPROM often during development.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
--Steve

Phil Pilgrim (PhiPi)
01-17-2009, 02:12 AM
Cluso99 said...
The only reason it is not all in LMM is to speed the load up.

The LMM won't slow things down at all. The EEPROM R/W code has to have major delays in it anyway to keep the data rate within the EEPROM's specs. I think this is a good approach, because it maximally reduces the hub footprint.

-Phil

Cluso99
01-17-2009, 11:02 AM
@Phil: Good point.

@Steve: Yes will need to minimise writes to eeprom. Development should debug the code being placed high while the code is still low.This will be a user problem that requires a warning. I will do the detail tonight (currently out)

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Prop Tools under Development or Completed (Index)
http://forums.parallax.com/showthread.php?p=753439

My cruising website http://www.bluemagic.biz

Carl Jacobs
01-17-2009, 02:29 PM
It's just one of those things - isn't it? We think about it for ages, without doing it. When we're finally challenged about it, it's not too hard.

I've taken the dumbest approach possible. If you've read my posts, then you'll know what I've done.

The zip file consists of three top-level applications, which I'll discuss shortly. All of them deal with the object JDCogSerial which was recently submitted to the object exchange.

JDCogSerial.spin has been split into two files:
x_JDCogSerial_pasm.spin - which is the pasm part and a call to the saver and verifyer.
x_JDCogSerial.spin - which is the spin that's leftover, and a call to the loader.

x_Config.spin is included in all x_??? objects and determines the memory map of the EEPROM.

x_saver.spin is used to put the object PASM into EEPROM, it is also used to verify the object.

x_loader.spin is used to get the object back out of EEPROM.

The loader and saver use simple bit bash I2C.

Now for the top-level applications:

1) MemoryWaster.spin - uses JDCogSerial directly, and compiles to 213 longs.

2) x_Main.spin - is used to save the object PASM to EEPROM. Who cares about the size?

3) Main.spin - load the object from EEPROM and use it, and compiles to 151 longs.

The loader itself compiles to 85 longs, so *most* of the HUBRAM has been reclaimed. The "leftover" x_loader contains generic I2C access routines that may be used for other purposes if desired.

Regards,

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Carl Jacobs


JDForth - Forth to Spin Compiler http://www.jacobsdesign.com.au/software/jdforth/jdforth.php
Includes: FAT16 support for SD cards. Bit-bash Serial at 2M baud. 32-bit floating point maths.·Fib(28) in 0.86 seconds. ~3x faster than spin, ~40% larger than spin.

Cluso99
01-17-2009, 09:24 PM
@Carl: How true. I had a quick look at your code - great stuff.

May I be so bold as to suggest the following. There should be at least·3 phases in getting this done.

Phase 1

Initially we use two distinctly separate code blocks. One is for the upper 32KB eeprom (for debugged cog pasm code), and the other is for the lower 32KB eeprom (for spin, data, cog stub loader from upper eeprom, and maybe fast or non-debugged cog pasm).

Both are independantly compiled and the user will be initially responsible for manually setting the upper load addresses (and optional length) within the lower code block·(Phase 2 will fix this by a compiler option).

Upper 32KB eeprom (for debugged cog pasm code)

This block contains a spin program which is executed upon load (by the normal PropLoader using ram option). Also contained in this file is(are) the cog pasm code block(s). The spin program will copy (program) the whole 32KB Hub Ram into the upper 32KB Eeprom (Phase 2 will check to see if the code is the same as already stored in the upper eeprom,·and if so, skip the reprogramming of the eeprom). The spin program may (or may not) include pasm code.

Carl, may I suggest you do this??? You seem to have the I2C·eeprom code under control.

This now covers getting the code into the upper 32KB eeprom. No changes are required to the compiler or proploader for this phase.

Lower 32KB eeprom (for spin, data, cog loader from upper eeprom, and maybe fast or non-debugged cog pasm)

This block contains the users spin programs and any cog pasm programs which the user decides should still be in lower eeprom (and hub resident). In addition to this will be a block of code which contains:

A table of cog upper eeprom addresses (18 bits, for later expansion) and optional length (9 bits) of longs to load. Length=0 means whole 496 longs. In phase 1, the user will be responsible for entering the addresses and lengths directly into the source code before compilation (Phase 2 will hopefully see Brads and Michaels compilers modified to do this automatically).

A stub boot code to load the upper eprom cog code into a temporary 2KB buffer in hub and then into cog (or directly into cog, or a mix of both). This code will also be responsible for any conflicts in use of the buffers. The user may be able to recover this space once all loading has taken place.

A 2KB buffer in hub as described in the previous paragraph. The user may be able to recover this space once all loading has taken place.

(Phase 2 will see the stub boot code and the 2KB buffer reduced in size to hopefully achieve less than 0.5KB footprint in hub space).

I am happy to write the stub boot code, but Carl, may I ask you to do the eeprom reading for me???

Phase 2

See notes above.

Request Brad and Michael modify their compilers to allow a single set of files. Brad has indicated he is willing to do this.

Phase 3

Once the compiler is done,·Brad has indicated he is interested in writing a windoze/linux/mac PropLoader, which would·load the prop in a single·function. This would see the project completed and very user friendly, with full backward compatability.

Does anyone have any objections, suggestions or comments ???





▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Prop Tools under Development or Completed (Index)
http://forums.parallax.com/showthread.php?p=753439

My cruising website http://www.bluemagic.biz

Post Edited (Cluso99) : 1/17/2009 1:38:43 PM GMT

Cluso99
01-17-2009, 09:30 PM
Steve, as requested

Phase1. PASM-HUB-loader (PHL)
1. Allocate 2K of HUB DAT for COG load swap space.
2. <2K HUB initially holds PASM loader.
3. PASM loader includes EEPROM read code.
4. PASM loads HUB and starts each COG.
5. Spin uses 2K for Heap or other purposes.

Phase2. PASM-HUB-loader (PHL)
1. PASM stub loader and COG load swap space in hub (target total <0.5KB)
2. PASM loader includes EEPROM read code.
3. PASM loads COG and starts each COG.

No other cogs used.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Prop Tools under Development or Completed (Index)
http://forums.parallax.com/showthread.php?p=753439

My cruising website http://www.bluemagic.biz

Cluso99
01-17-2009, 11:55 PM
@Carl: I gather from your eeprom code, that a read may begin at any byte offset, and may be of any length. Is this correct?
Do you know how many clocks it takes to read a byte?

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Prop Tools under Development or Completed (Index)
http://forums.parallax.com/showthread.php?p=753439

My cruising website http://www.bluemagic.biz

jazzed
01-18-2009, 08:09 AM
Nice efforts Carl and Ray. I've updated the "impact table" ... please check for clarifications.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
--Steve

Cluso99
01-18-2009, 01:59 PM
Jazzed:

Impact table B2: No cogs are wasted. Only the cog to be loaded is used to load itself and then execute and no cog space is required (it is reused via zero footprint shadow ram $1F0-1F3). http://forums.parallax.com/images/smilies/smile.gif

All coginit/cognew etc instructions work exactly as they do now, except the user points to a table instead of the real start address. PAR is as per normal. The table holds the real address (in eeprom) and its length, and is followed by the cog eeprom load code. This loads the cog (up to 496 longs) and then executes the loaded code at $000. The only issue is that the loading will be slower, and if multiple cogs are being loaded from eeprom, they will be queued and so will be slower to boot.

Carl: Would you like to do for me the pasm code (routine) to read a block of eeprom (from upper 32KB) into hub or cog? I will preset 2 longs, 1 with the 18 bit eeprom start address. Bits 17-15 will be set "001" indicating 2nd 32KB block being 32KB-64KB. This allows for later expansion. Bits 14-0 will indicate the byte offset within the block. The other long will be preset with the length in bytes (I will convert from longs). Is this the best way???

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Prop Tools under Development or Completed (Index)
http://forums.parallax.com/showthread.php?p=753439

My cruising website http://www.bluemagic.biz

Cluso99
01-18-2009, 06:27 PM
Here is some sample code of how I believe this will work in Phase 1.

The listings are from the homespun022 compiler.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Prop Tools under Development or Completed (Index)
http://forums.parallax.com/showthread.php?p=753439

My cruising website http://www.bluemagic.biz

Carl Jacobs
01-18-2009, 06:35 PM
@Cluso99: My I2C routines are as slow as a wet week. The contents of the EEPROM are loaded out at a baud rate of about 20000 bits per second. I'll probably not be persuing the loader algorithm any further, as what I was really interested in was process. In my own mind I have established a process that works and that is not a whole lot more difficult than the existing process for code development.

The only changes I'll probably make are to have a long word at the start of each high EEPROM object PASM block which consists of:
1) A word which tells how many bytes of code are stored - to cut loader time
2) A word which is a sum of all the bytes in the object
I might also think about adding another long that's an object signature. "JDCS" = JDCogSerial? But I'm really not too concerned about this.

Reads from EEPROM memory can be continuous, although it generally falls apart if you try and cross a 32K boundary. There may be cases where a continuous read is not permissable, but I'm not aware of any. There are excellent PASM routines for I2C available in the obex, and I would suggest these be investigated, as the SPIN bit-bash code I presented does not require any speed throttling - something which is absolutely essential when writing PASM based I2C. Great insight can be gained from studying the I2C code presented in FemtoBasic.

The loss of 2K of HUBRAM to the loader is not a problem for me as I generally need that amount of RAM for SD Card buffers, Ethernet buffers and other general buffering. (VGA Buffers?)

Please not that a lot of my code is an adhoc mixture of Forth, spin and PASM. The loader I've presented could just as easily be written in Forth - except that for that I first need a Forth kernel which I plan to store in high EEPROM. In Forth I can do bit-bash I2C at about 80000 bits per second without resorting to in-line PASM, which is probably as fast as I'd want to go with generic EEPROMs. If startup time becomes an issue I'll do further investigation (ie code writing) later.

Regards,

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Carl Jacobs


JDForth - Forth to Spin Compiler http://www.jacobsdesign.com.au/software/jdforth/jdforth.php
Includes: FAT16 support for SD cards. Bit-bash Serial at 2M baud. 32-bit floating point maths.·Fib(28) in 0.86 seconds. ~3x faster than spin, ~40% larger than spin.

Cluso99
01-19-2009, 12:39 PM
I've started on the I2C routines (using Chip's booter.spin as the I2C basis). I've updated to 128 byte pages (was 64) and using the upper address of $8000-$FFFF in eeprom. It is in pasm, but timing is using the RC oscillator, so once working I can speed it up. I am only checking it for the AT24C512 at present.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Prop Tools under Development or Completed (Index)
http://forums.parallax.com/showthread.php?p=753439

My cruising website http://www.bluemagic.biz

Carl Jacobs
01-19-2009, 01:22 PM
Cluso99 said...
I've started on the I2C routines (using Chip's booter.spin as the I2C basis). I've updated to 128 byte pages (was 64) ...

If you're reading, then this will be fine. If you're writing, then beware! There's a reason that both Chip and myself are using 64 byte blocks - it is all that some EEPROM devices will support.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Carl Jacobs


JDForth - Forth to Spin Compiler http://www.jacobsdesign.com.au/software/jdforth/jdforth.php
Includes: FAT16 support for SD cards. Bit-bash Serial at 2M baud. 32-bit floating point maths.·Fib(28) in 0.86 seconds. ~3x faster than spin, ~40% larger than spin.

Cluso99
01-19-2009, 03:50 PM
Thanks for the warning Carl http://forums.parallax.com/images/smilies/smile.gif

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Prop Tools under Development or Completed (Index)
http://forums.parallax.com/showthread.php?p=753439

My cruising website http://www.bluemagic.biz