PropForth V5.5 -- any memory map?
LoopyByteloose
Posts: 12,537
I know I can dig for this and make my own, but Tachyon Forth kindly provides a memory map.
The situation is this.
I want to set aside a section of memory either in CogRam, HubRam, or EEPROM for a look up table.
It seems if would be easiest to have 134 or so items using unused locations in CogRam for the task, but there is no memory map to show which actual contiguious memory locations are available.
Data would be written to and read with Cog! and Cog@.
I could presume that working backwards from the end of memory is the simpliest way to go. Is that okay.
The situation is this.
I want to set aside a section of memory either in CogRam, HubRam, or EEPROM for a look up table.
It seems if would be easiest to have 134 or so items using unused locations in CogRam for the task, but there is no memory map to show which actual contiguious memory locations are available.
Data would be written to and read with Cog! and Cog@.
I could presume that working backwards from the end of memory is the simpliest way to go. Is that okay.
Comments
My first guess is to have a file in EEprom and call it LookupTable, and just read records out of there.
But on to you request for memory map: It seems to spread out in a couple spots in the manual, as it fits into the narrative. I don't think there has been a specific requrest yet, but I thought I saw one in the comments during th build process review. But here the parts in the propforth.htm manual for starters
The first bit is inSection3.6
The next bit is in section 6.2 Technical Details (for I/O)
Next is Section 7.1 Assembler and Kernel Words
7.3 is The Stack in Detail.
I think 7.1 is the section that has the most of what you are looking for, but you might need the other section to make better sense of it.
9.3 starts the description of EEprom
Now to look through the code...
With the Cog Ram...
Both the front end and back end are used, so it is important to locate the area between.
$000-$139 320 longs of PropForth v5.5 code
$140-$1EF 176 longs of empty space to be used via Cog! and Cog2 (trashed by an reset or cog reset)
$1F0-$1FF 16 longs of Cog specific registers
This map only applies to Cog0 throuhg Cog6 as Cog7 is the Serial Console and has an entirely different code element loaded. It is best not to fool with that and I doubt one can reach it via PropForth. You would have to read and re-write the Source Code
I am sure I could break out some more about the 320 longs, but what I really want is to use those 176 longs for a look-up table of constants in the fastest loop possible.
HubRam, aka Main Memory, would seem to offer more space for a much longer table and I think I could work off the backend as it seems the Forth Dictionary is starting at 0 and going higher, but the Cogs have 8 224byte (56 long) regions that appear to break up contiguious address.
So one does have to be careful to work around those.
These facts can be confirmed by doing data dumps with Cog@ and L@.
I actually have gotten my information from Source code in the ForthKernel.spin file. But the PropForth.htm confirms the 176 longs available in CogRam... just doesn't give an actual address range.
BTW, I did read all that you recommended... before I asked.
did you check out this variable: (as defined in PropForthStartKernel.f and StartKernel.f)
\ coghere ( -- addr ) access as a word, the first unused register address in this cog
: coghere
hD8 _p+
It is the start of the 176 Long space (the 176 could change with future versions) - this is how the SD code used to decide where to put it's I/O buffer for the COG.
From coghere up to par should be good to use as long as you don't have SD words loaded (and haven't done an sd_init on the COG you are interested in).
Thank you. That is a bit of Class A info.
Class A -- need to know
Class B -- nice to know
Class C -- garbage
Now why isn't that in the Manual?
Oh, if I read the manual, it will explain why. Right?
(Not really, one has to read the .spin files to get the real details directly from the Source code. Documentation may not cover it all.)
You could be the first to ask that question.
I ended up looking in the sd*.f files to see how they allocated that buffer space. Some of it is in the .spin code.
Now I just understand why it will never be perfect.
Read the source files.. the .spin and the .f
After all, if those are not right, it won't work right.
Documentation always good? Unlikely. Even on paid products, this is an ambitious expectation.
Documentation sometimes present? More likely, but only when we (you, me, Rick, caskaz, etc) write it.
Remember, documentation is driven by user questions. It can't exist until we ask.
... then ask your specific question.
I've opened issue 193 for you, "Request for Memory Map" http://code.google.com/p/propforth/issues/detail?id=193
Hopefully there is enough there so Sal can answer. Please add more detail as needed.
For me, my knowledges ends at "Make a file in EEprom and use that". I don't really follow what you are trying to do or why a file in EEprom won't work, maybe Sal will.
Another thought: the Logic Analyzer implemented in software captures samples to HUB for slow sample rate (over 40 cycles) to COG RAM for fast sampling rate (under 40 cycles) and using three cogs for very fast samples (1 cycle). Perhaps LAC has an example of what you want to do (placing data in memory)?
Of course, there may be some rare cases where a user wants to use the Cog resident tbuf, but that is not my desire.
Well, it seems obvious that COG! and COG@ are handy ways to handle indexed data if the space is available. One can even create their own arrays within the space limits.
In my case, I am generating a series of stepper motor delays that will be read by delms.
For larger arrays, using the Hub Ram also offers the ! and @ in Long, Word, and Byte sizes... but a bit slower.
Why not exploit the empty spaces when available rather than having to extend the EEPROM or use an SDcard?
Users do try to buy into hardware with above average documentation. Developers generally are divided into zealous programmers that hate to write documentation, and those that enjoy doing so.
But when you are fast and furiously into development, documentation becomes a paper chase for the new users.
free ( -- ) display free main bytes and current cog longs [ifndef free : free dictend W@ here W@ - . ." bytes free - " par coghere W@ - . ." cog longs free" cr ;in PropForthDevKernel.f and ForthDevKernel.f, of course.
This pretty much sums up how to sum up memory.
Darn, you beat me to it! I was thinking of using 'free', but wanted to verify it before I mentioned it,
On a PropForth v5.5 eeprom installation you get back the following:
13836 bytes free 176 cog longs free
The 13836 bytes free are Hub ram, aka Main Memory
and obviously the 176 cog longs free are Cog Ram.
But if I insert d123 in register d321 via COG! on Cog6, I can read it with COG@, and still get d176 in 'free', no change to d175.
And if I add a word to the Forth Dictionary and run free...
The byte count in main memory does down. as it should.
~~~~~~~~~
I need to explore a bit as I am getting all sorts of values in Cog Ram registers that should be empty. I presume that empty does NOT mean that they were initialized to zero.
And maybe I have to study where the 'free' gets the Cog Ram longs... a constant or actually searching to verify.
Can't seem to make the 176 change in 'free' via COG! entries.
I'm probably going to go look in the SD words when I get a chance.....they take a buffer chunk out of COG space.....but now it's lunchtime!!!
320 176 cogdump
And finding some stretches of zeros, but other with h20202020 and other stuff. I am now beginning to wonder if the empty space is indeed contiguous.
You can run a reset and dump an image of it all with the following
0 512 cogdump
And of course, this will all lead to me doing some main memory dumps to seek out reality.
As Elmer Fudd says, "Just be vewy, vewy, careful!"
Yes, it is completely volatile and prone to loss....depends n your application's needs, of course, If you need the speed, then you load the lookup from some place safe (EEPROM?) in your initialization word and then go from there. If it is a dynamic lookup of some sort, you need to save it when changed, so then you write back to EEPROM. Depending on speed needed, you pick.
There isn't a COG register allocation word like there is for HUB memory (allot), so you need to do it manually like they do in sdcommon.f
Establish where your allocation is going to start and save the place so you can clean up: set up your space as needed and set constant pointer into the space.....
figure out where the end is: then updage coghere so you adjust the freespace after you took some:
when you are done, do something like this to clean up your mess (zero and "release" by adjusting coghere):
\ sd_uninit ( -- ) "releases" the cog memory \ : sd_uninit _sd_cogend v_sdbase do 0 i COG! loop v_sdbase coghere W!I guess I didn't see the real purpose of ALLOT before, as it makes sure that a contiguous spaces is made available for index manipulation of addresses.
free does return free cog longs and hub words. Look at HERE, this points to the next unallocated space after the common dictionary hub memory. Just look for the words that store something in coghere to see how the cog space is allocated.
Remember, the scratch pad is the first unallocated area in memory after the dictionary. So the stuff that is changing MIGHT be in the current scratch pad area(s). I think one scratch pad area is the 64 character buffer for serial I/O for each cog. The scratch area moves each time a new definition (etc) is defined, so no one area gets worn out.
I think ALLOT also updates HERE
Well, I have a bit of difficulty with calling a space unallocated and then using it as a scratch pad for systems operation if such operations are on-going after the initialization on a reboot or reset. it would be more tidy to clear available space to all zeros or some other pattern to indicate availability.
How is one to ever determine where the 176 free Cog registers are and if they are really available without a complete detailed analysis of the PASM for the PropForth code in the Cog? You seem to not know for sure.
I don't expect an answer to everything, but it would help to just not make definitions more unclear.
Sine the PASM compiles sequentially, I presumed the 320 longs occupied a contiguous space starting with $000. Now I am scratching my head or what you claim might be a scratch pad, or maybe not.
Now I am once again having to jump around in .spin and .f files looking for clues to what is reality.
And this is exactly why I try to use the 32k kernel, as I am trying to figure out what is the core of PropForth... For instance. it seems that 'onboot' may only be supported with an extended EEprom or an SDcard. But I can not find anything to clarify that. I have to run experiments to see what is what.
When a boot, a reboot, or an onboot is called, the 176 contiguous longs are all zero, as I was hoping for. This seemed to be in agreement wth what the .spin files were presenting and the PropForth.htm.
But..................
When a reset or a cogreset or a system reset due to stack error occurs, 4 regions are filled mostly with h2020_2020 that pretty much destroy any utility as a large contiguous block of Cog Ram for code or array storage.
I found this to be true fo the .spin image for the 32K eeprom and the .spin image for the 64k or more eeprom. And since Mindrobots mentioned that the SDcard images also use the space - maybe in the same way or maybe differently, it is pretty much useless for what I had hoped for.
My own feeling is that the 176 free longs is incorrect, and to me 'free' implies that I can use them in a stable manner without the system upsetting them. Sure you can say it is just 'scratch pad' use, but the simple fact is that if the system and developers feel it is there to be used by the system management, it just not really free to the end-user.
I have no idea if there are other conditions besides the reset in which the system will overwrite the space.
I can't seem to see the utility of 'coghere' or 'free' in relation to what they offer about the Cog Ram.
Don't be silly. If you want to reserve memory for your purpose, reserve it and use it. Remember that forth uses unallocated space as temporary scratch and don't let your app hog every last byte or risk a crash. Once you allocate it, the kernel will use the space AFTER yous as scratch. Simple.
The problem seems to be you are trying to use the same unallocated space as YOUR scratch, when the kernel already is using this technique. One party has to allocate, or you will step on each other's toes. It is suggested that you as the app allocate, and let the kernel know your are using this space, since that is the design. Otherwise the kernel has no way of knowing what you want.
Remember, this is a micro will limited space.
Also,did you look in the ../make/src directory for the kernel code? This is usually where to find the stuff we can't find in the beginner's package.
I seldom get the opportunity to say "no need to be dramatic", so this phrase does not immediately spring to mind. But now that I think about it, I will be using this more frequently in the coming days. I'm starting to freak out over booking the LittleRobot workshops, I imagine much drama is in store... A very intricate puzzle this is!
Um, wait until the little Brainiacs get a wee bit older.....your life will go from SitCom to Prime TimeDrama in a matter of minutes.
But, I digress from the Forth discussion.
"here" is a commonly understood Forth word used to mark the next entry point in the dictionary. Since everything in (normal) Forth lives in the dictionary (words, arrays, variables, constants), or on the stack, it's one of the critical addresses to know so new things can be created in the dictionary. IN PropForth, the dictionary lives in HUB memory, so 'here' refers to HUB memory. If you look at any of the creating type words, you;ll see that they will adjust 'here' by some amount before they are finished. "dictend" appears to be the actual end of HUB memory but I just haven't found it's definition yet. So for HUB memory, you have from 'here' to 'dictend' to use via the creating words or your own creating words AS LONG AS YOU PLAY BY THE RULES AND UPDATE 'here' WHEN DONE.
If the Prop, we also have COG registers (memory) to keep track of. Thus we have 'coghere' which is the the next available register in the COG. You have from 'coghere' to 'par' (since we have reserved space at the top of the COG registers) to do with as you'd like. Since it is an unusual commodity, there aren't really any creating words that us COG space. You could come up with a 'cogallot' if you wanted (probably just a matter of taking 'allot' and substituting 'coghere' for 'here' and maybe 'par' for 'dictend') and then use it. AS LONG AS YOU PLAY BY THE RULES AND UPDATE 'coghere' WHEN DONE *AND* EVERYBODY ELSE DOES, you should be OK.
The SD words: Yes, they do take a buffer out of COG space, SO, just don't run your word with the lookup table in the same COG as the SD words and all should be fine. If you need SD support from your SD COG, you will need to find a way to have the two of them exchange data (via a buffer space in HUB memory?) and notify each other (the joys of muliprocessing!!).
The RESET issue: if your COG gets reset (stack violation or any other reset event), no matter what you were doing that COG, all bets are off after the reset. The interpreter resets itself, your word is history, anything you set up is GONE. You could set the reset for that COG to point to your application word and start it up again which would cause it to rebuild the lookup table and start its exciting task again.
The nature of a reset then forces you to look at your lookup table again. if it is a static table with unchanging data, then just load it up again from where you got the data before and start looking things up. If it is dynamic and your application word changes it, then we are back to the persistence question. If you need to modify and change the data then it should either live in EEPROM (lookups from there will slow you down) or any changes need to be written back to the COG image PLUS the EEPROM images. Effectively you need to treat it like a cache and writes need to penetrate to your persistent layer.
Now, go Forth and CODE!!
I simply have found that the 176 longs are not reliably free (available) to the end user. It seems that what the 'reset' does should be mentioned somewhere, so I mentioned. I could easily understand if a cog reset replaced all the 176 with fresh zeros as the reboot does, or if the cog reset left the region untouched -- but it does neither. It overwrites four portions with data that may have some purpose .. I just don't know what.
Silly? dramatic? Portray it however you want. It seems to be overlooked in the source or documentation. Changing one register to indicate a reset had occured would be more than enough, it could be done by changing one bit.
Concerning COG initialization: COG memory is not guaranteed to come up in any particular state. IF you need it in a certain state, you must tend to in your application.
coghere points to the end of the memory used by the system, the rest is left to you.
SD uses memory after coghere, by then updates coghere properly.
The advice is a lookup table should be in main memory, allocate the space into the dictionary, and then use it as normal.
The space after here and dictend are used by the system for new definition, scratch etc, you must necessarily be aware of this if yo are also using these areas, the system doesn't mind as long as you don't step on the system function and don't get stepped on yourself.