Propeller Tool Not reporting Local Array Variable size correctly

JohnR2010

I have been working on a fairly large application based on the Spinneret and in troubleshooting memory problems I noticed that when I look at the Object Info’s Stack / Free space (F8 Key in the compiler) the free space does not correctly reflect the true free space. This is causing a ton of problems as my code gets closer and closer to filling the available RAM.

So here is what I’m seeing:
In a few of my private and public method blocks I use local variables that are arrays for example:

PRI sdRead | TxtPtr[25],x,y

When I hit F8 I get the same amount of free space regardless of the size of TxtPtr[xx]. Has anyone else seen this? It is easy to duplicate I see it in both version 1.3 and 1.2.7(R2) of the Propeller Tool. Does anyone have a suggestion on how I can accurately determine available RAM for my variables? I think this is only a problem if you have a large application as I do. Currently I'm manually adding up all the local arrays i declare. What other variables does the tool over look or not report correctly??

Please forgive me if this has been addressed in an earlier post I did some searching and I haven’t found anything close.

Thanks!

Heater.

JohnR2010,

The stack/free space reported is what is left after all your code, dat and var sections have been added up (minus whatever longs are used for Spin start up code and such).
That is to say it reports what is left after all the statically allocated things have been summed up. It is most probably correct.

Now, any local variables in methods do not exist at compile time, that is to say they are not statically defined in memory at any fixed location.
Local variables only come into existence when the method is called at run time. So they cannot be counted by the compiler at compile time.
So where are they?
They created on the stack that is in use when the method is called.
For this reason they do not show up in the free space report.

For your top level object they will be created in the top level objects stack which is indeed in the stack/free space. Where exactly depends on the value of the stack pointer when the method is called and may vary depending on how much stack has been used at the point that the method is called.

For your sub objects running in new cogs the local variables will be created in the stack space that you defined and passed to cognew. That stack space has already been counted in the program size and subtracted from free space of course.

Also be aware that if you have recursive method calls, i.e. a method that calls itself, then the same local variables will be created multiple times on the stack. Once per level of recursion.

Be sure your stacks are big enough!

Heater.

Most likely your program does not use recursion, methods calling themselves.
Therefore I would suggest not declaring arrays as local variables but just make them normal VAR or DAT arrays global within the object.
That way they will show up in the stack/free space report to give you a better idea of your real memory usage.

Rayman

Heater is right. Somebody here just had a big problem because they declared a huge local variable array and it broke their code.
So, big local variable arrays are not a good idea. Best to do like Heater recommends and put them in VAR or DAT instead...

MagIO2

Maybe now someone should also explain the differences and when to go which direction and why.

If you have a function that can be called by more than one COG and/or is called recursively and needs some variables it might make sense to use a local variable. A local variable - as already said - is placed on the stack (!AND! ... don't forget to initialize/set these variables before reading!). Each COG has it's own stack and each recursive call will create it's own set of these variables on the stack, which makes the code reentrant. BUT ... you have no build in possibility to check whether the stack is big enough!

Variables in the VAR section exist once per instance of an object. These don't live on a stack but are memory areas which are reserved statically during compile-time. They immediately eat up your HUB-RAM, BUT you also get immediate feedback when compiling about the RAM usage.
If we talk about one object that uses these variables, you have to be aware that:
1. A function using those variables should not be called from several COGs at the same time!
2. Instead of recursion you have to use iteration!

If you need to call the function from several COGs you can create one instance of this object per COG. Each object has it's own copy of the variables. This is again a static reservation of HUB-RAM and you will be acknowledged when you run out of memory.

A DAT variable only exists once, no matter how much objects you create. So, these should really contain general settings only or you have to take care by yourself that each function call that can happen concurrently only works in it's own area. Example: You have a TV driver which uses an array in a DAT section as screen-buffer. Now you could call printStr by addressing the upper half of the screen by one COG and call printStr addressing the bottom half of the screen by another COG.

JohnR2010

Thanks for the input. I have found a work around by keeping track of the temp stack space and making sure i have enough extra RAM to accommodate them. Some of my methods are called only once and I hate to use VAR and DAT blocks for their variables. I could sure use an advance class on memory usage. It makes sense that the local variables are not counted in stack space since not all the methods that declare them will be using them at the same time. I have found if I keep my available RAM above 100 longs everything works great. I wish I had some way to monitor the space used by temporary variables. Or specify a memory range they have to fit into so i can monitor its usage like I do my COG stack. I can monitor my COG's stack space since I know the address it takes up but temp variables I have no clue where they will be?? Thanks Heater for the response.

Rayman

I think there is some stack space utility code thing from Parallax in OBEX.
I believe it can calculate your actual stack space usage.
I've never used it though...

MagIO2

The local variables of a function are created on the stack of the COG that's calling the function.

Phil Pilgrim (PhiPi)

IIRC, accessing local variables beyond about the eight long on the stack gets expensive, time-wise, due to the byte-code interpreter having to use a longer instruction.

-Phil

Duane Degn

MagIO2 wrote: »

The local variables of a function are created on the stack of the COG that's calling the function.

I thought all Spin local variables are created on the stack in the hub. The cog is running the Spin interpreter but the code and local variables remain in the hub. (I think I learned this from one of Mike Green's posts.)

Duane

Phil Pilgrim (PhiPi)

The stack belongs to the cog that's accessing it, but it exists in the hub.

-Phil

Heater.

Duane Degan,
"...On the stack of the COG..."
Exactly what it says. The COG is full with and running the interpreter. The interpreter is running your Spin code. It uses a stack for your local vars and params in HUB.

Duane Degn

Thank you Phil and Heater. I see I read MagIO2's post incorrectly (I was surprised to think I might know something he didn't).

Duane Degn

Jorge P

Rayman wrote: »

I think there is some stack space utility code thing from Parallax in OBEX.
I believe it can calculate your actual stack space usage.
I've never used it though...

There is also a webinar video showing how to calculate the stack space, it is under "Memory Management:", "How can you determine how much stack space is needed for Spin code launched into another cog?" http://www.parallax.com/tabid/766/Default.aspx . It is a very simple and understandable explanation on how to do it.

Mike Green

You can also put a local variable in the outermost method running on a cog and store its address in a global variable. Any method where you're concerned about stack usage, you can calculate the difference between the address of a local variable in this method and that saved address. If the saved address is called stackBase and the new method's variable is called Temp, you'd calculate @Temp - stackBase. If the result is greater than some defined number of bytes, it would be an error. That's how FemtoBasic checks for expressions nested too deeply when it encounters a parenthesis or function call.