Memory Stomp: how to avoid it, how to detect it, how to rid it from the world?
ElectricAye
Posts: 4,561
One of my greatest anxieties when faced with learning how to work with microprocessors is this thing that happens where memory space and/or downloaded code gets eaten alive by processes that seem obscure and obscenely uncontrollable.· My own carbon-based brain hungers for a map of where all these bits and bytes are doing their thing in real time, showing me where they are coming from and where they are going to.· I yearn·for some kind of system to prevent variables and code from colliding and stomping over each other.· I get ulcers just thinking how it's possible that my computer program can call up methods that cannibalize each other and consume my precious data like it was last year's Halloween candy.· What is the answer to my breathless byte-based panic?· How do I map out all my variables and arrays and ascertain that various processes aren't going to gobble up each other like a barrel of snakes, each swallowing the tails of its neighbors and sometimes its very own?
Are there tools?· Are there tricks?· Is there some way to map out all of what's happening when you try to integrate warring tribes of code?
Or is voodoo involved?
Mark
Are there tools?· Are there tricks?· Is there some way to map out all of what's happening when you try to integrate warring tribes of code?
Or is voodoo involved?
Mark
Comments
It can also generate warnings for some common coding gotchas.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Michael Park
PS, BTW, and FYI:
To search the forum, use search.parallax.com (do not use the Search button).
Check out the Propeller Wiki: propeller.wikispaces.com/
Without array bounds checking and banning pointers there isn't much which can be done to stop disaster happening other than through careful design. And I'd presume that's done anyway; errors like this are always accidental and unexpected.
For other variables, they should be safe from each other unless something very major has happened which has caused a cascade failure.
One trick for testing array accesses out of bounds is to put small arrays either side of the suspect array. That way a small out of range error corrupts those unused arrays not any other used data. It doesn't always work. If it seems to fix a problem at least you have something to focus on when debugging.
A common array error is to define it like myVar[noparse][[/noparse] 10 ] and then write to myVar[noparse][[/noparse] 10 ] which is out of bounds.
Your discussion of warring tribes of code reminds me of an old game, Core War, that was designed around that very concept. Multiple programs are run on separate cores that all share the same main memory, and whichever core can keep its process running the longest, without getting overwritten by another core, wins the game. Wikipedia even has an article about it at en.wikipedia.org/wiki/Core_war. Although the game doesn't teach much in the way of avoiding memory contention, it does a lot to demonstrate how to work with multi-core processors. Many of the simulators even have a real-time graphic view of the main memory while the cores are battling it out.
As far as your Propeller programs, there are two things I would suggest. First make sure that all variables and arrays are declared, and refer to them by their declared names. Second, always perform bounds checking. Strings can be especially tricky, because you may not know their length before you start moving or copying them. If you try putting 80 bytes in a 64 byte buffer, you will just overwrite 16 bytes that probably store completely unrelated variables. Whenever I am writing multiple bytes directly to an array, I always look backward into the code to see if I can prove that it cannot, under any circumstances, create an overflow. If there is any possibility, I'll add bounds checking so that if there is an overflow it will either return with an error or truncate what is being written so that it fits.
What I have mentioned also applies to single-core microcontrollers. You do not need multiple threads to clobber your memory; it's just easier to figure where the problem is. One more Propeller specific thing to watch out for is when one cog is reading and writing variables that are being read from and written to by another. Make sure that modifying the variables in main memory never puts them in a state that would be considered invalid by the other cog. This is also true for if you have multiple cogs ready, modify, then write a variable. If you have two cogs read the same byte, add a number, then write the result, both cogs will read the same number, add their own offsets, then write there own version back. By the time the first cog writes its result, the second cog will have already read the original number, and it will overwrite the first cogs result and the changes the first cog made will be lost.
- David Carrier
This is a very interesting idea. But I have a question: how do I sandwich an array between these "array boundary checker" variables? Do you simply declare them in the order in which you want them to be physically located? In other words, is it simply a matter of declaring variable A, then declaring array B, then declaring variable C? For example...
Long CheckerA
Long MemorySpace[noparse][[/noparse]20]
Long CheckerC
Like that?
many thanks,
Mark
If you are being paranoid you add a Check routine to every object. It checks every guard variable and calls Check in every object that object has. Then you have a cog in the main object that repeats calling Check for the main object and every object the main object uses. If each Check returns true/false - false if a check fails. Then the main cog knows very quickly if a check fails. If you see a failure then you add debug to track which array in which object is causing the problem.
The reason for a value like $DEADDEAD (another common value is $DEADBEEF) is so its easy to spot in a memory dump. So byte arrays will need a check value of $DE, words $DEAD and longs $DEADDEAD.
If you are really paranoid you leave the code in shipping code with the cog doing a reboot if it detects a memory corruption.
You can put these guards in other places if you want e.g. between any variable or in DAT memory, etc.
If you are really, really paranoid you run checksums of code and DAT segments but thats being extreme, mostly done to try and stop reverse engineering.
Excellent. I think I'm desperately going to need this technique in the upcoming days (weeks???).
Thanks for taking the time to explain this to me.
Mark
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
It might be the Information Age but the Eon of Ignorance has yet to end.