Unexplained Propeller Hang Up
Juliet Oscar Echo
Posts: 31
Propeller Community,
I have an issue with my code that causes the propeller to hang up randomly. I have been searching for the bug in my code on and off for over a year now and still haven't been able to find it. It has a very unique signature:
It will sometimes run perfectly for over 24 hours before crashing. Sometimes, though, it only runs for a few hours before crashing.
From what I understand, it is very rare to have an issue in your code that takes so long to cause a hang up. The main routine executes about once / minute.
Has anyone ever experienced a similar issue with their code? If so, where was the gremlin hiding? For now I have a cog dedicated as a watchdog, but I would rather find the culprit than patch it with an extra watchdog cog.
Thanks,
JOE
I have an issue with my code that causes the propeller to hang up randomly. I have been searching for the bug in my code on and off for over a year now and still haven't been able to find it. It has a very unique signature:
It will sometimes run perfectly for over 24 hours before crashing. Sometimes, though, it only runs for a few hours before crashing.
From what I understand, it is very rare to have an issue in your code that takes so long to cause a hang up. The main routine executes about once / minute.
Has anyone ever experienced a similar issue with their code? If so, where was the gremlin hiding? For now I have a cog dedicated as a watchdog, but I would rather find the culprit than patch it with an extra watchdog cog.
Thanks,
JOE
Comments
I'm gazing into my crystal ball.....I'm concentrating....trying hard to imagine what your code might look like.....nope, give up..........can't see what your code is so I can't comment.
1) stack overflow
2) array access out of bounds
3) timing dependencies
I was powering the board over usb, so the program would run reliably for minutes at a time, then it would reset for no apparent reason.
So keep an open mind; problems aren't always caused by code.
I appreciate your insight. One of my first Propeller projects had me trying to write an auto pilot for my model airplane. For over a week I tried to find a random bug that turned out to be brownouts caused by too much load on one of the servos. All it took was a more robust power supply and the problem was solved.
APRS_Tracker.spin
APRS_Packet_Generator_V35.spin
Bell202_Modem_Mod.spin
umath.spin
GPS_Light_V1.spin
GPS_Str_NMEA_Heavy_V1.spin
FloatMath.spin
FullDuplexSerial.spin
MS5607.spin
(Don't include the Propeller Tool"
repeat MainAnd remove the call to "Main" from the end of "Main", your code should behave much better.
Alternatively, you could make "Main" an endless loop with a repeat at the beginning of the method and no call to "Main" at the end.
PUB Main''----------------------------------------------------------------------------------- repeat 'Transmitts position, temperatre, and pressure once every minute. Get_GPS_Data 'Get Sensor Data MS5607.Read MS_degC := MS5607.Temperature/10 MS_Pa := MS5607.Pressure 'Transmit Transmit_Data Sigma := Sigma + 1 Display_Data 'For debug only. Comment out otherwise. wait(60)I appreciate your advice. I never knew Spin had an issue with recursive methods.
Unfortunately, I have also run versions of the code where I used a repeat loop in the main method and still had the random hang up anywhere from a couple of hours to over a day later.
Again, we would need to see the code to identify the trouble. The recursive method was certainly a problem in the code you posted.
If the code still hangs with the changes I suggested, let us know.
If you have a version of the code which you think already has this change, then it would help if you posted the other version.
Attached is a version of my code without the recursive method issue.
To reiterate what I said earlier, my code hangs when I integrate APRS_Packet_Generator_V35.spin with Bell202_Modem_Mod.spin into my project. I believe the issue is most likely in APRS_Packet_Generator_V35.spin (maybe in Bell202_Modem_Mod.spin).
Thanks again to everyone who is helping me with this problem.
What do you mean when you say "Spin doesn't handle recursion well" and "all languages have problems with recursion"?
I don't know of any stack oriented language problems associated with recursion. Recursion certainly works just fine with the Spin compliler.
However, recursion requires stack space, but that's not a language problem ...
Agreed.
I think Heater said it better than I with the qualifier "run away".
It's not a problem with Spin. It's a problem with the structure of the program.
I spoke imprecisely. Thanks for the correction.
NMEA devices are notorious for not matching the NMEA specification, which is notoriously lax. The way you're parsing sentences relies on pretty compliant devices. There's no error check to be sure your assumptions are being met. Your parser does check checksums if they're present, but it's not clear that your NMEA device includes checksums, which are optional.
Your packet generator has a hard coded limit of 150 bytes, but there's no checking to ensure you don't exceed that limit.
Without understanding much about the application, its just not written in such a manner that staring at the code is likely to turn up anything. I'd strongly suggest you do the following:
Fill all unallocated memory with a constant pattern and check all that memory in your main loop where you wait. Similarly put a guard after the Packet in your packet generator and check that on each return. Try to figure out a similar strategy for your parser.
From a coding perspective, you should change "pub" to "pri" everywhere you can without getting compiler errors. As it stands, it's really impossible to understand what's calling what.
I think I'll but the NMEA parser on the back burner for now and focus on the Packet Generator as I have implemented my current NMEA parser in may projects without any problems... so far.
As for the 150 byte hard coded limit, it's significantly higher than any packet I've ever had to transmit. However, it may be possible that it doesn't start at 0 on each itteration. I haven't been able to prove this though.
As for your two suggestions on filling up all unallocated memory and placing a guard after the Packet in the packet generator, they both sound like great ideas. Is there any chance you have some example code to get me started?
Thanks
I posted some code here which does this with stack space. The program dumps the area being used as stack space to the terminal each loop. It also counts up the unchanged memory. If you don't get a better suggestion, you might want to modify the code to monitor your the "unallocated memory".
I guess you'd find the address of the unallocated memory by using F8?
If you're monitoring a large chunk of memory, you probably don't want to dump it all to the terminal. You could just keep track of how much memory remains unchanged.
Some of the unallocated memory will be used as the stack for cog #0 but I'm not sure how this works.
My recommendation would be to simply declare a very large array of longs - big enough that memory is nearly full ... and use that array to fill "unused" memory.