View Full Version : Help with math on prop

Old man Earl
04-06-2009, 10:36 AM
I have lost most of my math memories due to the fact I am 63 and never did know math ! Here is the problem. I need spin code for solving the formula to convert mmHg to altitude in feet. I have an i2c module that returns the mmHg number ~ 600 here in Mountainair,NM. I know the alt here is ~6500 feet. I could use the float32 math routines in the program, but I don't know how to write the code.
Someone please help....

mmHg is ~ 600
The formula is .....

altitudeF :=((1-(mmHg*1.333224/1013.25)^0.190284)*145366.45)

Result is altitude in feet.

I know the formula works because my 16 yo daughter showed me on her fancy dan calculator the answer comes out correct !
I know the answer is right because I go to an online calculator for solving mmHg to altitude.


04-06-2009, 01:28 PM
Hello Earl,

I made a XY-Chart with Excel to see if an easier to calculate function could give a good approximation.
Even a linear function has a correlation of 99,53 percent ! And a quadratic function fits to R^2 = 1 !
(see attached file)

with the Floating Point-object (http://obex.parallax.com/objects/202/) from the obex it will be easy to do
the calculations of the linear and the quadratic function

best regards


04-06-2009, 02:43 PM
Only an additional thought:
If you are short with memory (or need higher speed) then integer math will do it as well. Guess the result being in feet (without fractions) is accurate enough.
The linear solution given in the previous post is nearly integer.
y=-46,002x+34339 is not to different from y=-46x+34339 and should still be accurate enough if the linear solution is accurate enough.

If you prefer the polynomic solution you can simply convert the fractional numbers into integer fractions.
E.g. 0,0238 = 1/42,016806722
The max. mmHg of say 800 allows to multipy by 3355 without overflow at 32 bit integer, so we can multipy that by 1000 to have more digits:
1000 / 42017
So, the calculation for the x^2-part would be
(1000*x*x) / 42017
x part can be modified similarly
72346*x / 1000

So, the whole formula looks like:
y:= (1000*x*x) / 42017 - (72346*x) / 1000 + 41214
and the result for x=600 is y=6374
using float would be 6374,4 (used the windows calculator for that)

edit: put "or need higher speed" into brackets. This has to be testet. Maybe the two divisions need more time than the float multiplications?!

Post Edited (MagIO2) : 4/6/2009 7:51:02 AM GMT

04-06-2009, 06:39 PM
Okay, let's have a small lesson in physics. And then let's see what this implies to the math.

The formula is:

altitude [ft] :=((1 - (p [mmHg] * 1.333224 / 1013.25) ^ 0.190284) * 145366.45)

Look at the last factors:

145366.45. It has eight significant digits. Wow! Changing the least significant digit by one to 145366.46 would lift you up a bit more than a single inch. That's really great.
But wait. This only holds if there are no other errors in the calculation. The factor 1013.25 is the pressure measured in hPa at sea level in 'standard atmosphere.' But weather almost never is standard. At the closest airport to my current location we had 1015 hPa - pretty close.
The 1.333224 is the density of mercury in g/cm3. Wikipedia gives a different value with less accuracy. Six significant digits after the point mean an accuracy of 1 PPM. Maybe the creator of the formula has measured the volume of a mercury sample of 1l at a precision of 1/1000 ml and has put those some 11 kg on a scale that measured at an accuracy of some .01g.
The exponent of 0.190284 also has an empirical origin. It's from the fact that atmospheric pressure reduces to half every 5,400 ft or so. The 'or so' means that you can have a deviation of several ten feet here.

Oh, and there are lots of other variables. Outside temperature, humidity... to name only two of them.

So what does this mean to us? We are trying to compute at a resolution of six to eight digits with constants we only know at an accuracy of two, three, or at most four digits. Using an aviation barometric altimeter you can expect an error of a few feet if you calibrate it to current conditions. We are in safe water - sorry, 'safe air' if our computation adds an error that is smaller. Not necessarily several magnitudes, but a factor of two to five is completely sufficient. This would change the formula to something like

altitude [ft] :=((1 - (p [mmHg] * 1.333 / 1013) ^ 0.1903) * 145400)

What about integer calculation instead of FP? If you scale the values accordingly you can use the full 32 bits. Subtraction, multiplication, and division are no problem in SPIN. Exponentation can be done with the anti-log table. If I understand the manual correctly, this table has 2048 entries for the interval [1, 2) giving sufficient accuracy without further interpolation.

BTW, have a look at airspace-v.com/ggadgets/densityAltitude.htm (http://airspace-v.com/ggadgets/densityAltitude.htm) to see how weather changes air density, air pressure and altitude correlate and depend on uncontrollable factors - weather. Change the conditions only a little bit and watch the results. Then try rounding errors and watch the results, too. See what cause gives larger differences...

P.S.: Try sauna or steam bath conditions at high altitudes. When I did it I understood why I always have to yawn in the heat...

P.P.S.: Sorry for metric units regarding volumes and weights. I don't have a feeling for pounds, onces, or gallons.

Airspace V - international hangar flying!
www.airspace-v.com/ggadgets (http://www.airspace-v.com/ggadgets) for tools & toys

04-06-2009, 08:41 PM
Here is a straightforward calculation of your formula using floating-point math. If it isn't fast enough for you or uses too many cogs, consider using some of the alternative approaches the other guys have mentioned.

_clkfreq = 80_000_000
_clkmode = xtal1 + pll8x ' <== alter as appropriate
videoPin = 24 ' <== alter as appropriate

fp : "float32full"
floatstring : "floatstring"
print : "tv_text"

pub Main
print.start( videoPin )
print.str( floatstring.floatToString( ComputeAltitude( 600.0 ) ) ) ' prints 6392.983

pub ComputeAltitude( mmHg ) : altitudeF | temp1, temp2
'' input: mmHg -- air pressure in mmHg (floating-point)
'' output: altitude in feet (floating-point)

' The formula is
' altitudeF :=((1-(mmHg*1.333224/1013.25)^0.190284)*145366.45)
' We'll break it down into smaller pieces:
' temp1 := mmHg * 1.333224 / 1013.25
temp1 := fp.FDiv( fp.FMul( mmHg, 1.333224 ), 1013.25 )

' temp2 := 1 - temp1 ^ 0.190284
temp2 := fp.FSub( 1.0, fp.Pow( temp1, 0.190284 ) )

' altitudeF := temp2 * 145366.45
altitudeF := fp.FMul( temp2, 145366.45 )

Old man Earl
04-07-2009, 01:22 AM
Thanks to all !!!!

The program now reports altitude in feet !

One question tho... how much change in mmHg does it take to change the altitude ?

here is the program now

Post Edited (Old man Earl) : 4/6/2009 8:02:19 PM GMT

04-07-2009, 02:41 AM
Googling 'mmhg to feet' finds this table:


-50 feet = 761.37 mmHg
·· 0 feet = 760.00 mmHg

so a difference of 1.37 mmHg per 50 feet, or 0.0274 mmHg per foot, approximately.· Since the equasion isn't actually linear this will vary a little, but as previously mentioned, it's very close to linear within the range you're likely to use, so this should be fine.


Old man Earl
04-07-2009, 03:01 AM
thanks again...you may notice I am using the Integer math solving...
works great

04-07-2009, 03:15 AM
Well, in (metric) flight instruction you learn a rule of thumb that climbing 8 m decreases pressure by 1 hPa. This converts to 1 mmHg per 35 ft or .0286 mmHg per ft. Jason, this is rather close to the value you found!

But beware: This factor is (inverse) proportional to pressure. If you ski at Colorado you can get up to elevations of almost 11.000 ft. Up there pressure is less than 70% of that at sea level giving you some 50 ft per mmHg.

Airspace V - international hangar flying!
www.airspace-v.com/ggadgets (http://www.airspace-v.com/ggadgets) for tools & toys

04-07-2009, 03:29 AM
I like your code ... especially the line where it says "altitudeF:= ...."

Guess I made the things to easy for you ;o) Maybe all the helping hands should post a bit more in quizz-style.

04-07-2009, 10:47 AM
Oh, by the way. I don't like the next line of your code.

It tells me that you did not learn something. You divide the feet by 3 even if you have a more accurate divisor at hand (when I remember it right is something around 3.28.... I'm currently writing on my PS3 and can't unzip)
So, lets divide 1000 by 3 => 333 and now lets divide it by 3,28 => 304.
This means with each 1000ft you have add an error of 29m.

What I wanted to teach you is: Find out the number range of dividend and divisor and multiply both with the max. value possible without breaking the range of 32bit numbers. Mulltiplying both dividend and divisor does not change the result in float, in our integer math we want to calculate with numbers that in truth would be floats. Here we add accuracy by doing that. 1000*ft / 1000*divisor gives you a better result! (Of course you can pre-calculate whatever possible - 1000*divisor in your case)

So, height_m := 1000*height_ft / 328x (forgot the·last digit, please replace) would be by far better. Hope you leaned something·this time·;o)

Post Edited (MagIO2) : 4/7/2009 4:53:34 AM GMT

Ole Man Earl
04-08-2009, 03:37 AM
Thanks..I THINK I may have learned something..Ya gotta remember I am old, I do thank you and the others tho..
BTW the worlds best kept secret..... It HURTS to get old !

Post Edited (uphiearl) : 4/7/2009 8:44:02 PM GMT

04-08-2009, 04:01 AM
Can't be so old in mind, you're still playing with propellers ;o)

Ole Man Earl
04-08-2009, 04:08 AM
Redid my avatar....yup, that's me. My hippie part had now turned white...Oh well, couldn't have it when I was teacher !
Yes, playing with the prop does keep me going...as well as owning and running a WISP (wireless internet service provider) for a small (1200) town in NM !