Floating point!
172heavy
Posts: 55
So I am using fmath to perform a simple calculation (i thought) and I am getting my butt kicked!
I am trying to convert the string 2933.8621 to the decimal degree format. I have managed to convert from a string to a decimal so I have the decimal value of 29 the decimal value of 33 and the decimal value of 8631 in seperate variables. I multiplied 33x10000 and added 8631 to get 338631 I then (using fmath.fdiv(338631,10000) get the fstring value of 33.8631 all I need to do now is divide 33.8631/60 and add this to 29 so i should get something like 29.564385 and this is where I am lost. Is it not possible to add two floating point values? I tried to trunc them and I got a little closer but I need the precision for my project. I have included the code to be thorough but I am really just looking for some basic guidance regarding the re-usage of floating point values.
I am trying to convert the string 2933.8621 to the decimal degree format. I have managed to convert from a string to a decimal so I have the decimal value of 29 the decimal value of 33 and the decimal value of 8631 in seperate variables. I multiplied 33x10000 and added 8631 to get 338631 I then (using fmath.fdiv(338631,10000) get the fstring value of 33.8631 all I need to do now is divide 33.8631/60 and add this to 29 so i should get something like 29.564385 and this is where I am lost. Is it not possible to add two floating point values? I tried to trunc them and I got a little closer but I need the precision for my project. I have included the code to be thorough but I am really just looking for some basic guidance regarding the re-usage of floating point values.
Comments
F : "FloatMath" to use Spin floating point
seccon := F.FFloat(10000) ' seconds fraction divisor in floating point format
decconv := F.FFLoat(60) ' seconds divisor in floating point format
minutes := F.FFloat(fracMinutes) ' minutes fraction x 10000 in floating point format
minutes := F.FDiv(minutes, seccon) ' divide by 10000
minutes := F.FAdd(minutes,F.FFloat(intMinutes) ' add integer minutes
degrees := F.FDiv(minutes, decconv) ' convert minutes to degrees
degrees := F.FAdd(degrees, F.FFloat(intDegrees)) ' and in full degrees
Key thing is that you must convert integer values into floating point format before doing floating point operations.
John Abshier
John,
Thank you, I kinda have it working now but after looking at your suggestion I see my error and all of the "workaround" I ended up with is way harder than your way. I will try this.
Thanks again,
Josh