Question about FloatString

undermutt

I've been playing with F32 and FloatString and I have a question/concern about FloatString.

If I have this...

JDBase := 1_456_532.5
pst.str(fs.FloatToFormat(JDBase, 9, 1))

I get this...
1_456_532.5

if I change JDBase to ....
JDBase := 2_456_532.5

I get this ...
2_456_532.8

Is this a problem with FloatString or is the number too large?
If the problem is FloatString, how can I display results reliably to the terminal?

Thanks,

Rick.

Duane Degn

I'm pretty sure the problem lies with how many significant digits you're attempting to use. I think you're limited to somewhere around six significant figures with a 32-bit floating point number.

If you're just doing basic math, you could use scaled integers in order to retain more precision.

undermutt

Duane,

My concern is that I didn't do any math at all on the value, just changed it from 1 million something to 2 million something and the decimal piece went from .5 which is correct to .8 which is incorrect.

In the long run I'll be doing a little more than basic math so the floating point is essential. I just need to know that the results will be accurate and if so how to display the results accurately.

Thanks,

Rick.

Mike Green

Here's a description of the standard format used for storing floating point values. Note that there are 24 bits of precision available (approximately 7 decimal digits). What you're observing is what's normally expected. As Duane mentioned, you can use scaled integer arithmetic to get more precision or you could rewrite the floating point library to allow double precision floating point. Again, this is an inherent limitation of 32-bit IEEE standard floating point.

Duane Degn

undermutt wrote: »

My concern is that I didn't do any math at all on the value, just changed it from 1 million something to 2 million something and the decimal piece went from .5 which is correct to .8 which is incorrect.

You didn't do any math, but the compiler did. I believe packing the number into the floating point format involve taking the log of the number. 8 significant digits if beyond what can be reliably stored in a 32-bit floating point number. I think if you did do some math with the number successfully stored, you'd see rounding error very quickly.

If you want to use that many significant digits, you'll need to deal with the numbers in a different way than using F32 or any of the other 32-bit floating point math objects.

undermutt

Ok. Thanks. As I think I've said previously I'm relatively new to this whole microprocessor world. So I learn as I go.

I believe that the only place I will need to use a number that large is with the Julian date and I'm sure I can figure out a workaround for that little problem.

Now that I know what is going on I feel much better.