Shop OBEX P1 Docs P2 Docs Learn Events
Floating point rounding ? — Parallax Forums

Floating point rounding ?

BitsBits Posts: 414
edited 2011-12-30 10:01 in Propeller 1
Is there an object out there that will take a floating point number and round it as follows:

example numbers

1. 25.3600 to 25.3633
2. 25.3634 to 25.3666
3. 25.3667 to 25.3699

and the functions should crank out

1. 25.3600
2. 25.3633
3. 25.3666

Comments

  • Heater.Heater. Posts: 21,230
    edited 2011-12-29 23:45
    This looks like a really odd thing to want to do and you might have to specify the rules for the results you want a bit better to get any suggestions here.
    On a higher level what is it you actually want to do with this and/or why? Perhaps there is a better way.
  • Heater.Heater. Posts: 21,230
    edited 2011-12-29 23:50
    Of course you can use lonesock's F32 object (In the OBEX) to do floating point comparisons. Use it to write a method that checks numbers for the ranges specified and returns the required result. But this is not a general case solution.
  • Duane C. JohnsonDuane C. Johnson Posts: 955
    edited 2011-12-30 10:01
    It looks like you want to split a digit by thirds instead of the usual halves.

    A general method is to multiply the number by 1.5, round it in the normal way, then divide it by 1.5.

    In your example multiply by 150, round, then divide by 150.

    Duane
Sign In or Register to comment.