Shop Learn
Dividing a signed double by another, result to be a signed double ? — Parallax Forums

Dividing a signed double by another, result to be a signed double ?

Anyone have a Taqoz word for this they can share? It's the last operator needed to complete an air pressure sensor driver (BMP280). It's needed for the conversion from raw reading to calibrated hPa. 64 bit math is said to give best precision for altimeter use, for example.


  • Hi Bob,
    Do you know the possible range of these numbers? Normally even 16 bits are more than adequate for physical data, because noise does not allow higher resultion. I try to use scaling and use the */ operator.

  • bob_g4bbybob_g4bby Posts: 257
    edited 2021-11-25 23:23

    I also suspect excessive precision is being used - for now I'm following the recipe in the data sheet before refining it. I've written all the other 64 bit signed operators required, but can't remember how to do multi-cell signed division. There's a worked example to try after I've reproduced all the complicated formulae, if I get the same result, then I can experiment with simplifying the math.

  • evanhevanh Posts: 12,021

    Your friend is Spin's muldiv64(). Make a version of that in Forth and you'll be racing.

    Here's a Spin version with rounding to nearest, I named muldiv65() -
    And the subsequent update for IEEE de-biasing method of an exact 0.5 rounding -

  • That's a good tip, thanks Evanh, I'll try that

Sign In or Register to comment.