Dividing a signed double by another, result to be a signed double ? — Parallax Forums

# Dividing a signed double by another, result to be a signed double ?

Posts: 386

Anyone have a Taqoz word for this they can share? It's the last operator needed to complete an air pressure sensor driver (BMP280). It's needed for the conversion from raw reading to calibrated hPa. 64 bit math is said to give best precision for altimeter use, for example.

• Posts: 901

Hi Bob,
Do you know the possible range of these numbers? Normally even 16 bits are more than adequate for physical data, because noise does not allow higher resultion. I try to use scaling and use the */ operator.

• Posts: 386
edited 2021-11-25 23:23

I also suspect excessive precision is being used - for now I'm following the recipe in the data sheet before refining it. I've written all the other 64 bit signed operators required, but can't remember how to do multi-cell signed division. There's a worked example to try after I've reproduced all the complicated formulae, if I get the same result, then I can experiment with simplifying the math.

• Posts: 14,231

Your friend is Spin's muldiv64(). Make a version of that in Forth and you'll be racing.

Here's a Spin version with rounding to nearest, I named `muldiv65()` - https://forums.parallax.com/discussion/comment/1525687/#Comment_1525687
And the subsequent update for IEEE de-biasing method of an exact 0.5 rounding - https://forums.parallax.com/discussion/comment/1529224/#Comment_1529224

• Posts: 386

That's a good tip, thanks Evanh, I'll try that