Shop OBEX P1 Docs P2 Docs Learn Events
Integer only math (almost done) — Parallax Forums

Integer only math (almost done)

ArchiverArchiver Posts: 46,084
edited 2003-06-25 20:46 in General Discussion
Ok well I figured out I need to * by 100 to start things.

I start out with this:

4.96 * adcbits / 255

Now we will have:

4.96 * 100 = 496

and

255 * 100 = 25500

So now we have:

496 * adcbits / 25500

Does anyone see the problem? If you do 496 * 255 this will produce the
number 126480 which is to big for the basic stamp correct? The max
integer size is 65535 I believe? So what I found out in this case from
trial and error I found that 496 and 25500 divide into 4 even with out
producing a fraction. So now I have:

124 * adcbits / 6375

Ok? Now this *should* work if my calculations are correct. This also
solves my problem sense its a totally new formula then what the parallax
lab uses and it will work with both floating point numbers and integers!
The problem I'm having is finding out what to divide the 496 place of the
formula with. In this case 4 worked... I found it from trial and error...

So we have now:

a * b / c = e

I know what 'a' will be and I know what 'c' will be. I need to be able to
divide 'a' and 'c' into a number with out producing a fraction. I think
the biggest number like in 100 would be best because the smaller a is the
smaller number it will produce (use less memory?).

Is there a formula or way to go about finding what I can divide 'a' into
with out producing a fraction? This is what I'm trying to find:

a / x
c / x

I need to find x...
Sign In or Register to comment.