Integer Math

Archiver

There have been some posts on "Multiplication Overflow(Truncation)"
and I understand the concept within.

However, I have run into something with Integer math that I don't
understand. I too am working in VB to simulate integer math. I
have been very successful up to this point dealing with numbers >
65535 and < 0 and getting them to display as a BS2 would display
them.

The thing that questions me is when using this code...

a CON 65535
b CON a*a/2
DEBUG DEC b

I would think that the stamp would display zero, but it displays 1.

So here is what I come up with...

$FFFF * $FFFF = $FFFE0001
Truncated this equals 1.
Then dividing by 2 (1/2) this should equal 0.

But the Stamp insists that the result is one... [noparse];)[/noparse]

Can someone explain what is happening here?

Thank you,
Jared Hoylman

Archiver

OK, I think I am partially right in my first post.

The result should be zero. But since the constants are calculated
at compile time this is most likely a compiler bug, correct?

DEBUG DEC 65535*65535/2
...does diaply zero.

But when calculated at compile time it results in a one...

Is this a compiler(tokenizer) bug?
[noparse][[/noparse]I am using v2.0 Beta 2]

Jared



--- In basicstamps@yahoogroups.com, "microextra" <microextra@y...>
wrote:
>
> There have been some posts on "Multiplication Overflow
(Truncation)"
> and I understand the concept within.
>
> However, I have run into something with Integer math that I don't
> understand. I too am working in VB to simulate integer math. I
> have been very successful up to this point dealing with numbers >
> 65535 and < 0 and getting them to display as a BS2 would display
> them.
>
> The thing that questions me is when using this code...
>
> a CON 65535
> b CON a*a/2
> DEBUG DEC b
>
> I would think that the stamp would display zero, but it displays 1.
>
> So here is what I come up with...
>
> $FFFF * $FFFF = $FFFE0001
> Truncated this equals 1.
> Then dividing by 2 (1/2) this should equal 0.
>
> But the Stamp insists that the result is one... [noparse];)[/noparse]
>
> Can someone explain what is happening here?
>
> Thank you,
> Jared Hoylman

Archiver

That is interesting, but when you write,
b con a*a/2

it is the compiler does that computation, before it downloads the
program to the Stamp.

Run this:

a CON 65535
b var word
b = a*a/2 ' now, the Stamp computes this!
DEBUG DEC b ' the result is zero.

I would consider the thing you noticed a bug in the compiler (but not
in the Stamp!). The compiler should follow the truncation rules of
integer math when it precomputes those constants.

-- Tracy




>There have been some posts on "Multiplication Overflow(Truncation)"
>and I understand the concept within.
>
>However, I have run into something with Integer math that I don't
>understand. I too am working in VB to simulate integer math. I
>have been very successful up to this point dealing with numbers >
>65535 and < 0 and getting them to display as a BS2 would display
>them.
>
>The thing that questions me is when using this code...
>
>a CON 65535
>b CON a*a/2
>DEBUG DEC b
>
>I would think that the stamp would display zero, but it displays 1.
>
>So here is what I come up with...
>
>$FFFF * $FFFF = $FFFE0001
>Truncated this equals 1.
>Then dividing by 2 (1/2) this should equal 0.
>
>But the Stamp insists that the result is one... [noparse];)[/noparse]
>
>Can someone explain what is happening here?
>
>Thank you,
>Jared Hoylman

Archiver

Tracy,

Thanks for the quick answer. I agree with you that it is the
compiler that is wrong. I will forward this on to Parallax.

Thanks again,

Jared


--- In basicstamps@yahoogroups.com, Tracy Allen <tracy@e...> wrote:
> That is interesting, but when you write,
> b con a*a/2
>
> it is the compiler does that computation, before it downloads the
> program to the Stamp.
>
> Run this:
>
> a CON 65535
> b var word
> b = a*a/2 ' now, the Stamp computes this!
> DEBUG DEC b ' the result is zero.
>
> I would consider the thing you noticed a bug in the compiler (but
not
> in the Stamp!). The compiler should follow the truncation rules
of
> integer math when it precomputes those constants.
>
> -- Tracy
>
>
>
>
> >There have been some posts on "Multiplication Overflow
(Truncation)"
> >and I understand the concept within.
> >
> >However, I have run into something with Integer math that I don't
> >understand. I too am working in VB to simulate integer math. I
> >have been very successful up to this point dealing with numbers >
> >65535 and < 0 and getting them to display as a BS2 would display
> >them.
> >
> >The thing that questions me is when using this code...
> >
> >a CON 65535
> >b CON a*a/2
> >DEBUG DEC b
> >
> >I would think that the stamp would display zero, but it displays
1.
> >
> >So here is what I come up with...
> >
> >$FFFF * $FFFF = $FFFE0001
> >Truncated this equals 1.
> >Then dividing by 2 (1/2) this should equal 0.
> >
> >But the Stamp insists that the result is one... [noparse];)[/noparse]
> >
> >Can someone explain what is happening here?
> >
> >Thank you,
> >Jared Hoylman