Possible error in the Float32.compare-method
ChristianG
Posts: 29
I think it may be an error in the compare-method for the Float32 class.
·
Consider:
·
Float32 fVar1 = new Float32();
fvar1.set(0);
·
Float32 fvar2 = new Float32();
fVar2.set(10);
·
fVar2.compare(fvar1) returns 1 !!! and not –1 as expected.
In other words, comparing floats with zero, gives the wrong result.
It might be an error in the Fpu32.compareAB-method, but don’t really understand the code there (the CPU.carry() is undocumented).
Regards
·
Consider:
·
Float32 fVar1 = new Float32();
fvar1.set(0);
·
Float32 fvar2 = new Float32();
fVar2.set(10);
·
fVar2.compare(fvar1) returns 1 !!! and not –1 as expected.
In other words, comparing floats with zero, gives the wrong result.
It might be an error in the Fpu32.compareAB-method, but don’t really understand the code there (the CPU.carry() is undocumented).
Regards
Comments
A.compareTo(B);
will return:
A > B -> a positive value (typically 1)
A == B -> 0
A < B -> a negivite value (typically -1)
since 10 > 0 then
fVar2.compare(fvar1);
should return positive and
fvar1.compare(fVar2);
should yeild a negitive number (-1)
I've not used the Javelin, but that is how most Java comparison works.
The sample code was just from memory,·I may have switched A with B. I have to retest that...
Thanks,
ChristianG
·