Shop OBEX P1 Docs P2 Docs Learn Events
Possible error in the Float32.compare-method — Parallax Forums

Possible error in the Float32.compare-method

ChristianGChristianG Posts: 29
edited 2004-08-23 06:24 in General Discussion
I think it may be an error in the compare-method for the Float32 class.
·
Consider:
·
Float32 fVar1 = new Float32();
fvar1.set(0);
·
Float32 fvar2 = new Float32();
fVar2.set(10);
·
fVar2.compare(fvar1) returns 1 !!! and not –1 as expected.

In other words, comparing floats with zero, gives the wrong result.
It might be an error in the Fpu32.compareAB-method, but don’t really understand the code there (the CPU.carry() is undocumented).

Regards

Comments

  • J. A. StreichJ. A. Streich Posts: 158
    edited 2004-08-23 00:50
    If comapare works the same as compareTo() or a Comparator's compare() method then a positive value is expected.

    A.compareTo(B);
    will return:
    A > B -> a positive value (typically 1)
    A == B -> 0
    A < B -> a negivite value (typically -1)

    since 10 > 0 then
    fVar2.compare(fvar1);
    should return positive and
    fvar1.compare(fVar2);
    should yeild a negitive number (-1)

    I've not used the Javelin, but that is how most Java comparison works.
  • ChristianGChristianG Posts: 29
    edited 2004-08-23 06:24
    My code was working as expected with values unlike zero, but comparing to zero gave the wrong result.
    The sample code was just from memory,·I may have switched A with B. I have to retest that...
    Thanks,

    ChristianG
    ·
Sign In or Register to comment.