Shop OBEX P1 Docs P2 Docs Learn Events
math operation question — Parallax Forums

math operation question

emixemix Posts: 4
edited 2006-08-30 20:23 in BASIC Stamp
is it possible to solve a 16bit division in BS2?

like this:·· 59336/65536 = 0.90539


·i know that stamp only support integer math.

but is there a way to play with the decimals and solve it?

0· 9 0 5 3 9
0.90539

Comments

  • Tracy AllenTracy Allen Posts: 6,664
    edited 2006-08-30 19:38
    Sure! You can easily get 4 decimal digits from,

    result = 59336 ** 10000
    debug "0.",dec4 result,cr ' prints 0.9053

    And better roundoff with
    result = 59336 ** 20000 +1 / 2
    debug "0.",dec4 result,cr ' prints 0.9054

    There are ways to get another digit or more of precision, with a little more work. It is also possible for an algorithm to kick out decimal digits ad infinitum if necessary.

    ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
    Tracy Allen
    www.emesystems.com
  • Bruce BatesBruce Bates Posts: 3,045
    edited 2006-08-30 19:39
    emix -

    I think you will find these applications notes VERY helpful (link below). Thanks go to Dr. Tracy Allen.
    http://www.emesystems.com/BS2math6.htm

    Regards,

    Bruce Bates

    ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
    <!--StartFragment -->
  • emixemix Posts: 4
    edited 2006-08-30 20:23
    it works pretty good,

    Thanks for your help.
Sign In or Register to comment.