Issue with basic math calculation

Comments

• joeysr20det Posts: 8

  2013-12-10 20:47 edited 2013-12-10 20:47

  Oh geez... long day. I'm switching to a floating point. I'm good
• Duane Degn Posts: 10,588

  2013-12-10 21:54 edited 2013-12-10 21:54

  You could use 1077 instead of 107.7 and then divide by 10 at some point. Alternatively you could leave in the factor of 10 and just know the value is ten times larger than normally expected.
  
  Here's a method to display these sorts of scaled numbers.
  

  PUB DecPoint(value, denominator)
    if value < 0
      Pst.Char("-")
      -value
        
    if value => denominator
      result := value / denominator
      Pst.Dec(result)
      value //= denominator     
    else    
      Pst.Char("0")
    Pst.Char(".")  
    repeat while denominator > 1
      denominator /= 10
      if value => denominator
        result := value / denominator
        Pst.Dec(result)
        value //= denominator
      else
        Pst.Char("0")
        
  

  
  To display 107.7, you'd use.
  

    DecPoint(1077, 10)
  

  
  You can actually lose precision by switching to floating point.
• joeysr20det Posts: 8

  2013-12-11 08:12 edited 2013-12-11 08:12

  Thanks Duane. That's what I ended up doing last night (dropping the decimal out and changing my multiplier). It's sad how long I starred at the code before the light bulb went off 16hrs of coding will do that though.
  
  Thanks for the pointers about the floating point precision. I hadn't though of that, and in this case, I need to be very precise. I'm working on an FM Tuner object in SPIN for the Philips TEA5767 chip. I just now got it working and the sound quality is awesome. Best of all, I bought a lot of ten off ebay and the per chip price only cost me about 80 cents each.
  
  I will be submitting this object to OBEX for others to use when I get done. Everyone on this forum has been so helpful.
  
  Thanks again.
• Duane Degn Posts: 10,588

  2013-12-11 11:12 edited 2013-12-11 11:12

  joeysr20det wrote: »

  Is there something special I need to be doing if I'm specifying a number that is not a whole number (i.e. 107.7) for multiplication?

  
  I should have also mentioned the Prop Tool can do floating point calculations with pre-compiled constants but all the numbers need to be floats. You can't mix floating point and integers.
  
  You can use the "float" command to tell the compiler to treat a number as a floating point number such as " x = 107.7 * float(1_000_000)" but it's just as easy to add a ".0" to tell the compiler to treat the number as a floating point. The same constant could have been set with "x = 107.7 * 1_000_000.0".
  
  The constant "x" in the above examples would be formatted as a floating point number.
  
  I believe theres also a command to convert the pre-compiled number to an integer but I don't recall what it is.
  
  So you can kind switch between floats and integers in precompiled code but at runtime you need a floating point object (F32 is best) to deal with floating point numbers.
• joeysr20det Posts: 8

  2013-12-11 12:04 edited 2013-12-11 12:04

  Cool. Thanks for the tips. I really appreciate it.
  
  I've got the code working really well now, the radio tunes nice and clear