Issue with basic math calculation

joeysr20det · 2013-12-10 20:34

Can anyone tell me why the following in SPIN

frequencyCalc := (4*(107.7*1000000+225000))/32768

does not yield a result of

13174

I'm kinda stuck here. I've defined the frequencyCalc variable as a Long so I know I have a 32bit space to work in. The number at its largest point is 29bits (107.7*1000000+225000)*4

I've separated each step of the calculation out to eliminate order of operators as well and I still have the issue. Is there something special I need to be doing if I'm specifying a number that is not a whole number (i.e. 107.7) for multiplication?

Thanks for any input anyone has. This one has been a toughy

joeysr20det · 2013-12-10 20:47

Oh geez... long day. I'm switching to a floating point. I'm good

Duane Degn · 2013-12-10 21:54

You could use 1077 instead of 107.7 and then divide by 10 at some point. Alternatively you could leave in the factor of 10 and just know the value is ten times larger than normally expected.

Here's a method to display these sorts of scaled numbers.

PUB DecPoint(value, denominator)
  if value < 0
    Pst.Char("-")
    -value
      
  if value => denominator
    result := value / denominator
    Pst.Dec(result)
    value //= denominator     
  else    
    Pst.Char("0")
  Pst.Char(".")  
  repeat while denominator > 1
    denominator /= 10
    if value => denominator
      result := value / denominator
      Pst.Dec(result)
      value //= denominator
    else
      Pst.Char("0")

To display 107.7, you'd use.

  DecPoint(1077, 10)

You can actually lose precision by switching to floating point.

joeysr20det · 2013-12-11 08:12

Thanks Duane. That's what I ended up doing last night (dropping the decimal out and changing my multiplier). It's sad how long I starred at the code before the light bulb went off

16hrs of coding will do that though.

Thanks for the pointers about the floating point precision. I hadn't though of that, and in this case, I need to be very precise. I'm working on an FM Tuner object in SPIN for the Philips TEA5767 chip. I just now got it working and the sound quality is awesome. Best of all, I bought a lot of ten off ebay and the per chip price only cost me about 80 cents each.

I will be submitting this object to OBEX for others to use when I get done. Everyone on this forum has been so helpful.

Thanks again.

Duane Degn · 2013-12-11 11:12

joeysr20det wrote: »

Is there something special I need to be doing if I'm specifying a number that is not a whole number (i.e. 107.7) for multiplication?

I should have also mentioned the Prop Tool can do floating point calculations with pre-compiled constants but all the numbers need to be floats. You can't mix floating point and integers.

You can use the "float" command to tell the compiler to treat a number as a floating point number such as " x = 107.7 * float(1_000_000)" but it's just as easy to add a ".0" to tell the compiler to treat the number as a floating point. The same constant could have been set with "x = 107.7 * 1_000_000.0".

The constant "x" in the above examples would be formatted as a floating point number.

I believe theres also a command to convert the pre-compiled number to an integer but I don't recall what it is.

So you can kind switch between floats and integers in precompiled code but at runtime you need a floating point object (F32 is best) to deal with floating point numbers.

joeysr20det · 2013-12-11 12:04

Cool. Thanks for the tips. I really appreciate it.

I've got the code working really well now, the radio tunes nice and clear

Issue with basic math calculation

Comments