Working with Decimal values

I am having a terrible time time to figure out how to work with decimals.
I purchased one of the MS5607 Altimeters, which came with example code... but its completely formatted to displaying string output.

I can get a value from the altimeter driver, which returns a non-decimal (i.e. 87594 = 875.94ft), by doing this:
CM := alt.altitude(alt.average_press) ' Get the current altitude in cm, from new average local pressure.
ST := alt.convert(CM, alt#FEET) ' Record starting elevation, use alt.convert to change CM to FT

but if I try to divide by 100 or multiply by 0.01, it drops the decimal value and 87594 becomes 875 instead of 875.94.
I have tried using local var, byte, long and float.... I revisited the PEkit book and tried floatmath with fmath.fmul and fmath.fdiv. but those return a crazy value.

I could just use the non-decimal value, but I find it odd that I cannot work with or display a remainder.
What am I missing?


  • JBWolfJBWolf Posts: 336
    edited 2018-02-12 - 13:12:33
    I guess what im asking is.... why wouldn't this display "123.45", and how should I be working with decimals?

    a= 12345
  • This thread may be useful about floating point support (or the lack of it):
  • here is where im at... This code works great with the MS5607 Altimeter... Displays both altitude and temperature.
    just havent figured out how to work with and display decimals.

  • SeairthSeairth Posts: 2,465
    edited 2018-02-12 - 21:03:31
    In spin, integer division will always end up with an integer result. The remainder is just discarded. If you are simply trying to display the number, consider taking the following steps:
    fraction := CM // 100 
    whole := CM / 100

    Edit: Oh, yeah! Forgot about the left padding zero(es) for the fractional part. See the comments below for the proper solution. :D
  • ElectrodudeElectrodude Posts: 1,380
    edited 2018-02-12 - 15:53:40
    If you don't need a huge dynamic range (you usually don't), you could also fixed point instead, which is faster and simpler. Basically, you store numbers multiplied by some constant, say 100 or 256. If you made the constant 100, "123.45" would be stored as 12345. To print a number x, you'd first print x / 100 (which would get truncated), then you'd print a ".", then you'd print x // 100, ensuring that two digits of precision are used. In this comment, I'm using a multipler of 100 in examples for simplicity, but when I do it for real, I usually use the largest power of two that still gets me enough range, in order to maximize precision.

    To add or subtract fixed-point numbers, they need to have the same multiplier - scale one or both until they do. To multiply two numbers with a multiplier of 100, say 12.34 (stored as 1234) and 23.4 (stored as 2340), you multiply them (1234 * 2340 = 2887560) to get a number with a multiplier that's the product of the two input numbers: (12.34 * 100) * (23.40 * 100) = (288.7560 * 10000). If you want the result to have the original multiplier, you just divide the 2887560 by 100, which gives 28875 (printed as 288.75, which is the (truncated) correct answer). Watch out for overflow - divide before multiplication if you need to. You can split the division into two - one before, one after - to maximize precision while avoiding overflow.

    EDIT: use Spin modulus operator
  • how do you use the % in your example? --> "then you'd print x % 100"
    I cannot get any function out of a percentage sign... i.e.
    pst.dec((x % 100))
  • ElectrodudeElectrodude Posts: 1,380
    edited 2018-02-12 - 16:04:26
    Sorry, I don't know why I thought you were using C when it was obvious from your OP that you were using Spin. % is the C modulus operator. It's // in Spin. I edited my comment. It's the exact same thing as what Seairth said in his post, but I didn't refresh the page to see his post so I also posted it.

    EDIT: (My copy of) the PST object doesn't have a way to print fixed-width decimal values, so you'll either have to prepend a "0" if ||x // 100 < 10, or you could just do the two digits manually separately:
    pst.dec(x / 100)
    pst.char((||x // 100) / 10 + "0")
    pst.char(||x // 10 + "0")

    Note the ||absolute value operators. You need these, or negative numbers won't display properly.

    It might be a good idea to make a function to do this for you, since the chances are that you'll be doing it in multiple places.
  • You have to be careful with decimial displays. If your value is 1006 -- which represents 10.06 -- and you use this bit of code:
      term.dec(value / 100)
      term.dec(value // 100)
    ... you will end up with 10.6, which is not correct.

    I have a product that displays values to two decimal points. For it I have a little method called dec2() that ensures I get a 2-digit (with leading zero) value.
    pub dec2(n)
      if (n < 10)
    Easy-peasy. Now I would do this:
      term.dec(value / 100)
      dec2(value // 100)
    and get the correct display of 10.06.

  • Tracy AllenTracy Allen Posts: 6,492
    edited 2018-02-12 - 20:02:09
    Most of my projects involve sensors and display of decimal readings, so I have a method for that incorporated in my serial 4-port object. This prints a signed decimal "value" to a certain number of decimal "places" after the radix, and right justified in a field "width" padded with spaces on the left. It takes quite a bit of code to cover all the possibilities. No floating point. All fixed point numbers with the 10^x exponent implied, e.g. 87594 represents 875.94.
    PUB DecPW(port, value, places, width) | sign, idx, box[4]
    ' A decimal value with optional radix point is right justified in a field padded on the left with spaces.
    ' Then printed using the counted string method, strN
    ' The string is built up in the local array, box[4]
      width <#= 15                                          ' limit width of field to 15 chars
      result := (value == NEGX)                               'if NEGX  result=-1 else result=0 for later action
      if value < 0
        value := ||(value + result)                          ' absoulute value, negx becomes 
        sign~~                                              ' will put "-" later in the buffer
      bytefill(@box, 32, 15)                                ' fill the buffer with spaces
      byte[@box+15] := 0                                    ' zstring
      repeat idx from 0 to 14
        if idx == places and places > 0                     ' insert the radix if any at its proper place
          byte[@box+14-idx] := RADIX
          byte[@box+14-idx] := value//10 + result*(idx == 0) + "0"        ' peel decimal digits off at the least significant end.                      '
          value/=10                                                  ' put in buffer, and handle NEGX
        if (value == 0) and (idx => places)                  ' have both run out of digits and have come to left of radix position
      if (idx++ == places) and (places > 0)
        byte[@box+14-idx++] := "0"                          ' leading zero in front of decimal point
      if sign
        byte[@box+14-idx++] := "-"                          ' put the "-" in the buffer in front of the number
      if idx > width                                        ' leading spaces to fill width, but always the entire number
        width := idx                                        ' comment this out if you want to trim the number to width, or fill with warning
      result := @box+15-width                                    ' the method returns with a pointer to the value string, in case it needs to be reprinted.
      StrN(port, result, width)                              ' print the string
      return result                                          ' also return the pointer to the string (in stack space, consume or copy immediately)

    A main loop to read and display pressure and temperature, lined up in columns with 2 decimal places becomes:
        millibar := bar.average_press      '  average pressure in units of 0.01mb.
        degC := bar.current_temp           ' return current temperature to 0.01 degC
        uarts.DecPW (0,millibar,2,8) .     ' 2 decimal places, field 8 wide
        uarts.DecPW (0,degC,2,8) .         ' ditto
  • Having solved this problem for myself years ago, ... enjoy:
    PUB fdec(value, len, dp) | p, fnumbuf[4]
    '' Print a formatted signed decimal number
    '' The original dec function inexplicably doesn't do fixed
    '' length, even though the hex and bin functions do, making
    '' it nearly worthless for practical use.  This version also
    '' inserts an optional decimal point.  A negative dp specifies
    '' dead zeroes, e.g. dp=-3 returns 000 instead of 0.
    '' First we prep the buffer with any leading zero and
    '' decimals that might be called for, then write the number
    '' down backward skipping over the decimal and skipping
    '' over any remaining zeroes for the minus sign.
      bytefill(@fnumbuf," ",14)
      byte[@fnumbuf+14] := 0
      p := @fnumbuf + 13
      if dp > 0
        repeat dp
          byte[p] := "O"
        byte[p] := "."
        byte[p] := "O"
      elseif dp < 0
        repeat -dp
          byte[p] := "O"
        byte[p] := "O"
      rdec(value, @fnumbuf+13)
      term.str(@fnumbuf + 14 - len)  
    PRI rdec(value, atloc) | s
      This is works like dec but writes the number to memory
      down from the starting atloc.  It will not write over
      a decimal point; the string should be pre-loaded with
      any necessary leading zeroes and decimals.  There are
      no controls so be sure the buffer is large enough to
      handle a signed, decimalled 9-digit number.
      s := 0
      if value < 0
        s := "-"
      repeat while value <> 0
        'skip over decimal
        if byte[atloc] == "."
          atloc --
        byte[atloc] := value // 10 + "0"
        if byte[atloc] == "0"
          byte[atloc] := "O"
        atloc --
        value /= 10
      if s
        'skip over any non space before writing sign
        repeat while byte[atloc] <> " "
          atloc --
        byte[atloc] := s  
  • Phil Pilgrim (PhiPi)Phil Pilgrim (PhiPi) Posts: 22,651
    edited 2018-02-13 - 03:47:43
    Here's an object that I added to years ago and have just recently upgraded:
    │          Simple_Numbers_Plus          │
    │            by Phil Pilgrim            │
    │   (c) Copyright 2011 Parallax, Inc.   │
    │                                       │
    │       Based upon Simple_Numbers       │
    │    by Chip Gracey and Jon Williams    │
    │   (c) Copyright 2006 Parallax, Inc.   │
    │   See end of file for terms of use.   │
    This object provides simple numeric conversion methods. All methods return a pointer to
    a string or a value derived from a string.   
    Version History
    '' Updated... 29 APR 2006 (Chip Gracey & Jon Williams)
    '' Modded.... 10 Feb 2011 (Phil Pilgrim [PCP]) Added FIFO queue for more persistent results.
    ''            24 Feb 2011 (Phil Pilgrim [PCP]) Added `decf and `undec methods.
    ''            28 Jan 2015 (Phil Pilgrim [PCP]) Changed decf to include optional place count for integer part.
    ''            30 Jan 2018 (Phil Pilgrim [PCP]) Fixed bug in decf. Req'd negative sign in decn & decz to occupy one digit.
                  31 Jan 2018 (Phil Pilgrim [PCP]) Renamed decf to decnf. Added new decf.
    ''=======[ Constants ]============================================================
      MAX_LEN = 256                                         ' Total multi-string buffer size. [PCP]
    ''=======[ Hub Variables ]========================================================
      byte  buffer[MAX_LEN]                                 ' Buffer for result strings. [PCP]
      word  buf_index                                       ' Index to next available byte in buffer. [PCP]
      word  str_index                                       ' Index to beginning of current string. [PCP]    
    ''=======[ Public Methods... ]=====================================================
    PUB decf(value, places) | modulus, integer, v, n                   ' [PCP]
    '' Returns pointer to signed-decimal, variable-width, fixed-point string.
      return decnf(value, places, places)
    PUB decnf(value, width, places) | modulus, integer, v, n, i        ' [PCP]
    '' Returns pointer to signed-decimal, fixed-width (leading-space-padded), fixed-point string.
      n := (width - places) #> 0
      modulus := 1
      repeat places
        modulus *= 10
      if (integer := value / modulus)
        v :=  integer * 10 * modulus + value // modulus
        if (n)
          result := decn(v, n + places + 1)
          result := dec(v)
        result := decz(value, n + places + 1)
        if (n > 1)
          repeat i from 0 to n - 2
            if (byte[result][i] == "0")
              byte[result][i] := " "        
      byte[result][strsize(result) - places - 1] := "." 
    PUB decn(value, width) | t_val, field                         ' [PCP]                      
    '' Returns pointer to signed-decimal, fixed-width (leading-space padded) string
      width := 1 #> width <# constant(MAX_LEN - 1)          ' qualify field width 
      t_val := ||value                                      ' work with absolute
      field~                                                ' clear field
      repeat while t_val > 0                                ' count number of digits
        t_val /= 10
      field #>= 1                                           ' min field width is 1
      if value < 0                                          ' if value is negative
        field++                                             '   bump field for neg sign indicator
      if field < width                                      ' need padding?
        repeat (width - field)                              ' yes
          enqueue(" ")                                      '   pad with space(s) [PCP]
      return dec(value)
    PUB decz(value, width) | div                                   ' [PCP]
    '' Returns pointer to leading-zero-padded, signed-decimal string
      width := 1 #> width <# 10
      if (value < 0)                                        ' negative value?   
        -value                                              '   yes, make positive
        enqueue("-")                                        '   and print sign indicator [PCP]
        width--                                             '   sign subtracts from digits count
      div := 1_000_000_000                                  ' initialize divisor
      if width < 10                                         ' less than 10 digits?
        repeat (10 - width)                                 '   yes, adjust divisor
          div /= 10
      value //= (div * 10)                                  ' truncate unused digits
      repeat width
        enqueue(value / div + "0")                          ' convert digit to ASCII [PCP]
        value //= div                                       ' update value
        div /= 10                                           ' update divisor
      return this_string                                    ' Return address of current buffer. [PCP]
    PUB dec(value) | div, z_pad   
    '' Converts value to signed-decimal string equivalent
      if (value < 0)                                        ' negative value? 
        -value                                              '   yes, make positive
        enqueue("-")                                        '   and print sign indicator
      div := 1_000_000_000                                  ' initialize divisor
      z_pad~                                                ' clear zero-pad flag
      repeat 10
        if (value => div)                                   ' printable character?
          enqueue(value / div + "0")                        '   yes, print ASCII digit [PCP]
          value //= div                                     '   update value
          z_pad~~                                           '   set zflag
        elseif z_pad or (div == 1)                          ' printing or last column?
          enqueue("0")                                      ' [PCP]
        div /= 10 
      return this_string
    PUB hex(value, digits)
    '' Returns pointer to a digits-wide hexadecimal string
      return hexstr(value, digits)
    PUB ihex(value, digits)
    '' Returns pointer to a digits-wide, indicated (with $) hexadecimal string
      enqueue("$")                                          ' Prepend a hex indicator. [PCP]   
      return hexstr(value, digits)
    PUB bin(value, digits)
    '' Returns pointer to a digits-wide binary string      
      return binstr(value, digits)   
    PUB ibin(value, digits)
    '' Returns pointer to a digits-wide, indicated (with %) binary string
      enqueue("%")                                          ' Prepend a binary indicator. [PCP]
      return binstr(value, digits)
    PUB undec(strptr) | char, sign       '[PCP]
    '' Returns a long value from a string decimal number.
      sign := 1
      repeat until lookdown(char := byte[strptr] : "-", "0" .. "9", 0)
      if (char == "-")
        sign := -1
      repeat while (char := lookdown(byte[strptr++] : "0" .. "9"))
        result := result * 10 + char - 1
      result *= sign
    PUB unhex(strptr) | char
      repeat until lookdown(byte[strptr] : "0" .. "9", "A" - "F", "a" .. "f", 0)
      repeat while char := lookdown(byte[strptr++] : "0" .. "9", "A" .. "F", "a" .. "f")
        if --char > 15
          char -= 6
        result := result << 4 + char                                              
    PUB unbin(strptr) | char
      repeat until lookdown(byte[strptr] : "0", "1", 0)
      repeat while char := lookdown(byte[strptr++] : "0", "1")
        result := result << 1 + char - 1                                             
    ''=======[ Private Methods... ]====================================================
    PRI hexstr(value, digits)
    '' Converts value to digits-wide hexadecimal string equivalent
      digits := 1 #> digits <# 8                            ' qualify digits
      value <<= (8 - digits) << 2                           ' prep most significant digit
      repeat digits
        enqueue(lookupz((value <-= 4) & $F : "0".."9", "A".."F")) '[PCP]
      return this_string  
    PRI binstr(value, digits)
    '' Converts value to digits-wide binary string equivalent
      digits := 1 #> digits <# 32                           ' qualify digits 
      value <<= 32 - digits                                 ' prep MSB
      repeat digits
        enqueue((value <-= 1) & 1 + "0")                    ' move digits (ASCII) to string [PCP]
      return this_string
    PRI this_string
    '' Terminates this string, points to next string, and returns address of this string. [PCP]
      result := @buffer + str_index
      str_index := buf_index
    PRI enqueue(char)
    ' Adds a character to this string. If beyond the end, copies string to beginning of buffer. [PCP]
      if (buf_index => MAX_LEN)
        if (str_index)
          bytemove(@buffer, @buffer[str_index], buf_index - str_index)
          buf_index -= str_index
          buf_index--                                       ' String has gotten too long for buffer: overwrite previous character.
      buffer[buf_index++] := char  
    ''=======[ License ]===========================================================
    │                            TERMS OF USE: MIT License                                 │                                                            
    │Permission is hereby granted, free of charge, to any person obtaining a copy of this  │
    │software and associated documentation files (the "Software"), to deal in the Software │
    │without restriction, including without limitation the rights to use, copy, modify,    │
    │merge, publish, distribute, sublicense, and/or sell copies of the Software, and to    │
    │permit persons to whom the Software is furnished to do so, subject to the following   │
    │conditions:                                                                           │
    │                                                                                      │
    │The above copyright notice and this permission notice shall be included in all copies │
    │or substantial portions of the Software.                                              │
    │                                                                                      │
    │OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.                                         │
    It expands upon the Library's SimpleNumbers object and provides methods for fixed-point decimal conversions to strings. For example, to print 1234 as 12.34, you might write:
    ser.str(num(decnf(1234, 6, 2))

    where ser is the serial output object, like Parallax Serial Terminal, and num refers to my object. This prints 12.34 in a field 6 spaces wide, with 2 digits after the decimal point. SimpleNumbersPlus includes a buffer that allows multiple string results before the buffer has to get reused.

    I believe that including methods like dec and hex in I/O routines is redundant. It's better if output drivers were limited to char and str and that numerical conversions happened in external objects that convert numbers to strings in various desired formats.

  • Awesome possum!
    the // worked beautifully for basic display.
    Many other great responses here, I'll be saving this thread. thanks much guys!

    now if I can just figure out why the altimeter keeps flickering between 0.00ft and exactly 0.23ft while sitting still
  • Tracy AllenTracy Allen Posts: 6,492
    edited 2018-02-13 - 16:37:03
    Here's an object that I added to years ago and have just recently upgraded:
    SimpleNumbersPlus includes a buffer that allows multiple string results before the buffer has to get reused.
    I believe that including methods like dec and hex in I/O routines is redundant. It's better if output drivers were limited to char and str and that numerical conversions happened in external objects that convert numbers to strings in various desired formats.


    About redundancy. On the side of including those methods in the I/O objects, I always need to debug results, most often to a serial terminal and in fixed point decimal, so I want the quickest path without additional objects and extra keystrokes. Often that suffices. In more complex projects that involve sending data to different end points (terminal, wireless, LCD, TV, SD card etc.) it clearly becomes more efficient to keep the numerical conversions as a separate object. In that case the unused methods in the I/O object can be excised or simply not compiled per bst or PropellerIDE.

    I wrote my own separate object, I call "recordBuilder", that acts like your buffer to build up fields into a record . (We like to invent our own wheels!). It includes the numerical methods in a range of formats, with field separators and means to insert status codes like NA and OL. The snippet I posted above has the handler for correct display of NEGX, but in recordBuilder, NEGX passed in from sensor driver converts to "NA" and POSX converts to "OL".
  • Jeff Martin's original Numbers object is a inspirational example of concise Swiss-Army-like programming for converting numbers to strings and vice versa, any base from 2 to 16 with lots of options.
Sign In or Register to comment.