Rounding Errors With Math

Gerry Shand

Hi Folks:

Trying to solve a relatively simple problem but getting no where.

What I am trying to do is the following:

a. Read in a 12 bit number (0-4095) from and A/D chip (no problems - piece of cake when I read the manuals)
b. Compute an offset (no problems - done)
c. Do a calculation where 0 - 4095 is a linear equivalent between the offset and 4095-offset. So as an example using an offset of 500 counts, I should get an output of 0-4095 between an input of 500 (offset) and 4095. (My algorithms work but·have really high·rounding errors)
d. Take the result of the calculation and dump it into a D/A converter (got this part working too the first time without rewriting this portion of the code).

I set up a y=mx+b equation but once again I get rounding errors so there must be another elegant way to solve this problem. All incoming and outgoing variables are words so I have tried multiplying by 16 to expand the range and reduce the roundoff error due to the integer based math in the Stamp but nothing works well. I am going to take a coffee break and try another approach such as inputting the offset first. But if anyone has any other additional thoughts or ideas, please feel free to let me know.

Thanks and regards,

Gerry Shand

Tracy Allen

Is the offset fixed at compile time?

If so, precompute the mFactor:
mFactor = 4096 / (4096-offset) * 256 on a calculator
Then on the Stamp, the formula is:
DACvalue = mFactor */ (ADCreading MIN offset - offset)
The MIN operator is in there just to assure that the term in () does not become negative.

For example, with offset=500, the mFactor is 4096/3596 * 256 = 292.
ADCreading = ADCreading MIN offset - offset
DACvalue = 292 */ ADCreading
A more precise formula would use the ** operator instead of */
ADCreading = ADCreading MIN offset - offset
DACvalue = 9112 ** ADCreading + ADCreading

It is a little different if the offset needs to be determined at run time, for example, by means of a calibration procedure.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Tracy Allen
www.emesystems.com

Gerry Shand

Hi Tracy:

Thank you for the input. I have done offset precomputation in the past but this time it is a little different in that the offset is measured and stored as a variable everytime the circuit is started up (the system has a naturally occurring phenomenon where the zero reference point can and will shift just about every time).

Any other insights or suggestions?

Thanks and regards,

Gerry Shand

Tracy Allen

Hi Gary,

Here is a snippet. The core is the renormalization routine, which calculates the factor that you have to use with the ** operator. When the program starts up, it finds the offset, calculates the ** multipliers (mFactor and K), and later on uses them in the main loop.

gosub getOffset    '  offset value returned
  ' now calculate slope factor
  N = 4096                  ' N and D are temporary word variables
  D = 4096-offset    
  K = N/D     ' a small integer
  GOSUB renormalize   ' returns mFactor
  ' end of calibration procedure
   ' now keep mFactor and K for use later.
  
  ' this is later; in the main loop:
      gosub readProcess
  ADCreading = ADCreading MIN offset - offset
  DACvalue = mFactor ** ADCreading + (K*ADCreading)
  gosub sendDACvalue
  

' this is the routine that transforms N//D/D
' into new fraction mFactor/65536 for use with **
renormalize:
  FOR idx=15 TO 0               ' 16 bits 
  N=N//D<<1                   ' remainder*2 
   mFactor.bit0(J)=N/D               ' next bit 
  NEXT
  RETURN

The factor K will probably be a small integer. For example, if offset=500, then K = 4096/(4096-500) = 1 (integer division) In fixed point division, it is 1.139, and the mFactor takes care of the 0.139 part.

Another way to do it would be to put the renormalization step in the main loop, instead of in the calibration step. In that case, I would use the renormalization to calculate (4096-ADCreading)/(4096 - offset), which will be a number less than unity, so K=0. But that would not be as fast in the main loop as precomputing the multipliers.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Tracy Allen
www.emesystems.com

Gerry Shand

Hi Tracy:

I had some success implementing the code and below is what I have come up with. Success is elusive because it is linear but the end result is not 4096. This means I am having trouble with the slope calculation based on the //. I have also tried using */ in lieu of ** and I still get a linear result but still not 4096. Because the equation is y=mx+b, the actual equation becomes:

AF = (4096/4096-AZERO)*AD-AZERO*m

= m(AD-AZERO)

I am having trouble with EEPROM resources on this project so I am attempting to do all this with one line of code (I know this is not a good idea but I have no choice in the matter and I have gone through 5 iterations already to minimize the code).

Any further insights from the group? So close yet so far. Cheers for now and thanks in advance for any input.

Gerry Shand

' {$STAMP BS2p}
' {$PBASIC 2.5}

'DEFINE VARIABLES
AD VAR Word 'INPUT
AZERO VAR Word 'AUTOZERO CALIBRATION VARIABLE
AF VAR Word 'CALCULATED OUTPUT
n1 VAR Byte 'COUNTER BIT
config VAR Byte 'CONFIGURATION FOR A/D

'DEFINE PINS
MAINIO 'P0-P15
DIRL=%00000000 'ASSIGN DIGITAL I/O TO EACH MAIN PIN
DIRC=%0000
DIR15=%1
DIR14=%1
DIR12=%0

dio2 PIN 13 'A/D BIDIRECTIONAL DATA PIN
clk2 PIN 14 'A/D CLOCK PIN
cs2 PIN 15 'A/D CHIP SELECT PIN ("0" LEVEL)

AUXIO 'X0-X15
DIRS=%0000000000000000 'ASSIGN DIGITAL I/O TO EACH PIN

start PIN 7 'START PUSHBUTTON (N.O.)

PAUSE 1000 'Wait for things to settle down

Main:
AUXIO
IF start=1 THEN azeroroutine
GOTO MAIN

azeroroutine:
FOR n1=1 TO 10
PAUSE 1000
GOSUB Analog_digital:
DEBUG "AD=", DEC AD, CR
AZERO=AD+AZERO
DEBUG "AZERO=", DEC AZERO, CR
NEXT
AZERO=AZERO/10
GOTO mathroutine

mathroutine:
GOSUB Analog_digital
IF AD<AZERO THEN mathroutine
AF=((AD-AZERO)**(4096//4096-AZERO)+(AD-AZERO))
DEBUG "AD=", DEC AD, CR
DEBUG "AF=", DEC AF, CR
PAUSE 1000
GOTO mathroutine

Analog_digital: 'INPUT FROM A/D CONVERTER
MAINIO
config=%1011
cs2=0
SHIFTOUT dio2, clk2, LSBFIRST, [noparse][[/noparse]config\4]
SHIFTIN dio2, clk2, MSBPOST, [noparse][[/noparse]AD\12]
cs2=1
AUXIO
RETURN

Tracy Allen

Hi Gary,

I don't think the formula
{AF=((AD-AZERO)**(4096//4096-AZERO)+(AD-AZERO))}
will work in general.

Minimum eeprom? Try this in place of the above formula:

af = 4096 - azero
af = (ad - azero) */ ((65535/af * 16) + (65535//af * 16 / af))

The term on the right hand side after the */ is computing the multiplier, which only depends on the offset, azero. You could do that in the initial "azeroroutine" and not have to recompute it each time through the main loop.

The */ operator has an overall precision of 8 bits, which is not as good as the 12 bits resolution of your ADC. For example, if azero is 500, then the factor on the right in parentheses becomes 291. At full scale input, when ad=4096, the output is af=4087. Maybe it is good enough?

If you want more precision, I think you will have to do the binary division loop. Done in the main loop as follows:

af = ad - azero
  for idx=15 to 0
    af = af//(4096 - azero)
    factor.bit0(idx) = af / (4096 - azero)
  next
  af = 4096 ** factor

This routine maintains 12 bits of precision, but it is slower and eats up a little more eeprom. It is similar to the algorithm I laid out a couple of messages back, but this one here runs in the main loop instead of at the initialization.

I hope something will work for you!

-- Tracy

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Tracy Allen
www.emesystems.com

RussR

Gerry,

<< Success is elusive because it is linear but the end result is not 4096. >>

What result are you getting? Keep in mind that 4096 is the RANGE of values, which go from 0 to 4095. You will probably never calculate 4096. But how close are you getting?

Russ

Gerry Shand

Hi Folks:

I love it when a plan comes together (apologies to Col. Hannibal Smith in advance). With a little coding, some input and persistence I have come up with a solution.

This is the final code snippet that will calculate a 0 - 4095 output as a linear equivalent between an offset and 4095. So as an example using an offset of 500 counts, I should get an output of 0-4095 between an input of 500 (offset) and 4095. This has a lot of potential use in instrumentation interface, display and signal conditioning centered around a Stamp. The code is very minimal and the subroutine could be incorporated into the main body of the program.

AF VAR Word
AZERO VAR Word
AD VAR Word
Factor VAR Word
n VAR Byte

Main: 'Part of main program
AF=AD-AZERO
GOSUB Math 'Do math calculation


Math:
FOR n=15 TO 0 'Normalization routine
AF=AF//(4096-AZERO)<<1
Factor.BIT0(n)=AF/(4096-AZERO)
NEXT
AF=4096**Factor
RETURN

Where:

AD = the active value from 0-4095 counts returned from an actual end device via a serial link A/D device or through an A/D converter
AZERO = computed/pre-determined offset.
n = manipulated bits in the normalization subroutine
Factor = normalization factor used in the normalization subroutine
AF = output from this block that is 0-4095 counts between AZERO and 4095

If anyone is interested, I could write up a complete article with schematics, text and full code and post it on this forum (if that is permitted).

This post and the offer to write an article is my way of saying thanks to the group for their assistance in solving this problem by providing this medium to users like myself. In particular, a big "Thank You" goes out to Tracy Allen for his insight and input.

Have a good one.

Gerry Shand

Chris Savage

Gerry Shand said...(trimmed)
Hi Folks:
I love it when a plan comes together (apologies to Col. Hannibal Smith in advance). With a little coding, some input and persistence I have come up with a solution.

Gerry, I hate to ruin mood (And the memory of a great man), but the good Col. has left the building some time ago...Having been a child of the 70's & 80's, I do remember the A-Team...

And I'm sure everyone would love to see your complete article on your work!

Oh, and BTW, Elvis has left the building as well!·

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Chris Savage

Knight Designs
324 West Main Street
P.O. Box 97
Montour Falls, NY 14865
(607) 535-6777

Business Page:·· http://www.knightdesigns.com
Personal Page:··· http://www.lightlink.com/dream/chris
Designs Page:··· http://www.lightlink.com/dream/designs
·

Tracy Allen

Hi Gerry,

That little normalization routine would certainly go into my personal "most useful Stamp math tricks" book.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Tracy Allen
www.emesystems.com