double data types, floating point math, and the Propeller
idbruce
Posts: 6,197
Hello Everyone
I have two files of source code that I would like to modify to work with the Propeller chip, however these files include double data types and floating point math. I am wondering if my goal is attainable, before doing a vast amount of research to find out that it cannot be done. I have included the source code of one file that I want to modify.
Bruce
I have two files of source code that I would like to modify to work with the Propeller chip, however these files include double data types and floating point math. I am wondering if my goal is attainable, before doing a vast amount of research to find out that it cannot be done. I have included the source code of one file that I want to modify.
Bruce
// Ascended International
//
// Copyright (c) 2010
//
// All Rights Reserved
// ------------------------------------------------------------------------------
// * DO NOT REMOVE THIS HEADER. DO NOT MODIFY THIS HEADER *
/**********************************************************************************
* You may use this class for non-commerical purposes. If you wish to use this *
* software for a commercial pupose please contact Ascended International at: *
* [EMAIL="mark@ascended.com.au"]mark@ascended.com.au[/EMAIL] *
* *
* When using this free class, no warranty express or implied is offered. *
* In no event shall Ascended International, any of it's employees or shareholds *
* be held liable for any direct, indirect, special, exemplary, incidental or *
* consequential damages however caused. *
* *
* If you modify this class, please add your name and the date below as a *
* contributor, without removing this notice or modifying it in any way, shape *
* or form. *
**********************************************************************************/
/*
Contributors:
* 16 Sep 2010, Mark Harris - Initial code, maths and development [ [EMAIL="markh@rris.com.au"]markh@rris.com.au[/EMAIL] ]
* 17 Sep 2010, Mark Harris - increased accuracy of 4th order poly by 6 decimal places
* 26 Sep 2010, Mark Harris - Rewrote poly functions to get accuracty within 1mm
*/
using System;
using Microsoft.SPOT;
using GHIElectronics.NETMF.Hardware;
using GHIElectronics.NETMF.System;
using Microsoft.SPOT.Hardware;
namespace Ascended.SPOT.Sensors.Distance
{
/// <summary>
/// Settings for running the sensor
/// </summary>
public enum SensorMode
{
/// <summary>
/// Define your own settings using the properties on the class
/// </summary>
Custom,
/// <summary>
/// Read the sensor value out as fast as possible. No filtering or averaging will be done.
/// </summary>
FastestRead,
/// <summary>
/// Read the sensor as accurately as possible. All filtering and averaging will be done - if required.
/// </summary>
MostAccurate,
/// <summary>
/// The the accurately, but also try to do it quickly. This is best for most uses.
/// </summary>
BestTradeoff
}
/// <summary>
/// Driver for Sharp 2D120X (Infrared Proximity Sensor Short Range - Sharp GP2D120XJ00F).
/// Sparkfun SKU: sku: SEN-08959
/// TinyCLR SKU: GP2D120
///
/// Class written by Mark Harris, [email]mark@ascended.com.au/markh@rris.com.au[/email]
/// </summary>
public class Sharp2D120X
{
protected AnalogIn sensor;
protected double[] movingValues;
int movingPos;
double[] data;
double[] values;
/// <summary>
/// Create a new instance of Sharp2D120X. You should sleep for *at least* 50ms after instantiating this class.
/// The sensor does not produce accurate results for at least this time.
/// </summary>
/// <param name="pin">Pin the sensor's vout is connected to.</param>
public Sharp2D120X(Cpu.Pin pin)
{
Initialise(pin);
SetDefaults();
MinimumDistanceCapability = 4.75f;
MaximumDistanceCapability = 26;
}
/// <summary>
/// Create a new instance of Sharp2D120X. You should sleep for *at least* 50ms after instantiating this class.
/// The sensor does not produce accurate results for at least this time.
/// </summary>
/// <param name="pin">Pin the sensor's vout is connected to.</param>
/// <param name="mode">Set the sensor read mode defaults</param>
public Sharp2D120X(Cpu.Pin pin, SensorMode mode)
{
// initialise the senor
Initialise(pin);
SensorReadMode = mode;
if (mode == SensorMode.Custom)
{
SetDefaults();
}
MinimumDistanceCapability = 4.75f;
MaximumDistanceCapability = 26;
}
internal void SetDefaults()
{
UseMovingAverage = false;
MovingAverageSize = 5;
UseAveraging = true;
AverageSampleCount = 10;
movingValues = new double[MovingAverageSize];
FillMovingAverageValues();
}
internal void Initialise(Cpu.Pin pin)
{
sensor = new AnalogIn((AnalogIn.Pin)pin);
sensor.SetLinearScale(0, 3300);
}
/// <summary>
/// Returns the distance in centimetres and will apply any filtering or averaging before returning the value.
/// </summary>
/// <returns>The distance the sensor is reading in CM.</returns>
public double GetDistanceInCM()
{
double distance = 0;
if (UseAveraging)
{
data = new double[AverageSampleCount];
double min = Double.MaxValue;
double max = Double.MinValue;
for (int i = 0; i < AverageSampleCount; i++)
{
double val = ReadSensorValue();
data[i] = val;
distance += val;
if (UseFiltering)
{
if (min > val)
min = val;
if (max < val)
max = val;
}
}
distance = distance / AverageSampleCount; // faster than /=
if (UseFiltering)
{
// do we need to do cleanup through deviation?
if (!(max == min || max - min < 1))
{
// trim down the data
double stdDev = StandardDeviation(data, distance);
// Standard Dev is too high for our liking!
if (stdDev > 0.5)
{
int candidates = 0;
values = new double[AverageSampleCount];
for (int i = 0; i < AverageSampleCount; i++)
{
double dat = data[i];
if (dat < distance + stdDev && dat > distance - stdDev)
values[candidates++] = dat;
}
distance = Average(values, candidates - 1);
}
}
}
}
else
{
distance = ReadSensorValue();
}
if (UseMovingAverage)
{
AddMovingAverage(distance);
return Average(movingValues);
}
return distance;
}
protected double ReadSensorValue()
{
int x = sensor.Read();
// please do not change this unless you *really* know what you are doing. This took many hours to get right.
if (x > 670)
{
// handles 5cm ~23cm very accurately
return -0.00000000000000814807570435057 * MathEx.Pow(x, 5) +
0.0000000000677479681464555 * MathEx.Pow(x, 4) -
0.000000223468684525594 * MathEx.Pow(x, 3) +
0.000369147142846821 * MathEx.Pow(x, 2) -
0.313961024650693 * x +
121.96816964022;
}
else
{
// actually accurate from 17cm ~ 36cm however the above function is more accurate
return 0.00000000000000814807570435057 * MathEx.Pow(x, 5) +
0.0000000000677479681464555 * MathEx.Pow(x, 4) -
0.000000223468684525594 * MathEx.Pow(x, 3) +
0.000369147142846821 * MathEx.Pow(x, 2) -
0.313961024650693 * x +
121.96816964022;
}
static double StandardDeviation(double[] data, double avg)
{
//double avg = 0;
double totalVariance = 0;
int max = data.Length;
if (max == 0)
return 0;
// get variance
for (int i = 0; i < max; i++)
{
double variance = data[i] - avg;
totalVariance = totalVariance + (variance * variance);
}
return MathEx.Sqrt(totalVariance / max);
}
static double Average(double[] data)
{
return Average(data, data.Length);
}
static double Average(double[] data, int count)
{
double avg = 0;
for (int i = 0; i < count; i++)
avg += data[i];
if (avg == 0 || count == 0)
return 0;
return avg / count;
}
void AddMovingAverage(double nextValue)
{
movingValues[movingPos++] = nextValue;
if (movingPos >= movingValues.Length)
movingPos = 0;
}
protected void FillMovingAverageValues()
{
for (int i = 0; i < movingAverageSize; i++)
{
AddMovingAverage(ReadSensorValue());
}
}
private SensorMode sensorReadMode;
public SensorMode SensorReadMode
{
get { return sensorReadMode; }
set
{
if (sensorReadMode == value)
return;
sensorReadMode = value;
switch (value)
{
case SensorMode.FastestRead:
UseAveraging = false;
UseFiltering = false;
UseMovingAverage = false;
break;
case SensorMode.MostAccurate:
UseAveraging = true;
UseFiltering = true;
UseMovingAverage = true;
AverageSampleCount = 10;
MovingAverageSize = 5;
break;
case SensorMode.BestTradeoff:
UseAveraging = true;
UseFiltering = false;
UseMovingAverage = false;
AverageSampleCount = 5;
break;
case SensorMode.Custom:
default:
break;
}
}
}
bool useMovingAverage;
/// <summary>
/// Controls whether the system uses moving averages to smooth out values.
/// This should be True if you have a high speed update, or are movement is slow.
/// This should be False if you are moving fast or have a slow update rate.
/// Moving averages introduce some lag into the system, therefore it's only useful with high update speeds or slow movements.
/// The moving average can take out spikes and other crazy phenomenon efficiently.
/// </summary>
public bool UseMovingAverage
{
get { return useMovingAverage; }
set
{
if (useMovingAverage == value)
return;
useMovingAverage = value;
if (useMovingAverage == true)
{
movingValues = new double[movingAverageSize];
movingPos = 0;
FillMovingAverageValues();
}
}
}
int movingAverageSize;
/// <summary>
/// Gets/Sets the size of the moving average set. If the set is too large, there will be considerable lag. If the set is too small, it will be inefficient.
/// </summary>
public int MovingAverageSize
{
get { return movingAverageSize; }
set
{
if (movingAverageSize == value)
return;
movingAverageSize = value;
movingValues = new double[movingAverageSize];
movingPos = 0;
FillMovingAverageValues();
}
}
private bool useFiltering;
/// <summary>
/// Gets/Sets whether standard deviation filtering should be used. This only works with UseAveraging turned on.
/// If values read have too wide of a range, anything beyond one standard deviation will be culled and ignored.
/// If you have high speed movement, this will get a good workout and may not be of any benefit.
/// If you have slow movement with a lot of 'noise' this will clean it up considerably.
/// </summary>
public bool UseFiltering
{
get { return useFiltering; }
set
{
if (value && !UseAveraging)
throw new NotSupportedException("You cannot use filtering without averaging!");
useFiltering = value;
}
}
/// <summary>
/// When using averaging, the sensor will take <c>AverageSampleCount</c> samples and then average them to remove some fluctuations.
/// This works very well at all times and should be left on, vary the sample count based on the speed of code you require, and the speed you move at.
/// </summary>
public bool UseAveraging { get; set; }
/// <summary>
/// Gets/sets the number of samples to take for averaging.
/// </summary>
public int AverageSampleCount { get; set; }
#region IDisposable Members
public void Dispose()
{
sensor.Dispose();
}
#endregion
/// <summary>
/// The maximum distance (in centimetres) that the senor can be used.
/// </summary>
public float MaximumDistanceCapability { get; protected set; }
/// <summary>
/// The minimum distance (in centimetres) that the sensor can be used.
/// </summary>
public float MinimumDistanceCapability { get; protected set; }
public double GetDistanceInInches()
{
return GetDistanceInCM() / 2.54;
}
public double GetDistanceInMM()
{
return GetDistanceInCM() * 10;
}
public double GetDistanceInFeet()
{
return GetDistanceInCM() * 0.032808399;
}
}
}
}
Comments
PropGCC supports 64 bit doubles out of the box. I'm not sure if there are any Spin objects that do 64 bit floating point, but there certainly are some 32 bit floating point objects. I've written a 64 bit co-processor (using an additional COG) for GCC called fpucog.c. It's quite a bit faster than PropGCC's built-in 64 bit math, and could probably be adapted for use with Spin with a bit of work.
Depending on how much accuracy you need you might be able to get away with 32 bit floating point. As I mentioned, there are some Spin objects for this (F32 is the fastest). Catalina C supports 32 bit floating point (its doubles are also 32 bit), and I think that PropForth may support 32 bit floats too, but I'm not certain of that.
Eric
Then perhaps it would be a good time to get my feet wet with PropGCC. I have looked into it a few times, but I really have not had the necessity or projects that required learning it. I am not sure, but I believe the native code that I want to translate was written in C#, and I believe the doubles are 64 bit.
Thanks for your input Eric.
Bruce
What does the code do and why does it need to be so accurate?
As Eric suggests, if 32-bit precision is good enough then F32 should be able to handle the math portion of the code.
There's also the option of using a 64-bit floating point coprocessor (if you don't want to use GCC).
The code is used with the Sharp IR distance sensors. Supposedly this code will provide measurement accuracies within a couple millimeters. The author has written several objects for several different Sharp IR distance sensors.
The reason for using doubles is to keep the translation down to a minimum, because the math definitely appears to be a little complicated for translation purpores.
Can you try the original code with float instead of double ?
Float gives you 23-24bits of precision, so is well below 1ppm, an IR sensor will be WAY above that precision floor.
It seems to me double is chosen simply 'because they could'; on a PC it is a zero cost choice..
Back in the day I spent some time working on software for a three dimensional, phased array, radar (http://www.radartutorial.eu/19.kartei/karte113.en.html) that could give you bearing, distance and altitude of aircraft 250 miles away. We had no floating point support, it was all done with fixed point arithmetic on a 16 bit machine.
My boss on that project used to say "If you think you need floating point to solve the problem then you do not understand the problem".
Anyway that IT sensor sounds like a dinky radar, so what's the problem?
Here's an NV article.
http://www.parallax.com/Portals/0/Downloads/docs/cols/nv/vol5/col/nv114.pdf
Another good related read.
http://www.acroname.com/robotics/info/articles/irlinear/irlinear.html
So when I look at something like the following:
int x = sensor.Read(); // please do not change this unless you *really* know what you are doing. This took many hours to get right. if (x > 670) { // handles 5cm ~23cm very accurately return -0.00000000000000814807570435057 * MathEx.Pow(x, 5) + 0.0000000000677479681464555 * MathEx.Pow(x, 4) - 0.000000223468684525594 * MathEx.Pow(x, 3) + 0.000369147142846821 * MathEx.Pow(x, 2) - 0.313961024650693 * x + 121.96816964022; } else { // actually accurate from 17cm ~ 36cm however the above function is more accurate return 0.00000000000000814807570435057 * MathEx.Pow(x, 5) + 0.0000000000677479681464555 * MathEx.Pow(x, 4) - 0.000000223468684525594 * MathEx.Pow(x, 3) + 0.000369147142846821 * MathEx.Pow(x, 2) - 0.313961024650693 * x + 121.96816964022; }It scares the bejeezers out of me.
My viewpoint is that since the author used 64 bit doubles, why should I be the guy to rock the boat
I have already looked at those documents. I am looking for something that is HIGHLY accurate. I could simply setup the sensor, take some test readings, and create a few tables. However, there are a several people across the internet (robot enthuiasts) that attest to the accuracy of this software.
That is simply enough to code both ways and call both on a PC, and simply plot the errors you get, for the same input range, of float vs double.
I would expect maybe ~20um of difference ?
I can't help thinking that doing that in 64 bit floating point is going to take a lot of code space in the Prop and the same task could be done with a look up table and some linear interpolation, perhaps in less space.
I was just doing that as you speak. Attached is my effort in C which has float and double versions of that polynomial routine and exercises them for inputs from 0 to 1000. (What range should we expect as raw sensor input?)
The differences between float and double are down at the +-0.000002 level. like so:
Double is probably used because its the default for the architecture, not because the accuracy is required.
On high-end processors double is often faster than float because everything is at least 64 bit internal, and converting double->float is non-trivial (test for over/underflow, which may have to generate an exception, which requires flushing the instruction pipeline).
I think Dave's nailed it here. If 64 bits is fast enough, why sweat about reducing the precision? And conversely, switching from 64 to 32 is pretty trivial in PropGCC, since it's just a command line (or SimpleIDE) option, so it can always be done later.
Efficiency is important, but so is programmer effort required :-).
Eric
Propforth has modules for double and float. It is optimized and can probably be optimized further, but since its software it will only ever be "not fast". This applies to PASM and C equally. There is also support for the external math co-processor, in forth and other languages, but talking to the external chip is slow and ends up being almost the same as just doing it in software, due to the overhead for transfer etc.
So the question become how much processing and how often? IF YOU REALLY WANT TO BE CRUNCHING ON NUMBERS, the smart money says use something like a linux workstation , for example Raspberry Pi or better, running a language design for crunching for example GO LANG or better, and use a high speed comm protocol, CSP channels or better. Then collect the data on the prop as fast as you can, dump it to the Linux box as fast as you can, crunch it on the linux at your leisure, and send but your adjustments as fast as you need. Propforth v5.3 and the accompanying GO Lang modules are (going to be) pre configured for this out of the box, we don't have the box yet. Considering the cost of the prop and a Pi, this may be a nice bang for the buck option.
Smarter folks probably already have this set up in propgcc etc, but forth is likely going to be the fun option.
Side note: options like SPIN+Python might do the same thing, but would be a bit slower, which is ok if it does what you want.
@Heater
WOW... Thanks for the comparison. Definitely an eye opener.
Actually, according to Mark Harris's documentation, I will be pushing the Sharp GP2Y0A21YK0F beyond it's accurate measurement capabilities. To pick up the slack for this sensor, I am contemplating also using the Sharp GP2D120XJ00F. Basically, two sensors side by side, or an over/under configuration. However, each of these will be using different polynomial routines. As for the specific values I should expect to see, I have no idea except for the comparisons that he uses.
The Sharp GP2Y0A21YK0F uses the following comparisons:
- if (x > 970) // less than 28cm
- else if (x > 680) // less than ~44cm
- else
And the Sharp GP2D120XJ00F uses the following comparisons- if (x > 670)
- else
@Mark_TMakes perfect sense to me.
Well, I did not like your whole concept, but I definitely found a golden nugget of useful information.
Now that is a VERY useful suggestion. Considering that the readings will eventually be turned into useful data, and transferred to the PC, I could simply switch it around and let the PC do the work. By doing so, I could port this software to Visual C++ very, very quickly, basically a no brainer. However, that would eliminate my current need to get my feet wet with PropGCC
Thanks Prof, I really like that idea!!!!!!!!!!
John Abshier
I usually average at least 3 readings. With an 8-bit A/D you can get very close to 0.5cm accuracy with a 8-80cm Sharp IR sensor.
You did say in post #11 "constant to linearize the IR voltage output." and then talk about +/- 0.5cm.
Given the non-linearity of the thing I am prepared to belive that a polynomial with suitable coefficents can give you accuracy down to milimeters rather than using a single constant for a straight line fit.
So there is nothing wrong with the idea. It's just that I'm sure the same can be achieved with single precision floats, or even lesser precision fixed point arithmetic. A look up table and linear interpolation would be fine.
Ultimately the accuracy is determined by physical reality, the ADC resolution, the noise and so on. No point in going overboard on the arithmetic precision.
As per the copyright notice, I have sent him an email to listed address, and informed him that I would like to use his code in a commercial application.
However, I still think it might be beneficial to translate this code for use with the Propeller chip in non-commercial applications.
Bruce
If you want to use a fifth order polynomial or a look up table and interpolation just calibrate a sensor. Record output vs distance, find a ploynomial curve fitting program, there are many online like this one http://www.efunda.com/webM/numerical/curvefitpoly.cfm
and implement the resulting equation or put the equations values into a precomputed table.
If you are pushing any sensor, especially in a constrained-condition case like here, you are probably best calibrating each sensor.
Next step would be to add a Temperature compensation as well
ie setup an Auto-record system, such that you can easily capture and then Linear-fit, or compress however you like, the value.
Given the low speed, and small size here, you could just do a simple EEPROM table.
Just had a quick look at a data sheet for thhe device. They make no mention of expected accuracy. They do indicate that all sorts of things can detract from it's accuracy, like:
a) Ambient lighting.
b) Differnet colours of target.
c) Patterns of contrasting colour on the target.
d) Dirt and dust.
No doubt temperature may have an effect.
For sure to get maximum accuracy you are goint to need to operate the thing in carefully controlled conditions. And calibrate individual sensors. Question might then be what are you goint to calibrate against and what accuracy will that have.
Given all of this it seems having that software work down to dozen decimal places is just silly.
Do read this paper to see what you are up against.
http://www.gecad.isep.ipp.pt/iscies09/Papers/19November/iscies09_sharp_model.pdf