Problems with Floating point math
DiverBob
Posts: 1,116
I am testing the floating point object and have run into some difficulties. I have been trying various configurations but continue to not get the desired output
CON
· _CLKMODE····· = XTAL1 + PLL8X·······················
· _XINFREQ····· = 8_000_000
CON
· _CLKMODE····· = XTAL1 + PLL8X·······················
· _XINFREQ····· = 8_000_000
VAR
· word p1,p2,p3, cntr
OBJ
· OLED· : "uOLED-96-Prop_V4"
· DELAY : "Clock"
· Num : "Numbers"
· FLOATStr : "FloatString"
· FMath : "FloatMath"
PUB Main | p4
· DELAY.Init(8_000_000)
· OLED.InitOLED
· OLED.CLS
· FloatStr.SetPrecision(3)
· cntr := 0
'· p4 := 1.2
'· p4 := FMath.FFloat(Px1[noparse][[/noparse]cntr])
· p4 := FMath.Fdiv(FMath.FFloat(Px1[noparse][[/noparse]cntr]),100.0)
'· oled.PutText (0,0, 0, 255,255,0, Num.toStr(p4,Num#DEC))
'· oled.PutText (0,0, 0, 255,255,0, FloatStr.FloatToString(p4))
· oled.PutText (0,0, 0, 255,255,0, FloatStr.FloatToString(p4))
· delay.PauseSec(10)
DAT
Px1· byte· 40, 45, 52, 101
if I set p4 := 1.2 then FloatToString correctly displays the value
Then if I use the test statement·p4 := FMath.FFloat(Px1[noparse][[/noparse]cntr]) the output is something like 4.51e-41 where the desired output would be 40.0
The next test statement of·p4 := FMath.Fdiv(FMath.FFloat(Px1[noparse][[/noparse]cntr]),100.0) (where I eventually want to head up with) has another value similar to above. The desired output would be 0.4
It seems to be some issue with the integer pulled from DAT but I can't figure it out. I've gone over the FloatMath code several times and I'm just not seeing it. I even substituted in Float32, all I got was a blank display for that
I've done a search of the forum for floating point and looked for other examples but I'm missing something basic and could use another set of eyes on this
Thanks
Bob
· word p1,p2,p3, cntr
OBJ
· OLED· : "uOLED-96-Prop_V4"
· DELAY : "Clock"
· Num : "Numbers"
· FLOATStr : "FloatString"
· FMath : "FloatMath"
PUB Main | p4
· DELAY.Init(8_000_000)
· OLED.InitOLED
· OLED.CLS
· FloatStr.SetPrecision(3)
· cntr := 0
'· p4 := 1.2
'· p4 := FMath.FFloat(Px1[noparse][[/noparse]cntr])
· p4 := FMath.Fdiv(FMath.FFloat(Px1[noparse][[/noparse]cntr]),100.0)
'· oled.PutText (0,0, 0, 255,255,0, Num.toStr(p4,Num#DEC))
'· oled.PutText (0,0, 0, 255,255,0, FloatStr.FloatToString(p4))
· oled.PutText (0,0, 0, 255,255,0, FloatStr.FloatToString(p4))
· delay.PauseSec(10)
DAT
Px1· byte· 40, 45, 52, 101
if I set p4 := 1.2 then FloatToString correctly displays the value
Then if I use the test statement·p4 := FMath.FFloat(Px1[noparse][[/noparse]cntr]) the output is something like 4.51e-41 where the desired output would be 40.0
The next test statement of·p4 := FMath.Fdiv(FMath.FFloat(Px1[noparse][[/noparse]cntr]),100.0) (where I eventually want to head up with) has another value similar to above. The desired output would be 0.4
It seems to be some issue with the integer pulled from DAT but I can't figure it out. I've gone over the FloatMath code several times and I'm just not seeing it. I even substituted in Float32, all I got was a blank display for that
I've done a search of the forum for floating point and looked for other examples but I'm missing something basic and could use another set of eyes on this
Thanks
Bob
Comments
A related Floating Point question next; is there a object out there that allows the floating point result to be padded with 0? Example, .40 shows up as .4 and 1.00 shows as 1. I would like to retain the exponent at 2 char and the integer portion at 1 char so .4 would display as 0.40 and 1. display as 1.00.· I have been looking but haven't seen anything in the Float objects that seems to give this functionality. Before I try to re-create the wheel I thought I'd ask....
Now it's on to trying to average 3 floating point numbers
Bob