Stable Floating Point Calculations
Kye
Posts: 2,200
So... lets say I want to do:
atan2(y, x)
Where x and y are doubles. How do I make sure that this function works for even small values of y or x?
I'm asking this question because I'm trying to display the phase information from the output of an FFT and I get round off errors that are making the phase plot look bad. At the frequency of interest the phase is zero but then goes crazy like random noise every where else. I believe this is do to the division of very small numbers by each other in the atan2 function.
atan2(y, x)
Where x and y are doubles. How do I make sure that this function works for even small values of y or x?
I'm asking this question because I'm trying to display the phase information from the output of an FFT and I get round off errors that are making the phase plot look bad. At the frequency of interest the phase is zero but then goes crazy like random noise every where else. I believe this is do to the division of very small numbers by each other in the atan2 function.
Comments
In general the math functions in propgcc are pretty stable -- they come from the OpenBSD math library, which in turn is based on Sun's fdlibm, which is very widely used.
Eric
Apparently it is quite common to have this problem with making a spectrum analyzer.
It may depend on the compiler/library you're using. But in general handling floating point properly is hard (as you know!). I don't have any specific suggestions for atan2, I'm afraid, but you may be able to find some in "Numerical Recipes in C" or a similar numeric analysis book. Many university libraries will have some good resources like that.
Eric
Since I have no clue what atan2() is doing internally, this means I'm out of luck.
There has to be some way people fix this however. Otherwise the phase of a sin wave is 0 at the frequency of interest and random noise everywhere else. Really confusing.