Problem with Floating Point Routine - Page 2 — Parallax Forums

Problem with Floating Point Routine

mynet43 · 2007-09-23 00:18:20

I've been programming a routine that converts from barometric pressure to altitude. It first converts an adc count to the pressure in kPa and then calculates the altitude in feet from the pressure. The formula used is: Altitude = (10^(log10(Pressure/101.325) / 5.2558797) - 1.0) / (-6.8755856e-6) The code seemed to be…

or Register to comment.