Lazy way to convert floating point to integer

Mickster · 2025-07-31 10:28

I ported floating point code over to FlexBasic and the execution was more than acceptable but knowing that integer equivalents execute ~five times faster on the P2, it just niggled me.
So I just dumped the whole thing into Gemini and asked it to convert to integers and it was perfect. Simple stuff is a no-brainer so I just do it but this was a bit complicated (for me) and I knew that I wouldn't get it right first time but yep, ~five times faster

ManAtWork · 2025-07-31 10:58

Could you please post a small snippet of the original and the converted code, please? So it not only simply replaced the types of the variables which would result in terrible rounding errors I guess, but did also apply proper scaling and took care to avoid all possible overflows?

Mickster · 2025-07-31 12:11

I was surprised that it considered the possible overflow issue (this is part of my s-curve routine):

You're looking to convert a floating-point BASIC calculation into an integer-only version. The key is to manage the scaling factors to avoid losing precision while working with integers.

Let's break down your code and convert it.

Original Code Analysis:

ramp_segments = 0.002

reso = 500

samptime goes from 0 to 500.

frac = samptime * ramp_segments This means frac goes from 0 * 0.002 = 0 to 500 * 0.002 = 1.

f = (3 - 2 * frac) * frac * frac This is a cubic interpolation function, where f goes from 0 to 1 as frac goes from 0 to 1.

intspeed = f * 200000

Integer-Only Conversion Strategy:

The core idea is to multiply all floating-point numbers by a sufficiently large power of 10 (or another base) to turn them into integers, perform the calculations, and then potentially divide back down at the very end if needed for the final output.

Let's use a scaling factor that makes 0.002 an integer. Multiplying by 1000 seems appropriate.

Scaling ramp_segments:
ramp_segments = 0.002
Let's define a new integer variable, say INT_RAMP_SEGMENTS, and a scaling factor RAMP_SCALE = 1000.
INT_RAMP_SEGMENTS = 2 (which is 0.002 * 1000)

Scaling frac:
frac = samptime * ramp_segments
In integer form: INT_FRAC = samptime * INT_RAMP_SEGMENTS
However, INT_FRAC will be samptime * 2, which is effectively scaled by RAMP_SCALE. So INT_FRAC represents frac * RAMP_SCALE.

Scaling f:
f = (3 - 2 * frac) * frac * frac
Let INT_FRAC represent frac * RAMP_SCALE.
Then frac = INT_FRAC / RAMP_SCALE.

Substitute into the equation for f:
f = (3 - 2 * (INT_FRAC / RAMP_SCALE)) * (INT_FRAC / RAMP_SCALE) * (INT_FRAC / RAMP_SCALE)
f = ( (3 * RAMP_SCALE - 2 * INT_FRAC) / RAMP_SCALE ) * (INT_FRAC^2 / RAMP_SCALE^2)
f = ( (3 * RAMP_SCALE - 2 * INT_FRAC) * INT_FRAC^2 ) / RAMP_SCALE^3

To get an integer version of f, let INT_F be f * RAMP_SCALE^3.
INT_F = (3 * RAMP_SCALE - 2 * INT_FRAC) * INT_FRAC^2

Here, RAMP_SCALE^3 = 1000^3 = 1,000,000,000. This means INT_F will be a very large number, potentially exceeding standard integer limits if samptime is large.

Let's reconsider the scaling of f. The frac term ranges from 0 to 1. The f term also ranges from 0 to 1.
We need to choose a common scaling factor for frac and f. Let's use a COMMON_SCALE (e.g., 1000 or 10000) that allows for sufficient precision.

Let COMMON_SCALE = 10000 (this allows for more precision than 1000, giving us 4 decimal places).

ramp_segments is 0.002. To make this an integer, we need to multiply by COMMON_SCALE.
INT_RAMP_SEGMENTS = 0.002 * COMMON_SCALE = 20

Now, frac = samptime * ramp_segments.
We want INT_FRAC to represent frac * COMMON_SCALE.
So, INT_FRAC = samptime * INT_RAMP_SEGMENTS / COMMON_SCALE.
No, this isn't right. Let's restart the frac scaling.

Revised Integer-Only Conversion Strategy:

Let SCALE_FACTOR = 100000 (This is a good balance for the values you have, ensuring enough precision and avoiding overflow for intermediate calculations assuming a typical 32-bit integer, which can go up to ~2 * 10^9).

Define Scaled Constants:
INT_RAMP_SEGMENTS = 200 (which is 0.002 * SCALE_FACTOR / 100) - No, this is getting complicated.

Let's simplify. We are working with frac which goes from 0 to 1. Let's define INT_FRAC such that it goes from 0 to SCALE_FACTOR.

ramp_segments = 0.002.
We can rewrite frac = samptime * 0.002 as frac = samptime * 2 / 1000.

Let INT_FRAC be frac multiplied by some FRAC_SCALE. Let FRAC_SCALE = 100000.
INT_FRAC = samptime * 2 * FRAC_SCALE / 1000
INT_FRAC = samptime * 2 * 100
INT_FRAC = samptime * 200

Now, f = (3 - 2frac)frac*frac
Substitute frac = INT_FRAC / FRAC_SCALE:
f = (3 - 2 * (INT_FRAC / FRAC_SCALE)) * (INT_FRAC / FRAC_SCALE) * (INT_FRAC / FRAC_SCALE)
f = ( (3 * FRAC_SCALE - 2 * INT_FRAC) / FRAC_SCALE ) * (INT_FRAC^2 / FRAC_SCALE^2)
f = ( (3 * FRAC_SCALE - 2 * INT_FRAC) * INT_FRAC^2 ) / (FRAC_SCALE^3)

To get an integer version of f, let INT_F be f * FRAC_SCALE^3. This is problematic due to the large scale factor.

Better Approach: Incremental Scaling

Let's try to maintain a consistent scaling level through the calculation.
Let's aim for intspeed to be directly calculated as an integer.

The maximum value of intspeed is 1 * 200000 = 200000. This easily fits into a standard integer.

The intermediate frac and f values are where precision can be lost.

Let's define a MASTER_SCALE = 100000 (or 10^5). This will represent the "full" value when frac or f is 1.

ramp_segments = 0.002
We need to represent 0.002 as an integer scaled by MASTER_SCALE.
INT_RAMP_SEGMENTS_SCALED = 0.002 * MASTER_SCALE = 200

frac = samptime * ramp_segments
We want INT_FRAC to represent frac * MASTER_SCALE.
INT_FRAC = samptime * INT_RAMP_SEGMENTS_SCALED / MASTER_SCALE
This means: INT_FRAC = (samptime * 200) / MASTER_SCALE.
No, this is wrong. samptime * 0.002 directly.
INT_FRAC = samptime * (0.002 * MASTER_SCALE) / MASTER_SCALE
The INT_FRAC should be the integer representation of frac, so if frac is 0.5, INT_FRAC is 0.5 * MASTER_SCALE = 50000.

Let's rethink frac.
frac = samptime * 0.002
If samptime = 500, frac = 1.
We want INT_FRAC to range from 0 to MASTER_SCALE (100000).
So, INT_FRAC = samptime * (0.002 * MASTER_SCALE)
INT_FRAC = samptime * (200)

Now, f=(3-2frac)frac*frac
We have INT_FRAC which represents frac * MASTER_SCALE.
So, frac = INT_FRAC / MASTER_SCALE.

Let INT_F represent f * MASTER_SCALE.
INT_F = (3 - 2 * (INT_FRAC / MASTER_SCALE)) * (INT_FRAC / MASTER_SCALE) * (INT_FRAC / MASTER_SCALE) * MASTER_SCALE
This becomes:
INT_F = ( (3 * MASTER_SCALE - 2 * INT_FRAC) / MASTER_SCALE ) * (INT_FRAC^2 / MASTER_SCALE^2) * MASTER_SCALE
INT_F = ( (3 * MASTER_SCALE - 2 * INT_FRAC) * INT_FRAC^2 ) / (MASTER_SCALE^2)

This is the core of the integer math. Be careful about intermediate overflows for INT_FRAC^2.
INT_FRAC goes up to 500 * 200 = 100000.
INT_FRAC^2 goes up to 100000^2 = 10,000,000,000 (10 billion). This will overflow a 32-bit integer.
So MASTER_SCALE = 100000 is too large if samptime is 500.

Let's choose MASTER_SCALE based on avoiding overflow in intermediate steps.
The largest intermediate calculation is INT_FRAC^2.
Max samptime is 500. Max frac is 1.
If we want INT_FRAC to be, say, 1000, then INT_FRAC^2 is 1,000,000, which fits in a 32-bit integer.
So, let MASTER_SCALE = 1000.

INT_RAMP_SEGMENTS_SCALED = 0.002 * MASTER_SCALE = 2

INT_FRAC (represents frac * MASTER_SCALE):
INT_FRAC = samptime * INT_RAMP_SEGMENTS_SCALED
INT_FRAC = samptime * 2
Max INT_FRAC = 500 * 2 = 1000. This is good.

INT_F (represents f * MASTER_SCALE):
Recall INT_F = ( (3 * MASTER_SCALE - 2 * INT_FRAC) * INT_FRAC^2 ) / (MASTER_SCALE^2)
INT_F = ( (3 * 1000 - 2 * INT_FRAC) * INT_FRAC^2 ) / (1000^2)
INT_F = ( (3000 - 2 * INT_FRAC) * INT_FRAC^2 ) / 1000000

Max (3000 - 2 * INT_FRAC) is 3000 (when INT_FRAC is 0).
Max INT_FRAC^2 is 1000^2 = 1,000,000.
Max numerator is 3000 * 1,000,000 = 3,000,000,000 (3 billion). This still overflows a 32-bit signed integer (max ~2.1 billion).

We need to break down the calculation for f further or use 64-bit integers if available.

Assuming 32-bit integers are the limit and we want to preserve precision, we'll need to be clever with division.

Let INT_FRAC_CUBED = INT_FRAC * INT_FRAC * INT_FRAC.
INT_FRAC_CUBED could be 1000^3 = 1,000,000,000 (1 billion), which fits.

Let's rewrite f = (3-2frac)fracfrac as:
f = 3frac^2 - 2*frac^3

Now, substitute frac = INT_FRAC / MASTER_SCALE:
f = 3 * (INT_FRAC / MASTER_SCALE)^2 - 2 * (INT_FRAC / MASTER_SCALE)^3
f = 3 * (INT_FRAC^2 / MASTER_SCALE^2) - 2 * (INT_FRAC^3 / MASTER_SCALE^3)
To get INT_F (which is f * MASTER_SCALE):
INT_F = (3 * INT_FRAC^2 / MASTER_SCALE) - (2 * INT_FRAC^3 / MASTER_SCALE^2)

This still has potential for overflow, specifically in 3 * INT_FRAC^2.
If INT_FRAC is 1000, INT_FRAC^2 is 1,000,000. 3 * 1,000,000 = 3,000,000. This is fine.

Let's trace INT_F with MASTER_SCALE = 1000:

INT_F_TERM1 = (3 * INT_FRAC^2) / MASTER_SCALE
INT_F_TERM2 = (2 * INT_FRAC^3) / (MASTER_SCALE^2)

INT_F = INT_F_TERM1 - INT_F_TERM2

Let's check the maximum values for these terms:
Max INT_FRAC = 1000.
Max INT_FRAC^2 = 1,000,000.
Max INT_FRAC^3 = 1,000,000,000.

INT_F_TERM1: (3 * 1,000,000) / 1000 = 3,000,000 / 1000 = 3000. Fits.
INT_F_TERM2: (2 * 1,000,000,000) / (1000^2) = 2,000,000,000 / 1,000,000 = 2000. Fits.

The range of INT_F would be roughly from 0 to MASTER_SCALE (0 to 1000).

Scaling intspeed:
intspeed = f * 200000
We have INT_F which represents f * MASTER_SCALE.
So f = INT_F / MASTER_SCALE.

intspeed = (INT_F / MASTER_SCALE) * 200000
intspeed = (INT_F * 200000) / MASTER_SCALE

Max INT_F = 1000.
Max intspeed = (1000 * 200000) / 1000 = 200000. This is correct and fits.

Edit: I didn't inform Gemini that we do indeed support 64bit ints and so it didn't utilize them.

evanh · 2025-07-31 13:08

Just a plain translation. Doesn't need to know anything about the hardware. And barely any Basic really.

Get it to use the muldiv64() function to retain precision while not doing intermediate divides.

ManAtWork · 2025-07-31 13:16

Impressive. I was a bit sceptical because some people in the simulator forum I'm a member used AI to let them program their Arduinos and the AI made major mistakes, for example ignoring the rotational inertia of servo motors.

So it's getting better all the time and AI becomes a great way to save time. But you still need to know what you are doing and need to be able to verify that the results are somewhat plausible and sound. So good to know that humanity is not going to be totally expendable any time soon.

I also use floating point math whenever possible to avoid overflow and scaling issues. When a loop runs at 1kHz this is perfectly manageable with the P2. But in the PFC and SMPS for my PEMF project the loops run at 50 and 100kHz so efficient ASM code is mandatory.

evanh · 2025-07-31 13:43

@ManAtWork said:
So it's getting better all the time ...

Not at all. And not impressive either. It's just a translation of the codemaths Mickster supplied. LLMs have been good at translations all along.

evanh · 2025-07-31 13:57

This just appeared on Slashdot - https://venturebeat.com/ai/stack-overflow-data-reveals-the-hidden-productivity-tax-of-almost-right-ai-code/

It goes to the trouble of offering recommendations at the end. Although none of the recommendations are to address the elephant in the room, which is that LLMs can't think for themselves. Which is a problem because thinking for themselves is how they're being sold.

Mickster · 2025-07-31 15:09

@evanh said:
It's just a translation of the codemaths Mickster supplied. LLMs have been good at translations all along.

Agreed and that's all I want.
I have challenged "AI" with some really inefficient code and asked for the best possible optimization for speed....useless. But Gemini does come in handy.

Lazy way to convert floating point to integer

Comments