|
Re: Glitch in floor() function?
|
Arthur Crump
|
Jun 26, 2008 12:46 PDT
|
posted by: Arthur Crump <arthur.crump at ntlw?rld.?om>
Leaving in just the relevant parts of Mike Wever's message:
Mike Wever wrote:
| |
procedure round(atom x, integer precision)
integer mult
mult = power(10, precision)
printf(1, "initial: %f\n", {x})
x *= mult
printf(1, "shifted right: %f\n", {x})
x += 0.5
printf(1, "plus rounding half: %f\n", {x})
x = floor(x)
printf(1, "floor: %f\n", {x})
x /= mult
printf(1, "shifted left: %f\n", {x})
end procedure
round(1.005,2)
produces these results:
...
initial: 1.005000
shifted right: 100.500000
plus rounding half: 101.000000
floor: 100.000000
shifted left: 1.000000
It seems that floor(101) gave a result of 100. Is this an error or am I
missing something?
|
Effectively the result of:
floor( 1.005*100 + .5 )
is required.
The problem arises because 1.005 is converted into binary.
This cannot be done exactly as the binary number has a recurring fraction.
The last twenty binary digits of 1.000000010100011111001110 recur.
To fit it into an atom the series must be cut off and rounded in some way.
If the first digit cut off is a 1, the number is rounded up.
If it is a 0 it is rounded down. This must be what happened in this case.
So the binary number is very slightly less than 1.005
Multiplying by 100 preserves this deficiency.
Therefore adding .5 to it gives a number which is a minute amount less than 101.
All the printf statements apply a further rounding,
which eliminates the minute deficiency.
However, the floor function operates on the exact value.
This is not specifically a Euphoria problem.
I would expect most languages using this floating point standard to be the same.
Arthur Crump
|
|
 |
|
