|
Re: Glitch in floor() function?
|
Mike Wever
|
Jun 27, 2008 10:50 PDT
|
posted by: Mike Wever <mewever at gmai??com>
CChris wrote:
| | Yes, the usual penny drop.
1.000500 is not exactly representable in FPU hardware. According to the precise
sequence of operations you perform, this loss of acuracy shows up or doesn't.
If you were printing that "101" as sprint("%.16f",the_value), you'd see a number
a very tiny amount below 101, like 100.99999999857412. floor() correct,y returns
100.
CChris
|
Thanks to everyone who replied on this. I thought it was due to something
like that but I couldn't see the difference in the value even going out to 20
decimal places.
So does anyone have a suggestion for a better way to handle rounding? I'm
trying to verify a sales tax computation and after using this rounding
routine, I had a situation where the result of round(62.50 * 0.075, 2) was not
equal to the number that resulted from value("6.49").
|
|
 |
|
