|
Re: Glitch in floor() function?
|
CChris
|
Jun 28, 2008 02:35 PDT
|
posted by: CChris <christian.cuvier at agricu?t?re.gouv.fr>
Mike Wever wrote:
| |
CChris wrote:
| | Yes, the usual penny drop.
1.000500 is not exactly representable in FPU hardware. According to the precise
sequence of operations you perform, this loss of acuracy shows up or doesn't.
If you were printing that "101" as sprint("%.16f",the_value), you'd see a number
a very tiny amount below 101, like 100.99999999857412. floor() correct,y returns
100.
CChris
|
Thanks to everyone who replied on this. I thought it was due to something
like that but I couldn't see the difference in the value even going out to 20
decimal places.
So does anyone have a suggestion for a better way to handle rounding? I'm
trying to verify a sales tax computation and after using this rounding
routine, I had a situation where the result of round(62.50 * 0.075, 2) was not
equal to the number that resulted from value("6.49").
|
Use integers everywhere. Convert amounts which are, say in £, to pence or hundredths of pence thousandths if needed, and only work on integers. Note that some of those integers may become atoms in Eu, but there is no loss of precision in the process - unless you deal with amounts with 12+ digits, which I doubt.
CChris
|
|
 |
|
