There’s a past thread about floating arithmetic as it affected multiplication.
I’ve run into a similarly curious result on simply totaling a field of two decimal numbers. The field is set to float and has no output pattern that would hide more digits after a decimal.
601.49+198.44 = 799.9300000000001?
And the grand total works out to a dozen places after the decimal.
It’s easy enough to work around, but surprising to me.
I think you have an old copy of Panorama X there. The recent versions show 15 significant figures, not 16. 16 is right about the limit to the precision you can expect from a 64 bit floating point number. In some numeric ranges, 64 bits is more than enough. In others it’s not, and you can expect an error in that 16th digit.
My favorite analogy stems from my sophomore year in high school. They were adding a wing onto the building, and one wall, and all the lockers on that wall, had been demolished. This left us with fewer lockers than there were students. Some students had to share a locker.
Two students, who share a locker, have their geometry books in that locker. One of them needs a geometry book, and grabs one from the locker. She might get her own, or she might get her locker mate’s.
Floating point numbers have 53 binary significant figures. In some ranges of numbers, there are more 16 digit decimal numbers than there are 53 digit binaries. Some of those decimals need to share a binary locker. The result you get out, may be the number you want, or it might be its locker mate.
The original thread that you linked to links to a wealth of information about the mysteries of floating point arithmetic.
The problem is that neither 601.49 or 198.44 can be exactly represented in IEEE 64 bit floating point format. So approximations are used, and you get an approximate result. But it’s very close.
Wrestling with this same issue, especially as it relates to zero. I want to select all of the summary records that are not equal to zero. However apparently some records that appear to be zero, really are not presumably because of this issue.
If I do a select on Amount ≠ 0 it leaves some zero records. However if I use the round function, [ round(Amount,01) ≠ 0] then I get the desired result. A bit frustrating to have to go to that step but workable. However… when I try to do this programatically, it refuses to unselect any zero records, even those that are truly zero. Here is my code:
selectall
Removeallsummaries
Field DoneeID groupup
Field Amount Total
Outlinelevel “1"
select round(Amount,01)≠0
so it works from the Datasheet using the menu commands but won’t work in a procedure.
However if instead, I don’t use Outlinelevel “1” and instead use
select round(Amount,01) ≠ 0 and info(“summary”)=1
It works!
Seems bizarre but I’m sure there must be a logical reason…
Why are. you using a leading zero, your formula would be the same if it was
round(Amount,1)≠0
I wonder if you meant to use .01 instead of 01.
Anyway, I think it would be a lot simpler to check for non-zero by comparing with a small threshol value, something like this:
abs(Amount)<0.0000001
That makes no sense. The code being run is the same whether you run a menu command or a procedure. In fact, the menu command just quickly sets up and runs a procedure, then throws that procedure away. So there has to be no difference.
Well that was silly. Yes I meant to type the period. Thought I had, but clearly did not. Using the period makes the procedure work properly. However it is still odd that the results using the procedure were different than using the Find/Select menu from the Datasheet.
Your suggestion of the abs( function is good. Hadn’t thought of that.
However the larger issue of 0 not really being 0 remains. Just need to remember to use a modified function when that is an issue.
Thanks.
This will always be an issue in any program that does floating point arithmetic. If you add up a bunch of floating point numbers, and then subtract those same numbers, the result will be very close to zero, but probably not equal to it.