I'm testing some bits of code, a number which involves computation using floating-point values - often very large numbers of these. I have some generic (C++-templated, but it doesn't really matter for the sake of this question) code which compares my outputs, be they scalar or arrays, against their expected values.
I'm faced with the problem of choosing a precision threshold, at least for the two C/C++ floating-point types float and double - for various functions I'm testing. As is well known, there is no one-size-fits-all with respect to comparing floating-point values, nor a single precision value which fits and computation based solely on the data type: Relative vs. absolute error, numerous operations which may magnify floating-point rounding errors a lot, computations which are supposed to arrive at 0 so you can't really normalize by the expected value, etc.
What is a generally-reasonable approach/algorith/rule-of-thumb to choosing a comparsion method (and equality thresholds) for floating point values?
I like the approach used in googletest, e.g. EXPECT_DOUBLE_EQ(a,b) and EXPECT_FLOAT_EQ(a,b): the numbers are approximately equal if they are within 4 units in the last position (4 ULP). To do this, you
convert signed-magnitude to offset
subtract as though they were integers
check that the difference <= 4.
This automatically scales for magnitude and relaxes to absolute near zero.
There is no generally-reasonable approach :-(
One important property of numbers is that the set of numbers can be divided into equivalence classes where all members of the same equivalence class are "equal" in some sense and all members of two different equivalence classes are "not equal". That property is essential for sorting algorithms and hashing.
If you take double with 53 bit mantissa, and just replace the last bits of the mantissa with zeroes, then you still have equivalence classes, and sorting / hashing will work just fine. On the other hand, two numbers can be arbitrarily close together and still compare equal with this method.
The other method is having an algorithm that decides if two numbers are "possibly equal". You can base everything else on this. For example, a is "definitely greater" than b if a > b and a is not "possibly equal" to b. a is "possibly greater" than b if a > b or a is "possibly equal" to b.
Sorting is problematic. You could have a "possibly equal" to b, and b "possibly equal" to c, but a is not "possibly equal" to c.
If you use double with 53 bits mantissa, then it is unlikely that two unrelated numbers are equal within even 45 bits. So you could check quite reasonably whether the absolute value of the difference is less than the absolute value of the larger number, divided by 2^45. Your mileage will vary considerably. Important is whether you think 0 should be equal to very small numbers or not.
Related
The Python standard library contains the function math.isclose, which is equivalent to:
abs(a - b) <= max(rtol * max(abs(a), abs(b)), atol)
The Numpy library likewise contains numpy.isclose and numpy.allclose, which are equivalent to:
abs(a - b) <= (atol + rtol * abs(b))
Neither documentation page explains why you would want to use one of these formulas over the other, or provides any principled criteria for choosing sensible absolute and relative tolerances, written above as atol and rtol respectively.
I very often end up having to use these functions in tests for my code, but I never learned any principled basis for choosing between these two formulas, or for choosing tolerances that might be appropriate to my use case.
I usually just leave the default values as-is unless I happen to know that I'm doing something that could result in a loss of numerical precision, at which point I hand-tune the tolerances until the results seem right, largely based on gut feeling and checking examples by hand. This is tedious, imperfect, and seems antithetical to the purpose of software testing, particularly property-based testing.
For example, I might want to assert that two different implementations of the same algorithm produce "the same" result, acknowledging that an exact equality comparison doesn't make sense.
What are principled techniques that I can use for choosing a sensible formula and tolerances for comparing floating point numbers? For the sake of this question, I am happy to focus on the case of testing code that uses floating-point numbers.
For example, I might want to assert that two different implementations of the same algorithm produce "the same" result, acknowledging that an exact equality comparison doesn't make sense.
Consider instead of a singular true/false assessment of the "same" result, attempt to rate the algorithms same-ness on various properties.
If the assessments are within your tolerance/limits, functions are the "same".
Given g(x) and r(x) (the reference function).
Absolute difference: Try y = abs(g(x) - r(x)) for various (if not all) x. What is the largest y?
Relative difference: Try y = abs((g(x) - r(x))/r(x)) for various normal r(x) (not zeroes). What is the largest y?
Relative difference: Like above with r(x) with sub-normal results. Here relative difference may be far larger than with normals and so handled separately. r(x) == +/-0.0 deserves special assessment.
Range test/ edge cases: What is largest/smallest greatest/least x that "works". e.g. y = my_exp(x) and exp(x) may return infinity or 0.0 at different x, but are otherwise nearly the "same".
Total ordering difference: (a favorite). Map all non-NAN floating point values -inf to +inf to an integer: [-ORDER_N to ORDER_N] with a helper function called total order(). total order(+/-0.0) is 0. Find the maximum difference abs(total_order(g(x)) - total_order(r(x))) and use that metric to determine "same"-ness.
Various function deserve special handling. This field of study has many further considerations.
One question when using relative tolerance is - relative to what? If you want to know if 90 and 100 are "equal" with a 10% tolerance, you get different answers if you take 10% of 90 vs 10% of 100.
The standard library uses the larger of a or b when defining the "what" in that scenario, so it would use 10% of 100 as the tolerance. It also uses the larger of that relative tolerance or the absolute tolerance as the "ultimate" tolerance.
The numpy method simbly uses b for the "relative" tolerance and takes the total of the relative and absolute tolerance as the "ultimate" tolerance.
Which is better? Neither is better or worse- they are different ways of establishing a tolerance. You can choose which one to use based on how you want to define "close enough".
The tolerances you choose are contextual as well - are you comparing lengths of lumber or the distance between circuit paths in a microprocessor? Is 1% tolerance "good enough" or do you need ultra-precise tolerance? A tolerance too low might yield too many "false positives" depending on the application, while too high a tolerance will yield too many "false negatives" that might let some problems "slip through the cracks".
Note that the standard function is not vectorized, so if you want to use it on arrays you'll either have to use the numpy function or build a vertorized version of the standard one.
Nobody can choose the tolerances for you, they are problem dependent. Because in real-life the input data that you work on has (very) limited accuracy, be it the result of experimental measurement or of numerical computation that introduces truncation errors. So you need to know your data and understand the concepts and methods of error calculus to adjust them.
As regards the formulas, they were designed to be general-purpose, i.e. not knowing if the quantities to be compared can be strictly equal or not (when they are strictly equal, the relative error does not work). Again, this should not be a blind choice.
Quick Aside: I'm going to use the word "Float" to refer to both a .Net float and a SQL float with only 7 significant digits. I will use the word "Double" to refer to a .Net double and a SQL float with 15 significant digits. I also realize that this is very similar to some other posts regarding decimals/doubles, but the answers on those posts are really inconsistent, and I really want some recommendations for my specific circumstance...
I am part of a team that is rewriting an old application. The original app used floats (7 digits). This of course caused issues since the app conducted a lot of calculations and rounding errors accumulated very quickly. At some point, many of these floats were changed to decimals. Later, the floats (7) in the database all became doubles (15). After that we had several more errors with calculations involving doubles, and they too were changed to decimals.
Today about 1/3 of all of our floating point numbers in the database are decimals, the rest are doubles. My team wants to "standardize" all of our floating-point numbers in the database (and the new .Net code) to use either exclusively decimals or doubles except in cases where the other MUST be used. The majority of the team is set on using decimals; I'm the only person on my team advocating using doubles instead of decimals. Here's why...
Most of the numbers in the database are still doubles (though much of the application code still uses floats), and it would be a lot more effort to change all of the floats/doubles to decimals
For our app, none of the fields stored are "exact" decimal quantities. None of them are monetary quantities, and most represent some sort of "natural" measurement (e.g. mass, length, volume, etc.), so a double's 16 significant digits are already way more precise than even our initial measurements.
Many tables have measurements stored in two columns: 1 for the value; 1 for the unit of measure. This can lead to a HUGE difference in scale between the values in a single column. For example, one column can store a value in terms of pCi/g or Ci/m3 (1 Ci = 1000000000000 pCi). Since all the values in a single decimal columns must have the same scale (that is... an allocated number of digits both before and after the decimal point), I'm concerned that we will have overflow and rounding issues.
My teammates argue that:
Doubles are not as accurate nor as precise as decimals due to their inability to exactly represent 1/10 and that they only have 16 significant digits.
Even though we are not tracking money, the app is a inventory system that keeps track of material (mostly gram quantities) and it needs to be "as accurate as possible".
Even after the floats were changed to doubles, we continued to have bad results from calculations that used doubles. Changing these columns (and the application code) to decimals caused these calculations to produce the expected results.
It is my strong belief that the original issues where caused due to floats only having 7 significant digits and that simple arithmetic (e.g. 10001 * 10001) caused them to the data to quickly use up the few significant digits that they had. I do not believe this had anything to do with how binary-floating point numbers can only approximate decimal values, and I believe that using doubles would have fixed this issue.
I believe that the issue with doubles arose because doubles were used along side decimals in calculations that values were be converted back and forth between data types. Many of these calculations would round between intermediary steps in the calculation!
I'm trying to convince my team not to make everything under the sun into a decimal. Most values in the database don't have more than 5 or 6 significant digits anyway. Unfortunately, I am out-ranked by other members of my team that see things rather differently.
So, my question then is...
Am I worrying over nothing? Is there any real harm done by using almost exclusively decimals instead of doubles in an application with nearly 200 database tables, hundreds of transactions, and a rewrite schedule of 5 to 6 years?
Is using decimals actually solving an issue that doubles could not? From my research, both decimals and doubles are susceptible to rounding errors involving arbitrary fractions (adding 1/3 for example) and that the only way to account for this is to consider any value within a certain tolerance as being "equal" when comparing doubles and/or decimals.
If it is more appropriate to use doubles, what arguments could I make (other than what I have already made) could convince my team to not change everything to decimals?
Use decimal when you need perfect accuracy as a base-10 number (financial data, grades)
Use double or float when you are storing naturally imprecise data (measurements, temperature), want much faster mathematical operations, and can sacrifice a minute amount of imprecision.
Since you seem to be only storing various measurements (which have some precision anyways), float would be the logical choice (or double if you need more than 7 digits of precision).
Is using decimals actually solving an issue that doubles could not?
Not really - The data is only going to be as accurate as the measurements used to generate the data. Can you really say that a measured quantity is 123.4567 grams? Does the equipment used to measure it have that level of precision?
To deal with "rounding errors" I would argue that you can't really say whether a measurement of 1234.5 grams is exactly halfway - it could just as easily be 1234.49 grams, which would round down anyways.
What you need to decide is "what level of precision is acceptable" and always round to that precision as a last step. Don't round your data or intermediate calculations.
If it is more appropriate to use doubles, what arguments could I make (other than what I have already made) could convince my team to not change everything to decimals?
Other than the time spent switching, the only thing you're really sacrificing is speed. The only way to know how much speed is to try it both ways and measure the difference.
You'd better try your best not to lose precision. I guess my fault may convince you to choose double.
===> I did some wrong arithmetic, and it returns something very weird:
given 0.60, it returns 5
int get_index(double value) {
if (value < 0 || value > 1.00)
return -1;
return value / 0.10;
}
and I fixed it:
int get_index(double value) {
if (value < 0 || value > 1.00)
return -1;
return (value * 100000000) / (0.10 * 100000000);
}
I'm literately just doing a multiplication of two floats. How come these statements produce different results ? Should I even be using floats ?
500,000.00 * 0.001660 = 830
How come these statements produce different results ?
Because floating-point arithmetic is not exact and apparently you were not printing the multiplier precisely enough (i. e. with sufficient number of decimal digits). And it wasn't .00166 but something that seemed 0.00166 rounded.
Should I even be using floats ?
No. For money, use integers and treat them as fixed-point rational numbers. (They still aren't exact, but significantly better and less error-prone.)
You didn't show how you initialized periodicInterest, and presumably you think you set it to 0.00166, but in fact the error in your output is large enough that you must not have explicitly initialized it as periodicInterest = 0.00166. It must be closer to 0.00165975, and the difference between 0.00166 and 0.00165975 is definitely large enough not to just be a single floating-point rounding error.
Assuming you are working with monetary quantities, you should use NSDecimalNumber or NSDecimal.
One non-obvious benefit of using NSDecimalNumber is that it works with NSNumberFormatter, so you can let Apple take care of formatting currencies for all sorts of foreign locales.
UPDATE
In response to the comments:
“periodicInterest is clearly not a monetary quantity” and “decimal is no more free of error when dividing by 12 than binary is” - for inexact quantities, I can think of two concerns:
One concern is using sufficient precision to give accurate results. NSDecimalNumber is a floating-point number with 38 digits of precision and an exponent in the range -128…127. This is more than twice the number of decimal digits an IEEE 'double' can store. The exponent range is less than that of a double, but that's unlikely to matter in financial computing. So NSDecimalNumbers can definitely result in smaller error than floats or doubles, even though none of them can store 1/12 exactly.
The other concern is matching the results computed by some other system, like your bank or your broker or the NYSE. In that case, you need to figure out how that other system is storing numbers and computing with them. If the other system is using a decimal format (which is likely in the financial sector), then NSDecimalNumber will probably be useful.
“Wouldn't it be more efficient to use primitive types to do floating point arithmetic, specially thousands in real time.” Arithmetic on primitive types is far faster than arithmetic on NSDecimalNumbers. I haven't measured it, but a factor of 100 would not surprise me.
You have to strike a balance between your requirements. If decimal accuracy is paramount (as it often is in financial programming), you must sacrifice performance for accuracy. If decimal accuracy is not so important, you can consider carefully using a primitive type, but you should be aware of the accuracy you're sacrificing. Even then, the size of a float is so small (usually only 7 significant decimal digits) that you should probably be using double (at least 15, usually 16 significant decimal digits).
If you need to perform millions of arithmetic operations per second with true decimal accuracy, you might be able to do it using doubles, if you are an IEEE 754 expert capable of analyzing your code to figure out where errors are introduced and how to eliminate them. Few people have this level of expertise. (I don't claim to.) You must also understand how your compiler turns your Objective-C code into machine instructions.
Anyway, perhaps you are just writing a casual app to compute a rough estimate of net present value or future value. In that case, using double would probably suffice, but using NSDecimalNumber would probably also be sufficiently fast. Without knowing more about the app you're writing, I can't give you more specific advice.
When you run a VBA macro that performs numeric calculations which result in a decimal value, the result that is returned may be incorrect.
Here are a couple of examples:
Dim me_wrong as Double
me_wrong = 1000 - 999.59
RESULT = 0.409999999999968
Dim me_wrong_too as Double
me_wrong_too = 301.84 - 301
RESULT = 0.839999999999975
I have never ever noticed this before. What on earth is going on???
I have seen the following article about Office 97 but can't find anything about a bug in Excel 2007:
http://support.microsoft.com/default.aspx?scid=kb;en-us;165373
Plus it doesn't explain why I have never seen it before.
Please help!
The explanation for the problem from Office 97 and VBA is equally applicable going forward into Excel 2007. The core VBA system is largely unchanged despite the migration into later versions, hence the same kinds of accuracy gremlins that plague older VBA macros will persist.
The fundamental problem lies with the inherent inaccuracy in the representation of fractional numbers in binary, and how at least some effort to mitigate that inaccuracy was made with IEEE floating point representations. There is a very decent treatment of the subject of IEEE representation at this location.
*Edit: Just a minor bit of extra info for detail. *
For a very simple example that illustrates this issue in a trivial case, consider a situation in which decimals are represented as sums of inverse powers of two, eg 2-1, 2-2, 2-3, and so on. That ends up looking like .5, .25, .125, and so on. If you're representing exactly those numbers, all is good. However, consider a number like .761; 2-1+2-2 gets you to .750, but now you need .011. 2-3 (.125) is too big, but 2-4 (.0625) is too small...so you keep going to smaller powers of two, realizing you'll never quite represent the number precisely.
The choice becomes where you stop resolving and accept the inherent inaccuracy as being "good enough" for the problem you're solving/modeling.
It is, unfortunately, not a bug.
Double representation follows a fixed point notation, where the mantissa is a number "1,x" with "1" being implicit. There's an exponent and a sign, which makes the full representation in Base 2.
The pertinent issue is Base=2 which makes "x" in "1,x" to be a finite-precision (53bits of it) fractional binary. Think x= a52*1/2+a51*1/4+a50*1/8+...+a*1**1/(2^52)+a0*1/(2^53), where a< i > are the bits in the mantissa.
Try attaining 1,4 with this representation, and you hit the precision wall... there is no finite decomposition of 0.4 in binary weights. So the norm specifies you should represent the number immediately before the real one, which leaves you with 0,39999..9997346 (or whatever the tail is).
The "good" news is, and I've just burned four "c" coding days last week on that subject, you can do without Doubles if you represent your number using a very small scale (say 10^-9), store then in very large variables (long64), and do your display functions using nothing but integers (mathematically slicing away integral and fractional parts through integer division and their remainders). A treat, I tell you... not.
I have an sql:
SELECT Sum(Field1), Sum(Field2), Sum(Field1)+Sum(Field2)
FROM Table
GROUP BY DateField
HAVING Sum(Field1)+Sum(Field2)<>0;
Problem is sometimes Sum of field1 and field2 is value like: 9.5-10.3 and the result is -0,800000000000001. Could anybody explain why this happens and how to solve it?
Problem is sometimes Sum of field1 and
field2 is value like: 9.5-10.3 and the
result is -0.800000000000001. Could
anybody explain why this happens and
how to solve it?
Why this happens
The float and double types store numbers in base 2, not in base 10. Sometimes, a number can be exactly represented in a finite number of bits.
9.5 → 1001.1
And sometimes it can't.
10.3 → 1010.0 1001 1001 1001 1001 1001 1001 1001 1001...
In the latter case, the number will get rounded to the closest value that can be represented as a double:
1010.0100110011001100110011001100110011001100110011010 base 2
= 10.300000000000000710542735760100185871124267578125 base 10
When the subtraction is done in binary, you get:
-0.11001100110011001100110011001100110011001100110100000
= -0.800000000000000710542735760100185871124267578125
Output routines will usually hide most of the "noise" digits.
Python 3.1 rounds it to -0.8000000000000007
SQLite 3.6 rounds it to -0.800000000000001.
printf %g rounds it to -0.8.
Note that, even on systems that display the value as -0.8, it's not the same as the best double approximation of -0.8, which is:
- 0.11001100110011001100110011001100110011001100110011010
= -0.8000000000000000444089209850062616169452667236328125
So, in any programming language using double, the expression 9.5 - 10.3 == -0.8 will be false.
The decimal non-solution
With questions like these, the most common answer is "use decimal arithmetic". This does indeed get better output in this particular example. Using Python's decimal.Decimal class:
>>> Decimal('9.5') - Decimal('10.3')
Decimal('-0.8')
However, you'll still have to deal with
>>> Decimal(1) / 3 * 3
Decimal('0.9999999999999999999999999999')
>>> Decimal(2).sqrt() ** 2
Decimal('1.999999999999999999999999999')
These may be more familiar rounding errors than the ones binary numbers have, but that doesn't make them less important.
In fact, binary fractions are more accurate than decimal fractions with the same number of bits, because of a combination of:
The hidden bit unique to base 2, and
The suboptimal radix economy of decimal.
It's also much faster (on PCs) because it has dedicated hardware.
There is nothing special about base ten. It's just an arbitrary choice based on the number of fingers we have.
It would be just as accurate to say that a newborn baby weighs 0x7.5 lb (in more familiar terms, 7 lb 5 oz) as to say that it weighs 7.3 lb. (Yes, there's a 0.2 oz difference between the two, but it's within tolerance.) In general, decimal provides no advantage in representing physical measurements.
Money is different
Unlike physical quantities which are measured to a certain level of precision, money is counted and thus an exact quantity. The quirk is that it's counted in multiples of 0.01 instead of multiples of 1 like most other discrete quantities.
If your "10.3" really means $10.30, then you should use a decimal number type to represent the value exactly.
(Unless you're working with historical stock prices from the days when they were in 1/16ths of a dollar, in which case binary is adequate anyway ;-) )
Otherwise, it's just a display issue.
You got an answer correct to 15 significant digits. That's correct for all practical purposes. If you just want to hide the "noise", use the SQL ROUND function.
I'm certain it is because the float data type (aka Double or Single in MS Access) is inexact. It is not like decimal which is a simple value scaled by a power of 10. If I'm remembering correctly, float values can have different denominators which means that they don't always convert back to base 10 exactly.
The cure is to change Field1 and Field2 from float/single/double to decimal or currency. If you give examples of the smallest and largest values you need to store, including the smallest and largest fractions needed such as 0.0001 or 0.9999, we can possibly advise you better.
Be aware that versions of Access before 2007 can have problems with ORDER BY on decimal values. Please read the comments on this post for some more perspective on this. In many cases, this would not be an issue for people, but in other cases it might be.
In general, float should be used for values that can end up being extremely small or large (smaller or larger than a decimal can hold). You need to understand that float maintains more accurate scale at the cost of some precision. That is, a decimal will overflow or underflow where a float can just keep on going. But the float only has a limited number of significant digits, whereas a decimal's digits are all significant.
If you can't change the column types, then in the meantime you can work around the problem by rounding your final calculation. Don't round until the very last possible moment.
Update
A criticism of my recommendation to use decimal has been leveled, not the point about unexpected ORDER BY results, but that float is overall more accurate with the same number of bits.
No contest to this fact. However, I think it is more common for people to be working with values that are in fact counted or are expected to be expressed in base ten. I see questions over and over in forums about what's wrong with their floating-point data types, and I don't see these same questions about decimal. That means to me that people should start off with decimal, and when they're ready for the leap to how and when to use float they can study up on it and start using it when they're competent.
In the meantime, while it may be a tad frustrating to have people always recommending decimal when you know it's not as accurate, don't let yourself get divorced from the real world where having more familiar rounding errors at the expense of very slightly reduced accuracy is of value.
Let me point out to my detractors that the example
Decimal(1) / 3 * 3 yielding 1.999999999999999999999999999
is, in what should be familiar words, "correct to 27 significant digits" which is "correct for all practical purposes."
So if we have two ways of doing what is practically speaking the same thing, and both of them can represent numbers very precisely out to a ludicrous number of significant digits, and both require rounding but one of them has markedly more familiar rounding errors than the other, I can't accept that recommending the more familiar one is in any way bad. What is a beginner to make of a system that can perform a - a and not get 0 as an answer? He's going to get confusion, and be stopped in his work while he tries to fathom it. Then he'll go ask for help on a message board, and get told the pat answer "use decimal". Then he'll be just fine for five more years, until he has grown enough to get curious one day and finally studies and really grasps what float is doing and becomes able to use it properly.
That said, in the final analysis I have to say that slamming me for recommending decimal seems just a little bit off in outer space.
Last, I would like to point out that the following statement is not strictly true, since it overgeneralizes:
The float and double types store numbers in base 2, not in base 10.
To be accurate, most modern systems store floating-point data types with a base of 2. But not all! Some use or have used base 10. For all I know, there are systems which use base 3 which is closer to e and thus has a more optimal radix economy than base 2 representations (as if that really mattered to 99.999% of all computer users). Additionally, saying "float and double types" could be a little misleading, since double IS float, but float isn't double. Float is short for floating-point, but Single and Double are float(ing point) subtypes which connote the total precision available. There are also the Single-Extended and Double-Extended floating point data types.
It is probably an effect of floating point number implementations. Sometimes numbers cannot be exactly represented, and sometimes the result of operations is slightly off what we may expect for the same reason.
The fix would be to use a rounding function on the values to cut off the extraneous digits. Like this (I've simply rounded to 4 significant digits after the decimal, but of course you should use whatever precision is appropriate for your data):
SELECT Sum(Field1), Sum(Field2), Round(Sum(Field1)+Sum(Field2), 4)
FROM Table
GROUP BY DateField
HAVING Round(Sum(Field1)+Sum(Field2), 4)<>0;