using awk and printf not rounding correctly [duplicate] - awk

Question

This question already has answers here:
Is floating point math broken?
(31 answers)
Closed 6 years ago.
I have an issue where I'm using printf to round a float to the proper number of decimal points. I'm getting inconsistent results as shown below.
echo 104.45 | awk '{printf "%.1f\n",$1}'
104.5 <-- seem to be correct behaviour
echo 104.445 | awk '{printf "%.2f\n",$1}'
104.44 (should be 104.45) <-- seems to be INCORRECT behaviour
echo 104.4445 | awk '{printf "%.3f\n",$1}'
104.445 <-- seems to be correct behaviour
I've seen examples where float number in calculations may cause problems, but did not expect this with formatting.

Answer 1

The number 104.4445 cannot be represented exactly as a binary number. In other words, your computer doesn't know such a number.
# echo 104.4445 | awk '{printf "%.20f\n",$1}'
104.44450000000000500222
# echo 104.445 | awk '{printf "%.20f\n",$1}'
104.44499999999999317879
That's why the former is rounded to 104.445, while the latter is rounded to 104.44 .
The sjsam's answer is relevant only to numbers which can be represented exactly as a binary number, i. e. m/2**n , where m and n are integers and not too big. Changing ROUNDINGMODE to "A" has absolutely no effect on printing 104.45, 104.445, or 104.4445 :
# echo 104.4445 | awk -v ROUNDMODE="A" '{printf "%.3f\n",$1}'
104.445
# echo 104.4445 | awk '{printf "%.3f\n",$1}'
104.445
# echo 104.445 | awk -v ROUNDMODE="A" '{printf "%.2f\n",$1}'
104.44
# echo 104.445 | awk '{printf "%.2f\n",$1}'
104.44

Answer 2

I tried something analogous in Python and got similar results to you:
>>> round(104.445, 2)
104.44
>>> round(104.4445, 3)
104.445
This seems to be run-of-the-mill wonky floating point wonkiness, especially considering that the floating-point representation of 104.445 is less than the actual mathematical value of 104.445:
>>> 104.445 - 104.44
0.0049999999999954525
>>> 104.445 - 104.44 + 104.44
104.445

Answer 3

I strongly suspect that the reason for this behavior has less to do with awk than with how computers store numbers. As user31264 states: "Your computer doesn't know such a number [as 104.4445]."
Here are the results of an experiment I just did with the JavaScript Scratchpad in Pale Moon Web browser:
(104.45).toFixed(55)
/*
104.4500000000000028421709430404007434844970703125000000000
*/
(104.445).toFixed(55)
/*
104.4449999999999931787897367030382156372070312500000000000
*/
(104.4445).toFixed(55)
/*
104.4445000000000050022208597511053085327148437500000000000
*/
In all probability, your awk interpreter is not dealing with the decimal numbers 104.45, etc., but rather with the "wonky" values shown here. Rounding the first, second, and third of these "wonky" values to, respectively, 1, 2, and 3 decimal places will give the same results as your awk interpreter is giving you.