I have a problem with round in SQL Server 2014: when I round a number to 2 decimal places sometimes the rounded number is different if I cast to float before or not.
For example, if I execute:
select round(cast(3.945 as float),2)
select round(3.945,2)
I have:
3.94
3.950
But if I execute:
select round(cast(3.935 as float),2)
select round(3.935,2)
I have:
3.94
3.940
It seems incorrect, rounding 3.935 and 3.945 casting to float before, I obtain the same value. Is this a bug?
The problem here is that float is a binary floating point type, where the representation is an approximation of the value. Floats do not losslessly convert to or from base 10, because there is no power of 10 that is also a power of 2. So when this is converted it is done in a way that leaves a roundoff error that pushes the value just before the rounding threshold.
Oddly I cannot reproduce the same behaviour on PostgreSQL and I am not entirely sure why (it may be that on PostgreSQL, round takes a numeric value and this forces a conversion back).
Never use floats where absolute accuracy is required. This occurs not only in databases, but in almost every programming language as well.
As #ChrisTravers says in his answer the issue with rounding a float is that you're not getting exact arithmetic. i.e. That explains why round(3.945,2) rounds up to 3.95 whilst round(3.945E0,2) effectively rounds down to 3.94.
If you're wondering why you see more than 2 decimal places in some cases, that's because of the type you're dealing with. i.e. 3.94 is a float, so doesn't have a specified number of decimal places; whilst 3.950 is the result of rounding a decimal(4,3); which even though we've rounded to 2 decimal places doesn't affect the precision of the type (i.e. it's still decimal(4,3); not converted to decimal(4,2) or decimal(3,2)).
If the purpose of this rounding is for display purposes, you're best of using the str function. i.e.
select str(3.945,4,2) --decimal
select str(3.945E0,4,2) --float
In the above the 4 is the length of the string (i.e. includes the decimal point as a character), and the 2 is the number of decimal places to show.
NB: In this scenario you're chaning the data type to varchar(4).
The below code allows you to see what type you get after performing an operation:
declare #result sql_variant = str(3.945E0,4,2)
select sql_variant_property(#result, 'BaseType') [BaseType]
,sql_variant_property(#result, 'MaxLength') [MaxLength]
,sql_variant_property(#result, 'Precision') [Precision]
,sql_variant_property(#result, 'Scale') [Scale]
Related
I know REAL data type is not accurate and normally for currency I should use numeric data type.
But, I'm asked to do some stuff and one of the conditions is that the data type is real.
When I try to do round((....),2) for example, I get that round function does not exist for this data type.
My question is, without converting, is there any function that can return a REAL value rounded to 0?
Many thanks!1
As you can see here it's no way to round without any type cast. It's only two kinds of function exists:
round(dp or numeric) - round to nearest integer
round(v numeric, s int) - round to s decimal places
Real = double precision. So you need to use convert anyway if you want to get some decimal places:
select round('123.456789'::real::numeric,2)
upd. Keep care about rounding+cast at big real numbers:
select round('12122156.567'::real::numeric, 2); --< rounding up to 6 digits, result = 12122200
select round('12122156.567'::real::DOUBLE PRECISION::numeric,2); --<< rounding result = 12122157
Or you can use round without decimal places:
select round('123.456789'::real)
round a numeric value to 0 after the dot?
ROUND(numeric_value, 0)
After investigation, converting to ::numeric is the only way around
For a SQL int that is being converted to a float, how do I set the precision of the floating point number?
This is the selection I would like to truncate to two or 3 decimal places:
AVG(Cast(e.employee_level as Float))avg_level,
Thanks!
In TSQL, you can specify two different sizes for float, 24 or 53. This will set the precision to 7 or 15 digits respectively.
If all you want to do is truncate to a set number of decimal places, you can use ROUND, ie:
ROUND(AVG(CAST(e.employee_level as float)), 3)
As a general rule, you can't specify the number of digits after the decimal point for a floating-point number. Floating point data types store the closest floating-point approximation to any given value. The closest floating-point approximation is unlikely to have the number of digits you want. Although you might be able to suppress every digit after the third one, that will only change the appearance of the value, not the value itself.
Integers are a different story. An integer--stored, converted, or cast to a floating-point data type--will be stored exactly over a large range. Floating-point data types don't have to store any fractional units for integers.
I'd suggest, though that the best practice for you is to
avoid casting integers to floating-point if you don't need fractional units, or
cast integers to decimal or numeric if you do need fractional units, or
handle display issues entirely in application code.
I have had the same issue when calculating a percentage and needing a resulting string value.
Example: 68 is what % of 379
Result is a float = 17.9419525065900
You can cast/convert to Numeric with the Precision = 2 and get 17.94
If you need the value as a string you can then cast it as a VarChar if needed.
You can use Round() as well but in this case it only makes 17.9419525065900 = 17.9400000000000.
You can also use Ceiling() and Floor() to get the next highest or lowest integer.
Ceiling(17.9419525065900) = 18
Floor(17.9419525065900) = 17
Using these combinations you should be able to achieve a result in any format you need.
Let's say I have the following 2 queries:
select sum(cast(2666 as float)) * cast(.3 as float)
select sum(cast(2666 as real)) * cast(.3 as real)
The 1st query returns: 799.8
The 2nd query returns: 799.800031781197
Why does the 2nd query not return the same thing as the 1st?
Binary floating point types (like real and float) cannot exactly represent decimal numbers. In particular it is not possible to exactly store 0.3 as a binary floating point number. Instead a number very close to 0.3 is stored. This is called a representation error.
The size of the error is different for real and float because they have different precision.
If you want to store decimal numbers more accurately, consider using decimal or numeric. But note that even though these types can accurately store decimal values up to a certain number of digits, calculations can still produce numbers that cannot be represented exactly. For example the result of 0.1 / 0.3 can not be stored exactly in a decimal even though both 0.1 and 0.3 can. In this case the result will be rounded to the nearest value that can be stored in the type (e.g. 0.333333333 depending on the precision).
So I'm dividing an integer by a decimal, and storing the result in a decimal column. However, it always drops the fractional component(the part after the decimal point). If I multiply the result by 10 or 100 I get a more accurate result, but dividing again drops the fractional part again.
The two fields I've inserted into were a precision 5, scale 0 decimal and a precision 5, scale 3 decimal.
I've also tried casting the integer into a decimal and that doesn't make a difference, neither does multiplying by 1.0.
I'm out of ideas or tricks to try.
Thanks, Buzkie
Turns out I was casting incorrectly. After doing using the correct format
CAST(int AS DECIMAL(5,3))
it worked. I had left off the precision and scale before.
I am running this sql code in SQL 2005
declare #a as float(24)
set #a=0.85
select cast ( #a as float(53))
and the result is
0.850000023841858
does anyone know why?
Thanks.
The first seven digits are the default level of precision for a float when in the declaration float(N) the value of N is 24 or less. After that, the digits could show up as pretty much anything when cast to a float of greater precision. That's the 'floating point'.
http://msdn.microsoft.com/en-us/library/ms173773.aspx
The number you see is as close as the computer can get within however many binary digits it has available to use.
If you try and write 1/3 in decimal, but you only have enough space for 8 digits, the closest you can get is 0.33333333. That's still off by a quite a way, but if you had more decimal places you could get more accurate. This is exactly the same probably as the computer faces, but whereas each of your successive digits represents 1/10ths, 1/100ths, 1/1000ths, the computer works in 1/2, 1/4, 1/8, 1/16.