My question is conceptual.
In the code snippet below my expectation is to get exactly the same result for all "round" activations.
However, when using a constant as an argument, the result is different than when using a calculated or stored value.
In the documentation I did not find any comment to help me understand the reason for the difference.
In response to the similar question (sql server round function not working well), I was referred to the excellent article https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html, but in the following example, all cases are "float", so I don't think there are reasons for different results.
--
DECLARE #value FLOAT;
SET #value = 4255.0;
CREATE TABLE TESTTABLE (
FLOAT_VALUE FLOAT
)
INSERT INTO TESTTABLE(FLOAT_VALUE) VALUES (#value);
SELECT 0.015 * 4255.0 AS ORIGINAL_VALUE,
ROUND(0.015 * 4255.0, 2) AS ROUND_FROM_CONSTANT,
ROUND(0.015 * CONVERT(FLOAT, 4255.0), 2) AS ROUND_FROM_CAST,
ROUND(0.015 * #value, 2) AS ROUND_FROM_VARIABLE,
ROUND(0.015 * FLOAT_VALUE, 2) AS ROUND_FROM_FIELD
FROM TESTTABLE
The reason you're getting different results is because you have different data types throughout the expressions.
As a literal, 4225.0 is a decimal(5,1) not a float, so only when you explicitly cast the value to a float will it be treated as one. As a float and decimal don't operate the same as well, you get different results.
We'll go through each expression:
0.015 * 4255.0
These are both decimal values, a decimal(3,3) and a decimal(5,1). When dealing with different scales and precisions and multiplication, the following is used to calculate them:
Precision = p1 + p2 + 1
Scale = s1 + s2
Therefore the resulting precision and scale are 3+5+1 and 1+3 = decimal(9,4) This is why you get the result 63.8250.
ROUND(0.015 * 4255.0, 2)
Same as above, but then you round to 2 decimal places; little explanation needed here.
ROUND(0.015 * CONVERT(float, 4255.0), 2)
There are 2 data types here, the decimal(3,3) and the float. Use we have 2 different data types, data type precedence comes into play. As a 0.015 is implicitly cast to a float.
CONVERT(float,0.015) * CONVERT(float, 4255.0) returns 63.825, however, as float is an imprecision value, using ROUND on that ends up returning 63.82; because because the number is actually < 63.825 in the imprecise stored value (perhaps 63824999999999~).
ROUND(0.015 * #value, 2)
Same as 3.
ROUND(0.015 * FLOAT_VALUE, 2)
Same as 3.
References:
Data type precedence (Transact-SQL)
Precision, scale, and Length (Transact-SQL)
Related
I am dividing 100000=Amount into 9=Shift into Days=30
But when I need the actual amount so how do I get it?
In SQL:
Declare #R1 as money, #R2 as money, #R3 as money
Set #R1 = 100000 / 9 / 30
Set #R2 = #R1 * 9 * 30
select #R2 As 'ActualAmount'
I need exactly 100,000, but the result is 99900.00 - why?
By default sql will do integer division. Just add *1.0 to make it decimal division
Set #R1=100000*1.0/9/30
SQL DEMO
Use float instead of money
SELECT (100000/9/30)*9*30 result, 'v1' as version
UNION
SELECT (100000*1.0/9/30)*9*30, 'v2'
UNION
SELECT (CAST(100000 as float)/9/30)*9*30, 'v3'
ORDER BY version
OUTPUT
Welcome to the wonderful world of floating point rounding errors. 100000 / 9 = 11111.1111... repeating. Let's simplify the example and pretend we just want 10 / 3; different function, same problem. When 10 / 3 is expressed as a fraction, you could do (10 / 3) * 3, cancel out the threes, and get back to 10. However a computer stores 10 / 3 as 3.33333... repeating, out to as high a precision as it can get. What that precision is, is actually a somewhat complicated question to answer, but suffice it to say, it's not infinity. So what actually gets stored is something like 3.333333....0. When you multiply that by 3, the computer says "ah, you asked me to multiply 3.3333...0 * 3, which is of course 9.99999...0".
Mathematically, your expectation to get 10 back (or in your case, 100,000) is completely reasonable. But because of how a computer stores numbers with repeating decimals, the computer doesn't recognize there's an equivalence between 10 / 3 and 3.33333... (Actually, it sort of does, but the reason it does is probably more to do with luck; more on this later).
As was pointed out in another answer, the reason you're getting 99900 is because it's doing integer division rather than floating point division. So the question then becomes what do you cast 100000 as to get the answer you want?
In your case, float seems to get it right, although it's worth noting that this may not be the case in all circumstances. This probably turns out right because float has 53 decimal places available in its mantissa (significant digits) and with that level of precision, SQL makes an educated guess that you were, in fact, trying to re-form a fraction with infinitely repeating decimal places rather than multiply three numbers very close to, but not quite equal to the original number.
Money on the other hand is usually a bad choice as a data type because it's effectively just a small precision float and leads to problems like this. Decimal also has some problems in this case, as it will actually overshoot the re-formed value.
declare
#Money money = cast(100000 as money) / 2700,
#Float float = cast(100000 as float) / 2700,
#Decimal decimal(38,2) = cast(100000 as decimal(38, 2)) / 2700
select
_Money = #Money,
_MoneyReformed = #Money * 2700,
_Float = #Float,
_FloatReformed = #Float * 2700,
_Decimal = #Decimal,
_DecimalReformed = #Decimal * 2700
If you need a true, universally correct representation of fractions, you'll have to store the numerator and denominator in separate columns so you can manipulate each independently.
I'm trying the following using SQL Server 2008 R2 and SQL Server 2012 and I get the same results in both.
If I write the following statement:
select round(4.005, 2)
I get the expected result: 4.01
However if I write the following statements:
declare #result float
select #result = 4.005
select round(#result, 2)
I get an unexpected result: 4
But if I replace float with real in the previous statements:
declare #result real
select #result = 4.005
select round(#result, 2)
I get the expected result.
Can anyone tell me why is this happening?
because you haven't specify the number of bits that are used to store the mantissa of the float number in scientific notation , try this,
declare #result float(5)
select #result = 4.005
select round(#result, 2)
SQLFiddle Demo
float and real (Transact-SQL)
This has to do with float being an approximate data type.
see: http://technet.microsoft.com/en-us/library/ms187912(v=sql.105).aspx
The float and real data types are known as approximate data types. The behavior of float and real follows the IEEE 754 specification on approximate numeric data types.
Approximate numeric data types do not store the exact values specified for many numbers; they store an extremely close approximation of the value. For many applications, the tiny difference between the specified value and the stored approximation is not noticeable. At times, though, the difference becomes noticeable. Because of the approximate nature of the float and real data types, do not use these data types when exact numeric behavior is required, such as in financial applications, in operations involving rounding, or in equality checks. Instead, use the integer, decimal, money, or smallmoney data types.
Avoid using float or real columns in WHERE clause search conditions, especially the = and <> operators. It is best to limit float and real columns to > or < comparisons.
The IEEE 754 specification provides four rounding modes: round to nearest, round up, round down, and round to zero. Microsoft SQL Server uses round up. All are accurate to the guaranteed precision but can result in slightly different floating-point values. Because the binary representation of a floating-point number may use one of many legal rounding schemes, it is impossible to reliably quantify a floating-point value.
yes I had the same issue. I tested out some solutions...
SELECT round(cast(3.175 as float), 2) as roundingBAD,
CAST(round(TRY_CONVERT(DECIMAL(28,2), cast(3.175 as float)), 2) as
nvarchar(max)) as roundconvertBAD, cast(CAST(round(CAST(3.175 as
DECIMAL(18,10)), 4) as decimal(18,2)) as nvarchar(max)) as
roundconvert3, cast(FORMAT(round(CAST(3.175 as DECIMAL(18,10)), 2),
'0.######') as nvarchar(max)) as roundconvert4, cast(CAST(CAST(3.175
as DECIMAL(18,10)) as decimal(18,2)) as nvarchar(max)) as
roundconvert5P, cast(CAST(CAST(3.175 as DECIMAL(18,10)) as
decimal(18,1)) as nvarchar(max)) as roundconvert1DP
output: 3.17 3.17 3.18 3.18 3.18 3.2
I went with the last two options in my production Microsoft SQL Server code for rounding floats correctly.
Using SQL Server 2000
I want to reduce the decimal length
Query
Select 23/12 as total
Output is showing as 1.99999999999
I don't want to round the value, I want to diplay like this 1.99
Tried Query
Select LEFT(23/12, LEN(23/12) - 3) as total
The above query is working only if there is decimal value like 12.444444, but if the total is single digit means like 12 or 4 or 11...., i am getting error at run time.
How to do this.
Need Query Help
There is a very simple solution. You can find it in BOL. Round takes an optional 3rd argument, which is round type. The values are round or truncate.
ROUND numeric_expression , length [ ,function ] )
...
function Is the type of operation to perform. function must be
tinyint, smallint, or int. When function is omitted or has a value of
0 (default), numeric_expression is rounded. When a value other than 0
is specified, numeric_expression is truncated.
So just do
Select ROUND(cast(23 as float)/12, 2, 1) as total
That gives 1.91. Note, if you were really seeing 1.999 - something is really wrong with your computer. 23/12 = 1.916666666(ad infinitum). You need to cast one of the numbers as float since sql is assuming they're integers and doing integer division otherwise. You can of course cast them both as float, but as long as one is float the other will be converted too.
Not terribly elegant, but works for all cases:
Select CONVERT(float,LEFT(CONVERT(nvarchar, 23.0/12.0),CHARINDEX('.',CONVERT(nvarchar, 23.0/12.0)) + 2)) as total
Scalar Function
-- Description: Truncate instead of rounding a float
-- SELECT dbo.TruncateNumber(23.0/12.0,2)
-- =============================================
CREATE FUNCTION TruncateNumber
(
-- Add the parameters for the function here
#inFloat float,
#numDecimals smallint
)
RETURNS float
AS
BEGIN
IF (#numDecimals < 0)
BEGIN
SET #numDecimals = 0
END
-- Declare the return variable here
RETURN CONVERT(float,LEFT(CONVERT(nvarchar, #inFloat),CHARINDEX('.',CONVERT(nvarchar, #inFloat)) + #numDecimals))
END
GO
I am trying to divide one number by another in a SQL view. I'm dividing two columns that are each of type int, and the problem is that its giving me a rounded output, rather than an accurate one.
Here is my select:
SELECT numberOne, numberTwo, (numberOne*100/NULLIF(numberTwo, 0)) as PctOutput
This is working, but when it dives 3 by 37 its giving me 8 instead of 8.108.
How would I modify this to give me an accurate answer? Do I need to cast the numbers out of ints? if so - into what?
Try an implicit cast:
SELECT
numberOne,
numberTwo,
((1.0*numberOne)*100/NULLIF(1.0*numberTwo, 0)) as PctOutput
**--Cast the denominator as Float**
SELECT 3 * 100 / NULLIF(CAST(37 AS FLOAT),0) -- 8.10810810810811
**--Multiply by 1.0 in either numerator OR denominator**
SELECT 3 * 100 / NULLIF(37 * 1.0,0) -- 8.108108
SELECT 3.0 * 100 / NULLIF(37,0) -- 8.108108
**--Convert it to decimal**
SELECT CONVERT(DECIMAL(25,13), 3) * 100 /
NULLIF(CONVERT(DECIMAL(25,13), 37),0) -- 8.108108108
You need to cast the numbers from INT into either float or decimal.
If you use literals, they will likely be decimal (NUMERIC), not float (http://stackoverflow.com/questions/1072806/sql-server-calculation-with-numeric-literals)
Note that if you use decimal, you should be aware of the rules of scale and precision in division if you have numbers near the boundaries where you might lose precision:
http://msdn.microsoft.com/en-us/library/ms190476.aspx
In SQL Server the result is always of the same type as the inputs. So yes you do need to convert the inputs.
I generally convert using convert(decimal(9,2),numberOne) but depending you may want to convert(real,numberOne) or use a different level of precision.
I understand the host of issues in comparing floats, and lament their use in this case - but I'm not the table author and have only a small hurdle to climb...
Someone has decided to use floats as you'd expect GUIDs to be used. I need to retrieve all the records with a specific float value.
sp_help MyTable
-- Column_name Type Computed Length Prec
-- RandomGrouping float no 8 53
Here's my naive attempt:
--yields no results
SELECT RandomGrouping
FROM MyTable
WHERE RandomGrouping = 0.867153569942739
And here's an approximately working attempt:
--yields 2 records
SELECT RandomGrouping
FROM MyTable
WHERE RandomGrouping BETWEEN 0.867153569942739 - 0.00000001
AND 0.867153569942739 + 0.00000001
-- 0.867153569942739
-- 0.867153569942739
In my naive attempt, is that literal a floating point literal? Or is it really a decimal literal that gets converted later?
If my literal is not a floating point literal, what is the syntax for making a floating point literal?
EDIT: Another possibility has occurred to me... it may be that a more precise number than is displayed is stored in this column. It may be impossible to create a literal that represents this number. I will accept answers that demonstrate that this is the case.
EDIT: response to DVK.
TSQL is MSSQLServer's dialect of SQL.
This script works, and so equality can be performed deterministically between float types:
DECLARE #X float
SELECT top 1 #X = RandomGrouping
FROM MyTable
WHERE RandomGrouping BETWEEN 0.839110948199148 - 0.000000000001
AND 0.839110948199148 + 0.000000000001
--yields two records
SELECT *
FROM MyTable
WHERE RandomGrouping = #X
I said "approximately" because that method tests for a range. With that method I could get values that are not equal to my intended value.
The linked article doesn't apply because I'm not (intentionally) trying to straddle the world boundaries between decimal and float. I'm trying to work with only floats. This isn't about the non-convertibility of decimals to floats.
Response to Zinglon:
A literal value that can find my records, thanks.
DECLARE #Y binary(8)
SET #Y = 0x3FEAD9FF34076378
SELECT *
FROM MyTable
WHERE convert(binary(8), RandomGrouping) = #Y
is that literal a floating point literal? Or is it really a decimal
literal that gets converted later?
If my literal is not a floating point literal, what is the syntax for
making a floating point literal?
The 0.867153569942739 literal in SQL Server is a decimal type, not float.
The engine automatically picks appropriate scale and precision to represent the given literal.
To write a literal of the float type you should use the scientific notation, like this:
0.867153569942739E0
This is documented in Constants (Transact-SQL)
decimal constants
decimal constants are represented by a string of
numbers that are not enclosed in quotation marks and contain a decimal
point.
The following are examples of decimal constants:
1894.1204
2.0
float and real constants
float and real constants are represented by
using scientific notation.
The following are examples of float or real values:
101.5E5
0.5E-2
The sp_describe_first_result_set can tell us the types of columns
EXEC sp_describe_first_result_set N'SELECT 0.867153569942739, 0.867153569942739E0'
It returns numeric(15,15) for the first column and float for the second.
If your column RandomGrouping is indexed, it is much more efficient to use a float literal, because when you wrap RandomGrouping in convert(), an index can't be used.
The following query will use an index:
SELECT *
FROM MyTable
WHERE RandomGrouping = 0.867153569942739E0
The following query will not use an index:
SELECT *
FROM MyTable
WHERE convert(binary(8), RandomGrouping) = #Y
It is possible that the values are being truncated on display. I'm assuming the column doesn't have a unique constraint on it, otherwise the question would be moot. On my setup, SSMS truncates the more precise value in this script.
create table flt ( f float not null primary key )
insert into flt
select 0.111111111111111
union all
select 0.1111111111111111
select f, cast(f as binary(8)) from flt
Similarly, if these values are distinct you can cast them to binary(8) and identify them based on that value, like this:
select f from flt
where cast(f as binary(8)) = 0x3FBC71C71C71C71C
The problem is not whether it's a floating point literal or not.
The problem is that comparing two floats for equality in Sybase (or any DB server) is not deterministic, since 4.00000000000000000000... and 3.99999999999999999999... are the same exact number but aren't equal.
Your second solution is the only correct way to compare floats for "equality" (that is, are they the same up to a precision).
Why are you saying "approximately working" about your second approach?
Since you didn't provide the specific DB server you use, here's a fairly decent write-up of the problem (with basically the same conclusions as above) for MySQL
http://dev.mysql.com/doc/refman/5.0/en/problems-with-float.html