I'm trying the following using SQL Server 2008 R2 and SQL Server 2012 and I get the same results in both.
If I write the following statement:
select round(4.005, 2)
I get the expected result: 4.01
However if I write the following statements:
declare #result float
select #result = 4.005
select round(#result, 2)
I get an unexpected result: 4
But if I replace float with real in the previous statements:
declare #result real
select #result = 4.005
select round(#result, 2)
I get the expected result.
Can anyone tell me why is this happening?
because you haven't specify the number of bits that are used to store the mantissa of the float number in scientific notation , try this,
declare #result float(5)
select #result = 4.005
select round(#result, 2)
SQLFiddle Demo
float and real (Transact-SQL)
This has to do with float being an approximate data type.
see: http://technet.microsoft.com/en-us/library/ms187912(v=sql.105).aspx
The float and real data types are known as approximate data types. The behavior of float and real follows the IEEE 754 specification on approximate numeric data types.
Approximate numeric data types do not store the exact values specified for many numbers; they store an extremely close approximation of the value. For many applications, the tiny difference between the specified value and the stored approximation is not noticeable. At times, though, the difference becomes noticeable. Because of the approximate nature of the float and real data types, do not use these data types when exact numeric behavior is required, such as in financial applications, in operations involving rounding, or in equality checks. Instead, use the integer, decimal, money, or smallmoney data types.
Avoid using float or real columns in WHERE clause search conditions, especially the = and <> operators. It is best to limit float and real columns to > or < comparisons.
The IEEE 754 specification provides four rounding modes: round to nearest, round up, round down, and round to zero. Microsoft SQL Server uses round up. All are accurate to the guaranteed precision but can result in slightly different floating-point values. Because the binary representation of a floating-point number may use one of many legal rounding schemes, it is impossible to reliably quantify a floating-point value.
yes I had the same issue. I tested out some solutions...
SELECT round(cast(3.175 as float), 2) as roundingBAD,
CAST(round(TRY_CONVERT(DECIMAL(28,2), cast(3.175 as float)), 2) as
nvarchar(max)) as roundconvertBAD, cast(CAST(round(CAST(3.175 as
DECIMAL(18,10)), 4) as decimal(18,2)) as nvarchar(max)) as
roundconvert3, cast(FORMAT(round(CAST(3.175 as DECIMAL(18,10)), 2),
'0.######') as nvarchar(max)) as roundconvert4, cast(CAST(CAST(3.175
as DECIMAL(18,10)) as decimal(18,2)) as nvarchar(max)) as
roundconvert5P, cast(CAST(CAST(3.175 as DECIMAL(18,10)) as
decimal(18,1)) as nvarchar(max)) as roundconvert1DP
output: 3.17 3.17 3.18 3.18 3.18 3.2
I went with the last two options in my production Microsoft SQL Server code for rounding floats correctly.
Related
DECLARE #TAX VARCHAR(30)=120.45
DECLARE #TaxRoundOf VARCHAR(30)
SET #TaxRoundOf=ROUND(#TAX,1)
SELECT #TaxRoundOf
This Gives Result (#TaxRoundOf=120.5)
DECLARE #TAX VARCHAR(30)=146.45
DECLARE #TaxRoundOf VARCHAR(30)
SET #TaxRoundOf=ROUND(#TAX,1)
SELECT #TaxRoundOf
This Gives Result (#TaxRoundOf=146.4)
But I need to return 146.50 . why this mismatch between two results?
any one can help plz?
Since you are using VARCHAR to store your numbers, SQL Server is having to do implicit conversion to float behind the scenes, which is having knock on effects on your calculations. You can reproduce this using the below query:
SELECT ROUND(CONVERT(FLOAT, 120.45),1), -- 120.5
ROUND(CONVERT(FLOAT, 146.45),1), -- 146.4
ROUND(CONVERT(DECIMAL(10, 2), 120.45),1), -- 120.50
ROUND(CONVERT(DECIMAL(10, 2), 146.45),1) -- 146.50
Since floating point numbers are not exact, 146.45 cannot be exactly represented as a float, and ends up being stored as a very slightly smaller number, so when this is passed to the round function, it is rounded down, instead of up.
The solution, as demonstrated by the 3rd and 4th columns in the above query, is to use a more precise data type.
You can use this:
SET #TaxRoundOf=ROUND(10 * CAST(#TAX AS FLOAT)) / 10
instead of:
SET #TaxRoundOf=ROUND(#TAX,1)
DEMO
PS as #GarethD already mentioned I wouldn't use #TAX as VARCHAR type.
You can also rely on numeric rounding instead of converting your string to a float, which can lose information.
Cast a string to numeric and then round:
select round(cast('146.45' as numeric(18,2)), 1)
-- 146.50
A decimal constant is already a decimal so there's no need to cast it:
select round(146.45, 1)
-- 146.50
We are doing some validation of data which has been migrated from one SQL Server to another SQL Server. One of the things that we are validating is that some numeric data has been transferred properly. The numeric data is stored as a float datatype in the new system.
We are aware that there are a number of issues with float datatypes, that exact numeric accuracy is not guaranteed, and that one cannot use exact equality comparisons with float data. We don't have control over the database schemas nor data typing and those are separate issues.
What we are trying to do in this specific case is verify that some ratio values were transferred properly. One of the specific data validation rules is that all ratios should be transferred with no more than 4 digits to the right of the decimal point.
So, for example, valid ratios would look like:
.7542
1.5423
Invalid ratios would be:
.12399794301
12.1209377
What we would like to do is count the number of digits to the right of the decimal point and find all cases where the float values have more than four digits to the right of it. We've been using the SUBSTRING, LEN, STR, and a couple of other functions to achieve this, and I am sure it would work if we had numeric fields typed as decimal which we were casting to char.
However, what we have found when attempting to convert a float to a char value is that SQL Server seems to always convert to decimal in between. For example, the field in question shows this value when queried in SQL Server Enterprise Manager:
1.4667
Attempting to convert to a string using the recommended function for SQL Server:
LTRIM(RTRIM(STR(field_name, 22, 17)))
Returns this value:
1.4666999999999999
The value which I would expect if SQL Server were directly converting from float to char (which we could then trim trailing zeroes from):
1.4667000000000000
Is there any way in SQL Server to convert directly from a float to a char without going through what appears to be an intermediate conversion to decimal along the way? We also tried the CAST and CONVERT functions and received similar results to the STR function.
SQL Server Version involved: SQL Server 2012 SP2
Thank you.
Your validation rule seems to be misguided.
An SQL Server FLOAT, or FLOAT(53), is stored internally as a 64-bit floating-point number according to the IEEE 754 standard, with 53 bits of mantissa ("value") plus an exponent. Those 53 binary digits correspond to approximately 15 decimal digits.
Floating-point numbers have limited precision, which does not mean that they are "fuzzy" or inexact in themselves, but that not all numbers can be exactly represented, and instead have to be represented using another number.
For example, there is no exact representation for your 1.4667, and it will instead be stored as a binary floating-point number that (exactly) corresponds to the decimal number 1.466699999999999892708046900224871933460235595703125. Correctly rounded to 16 decimal places, that is 1.4666999999999999, which is precisely what you got.
Since the "exact character representation of the float value that is in SQL Server" is 1.466699999999999892708046900224871933460235595703125, the validation rule of "no more than 4 digits to the right of the decimal point" is clearly flawed, at least if you apply it to the "exact character representation".
What you might be able to do, however, is to round the stored number to fewer decimal places, so that the small error at the end of the decimals is hidden. Converting to a character representation rounded to 15 instead of 16 places (remember those "15 decimal digits" mentioned at the beginning?) will give you 1.466700000000000, and then you can check that all decimals after the first four are zeroes.
You can try using cast to varchar.
select case when
len(
substring(cast(col as varchar(100))
,charindex('.',cast(col as varchar(100)))+1
,len(cast(col as varchar(100)))
)
) = 4
then 'true' else 'false' end
from tablename
where charindex('.',cast(col as varchar(100))) > 0
For this particular number, don't use STR(), and use a convert or cast to varchar. But, in general, you will always have precision issues when storing in float... it's the nature of the storage of that datatype. The best you can do is normalize to a NUMERIC type and compare with threshold ranges (+/- .0001, for example). See the following for a breakdown of how the different conversions work:
declare #float float = 1.4667
select #float,
convert(numeric(18,4), #float),
convert(nvarchar(20), #float),
convert(nvarchar(20), convert(numeric(18,4), #float)),
str(#float, 22, 17),
str(convert(numeric(18,4), #float)),
convert(nvarchar(20), convert(numeric(18,4), #float))
Instead of casting to a VarChar you might try this: cast to a decimal with 4 fractional digits and check if it's the same value as before.
case when field_name <> convert(numeric(38,4), field_name)
then 1
else 0
end
The issue you have here is that float is an approximate number data type with an accuracy of about seven digits. That means it approaches the value while using less storage than a decimal / numeric. That's why you don't use float for values that require exact precision.
Check this example:
DECLARE #t TABLE (
col FLOAT
)
INSERT into #t (col)
VALUES (1.4666999999999999)
,(1.4667)
,(1.12399794301)
,(12.1209377);
SELECT col
, CONVERT(NVARCHAR(MAX),col) AS chr
, CAST(col as VARBINARY) AS bin
, LTRIM(RTRIM(STR(col, 22, 17))) AS rec
FROM #t
As you see the float 1.4666999999999999 binary equals 1.4667. For your stated needs I think this query would fit:
SELECT col
, RIGHT(CONVERT(NVARCHAR(MAX),col), LEN(CONVERT(NVARCHAR(MAX),col)) - CHARINDEX('.',CONVERT(NVARCHAR(MAX),col))) AS prec
from #t
We are undergoing a migration in our tool. Some of the datatypes in our tables have been changed from real to float. To understand the impact I have run the following code:
declare #realVariable as real,
#floatVariable as float
set #realVariable=152304.11999512
set #floatVariable=152304.11999512
select cast(#realVariable AS decimal(15, 4)) as realcol,str(#floatVariable,15,4) as floatcol
The result is different for both real and float variables:
real=152304.1250
float= 152304.1200
How can I make sure that the float result is same as real?
float and real are both approximate-number data types for floating point numeric data.
real is the same as float(24). If you declare float without a suffix, then it means float(53).
So, you are converting from real (which is float(24)) to float(53). You are simply making the numbers more precise.
float(24) has a precision of 7 digits and float(53) has a precision of 15 digits.
See Books Online > float and real (Transact-SQL): https://msdn.microsoft.com/en-GB/library/ms173773.aspx
If we repeat your example and display the variables without conversion, we can see that the real variable is displayed with 7 digits precision, and float variable is displayed with 14 digits precision, which is exactly what you set it to.
DECLARE #realVariable as real,
#floatVariable as float;
-- Set to a number with 14 digits precision
SET #realVariable=152304.11999512;
SET #floatVariable=152304.11999512;
SELECT #realVariable AS '#realVariable', #floatVariable AS '#floatVariable';
The simple answer to your question is that the two numbers are not the same. The one with float(53) is more precise.
Calculating the ABS of the difference of the two numbers and then comparing this to an acceptable threshold is one way to do it. However this would not be satisfactory if the scale can vary widely.
To illustrate why this might not be satisfactory:
r = 0.0001234567
f = 0.00012345678901234
Comparing the difference to a threshold of 0.0005 would not be useful.
So it might be better to use a threshold that is a percentage of the number.
DECLARE #threshold AS float, #delta AS float;
SET #threshold = ABS(#floatVariable / 1000000);
SET #delta = ABS(#floatVariable - CAST(#realVariable AS float));
SELECT #realVariable AS '#realVariable',
#floatVariable AS '#floatVariable',
#threshold AS '#threshold',
#delta AS '#delta',
CASE
WHEN #delta < #threshold THEN N'OK'
ELSE N'Different'
END AS 'Comparison';
Note: This solution might need a bit of adjustment for numbers that are very close to zero.
#RichardCL's description describes the way things work (+1). Now, if you are converting a value stored as a real datatype value into a value stored as a float datatype value, then the entire and precise "original" value will be stored in the resulting value--you will lose no information from your existing set of real values.
Referring to your example, your value of 152304.11999512 cannot be stored precisely as a real value--it will be rounded and stored as 152304.125. Convert this to a float, and you will still have 152304.125.
You need a delta. Compute the absolute value of the difference between the real and float values then make sure that is less than your delta. The delta should be sufficiently small to meet your needs.
For example using your values:
abs(152304.1250 - 152304.1200) <= .0050 so the use .0050 as your delta value.
At some point I have a numeric(28,10) and I cast it in money (I know its bad but for legacy reason I have to return money) in the same time I also have to set the sign (multiplying by +1/-1).
In a first attempt I had cast the +/-1 to match the numeric type.
For the value 133.3481497944 we encounter a strange behavior (I have simplified the actual code in order to keep only the elements needed to demonstrate the problem):
SELECT CAST(CAST(133.3481497944 AS numeric(28,10))*cast(1 AS numeric(28,10)) AS money)
133.3482
which is not correctly rounded...
Removing the cast solve the problem
SELECT CAST(CAST(133.3481497944 AS numeric(28,10)) * 1 AS money)
133.3481
Did someone know what is happening in SQL? How can a multiplication by 1 and cast(1 AS numeric(28,10)) affect the result of the rounding?
When multiplying numerics, SQL uses the following rules to determine the precision and scale of the output:
p = p1 + p2 + 1
s = s1 + s2
which makes sense - you wouldn't want 1.5 * 2.5 to be truncated to one digit past the decimal. Nor would you want 101 * 201 to be limited to 3 digits of precision, giving you 20300 instead of 20301.
In your case that would result in a precision of 57 and a scale of 20, which isn't possible - the maximum precision and scale is 38.
If the resulting type is too big, decimal digits are sacrificed in order to preserve the integral (most significant) part of the result.
From the SQL Programmability & API Development Team Blog:
In SQL Server 2005 RTM (and previous versions), we decided preserve a minimum scale of 6 in both multiplication and division.
So your answer depands on how big and precise you need the multiplier to be. In order to preserve 10 digits of decimal precision. If the multiplier needs a scale bigger than 9, then decimal digits may be truncated. If you use a smaller precision and scale, you should be fine:
SELECT CAST(CAST(133.3481497944 AS numeric(28,10))*cast(1 AS numeric(9,7)) AS money)
yields 133.3481.
I don't see any ROUNDing here. I only see casting. Don't assume that it will round, when you CAST. Historically, when we cast the environment truncates (SQL server or not) or behaves not as we expect - especially when we're talking about FLOATs.
SELECT
CAST(CAST(133.3481497944 AS numeric(28,10))*cast(1 AS numeric(28,10)) AS money) --Your original,
CAST(1 AS numeric(28,10)) --Just the 1 casted,
CAST(133.3481497944 AS numeric(28,10)) --Your expected calculation,
CAST(133.3481497944 AS numeric(28,10))*cast(1 AS numeric(28,10)) -- The actual calculation
SELECT
CAST(133.3481497944 AS numeric(28,10))*cast(1.5 AS numeric(28,10)),
CAST(133.3481497944 AS numeric(28,10))*1.5,
CAST((133.3481497944*1) AS money),
133.3481497944*1
Returns
133.3482
1.0000000000
133.3481497944
133.348150
200.022225
200.02222469160
133.3481
133.3481497944
So as mentioned above, there really isn't any true rounding, but a loss of precision during the cast. As to exactly why, I don't know. Most likely during the calculation(multiplication) while using the Numeric(28,10) it cuts off some precision.
I added the second lines to show that really you may not need your numeric casting.
I understand the host of issues in comparing floats, and lament their use in this case - but I'm not the table author and have only a small hurdle to climb...
Someone has decided to use floats as you'd expect GUIDs to be used. I need to retrieve all the records with a specific float value.
sp_help MyTable
-- Column_name Type Computed Length Prec
-- RandomGrouping float no 8 53
Here's my naive attempt:
--yields no results
SELECT RandomGrouping
FROM MyTable
WHERE RandomGrouping = 0.867153569942739
And here's an approximately working attempt:
--yields 2 records
SELECT RandomGrouping
FROM MyTable
WHERE RandomGrouping BETWEEN 0.867153569942739 - 0.00000001
AND 0.867153569942739 + 0.00000001
-- 0.867153569942739
-- 0.867153569942739
In my naive attempt, is that literal a floating point literal? Or is it really a decimal literal that gets converted later?
If my literal is not a floating point literal, what is the syntax for making a floating point literal?
EDIT: Another possibility has occurred to me... it may be that a more precise number than is displayed is stored in this column. It may be impossible to create a literal that represents this number. I will accept answers that demonstrate that this is the case.
EDIT: response to DVK.
TSQL is MSSQLServer's dialect of SQL.
This script works, and so equality can be performed deterministically between float types:
DECLARE #X float
SELECT top 1 #X = RandomGrouping
FROM MyTable
WHERE RandomGrouping BETWEEN 0.839110948199148 - 0.000000000001
AND 0.839110948199148 + 0.000000000001
--yields two records
SELECT *
FROM MyTable
WHERE RandomGrouping = #X
I said "approximately" because that method tests for a range. With that method I could get values that are not equal to my intended value.
The linked article doesn't apply because I'm not (intentionally) trying to straddle the world boundaries between decimal and float. I'm trying to work with only floats. This isn't about the non-convertibility of decimals to floats.
Response to Zinglon:
A literal value that can find my records, thanks.
DECLARE #Y binary(8)
SET #Y = 0x3FEAD9FF34076378
SELECT *
FROM MyTable
WHERE convert(binary(8), RandomGrouping) = #Y
is that literal a floating point literal? Or is it really a decimal
literal that gets converted later?
If my literal is not a floating point literal, what is the syntax for
making a floating point literal?
The 0.867153569942739 literal in SQL Server is a decimal type, not float.
The engine automatically picks appropriate scale and precision to represent the given literal.
To write a literal of the float type you should use the scientific notation, like this:
0.867153569942739E0
This is documented in Constants (Transact-SQL)
decimal constants
decimal constants are represented by a string of
numbers that are not enclosed in quotation marks and contain a decimal
point.
The following are examples of decimal constants:
1894.1204
2.0
float and real constants
float and real constants are represented by
using scientific notation.
The following are examples of float or real values:
101.5E5
0.5E-2
The sp_describe_first_result_set can tell us the types of columns
EXEC sp_describe_first_result_set N'SELECT 0.867153569942739, 0.867153569942739E0'
It returns numeric(15,15) for the first column and float for the second.
If your column RandomGrouping is indexed, it is much more efficient to use a float literal, because when you wrap RandomGrouping in convert(), an index can't be used.
The following query will use an index:
SELECT *
FROM MyTable
WHERE RandomGrouping = 0.867153569942739E0
The following query will not use an index:
SELECT *
FROM MyTable
WHERE convert(binary(8), RandomGrouping) = #Y
It is possible that the values are being truncated on display. I'm assuming the column doesn't have a unique constraint on it, otherwise the question would be moot. On my setup, SSMS truncates the more precise value in this script.
create table flt ( f float not null primary key )
insert into flt
select 0.111111111111111
union all
select 0.1111111111111111
select f, cast(f as binary(8)) from flt
Similarly, if these values are distinct you can cast them to binary(8) and identify them based on that value, like this:
select f from flt
where cast(f as binary(8)) = 0x3FBC71C71C71C71C
The problem is not whether it's a floating point literal or not.
The problem is that comparing two floats for equality in Sybase (or any DB server) is not deterministic, since 4.00000000000000000000... and 3.99999999999999999999... are the same exact number but aren't equal.
Your second solution is the only correct way to compare floats for "equality" (that is, are they the same up to a precision).
Why are you saying "approximately working" about your second approach?
Since you didn't provide the specific DB server you use, here's a fairly decent write-up of the problem (with basically the same conclusions as above) for MySQL
http://dev.mysql.com/doc/refman/5.0/en/problems-with-float.html