I recently came across a weird case in an ETL process where the results seem unpredictable to me. I read Difference between numeric, float and decimal in SQL Server, but I don't think it's an overflow or decimal precision issue.
Scenario:
Source table "test" in SQL Server 2008 SP3, column a declared as numeric (38,6).
The result is cast first to real, and then to int. The issue doesn't occur if there is a direct cast from numeric to int.
Results of:
SELECT a,CAST(a as real) as real_a,CAST(CAST(a as real) as int) as int_a FROM test;
a: 778881838.810000
real_a: 7.78819E+08
int_a: 778881856
The same experiment, run in SQL Server 2017 (sql fiddle) gives this:
http://sqlfiddle.com/#!18/45aca/2
a: 778881838.81
real_a: 778881860
int_a: 778881856
I can (vaguely) understand the ..19E+08 case, but why is there a +18 difference in the double conversion case? The number seems completely arbitrary to me.
OK, first of all, the result in SQL Server 2017 for real_a is not 778881860. It is 778881856, exactly, just as in SQL Server 2008. How this floating-point value is presented by the client is another matter -- Management Studio shows me 7.788819E+08, sqlcmd produces 7.7888186E+8, and apparently SQL Fiddle uses another library altogether (one I would personally have issue with, seeing as how it obscures significant figures!)
This value is not arbitrary. REAL is a single-precision floating point type that cannot represent 778881838.81 exactly. The closest representable value is 778881856, hence your result (the next lower representable value is 778881792). Without casting to INT, you can see this value using
SELECT STR(CONVERT(REAL, CONVERT(NUMERIC(38, 6), 778881838.810000)), 40, 16)
778881856.0000000000000000
Your use of the term "double" makes me think you're confusing this with FLOAT, which is the double-precision floating point type. FLOAT cannot represent this value exactly either, but it comes much closer:
SELECT STR(CONVERT(FLOAT, CONVERT(NUMERIC(38, 6), 778881838.810000)), 40, 16)
778881838.8099999400000000
Converting this value to an INT yields the (truncated) 778881838. (This truncation is documented and does not happen for conversions to NUMERIC; you'll need to ROUND first before converting if you'd prefer 778881839 instead.)
Easy example for other people that want to test locally:
DECLARE #test numeric (38,6)='778881838.810000'
SELECT #test as [Original],CAST(#test as real) as real_a,CAST(CAST(#test as real) as int) as int_a;
Original real_a int_a
778881838.810000 7.788819E+08 778881856
You would likely need someone from Microsoft to explain the way it works inside the SQL engine (and certainly to know why they made that decision), but I'll take a stab at the reasoning:
If the output is in scientific notation on the first cast and is then needed to cast to an int, it sets the int to the minimum value that would result in that scientific notation. It ends in 6 instead of 5 because rounding on 5 does not consistently round up on all cases (Alternating tie-breaking for example).
But, no matter the reason, if precision is important, you should explicitly cast to a numeric data type with a defined precision.
When you want to convert from float or real to character data, using the STR string function is usually more useful than CAST( ). This is because STR enables more control over formatting. For more information, see STR (Transact-SQL) and Functions (Transact-SQL).
Please find the below links
USE STR Instead of real
STR example
Use the below query : -
SELECT a,STR(a ,38,6) as real_a,CAST(CAST(a as real) as int) as int_a FROM test;
Please let me know if you find any issue.
Related
This is something that has only just started happening today for me.
If I run this:
select '185.37' * 2.00
select '0.16' * 2.00
The first query returns this error:
Arithmetic overflow error converting varchar to data type numeric.
Where as the second query returns 0.32 as expected.
This is on SQL Server 15.0.2080.9.
I know I can fix the issue by casting to decimal or numeric, but why does the error only occur for the higher numbers? And why has this only started occurring today? This query has been used as part of an ETL that we've been using without changing anything for the past few years.
It tries to convert your string to a numeric(3,2) because that's the type on the right of the multiplication1. If you can force your value to be a larger numeric type:
select '185.37' * (102.00 - 100.00)
Then it works fine (produces 370.74) because it'll now attempt to convert it to a numeric(5,2).
Rather than doing it by a trick, however, I'd pick an appropriate numeric type for this and explicitly perform the required conversion, to better document what you want to occur:
select CONVERT(numeric(5,2),'185.37') * 2.00
1And it has a higher precedence.
EDIT (by Gordon Linoff):
SQL Server's use of type precedence for this purpose is explicitly stated in the documentation:
For comparison operators or other expressions, the resulting data type will depend on the rules of data type precedence.
There might be just a little confusion because the documentation is not clear that the scale and precision of numerics is explicitly part of the type (that is, what gets converted is numeric(3, 2) rather than numeric(?, ?) with appropriate scale and precision).
For below mentioned SQL Left side is varchar datatype and right side is a numeric datatype numeric(3,2) .
select '185.37' * 2.00
After implicit conversion from varchar to numeric, result data type will be numeric(3,2) but actual result 370.74 need a datatype of numeric(5,2) so this expression fails with overflow error. you can see the datatype of these in below sample code.
select 2.00 as colDatatype,185.37 as colDatatype2, 370.74 as colDatatype3 into #tempdata
SELECT * FROM tempdb.INFORMATION_SCHEMA.columns WHERE table_name like'#tempdata%';
In such cases explicitly cast the expression to desired out datatype and not merely
depend on implicit conversion of data types. You can refer the Datatype conversion chart on the website but here result data type after implicit conversion is not know.
Microsoft link
Without using scientific notation, I need to convert a FLOAT to a string, without showing scientific notation, and capturing all possible precision.
For example, when I execute SELECT 1E0 / 1346E0 I get the following result:
This is how SQL Server displays a FLOAT value by default.
In this case, it displays 18 decimal places, which is more than the STR function can provide.
It also does not add any trailing zeros.
If SQL Server Management Studio can do this, can I also get this conversion in my code?
I need to avoid scientific notation at all costs, even if there are 20 leading zeros after the decimal point. A long string is not a problem.
Unfortunately, the CONVERT function does not provide what I need, even with style 3
try format()
SELECT
1E0 / 1346E0
, format(1E0 / 1346E0,'N18')
declare #float float = 0.000742942050520059
select cast(cast(#Float as decimal(38,35)) as varchar(200))
As was also noted, format works too, although I'm not a huge fan of it as it's a kind of heavy hitting CLR. but for one offs, it's fine.
I have a table in which one of field is Real data type. I need to show the values in decimal format like #.###. So i'm converting the real values to decimal. But when i convert for some values it is not generating actual value. For eg:- 20.05 is the actual value. multiple it by 100 and then it to decimal(9,4) it will return like 2004.9999.
select cast(cast(20.05 as real)*100.00 as decimal(9,4))
Why this is returning like this ?
Real or Float are not precise...
Even if you see the value as "20.05", even if you type it in like this, there will be tiny differences.
Your value 2004.9999 (or similar something like 2005.00001) is due to the internal representation of this type.
If you do the conversion to decimal first, it should work as expected:
select cast(cast(20.05 as real) as decimal(9,4))*100.00
But you should really think about, where and why you use floating point numbers...
UPDATE: Format-function
With SQL-Server 2012+ you might use FORMAT() function:
SELECT FORMAT(CAST(20.05 AS REAL)*100,'###.0.000')
This will allow you, to sepcify the format, and you will get text back.
This is fine for presentation output (lists, reports), but not so fine, if you want to continue with some kinds of calculations.
I am trying to replicate tables from a remote SQL 2000 database into my local SQL 2012 instance.
As a quick way of checking for values which have changed, I am using the "UNION ALL...GROUP BY" technique found on Simple Talk (scroll about half-way down).
Unfortunately, the remote data types are set as REAL and as this is an approximate data type this is not very reliable as it finds differences where I don't want it to (even though those differences exist computationally).
I have tried using CONVERT to change the values to a NUMERIC (exact) data type. However, different columns have different numbers of decimal places and finding a one size fits all solution is proving difficult.
One thing I noticed is that if I run the following query (TimeID is an INT and Value1 is a REAL):
SELECT [TimeID], [Value1], CONVERT(DECIMAL(19,10), [Value1]) AS [CONV19,10], CONVERT(DECIMAL(19,3), [Value1]) AS [CONV19,3], CONVERT(DECIMAL(19,4), [Value1]) AS [CONV19,4]
FROM [DATABASE].[SCHEMA].[TABLE]
WHERE [TimeID] = 12345
I get the following results:
[TimeID] [Value1] [CONV19,10] [CONV19,3] [CONV19,4]
12345 1126.089 1126.0885009766 1126.089 1126.0885
Note that SQL Server Management Studio displays Value1 to 3 decimal places when in its native format (i.e. without me converting it).
So my question is: how does SSMS know that it should be displayed to 3 decimal places? How does it know that 1126.0885 is not the actual number stored, but instead is 1126.089?
Ideally I'd like to understand it's algorithm so I can replicate it to convert my data to the correct number of decimal places.
This won't answer your question but will give you a starting point to answer it yourself?
First read this:
http://msdn.microsoft.com/en-us/library/ms187912.aspx
Notably, "The behavior of float and real follows the IEEE 754 specification on approximate numeric data types."
Now read this:
http://en.wikipedia.org/wiki/IEEE_floating_point
So now you should know how float/ real numbers are stored and why they are "approximate" numbers.
As for how SSMS "knows" how many decimals are in a real/ float I don't really know, but it is going to be something to do with the IEEE 754 specification?
A simple script to demonstrate this is:
DECLARE #MyNumber FLOAT(24) = 1.2345;
SELECT #MyNumber, CONVERT(NUMERIC(19, 4), #MyNumber), CONVERT(NUMERIC(19, 10), #MyNumber), CONVERT(NUMERIC(19, 14), #MyNumber);
I don't know if this is the case, but I suspect that SSMS is using .NET numeric string formatting.
I was having a similar situation, I simply wanted to SELECT into a VARCHAR the exact same thing that SSMS was displaying in the query results grid.
In the end I got what I wanted with the FORMAT function, using the General format specifier.
For example:
DECLARE #Floats TABLE([FloatColumn] FLOAT);
INSERT INTO #Floats
VALUES
(123.4567),
(1.23E-7),
(PI());
SELECT
Number = f.FloatColumn,
Text = FORMAT(f.FloatColumn, 'G'),
IncorrectText = CONVERT(NVARCHAR(50), f.FloatColumn)
FROM #Floats f;
I have to give the disclaimer that I don't know if this will work as desired in all cases, but it worked for everything I needed it to.
I'm sure this is very useful after six years.
I'm dividing some integers x & y in MS SQL, and I wan the result to be in a floating-point form. 5/2 should equal 2.5. When I simply do
SELECT 5/2
I get 2, which doesn't suprise me, since it's creating an int from two ints. I know I can force it to a float by doing:
SELECT CAST(5 AS FLOAT)/CAST(2 AS FLOAT);
but that seems like overkill. I find that I can just as easily (and much more readably) get the same result by using
SELECT (0.0+5)/2;
I'm guessing that this is just some sort of implicit type-casting? Is there some reason either method is better/worse?
Under the covers there's no difference in implementation. The implicit casts accomplish the same thing as your explicit casts.
Since you write 0.0, TSQL interprets this as float value. All following ints are implicitly cast to float.
See also the implicit data type conversion matrix in the section Implicit Conversions
Not sure something being shorter is more readable since true reading involves comprehension.
SELECT CAST(5 AS FLOAT)/CAST(2 AS FLOAT);
There really is no doubt what the intention is here and will be understood when you come back to the code 6 months from now or when another developer looks at it for the first time.