Why does this SQL query fail - sql

The query below is failing unexpectedly with an arithmatic overflow error.
select IsNull(t2.val, 5005)
from(
SELECT 336.6 as val UNION ALL
SELECT NULL
) as t2
"Arithmetic overflow error converting int to data type numeric."
Strangely if the query is modified to remove the NULL and replace it with the same value as in the null coalesce (5005), it runs without issue
select IsNull(t2.val, 5005)
from(
SELECT 336.6 as val UNION ALL
SELECT 5005
) as t2
Also, omitting the SELECT NULL line entirely allows the query to run without issue
select IsNull(t2.val, 5005)
from(
SELECT 336.6 as val
) as t2
If the coalesce value in the IsNull function is changed to an integer which is small enough to convert to the decimal in the subquery without widening, the query runs
select IsNull(t2.val, 500)
from(
SELECT 336.6 as val UNION ALL
SELECT NULL
) as t2
Tested this in both SQL Server 2005 and SQL Server 2008.
Ordinarily combining integers with decimals is seamless and SQL Server will convert both the integer and the decimal into a decimal type large enough to accommodate both. But for some reason running a query where the cast occurrs from both the UNION and the IsNull, causes the cast to fail.
Does anyone know why this is?

Try doing this
select * into t2
from(
SELECT 336.6 as val
UNION ALL
SELECT NULL
) as x
If you now look at the columns, you see a numeric with numeric precision of 4 and scale of 1
select * from INFORMATION_SCHEMA.COLUMNS where TABLE_NAME='T2'
SQL made that decision based on the smallest numeric precision to hold 336.6. Now, when you ask it to convert the NULL to 5005, you are saying, convert any NULL values to a number too big to fit in a numeric with the precision of 4 and a scale of 1. The error message indicates that 5005 won't fit in Numeric(4,1)
This will work because the table will now generate a larger numeric field, since SQL needs to accommodate 5005. Create the table using the new contents of T2 from below, and the field type should go to Numeric(5,1) allowing the 5005 to fit.
select IsNull(t2.val, 5005)
from(
SELECT 336.6 as val UNION ALL
SELECT 5005
) as t2
When you run the statement without a NULL in your inner query, SQL never evaluates the 5005, so it doesn't reach a condition where it needs to put 5005 into a numeric(4,1) field.
select IsNull(t2.val, 5005)
from(
SELECT 336.6 as val
) as t2

I think the problem is that when SQL Server resolves the union, it decides on a decimal type that is only large enough to fit 333.6 (which is decimal(4,1)). Trying to put 5005 into that results in an overflow.
You can get around that specifying the precision of decimal yourself:
select IsNull(t2.val, 5005)
from(
SELECT CONVERT(DECIMAL(5,1), 336.6) as val UNION ALL
SELECT NULL
) as t2

I believe its because your 336.6 value is being inferred to be of data type NUMERIC. If you want to be more specific, then explicity cast it to DECIMAL;
SELECT IsNull(t2.val, CAST(5005 AS DECIMAL))
FROM (
SELECT CAST(336.6 AS DECIMAL) AS val
UNION ALL
SELECT NULL
) AS t2

Related

Datatype mismatch error because of NULL values in union operation

AFAIU, an SQL engine internally assigns a datatype to NULL values, for example, some engines use the datatype integer as default. I know, that this may cause an error in UNION operations when the column of the other table is not compatible (e.g. VARCHAR) (here). Still, I struggle to understand why the following code works/does not work (on Exasol DB):
A)
This works
SELECT NULL AS TEST
UNION
SELECT DATE '2022-11-03' AS TEST
;
B)
But when I do 'the same' using a subquery, it throws a datatypes are not compatible error.
SELECT A.* FROM (SELECT NULL AS TEST) A
UNION
SELECT DATE '2022-11-03' AS TEST
;
C)
B can be fixed by explicit type casting of the NULL value:
SELECT A.* FROM (SELECT CAST(NULL AS DATE) AS TEST) A
UNION
SELECT DATE '2022-11-03' AS TEST
;
Still, I do not understand what happens in B behind the scenes, so A works but B does not. Apparently, the subquery (or a join) makes a difference for the type of the NULL column. But why?
Can anyone explain what exactly happens here?
PS. The same is the case for JOINS.
B2)
Does not work.
SELECT 'Dummy' AS C1, SELECT NULL AS TEST
UNION
SELECT 'Dummy' AS C1, SELECT DATE '2022-11-03' AS TEST
;
C2)
Does work.
SELECT 'Dummy' AS C1, SELECT CAST(NULL AS DATE) AS TEST
UNION
SELECT 'Dummy' AS C1, SELECT DATE '2022-11-03' AS TEST
;
In general, NULL has the type BOOLEAN in Exasol. You can check this either by using TYPEOF or DESCRIBE:
select TYPEOF(NULL); -- The TYPEOF function is available since version 7.1
-- Old way to see the type of an expression:
create schema s;
create or replace table tmp as select null expr;
describe tmp;
However, sometimes Exasol changes this type internally. One of these cases is if a NULL literal occurs directly inside an UNION clause. It only does this if it is an explicit NULL literal. E.g. the following also doesn't work:
SELECT NULL+NULL AS TEST
UNION
SELECT DATE '2022-11-03' AS TEST;
But it works if the expected type is BOOLEAN:
SELECT NULL+NULL AS TEST
UNION
SELECT TRUE AS TEST;
As you mentioned, the workaround is to wrap the expression in a CAST.

Oracle Case Failure Not An Object Or Ref

I'm getting the failure ORA-22806: not an object or REF
when querying
SELECT (CASE WHEN "SHAPE" IS NULL THEN NULL
ELSE ("SHAPE").SDO_POINT.X END)
FROM (SELECT null AS "SHAPE" FROM DUAL)
in Oracle. It appears Oracle is evaluating the else statement even though the case expression is true. Anyone know a way to make this work? Requirements are to return null if "SHAPE" is null, otherwise return ("SHAPE").SDO_POINT.X. Edit: This also needs to be done without modifying the subquery.
Oracle isn't evaluating the ELSE statement, but it seems it is checking the types of columns and expressions in your query.
I suspect that it starts type-checking your query by type-checking the subquery. The subquery is SELECT null AS "SHAPE" FROM DUAL. While type-checking this subquery Oracle doesn't know what the type the SHAPE column should be, so it chooses a type for it rather than make do without knowing the type for the time being and see if it can find out the real type later on.
For me (Oracle 18c XE), the error goes away if I cast null to SDO_GEOMETRY:
SQL> SELECT (CASE WHEN "SHAPE" IS NULL THEN NULL
2 ELSE ("SHAPE").SDO_POINT.X END)
3 FROM (SELECT CAST(null AS SDO_GEOMETRY) AS "SHAPE" FROM DUAL);
(CASEWHEN"SHAPE"ISNULLTHENNULLELSE("SHAPE").SDO_POINT.XEND)
-----------------------------------------------------------
SQL>
I am guessing that your database has a table with a SHAPE column of type SDO_GEOMETRY, but I could of course be wrong.
I forgot to mention another requirement - I can't modify the subquery
#SStrong: this additional requirement boils down to: "my unmodified code has a bug: how can I remove the bug without modifying my code?"
So all you can is put in an intermediary wrapper around the subquery which handles NULL values without applying case, using #lukewoodward suggestion. Something like this:
SELECT (CASE WHEN "SHAPE" IS NULL THEN NULL
ELSE ("SHAPE").SDO_POINT.X END)
FROM ( select case when "SHAPE" is null then cast(null as sdo_geometry)
else "SHAPE" end as "SHAPE"
from (SELECT null AS "SHAPE" FROM DUAL) --<-- your unmodifiable subquery
)
/
The problem is not the CASE expression but that in the sub-query you are using SELECT null FROM DUAL and the NULL does not have a data type so Oracle assumes that it is VARCHAR2 and then when it is passed to the outer query and you try to use ("SHAPE").SDO_POINT.X it will raise the exception ORA-22806: not an object or REF because SHAPE is a VARCHAR2 and is not an object or a REF so you cannot use the . syntax to extract member attributes from it.
What you can do is to use CAST in the outer query (and you do not need a CASE expression) to wrangle the NULL to a different data type:
SELECT (CAST(SHAPE AS SDO_GEOMETRY)).SDO_POINT.X AS x
FROM (SELECT null AS "SHAPE" FROM DUAL)
However, that will not work if the inner sub-query can also return a non-NULL object and for that you can use some trickery with UNION ALL and ROWNUM to ensure the sub-query has the correct data type:
SELECT (SHAPE).SDO_POINT.X
FROM (
<your inner query> UNION ALL
SELECT CAST(NULL AS SDO_GEOMETRY) FROM DUAL
) s
WHERE ROWNUM = 1;
Then:
SELECT (SHAPE).SDO_POINT.X
FROM (
SELECT null AS "SHAPE" FROM DUAL UNION ALL
SELECT CAST(NULL AS SDO_GEOMETRY) FROM DUAL
) s
WHERE ROWNUM = 1;
Outputs NULL and:
SELECT (SHAPE).SDO_POINT.X
FROM (
SELECT SDO_GEOMETRY( 2001, 4326, SDO_POINT_TYPE(12,14,NULL), NULL, NULL) AS "SHAPE" FROM DUAL UNION ALL
SELECT CAST(NULL AS SDO_GEOMETRY) FROM DUAL
) s
WHERE ROWNUM = 1;
Outputs 12.
db<>fiddle here

Why this sql will cause type conversion error?

WITH tb_testl AS (
SELECT 1 AS id ,'hehe' AS value
UNION ALL
SELECT 1 AS id, '1' AS value
UNION ALL
SELECT 2 AS id, '2' AS value
UNION ALL
SELECT 2 AS id, '2' AS value
), tb_test2 AS (
SELECT CONVERT(INT , value) AS value FROM tb_testl WHERE id = 2
)
SELECT * FROM tb_test2 WHERE value = 2;
this sql will cause error
Conversion failed when converting the varchar value 'hehe' to data
type int.
but the table tb_test2 dosen't have the value 'hehe' which is in the anthor table tb_test1. And I found that this sql will work well if I don't append the statement WHERE value = 2; .I've tried ISNUMBERIC function but it didn't work.
version:mssql2008 R2
With respect to the why this occurs:
There is a Logical Processing Order, which describes the order in which clauses are evaluated. The order is:
FROM
ON
JOIN
WHERE
GROUP BY
WITH CUBE or WITH ROLLUP
HAVING
SELECT
DISTINCT
ORDER BY
TOP
You can also see the processing order when you SET SHOWPLAN_ALL ON. For this query, the processing is as follows:
Constant scan - this is the FROM clause, which consists of hard coded values, hence the constants.
Filter - this is the WHERE clause. While it looks like there are two where clauses (WHERE id = 2 and WHERE value = 2). SQL Server sees this differently, it considers a single WHERE clause: WHERE CONVERT(INT , value) = 2 AND id = 2.
Compute scaler. This is the CONVERT function in the select.
Because both WHERE clauses are executed simultaneously, the hehe value is not filtered out of the CONVERT scope.
Effectively, the query is simplified to something like:
SELECT CONVERT(INT, tb_testl.value) AS Cvalue
FROM (
SELECT 1 AS id
, 'hehe' AS value
UNION ALL
SELECT 1 AS id
, '1' AS value
UNION ALL
SELECT 2 AS id
, '2' AS value
UNION ALL
SELECT 2 AS id
, '2' AS value
) tb_testl
WHERE CONVERT(INT, tb_testl.value) = 2
AND tb_testl.id = 2
Which should clarify why the error occurs.
With SQL, you cannot read code in the same way as imperative languages like C. Lines of SQL code are not necessarily (mostly not at all, in fact) executed in the same order it is written in. In this case, it's an error to think the inner where is executed before the outer where.
SQL Server does not guarantee the order of processing of statements (with one exception below). That is, there is no guarantee that WHERE filtering happens before the SELECT. Or that one CTE is evaluated before another. This is considered an advantage because it allows SQL Server to rearrange the processing to optimize performance (although I consider the issue that you are seeing a bug).
Obviously, the problem is in this part of the code:
tb_test2 AS (
SELECT CONVERT(INT, value) AS value
FROM tb_testl
WHERE id = 2
)
(Well, actually, it is where tb_test2 is referenced.)
What is happening is that SQL Server pushes the CONVERT() to where the values are being read, so the conversion is attempted before the WHERE clause is processed. Hence, the error.
In SQL Server 2012+, you can easily solve this using TRY_CNVERT():
tb_test2 AS (
SELECT TRY_CONVERT(INT, value) AS value
FROM tb_testl
WHERE id = 2
)
However, that doesn't work in SQL Server 2008. You can use the fact that CASE does have some guarantees on the order of processing:
tb_test2 AS (
SELECT (CASE WHEN value NOT LIKE '%[^0-9]%' THEN CONVERT(INT, value)
END) AS value
FROM tb_testl
WHERE id = 2
)
error caused by this part of statement
), tb_test2 AS (
SELECT CONVERT(INT , value) AS value FROM tb_testl WHERE id = 2
value has type of varchar and 'hehe' value cannot be converted to integer
WITH tb_testl AS (
SELECT 1 AS id ,'hehe' AS value
UPDATE: sql try convert all value(s) to integer in you statement. to avoid error rewrite statement as
WITH tb_testl AS (
SELECT 1 AS id ,'hehe' AS value
UNION ALL SELECT 1 AS id, '1' AS value
UNION ALL SELECT 2 AS id, '2' AS value
UNION ALL SELECT 2 AS id, '2' AS value
), tb_test2 AS (
SELECT value AS value FROM tb_testl WHERE id = 2
),
tb_test3 AS (
SELECT cast(value as int) AS value FROM tb_test2
)
SELECT * FROM tb_test3

Oracle SQL - group by char cast

Using the a char-cast in a group-by clause results something unexpected:
select cast(col as char(2)) from (
select 'Abc' as col from dual
union all
select 'Abc' as col from dual
) group by cast(col as char(10));
The result is 'Abc ' (10 characters long).
Intuitively, I would have expected Oracle to return one of the following:
An error: 'not a group-by expression', as the group-by clause is another than the selection clause
A result of length 2 'Ab'.
Replacing cast(col as char(2)) with cast(col as char(3)), Oracle returns an error 'not a group-by expression'. This, again is a very strange behavior.
How can this be explained? What's the reason behind it?
I'm using Oracle SQL 11g.
As was mentioned above, I think there is a misunderstanding going on. o.O
I can't explain why it's doing this, but here's the pattern for the type of query you have:
If you generalize it a bit like this, where [A] and [B] are integers, and [STRING] is whatever text you want:
select cast(col as char([A])) from (
select '[STRING]' as col from dual
union all
select '[STRING]' as col from dual
) group by cast(col as char([B]));
it looks like this always fails if one of the two conditions below is true (there may be others):
( LENGTH([STRING]) < [B] OR LENGTH([STRING] > [B]) and [A] = LENGTH([STRING])
( LENGTH([STRING]) = [B] AND [A] <> LENGTH([STRING]) )
Otherwise, it'll return a row.
But if you take your example that runs and use it in a CREATE TABLE statement, it's going to fail as it sets up the column width to be the 2 and can't fit the 3 character string coming in.
To add to the oddity, if you append something at the start and the end of the string like this:
select '\*'||cast(col as char([A]))||'\*' from (
select '[STRING]' as col from dual
union all
select '[STRING]' as col from dual
) group by cast(col as char([B]));
This will only work if [A] >= [B], otherwise it fails on ORA-01489: result of string concatenation is too long.
Curious...

T-SQL average rounded to the closest integer

I'm not sure if this has been asked before, but how do I get the average rounded to the closest integer in T-SQL?
This should do it. You might need a GROUP BY on the End depending on what you are looking for the average of.
SELECT CONVERT(int,ROUND(AVG(ColumnName),0))
FROM
TableName
EDIT: This question is more interesting than I first thought.
If we set up a dummy table like so...
WITH CTE
AS
(
SELECT 3 AS Rating
UNION SELECT 4
UNION SELECT 7
)
SELECT AVG(Rating)
FROM
CTE
We get an integer average of 4
However if we do this
WITH CTE
AS
(
SELECT 3.0 AS Rating
UNION SELECT 4.0
UNION SELECT 7.0
)
SELECT AVG(Rating)
FROM
CTE
We get a decimal average of 4.666..etc
So it looks like the way to go is
WITH CTE
AS
(
SELECT 3 AS Rating
UNION SELECT 4
UNION SELECT 7
)
SELECT CONVERT(int,ROUND(AVG(CONVERT(decimal,Rating)),0))
FROM CTE
Which will return an integer value of 5 which is what you are looking for.
If you are in SQL Server, just use round(avg(column * 1.0), 0).
The reason for * 1.0 is because sql server in some cases returns calculations using the same datatype of the values used in the calculation. So, if you calculate the average of 3, 4 and 4, the result is 3.66..., but the datatype of the result is integer, therefore the sql server will truncate 3.66... to 3, using * 1.0 implicit convert the input to a decimal.
Alternatively, you can convert or cast the values before the average calculation, like cast(column as decimal) instead of using the * 1.0 trick.
If your column it's not a integer column, you can remove the * 1.0.
PS: the result of round(avg(column * 1.0), 0) still is a decimal, you can explicit convert it using convert(int, round(avg(column * 1.0), 0), 0) or just let whatever language you are using do the job (it's a implicit conversion)
Select cast(AVG(columnname) as integer)
This worked for it:
CONVERT(int,ROUND(AVG(CAST(COLUMN-NAME AS DECIMAL)) ,0))
Isn't there a shorter way of doing it though?
T-SQL2018.
CAST(ROUND(COLUMN, 0) AS INT) This code does the job for me and gives the output I require so a 4.8 becomes 5.
whereas
CAST(AVG(COLUMN) AS INT) This code almost does the job but rounds down, so 4.8 becomes a 4 and not 5.
select cast(avg(a+.5) as int) from
(select 1 a union all select 2) b
If you don't like shortcuts, you could use the long way:
select round(avg(cast(a as real)), 0)
from (select 1 a union all select 2) b
The following statements are equivalent:
-- the original code
CONVERT(int, ROUND(AVG(CAST(mycolumn AS DECIMAL)) ,0))
-- using '1e0 * column' implicitly converts mycolumn value to float
CONVERT(int, ROUND(AVG(1e0 * mycolumn) ,0))
-- the conversion to INT already rounds the value
CONVERT(INT, AVG(1e0 * mycolumn))
On SQL 2014,
select round(94,-1)
select round(95,-1)