I am trying to achieve the following:
Get all Field values where the FieldValue is greater than 100 when the
value is stored as a number.
The indicator of whether or not if a value was stored as a number is given by the field type, which is another where clause.
The issue I am facing is that when I try to do the data field conversion in my WHERE statement, it failed.
I run:
SELECT FieldValue FROM CARData A
JOIN Fields B ON A.FieldId = B.FieldId
WHERE FieldTypeId = 3 AND FieldValue IS NOT NULL
And this returns the expected result of:
But if I added the WHERE clause to filter by the value:
SELECT FieldValue FROM CARData A
JOIN Fields B ON A.FieldId = B.FieldId
WHERE FieldTypeId = 3 AND FieldValue IS NOT NULL
AND CAST(FieldValue AS BIGINT) > 100
It throws the error:
Error converting data type nvarchar to bigint.
I somewhat understand what the issue is- it is trying to convert ALL values in the table to a bigint and is failing when it hits a non-numeric value.
I attempted to solve this by nesting the first query in a second like so:
SELECT RESULT.FieldValue FROM (
SELECT FieldValue FROM CARData A
JOIn Fields B ON A.FieldId = B.FieldId
WHERE
FieldTypeId = 3
AND FieldValue IS NOT NULL
AND ISNUMERIC(A.FieldValue) = 1) RESULT
WHERE CAST(FieldValue AS BIGINT) > 100
But even that does not return anything other than the aforementioned error.
While it's true that changing the structure of your query can cause SQL to pick a different plan, you should limit that technique to trying to help the optimizer pick a performant plan. The reason for this is that if successful execution of the logic depends on a particular choice of plan, then your query might work today but fail tomorrow (or in production) when SQL decides to pick a different plan.
Fortunately you don't have to rely on that here! Use try_cast
SELECT FieldValue FROM CARData A
JOIN Fields B ON A.FieldId = B.FieldId
WHERE FieldTypeId = 3
AND TRY_CAST(FieldValue AS BIGINT) > 100
I'm also curious... is this part of a SQL course? If so, tell your instructor to come visit StackOverflow so we can tell them to stop teaching students to use EAV's. Unless the whole point is to show you how horrible they are! :)
Related
It is Microsoft SQL Server.
In this column PostalCode from the AdventureWorks 2012 Person.Address table, there are numeric and string values.
I want to get table with rows WHERE PostalCode < 7000
This does not work as expected:
USE [AdventureWorks2012]
SELECT *
FROM Person.Address
WHERE ISNUMERIC(PostalCode) = 1
AND PostalCode < 7000
because I get this error:
Conversion failed when converting the nvarchar value 'K4B 1T7' to data type int.
I can make it, by creating temporally table like this:
/* creating of temp table */
USE AdventureWorks2012
SELECT *
INTO temp2
FROM Person.Address
WHERE ISNUMERIC(PostalCode) = 1
/* get data from temp table */
SELECT *
FROM temp2
WHERE PostalCode < 7000
But it is a bad way, cause of low productivity and needless temp-table.
What is the better way to get table with rows WHERE PostalCode < 7000 but data has not only numeric values?
If you're in SQL Server 2012 or newer you should use try_convert instead of isnumeric. Isnumeric has some funny issues that it returns 1 even for strings that can't be converted into a number. So something like this should work:
SELECT *
FROM Person.Address
WHERE try_convert(int, PostalCode) < 7000
If the string can't be converted, try_convert returns null.
MSDN: https://msdn.microsoft.com/en-us/library/hh230993.aspx
The error is being returned because the conditions being evaluated are not short-circuiting - the condition PostalCode<7000 is being evaluated even where the postal code is non-numeric.
Instead, try:
SELECT *
from Person.Address
WHERE CASE WHEN PostalCode NOT LIKE '%[^0-9]%'
THEN CAST(PostalCode AS NUMERIC)
ELSE CAST(NULL AS NUMERIC)
END <7000
(Updated following comments)
The text is from 70-461 Training kit
(Exam 70-461: Querying Microsoft SQL Server 2012):
Recall from Chapter 1 that all expressions that appear in the same
logical query processing phase—for example, the WHERE phase—are
conceptually evaluated at the same point in time. For example,
consider the following filter predicate.
WHERE propertytype = 'INT' AND CAST(propertyval AS INT) > 10
Suppose that the table being queried
holds different property values. The propertytype column represents
the type of the property (an INT, a DATE, and so on), and the
propertyval column holds the value in a character string. When
propertytype is 'INT', the value in propertyval is convertible to INT;
otherwise, not necessarily.
Some assume that unless precedence rules
dictate otherwise, predicates will be evaluated from left to right,
and that short circuiting will take place when possible. In other
words, if the first predicate propertytype = 'INT' evaluates to false,
SQL Server won’t evaluate the second predicate CAST(propertyval AS
INT) > 10 because the result is already known. Based on this
assumption, the expectation is that the query should never fail trying
to convert something that isn’t convertible.
The reality, though, is
different. SQL Server does internally support a short-circuit concept;
however, due to the all-at-once concept in the language, it is not
necessarily going to evaluate the expressions in left-to-right order.
It could decide, based on cost-related reasons, to start with the
second expression, and then if the second expression evaluates to
true, to evaluate the first expression as well. This means that if
there are rows in the table where propertytype is different than
'INT', and in those rows propertyval isn’t convertible to INT, the
query can fail due to a conversion error.
The only safe way of doing this is by getting first the Ids of the fields you are interested in and then join with them in different statements, otherwise the query planner could decide it want to do first the numeric comparison. I started to get a lot of this problems when we upgraded to SQL Server 2008 that didn't happened before.
You can however do a conversion:
USE [AdventureWorks2012]
SELECT *
from Person.Address
WHERE ISNUMERIC(PostalCode) =1 AND CAST(CAST(PostalCode AS INT) AS VARCHAR)<'7000'
I have done the castings to try to avoid any data that could be numeric but with 0 padding on the left that could screw up the ordering.
Beware that the performance of this won't be the best and Indexes on PostalCode aren't going to be used.
Here in Denmark, postal code always have the same length, so I would use this script to avoid strange issues* with isnumeric and conversion issues.
It will check that postalCode has 4 digits and compare the string value.
SELECT *
FROM temp2
WHERE
PostalCode < '7000' and
PostalCode like '[0-9][0-9][0-9][0-9]'
*An example of strange issues with isnumeric
SELECT isnumeric('£1.1')
SELECT isnumeric('-')
Both returns 1
You can use subquery to do that :
select * from (
SELECT * from Person.Address WHERE ISNUMERIC(PostalCode) =1 ) t
where PostalCode<7000
I encountered a strange behavior this morning and am not sure why is it happening?
My table and values :
create table tblA (acc varchar(10),accname varchar(10))
insert into tblA values('1','A')
insert into tblA values('2','B')
insert into tblA values('3','C')
insert into tblA values('Z','D')
insert into tblA values('4','E')
Query 1 to fetch acc with 1
select * from tblA where acc = 1
error : Conversion failed when converting the varchar value 'Z' to data type int
I understand the error very well.
Query 2 to fetch all numeric records
select * from tblA where ISNUMERIC(acc)<>0
gives me all records with proper numeric accounts. Excludes acc 'Z'
Query 3 to fetch acc 1 from all numeric accs
select * from (select * from tblA where ISNUMERIC(acc)<>0) A
where A.acc=1
error : Conversion failed when converting the varchar value 'Z' to data type int.
This is what I didn't understand, how come is it showing an error for value 'Z' that is not even present in table from subquery aliased as A? I know there are work arounds like converting or casting to varchar and all, I just want to know the reason behind this behavior
Your conversion in query 3 is a filtering predicate, meaning it is performing the check to return all rows where acc CAN be converted to a numeric. It is not, however, returning a result set with those values converted, as it would be here:
select *
from (select CONVERT(INT, acc) AS acc, accname
from tblA
where ISNUMERIC(acc)<>0) A
where A.acc=1
In other words, your query is still returning varchars, not integers. As a result, it may be that behind-the-scenes, the query engine is just doing the conversion on every value in the table because it will not simply follow the order of the query if it estimates it will be faster. Therefore, even though you logically should have ruled out the need to perform an integer conversion on non-integer values, the query optimizer is still doing it in the background because it thinks it will save time. This is just my speculation.
you are having datatype of Id acc as varchar and you are checking acc with integer type . just pass acc as varchar.
select * from (select * from tblA where ISNUMERIC(acc)<>0) A
where A.acc='1'
this will work fine.
here is the screenshot for the reference
Try the following nested case query
select *
from #tblA
where 1=
case when ISNUMERIC(acc)=1 then case when acc=1 then 1 end end
I have two tables Backup and Requests.
Below is the script for both the tables
Backup
CREATE TABLE UserBackup(
FileName varchar(70) NOT NULL,
)
File name is represented by a guid. Sometimes there is some additional information related to the file. Hence we have entries like guid_ADD entried in table.
Requests
CREATE TABLE Requests(
RequestId UNIQUEIDENTIFIER NOT NULL,
Status int Not null
)
Here are some sample rows :
UserBackup table:
FileName
15b993cc-e8be-405d-bb9f-0c58b66dcdfe
4cffe724-3f68-4710-b785-30afde5d52f8
4cffe724-3f68-4710-b785-30afde5d52f8_Add
7ad22838-ddee-4043-8d1f-6656d2953545
Requests table:
RequestId Status
15b993cc-e8be-405d-bb9f-0c58b66dcdfe 1
4cffe724-3f68-4710-b785-30afde5d52f8 1
7ad22838-ddee-4043-8d1f-6656d2953545 2
What I need is to return all the rows from userbackup table whose name (the guid) is matches RequestId in the Requests table and the status is 1. So here is the query I wrote
Select *
from UserBackup
inner join Requests on UserBackup.FileName = Requests.RequestId
where Requests.Status = 1
And this works fine. It returns me the following result
FileName RequestId Status
15b993cc-e8be-405d-bb9f-0c58b66dcdfe 15b993cc-e8be-405d-bb9f-0c58b66dcdfe 1
4cffe724-3f68-4710-b785-30afde5d52f8 4cffe724-3f68-4710-b785-30afde5d52f8 1
4cffe724-3f68-4710-b785-30afde5d52f8_Add 4cffe724-3f68-4710-b785-30afde5d52f8 1
This is exactly what I want. But what I don't understand is how it is working. If you notice the result is returning 4cffe724-3f68-4710-b785-30afde5d52f8_Add row as well. The inner join is on varchar and uniqueidentifier, and this join instead of working like "Equals to" comparison works like "contains" comparison. I want to know how this works so that I can be sure to use this code without any unexpected scenarios.
The values on both sides of a comparison have to be of the same data type. There's no such thing as, say, comparing a uniqueidentifier and a varchar.
uniqueidentifier has a higher precedence than varchar so the varchars will be converted to uniqueidentifiers before the comparison occurs.
Unfortunately, you get no error or warning if the string contains more characters than are needed:
select CONVERT(uniqueidentifier,'4cffe724-3f68-4710-b785-30afde5d52f8_Add')
Result:
4CFFE724-3F68-4710-B785-30AFDE5D52F8
If you want to force the comparison to occur between strings, you'll have to perform an explicit conversion:
Select *
from UserBackup
inner join Requests
on UserBackup.FileName = CONVERT(varchar(70),Requests.RequestId)
where Requests.Status = 1
When you compare two columns of different data types SQL Server will attempt to do implicit conversion on lower precedence.
The following comes from MSDN docs on uniqueidentifier
The following example demonstrates the truncation of data when the
value is too long for the data type being converted to. Because the
uniqueidentifier type is limited to 36 characters, the characters that
exceed that length are truncated.
DECLARE #ID nvarchar(max) = N'0E984725-C51C-4BF4-9960-E1C80E27ABA0wrong';
SELECT #ID, CONVERT(uniqueidentifier, #ID) AS TruncatedValue;
http://msdn.microsoft.com/en-us/library/ms187942.aspx
Documentation is clear that data is truncated
When ever you are unsure about your join operation you can verify Actual Execution Plan.
Here is test sample that you can run inside SSMS or SQL Sentry Plan Explorer
DECLARE #userbackup TABLE ( _FILENAME VARCHAR(70) )
INSERT INTO #userbackup
VALUES ( '15b993cc-e8be-405d-bb9f-0c58b66dcdfe' ),
( '4cffe724-3f68-4710-b785-30afde5d52f8' ),
( '4cffe724-3f68-4710-b785-30afde5d52f8_Add' )
, ( '7ad22838-ddee-4043-8d1f-6656d2953545' )
DECLARE #Requests TABLE
(
requestID UNIQUEIDENTIFIER
,_Status INT
)
INSERT INTO #Requests
VALUES ( '15b993cc-e8be-405d-bb9f-0c58b66dcdfe', 1 )
, ( '4cffe724-3f68-4710-b785-30afde5d52f8', 1 )
, ( '7ad22838-ddee-4043-8d1f-6656d2953545', 2 )
SELECT *
FROM #userbackup u
JOIN #Requests r
ON u.[_FILENAME] = r.requestID
WHERE r.[_Status] = 1
Instead of regular join operation SQL Server is doing HASH MATCH with EXPR 1006 in SSMS it is hard to see what is doing but if you open XML file you will find this
<ColumnReference Column="Expr1006" />
<ScalarOperator ScalarString="CONVERT_IMPLICIT(uniqueidentifier,#userbackup.[_FILENAME] as [u].[_FILENAME],0)">
When ever in doubt check execution plan and always make sure to match data types when comparing.
This is great blog Data Mismatch on WHERE Clause might Cause Serious Performance Problems from Microsoft engineer on exact problem.
What is happening here is the FileName is being converted from varchar to a UniqueIdentifier, and during that process it ignores anything after the first 36 characters.
You can see it in action here
Select convert(uniqueidentifier, UserBackup.FileName), FileName
from UserBackup
It works, but to reduce confusion for the next person to come along, you might want to store the RequestId associated with the UserBackup as a GUID in the UserBackup table and join on that.
At the very least put a comment in ;)
This may be simple, but I am no SQL whiz so I am getting lost. I understand that sql takes your query and executes it in a certain order, which I believe is why this query does not work:
select * from purchaseorders
where IsNumeric(purchase_order_number) = 1
and cast(purchase_order_number as int) >= 7
MOST of the purchar_order_number fields are numeric, but we introduce alphanumeric ones recently. The data I am trying to get is to see if '7' is greater than the highest numeric purchase_order_number.
The Numeric() function filters out the alphanumeric fields fine, but doing the subsequent cast comparison throws this error:
Conversion failed when converting the nvarchar value '124-4356AB' to data type int.
I am not asking what the error means, that is obvious. I am asking if there is a way to accomplish what I want in a single query, preferably in the where clause due to ORM constraints.
does this work for you?
select * from purchaseorders
where (case when IsNumeric(purchase_order_number) = 1
then cast(purchase_order_number as int)
else 0 end) >= 7
You can do a select with a subselect
select * from (
select * from purchaseorders
where IsNumeric(purchase_order_number) = 1) as correct_orders
where cast(purchase_order_number as int) >= 7
try this:
select * from purchaseorders
where try_cast(purchase_order_number as int) >= 7
have to check which column has numeric values only.
Currently, in a table every field is setted with nvarchar(max) Like tableName (field1 nvarchar(max),field2 nvarchar(max),field3 nvarchar(3)) and tableName has 25lac Rows.
But on manually Check Field2 Contain the numeric Values Only... How to Check With t-sql that in the Complete Column (Field2) has numeric Value or not/null value with Longest Length in the Column!
A puzzler from a coworker that I cannot figure out...
update btd.dbo.tblpayroll
set empname = ( select b.Legal_Name
from ( SELECT Legal_Name,
Employee_ID
FROM Com.dbo.Workers
WHERE isnumeric(Employee_ID) = 1
) b
where b.Employee_ID = empnum
and b.Legal_name is not NULL
)
where empname is NULL
Msg 245, Level 16, State 1, Line 1
Conversion failed when converting the varchar value 'N0007 ' to data type int. The table alias b would actually be a view.
The value 'N0007 ' is in the Workers table. I don't see why it is not being filtered from the results that are being joined.
EDIT:
The alias does, in fact, return the correct rows - so isNumeric is doing the job.
I suspect that the optimizer is attempting to apply the where clause of the outer select before the inner select. Presumably it thinks it would be able to do an index lookup on Employee_ID resulting in a faster query in this case. Try:
update btd.dbo.tblpayroll
set empname = ( select Legal_Name
from Com.dbo.Workers
where isnumeric(Employee_ID) = 1
and convert(varchar,Employee_ID)
= convert(varchar,empnum)
and Legal_name is not NULL)
where empname is NULL
Converting them all to varchar should take care of it. I don't think it's much less efficient than you wanted orginally since the isnumeric, if done first, would have forced a table scan anyway.
ISNUMERIC() is famously unreliable for what you are trying to do. You'll need an alternative, which I've been asking for here like this one.
The obvious thing is to force the order of comparison, perhaps by getting then name from a view with only numeric Employee_IDs, rather than the full Workers table.
Maybe N is considered currency symbol? You can try to replace IsNumeric with
LIKE REPLICATE('[0-9]',/*length of Employee_ID*/)
or just
LIKE '[0-9]%'
if letter cannot be in the middle