Related
Take the following table as an instance:
CREATE TABLE TBL_Names(Name VARCHAR(32))
INSERT INTO TBL_Names
VALUES ('Ken'),('1965'),('Karen'),('2541')
sqlfiddle
Executing following query throws an exception:
SELECT [name]
FROM dbo.tblNames AS tn
WHERE [name] IN ( SELECT [name]
FROM dbo.tblNames
WHERE ISNUMERIC([name]) = 1 )
AND [name] = 2541
Msg 245, Level 16, State 1, Line 1 Conversion failed when converting
the varchar value 'Ken' to data type int.
While the following query executes without error:
SELECT [name]
FROM dbo.tblNames AS tn
WHERE ISNUMERIC([name]) = 1
AND [name] = 2541
I know that this is because of SQL Server Query Optimizer's decision. but I am wondering if there is any way to make sql server evaluate clauses in a certain order. this way, in the first query,the first clause filters out those Names that are not numeric so that the second clause will not fail at converting to a number.
Update: As you may noticed, the above query is just an instance to exemplify the problem. I know the risks of that implicit conversion and appreciate those who tried to warn me of that. However my main question is how to change Optimizer's behavior of evaluating clauses in a certain order.
There is no "direct" way of telling the engine to perform operations in order. SQL isn't an imperative language where you have complete control of how to do things, you simply tell what you need and the server decides how to do it itself.
For this particular case, as long as you have [name] = 2541, you are risking a potential conversion failure since you are comparing a VARCHAR column against an INT. Even if you use a subquery/CTE there is still room for the optimizer to evaluate this expression first and try to convert all varchar values to int (thus failing).
You can evade this with workarounds:
Correctly comparing matching data types:
[name] = '2541'
Casting [name] to INT beforehand and only whenever possible and on a different statement, do the comparison.
DECLARE #tblNamesInt TABLE (nameInt INT)
INSERT INTO #tblNamesInt (
nameInt)
SELECT
[nameInt] = CONVERT(INT, [name])
FROM
dbo.tblNames
WHERE
TRY_CAST([name] AS INT) IS NOT NULL -- TRY_CAST better than ISNUMERIC for INT
SELECT
*
FROM
#tblNamesInt AS T
WHERE
T.nameInt = 2351 -- data types match
Even an index hint won't force the optimizer to use an index (that's why it's called a hint), so we have little control on how it gets stuff done.
There are a few mechanics that we know are evaluated in order and we can use to our advantage, such as the HAVING expressions will always be computed after grouping values, and the grouping always after WHERE conditions. So we can "safely" do the following grouping:
DECLARE #Table TABLE (IntsAsVarchar VARCHAR(100))
INSERT INTO #Table (IntsAsVarchar)
VALUES
('1'),
('2'),
('20'),
('25'),
('30'),
('A') -- Not an INT!
SELECT
CASE WHEN T.IntsAsVarchar < 15 THEN 15 ELSE 30 END,
COUNT(*)
FROM
#Table AS T
WHERE
TRY_CAST(T.IntsAsVarchar AS INT) IS NOT NULL -- Will filter out non-INT values first
GROUP BY
CASE WHEN T.IntsAsVarchar < 15 THEN 15 ELSE 30 END
But you should always avoid writing code that implies implicit conversions (like T.IntsAsVarchar < 15).
Try like this
SELECT [name]
FROM #TBL_Names AS tn
WHERE [name] IN ( SELECT [name]
FROM #TBL_Names
WHERE ISNUMERIC([name]) = 1 )
AND [name] = '2541'
2)
AND [name] = convert(varchar,2541 )
Since You are storing name as varchar(32) varchar will accept integer datatype values also called precedence value
What about:
SELECT *
FROM dbo.tblNames AS tn
WHERE [name] = convert(varchar, 2541)
Why do you need ISNUMERIC([name]) = 1) since you only care about the value '2541'?
You can try this
SELECT [name]
FROM dbo.TBL_Names AS tn
WHERE [name] IN ( SELECT [name]
FROM dbo.TBL_Names
WHERE ISNUMERIC([name]) = 1 )
AND [name] = '2541'
You need to just [name] = 2541 to [name] = '2541'. You are missing ' (single quote) with name in where condition.
You can find the live demo Here.
Honestly, I wouldn't apply the implicit cast to your column [name], it'll make the query non-SARGable. Instead, convert the value of your input (or pass it as a string)
SELECT [name]
FROM dbo.TBL_Names tn
WHERE [name] = CONVERT(varchar(32),2541);
If you "must", however, wrap [name] (and suffer performance degradation) then use TRY_CONVERT:
SELECT [name]
FROM dbo.TBL_Names tn
WHERE TRY_CONVERT(int,[name]) = 2541;
I have an INSERT statement which looks like this:
INSERT INTO CLIENT_TABLE
SELECT NAME, SURNAME, AGE FROM CONTACT_TABLE
My example above is a basic one, but is there a way to pass in a SELECT statement and then check the returned column values against what the actual field sizes are?
Checking LEN against every column isnt practical. I am looking for something that is automated.
My debugging in that kind of problem is..
I am removing columns in the SELECT one by one, if did not return error, then you know what column is the cause of truncation problem.. but here are some tips on debugging.
Option 1: Start first with the columns that hold more character.. like VARCHAR, for example in your case, i think the column NAME, SURNAME are the one causes an error since AGE column does not hold many characters because its integer. You should debug something like that.
Option 2: You can investigate the column in your final output. The final SELECT will return all columns and its values, then you can counter check if the values matches what you input on the UI etc.
Ex. See the Expected vs. Actual Output result on the image below
Expected:
Actual Output:
My example in option 2 shows that the truncated string is the SURNAME as you can see..
NOTE: You can only use the Option 2 if the query did not return execution error, meaning to say that the truncated string did not return an error BUT created an unexpected split string which we don't want.
IF the query return an error, your best choice is Option 1, which consume more time but worth it, because that is the best way to make sure you found the exact column that causes the truncation problem
Then if you already found the columns that causes the problem, you can now adjust the size of the column or another way is to limit the input of the user ?, you can put some validation to users to avoid truncation problem, but it is all up to you on how you want the program works depending on your requirement.
My answers/suggestion is base on my experience in that kind of situation.
Hope this answer will help you. :)
Check max length for each field, this way you can identify the fields that are over char limit specified in your table e.g CLIENT_TABLE.
SELECT Max(Len(NAME)) MaxNamePossible
, Max(Len(SURNAME)) MaxSurNamePossible
, Max(Len(AGE)) MaxAgePossible
FROM CONTACT_TABLE
Compare the result with Client_Table Design
Like if in Client_Table "Name" is of Type Varchar(50) and validation query( written above) return more than 50 chars than "Name" field is causing over flow.
There is a great answer by Aaron Bertrand to the question:
Retrieve column definition for stored procedure result set
If you used SQL Server 2012+ you could use sys.dm_exec_describe_first_result_set. Here is a nice article with examples. But, even in SQL Server 2008 it is possible to retrieve the types of columns of the query. Aaron's answer explains it in details.
In fact, in your case it is easier, since you have a SELECT statement that you can copy-paste, not something that is hidden in a stored procedure. I assume that your SELECT is a complex query returning columns from many tables. If it was just one table you could use sys.columns with that table directly.
So, create an empty #tmp1 table based on your complex SELECT:
SELECT TOP(0)
NAME, SURNAME, AGE
INTO #tmp1
FROM CONTACT_TABLE;
Create a second #tmp2 table based on the destination of your complex SELECT:
SELECT TOP(0)
NAME, SURNAME, AGE
INTO #tmp2
FROM CLIENT_TABLE;
Note, that we don't need any rows, only columns for metadata, so TOP(0) is handy.
Once those #tmp tables exist, we can query their metadata using sys.columns and compare it:
WITH
CTE1
AS
(
SELECT
c.name AS ColumnName
,t.name AS TypeName
,c.max_length
,c.[precision]
,c.scale
FROM
tempdb.sys.columns AS c
INNER JOIN tempdb.sys.types AS t ON
c.system_type_id = t.system_type_id
AND c.user_type_id = t.user_type_id
WHERE
c.[object_id] = OBJECT_ID('tempdb.dbo.#tmp1')
)
,CTE2
AS
(
SELECT
c.name AS ColumnName
,t.name AS TypeName
,c.max_length
,c.[precision]
,c.scale
FROM
tempdb.sys.columns AS c
INNER JOIN tempdb.sys.types AS t ON
c.system_type_id = t.system_type_id
AND c.user_type_id = t.user_type_id
WHERE
c.[object_id] = OBJECT_ID('tempdb.dbo.#tmp2')
)
SELECT *
FROM
CTE1
FULL JOIN CTE2 ON CTE1.ColumnName = CTE2.ColumnName
WHERE
CTE1.TypeName <> CTE2.TypeName
OR CTE1.max_length <> CTE2.max_length
OR CTE1.[precision] <> CTE2.[precision]
OR CTE1.scale <> CTE2.scale
;
Another possible way to compare:
WITH
... as above ...
SELECT * FROM CTE1
EXCEPT
SELECT * FROM CTE2
;
Finally
DROP TABLE #tmp1;
DROP TABLE #tmp2;
You can tweak the comparison to suit your needs.
A manual solution is very quick if you are using SQL Server Manager Studio (SSMS). First capture the table structure of your SELECT statement into a working table:
SELECT COL1, COL2, ... COL99 INTO dbo.zz_CONTACT_TABLE
FROM CONTACT_TABLE WHERE 1=0;
Then in SSMS, right-click your original destination table (CLIENT_TABLE) and script it as create to a new SSMS window. Then right-click your working table (zz_CONTACT_TABLE) and script the creation of this table to a second SSMS window. Arrange both windows side by side and check the columns of zz_CONTACT_TABLE against CLIENT_TABLE. Differences in length and out-of-order columns will be immediately seen, even if there are hundreds of output columns.
Finally drop your working table:
DROP TABLE dbo.zz_CONTACT_TABLE;
Regarding an automated solution, it is difficult to see how this could work. Basically you are comparing a destination table (or a subset of columns in a destination table) against the output of a SELECT statement. I suppose you could write a stored procedure that takes two varchar parameters: the name of the destination table and the SELECT statement that would populate it. But this would not handle the case where only some columns of the destination are populated, and it would be more work than the manual solution above.
Here is some code to compare two row producing SQL statements to compare the columns. It takes as parameters two row-sets specified with server name, database name, and T-SQL query. It can compare data in different databases and even on different SQL Servers.
--setup parameters
declare #Server1 as varchar(128)
declare #Database1 as varchar(128)
declare #Query1 as varchar(max)
declare #Server2 as varchar(128)
declare #Database2 as varchar(128)
declare #Query2 as varchar(max)
set #Server1 = '(local)'
set #Database1 = 'MyDatabase'
set #Query1 = 'select * from MyTable' --use a select
set #Server2 = '(local)'
set #Database2 = 'MyDatabase2'
set #Query2 = 'exec MyTestProcedure....' --or use a procedure
--calculate statement column differences
declare #SQLStatement1 as varchar(max)
declare #SQLStatement2 as varchar(max)
set #Server1 = replace(#Server1,'''','''''')
set #Database1 = replace(#Database1,'''','''''')
set #Query1 = replace(#Query1,'''','''''')
set #Server2 = replace(#Server2,'''','''''')
set #Database2 = replace(#Database2,'''','''''')
set #Query2 = replace(#Query2,'''','''''')
CREATE TABLE #Qry1Columns(
[colorder] [smallint] NULL,
[ColumnName] [sysname] COLLATE SQL_Latin1_General_CP1_CI_AS NULL,
[TypeName] [sysname] COLLATE SQL_Latin1_General_CP1_CI_AS NOT NULL,
[prec] [smallint] NULL,
[scale] [int] NULL,
[isnullable] [int] NULL,
[collation] [sysname] COLLATE SQL_Latin1_General_CP1_CI_AS NULL
) ON [PRIMARY]
CREATE TABLE #Qry2Columns(
[colorder] [smallint] NULL,
[ColumnName] [sysname] COLLATE SQL_Latin1_General_CP1_CI_AS NULL,
[TypeName] [sysname] COLLATE SQL_Latin1_General_CP1_CI_AS NOT NULL,
[prec] [smallint] NULL,
[scale] [int] NULL,
[isnullable] [int] NULL,
[collation] [sysname] COLLATE SQL_Latin1_General_CP1_CI_AS NULL
) ON [PRIMARY]
set #SQLStatement1 =
'SELECT *
INTO #Qry1
FROM OPENROWSET(''SQLNCLI'',
''server=' + #Server1 + ';database=' + #Database1 + ';trusted_connection=yes'',
''select top 0 * from (' + #Query1 + ') qry'')
select colorder, syscolumns.name ColumnName, systypes.name TypeName, syscolumns.prec, syscolumns.scale, syscolumns.isnullable, syscolumns.collation
from tempdb.dbo.syscolumns
join tempdb.dbo.systypes
on syscolumns.xtype = systypes.xtype
where id = OBJECT_ID(''tempdb.dbo.#Qry1'')
order by 1'
insert into #Qry1Columns
exec(#SQLStatement1)
set #SQLStatement2 =
'SELECT *
INTO #Qry1
FROM OPENROWSET(''SQLNCLI'',
''server=' + #Server2 + ';database=' + #Database2 + ';trusted_connection=yes'',
''select top 0 * from (' + #Query2 + ') qry'')
select colorder, syscolumns.name ColumnName, systypes.name TypeName, syscolumns.prec, syscolumns.scale, syscolumns.isnullable, syscolumns.collation
from tempdb.dbo.syscolumns
join tempdb.dbo.systypes
on syscolumns.xtype = systypes.xtype
where id = OBJECT_ID(''tempdb.dbo.#Qry1'')
order by 1'
insert into #Qry2Columns
exec(#SQLStatement2)
select ISNULL( #Qry1Columns.colorder, #Qry2Columns.colorder) ColumnNumber,
#Qry1Columns.ColumnName ColumnName1,
#Qry1Columns.TypeName TypeName1,
#Qry1Columns.prec prec1,
#Qry1Columns.scale scale1,
#Qry1Columns.isnullable isnullable1,
#Qry1Columns.collation collation1,
#Qry2Columns.ColumnName ColumnName2,
#Qry2Columns.TypeName TypeName2,
#Qry2Columns.prec prec2,
#Qry2Columns.scale scale2,
#Qry1Columns.isnullable isnullable2,
#Qry2Columns.collation collation2
from #Qry1Columns
join #Qry2Columns
on #Qry1Columns.colorder=#Qry2Columns.colorder
You can tweak the finally select statement to highlight any differences that you wish. You can also wrap this up in a procedure and make a nice little user interface for it if you like, so that it's literally a cut and paste away to quick results.
I followed this post How do I perform an accent insensitive compare (e with è, é, ê and ë) in SQL Server? but it doesn't help me with " ş ", " ţ " characters.
This doesn't return anything if the city name is " iaşi " :
SELECT *
FROM City
WHERE Name COLLATE Latin1_general_CI_AI LIKE '%iasi%' COLLATE Latin1_general_CI_AI
This also doesn't return anything if the city name is " iaşi " (notice the foreign ş in the LIKE pattern):
SELECT *
FROM City
WHERE Name COLLATE Latin1_general_CI_AI LIKE '%iaşi%' COLLATE Latin1_general_CI_AI
I'm using SQL Server Management Studio 2012.
My database and column collation is "Latin1_General_CI_AI", column type is nvarchar.
How can I make it work?
The characters you've specified aren't part of the Latin1 codepage, so they can't ever be compared in any other way than ordinal in Latin1_General_CI_AI. In fact, I assume that they don't really work at all in the given collation.
If you're only using one collation, simply use the correct collation (for example, if your data is turkish, use Turkish_CI_AI). If your data is from many different languages, you have to use unicode, and the proper collation.
However, there's an additional issue. In languages like Romanian or Turkish, ş is not an accented s, but rather a completely separate character - see http://collation-charts.org/mssql/mssql.0418.1250.Romanian_CI_AI.html. Contrast with eg. š which is an accented form of s.
If you really need ş to equal s, you have replace the original character manually.
Also, when you're using unicode columns (nvarchar and the bunch), make sure you're also using unicode literals, ie. use N'%iasi%' rather than '%iasi%'.
In SQL Server 2008 collations versioned 100 were introduced.
Collation Latin1_General_100_CI_AI seems to do what you want.
The following should work:
SELECT * FROM City WHERE Name LIKE '%iasi%' COLLATE Latin1_General_100_CI_AI
Not tidiest solution I guess, but if you know that it's just the "ş" and "ţ" characters that are the problem, would it be acceptable to do a replace?
SELECT *
FROM City
WHERE replace(replace(Name,'ş','s'),'ţ','t') LIKE COLLATE Latin1_general_CI_AI '%iasi%' COLLATE Latin1_general_CI_AI
You just need to change collation of name field before like operation. Check test code below
DECLARE #city TABLE ( NAME NVARCHAR(20) )
INSERT INTO #city
VALUES ( N'iaşi' )
SELECT *
FROM #city
WHERE name LIKE 'iasi'
--No return
SELECT *
FROM #city
WHERE name COLLATE Latin1_general_CI_AI LIKE '%iasi%'
--Return 1 row
This problem was haunting me for some time, until now, when I've finally figured it out.
Presuming your table or column is of SQL_Latin1_General_CP1_CI_AS collation, if you do:
update
set myCol = replace(myCol , N'ș', N's')
from MyTable
and
update
set myCol = replace(myCol,N'ț',N't')
from MyTable
the replace function will not find these characters, because the "ș" made from your keyboard (Romanian Standard keyboard) differs from the "ş" or "ţ" found in your database.
As a comparison: ţț and şș - you can see that they differ because the accents are closer to the "s" or "t" character.
Instead, you must do:
update
set myCol = replace(myCol , N'ş', N's')
from MyTable
and
update
set myCol = replace(myCol,N'ţ',N't')
from MyTable
i am trying to create a View in a Database A, that is filled by a select from the Database B and i am having a collation conflict, to be more exactly , its between ( Latin1_General_CI_AS" and "Latin1_General_BIN ). WHere(in the code) i need to put the collate?
Best Regards.
The code is here:
CREATE VIEW [dbo].[CML_SDG_MENSAL_ESTOQUE]
AS
select
SUM(dw_fato_faturmes.val_fatur) val_fatur,
SUM(dw_fato_faturmes.val_receita) val_receita,
SUM(dw_fato_faturmes.qtd_bonif_item) qtd_bonif_item,
SUM(dw_fato_faturmes.val_bonif_fatur) val_bonif_fatur,
SUM(dw_fato_faturmes.val_bonif_receita) val_bonif_receita,
SUM(dw_fato_faturmes.val_devol_fatur) val_devol_fatur,
SUM(dw_fato_faturmes.val_devol_receita) val_devol_receita,
DW_DIM_PRODUTO.B1_CODDB B1_CODDB,
dw_fato_faturmes.cod_produto cod_produto,
SUM(dw_fato_faturmes.qtd_estoque) qtd_estoque,
SUM(dw_fato_faturmes.qtd_devol) qtd_devol,
SUM(dw_fato_faturmes.qtd_item) qtd_item,
SUM(dw_fato_faturmes.qtd_meta) qtd_meta,
SUM(dw_fato_faturmes.qtd_pedido) qtd_pedido,
SUM(dw_fato_faturmes.qtd_item)+
SUM(dw_fato_faturmes.qtd_bonif_item)+
SUM(dw_fato_faturmes.qtd_devol) venda_liquida
(SUM(dw_fato_faturmes.qtd_item)
+SUM(dw_fato_faturmes.qtd_bonif_item)
+SUM(dw_fato_faturmes.qtd_devol))
+SUM(dw_fato_faturmes.qtd_pedido) venda___pedido
FROM
logixbi.dbo.dw_fato_faturmes dw_fato_faturmes,
logixbi.dbo.DW_DIM_CLIENTE DW_DIM_CLIENTE,
DW_DIM_EMPRESA DW_DIM_EMPRESA,
logixbi.dbo.DW_DIM_MARCA DW_DIM_MARCA,
logixbi.dbo.DW_DIM_PRODUTO DW_DIM_PRODUTO,
logixbi.dbo.DW_DIM_REPRESENTANTE DW_DIM_REPRESENTANTE
where
DW_DIM_EMPRESA.SM0_FILIAL=dw_fato_faturmes.filial and
DW_DIM_MARCA.BM_GRUPO=dw_fato_faturmes.grupo and
DW_DIM_PRODUTO.B1_COD=dw_fato_faturmes.cod_produto and
DW_DIM_REPRESENTANTE.A3_COD=dw_fato_faturmes.vendedor and
DW_DIM_CLIENTE.A1_COD=dw_fato_faturmes.cliente and
DW_DIM_CLIENTE.A1_LOJA=dw_fato_faturmes.loja
group by DW_DIM_PRODUTO.B1_CODDB,dw_fato_faturmes.cod_produto
In order to find wich column has wich collation use this snippet:
SELECT name, collation_name
FROM sys.columns
WHERE OBJECT_ID IN (SELECT OBJECT_ID
FROM sys.objects
WHERE type = 'U'
AND name = 'your_table_name'
)
AND name = 'your_column_name'
Once you find the columns try this:
column_1 COLLATE your_collation = column_2 COLLATE your_collation
It is better to stick to a single collation globally. Otherwise you will have problems. Here is a snippet that will give you all the columns on your database with a COLLATION different than the one in the database
SELECT [TABLE_NAME] = OBJECT_NAME([id]),
[COLUMN_NAME] = [name],
[COLLATION_NAME] = collation
FROM syscolumns
WHERE collation <> 'your_database_collation_type'
AND collation IS NOT NULL
AND OBJECTPROPERTY([id], N'IsUserTable')=1
Where to put it depends on where the conflict is.
I'd suggest on the joins
ie
DW_DIM_EMPRESA.SM0_FILIAL COLLATE Latin1_General_CI_AS =dw_fato_faturmes.filial COLLATE Latin1_General_CI_AS
This is happening due to operation between different collation types so try this for statement for comparison.
ColumnA = ColumnB collate database_default
Try to use this in all your character matching conditions in where clause:
colnameA COLLATE Latin1_General_CI_AS = columnnameB COLLATE Latin1_General_CI_AS
I have a series of tables that each have a dedicated column to a foreign language. Languages vary from Japanese, Thai, English, Italian, French, more than 20 in all.
All of these tables are set up with Latin Case Insensitive collation. DB works fine.
But now I am trying to query against the specific foreign language column of each table. Lets take Japanese for starters. I'd like a foreign language user to enter foreign text and find the record based on the foreign language column.
DECLARE #myVar nvarchar(max);
SET #myVar = 'ｴﾝｼﾞﾝ ｽﾄｯﾌﾟ ﾘﾚｰ' = 'Engine Stop Relay' in english
Select *
FROM tableJapanese
WHERE langString = #myVar;
I have tried a multitude of collation combinations. I even copied the table and changed the collation of the column to Japanese_CI_AI and tried to query it that way.
None of these WHERE clauses work on either table/columm collation, when the column was Latin or Japanese...
WHERE lang_String collate Japanese_CI_AI = #myVar;
WHERE lang_String = #myVar collate Japanese_CI_AI;
WHERE lang_String collate Japanese_CI_AI = #myVar collate Japanese_CI_AI;
WHERE lang_String collate Japanese_CI_AI = #myVar;
WHERE lang_String = #myVar collate Japanese_CI_AI;
WHERE lang_String collate Japanese_CI_AI = #myVar collate Japanese_CI_AI;
I would like to leave the columns/database as Latin collation and code the queries for each language if possible.
This seems like one of those problems that if were a snake I'd been bitten already. Can anyone see what I am missing?
MSSQL Express 2008 R2
SOLUTION:
Add N in front of the field, it indicates unicode to SQL...
Select *
FROM tblLangJAP_test
WHERE lang_String = N'ｴﾝｼﾞﾝ ｽﾄｯﾌﾟ ﾘﾚｰ';
Works flawlessly.
Thanks,
Add N in front of the field, it indicates unicode to SQL...
Select *
FROM tblLangJAP_test
WHERE lang_String = N'ｴﾝｼﾞﾝ ｽﾄｯﾌﾟ ﾘﾚｰ';
Works flawlessly.