My server database default collation is Arabic_CI_AS. In my application, I am never setting collation anywhere. But when I run this simple SQL,
CREATE TABLE #TempProductInventory
(
PID int
,InvTypeValue nvarchar(MAX)
,InvTypeKey nvarchar(MAX)
);
SELECT *
FROM #TempProductInventory TP
INNER JOIN dbo.[Sources] S ON (S.Code = TP.InvTypeKey)
I am getting,
Cannot resolve the collation conflict between "SQL_Latin1_General_CP1_CI_AS" and "Arabic_CI_AS" in the equal to operation. When I un,
When I run,
SELECT col.name, col.collation_name FROM sys.columns col WHERE object_id = OBJECT_ID('Sources')
I will get,
name collation_name
ID NULL
Code Arabic_CI_AS
Try this, it will set the collation of temporary table to the one that you specify.
Most likely reason for this happening is as stated by GarethD that temp database collition is set not to Arabic_CI_AS.
CREATE TABLE #TempProductInventory( PID int ,InvTypeValue nvarchar(MAX) ,InvTypeKey nvarchar(MAX) Collate "Arabic_CI_AS")
Related
I'm getting 'Cannot resolve the collation conflict between "Latin1_General_CI_AS" and "SQL_Latin1_General_CP1_CI_AS" in the equal to operation.' error.
The error is in a SP, I'm not specifying the collation anywhere, neither in the create table and when I use
select t.name, c.*, c.collation_name
from sys.columns c
left join sys.tables t
on c.object_id = t.object_id
order by c.collation_name
It says there is no 'Latin1_General_CI_AS' fields, all the fields are using the default collation. I'm returning a dataset but can't see where the error is. There are no joins. I am populating a temporary table, again, not specifying the collation yet it says there's an error. Any suggestions. Using SQL Server 2014.
Temporary tables use the collation of tempdb by default except if used in the context of a contained database.
You can specify COLLATE DATABASE_DEFAULT to have it use the collation of the current database.
CREATE TABLE #T
(
SomeColumn VARCHAR(50) COLLATE DATABASE_DEFAULT
)
I have an INSERT statement which looks like this:
INSERT INTO CLIENT_TABLE
SELECT NAME, SURNAME, AGE FROM CONTACT_TABLE
My example above is a basic one, but is there a way to pass in a SELECT statement and then check the returned column values against what the actual field sizes are?
Checking LEN against every column isnt practical. I am looking for something that is automated.
My debugging in that kind of problem is..
I am removing columns in the SELECT one by one, if did not return error, then you know what column is the cause of truncation problem.. but here are some tips on debugging.
Option 1: Start first with the columns that hold more character.. like VARCHAR, for example in your case, i think the column NAME, SURNAME are the one causes an error since AGE column does not hold many characters because its integer. You should debug something like that.
Option 2: You can investigate the column in your final output. The final SELECT will return all columns and its values, then you can counter check if the values matches what you input on the UI etc.
Ex. See the Expected vs. Actual Output result on the image below
Expected:
Actual Output:
My example in option 2 shows that the truncated string is the SURNAME as you can see..
NOTE: You can only use the Option 2 if the query did not return execution error, meaning to say that the truncated string did not return an error BUT created an unexpected split string which we don't want.
IF the query return an error, your best choice is Option 1, which consume more time but worth it, because that is the best way to make sure you found the exact column that causes the truncation problem
Then if you already found the columns that causes the problem, you can now adjust the size of the column or another way is to limit the input of the user ?, you can put some validation to users to avoid truncation problem, but it is all up to you on how you want the program works depending on your requirement.
My answers/suggestion is base on my experience in that kind of situation.
Hope this answer will help you. :)
Check max length for each field, this way you can identify the fields that are over char limit specified in your table e.g CLIENT_TABLE.
SELECT Max(Len(NAME)) MaxNamePossible
, Max(Len(SURNAME)) MaxSurNamePossible
, Max(Len(AGE)) MaxAgePossible
FROM CONTACT_TABLE
Compare the result with Client_Table Design
Like if in Client_Table "Name" is of Type Varchar(50) and validation query( written above) return more than 50 chars than "Name" field is causing over flow.
There is a great answer by Aaron Bertrand to the question:
Retrieve column definition for stored procedure result set
If you used SQL Server 2012+ you could use sys.dm_exec_describe_first_result_set. Here is a nice article with examples. But, even in SQL Server 2008 it is possible to retrieve the types of columns of the query. Aaron's answer explains it in details.
In fact, in your case it is easier, since you have a SELECT statement that you can copy-paste, not something that is hidden in a stored procedure. I assume that your SELECT is a complex query returning columns from many tables. If it was just one table you could use sys.columns with that table directly.
So, create an empty #tmp1 table based on your complex SELECT:
SELECT TOP(0)
NAME, SURNAME, AGE
INTO #tmp1
FROM CONTACT_TABLE;
Create a second #tmp2 table based on the destination of your complex SELECT:
SELECT TOP(0)
NAME, SURNAME, AGE
INTO #tmp2
FROM CLIENT_TABLE;
Note, that we don't need any rows, only columns for metadata, so TOP(0) is handy.
Once those #tmp tables exist, we can query their metadata using sys.columns and compare it:
WITH
CTE1
AS
(
SELECT
c.name AS ColumnName
,t.name AS TypeName
,c.max_length
,c.[precision]
,c.scale
FROM
tempdb.sys.columns AS c
INNER JOIN tempdb.sys.types AS t ON
c.system_type_id = t.system_type_id
AND c.user_type_id = t.user_type_id
WHERE
c.[object_id] = OBJECT_ID('tempdb.dbo.#tmp1')
)
,CTE2
AS
(
SELECT
c.name AS ColumnName
,t.name AS TypeName
,c.max_length
,c.[precision]
,c.scale
FROM
tempdb.sys.columns AS c
INNER JOIN tempdb.sys.types AS t ON
c.system_type_id = t.system_type_id
AND c.user_type_id = t.user_type_id
WHERE
c.[object_id] = OBJECT_ID('tempdb.dbo.#tmp2')
)
SELECT *
FROM
CTE1
FULL JOIN CTE2 ON CTE1.ColumnName = CTE2.ColumnName
WHERE
CTE1.TypeName <> CTE2.TypeName
OR CTE1.max_length <> CTE2.max_length
OR CTE1.[precision] <> CTE2.[precision]
OR CTE1.scale <> CTE2.scale
;
Another possible way to compare:
WITH
... as above ...
SELECT * FROM CTE1
EXCEPT
SELECT * FROM CTE2
;
Finally
DROP TABLE #tmp1;
DROP TABLE #tmp2;
You can tweak the comparison to suit your needs.
A manual solution is very quick if you are using SQL Server Manager Studio (SSMS). First capture the table structure of your SELECT statement into a working table:
SELECT COL1, COL2, ... COL99 INTO dbo.zz_CONTACT_TABLE
FROM CONTACT_TABLE WHERE 1=0;
Then in SSMS, right-click your original destination table (CLIENT_TABLE) and script it as create to a new SSMS window. Then right-click your working table (zz_CONTACT_TABLE) and script the creation of this table to a second SSMS window. Arrange both windows side by side and check the columns of zz_CONTACT_TABLE against CLIENT_TABLE. Differences in length and out-of-order columns will be immediately seen, even if there are hundreds of output columns.
Finally drop your working table:
DROP TABLE dbo.zz_CONTACT_TABLE;
Regarding an automated solution, it is difficult to see how this could work. Basically you are comparing a destination table (or a subset of columns in a destination table) against the output of a SELECT statement. I suppose you could write a stored procedure that takes two varchar parameters: the name of the destination table and the SELECT statement that would populate it. But this would not handle the case where only some columns of the destination are populated, and it would be more work than the manual solution above.
Here is some code to compare two row producing SQL statements to compare the columns. It takes as parameters two row-sets specified with server name, database name, and T-SQL query. It can compare data in different databases and even on different SQL Servers.
--setup parameters
declare #Server1 as varchar(128)
declare #Database1 as varchar(128)
declare #Query1 as varchar(max)
declare #Server2 as varchar(128)
declare #Database2 as varchar(128)
declare #Query2 as varchar(max)
set #Server1 = '(local)'
set #Database1 = 'MyDatabase'
set #Query1 = 'select * from MyTable' --use a select
set #Server2 = '(local)'
set #Database2 = 'MyDatabase2'
set #Query2 = 'exec MyTestProcedure....' --or use a procedure
--calculate statement column differences
declare #SQLStatement1 as varchar(max)
declare #SQLStatement2 as varchar(max)
set #Server1 = replace(#Server1,'''','''''')
set #Database1 = replace(#Database1,'''','''''')
set #Query1 = replace(#Query1,'''','''''')
set #Server2 = replace(#Server2,'''','''''')
set #Database2 = replace(#Database2,'''','''''')
set #Query2 = replace(#Query2,'''','''''')
CREATE TABLE #Qry1Columns(
[colorder] [smallint] NULL,
[ColumnName] [sysname] COLLATE SQL_Latin1_General_CP1_CI_AS NULL,
[TypeName] [sysname] COLLATE SQL_Latin1_General_CP1_CI_AS NOT NULL,
[prec] [smallint] NULL,
[scale] [int] NULL,
[isnullable] [int] NULL,
[collation] [sysname] COLLATE SQL_Latin1_General_CP1_CI_AS NULL
) ON [PRIMARY]
CREATE TABLE #Qry2Columns(
[colorder] [smallint] NULL,
[ColumnName] [sysname] COLLATE SQL_Latin1_General_CP1_CI_AS NULL,
[TypeName] [sysname] COLLATE SQL_Latin1_General_CP1_CI_AS NOT NULL,
[prec] [smallint] NULL,
[scale] [int] NULL,
[isnullable] [int] NULL,
[collation] [sysname] COLLATE SQL_Latin1_General_CP1_CI_AS NULL
) ON [PRIMARY]
set #SQLStatement1 =
'SELECT *
INTO #Qry1
FROM OPENROWSET(''SQLNCLI'',
''server=' + #Server1 + ';database=' + #Database1 + ';trusted_connection=yes'',
''select top 0 * from (' + #Query1 + ') qry'')
select colorder, syscolumns.name ColumnName, systypes.name TypeName, syscolumns.prec, syscolumns.scale, syscolumns.isnullable, syscolumns.collation
from tempdb.dbo.syscolumns
join tempdb.dbo.systypes
on syscolumns.xtype = systypes.xtype
where id = OBJECT_ID(''tempdb.dbo.#Qry1'')
order by 1'
insert into #Qry1Columns
exec(#SQLStatement1)
set #SQLStatement2 =
'SELECT *
INTO #Qry1
FROM OPENROWSET(''SQLNCLI'',
''server=' + #Server2 + ';database=' + #Database2 + ';trusted_connection=yes'',
''select top 0 * from (' + #Query2 + ') qry'')
select colorder, syscolumns.name ColumnName, systypes.name TypeName, syscolumns.prec, syscolumns.scale, syscolumns.isnullable, syscolumns.collation
from tempdb.dbo.syscolumns
join tempdb.dbo.systypes
on syscolumns.xtype = systypes.xtype
where id = OBJECT_ID(''tempdb.dbo.#Qry1'')
order by 1'
insert into #Qry2Columns
exec(#SQLStatement2)
select ISNULL( #Qry1Columns.colorder, #Qry2Columns.colorder) ColumnNumber,
#Qry1Columns.ColumnName ColumnName1,
#Qry1Columns.TypeName TypeName1,
#Qry1Columns.prec prec1,
#Qry1Columns.scale scale1,
#Qry1Columns.isnullable isnullable1,
#Qry1Columns.collation collation1,
#Qry2Columns.ColumnName ColumnName2,
#Qry2Columns.TypeName TypeName2,
#Qry2Columns.prec prec2,
#Qry2Columns.scale scale2,
#Qry1Columns.isnullable isnullable2,
#Qry2Columns.collation collation2
from #Qry1Columns
join #Qry2Columns
on #Qry1Columns.colorder=#Qry2Columns.colorder
You can tweak the finally select statement to highlight any differences that you wish. You can also wrap this up in a procedure and make a nice little user interface for it if you like, so that it's literally a cut and paste away to quick results.
EDIT original question:
Our UDW is broken out into attribute and attribute list tables.
I would like to write a data dictionary query that dynamically pulls in all column values from all tables that are like %attr_list% without having to write a series of unions and update or add every time a new attribute list is created in our UDW.
All of our existing attribute list tables follow the same format (number of columns, most column names, etc). Below is the first two unions in our existing view which I want to avoid updating each time a new attribute list table is added to our UDW.
CREATE VIEW [dbo].[V_BI_DATA_DICTIONARY]
( ATTR_TABLE
,ATTR_LIST_ID
,ATTR_NAME
,ATTR_FORMAT
,SHORT_DESCR
,LONG_DESCR
,SOURCE_DATABASE
,SOURCE_TABLE
,SOURCE_COLUMN
,INSERT_DATETIME
,INSERT_OPRID
)
AS
SELECT 'PREAUTH_ATTR_LIST' ATTR_TABLE
,[PREAUTH_ATTR_LIST_ID] ATTR_LIST_ID
,[ATTR_NAME] ATTR_NAME
,[ATTR_FORMAT] ATTR_FORMAT
,[SHORT_DESCR] SHORT_DESCR
,[LONG_DESCR] LONG_DESCR
,[SOURCE_DATABASE] SOURCE_DATABASE
,[SOURCE_TABLE] SOURCE_TABLE
,[SOURCE_COLUMN] SOURCE_COLUMN
,[INSERT_DATETIME] INSERT_DATETIME
,[INSERT_OPRID] INSERT_OPRID
FROM [My_Server].[MY_DB].[dbo].[PREAUTH_ATTR_LIST]
UNION
SELECT 'SAVINGS_ACCOUNT_ATTR_LIST'
,[SAVINGS_ACCOUNT_ATTR_LIST_ID]
,[ATTR_NAME]
,[ATTR_FORMAT]
,[SHORT_DESCR]
,[LONG_DESCR]
,[SOURCE_DATABASE]
,[SOURCE_TABLE]
,[SOURCE_COLUMN]
,[INSERT_DATETIME]
,[INSERT_OPRID]
FROM [My_Server].[MY_DB].[dbo].[SAVINGS_ACCOUNT_ATTR_LIST]'
Something like this might work for you if all tables contain the same columns.
Just change the temp table and the selected columns to match your own columns.
CREATE TABLE #results (
ATTR_TABLE SYSNAME,
ATTR_LIST_ID INT,
ATTR_NAME NVARCHAR(50),
ATTR_FORMAT NVARCHAR(50),
SHORT_DESCR NVARCHAR(50),
LONG_DESCR NVARCHAR(255),
SOURCE_DATABASE NVARCHAR(50),
SOURCE_TABLE NVARCHAR(50),
SOURCE_COLUMN NVARCHAR(50),
INSERT_DATETIME DATETIME,
INSERT_OPRID INT
);
INSERT INTO #results
EXEC sp_MSforeachtable #command1 =
'
SELECT ''?''
, *
FROM ?
WHERE ''?'' LIKE ''%ATTR_LIST%''
'
SELECT *
FROM #results
DROP TABLE #results
EDIT: Updated my example with your columns. Because you use different column name for ATTR_LIST_ID in each table I changed the select to SELECT *. Obviously I don't know the data types of your columns so you have to change them.
This won't work in a view but you could create a stored procedure.
For SQL Server you should be able to use something like this:
SELECT c.name AS ColName, t.name AS TableName
FROM sys.columns c
JOIN sys.tables t ON c.object_id = t.object_id
WHERE t.name LIKE '%attr_list%'
And this will include views as well as tables
SELECT COLUMN_NAME, TABLE_NAME
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME LIKE '%attr_list%'
If using MS SQL Server check out the sys catalog views. You can use sys.tables and join to sys.columns to get your tables and columns. sys.extended_properties can get you description information, if entered.
I can't update temp table. This is my query
CREATE TABLE #temp_po(IndentID INT, OIndentDetailID INT, OD1 VARCHAR(50), OD2 VARCHAR(50),
OD3 VARCHAR(50), ORD VARCHAR(50), NIndentDetailID INT, ND1 VARCHAR(50), ND2 VARCHAR(50),
ND3 VARCHAR(50), NRD VARCHAR(50), Quantity DECIMAL(15,3))
INSERT INTO #temp_po(IndentID, OIndentDetailID, OD1, OD2, OD3, ORD)
SELECT ID.IndentID, ID.IndentDetailID, ID.D1, ID.D2, ID.D3, ID.RandomDimension
FROM STR_IndentDetail ID WHERE ID.IndentID = #IndentID
UPDATE
t
SET
t.ND1 = CASE WHEN D.D1 = '' THEN NULL ELSE D.D1 END,
t.ND2 = CASE WHEN D.D2 = '' THEN NULL ELSE D.D2 END,
t.ND3 = CASE WHEN D.D3 = '' THEN NULL ELSE D.D3 END,
t.NRD = CASE WHEN D.RandomDim = '' THEN NULL ELSE D.RandomDim END,
t.Quantity = D.PurchaseQty
FROM
#temp_po t INNER JOIN #detail D ON D.IndentDetailID = t.OIndentDetailID
WHERE
t.IndentID = #IndentID
But it gives the error
Cannot resolve the collation conflict between "Latin1_General_CI_AI" and "SQL_Latin1_General_CP1_CI_AS" in the equal to operation.
How to resolve this problem?
My tempdb collation is Latin1_General_CI_AI and my actual database collation is SQL_Latin1_General_CP1_CI_AS.
This happens because the collations on #tempdb.temp_po.OD1 and STR_IndentDetail.D1 are different (and specifically, note that #tempdb is a different, system database, which is generally why it will have a default opinion for collation, unlike your own databases and tables where you may have provided more specific opinions).
Since you have control over the creation of the temp table, the easiest way to solve this appears to be to create *char columns in the temp table with the same collation as your STR_IndentDetail table:
CREATE TABLE #temp_po(
IndentID INT,
OIndentDetailID INT,
OD1 VARCHAR(50) COLLATE SQL_Latin1_General_CP1_CI_AS,
.. Same for the other *char columns
In the situation where you don't have control over the table creation, when you join the columns, another way is to add explicit COLLATE statements in the DML where errors occur, either via COLLATE SQL_Latin1_General_CP1_CI_AS or easier, using COLLATE DATABASE_DEFAULT
SELECT * FROM #temp_po t INNER JOIN STR_IndentDetail s
ON t.OD1 = s.D1 COLLATE SQL_Latin1_General_CP1_CI_AS;
OR, easier
SELECT * FROM #temp_po t INNER JOIN STR_IndentDetail s
ON t.OD1 = s.D1 COLLATE DATABASE_DEFAULT;
SqlFiddle here
Changing the server collation is not a straight forward decision, there may be other databases on the server which may get impacted. Even changing the database collation is not always advisable for an existing populated database. I think using COLLATE DATABASE_DEFAULT when creating temp table is the safest and easiest option as it does not hard code any collation in your sql. For example:
CREATE TABLE #temp_table1
(
column_1 VARCHAR(2) COLLATE database_default
)
By default temp table take the collation of server. So instead updating all stored procedure with temp table change only server collation.
Check this link for Set or Change the Server Collation
This worked for me.
We ran into the same problem right now. Instead of adding the collation to the temp table creation (or to each temp table join), we just changed the temp table creation to a table variable declaration.
i am trying to create a View in a Database A, that is filled by a select from the Database B and i am having a collation conflict, to be more exactly , its between ( Latin1_General_CI_AS" and "Latin1_General_BIN ). WHere(in the code) i need to put the collate?
Best Regards.
The code is here:
CREATE VIEW [dbo].[CML_SDG_MENSAL_ESTOQUE]
AS
select
SUM(dw_fato_faturmes.val_fatur) val_fatur,
SUM(dw_fato_faturmes.val_receita) val_receita,
SUM(dw_fato_faturmes.qtd_bonif_item) qtd_bonif_item,
SUM(dw_fato_faturmes.val_bonif_fatur) val_bonif_fatur,
SUM(dw_fato_faturmes.val_bonif_receita) val_bonif_receita,
SUM(dw_fato_faturmes.val_devol_fatur) val_devol_fatur,
SUM(dw_fato_faturmes.val_devol_receita) val_devol_receita,
DW_DIM_PRODUTO.B1_CODDB B1_CODDB,
dw_fato_faturmes.cod_produto cod_produto,
SUM(dw_fato_faturmes.qtd_estoque) qtd_estoque,
SUM(dw_fato_faturmes.qtd_devol) qtd_devol,
SUM(dw_fato_faturmes.qtd_item) qtd_item,
SUM(dw_fato_faturmes.qtd_meta) qtd_meta,
SUM(dw_fato_faturmes.qtd_pedido) qtd_pedido,
SUM(dw_fato_faturmes.qtd_item)+
SUM(dw_fato_faturmes.qtd_bonif_item)+
SUM(dw_fato_faturmes.qtd_devol) venda_liquida
(SUM(dw_fato_faturmes.qtd_item)
+SUM(dw_fato_faturmes.qtd_bonif_item)
+SUM(dw_fato_faturmes.qtd_devol))
+SUM(dw_fato_faturmes.qtd_pedido) venda___pedido
FROM
logixbi.dbo.dw_fato_faturmes dw_fato_faturmes,
logixbi.dbo.DW_DIM_CLIENTE DW_DIM_CLIENTE,
DW_DIM_EMPRESA DW_DIM_EMPRESA,
logixbi.dbo.DW_DIM_MARCA DW_DIM_MARCA,
logixbi.dbo.DW_DIM_PRODUTO DW_DIM_PRODUTO,
logixbi.dbo.DW_DIM_REPRESENTANTE DW_DIM_REPRESENTANTE
where
DW_DIM_EMPRESA.SM0_FILIAL=dw_fato_faturmes.filial and
DW_DIM_MARCA.BM_GRUPO=dw_fato_faturmes.grupo and
DW_DIM_PRODUTO.B1_COD=dw_fato_faturmes.cod_produto and
DW_DIM_REPRESENTANTE.A3_COD=dw_fato_faturmes.vendedor and
DW_DIM_CLIENTE.A1_COD=dw_fato_faturmes.cliente and
DW_DIM_CLIENTE.A1_LOJA=dw_fato_faturmes.loja
group by DW_DIM_PRODUTO.B1_CODDB,dw_fato_faturmes.cod_produto
In order to find wich column has wich collation use this snippet:
SELECT name, collation_name
FROM sys.columns
WHERE OBJECT_ID IN (SELECT OBJECT_ID
FROM sys.objects
WHERE type = 'U'
AND name = 'your_table_name'
)
AND name = 'your_column_name'
Once you find the columns try this:
column_1 COLLATE your_collation = column_2 COLLATE your_collation
It is better to stick to a single collation globally. Otherwise you will have problems. Here is a snippet that will give you all the columns on your database with a COLLATION different than the one in the database
SELECT [TABLE_NAME] = OBJECT_NAME([id]),
[COLUMN_NAME] = [name],
[COLLATION_NAME] = collation
FROM syscolumns
WHERE collation <> 'your_database_collation_type'
AND collation IS NOT NULL
AND OBJECTPROPERTY([id], N'IsUserTable')=1
Where to put it depends on where the conflict is.
I'd suggest on the joins
ie
DW_DIM_EMPRESA.SM0_FILIAL COLLATE Latin1_General_CI_AS =dw_fato_faturmes.filial COLLATE Latin1_General_CI_AS
This is happening due to operation between different collation types so try this for statement for comparison.
ColumnA = ColumnB collate database_default
Try to use this in all your character matching conditions in where clause:
colnameA COLLATE Latin1_General_CI_AS = columnnameB COLLATE Latin1_General_CI_AS