Im pivoting sys.[views] into key value pairs to compare with values on another server for consistency testing. Im running into an issue which returns the error.
Msg 8167, Level 16, State 1, Line 51
The type of column "type" conflicts with the type of other columns specified in the UNPIVOT list.
Query:
SELECT
sourceUnpivoted.idServer,
sourceUnpivoted.sourceServerName,
sourceUnpivoted.name,
sourceUnpivoted.columnName,
sourceUnpivoted.columnValue
FROM (
SELECT
CAST('1' AS VARCHAR(255)) AS idServer,
CAST('thisOne' AS VARCHAR(255)) AS sourceServerName,
CAST('theDatabase' AS VARCHAR(255)) AS sourceDatabaseName,
CAST(name AS VARCHAR(255)) AS name,
CAST(object_id AS VARCHAR(255)) AS object_id,
CAST(principal_id AS VARCHAR(255)) AS principal_id,
CAST(schema_id AS VARCHAR(255)) AS schema_id,
CAST(parent_object_id AS VARCHAR(255)) AS parent_object_id,
CAST(type AS VARCHAR(255)) AS type,
CAST(type_desc AS VARCHAR(255)) AS type_desc,
CAST(create_date AS VARCHAR(255)) AS create_date,
CAST(lock_escalation_desc AS VARCHAR(255)) AS lock_escalation_desc
...
FROM noc_test.dbo.stage_sysTables
) AS databaseTables
UNPIVOT (
columnValue FOR columnName IN (
object_id,
principal_id,
schema_id,
parent_object_id,
type,
type_desc,
create_date,
lock_escalation_desc
)
) AS sourceUnpivoted
Why does this not like [type],[type_desc],[lock_escalation_desc] ???
Ive also tried CONVERT(VARCHAR(255),type) AS type
It's actually a collation issue. I can resolve it by changing these lines:
CAST([type] collate database_default AS VARCHAR(255)) AS [type],
CAST(type_desc collate database_default AS VARCHAR(255)) AS type_desc,
CAST(create_date AS VARCHAR(255)) AS create_date,
CAST(lock_escalation_desc collate database_default AS VARCHAR(255)) AS lock_escalation_desc
The specific issue is that name is collated as Latin1_General_CI_AS, whereas the other 3 columns you mentioned are collated as Latin1_General_CI_AS_KS_WS (At least, on my machine, I'm not sure what it would be like on a server/database with different default collation).
This is one of the solution for this type error
1: create the this table
CREATE TABLE People
(
PersonId int,
Firstname varchar(50),
Lastname varchar(25)
)
2: Then insert
INSERT INTO People VALUES (1, 'Jim', 'Smith');
INSERT INTO People VALUES (2, 'Jane', 'Jones');
INSERT INTO People VALUES (3, 'Bob', 'Unicorn');
3: run this script you get the error
Msg 8167, Level 16, State 1, Line 3
The type of column "Lastname" conflicts with the type of other columns specified in the UNPIVOT list.
SELECT PersonId, ColumnName, Value
FROM People
unpivot(Value FOR ColumnName IN (FirstName, LastName)) unpiv;
4: the solution is you must use a subquery to first cast the Lastname column to have the same length as Firstname
SELECT PersonId, ColumnName, Value
FROM (
SELECT personid, firstname, cast(lastname AS VARCHAR(50)) lastname
FROM People
) d
unpivot(Value FOR ColumnName IN (FirstName, LastName)) unpiv;
Ran into this same error and I just made all the columns in the table of the same data type - I had a mix of int, varchar, nvarchar of various lengths. Once I converted all the columns in my table to the same type - nvarchar(255) it worked perfectly.
The PIVOT/UNPIVOT clause is sensitive to the ANSI Padding Status of the column (right-click -> properties in SSMS) as well as the type, size and collation. Try specifying SET ANSI_PADDING ON|OFF in the session before adding or recreating the column in question so it matches the others in the PIVOT/UNPIVOT clause.
I had the same issue. Fixed it by right clicking on the column header and selecting change type "using locale". See attached screen shot
1:
Related
i am trying to pivot all values in different type in MSSQL 2016. I could not find a way how i can pivot different data types..
The first table are initial form / structure. The second table is the desired shape.
I was trying the following SQL code to pivot my values
SELECT
[id] AS [id],
FIRSTNAME,
LASTNAME,
BIRTHDATE,
ADDRESS,
FLAG,
NUMBER
FROM (
SELECT
[cm].[key] AS [id],
[cm].[column] AS [column],
[cm].[string] AS [string],
[cm].[bit] AS [bit],
[cm].[xml] AS [xml],
[cm].[number] AS [number],
[cm].[date] AS [date]
FROM [cmaster] AS [cm]
) AS [t]
PIVOT (
MAX([string]) --!?!?
FOR [column] IN (
FIRSTNAME,
LASTNAME,
BIRTHDATE,
ADDRESS,
FLAG,
NUMBER
)
) AS [p]
I think your best bet is to use conditional aggregation, e.g.
SELECT cm.id,
FIRSTNAME = MAX(CASE WHEN cm.[property] = 'firstname' THEN cm.[string] END),
LASTNAME = MAX(CASE WHEN cm.[property] = 'lastname' THEN cm.[string] END),
BIRTHDATE = MAX(CASE WHEN cm.[property] = 'birthddate' THEN cm.[date] END),
FLAG = CONVERT(BIT, MAX(CASE WHEN cm.[bit] = 'flag' THEN CONVERT(TINYINT, cm.[boolean]) END)),
NUMBER = MAX(CASE WHEN cm.[property] = 'number' THEN cm.[integer] END)
FROM cmaster AS cm
GROUP BY cm.id;
Although, as you can see, your query becomes very tightly coupled to your EAV model, and why EAV is considered an SQL antipattern. Your alternative is to create a single column in your subquery and pivot on that, but you have to convert to a single data type, and lose a bit of type safety:
SELECT id, FIRSTNAME, LASTNAME, BIRTHDATE, ADDRESS, FLAG, NUMBER
FROM (
SELECT id = cm.[key],
[column] = cm.[column],
Value = CASE cm.type
WHEN 'NVARCHAR' THEN cm.string
WHEN 'DATETIME' THEN CONVERT(NVARCHAR(MAX), cm.date, 112)
WHEN 'XML' THEN CONVERT(NVARCHAR(MAX), cm.xml)
WHEN 'BIT' THEN CONVERT(NVARCHAR(MAX), cm.boolean)
WHEN 'INT' THEN CONVERT(NVARCHAR(MAX), cm.integer)
END
FROM cmaster AS cm
) AS t
PIVOT
(
MAX(Value)
FOR [column] IN (FIRSTNAME, LASTNAME, BIRTHDATE, ADDRESS, FLAG, NUMBER)
) AS p;
In order to make the result as per your request, first thing is we need to bring the data in to one format which is compatible with all data types. VARCHAR is ideal for that. Then prepare the base table using a simple select query, then PIVOT the result.
In the last projection, if you want, you can convert the data back in to the original format.
This query can be written dynamically as well to obtain the result as records are added. Here I provide the static answer according to your data. If you need a more generic dynamic answer, let me know. So I can post here.
--data insert scripts I used:
CREATE TABLE First_Table
(
[id] int,
[column] VARCHAR(10),
[string] VARCHAR(20),
[bit] BIT,
[xml] [xml],
[number] INT,
[date] DATE
)
SELECT GETDATE()
INSERT INTO First_Table VALUES(1, 'FIRST NAME', 'JOHN' , NULL, NULL, NULL, NULL)
INSERT INTO First_Table VALUES(1, 'LAST NAME', 'DOE' , NULL, NULL, NULL, NULL)
INSERT INTO First_Table VALUES(1, 'BIRTH DATE', NULL , NULL, NULL, NULL, '1985-02-25')
INSERT INTO First_Table VALUES(1, 'ADDRESS', NULL , NULL, 'SDFJDGJOKGDGKPDGKPDKGPDKGGKGKG', NULL, NULL)
INSERT INTO First_Table VALUES(1, 'FLAG', NULL , 1, NULL, NULL, NULL)
INSERT INTO First_Table VALUES(1, 'NUMBER', NULL , NULL, NULL, 20, NULL)
SELECT
PIVOTED.* FROM
(
--MAKING THE BASE TABLE FOR PIVOT
SELECT
[id]
,[column] AS [COLUMN]
, CASE WHEN [column] = 'FIRST NAME' then [string]
WHEN [column] = 'LAST NAME' then [string]
WHEN [column] = 'BIRTH DATE' then CAST([date] AS VARCHAR(100))
WHEN [column] = 'ADDRESS' then CAst([xml] as VARCHAR(100))
WHEN [column] = 'FLAG' then CAST([bit] AS VARCHAR(100))
else CAST([number] AS VARCHAR(100)) END AS [VALUE]
FROM First_Table
) AS [P]
PIVOT
(
MIN ([P].[VALUE])
FOR [column] in ([FIRST NAME],[LAST NAME],[BIRTH DATE],[ADDRESS],[FLAG],[NUMBER])
) AS PIVOTED
RESULT:
SQL：
SELECT
ID,
FIRSTNAME,
...,
FLAG = CAST (FLAG AS INT),
...
FROM
(
SELECT
*
FROM
(
SELECT
f.ID,
f.PROPERTY,
f.STRING + f."INTEGER" + f.DATETIME + f.BOLLEAN + f.XML AS COLS
FROM
FIRSTTBL f)
PIVOT(
min(COLS) FOR PROPERTY IN
(
'firstname' AS firstname,
'lastname' AS lastname,
'birthdate' AS birthdate,
'address' AS address,
'flag' AS flag,
'number' AS "NUMBER"
)
)
)
According to the original table, there is one and only one non-null value among STRING, INTEGER, DATETIME, BOLLEAN and XML columns for any row, so we just need to get the first non-null value and assign it to the corresponding new column. It is not difficult to perform the transposition using PIVOT function, except that we need to handle different data types according to the SQL rule, which requires that each column have a consistent type. For this task, first we need to convert the combined column values into a string, perform row-to-column transposition, and then convert string back to the proper types. When there are a lot of columns, the SQL statement can be tricky, and dynamic requirements are even hard to achieve.
Yet it is easy to write the code using the open-source esProc SPL:
A
1
=connect("MSSQL")
2
=A1.query#x("SELECT * FROM FIRSTTBL")
3
=A2.pivot(ID;PROPERTY,~.array().m(4:).ifn();"firstname":"FIRSTNAME", "lastname":"LASTANME","birthdate":"BIRTHDAY","address":"ADDRESS","flag":"FLAG","number":"NUMBER")
SPL does not require that data in the same column have consistent type. It is easy for it to maintain the original data types while performing the transposition.
Question updated.
What I want to achive is to get list of new tables which are empty or null in description field. (new tables means with prefix new_) and all tables have description field.
Table definition:
create table topic (id int, description varchar(255));
create table comment (id int, description varchar(255));
create table author (id int, description varchar(255));
create table new_topic (id int, description varchar(255));
create table new_comment (id int, description varchar(255));
create table new_author (id int, description varchar(255));
Sample data and description:
insert into new_topic (id, description) values (1, null);
insert into new_topic (id, description) values (2, 'This is topic description');
insert into new_comment (id, description) values (1, null);
insert into new_comment (id, description) values (2, null);
insert into new_author (id, description) values (1, 'This is casual first author.');
insert into new_author (id, description) values (2, 'This is casual second author.');
Like you can notice on my example ideal output for my sample data would've be:
table_name:
new_topic
new_comment
My actual solution works, but I need to manually add tables and I make a lot of repetitions.
select distinct 'new_topic' as table_name
from new_topic where description is null
select distinct 'new_comment' as table_name
from new_comment where description is null
select distinct 'new_author' as table_name
from new_author where description is null
And output of my solution is like below:
table_name
new_topic
table_name
new_comment
table_name
I also created SELECT to get all new tables:
SELECT TABLE_NAME FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_NAME LIKE 'new_%' AND TABLE_TYPE = 'BASE TABLE'
Which could've be an entry point for my previous select, but I don't know how to connect those two.
Also my solution is avaiable on dbfiddle
Oh I think I understand what you are after. Yes this requires dynamic sql. Also, please note that your query to find all tables with a name like new_ is not quite right. The underscore is a wildcard pattern check. So that would return a table named "news" when you don't want it to. Wrap the underscore in square brackets to solve this. Here is how I would go about this type of query. The comments in the code should explain this.
declare #SQL nvarchar(max) = '' --this must be initialized to an empty string for this to work.
select #SQL = #SQL + 'select distinct TableName = ''' + t.name + ''' from ' + quotename(t.name) + ' where description is null union all '
from sys.tables t
where name like 'new[_]%' --need the square brackets because the underscore is a wildcard so you might get false positives
select #SQL = left(#SQL, len(#SQL) - 10)
--this will show you the dynamic sql
select #SQL
--once you are satisfied the dynamic sql is correct uncomment the next line to execute it
--exec sp_executesql #SQL
Could you not just do:-
select table_name from information_schema.columns
where table_name like 'prefix_%' and (column_name is null or column_name='')
How do I extract values from a string? I'm trying to separate into 3 new columns. A separate column for city, state and zipcode.
I've tried
select address2,
left(address2, charindex('',address2)-1)
from table
and ---when I try the below code I get "Invalid length parameter passed to the left or substring function"
,LTRIM(substring(a.Address2, CHARINDEX(' ', a.Address2)+1, CHARINDEX(' ', substring(a.address2, charindex(' ',
a.address2)+1, len(a.address2)))-1))
I can break out the city (except for West Warwick) using the following code, but not sure how to make it work for state and zip. This also removes the error.
SUBSTRING(Address2,1,CHARINDEX(' ', a.address2+ ' ')-1) as city
Any ideas what to try?
It looks like your zip codes and your states are all the same length. If that is true, you should be able to use something like this:
SELECT
LEFT(a.Address2,LEN(a.Address2) - 13) AS City,
RIGHT(LEFT(a.Address2,LEN(a.Address2) - 11),2) AS State,
RIGHT(a.Address2,10) AS Zip_Code
FROM
table;
DEMO CODE
Create the table and data:
CREATE TABLE MyTable (Address2 VARCHAR(100));
INSERT INTO MyTable
VALUES
('SAN DIEGO CA 92128-1234'),
('WEST WARWICK RI 02893-1349'),
('RICHMOND IN 47374-9409');
The query:
SELECT
LEFT(Address2,LEN(Address2) - 13) AS City,
RIGHT(LEFT(Address2,LEN(Address2) - 11),2) AS State,
RIGHT(Address2,10) AS Zip_Code
FROM
MyTable;
The output:
Since you only have 3 parts (City/State/Zip) you can take advantage of a function called parsename in SQL Server 2008 and later. (The original intent of the function is to parse out object names.)
Using a combination of the replace and parsename functions will allow you to be able to separate the data into 3 parts, even if the length of the State (not likely) or the Zip (more likely) change.
Example Data:
create table #my_table
(
address2 varchar(75) not null
)
insert into #my_table values ('CONNERSVILLE IN 47331-3351')
insert into #my_table values ('WEST WARWICK RI 02893-1349')
insert into #my_table values ('RICHMOND IN 47374-9409')
insert into #my_table values ('WILLIAMSBURG IN 47393-9617')
insert into #my_table values ('FARMERSVILLE OH 45325-9226')
--this record is an example of a likely scenario for when the zip length would change.
insert into #my_table values ('WILLIAMSBURG IN 47393')
Solution:
with len_vals as
(
select t.address2
, len(parsename(replace(t.address2,' ','.'), 1)) as zip_len
, len(parsename(replace(t.address2,' ','.'), 2)) as st_len
from #my_table as t
group by t.address2
)
select left(a.address2, len(a.address2) - b.zip_len - b.st_len - 2) as city
, substring(a.address2, len(a.address2) - b.zip_len - 2, b.st_len) as st
, right(a.address2, b.zip_len) as zip_code
from #my_table as a
inner join len_vals as b on a.address2 = b.address2
Results:
SELECT * FROM (
SELECT ID,
CAST('EDIT' AS NVARCHAR(MAX)) AS 'test',
CAST([dbo].[SC_GetVersionedFieldValue](ID, '{8DBF084C-E575-4739-B37A-F732F72CFF69}') AS NVARCHAR(MAX)) AS 'City',
CAST([dbo].[SC_GetVersionedFieldValue](ID, '{32ED6082-1145-4331-9D29-F47E19090A0A}') AS NVARCHAR(MAX)) AS 'Sectors'
FROM [dbo].[Items] WHERE [TemplateID] = '{000C7660-E35B-4734-B1DA-A42A79A7B827}')
mytable
WHERE [City] like 'l%'
OR [Sectors] LIKE '%{20090D52-4C76-4C93-A249-F6E0883F4663}%'
OR [Sectors] LIKE '%{FAE9FCB9-8D1C-439E-BA6C-00804C58361E}%'
OR [Sectors] LIKE '%{844C2884-4E6A-4F38-9DE4-7CCF7DDF06C6}%'
I need to retrieve the rows where city name start with ‘some alphabet’ AND sectors should be in (id1, id2, id3…n) not OR. Some of the ids were located in between two ids that is delimited data (refer the picture). Any advice?
You will need to write a multistatement table-valued function that translates your multi-valued column into multiple rows. (see the documentation for CREATE FUNCTION.)
Once you have that function, you can then call it via CROSS APPLY.
(And, as Joel Coehoorn noted in a comment, you shouldn't store data like this in SQL Server. Instead, moving the GUID's for "Sectors" to a foreign table. Or, at least, an XML field.)
I Don't understand your need in where clause but try this
SELECT * FROM (
SELECT ID,
CAST('EDIT' AS NVARCHAR(MAX)) AS 'test',
CAST([dbo].[SC_GetVersionedFieldValue](ID, '{8DBF084C-E575-4739-B37A-F732F72CFF69}') AS NVARCHAR(MAX)) AS 'City',
CAST([dbo].[SC_GetVersionedFieldValue](ID, '{32ED6082-1145-4331-9D29-F47E19090A0A}') AS NVARCHAR(MAX)) AS 'Sectors'
FROM [dbo].[Items] WHERE [TemplateID] = '{000C7660-E35B-4734-B1DA-A42A79A7B827}')
mytable
WHERE [City] like '[A-Z]%' --'[a-z]%'
and (
[Sectors] LIKE '%{20090D52-4C76-4C93-A249-F6E0883F4663}%'
OR [Sectors] LIKE '%{FAE9FCB9-8D1C-439E-BA6C-00804C58361E}%'
OR [Sectors] LIKE '%{844C2884-4E6A-4F38-9DE4-7CCF7DDF06C6}%'
)
HOPE GOT SOME IDEA.
I am having problems using unpivot on columns, that are not the exact same datatype, and I can't figure out how to convert the columns on the fly, because the syntax for UNPIVOT does not seem to support it.
Consider this example:
DECLARE #People TABLE
(PersonId int, Firstname varchar(50), Lastname varchar(50))
-- Load Sample Data
INSERT INTO #People VALUES (1, 'Abe', 'Albertson')
INSERT INTO #People VALUES (2, 'Benny', 'Boomboom')
SELECT PersonId, ColumnName, Value FROM #People
UNPIVOT
(
ColumnName FOR
Value IN (FirstName, LastName)
)
The result would be this:
PersonId ColumnName Value
----------- ----------------- ----------------
1 Abe Firstname
1 Albertson Lastname
2 Benny Firstname
2 Boomboom Lastname
Everything is unicorns and rainbows. Now I change the datatype of Lastname to varchar(25) and everything breaks. The output is:
The type of column "Lastname" conflicts with the type of other columns
specified in the UNPIVOT list.
How can I get around this and convert everything to say a varchar(50) on the fly, without tampering with the actual data types on the table?
SqlFiddle working example (same datatype): http://sqlfiddle.com/#!3/f3719
SqlFiddle broken example (diff datatypes): http://sqlfiddle.com/#!3/5dca13/1
You cannot convert inside the UNPIVOT syntax but you can convert the data inside a subquery similar to the following:
select PersonId, ColumnName, Value
from
(
select personid,
firstname,
cast(lastname as varchar(50)) lastname
from People
) d
unpivot
(
Value FOR
ColumnName in (FirstName, LastName)
) unpiv;
See SQL Fiddle with Demo
Another way to do this would be to use CROSS APPLY, depending on your version of SQL Server you can use CROSS APPLY with VALUES or UNION ALL:
select PersonId, ColumnName, Value
from People
cross apply
(
select 'firstname', firstname union all
select 'lastname', cast(lastname as varchar(50))
) c (ColumnName, value)
See SQL Fiddle with Demo.
select col1, col2, Answer, Question_Col
from (
SELECT
col1,
col2,
col3,
CAST(col4 AS NVARCHAR(MAX)) col4,
-- Note: Here datatype should be same as that of previous columns
CAST(col5 AS NVARCHAR(MAX)) col5
FROM Table_Name )a
unpivot
(
Answer
for Question_Col in (
col3,
col4,
col5
)
) unpiv;