I have a table which has been designed to have columns named the same as a value I need to reference it with. I know I need to unpivot the table and have got it working if I manually type all the column names, however this table keeps getting new columns added to it so I want to capture all the columns in the unpivot rather than script them manually.
I can get the column names using the column_name function and was wondering if this can be added to the unpivot at all, ive been playing around with it and its not looking possible at the moment to me so thought id check to see if there were any other suggestions.
Sadly I cant redesign the table with where the column names keep getting added to although that would be the ideal solution.
select Day, Rota, RotaTemplate
from table1 t1
unpivot
(
Rota
for RotaTemplate in (select Column_name
from INFORMATION_SCHEMA.COLUMNS
where TABLE_NAME = 'table1')
) unpiv;
You are not allowed to do this. You need to build dynamic SQL statement like follows:
DECLARE #DynamicSQL NVARCHAR(MAX)
SET #DynamicSQL = N'select Day, Rota, RotaTemplate' + CHAR(10) +
'from table1 t1'+ CHAR(10) +
'unpivot' + CHAR(10) +
'(' + CHAR(10) + CHAR(9) +
'Rota for RotaTemplate in ('
+
STUFF
(
(
SELECT ',[' + [COLUMN_NAME] + '] '
FROM [INFORMATION_SCHEMA].[COLUMNS]
WHERE [TABLE_NAME] = 'table1'
FOR XML PATH('')
)
,1
,1
,''
)
+')' + CHAR(10) +
') unpiv;'
EXEC sp_executesql #DynamicSQL
Related
I am using dynamic SQL to concatenate a bunch of scripts that I have loaded into a table. It will take those individual scripts, concatenate them into one script, and then put that finished script into another table. It has worked beautifully so far, concatenating about 17 scripts successfully. But yesterday, I tried 80. And there is the issue. It is capping out at 78,165 characters every time I try it. The guts of it go like this:
SET #FinScriptSQL
= N'SELECT script, row_number( ) over( order by scriptTypeId ) as execOrder INTO #' + #FinScriptName
+ N' from dbo.' + #FinScriptName
+ N';
DECLARE #tsqlStmt NVARCHAR(MAX);
SELECT #tsqlStmt = STUFF(
(
SELECT (CHAR(13) + CHAR(10) + script) AS [text()]
FROM #' + #FinScriptName
+ N'
ORDER BY execOrder
FOR XML PATH(''''), TYPE).value(''.'', ''nvarchar( max )''),
1 ,
1 ,
NULL
);
CREATE TABLE dbo.' + #FinScriptName + N'_FinishedScripts
(
Name sysname NULL,
FinishedScript NVARCHAR(MAX) NULL
);
INSERT INTO dbo.' + #FinScriptName + N'_FinishedScripts
SELECT ''' + #FinScriptName + N''', #tsqlStmt;'
PRINT #FinScriptSQL;
EXECUTE sp_executesql #FinScriptSQL;
The column of the table where the individual scripts are is an NVARCHAR(MAX), as is the destination column of the new table. I thought it might be a setting in SQL Server, so I have already ensured that the maximum characters retrieved is wide open. Any ideas?
I am using SQL Server 2012. i have a table with 90 columns. I am trying to select only columns that contains data. After searching i used the following procedure:
1- Getting all columns count using one select query
2- Pivoting Result Table into a Temp table
3- Creating Select query
4- Executing this query
Here is the query i used:
DECLARE #strTablename varchar(100) = 'dbo.MyTable'
DECLARE #strQuery varchar(max) = ''
DECLARE #strSecondQuery varchar(max) = 'SELECT '
DECLARE #strUnPivot as varchar(max) = ' UNPIVOT ([Count] for [Column] IN ('
CREATE TABLE ##tblTemp([Column] varchar(50), [Count] Int)
SELECT #strQuery = ISNULL(#strQuery,'') + 'Count([' + name + ']) as [' + name + '] ,' from sys.columns where object_id = object_id(#strTablename) and is_nullable = 1
SELECT #strUnPivot = ISNULL(#strUnPivot,'') + '[' + name + '] ,' from sys.columns where object_id = object_id(#strTablename) and is_nullable = 1
SET #strQuery = 'SELECT [Column],[Count] FROM ( SELECT ' + SUBSTRING(#strQuery,1,LEN(#strQuery) - 1) + ' FROM ' + #strTablename + ') AS p ' + SUBSTRING(#strUnPivot,1,LEN(#strUnPivot) - 1) + ')) AS unpvt '
INSERT INTO ##tblTemp EXEC (#strQuery)
SELECT #strSecondQuery = #strSecondQuery + '[' + [Column] + '],' from ##tblTemp WHERE [Count] > 0
DROP TABLE ##tblTemp
SET #strSecondQuery = SUBSTRING(#strSecondQuery,1,LEN(#strSecondQuery) - 1) + ' FROM ' + #strTablename
EXEC (#strSecondQuery)
The problem is that this query is TOO SLOW. Is there a best way to achieve this?
Notes:
Table have only one clustered index on primary key Column ID and does not contains any other indexes.
Table is not editable.
Table contains very large data.
Query is taking about 1 minute to be executed
Thanks in advance.
I do not know if this is faster, but you might use one trick: FOR XML AUTO will ommit columns without content:
DECLARE #tbl TABLE(col1 INT,col2 INT,col3 INT);
INSERT INTO #tbl VALUES (1,2,NULL),(1,NULL,NULL),(NULL,NULL,NULL);
SELECT *
FROM #tbl AS tbl
FOR XML AUTO
This is the result: col3 is missing...
<tbl col1="1" col2="2" />
<tbl col1="1" />
<tbl />
Knowing this, you could find the list of columns, which are not NULL in all rows, like this:
DECLARE #ColList VARCHAR(MAX)=
STUFF
(
(
SELECT DISTINCT ',' + Attr.value('local-name(.)','nvarchar(max)')
FROM
(
SELECT
(
SELECT *
FROM #tbl AS tbl
FOR XML AUTO,TYPE
) AS TheXML
) AS t
CROSS APPLY t.TheXML.nodes('/tbl/#*') AS A(Attr)
FOR XML PATH('')
),1,1,''
);
SELECT #ColList
The content of #ColList is now col1,col2. This string you can place in a dynamically created SELECT.
UPDATE: Hints
It would be very clever, to replace the SELECT * with a column list created from INFORMATION_SCHEMA.COLUMNS excluding all not-nullable. And - if needed and possible - types, wich contain very large data (BLOBs).
UPDATE2: Performance
Don't know what your very large data means actually... Just tried this on a table with about 500.000 rows (with SELECT *) and it returned correctly after less than one minute. Hope, this is fast enough...
Try using this condition:
where #columnname IS NOT NULL AND #columnname <> ' '
My table has all the column names
(There are more than 80 columns, I can't change the column names now)
in the format of '_'. Like First_Name, Last_Name,...
So i want to use select * from table instead
of using AS.
I want to select them by removing '_' in one statement. Anyway i can do it?
something like Replace(coulmnName, '_','') in select statement ?
Thanks
You can simply rename the column in your query. For example:
SELECT FIRST_NAME [First Name],
LAST_NAME [Last Name]
FROM UserTable
You can also use the AS keyword but this is optional. Also note that if you don't want to do this on every query you can use this process to create a view with renamed columns. Then you can use SELECT * the way you want to (although this is considered a bad idea for many reasons).
Best of luck!
Alternative - Map In The Client Code:
One other alternative is to do the mapping in the client code. This solution is going to depend greatly on your ORM. Most ORM's (such as LINQ or EF) will allow you to remap. If nothing else you could use AutoMapper or similar to rename the columns on the client using convention based naming.
You can't do this in a single statement unless you're using dynamic SQL. If you're just trying to generate code, you can run a query against Information_Schema and get the info you want ...
DECLARE #MaxColumns INT
DECLARE #TableName VARCHAR(20)
SET #TableName = 'Course'
SELECT #MaxColumns = MAX(ORDINAL_POSITION) FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME = #TableName
SELECT Col
FROM
(
SELECT 0 Num, 'SELECT' Col
UNION
SELECT ROW_NUMBER() OVER (PARTITION BY TABLE_NAME ORDER BY ORDINAL_POSITION) Num, ' [' + COLUMN_NAME + '] AS [' + REPLACE(COLUMN_NAME, '_', '') + ']' + CASE WHEN ORDINAL_POSITION = #MaxColumns THEN '' ELSE ',' END
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = #TableName
UNION
SELECT #MaxColumns + 1 Num, 'FROM ' + #TableName
) s
ORDER BY num
The question intrigued me and I did find one way. It makes it happen but if you just wanted to give a lot of aliases one time in one query I wouldn't recommend it though.
First I made a stored procedure that extracts all the column names and gives them an alias without '_'.
USE [DataBase]
GO
IF OBJECT_ID('usp_AlterColumnDisplayName', 'P') IS NOT NULL
DROP PROCEDURE usp_AlterColumnDisplayName
GO
CREATE PROCEDURE usp_AlterColumnDisplayName
#TableName VARCHAR(50)
,
#ret nvarchar(MAX) OUTPUT
AS
Select #ret = #ret + [Column name]
From
(
SELECT ([name] + ' AS ' + '[' + REPLACE([name], '_', ' ') + '], ') [Column name]
FROM syscolumns
WHERE id =
(Select id
From sysobjects
Where type = 'U'
And [name] = #TableName
)
) T
GO
Then extract that string and throw it into another string with a query-structure.
Execute that and you are done.
DECLARE #out NVARCHAR(MAX), #DesiredTable VARCHAR(50), #Query NVARCHAR(MAX)
SET #out = ''
SET #DesiredTable = 'YourTable'
EXEC usp_AlterColumnDisplayName
#TableName = #DesiredTable,
#ret = #out OUTPUT
SET #out = LEFT(#out, LEN(#out)-1) --Removing trailing ', '
SET #Query = 'Select ' + #out + ' From ' + #DesiredTable + ' WHERE whatever'
EXEC sp_executesql #Query
If you just wanted to give a lot of aliases at once without sitting and typing it out for 80+ columns I would rather suggest doing that with one simple SELECT statement, like the one in the sp, or in Excel and then copy paste into your code.
I'm working with MS SQL Server 2008. I'm trying to create a stored procedure to Merge (perhaps) several rows of data (answers) into a single row on target table(s). This uses a 'table_name' field and 'column_name' field from the answers table. The data looks like something like this:
answers table
--------------
id int
table_name varchar
column_name varchar
answer_value varchar
So, the target table (insert/update) would come from the 'table_name'. Each row from the anwsers would fill one column on the target table.
table_name_1 table
--------------
id int
column_name_1 varchar
column_name_2 varchar
column_name_3 varchar
etc...
Note, there can be many target tables (variable from answers table: table_name_1, table_name_2, table_name_3, etc.) that insert into many columns (column_name_1...2...3) on each target table.
I thought about using a WHILE statement to loop through the answers table. This could build a variable which would be the insert/update statement(s) for the target tables. Then executing those statements somehow. I also noticed Merge looks like it might help with this problem (select/update/insert), but my MS SQL Stored Procedure experience is very little. Could someone suggestion a strategy or solution to this problem?
Note 6/23/2014: I'm considering using a single Merge statement, but I'm not sure it is possible.
I'm probably missing something, but the basic idea to solve the problem is to use meta-programming, like a dynamic pivot.
In this particular case there is another layer to make the solution more difficult: the result need to be in different execution instead of beeing grouped.
The backbone for a possible solution is
DECLARE #cols AS NVARCHAR(MAX)
DECLARE #query AS NVARCHAR(MAX)
--using a cursor on SELECT DISTINCT table_name FROM answers iterate:
--*Cursor Begin Here*
--mock variable for the first value of the cursor
DECLARE #table AS NVARCHAR(MAX) = 't1'
-- Column list
SELECT #cols = STUFF((SELECT distinct
',' + QUOTENAME(column_name)
FROM answers with (nolock)
WHERE table_name = #table
FOR XML PATH(''), TYPE
).value('.', 'NVARCHAR(MAX)')
, 1, 1, '')
--Query definition
SET #query = '
SELECT ' + #cols + '
INTO ' + #table + '
FROM (SELECT column_name, answer_value
FROM answers
WHERE table_name = ''' + #table + ''') b
PIVOT (MAX(answer_value) FOR column_name IN (' + #cols + ' )) p '
--select #query
EXEC sp_executesql #query
--select to verify the execution
--SELECT * FROM t1
--*Cursor End Here*
SQLFiddle Demo
The cursor definition is omitted, because I'm not sure if it'll work on SQLFiddle
In addition to the template for a Dynamic Pivot the columns list is filtered by the new table name, and in the query definition there is a SELECT ... INTO instead of a SELECT.
This script does not account for table already in the database, if that's a possibility the query can be divided in two:
SET #query = '
SELECT TOP 0 ' + #cols + '
INTO ' + #table + '
FROM (SELECT column_name, answer_value
FROM answers
WHERE table_name = ''' + #table + ''') b
PIVOT (MAX(answer_value) FOR column_name IN (' + #cols + ' )) p '
to create the table without data, if needed, and
SET #query = '
INSERT INTO ' + #table + '(' + #cols + ')'
SELECT ' + #cols + '
FROM (SELECT column_name, answer_value
FROM answers
WHERE table_name = ''' + #table + ''') b
PIVOT (MAX(answer_value) FOR column_name IN (' + #cols + ' )) p '
or a MERGE to insert/update the values in the table.
Another possibility will be to DROP and recreate every table.
Approach I took to this complex problem:
Create several temporary tables to work with your data
Select and populate the temporary tables with the data
Use dynamic pivoting to pivot the rows into one row
Use a CURSOR with WHILE loop for multiple table entries
SET #query with the dynamically built MERGE statement
EXECUTE(#query)
Drop temporary tables
I need to switch from using a #temp table to a #table variable so that I can use it in a function.
My query uses insert into #temp (from multiple tables) like so:
SELECT
a.col1,
a.col2,
b.col1...
INTO #temp
FROM ...
Is there an easy way to find out the data types of the columns in the #temp table so that I can create the #table variable with the same columns and data types as #temp?
You need to make sure sp_help runs in the same database where the table is located (tempdb). You can do this by prefixing the call directly:
EXEC tempdb.dbo.sp_help #objname = N'#temp';
Or by prefixing a join against tempdb.sys.columns:
SELECT [column] = c.name,
[type] = t.name, c.max_length, c.precision, c.scale, c.is_nullable
FROM tempdb.sys.columns AS c
INNER JOIN tempdb.sys.types AS t
ON c.system_type_id = t.system_type_id
AND t.system_type_id = t.user_type_id
WHERE [object_id] = OBJECT_ID(N'tempdb.dbo.#temp');
This doesn't handle nice things for you, like adjusting max_length for varchar differently from nvarchar, but it's a good start.
In SQL Server 2012 or better, you can use a new DMF to describe a resultset, which takes that issue away (and also assembles max_length/precision/scale for you). But it doesn't support #temp tables, so just inject the query without the INTO:
SELECT name, system_type_name, is_nullable
FROM sys.dm_exec_describe_first_result_set(N'SELECT
a.col1,
a.col2,
b.col1...
--INTO #temp
FROM ...;',NULL,1);
The accepted answer does not give the data type.Joining tempdb.sys.columns with sys.types gives the data type as mentioned in the comment of the answer.But joining on system_type_id yields one extra row with datatype "sysname". Instead "user_type_id" gives the exact solution as given below.
SELECT cols.NAME
,ty.NAME
FROM tempdb.sys.columns cols
JOIN sys.types ty ON cols.user_type_id = ty.user_type_id
WHERE object_id = OBJECT_ID('tempdb..#temp')
you need to qualify the sp_help process to run from the tempdb database to get details about a hash table, because that's where the hash table is actually stored. If you attempt to run sp_help from a different database you'll get an error that the table doesn't exist in that database.
If your query is executing outside of tempdb, as I assume it is, you can run the following:
exec tempdb..sp_help #temp
One benefit of this procedure is it includes a text description of the column datatypes for you. This makes it very easy to copy and paste into another query, e.g. if you're trying use the definition of a temp table to create a table variable.
You could find the same information in the Syscolumns table, but it will give you numeric indentifiers for the types which you'll have to map yourself. Using sp_help will save you a step.
The other answers will give you the information that you need, but still require you to type it all out when you define the table variable.
The following TSQL will allow you to quickly generate the table variable's definition for any given table.
This can save you a lot of time instead of manually typing table definitions like:
table(Field1Name nvarchar(4), Field2Name nvarchar(20), Field3Name int
, Field4Name numeric(28,12))
TSQL:
select top 10 *
into #temp
from db.dbo.myTable
declare #tableName nvarchar(max)
set #tableName = '#temp'
use tempdb
declare #tmp table(val nvarchar(max))
insert into #tmp
select case data_type
when 'binary' then COLUMN_NAME + ' ' + DATA_TYPE + '(' + cast(CHARACTER_MAXIMUM_LENGTH AS nvarchar(max)) + ')'
when 'char' then COLUMN_NAME + ' ' + DATA_TYPE + '(' + cast(CHARACTER_MAXIMUM_LENGTH AS nvarchar(max)) + ')'
when 'datetime2' then COLUMN_NAME + ' ' + DATA_TYPE + '(' + CAST(DATETIME_PRECISION as nvarchar(max)) + ')'
when 'datetimeoffset' then COLUMN_NAME + ' ' + DATA_TYPE + '(' + CAST(DATETIME_PRECISION as nvarchar(max)) + ')'
when 'decimal' then COLUMN_NAME + ' ' + DATA_TYPE + '(' + cast(NUMERIC_PRECISION as nvarchar(max)) + ',' + cast(NUMERIC_SCALE as nvarchar(max)) + ')'
when 'nchar' then COLUMN_NAME + ' ' + DATA_TYPE + '(' + cast(CHARACTER_MAXIMUM_LENGTH AS nvarchar(max)) + ')'
when 'numeric' then COLUMN_NAME + ' ' + DATA_TYPE + '(' + cast(NUMERIC_PRECISION as nvarchar(max)) + ',' + cast(NUMERIC_SCALE as nvarchar(max)) + ')'
when 'nvarchar' then COLUMN_NAME + ' ' + DATA_TYPE + '(' + cast(CHARACTER_MAXIMUM_LENGTH AS nvarchar(max)) + ')'
when 'time' then COLUMN_NAME + ' ' + DATA_TYPE + '(' + CAST(DATETIME_PRECISION as nvarchar(max)) + ')'
when 'varbinary' then COLUMN_NAME + ' ' + DATA_TYPE + '(' + cast(CHARACTER_MAXIMUM_LENGTH AS nvarchar(max)) + ')'
when 'varchar' then COLUMN_NAME + ' ' + DATA_TYPE + '(' + cast(CHARACTER_MAXIMUM_LENGTH AS nvarchar(max)) + ')'
-- Most standard data types follow the pattern in the other section.
-- Non-standard datatypes include: binary, char, datetime2, datetimeoffset, decimal, nvchar, numeric, nvarchar, time, varbinary, and varchar
else COLUMN_NAME + ' ' + DATA_TYPE
end + case when IS_NULLABLE <> 'YES' then ' NOT NULL' else '' end 'dataType'
from INFORMATION_SCHEMA.COLUMNS
where TABLE_NAME like #tableName + '%'
declare #result nvarchar(max)
set #result = ''
select #result = #result + [val] + N','
from #tmp
where val is not null
set #result = substring(#result, 1, (LEN(#result)-1))
-- The following will replce '-1' with 'max' in order to properly handle nvarchar(max) columns
set #result = REPLACE(#result, '-1', 'max')
select #result
Output:
Field1Name nvarchar(4), Field2Name nvarchar(20), Field3Name int
, Field4Name numeric(28,12)
to get columns name with data type use this
EXEC tempdb.dbo.sp_help N'#temp';
or
To get only columns name to use this
SELECT *
FROM tempdb.sys.columns
WHERE [object_id] = OBJECT_ID(N'tempdb..#temp');
Finding the data types of a SQL temporary table
METHOD 1 – Using SP_HELP
EXEC TempDB..SP_HELP #TempTable;
Note-
In the Table Structure, the Table Name shows something like ‘#TempTable__________________________________________________________________________________________________________0000000004CB’. Actually, the total length of each and every Temp Table name will be 128 . To handle the Same Temp Table name in Multiple Sessions differently, SQL Server will automatically add some underscores in between and alphanumeric’s at end.
METHOD 2 – Using SP_COLUMNS
EXEC TempDB..SP_COLUMNS '#TempTable';
METHOD 3 – Using System Tables like INFORMATION_SCHEMA.COLUMNS, SYS.COLUMNS, SYS.TABLES
SELECT * FROM TempDB.INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME IN (
SELECT NAME FROM TempDB.SYS.TABLES WHERE OBJECT_ID=OBJECT_ID('TempDB.dbo.#TempTable')
);
GO
SELECT * FROM TempDB.SYS.COLUMNS WHERE OBJECT_ID=OBJECT_ID('TempDB.dbo.#TempTable');
GO
SELECT * FROM TempDB.SYS.TABLES WHERE OBJECT_ID=OBJECT_ID('TempDB.dbo.#TempTable');
GO
I'd go the lazy route and use
use tempdb
GO
EXECUTE sp_help #temp
What you are trying to do is to get information about the system types of the columns you are querying.
For SQL Server 2012 and later you can use sys.dm_exec_describe_first_result_set function. It returns very detailed information about the columns and the system_type_column holds the complete system type definition (ready to use in your table definition):
For example:
SELECT *
FROM [sys].[dm_exec_describe_first_result_set] (N'SELECT object_id, name, type_desc FROM sys.indexes', null, 0);
Yes, the data types of the temp table will be the data types of the columns you are selecting and inserting into it. So just look at the select statement and determine each data type based on the column you select.