I am trying to select all the contents of all the columns stored in a variable from a table in SQL Server using dynamic SQL.
Here is my code:
IF #UserChoice# = 'MANGO' BEGIN
DECLARE #columns NVARCHAR(MAX)
SELECT #columns = COALESCE(#columns + ', ', '') + Cols
FROM (
SELECT DISTINCT x.value('local-name(.)', 'SYSNAME') AS Cols
FROM DBO.Fruits AS t
CROSS APPLY (SELECT t.* FOR XML PATH(''), TYPE, ROOT('root')) AS t1(c)
CROSS APPLY c.nodes('/root/*') AS t2(x)
) temp
DECLARE #sql NVARCHAR(MAX) = 'SELECT ' + #columns + ' FROM DBO.Foods;'
EXEC sp_executesql #sql;
END
In this code, there are two tables dbo.Fruits has the columns and dbo.Foods has all the columns existing in dbo.Fruits and additional columns and rows. However, dbo.Fruits has some null columns that is columns with no data in all of its rows. Thus with the help of XML null columns are removed and only non-null columns are stored in a variable #columns.
Then a dynamic SQL is written that performs select statement of the #columns from the table dbo.Foods.
Filtering of null columns is working however when I try to run the dynamic SQL. I get an error saying SQL statement failed.
FYI: I have huge data in the table. I have also tried timeout feature but not working.
Any help is appreciated.
Thanks in advance.
Related
I have a table with 122 columns and ~200K rows. There are duplicate rows in this table. What query can I use to return these duplicate rows? Normally I would GROUP BY all rows and COUNT, but with 122 columns, this becomes unwieldy.
Basically, I'm looking for a query that does this:
SELECT *, COUNT(*) AS NoOfOccurrences
FROM TableName
GROUP BY *
HAVING COUNT(*) > 1
If you are using SSMS you can right-click on the table and pick "Select Top 1000 rows..." - SSMS will generate the select query for you - then all you have to do is add GROUP BY then copy the column list and paste it after that. Add the HAVING COUNT(*) > 1 and the COUNT(*) AS NoOfOccurrences and run.
I suggest that you include an ORDER BY clause as well so that the duplicated rows are displayed together in your results.
If you are not using SSMS then you can run this dynamic SQL
-- Get the list of columns
-- There are loads of ways of doing this
declare #colList nvarchar(MAX) = '';
select #colList = #colList + c.[name] +','
from sys.columns c
join sys.tables t on c.object_id =t.object_id
where t.[name] = 'tblFees';
-- remove the trailing comma
set #colList = LEFT(#colList,LEN(#colList)-1);
-- generate dynamic SQL
declare #sql1 nvarchar(max) = 'SELECT *, COUNT(*) AS NoOfOccurrences FROM TableName GROUP BY '
declare #sql2 nvarchar(max) = ' HAVING COUNT(*) > 1'
declare #sql nvarchar(max) = CONCAT(#sql1,#colList, #sql2)
--print #sql
-- run the SQL
exec sp_executesql #sql
For other ways of generating comma separated lists see Converting row values in a table to a single concatenated string
I am getting temp table with dynamically generated columns let say it is columns A,B,C,D etc from other source.
Now in my hand I have temp table with column generated. I had to write stored procedure with the use of temp table.
So my stored procedure is like
create proc someproc()
as
begin
Insert into #searchtable
select isnull(#temp.*,0.00)
End
Now #searchresult is table created by me to store temp table columns. The problem arises when I want to check isnull for #tempdb columns. Because from source it comes it may be 3 columns, again next time it may be 4 columns. It changes.
Since it is dynamically generated I cannot use each column name and use like below:
isnull(column1,0.00)
isnull(column2,0.00)
I had to use all column generated and check if value is empty use 0.00
I tried this below but not working:
isnull(##temp.*,0.00),
Try with Dynamic code by fetching the column name for your dynamic table from [database].NFORMATION_SCHEMA.COLUMNS
--Get the Column Names for the your dynamic table and add the ISNULL Check:
DECLARE #COLS VARCHAR(MAX) = ''
SELECT #COLS = #COLS + ', ISNULL(' + COLUMN_NAME + ', 0.00) AS ' + COLUMN_NAME
FROM tempdb.INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME LIKE '#temp[_]%' -- Dynamic Table (here, Temporary table)
DECLARE #COLNAMES VARCHAR(MAX) = STUFF(#COLS, 1, 1, '')
--Build your Insert Command:
DECLARE #cmd VARCHAR(MAX) = '
INSERT INTO #temp1
SELECT ' + #COLNAMES + ' FROM #temp'
--Execute:
EXEC (#cmd)
Hope, I understood your comment right:
CREATE PROCEDURE someproc
AS
IF OBJECT_ID(N'#searchtable') IS NOT NULL DROP TABLE #searchtable
IF OBJECT_ID(N'#temp') IS NOT NULL
BEGIN
DECLARE #sql nvarchar(max),
#cols nvarchar(max)
SELECT #cols = (
SELECT ',COALESCE('+QUOTENAME([name])+',0.00) as '+QUOTENAME([name])
FROM sys.columns
WHERE [object_id] = OBJECT_ID(N'#temp')
FOR XML PATH('')
)
SELECT #sql = N'SELECT '+STUFF(#cols,1,1,'')+' INTO #searchtable FROM #temp'
EXEC sp_executesql #sql
END
This SP checks if #temp table exists. If exists then it takes all column names from sys.columns table and we make a string like ,COALESCE([Column1],0.00) as [Column1], etc. Then we make a dynamic SQL query like:
SELECT COALESCE([Column1],0.00) as [Column1] INTO #searchtable FROM #temp
And execute it. This query result will be stored in #searchtable.
Notes: Use COALESCE instead of ISNULL, and sp_executesql instead of direct exec. It is a good practice.
I have a tsql dynamic pivot table query that works fine although I´m not happy with the column names in the final output.
To make it work, the user has to select up to 20 fruit names from a list of 200 fruit names. Then I build the pivot table so I end up with different column names everytime the select is run.
For example:
First time the column names are: apple, orange and pear
Second time is: .orange, banana, kiwi and apple
My question is: Is there a posibility to have static names, like for example: name of the first column always "col_1", second column "col_2" etc?
The select statement is as follows:
DECLARE #idList varchar(800)
DECLARE #sql nvarchar(max)
SELECT #idList = coalesce(#idList + ', ', '') + '['+ltrim(rtrim(id_producto)) +']'
from gestor_val_pos
group by id_producto order by id_producto
SELECT #sql = 'select * from #correlaciones pivot (max (correl)
for codigo2 in (' + #IDlist + ')) AS pvt order by codigo1;'
exec sp_executeSQL #sql
sure.. just created a new variable to hold the column aliases and Row_Number to get the column number.
DECLARE #idList varchar(800)
DECLARE #idListAlias varchar(800)
DECLARE #sql nvarchar(max)
SELECT
#idList = coalesce(#idList + ', ', '') + '['+ltrim(rtrim(id_producto)) +']',
#idListAlias = coalesce(#idListAlias + ', ', '') + '['+ltrim(rtrim(id_producto)) +'] as col_' + CONVERT(VARCHAR(10), ROW_NUMBER() OVER(ORDER BY id_producto))
from gestor_val_pos
group by id_producto order by id_producto
SELECT #sql = 'select ' + #idListAlias + ' from #correlaciones pivot (max (correl)
for codigo2 in (' + #IDlist + ')) AS pvt order by codigo1;'
exec sp_executeSQL #sql
Yes, but it'll make your query significantly more complex.
You'll need to return a list of the possible column names generated from #IDList and convert that into a SELECT clause more sophisticated than your current SELECT *.
When you've got that, use some SQL string splitting code to convert the #IDList into a table of items with a position parameter. Append AS <whatever> onto the end of any you want and use the FOR XML PATH trick to flatten it back, and you've got a SELECT clause that'll do what you want. But, as I said, your code is now significantly more complicated.
As an aside - I really hope that #idList is either completely impossible for any user input to ever reach or hugely simplified from your real code for this demonstration. Otherwise you've got a big SQL injection hole right there.
I'm working with MS SQL Server 2008. I'm trying to create a stored procedure to Merge (perhaps) several rows of data (answers) into a single row on target table(s). This uses a 'table_name' field and 'column_name' field from the answers table. The data looks like something like this:
answers table
--------------
id int
table_name varchar
column_name varchar
answer_value varchar
So, the target table (insert/update) would come from the 'table_name'. Each row from the anwsers would fill one column on the target table.
table_name_1 table
--------------
id int
column_name_1 varchar
column_name_2 varchar
column_name_3 varchar
etc...
Note, there can be many target tables (variable from answers table: table_name_1, table_name_2, table_name_3, etc.) that insert into many columns (column_name_1...2...3) on each target table.
I thought about using a WHILE statement to loop through the answers table. This could build a variable which would be the insert/update statement(s) for the target tables. Then executing those statements somehow. I also noticed Merge looks like it might help with this problem (select/update/insert), but my MS SQL Stored Procedure experience is very little. Could someone suggestion a strategy or solution to this problem?
Note 6/23/2014: I'm considering using a single Merge statement, but I'm not sure it is possible.
I'm probably missing something, but the basic idea to solve the problem is to use meta-programming, like a dynamic pivot.
In this particular case there is another layer to make the solution more difficult: the result need to be in different execution instead of beeing grouped.
The backbone for a possible solution is
DECLARE #cols AS NVARCHAR(MAX)
DECLARE #query AS NVARCHAR(MAX)
--using a cursor on SELECT DISTINCT table_name FROM answers iterate:
--*Cursor Begin Here*
--mock variable for the first value of the cursor
DECLARE #table AS NVARCHAR(MAX) = 't1'
-- Column list
SELECT #cols = STUFF((SELECT distinct
',' + QUOTENAME(column_name)
FROM answers with (nolock)
WHERE table_name = #table
FOR XML PATH(''), TYPE
).value('.', 'NVARCHAR(MAX)')
, 1, 1, '')
--Query definition
SET #query = '
SELECT ' + #cols + '
INTO ' + #table + '
FROM (SELECT column_name, answer_value
FROM answers
WHERE table_name = ''' + #table + ''') b
PIVOT (MAX(answer_value) FOR column_name IN (' + #cols + ' )) p '
--select #query
EXEC sp_executesql #query
--select to verify the execution
--SELECT * FROM t1
--*Cursor End Here*
SQLFiddle Demo
The cursor definition is omitted, because I'm not sure if it'll work on SQLFiddle
In addition to the template for a Dynamic Pivot the columns list is filtered by the new table name, and in the query definition there is a SELECT ... INTO instead of a SELECT.
This script does not account for table already in the database, if that's a possibility the query can be divided in two:
SET #query = '
SELECT TOP 0 ' + #cols + '
INTO ' + #table + '
FROM (SELECT column_name, answer_value
FROM answers
WHERE table_name = ''' + #table + ''') b
PIVOT (MAX(answer_value) FOR column_name IN (' + #cols + ' )) p '
to create the table without data, if needed, and
SET #query = '
INSERT INTO ' + #table + '(' + #cols + ')'
SELECT ' + #cols + '
FROM (SELECT column_name, answer_value
FROM answers
WHERE table_name = ''' + #table + ''') b
PIVOT (MAX(answer_value) FOR column_name IN (' + #cols + ' )) p '
or a MERGE to insert/update the values in the table.
Another possibility will be to DROP and recreate every table.
Approach I took to this complex problem:
Create several temporary tables to work with your data
Select and populate the temporary tables with the data
Use dynamic pivoting to pivot the rows into one row
Use a CURSOR with WHILE loop for multiple table entries
SET #query with the dynamically built MERGE statement
EXECUTE(#query)
Drop temporary tables
Good day StackOverflow
The table that I create from my dynamic sql can have any number of columns as it is a pivot table.
-- Pivot the table so we get the UW as columns rather than rows
DECLARE #SQL NVARCHAR(MAX)
SET #SQL = '
SELECT *
FROM #PreProcessed
PIVOT (SUM(Quotes)
FOR [UW] IN (' + #UWColumns + ')
) AS bob'
I run this code to run my dynamic sql.
EXEC sp_executesql #SQL,
N'#UWColumns nvarchar(MAX)',
#UWColumns
My question is, how do I store the resulting table? Especially when I don't know how many columns it will have or even what the columns will be called?
I tried the code below but it doesn't work
INSERT INTO #Temp
EXEC sp_executesql #SQL,
N'#UWColumns nvarchar(MAX)',
#UWColumns
Thanks everyone
SQL Server uses SELECT * INTO ...., as opposed to the CREATE TABLE AS syntax. So you'll need to modify your dynamic sql to:
SELECT * INTO <YOUR TABLE>
FROM #PreProcessed
PIVOT (SUM(Quotes)
FOR [UW] IN (' + #UWColumns + ')
) AS bob'
The only way I could find around this problem was to do all of my calculations in the dynamic sql. Which meant I had to work on two tables.
-- Pivot the table so we get the UW as columns rather than rows
DECLARE #SQL NVARCHAR(MAX)
SET #SQL = '
SELECT * INTO #temp
FROM #PreProcessed
PIVOT (SUM(Quotes)
FOR [UW] IN (' + #UWColumns + ')
) AS bob
SELECT DISTINCT t1.Date, d.Declines AS ''Declines'' , '+#UWColumns+'
FROM #temp AS t1 LEFT OUTER JOIN
#Declines AS d ON t1.DATE = d.DATE
'
PRINT #SQL
EXEC(#SQL)