Is it possible to remove the cursor from this dynamic query? - sql

Simplified example of what I have:
Two tables (table_1 and table_2) which have one similar column (Id) but also several "payload" columns with different names (col_1_1, col_2_1, col_2_2). Amount of "payload" columns is different for different tables.
I am interested in extracting the IDs from both tables into another table for rows which have all "payload" columns empty.
There is a list of all "payload" columns for all tables which can be used (#temp)
This is how it is done with cursor:
CREATE TABLE #temp (tab nvarchar(20) not null, col nvarchar(20) not null)
INSERT INTO #temp SELECT 'table_1','col_1_1' UNION SELECT 'table_2','col_2_1' UNION SELECT 'table_2','col_2_2'
DECLARE #table_name nvarchar(20)
DECLARE #sql nvarchar(max)
DECLARE curs CURSOR FOR (SELECT DISTINCT tab FROM #temp)
OPEN curs
FETCH NEXT FROM curs INTO #table_name
WHILE ##FETCH_STATUS = 0
BEGIN
SELECT #sql = ISNULL(#sql,'')+col+' IS NULL AND ' FROM #temp WHERE tab = #table_name
SET #sql += 'Id IS NOT NULL'
SET #sql = 'INSERT INTO #temp_master SELECT ID FROM '+#table_name+' WHERE '+#sql
print #sql
SET #sql = ''
FETCH NEXT FROM curs INTO #table_name
END
CLOSE curs
DEALLOCATE curs
This is the result:
INSERT INTO #temp_master SELECT ID FROM table_1 WHERE col_1_1 IS NULL AND Id IS NOT NULL
INSERT INTO #temp_master SELECT ID FROM table_2 WHERE col_2_1 IS NULL AND col_2_2 IS NULL AND Id IS NOT NULL
Is it possible to remove the cursor to get the same resulting dynamic query? The problem is that I am unable to have dynamic "IS NULL AND" part for different tables when I remove the cursor.

It's possible to get rid of that cursor. This is probably what you need:
CREATE TABLE #temp (tab nvarchar(20) not null, col nvarchar(20) not null)
INSERT INTO #temp SELECT 'table_1','col_1_1' UNION SELECT 'table_2','col_2_1' UNION SELECT 'table_2','col_2_2'
DECLARE #table_name nvarchar(20)
DECLARE #sql nvarchar(max) = ''
select #sql = 'INSERT INTO #temp_master SELECT ID FROM ' + t.tab + ' WHERE Id IS NOT NULL AND ' + substring(t.cols, 0, len(t.cols)-3) + '
' + #sql from
(
SELECT
distinct
t2.tab,
stuff(
(
select t1.col + cast(' IS NULL AND ' as varchar(max))
from #temp t1
WHERE t1.tab = t2.tab
order by t1.tab
for xml path('')
), 1, 0, '') AS cols
FROM
#temp t2
) as t
order by t.tab desc
print #sql
drop table #temp

That is a regular CONCAT question, you can find many approaches to accomplish it without cursor. One of approaches is a cursor by the way and it's not that bad for such a task.
Another and more popular - FOR XML which can guarantee row order if any defined:
DECLARE #sql VARCHAR(MAX)
CREATE TABLE #temp (tab nvarchar(20) not null, col nvarchar(20) not null)
INSERT INTO #temp SELECT 'table_1','col_1_1' UNION SELECT 'table_2','col_2_1' UNION SELECT 'table_2','col_2_2'
SET #sql = (SELECT (
SELECT '
INSERT INTO #temp_master (ID) SELECT t.ID FROM '+t.tab +' t WHERE t.Id IS NOT NULL'
+ (select ' AND t.' + tt.col + ' is NULL' from #temp tt WHERE tt.tab = t.tab FOR XML PATH(''), TYPE).value('.', 'varchar(max)')
FROM #temp t
GROUP BY t.tab
FOR XML PATH(''), TYPE).value('.', 'varchar(max)'))
PRINT #sql
DROP TABLE #temp
A little "tricky" thing is that you have two things to collaps:
many queries (separate for specific table)
many columns per table
So you there is one inner FOR XML to collapse columns per table and another - to combine all queries into one big script.

Related

How to retrieve a column name from a table that is stored as a value in another table

I'm pretty new to sql so any help will be much appreciated
I have a table containing a list of table names in a column of a table and I need to retrieve a column called [Last Refreshed] from all the tables listed. The tables all have different structures but they all have the [Last Refreshed] Column. I have managed to insert the tablenames into a sql variable but up to this point I am kind of stuck.
I hope I managed to explain what I need but I have attached my code as well.
Declare #tables nvarchar(max)
Declare #sql nvarchar(max)
Declare #cnt int
DECLARE #Counter int
SET #Counter = 1
DECLARE #RowCount INT
SET #RowCount = (SELECT COUNT(*)
FROM (
SELECT * FROM TABLE_LIST1
UNION
SELECT * FROM TABLE_LIST2) data )
DROP TABLE #DB_DUMMY
CREATE TABLE #DB_DUMMY (
[TABLENAME] VARCHAR(512),
[LAST_REFRESHED] VARCHAR(533)
);
WHILE ( #Counter <= #RowCount)
BEGIN
SELECT #tables = FinalTable, #cnt = Row_num from (
SELECT FinalTable , ROW_NUMBER() OVER(ORDER BY FinalTable DESC) AS Row_num
FROM (
SELECT FinalTable FROM TABLE_LIST1
UNION
SELECT FinalTable FROM ABLE_LIST2) data
group by FinalTable) a
where Row_num = #Counter
--This part doesnt work
INSERT INTO #DB_DUMMY(TABLENAME,LAST_REFRESHED)
SELECT #tables , [Last Refreshed] from #tables
SET #Counter = #Counter + 1
END
exec(#sql)
I expect to see a list of tablenames as well as the last refresh in the temporary table #DB_DUMMY
i add the [Last Refreshed] column to my tables and write this query and give me the correct answer
DROP TABLE IF EXISTS #DB_DUMMY
CREATE TABLE #DB_DUMMY (
[TABLENAME] VARCHAR(512),
[LAST_REFRESHED] VARCHAR(533)
);
DECLARE #COMMAND NVARCHAR(MAX)
SELECT #COMMAND = STRING_AGG(' INSERT INTO #DB_DUMMY SELECT DISTINCT '+CHAR(39)+T.name+CHAR(39)+',['+C.name+'] FROM '+S.name+'.'+T.name + ' GO', CHAR(13)+CHAR(10))
FROM sys.all_columns C
INNER JOIN sys.tables T ON C.object_id = T.object_id
INNER JOIN sys.schemas S ON T.schema_id = S.schema_id
WHERE C.name = 'Last Refreshed'
PRINT(#COMMAND)
EXEC(#COMMAND)
SELECT * FROM #DB_DUMMY
two first line with IF EXISTS is new syntax in sql server 2017
Just a suggestion You could use a INSERT SELECT
INSERT INTO #DB_DUMMY(TABLENAME,LAST_REFRESHED)
SELECT 'TABLE_LIST1', LAST_REFRESHED
FROM TABLE_LIST1
UNION ALL
SELECT 'TABLE_LIST2', LAST_REFRESHED
FROM TABLE_LIST2
UNION ALL
.....
SELECT 'TABLE_LISTN', LAST_REFRESHED
FROM TABLE_LISTN
Try something like this:
declare cur cursor for Select TableName From TABLE_LIST
declare #tablename nvarchar(max)
declare #sqlstring nvarchar(max)
open cur
fetch next from cur into #tablename
while ##fetch_status=0
begin
set #sqlstring = 'SELECT ''' + #tablename + ''' AS ''TABLE'', [LAST_REFRESHED] FROM ' + #tablename
exec sp_executesql #sqlstring
fetch next from cur into #tablename
end
close cur
deallocate cur
;
It is the weekend and I don't have access to a database to test on, so it may need some adjusting. Here is a fiddle with the sample code, but it only returns the first table http://sqlfiddle.com/#!18/a5b55b/2 (I think the fiddle execution mechanism interferes with the cursor.)
This answer is based upon the code here: I have the same column in multiple tables, and want to update that column in all tables to a specific value. How can I do this?
Note that there is no need to maintain a list of tables with the column. You can generate it dynamically from INFORMATION_SCHEMA.COLUMNS
Another possible approach is to generate and execute a dynamic statement (it's not possible to use a variable for the name of a column or a table):
Table:
CREATE TABLE #TableNames (
[TableName] nvarchar(128)
)
INSERT INTO #TableNames
([TableName])
VALUES
(N'Table1'),
(N'Table2'),
(N'Table3'),
(N'Table4'),
(N'Table5')
Statement:
-- Generate statement
DECLARE #stm nvarchar(max) = N''
SELECT #stm = CONCAT(
#stm,
N'INSERT INTO #DB_DUMMY (TABLENAME, LAST_REFRESHED) ',
N'SELECT ''',
[TableName],
N''' AS [TableName], [LastRefreshed] FROM ',
QUOTENAME([TableName]),
N'; '
)
FROM #TableNames
-- Execute statement
PRINT #stm
EXEC sp_executesql #stm

Iterate through temporary table columns to select them

I have a final temporary table (#tempTable) with unknown columns number.
My final select is like this, it works :
SELECT temp.* FROM #tempTable temp
But instead of a '*' I would like to call each columns individually :
SELECT temp.col1, temp.col2 FROM #tempTable temp
To do so I need to iterate through my columns names and create a procedure, I tried something like this :
DECLARE #ColName VARCHAR(255)
SELECT #ColName = min(name) FROM tempdb.sys.columns
WHERE object_id = Object_id('tempdb..#TEMPTABLE');
WHILE #ColName is not null
BEGIN
-- i need to do it all in once and not each time....
declare #sql varchar(max) = 'SELECT tp.'+'#COlName'+'FROM #TEMPTABLE tp'
exec(#sql)
-- Increment the value, how to go to next column ?
select #ColName = min(name) FROM tempdb.sys.columns WHERE object_id =
Object_id('tempdb..#TEMPTABLE') > #ColName -- does not work because it is a string (column name)
END
Try this:
DECLARE #ColName VARCHAR(2000) = 'select '
SELECT #ColName = #ColName + ' temp.' + name + ',' FROM tempdb.sys.columns
WHERE object_id = Object_id('tempdb..#TEMPTABLE')
--delete last character, which is comma and append table name
#ColName = substring(#ColName, 1, LEN(#ColName) - 1) + ' from #TEMPTABLE temp'
exec(#ColName)
This query construct whole table list combined in select ... from ... statement. I increased size of the varchar variable, so it can accomodate long queries.
Also, IMO variable name such as #sql or #query would be more meaningful.
A set based approach
IF OBJECT_ID('tempdb..#TEMPTABLE','U') IS NOT NULL
DROP TABLE #TEMPTABLE;
CREATE TABLE #TEMPTABLE (
Id INT IDENTITY(1,1)
,Col1 INT
,Col2 BIGINT
,Col3 BIGINT
,Col4 DATETIME
,Col5 DATETIME
) ;
DECLARE #SQL NVARCHAR(MAX)
SELECT #SQL = N'SELECT ' + SUBSTRING((
SELECT N', temp.' + S.name
FROM
tempdb.sys.columns S
WHERE
S.object_id = OBJECT_ID('tempdb..#TEMPTABLE')
ORDER BY
S.column_id
FOR XML PATH('')
)
,2
,200000
) + N' FROM #TEMPTABLE temp'
EXEC sys.sp_executesql #SQL

Select all from tables where table names are from another table in SQL

I have a temp table which has a TableName column. I would like to loop through the temporary table and select everything in the the table (where table is the TableName column in the temp table).
I have been looking through the following link and related links however I am unable to adapt it to my needs. Any help is greatly appreciated.
I am using SQL Server 2014
Something which i have tried
Declare #id int
WHILE EXISTS(SELECT * FROM ##tt_tableList)
BEGIN
Select Top 1 #id = Id from ##tt_tableList
-- Do the work --
declare #query nvarchar(max)
set #query = 'Select * from (select TableName from ##tt_tablelist where id = '' +Cast(#id as nvarchar(50))+'')'
select #query
declare #tableName nvarchar(50)
set #tableName = (select TableName from ##tt_tableList where id = #id)
select #tableName
execute(#query)
-- Scrap the ID and Move On --
Delete ##tt_tableList where ID = #id
END
If I understood you correctly this is what you are asking for:
DECLARE #tbl table (TableName varchar(50))
insert into #tbl values ('SomeTableName')
insert into #tbl values ('AnotherTableName')
DECLARE #Tables VARCHAR(8000)
SELECT #Tables = COALESCE(#Tables + CHAR(13), '') + 'SELECT * FROM '+ TableName
FROM #tbl
exec(#Tables)
Just insert your table names in #tbl
I tried this based on answer from one of our fellow stack overflower and it works.
DECLARE #Tables VARCHAR(8000)
SELECT #Tables = COALESCE(#Tables + CHAR(13), '') + 'SELECT * FROM '+ TableName + ' Where Event like ''%CM_Manual_Change%'''
FROM ##tt_tableList
select #Tables
exec(#Tables)

How do I get a collection of every value in every column of a table?

I have two tables, Values and SpecialValues.
Values has two columns, RecordID and ValueName.
SpecialValues is a table which contains a single row, and thirty columns named SpecialValueName1, SpecialValueName2, SpecialValueName3, etc.
There are obvious database design problems with this system.
That aside, can someone explain to me how to query SpecialValues so that I can get a collection of all the values of every row from the table, and exclude them from a Select from Values?
There's probably some easy way to do this or create a View for it or something, but I think looking at this code might have broken me for the moment...
EDIT: I'd like a query to get all the individual values from every row and column of a given table (in this case the SpecialValues table) so that the query does not need to be updated the next time someone adds another column to the SpecialValues table.
This creates a #SpecialValuesColumns temporary table to store all the column names from SpecialValues.
It then uses a cursor to insert all the values from each of those columns into another temporary table #ProtectedValues.
It then uses a NOT IN query to exclude all of those values from a query to Values.
This code is bad and I feel bad for writing it, but it seems like the least-worst option open to me right now.
DECLARE #SpecialColumnsCount INT;
DECLARE #Counter INT;
DECLARE #CurrentColumnName VARCHAR(255);
DECLARE #ExecSQL VARCHAR(1024);
SET #Counter = 1;
CREATE TABLE #ProtectedValues(RecordID INT IDENTITY(1,1) PRIMARY KEY NOT NULL, Value VARCHAR(255));
DECLARE #SpecialValuesColumns TABLE (RecordID INT IDENTITY(1,1) PRIMARY KEY NOT NULL, ColumnName VARCHAR(255));
INSERT INTO #SpecialValuesColumns (ColumnName)
SELECT COLUMN_NAME
FROM INFORMATION_SCHEMA.COLUMNS
WHERE
TABLE_NAME = 'SpecialValues' AND
DATA_TYPE = 'varchar' AND
CHARACTER_MAXIMUM_LENGTH = 255
SELECT #SpecialColumnsCount = COUNT(*) FROM #SpecialValuesColumns
WHILE #Counter <= #SpecialColumnsCount
BEGIN
SELECT #CurrentColumnName = ColumnName FROM #SpecialValuesColumns WHERE RecordID = #Counter;
SET #ExecSQL = 'INSERT INTO #ProtectedValues (Value) SELECT ' + #CurrentColumnName + ' FROM SpecialValues'
EXEC (#ExecSQL)
SET #Counter = #Counter + 1;
END
SELECT * FROM Values WHERE ValueName NOT IN (SELECT ValueName COLLATE DATABASE_DEFAULT FROM #ProtectedValues)
DROP TABLE #ProtectedValues;
I might have misunderstood but doesn't this do it?
SELECT * FROM Values
WHERE ValueName NOT IN (
SELECT SpecialValueName1 FROM SpecialValues
UNION SELECT SpecialValueName2 FROM SpecialValues
UNION SELECT SpecialValueName3 FROM SpecialValues
etc..
)
You could of course make the subquery into a view instead.
*Edit:
This is quite ugly but should solve your problem:
First Create procedure #1
CREATE PROCEDURE [dbo].[SP1]
As
DECLARE
#Query nvarchar(MAX),
#Table nvarchar(255),
#Columns nvarchar(255)
CREATE TABLE #TempTable (Value nvarchar(255))
SET #Table = 'SpecialValues'
SELECT [COLUMN_NAME]
FROM [INFORMATION_SCHEMA].[COLUMNS]
WHERE [TABLE_NAME] = #Table
DECLARE Table_Cursor CURSOR FOR
SELECT COLUMN_NAME
FROM [INFORMATION_SCHEMA].[COLUMNS]
WHERE [TABLE_NAME] = #Table
OPEN Table_Cursor
FETCH NEXT FROM Table_Cursor INTO #Columns
WHILE ##FETCH_STATUS = 0
BEGIN
INSERT INTO #TempTable EXEC SP2 #Columns = #Columns, #Table = #Table
FETCH NEXT FROM Table_Cursor INTO #Columns
END
CLOSE Table_Cursor
DEALLOCATE Table_Cursor
SELECT ValueName FROM Value WHERE Value NOT IN (SELECT * FROM #TempTable)
TRUNCATE TABLE #TempTable
DROP TABLE #TempTable
Then Create procedure #2
CREATE PROCEDURE [dbo].[SP2]
#Columns nvarchar(255) = '',
#Table nvarchar(255)
AS
DECLARE
#Query nvarchar(MAX)
SET #Query = 'SELECT TOP 1 CONVERT(nvarchar, ' + #Columns + ') FROM ' + #Table
EXEC (#Query)
Then lastly execute the procedure
EXEC SP1
You need to unpivot the values in specialvalues. A pretty easy way to do that is with cross apply syntax:
select sv.value
from specialvalues sv cross apply
(values(sv.SpecialValueName1), (sv.SpecialValueName2), . . .
) sv(value)
where sv.value is not null;
You can exclude these from the list using not in, not exists or a left join.
What ever way you cut it, you have to specify the columns in SpecialValues, you can do this with a long set of UNION queries, or use UNPIVOT:
select SpecialValue
from (select SpecialValueName1,SpecialValueName2,SpecialValueName3 from #SpecialValues) p
unpivot (SpecialValue FOR ROW IN (SpecialValueName1,SpecialValueName2,SpecialValueName3))
AS unpvt
You can then incorporate this into a query on Values using NOT IN
select * from [Values] where ValueName not in (
select SpecialValue
from (select SpecialValueName1,SpecialValueName2,SpecialValueName3 from #SpecialValues) p
unpivot (SpecialValue FOR ROW IN (SpecialValueName1,SpecialValueName2,SpecialValueName3))
AS unpvt
)

Determine all columns from a table and write them separated by commas in a variable

--Dummy table
create table table1 (
column_order varchar (100)
)
insert into table1 values ('column1')
insert into table1 values ('column2')
insert into table1 values ('column3')
insert into table1 values ('column4')
insert into table1 values ('column5')
insert into table1 values ('column6')
--Start of select
declare #rowsCount INT
declare #i INT = 1
declare #column varchar(1000) = ''
set #rowsCount = (select COUNT(*) from table1)
while #i <= #rowsCount
begin
set #column = #column + (select column_order from table1 where rowid(table1) = #i) + ', '
set #i = #i + 1
end
select #column
This code has the function ROWID thats an IQ-Sybase funktion, and im not sure what other DBMS can use it. And above you have a example what i want my select to look like.
My problem is, you cant use the ROWID function with sys.column or any other systables. Has anyone an idea how to get the same select as mine without using the ROWID function.
If you are using IQ, i constructed the code so you can just type f5 and see the select statement, after that just drop the dummy table.
Use list(). It works in both the ASA system and IQ catalogs.
drop table if exists table1
go
create local temporary table table1 (
column_order varchar (100)
) in system --create table in system
insert into table1 values ('column1')
insert into table1 values ('column2')
insert into table1 values ('column3')
insert into table1 values ('column4')
insert into table1 values ('column5')
insert into table1 values ('column6')
declare #columns varchar(100)
select #columns = list(column_order) from table1
select #columns
go
I may be not understand your need, because I can't see why you need rowdid.
Usually, in TSQL, I do as follow:
declare #someVar as nvarchar(max)
set #someVar = (select
'[' + c.name + '],' as 'data()'
from
sys.columns c
join sys.tables t on c.object_id = t.object_id
where
t.name = 'SomeTableName'
for xml path(''))
print Left(#someVar, Len(#someVar) - 1)
Maybe you will need to use a cursor:
-- #MyTableID has to be definded somewhere or replace
DECLARE #columns varchar(32000)
DECLARE my_cursor CURSOR FOR
SELECT syscolumn.column_name FROM syscolumn WHERE syscolumn.table_id = #MyTableID
OPEN my_cursor
FETCH NEXT my_cursor into #column
WHILE ##FETCH_STATUS = 0
BEGIN
-- put your magic here
-- e.g.
-- SET #columns = #columns + column
FETCH NEXT my_cursor_pkey into #column
END
CLOSE my_cursor
DEALLOCATE my_cursor
Not tested yet, but something like that should work.