I have a number of databases with tables Table1 with the same structures. All table names are listed in the main1 database in the table testbases. It is necessary in a loop to go through all the rows from the Table1 and insert them into query.
SELECT * FROM [DBNAME].[DBO].Table1.Client
UNION
To end up with a big query like this:
SELECT * FROM [DBNAME1].[DBO].Table1.Client
UNION
SELECT * FROM [DBNAME2].[DBO].Table1.Client
UNION
SELECT * FROM [DBNAME3].[DBO].Table1.Client
UNION
SELECT * FROM [DBNAME4].[DBO].Table1.Client
UNION
etc...
How can I do this efficiently and automatically, so I don't need to manually change the query every time we add a client?
If all of the tables are identical, you could use sp_MSforeachdb along with an IF EXISTS to go through all of your databases, then insert the table if found into a temp table. You would just need to change the column names from col1, col2, ... to whatever your actual table schema is. Although you'd need to exclude any databases that contain a Table1 table that you don't want included in your data set.
DROP TABLE IF EXISTS #insertTable (col1 varchar(100), col2 varchar(100),...)
DECLARE #command varchar(1000)
SELECT #command = 'USE ?
EXEC(''IF EXISTS (SELECT 1 FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = ''''Table1'''')
SELECT col1, col2, ... FROM Table1'')'
INSERT INTO #insertTable (col1, col2, ...)
EXEC sp_MSforeachdb #command
Related
I need to use a temp table and CTE in SQL task in my SSIS package. But I got the following errors
" Metadata discovery only supports temp tables when analyzing a single- statement batch "
I tried to refer SSIS Package not wanting to fetch metadata of temporary table but I don't know how to do it in my case
The query that I'm using is
--Creating multiple temp table to increase preforming
SELECT *
INTO #TEMP_1
FROM MYTABLE
SELECT *
INTO #TEMP_2
FROM MYTABLE
SELECT *
INTO #TEMP_3
FROM MYTABLE
--create a final result set
;WITH CTE AS (
SELECT * FROM #TEMP_1
UNION
SELECT * FROM #TEMP_2
UNION
SELECT * FROM #TEMP_3
)
--data I eventually need
SELECT col1,col2,col3
FROM CTE
WHERE some_condition
All those code is working fine in SSMS but failed when I load to SSIS.
Any suggestion?
If you're working with temporary tables and you're on SQL Server 2012+, an option is to use EXECUTE with the RESULT SETS option.
Given your supplied query, I wrapped it with an Execute and specified the column names and types.
EXECUTE(N'
SELECT 1 AS col1, NULL AS col2, NULL AS col3, ''A'' AS col4 INTO #TEMP_1
SELECT NULL AS col1, 2 AS col2, NULL AS col3, ''B'' AS col4 INTO #TEMP_2
SELECT NULL AS col1, NULL AS col2, 3 AS col3, ''John'' AS col4 INTO #TEMP_3
--Creating multiple temp table to increase preforming
--create a final result set
;WITH CTE AS (
SELECT * FROM #TEMP_1
UNION
SELECT * FROM #TEMP_2
UNION
SELECT * FROM #TEMP_3
)
--data I eventually need
SELECT col1,col2,col3
FROM CTE
WHERE 1=1 AND col4=''John''
') WITH RESULT SETS
(
(Column1 int, Column2 int, Column3 int)
);
The OLE DB Source component was able to interpret that metadata just fine and out comes my data.
This solution works because it's is dynamic SQL. I am creating a big string and passing it to the EXEC call and specifying the RESULT SET which allows SSIS to properly infer the metadata. If, as I suspect, you need to fool around with dynamic filters and such, then you can do some fancy string building early on and reduce the EXEC call.
-- Do whatever needs to be done to properly define your queries
-- add filtering, etc
-- Before you EXEC, I find printing the resulting SQL handy so I can manually run it through a parser looking for mistakes
DECLARE #DynamicSQL nvarchar(max) = N'
SELECT 1 AS col1, NULL AS col2, NULL AS col3, ''A'' AS col4 INTO #TEMP_1
SELECT NULL AS col1, 2 AS col2, NULL AS col3, ''B'' AS col4 INTO #TEMP_2
SELECT NULL AS col1, NULL AS col2, 3 AS col3, ''John'' AS col4 INTO #TEMP_3
--Creating multiple temp table to increase preforming
--create a final result set
;WITH CTE AS (
SELECT * FROM #TEMP_1
UNION
SELECT * FROM #TEMP_2
UNION
SELECT * FROM #TEMP_3
)
--data I eventually need
SELECT col1,col2,col3
FROM CTE
WHERE 1=1 AND col4=''John''
';
EXECUTE(#DynamicSQL) WITH RESULT SETS
(
(Column1 int, Column2 int, Column3 int)
);
To embed a quote within the existing single quoted mess of statements, you need to double them - two single quotes, not a double
Im trying to create a temp table #TempTable with columns. There are many columns and I do not want to type them all out by hand. Is there a way to pre-fill, if that makes sense?
Instead of
CREATE #TempTable (col1, col2 ... col1000) -- Im not saying we have 1000
But doing:
CREATE #TempTable (SELECT column_name
from information_schema.columns where table_name = 'OriginalTable')
Is this possible? Im using MS SQL.
You can do SELECT . . . INTO :
SELECT ot.* INTO #TempTable
FROM OriginalTable ot
WHERE 1 = 0;
Note : When using the SELECT . . . INTO statement, the #TempTable must not already exist.
When creating a temp table, it's good to clean up before creating it. repeating this step will cause an error if the table already exists
IF OBJECT_ID('tempdb..#tmp') IS NOT NULL
DROP TABLE #tmp
select *
into #tmp
from OriginalTable
One way of doing -
select top 0 * into #TempTable from OriginalTable
The above creates an empty copy of temp table
If you don't want to specify *(wildcard) and want specific columns from OriginalTable to be created in your temporary table -
select top 0 col1,col2,col3 into #TempTable from OriginalTable
I am working on a tool (using c# winforms) which generates SQL scripts of data which are independent of identities but yet not violating constraints. While walking through all possibilities I got into below trouble. I need to insert data considering the fact a column (which allows null) may exists or not.
For example, a column as VAL2 exists in dev box, but may or may not exists in prod box.
CREATE TABLE TEMP ( COL1 VARCHAR(50) , VAL1 VARCHAR(50)) -- OPTIONAL/MAY EXISTS VAL2 VARCHAR(50)
IF EXISTS (SELECT TOP(1) COLUMN_NAME FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME = 'TEMP' AND COLUMN_NAME = 'VAL2')
BEGIN
INSERT INTO TEMP (COL1,VAL1,VAL2) VALUES ('1','YES','NO')
INSERT INTO TEMP (COL1,VAL1,VAL2) VALUES ('2','NO','NO')
END
ELSE
BEGIN
INSERT INTO TEMP (COL1,VAL1) VALUES ('1','YES')
INSERT INTO TEMP (COL1,VAL1) VALUES ('2','NO')
END
GO
But this method fails miserably if that column is missing even though its should be flowing through else by syntax.
Note: I had for now made a work around but adding that column and dropping it at end if only created through current script. But the problem is the execution is through limited access grant which rules possibility of CREATE/DROP statement.This may be wrong, but that's the way it is for me.
If there is any alternate way preferably which doesn't mandates adding this column , please do guide me.
You'll have to use dynamic sql to accomplish that:
CREATE TABLE TEMP
(
COL1 VARCHAR(50) ,
VAL1 VARCHAR(50)
)
-- OPTIONAL/MAY EXISTS VAL2 VARCHAR(50)
IF EXISTS ( SELECT TOP ( 1 )
COLUMN_NAME
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = 'TEMP'
AND COLUMN_NAME = 'VAL2' )
BEGIN
DECLARE #sql NVARCHAR(MAX)
SELECT #sql = 'INSERT INTO TEMP (COL1,VAL1,VAL2) VALUES (''1'',''YES'',''NO'')
INSERT INTO TEMP (COL1,VAL1,VAL2) VALUES (''2'',''NO'',''NO'')'
EXEC sp_executesql #sql
END
ELSE
BEGIN
INSERT INTO TEMP
( COL1, VAL1 )
VALUES ( '1', 'YES' )
INSERT INTO TEMP
( COL1, VAL1 )
VALUES ( '2', 'NO' )
END
I have a DataBase with around +100 tables, like half of tables have column A & column B.
My question is, Can I query all tables that have this columns with a specific values e.g.
SELECT * FROM DATABASE
WHERE
EACHTABLE HAS COLUMN A = 21 //only if table has columns and then values
AND
COLUMN B = 13
I am not sure how exact I will do it, nothing is coming up on google either
You can use the undocumented MS stored procedure sp_MSforeachtable, if you fancy living life recklessly:
create table T1 (
ColumnA int not null,
ColumnB int not null
)
go
create table T2 (
ColumnA int not null,
Column2 int not null
)
go
create table T3 (
Column1 int not null,
ColumnB int not null
)
go
create table T4 (
ColumnA int not null,
ColumnB int not null
)
go
insert into T1 values (1,2);
insert into T2 values (3,4);
insert into T3 values (5,6);
insert into T4 values (7,8);
go
create table #Results (TableName sysname,ColumnA int,ColumnB int)
exec sp_MSforeachtable 'insert into #Results select ''?'',ColumnA,ColumnB from ?',
#whereand = ' and syso.object_id in (select object_id from sys.columns where name=''ColumnA'') and syso.object_id in (select object_id from sys.columns where name=''ColumnB'')'
select * from #Results
drop table #Results
Result:
TableName ColumnA ColumnB
------------------------------------- ----------- -----------
[dbo].[T1] 1 2
[dbo].[T4] 7 8
By default, sp_MSforeachtable will, as its name implies, perform the same task for each table in the database. However, one optional parameter to this procedure, called #Whereand, can be used to modify the WHERE clause of the internal query that enumerates the tables in the database. It helps to know that this internal query has already established two aliases to some of the system views. o is an alias for sysobjects (the legacy system view). syso is an alias for sys.all_objects (a more modern system view).
Once sp_MSforeachtable has decided which tables to run against, it will execute the query given to it as its first parameter. But, it will replace ? with the schema and table name (? is the default replacement character. This can be changed as needed)
In this case, I chose to create a temp table, then have each selected table store its results into this temp table, and after sp_MSforeachtable has finished running, to select the combined results out with no further processing.
There is a similar (and similarly undocumented) procedure called sp_MSforeachdb which will access each user database on the server. These can even be combined (although you have to be careful with doubling up ' quote characters twice, at times). However, there's no equivalent sp_MSforeachcolumn.
Try this:
select t.name from sys.objects t inner join sys.columns c
on t.name=OBJECT_NAME(c.object_id)
where t.type='U'
and c.name in('col1','col2')
group by t.name
having COUNT(*) = 2
order by 1
Then you just loop through all the tables and fine the values for these columns.
Like
Declare #out TABLE(tblname varchar(100))
if exists(select * from tbl1 where col1='21' and col2='22')
BEGIN
INSERT INTO #out
select tbl1
END
You can try like this using dynamic query.
select 'select * from '+table_name+ ' where'+column_name+'=21'
from information_schema.columns where column_name = 'A'
I suggest to use two steps:
First, find out all tables in your database that have these two columns and use it for a temporal derived table. For I am not an expert in SQL-Server 2008 I recommend to have a look at the whitepages.
The expression might look like this:
SELECT tablename
FROM information_schema.tables sdbt
WHERE "column a" IN
(SELECT columns
FROM information_schema.columns col
WHERE col.tablename = sdbt.tablename)
Second, use a expresssion to filter the results according to your demanded values.
This command should do the trick in one go, only for column A, amend accordingly to include any other columns you need:
exec sp_MSforeachtable
#command1=N'SELECT * FROM ? WHERE A = 21',
#whereand=' and o.name IN (SELECT TABLE_NAME FROM INFORMATION_SCHEMA.COLUMNS WHERE COLUMN_NAME = ''A'') '
i want use select statement on the a table and inserting result into a temp table variable, but i don't declare temp table with columns and i want use like this:
Declare #tmp table;
SELECT * INTO #tmp FROM myTable
this want declare columns and data types for #tmp
please help me
You can do this simply without the DECLARE command - which is not valid for #temp tables anyway, only #table variables. Did you try just the following without trying to define #tmp first:
SELECT * INTO #tmp FROM myTable;
With data:
select *
into #tmp
from myTable
No data:
select *
into #tmp
from myTable
where 0=1
BTW, you can not do this with table variables.
select *
into #tmp
from myTable
Table variables need to be declared with the columns.