I'm trying to run a query across multiple databases. Each database is a different customer but I'm querying the same tables across all of them. I want to loop through all the databases (about 100) and put all the results in a table. I've tried a few different ways but I can't quite seem to get it working and it seems like it might be a syntax thing. Below is my code:
IF OBJECT_ID('KW.dbo.Result') IS NOT NULL DROP TABLE KW.dbo.Result;
SELECT name as DBName, ROW_NUMBER() OVER(PARTITION BY 1 ORDER BY name) AS RNo
INTO #DBName
FROM sys.databases
WHERE name LIKE 'Customer%'
ORDER BY name;
SELECT * FROM #DBName
DECLARE #sql varchar;
SET #sql = N'SELECT bh.ID,
b.LINE AS Line#,
c.State,
bh.ZIP,
REPLACE(p.Phone, '-', '') AS Phone,
cc.COMPANY_NAME,
b.Amount
INTO KW.dbo.Result
FROM dbo.Customer c
LEFT JOIN dbo.Change ch ON ch.CIDNo = c.CIDNo
LEFT JOIN dbo.Provider p ON p.PIDNo = ch.PIDNo
LEFT JOIN dbo.BossHead bh ON bh.CHIDNo = ch.CHIDNo
LEFT JOIN dbo.Bills b ON b.BIDNo = bh.BIDNo
LEFT JOIN dbo.PartnerTerms pt ON pt.BIdNO = b.BIdNo
LEFT JOIN dbo.CompanyCode cc ON cc.CompanyCode = pt.CompanyCode
WHERE REPLACE(p.Phone, '-', '') IN
(
SELECT *
FROM KW.dbo.PhoneNumbers
)
AND bh.CreateDate BETWEEN "09-01-2016" AND "10-01-2017"
AND pt.CompanyCode IS NOT NULL
ORDER BY 1';
DECLARE #DB varchar;
DECLARE #i int;
BEGIN TRANSACTION;
SET #i = 1;
WHILE #i <= (SELECT MAX(RNo) FROM #DBName)
BEGIN
SET #DB = (SELECT DBName FROM #DBName WHERE RNo = #i)
USE #DB;
EXECUTE sp_executesql #sql
SET #i = #i + 1
END
COMMIT TRANSACTION;
GO
Error Message: "Msg 102, Level 15, State 1, Line 55
Incorrect syntax near '#DB'."
Any help will be greatly appreciated, thank you!
You can't use a variable to switch tables. What you can do is create a dynamic query that uses the fully qualified database and table name to go across all of the databases at the same time.
Create a query that creates your query:
Use master
Select top(1) case when Row_Number() over (order by d.name) = 1 then '' else
'union ' end + 'select c.name from ' + quotename(d.name) +'.' +'.[dbo].[Customer]'
From sys.databases d
Order by d.name
Work with that until the query does what you want, then remove the top 1 and run the whole query against all of your databases.
Related
I'm trying to run a query against multiple databases on the same server.
I need to pull all the values from 2 tables in a database based on a criteria from a 3rd table in the database, if the database was created after a certain date.
I have a query to find when the database was created:
SELECT *
FROM sys.databases
WHERE STATE = 0 --ignores offline databases
AND database_id > 4 --does not include master, model, msdb, tempdb
AND create_date > CONVERT(datetime, '2021-01-01')
And the query that I need run on each database is generally as follows:
SELECT *
FROM Table1
INNER JOIN Table3 ON Table3.Column6=Table1.Column2
AND Table3.Column3='Value1'
AND Table3.Column4='Value2'
INNER JOIN Table2 ON Table3.Column6=Table2.Column2
I did find this question, which is essentially would I would like to do, but when I look at the INFORMATION_SCHEMA.TABLES the TABLE_CATALOG column does not have the table names I would like to query against. I thought I could try pulling the names from the sys.databases table like above, so I tried modifying it to:
DECLARE #cmd VARCHAR(max) = N''
SELECT #cmd += COALESCE(#cmd + ' UNION ALL ', '') + 'SELECT *
FROM [' + name + '].dbo.Table1
INNER JOIN [' + name + '].dbo.Table3 on Table3.Column6=Table1.Column2
AND Table3.Column3= ''Value1''
AND Table3.Column4=''Value2''
INNER JOIN [' + name + '].dbo.Table2 on Table3.Column6=Table2.Column2'
FROM sys.databases
WHERE STATE = 0
AND database_id>4
AND create_date>CONVERT(datetime,'2021-08-26')
SET #cmd = STUFF(#cmd, CHARINDEX('UNION ALL', #cmd), 10, '')
PRINT #cmd
EXEC(#cmd)
But when I run it with a date earlier than 2021-08-26 (which grabs more than 5 tables), I get a memory error. I need to run this at least to the beginning of April (preferably up to the beginning of the year) which will grab around 500 tables.
What is the recommended way to run a query against multiple databases in SQL?
My recommendation would be instead of trying to build one massive UNION ALL dynamic SQL statement, that you build a #temp table to hold the results of each output, and then it's much easier to send the same string to each database:
CREATE TABLE #hold(dbname sysname, Column1 {data type}, ...);
DECLARE #sql nvarchar(max), #exec nvarchar(1024);
SET #sql = N'SELECT DB_NAME(), *
FROM dbo.Table1
INNER JOIN dbo.Table3
ON Table3.Column6 = Table1.Column2
AND Table3.Column3 = ''Value1''
AND Table3.Column4 = ''Value2''
INNER JOIN dbo.Table2
ON Table3.Column6 = Table2.Column2;';
DECLARE #dbname sysname, #c cursor;
SET #c = CURSOR FORWARD_ONLY STATIC READ_ONLY FOR
SELECT name FROM sys.databases
WHERE state = 0 -- ignores offline databases
AND database_id > 4 -- does not include master, model, msdb, tempdb
AND create_date > CONVERT(datetime, '20210101');
OPEN #c;
FETCH NEXT FROM #c INTO #dbname;
WHILE ##FETCH_STATUS = 0
BEGIN
SET #exec = QUOTENAME(#dbname) + N'.sys.sp_executesql';
INSERT #hold EXEC #exec #sql;
FETCH NEXT FROM #c INTO #dbname;
END;
SELECT * FROM #hold;
You might also consider investing in sp_ineachdb, a procedure I wrote to help simplify running the same command in the context of each database.
Execute a Command in the Context of Each Database in SQL Server using sp_ineachdb
Execute a Command in the Context of Each Database in SQL Server - Part 2
I need to query multiple tables that contain the column name 'idClient' for a specific condition.
So far I acquire the tables that I need to query using the following query:
SELECT c.name AS 'ColumnName',
t.name AS 'TableName'
FROM sys.columns c (nolock)
JOIN sys.tables t (nolock) ON c.object_id = t.object_id
WHERE c.name LIKE '%idClient%'
ORDER BY TableName,
ColumnName;
This gives me the following result (in reality there are roughly 100 tables returned):
+------------+-----------------+
| ColumnName | TableName |
+------------+-----------------+
| idClient | tbClient |
| idClient | tbClientContact |
| idClient | tbInvoice |
+------------+-----------------+
In order for me to find all client records in each of the tables I am currently running 3 separate queries for each table name. For example:
SELECT * FROM tbClientContact (nolock)
JOIN tbClient (nolock)
ON tbClientContact.idClient = tbClient.idClient
WHERE tbClient.vcSurname = 'Smith'
Instead of running the above query 3 times for each table, is there an easier way to run the same query on all results that are returned as TableName?
GOAL: I have been tasked with - in the example above - removing any client records from a database where the client surname is 'Smith'. I am running the above SELECT query to find whether the idClient of all clients with the surname of 'Smith' will leave orphan records in tables where there is a link of 'idClient'. I am joining tbClient as the column vcSurname does not exist in any other table than tbClient.
You could try
SELECT idClient FROM tbClient WHERE vcSurname = 'Smith'
UNION ALL
SELECT idClient FROM tbClientContact WHERE vcSurname = 'Smith'
UNION ALL
SELECT idClient FROM tbInvoice WHERE vcSurname = 'Smith'
If you need more columns output, all three queries must have the same number of ouput columns and all of the same type
Edit
As others have suggested, it would be helpful to know what you are doing as the method you are attempting is never the best way of doing something. However, the cursor solution below should build a dynamic query to do what you want
DECLARE #table AS NVARCHAR(128)
DECLARE #sql AS NVARCHAR(4000)
DECLARE c CURSOR FOR
-- hey all the tables with the columns LIKE '%idClient%'
SELECT t.name AS 'TableName'
FROM sys.columns c (nolock)
JOIN sys.tables t (nolock) ON c.object_id = t.object_id
WHERE c.name LIKE '%idClient%'
ORDER BY TableName
-- loop through the results of the query line by line
OPEN c
FETCH NEXT FROM c INTO #table
WHILE ##FETCH_STATUS = 0
BEGIN
-- build the query dynamically
SET #sql =CONCAT(#sql,'SELECT idClient FROM ' + #table + ' WHERE vcSurname = ''Smith'' UNION ALL ')
FETCH NEXT FROM c INTO #table
END
-- remove last "UNION ALL" text
SET #sql = STUFF(#sql,LEN(#sql)-9,11,'')
EXEC sp_executesql #stmt = #sql
CLOSE c
DEALLOCATE c
EDIT EDIT
Just seen your edit. Does your table have foreign / primary key pairs, do the foreign keys have ON DELETE CASCADE? If so, it should just be a case of
DELETE from tbClient WHERE vcSurname = 'Smith'
For automated query:
After question Edit:
SELECT 'SELECT '+c.name+' FROM '+t.name+' T WITH(nolock)
JOIN tbClient (NOLOCK)
ON T.idClient = tbClient.idClient
WHERE vcSurname = ''Smith''
UNION ALL
'
FROM sys.columns c (nolock)
JOIN sys.tables t (nolock) ON c.object_id = t.object_id
WHERE c.name LIKE '%idClient%'
PRINT the result in text format (ctrl+T).
Copy the complete result and then remove last UNION ALL
1 Determine involved tables by querying systables (sysforeignkeys or syscolumns if you don't have FKs).
2 Manually write a comprehensive stored proc that is meant to do everything right
create proc dbo.Client_Del
#client_id int
as
begin try
if not exists(select 1 from dbo.Client c where c.id = #client_id)
raiserror("Client %d not found", 16, 1, #client_id)
begin tran
delete ct
from dbo.ClientContacts ct
where ct.client_id = #client_id
delete idt
from dbo.InvoiceDetail idt
inner join dbo.Invoice i
on i.invoice_id = idt.invoice_id
where i.client_id = #client_id
delete i
from dbo.Invoice i
where i.client_id = #client_id
delete c
from dbo.Client c
where c.client_id = #client_id
commit tran
end try
begin catch
if ##trancount > 0
rollback tran
throw
end catch
GO
3 Invoke your stored proc with an argument
declare #id int
set #id = (select c.client_id from dbo.Client c where c.LastName = 'Smith')
exec dbo.Client_Del
#client_id = #id
I am not sure on what is your need. And this post is enhanced code of #Luv. Use of this, you just run it.
declare #select nvarchar(max) = N''
set #select =
N'
declare #sql nvarchar(max) = N''''
SELECT #sql += '' UNION ALL SELECT ''+c.name+'' FROM ''+t.name+'' T WITH(nolock)
JOIN tbClient (NOLOCK)
ON T.idClient = tbClient.idClient
WHERE vcSurname = ''''Smith''''''
FROM sys.columns c (nolock)
JOIN sys.tables t (nolock) ON c.object_id = t.object_id
WHERE c.name LIKE ''%yourColumnName%''
set #sql = STUFF(#sql, 1, 10, '''')
print #sql
exec sp_executesql #sql
'
exec sp_executesql #select
I have a table with 700 columns. I am trying to get a list of distinct values for each column and their count. I am using the below query to get the result for 1 column
Select distinct col1, count(*) from MyTable group by 1.
Result:
col1 count(*)
a 10
b 20
c 40
How can I get the result for all columns using a single query in the most optimal way?
The basic query is:
select col001, count(*) from MyTable group by col001 union all
select col002, count(*) from MyTable group by col002 union all
. . .
select col700, count(*) from MyTable group by col700 ;
Not pleasant, but that is basically the query you need to run. SQL doesn't really do multiple independent aggregations more efficiently than doing them separately (even using grouping sets, in my experience).
You can construct the query. One way is to run something like this:
select replace(replace('select [col], count(*) as cnt from [tab] group by [col] union all ',
'[tab]', table_name
), '[col]', column_name
)
from information_schema.columns
where table_name = 'mytable' and table_schema = ??;
You can then copy the generated SQL (removing the final union all) and run it.
Note: That above is generic; the exact code might differ by database.
A list with distinct values for each column is impossible. What if column A has 5 distinct values and column B has 7. What would your list look like?
The other question is easier, but as #Gordon Linoff states, takes 2 steps. Elaborating on his answer, for MS SQL:
select replace(replace(' count(distinct([col])) as [col],',
'[tab]', table_name
), '[col]', column_name
)
from information_schema.columns
where table_name = 'your_table';
Copy the results and paste them in a new query window between.
SELECT
[[results query 1]]
FROM your_table
Remember to delete the last ',' from query 1 results.
Replace [table name] with the table you need counts for.
DECLARE #table varchar(100) = '[table name]'
DECLARE #i INT = 1, #cntOUT int, #SQL nvarchar(500) = ''
DECLARE #ParmDef nvarchar(500) = N'#cnt int OUTPUT';
SELECT column_id, name, 0 as record_count
INTO #T1
FROM sys.all_columns c
WHERE c.object_id = (SELECT object_id FROM sys.objects WHERE name = #table AND type = 'U')
WHILE #i <= (SELECT MAX(column_id) FROM #T1)
BEGIN
SELECT #SQL = 'SELECT #cnt = COUNT(DISTINCT ' + name + ') FROM ' + #table + ';'
FROM #T1 WHERE column_id = #i;
EXECUTE sp_executesql #stmt = #SQL, #ParmDefinition = #ParmDef, #cnt = #cntOUT OUTPUT;
UPDATE #T1 SET record_count = #cntOUT WHERE column_id = #i
SET #i = #i + 1
END
SELECT * FROM #T1
--DROP TABLE #T1
I want to alter a view using a T-SQL script dynamic. Each month I have a new table in my database, I want to include the new table in my view. My idea is to create a var inside a T-SQL procedure and then build the sql statment to create the code I will use to alter the view. With this I only need to EXEC (#SqlView). The challenge now is to get (#SqlResults) in a string. Any ideas?
SQL for the view (#SqlView)
select a, b from table01
union all
select a, b from table02
union all
select a, b from table03
SQL statement for the view code (#SqlResults)
select 'select a,b from '+so.name' union all' from sysobjects so
join sys.schemas s
On so.uid = s.schema_id
where so.xtype = 'U'
and so.name like '%table0%'
Here is another approach that doesn't use a loop.
declare #SqlResults nvarchar(max) = ''
select #SqlResults = #SqlResults + 'select a,b from ' + t.name + ' union all '
from sys.tables t
where t.name like '%table0%'
select #SqlResults = 'ALTER VIEW SomeView as ' + left(#SqlResults, LEN(#SqlResults) - 10)
select #SqlResults
--Uncomment the exec line when you are comfortable
--exec sp_executesql #SqlResults
This is the SQL that I use when trying to generate a script for every table, altered to plug in your specific details:
DECLARE #SQLTable TABLE
(
ID INT IDENTITY(1,1) NOT NULL,
Sequel VARCHAR(4000) NOT NULL
)
INSERT INTO #SQLTable (Sequel)
SELECT
'SELECT A, B FROM ' + TABLE_NAME + ' UNION ALL'
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_NAME LIKE '%table0%'
DECLARE #Looper INT = (SELECT MAX(ID) FROM #SQLTable)
DECLARE #Counter INT = 0
DECLARE #ExecSQL VARCHAR(MAX) =
--'' -- use this if you just want to run the SQL
'ALTER VIEW MyView AS
' -- use this if you want to alter a view
WHILE #Counter <= #Looper
BEGIN
SET #Counter = #Counter + 1
SELECT #ExecSQL = #ExecSQL + '
' + Sequel
FROM #SQLTable
WHERE ID = #Counter
END
SET #ExecSQL =
LEFT (#ExecSQL, LEN(#ExecSQL) - 10) -- the LEFT() function is to remove the final UNION ALL
+ '
GO'
PRINT (#ExecSQL)
There is probably a way to do this with cursors, but I'm not too familiar with that syntax. What I've done in similar situations is used a few variables and either a table variable or a temp table. I also don't have much opportunity to work with dynamic sql very much, so this is a bit of a shot in the dark.
declare #view_script as varchar(4000) = 'alter view dbo.my_view as select a, b from table01'
declare #cur_table as integer = 2
declare #table_list as table (table_num integer identity(1,1), table_nm varchar(100))
insert into #table_list (table_nm)
select name
from sys.tables
where name like '%table0%'
while #cur_table <= (select max(table_num) from #table_list)
begin
select #view_script = #view_script + 'union all ' + char(10) + select a, b from ' + table_nm
from #table_list
where #table_list.table_num = #cur_table
#cur_table = #cur_table + 1
end
exec #view_script
Let me know if this works or if there was something you had to change to make it work.
I'm looking to pull a specific line from a number of table that have a field name criteria1. The problem I'm having is that when I combine the Owner and Table Name and try to call "select criteria1 from #t where linenum = 1" SQL is expecting #t to be a table. I need to know how to construct the full table name and then pass it to this query. I know I can us a programming language to access the DB but i need this to be in SQL. If someone knows of a better way of doing this that would be great too.
declare #next as varchar
declare #owner varchar
while 1=1
begin
set #next = (select top 1 o.name FROM syscolumns c inner join sysobjects o on c.id = o.id
where c.name = 'criteria1' and o.id > #next order by o.id)
if #next is null
break
else
begin
set #owner = (select top 1 u.name
FROM syscolumns c inner join
sysobjects o on c.id = o.id left join
sysusers u on o.uid=u.uid
where c.name = 'criteria1' and o.id = #next order by o.id)
declare #t as varchar
set #t = #owner+'.'+#next
select criteria1 from #t where linenum = 1
end
continue
end
You can build the entire query you want as a varchar() and then execute it with the sp_executesql stored procedure.
http://msdn.microsoft.com/en-us/library/ms188001.aspx
In your case, that bit at the end becomes
declare #sql varchar(512);
set #sql = 'select criteria1 from ' + #t + ' where linenum = 1'
sp_executesql #sql
Have you considered the following construct in a stored procedure?
CASE #tablename
WHEN 'table1' THEN SELECT * FROM table1
WHEN 'table2' THEN SELECT * FROM table2
WHEN 'table3' THEN SELECT * FROM table3
WHEN 'table4' THEN SELECT * FROM table4
END
In case you're married to dynamic SQL (considered to be a bad choice for this problem space), this guide to dynamic SQL should help a lot. It helped me and I've used dynamic SQL extensively.
Thanks for all the help. This is what I ended up with.
declare cur cursor for
select u.name + '.' + o.name tname
FROM sysobject o left join
syscolumns c on c.id = o.id left join
sysusers u on o.uid=u.uid
where c.name = 'criteria1'
declare #tn as varchar(512)
open cur
fetch next from cur into #tn
create table holding_table ( val varchar(512), table_name varchar(512))
declare #sql nvarchar(1000)
while ##FETCH_STATUS = 0
begin
set #sql = 'insert into holding_table select criteria1, ''' + #tn + ''' from ' + #tn + ' where linenum = 1'
execute sp_executesql #sql
fetch next from cur into #tn
end
close cur
deallocate cur
Maybe a view can be used here?
CREATE VIEW vCriterias
AS
SELECT 'Table1' AS TableName,
linenum,
criteria1
FROM Table1
UNION ALL
SELECT 'Table2' AS TableName,
linenum,
criteria1
FROM Table2
UNION ALL
SELECT 'Table3' AS TableName,
linenum,
criteria1
FROM Table3
go
Then selection is like:
SELECT criteria1
FROM vCriterias
WHERE linenum = 3
AND TableName IN ('Table1','Table3')