I am using SQL Server 2008. My question is, is it possible, given a table name, to construct a query that returns a list of databases that contain that table WITH values (meaning it is not empty)?
For example, I have a table called tbl_Name. I have 100 databases, and in 90 of them tbl_Name is empty. Can I get a list of the 10 databases where tbl_Name has values?
You can do this with a cursor, and the stored procedure sp_msforeachdb.
sp_msforeachdb is, as the name suggests, a proc that runs something for each database. You can use this to list all db's with a given table name:
sp_msforeachdb 'SELECT "?" AS db from [?].sys.tables WHERE name = ''tbl_Name'''
Inserting those records into a temp table makes it easy to iterate over in a cursor:
DROP TABLE #db_List
DROP TABLE #Not_Empty
GO
CREATE TABLE #db_List (db VARCHAR(MAX))
CREATE TABLE #Not_Empty (db VARCHAR(MAX))
GO
sp_msforeachdb 'INSERT INTO #db_List SELECT "?" AS db from [?].sys.tables WHERE name = ''tbl_Name'''
GO
SET NOCOUNT ON
DECLARE #Iterator varchar(MAX)
,#strSQL varchar(MAX)
DECLARE xyz CURSOR
FOR
--Select stuff to iterate over
SELECT *
FROM #db_List
OPEN xyz
FETCH NEXT FROM xyz
INTO #Iterator
WHILE ##FETCH_STATUS = 0
BEGIN
--Do stuff
SET #strSQL = 'INSERT INTO #Not_Empty
SELECT '''+#Iterator+'''
WHERE EXISTS (SELECT TOP 1 * FROM '+#Iterator+'..tbl_Name)
'
Exec (#strSQL)
FETCH NEXT FROM xyz
INTO #Iterator
END
CLOSE xyz
DEALLOCATE xyz
GO
SET NOCOUNT OFF
Quick summary:
Start in master DB select from sys.databases, get all the DB's, cursor thru each one (USE the DB). Use EXEC( sqlStatement ) to perform your command etc...
Now that you're in the DB (Each DB has a table named sys.objects, so your goal from the above paragraph is to simply issue a command like EXEC(USE dbName) ) list through the list of tables, select from sys.objects, look for object types of 'U' (user table). Get the name, construct your SELECT * FROM INTO #yourTemp WHERE blah blah blah insert your results in a #temp table you created at the very beginning.. Issue the SQL via EXEC()
Related
We are writing a stored procedure responsible for getting a stored procedure name and returning a result containing the stored procedure columns and their data types.
However, we bumped into a problem executing a dynamic query to return the results of stored procedure, but we can't store it in a temp table!
You can see our query below:
DECLARE #ProcName VARCHAR(100)='spGetOraganizationsList',
#ParamName VARCHAR(100),#DataType VARCHAR(20),
#Query NVARCHAR(MAX)='EXEC '+'spGetOraganizationsList '
SELECT PARAMETER_NAME,DATA_TYPE
INTO #Tmp
FROM information_schema.PARAMETERS
WHERE SPECIFIC_NAME=#ProcName
DECLARE ParamCursor CURSOR
FOR SELECT * FROM #Tmp
OPEN ParamCursor
FETCH NEXT FROM ParamCursor
INTO #ParamName,#DataType
WHILE ##FETCH_STATUS = 0
BEGIN
SET #Query=#Query+#ParamName+'=Null,'
FETCH NEXT FROM ParamCursor INTO #ParamName,#DataType
END
CLOSE ParamCursor
DEALLOCATE ParamCursor
DROP TABLE #Tmp
EXEC sp_executesql #Query
The thing is I can't store the results of it in a temp table,
and OPENROWSET does not accept variables.
I think it comes from sql concept that it doesn't trust in result of stored procedures and because of that we cannot select on it or store it in a table by 'making in query table' method.
Unless you create a table and define it's columns and sql trust to you and you insert result of it into this table for example take below situation
Create table test (name varchar(10),family varchar(20))
Insert into test
Exec sp-testResult
Now if you define wrong column for your table you will receive query runtime error .actually sql doesn't predict result of sp and leaves it to you to define result of your stored procedure.
You can certainly INSERT the results of a stored procedure into a TEMP table:
CREATE PROCEDURE PurgeMe
AS
SELECT convert(int, 1) AS DaData
UNION
SELECT convert(int, 2)
GO
CREATE TABLE #Doodles (AnInteger int)
INSERT #Doodles EXECUTE PurgeMe
SELECT * FROM #Doodles
Questions arise about the SCOPE of TEMP tables, however. You might find that in your calling routine you will not be able to see a TEMP table created within your routine.
The solution to the SCOPE problem is to do the following:
Create a minimal TEMP table (say, with one column)
Use ALTER TABLE on the TEMP table within your routine to make its schema match
your needs (this can be tricky, but it can be done)
Put data into the TEMP table
return from your routine - the calling routine will now be able to access the temp
table
If this is of interest I can make a longer post with a stored procedure to do the above. It was written to facilitate dynamic SQL
Write select query as you want in the stored procedure. You will get the result without creating temp table.
Use global temp table and dynamic OPENROWSET
DROP TABLE ##Tmp;
GO
DECLARE #ProcName VARCHAR(100)='spGetOraganizationsList',
#ParamName VARCHAR(100), #DataType VARCHAR(20),
-- Mind to specify database and schema of the SP
#Query NVARCHAR(MAX)=' EXEC [mydb].[dbo].spGetOraganizationsList ';
SELECT PARAMETER_NAME,DATA_TYPE
INTO #Tmp
FROM information_schema.PARAMETERS
WHERE SPECIFIC_NAME=#ProcName;
-- Build SP exec
DECLARE ParamCursor CURSOR
FOR SELECT * FROM #Tmp
OPEN ParamCursor
FETCH NEXT FROM ParamCursor
INTO #ParamName,#DataType
WHILE ##FETCH_STATUS = 0
BEGIN
SET #Query=#Query+#ParamName+'=Null,'
FETCH NEXT FROM ParamCursor INTO #ParamName,#DataType
END
CLOSE ParamCursor
DEALLOCATE ParamCursor
SET #Query = left(#Query, len(#Query) - 1);
-- Build ad hoc distributed query which creates ##Tmp from SP exec.
SET #Query = 'SELECT * INTO ##Tmp FROM OPENROWSET(''SQLNCLI'', ''Server=localhost;Trusted_Connection=yes;'',''' + #Query + ''')';
EXEC (#Query);
-- Created by dynamic sql `##Tmp` is availabe in the current context.
SELECT *
FROM ##Tmp;
Don't forget to enable ad hoc distributed queries first.
sp_configure 'Show Advanced Options', 1
GO
RECONFIGURE
GO
sp_configure 'Ad Hoc Distributed Queries', 1
GO
RECONFIGURE
GO
EDIT
My answer solves only one problem, storing the result of a dynamic proc call in a temp table. And there are more problems.
First, #p=null just will not compile if the type of #p is user-defined table type. You need kind of declare #t myType;
exec mySp ... ,#p=#t ....
Next is the 'cannot retrieve matadata for sp because contain dynamic query' error you commented on. Looks like you need an application, SqlClr or standalone, which would be capable to read and parse Datasets returned by procs.
Finally, if an SP contains conditional sql which can return a result set of different schema depending on parameter values, the result of all those efforts is still questionable.
In C#, you can use an SqlDataReader or a DataTable to get the results from a stored procedure without knowing the schema beforehand. If you then want to write that data to a temporary table, I think you can do that from C# (though I've never tried to do it).
I am working in an environment where many users have the same (or almost identical) test database set up on a common MSSQL server. We are talking about well over 100 databases for testing purposes. And at the very least, 95+% of them will contain the table I am trying to target.
These test databases are only filled with junk data - I will not be impacting anyone by doing any kind of a search. I am looking at one table, specifically, and I need to determine if any test database has that table actually containing any data at all. It doesn’t matter what the data is, I just need to find a table actually containing any data, so I can determine why that data exists in the first place. (This DB is quite old - almost two decades, so sometimes no-one has a clear answer why something in it exists).
I have been trying to build an SQL statement that iterates through all the databases, and checks that particular table specifically to see if it has any content, to bring back a list of databases that have that table containing data.
So to be specific: I need to find all databases where a specific table has any content at all (COUNT(*) > 0). Right now totally stuck with not much of any clues as to how to proceed.
In both methods replace <tablename> with the table name
Using sp_foreachdb
You can use sp_foreachDb
CREATE TABLE ##TBLTEMP(dbname varchar(100), rowscount int)
DECLARE #command varchar(4000)
SELECT #command =
'if exists(select 1 from [?].INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME =''<TABLE NAME>'') insert into ##TBLTEMP(dbname,rowscount) select ''[?]'',count(*) from [?].dbo.<tablename>'
EXEC sp_MSforeachdb #command
SELECT * FROM ##TBLTEMP WHERE rowscount > 0
DROP TABLE ##TBLTEMP
Using CURSOR
CREATE TABLE ##TBLTEMP(dbname varchar(100), rowscount int)
DECLARE #dbname Varchar(100), #strQuery varchar(4000)
DECLARE csr CURSOR FOR SELECT [name] FROM sys.databases
FETCH NEXT FROM csr INTO #dbname
WHILE ##FETCH_STATUS = 0
BEGIN
SET #strQuery = 'if exists(select 1 from [' + #dbname +'].INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME =''<TABLE NAME>'') INSERT INTO ##TBLTEMP(dbname,rowscount) SELECT ''' + #dbname + '' ', COUNT(*) FROM [' + #dbname + '].[dbo].<table name>'
EXEC(#strQuery)
FETCH NEXT FROM csr INTO #dbname
END
CLOSE csr
DEALLOCATE csr
SELECT * FROM ##TBLTEMP where rowscount > 0
References
Sp MSforeachDB
Run same command on all SQL Server databases without cursors
DECLARE CURSOR (Transact-SQL)
The db being accessed is on Snowflake; not certain on the storage details behind the scenes.
I have a query right now that creates a new view from 41 data tables stored in separate schemas under the same database, looks something like this:
CREATE VIEW all_data AS
SELECT * FROM db.schema1.data UNION ALL
SELECT * FROM db.schema2.data UNION ALL
SELECT * FROM db.schema3.data
This query is run daily. My issue is I get new data tables added every few days and I have to go manually edit the query to include those new tables, as they're stored under separate schemas (and the naming scheme for the schemas isn't consistent either, for reasons outside my control). Is there a way I can select all the schemas inside a database with a subquery that would allow me to run the query daily without needing manual updates when new schemas + tables are added?
I'd like the resulting query to have a structure somewhat like
CREATE VIEW all_data as
SELECT * FROM [SELECT schemas from db].data
but not sure how that would work, and how to union the resulting data correctly.
Unfortunately, in Snowflake you can't dynamically construct SQL statements (yet). You can of course do what you want to achieve via a script in one of the supported languages (e.g. Python, JS), by first finding all the schemas and then constructing a full SQL statement.
Hope this helps.
You can definitely query the table and schema list available. SQL Authority has a good article on it:
http://blog.sqlauthority.com/2009/06/17/sql-server-list-schema-name-and-table-name-for-database/
In short the query winds up being along these lines to pull the list of tables and schema:
SELECT '['+SCHEMA_NAME(schema_id)+'].['+name+']'
AS SchemaTable
FROM sys.tables
Though you will have to add a database name to the where clause to point to the proper DB.
With the release of Snowflake Scripting dynamic recreation of the view inside Snowflake is now very possible.
create database dynamic_views;
create schema dynamic_views.schema_base;
create schema dynamic_views.schema1;
create table dynamic_views.schema1.data(id int) as select * from values (1);
We can use the INFORMATION_SCEMA.TABLES to find all DATA tables:
SELECT table_schema
FROM dynamic_views.information_schema.tables
WHERE table_name = 'DATA';
TABLE_SCHEMA
SCHEMA1
and now push that into a cursor and build up a view creation SQL
This SQL needs to be run in the new Snowsight Console see (Working with Classic Console):
declare
sql text;
add_union boolean := false;
c1 cursor for SELECT TABLE_SCHEMA
FROM dynamic_views.information_schema.TABLES
WHERE TABLE_NAME = 'DATA';
begin
sql := 'CREATE OR REPLACE VIEW dynamic_views.schema_base.all_data AS ';
for record in c1 do
if (add_union) then
sql := sql || 'UNION ALL ';
end if;
sql := sql || 'SELECT * FROM dynamic_views.'|| record.TABLE_SCHEMA ||'.data ';
add_union := true;
end for;
EXECUTE IMMEDIATE sql;
return sql;
end;
;
and we can use it:
select * from dynamic_views.schema_base.all_data;
ID
1
and add more:
create schema dynamic_views.schema2;
create table dynamic_views.schema2.data(id int) as select * from values (2);
rebuild:
anonymous block
CREATE OR REPLACE VIEW dynamic_views.schema_base.all_data AS SELECT * FROM dynamic_views.SCHEMA1.data UNION ALL SELECT * FROM dynamic_views.SCHEMA2.data
use it again:
select * from dynamic_views.schema_base.all_data;
ID
1
2
Note: You should not use SELECT * in production as the order of the table columns will be dependent of the create orders, and if newer tables have a different shape you view will become invalid.
So the explicit form really should be used:
'SELECT column1, column2, column4 FROM dynamic_views.'|| record.TABLE_SCHEMA ||'.data ';
for anyone who want the solution for this question this is my Idea with useing FETCH
Declare #str nvarchar(maX)
Declare #i int
Set #i =(Select max(id ) from Clinics );
set #str='';
declare #Id int
declare cur CURSOR LOCAL for
select [Id] from [dbo].[Clinics]
GROUP BY [Id]
open cur
fetch next from cur into #Id
while ##FETCH_STATUS = 0 BEGIN
if #i>#id
begin
set #str=#str+ 'sELECT '+ LTRIM(RTRIM(Convert(varchar(6),#Id))) + ',* fROM ' + quotename(LTRIM(RTRIM(CONVERT(VARCHAR(8),#Id))))+'.[Clinic_Benefits] UNION ALL ';
end
else
begin
set #str=#str+ 'sELECT '+ LTRIM(RTRIM(Convert(varchar(6),#Id))) + ',* fROM ' + quotename(LTRIM(RTRIM(CONVERT(VARCHAR(8),#Id))))+'.[Clinic_Benefits] ';
end
fetch next from cur into #Id
END
close cur
deallocate cur
print #str;
exec (#str);
My COLUMNS can contain only three values or var chars - economy, basic, luxury. I want to select a ROW and display only those COLUMNS which contain luxury. The problem is that there are many such columns - about 50. I don't want to type the names of all those columns in my select query. Is there a shorter and simpler alternative to this ? Which query should I use ?
I am thinking of something like this (this is a FAKE query) -
#declare Column_Name varchar(30)
select Column_Name where Column_Value = 'luxury'
from ATable
where rowId = 'row 5';
Table structure -
rowId | Column1 | Column2 | Column3.....
I've created a stored procedure for you.
This procedure examines the MSSQL meta to build a dynamic SQL string that returns a result containing column names N and their values V, and the corresponding row key K from which that value was retrieved, for a specified table.
When this is executed, the results stored in a global temporary table called ##ColumnsByValue, which can then be queried directly.
Create the GetColumnsByValue stored procedure, by executing this script:
-- =============================================
-- Author: Ben Roberts (sepster#internode.on.net)
-- Create date: 22 Mar 2013
-- Description: Returns the names of columns that contain the specified value, for a given row
-- =============================================
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
IF OBJECT_ID ( 'dbo.GetColumnsByValue', 'P' ) IS NOT NULL
DROP PROCEDURE dbo.GetColumnsByValue;
GO
CREATE PROCEDURE dbo.GetColumnsByValue
-- Add the parameters for the stored procedure here
#idColumn sysname,
#valueToFind nvarchar(255),
#dbName sysname,
#tableName sysname,
#schemaName sysname,
#debugMode int = 0
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from interfering with SELECT statements.
SET NOCOUNT ON;
DECLARE #SQL nvarchar(max);
DECLARE #SQLUnion nvarchar(max);
DECLARE #colName sysname;
DECLARE #dbContext nvarchar(256);
DECLARE #Union nvarchar(10);
SELECT #dbContext = #dbName + '.' + #schemaName + '.sp_executeSQL';
SELECT #SQLUnion = '';
SELECT #Union = '';
IF OBJECT_ID ( 'tempdb..##GetColumnsByValueIgnoreList') IS NULL -- no columns to ingore have been specified, need to create an empty list.
BEGIN
CREATE TABLE ##GetColumnsByValueIgnoreList (column_name nvarchar(255));
END
DECLARE DBcursor CURSOR FOR
SELECT
COLUMN_NAME
FROM
INFORMATION_SCHEMA.COLUMNS
WHERE
TABLE_NAME = #tableName
AND
TABLE_SCHEMA = #schemaName;
OPEN DBcursor;
FETCH DBcursor INTO #colName;
WHILE (##FETCH_STATUS = 0)
BEGIN
IF (
#colName != #idColumn
AND
#colName NOT IN (SELECT column_name FROM ##GetColumnsByValueIgnoreList)
)
BEGIN
SELECT #SQL = 'SELECT '+#idColumn+' as K, '''+#colName+''' as N, ' +#colName+ ' as V FROM ' + #dbName + '.' + #schemaName + '.' + #tableName;
--PRINT #SQL;
SELECT #SQLUnion = #SQL + #Union + #SQLUnion;
SELECT #Union = ' UNION ';
END
FETCH DBcursor INTO #colName;
END; -- while
CLOSE DBcursor; DEALLOCATE DBcursor;
IF (#debugMode != 0)
BEGIN
PRINT #SQLUnion;
PRINT #dbContext;
END
ELSE
BEGIN
-- Delete the temp table if it has already been created.
IF OBJECT_ID ('tempdb..##ColumnsByValue') IS NOT NULL
BEGIN
DROP TABLE ##ColumnsByValue
END
-- Create a new temp table
CREATE TABLE ##ColumnsByValue (
K nvarchar(255), -- Key
N nvarchar(255), -- Column Name
V nvarchar(255) -- Column Value
)
-- Populate it with the results from our dynamically generated SQL.
INSERT INTO ##ColumnsByValue EXEC #dbContext #SQLUnion;
END
END
GO
The SP takes several inputs as parameters, these are explained in the following code.
Note also I've provided a mechanism to add an "ignore list" as an input:
This allows you to list any column names that should not be included
in the results.
You do NOT need to add the columnn that you're using as your key, ie the row_id from your example structure.
You MUST include other columns that are not varchar as
these will cause an error (as the SP just does a varchar comparison
on all columns it looks at).
This is done via a temp table that you must create/populate
Your example table structure suggests
the table contains only columns of interest, so this may not apply to
you.
I've included example code for how to do this (but only do this if you need to):
IF OBJECT_ID ( 'tempdb..##GetColumnsByValueIgnoreList') IS NOT NULL
BEGIN
DROP TABLE ##GetColumnsByValueIgnoreList;
END
CREATE TABLE ##GetColumnsByValueIgnoreList (column_name nvarchar(255));
INSERT INTO ##GetColumnsByValueIgnoreList VALUES ('a_column');
INSERT INTO ##GetColumnsByValueIgnoreList VALUES ('another_column');
INSERT INTO ##GetColumnsByValueIgnoreList VALUES ('yet_another_column');
Now, to fire off the procedure that build your temp table of results, use the following code (and modify as appropriate, of course).
-- Build the ##ColumnsByValue table
EXEC dbo.GetColumnsByValue
#idColumn = 'row_id', -- The name of the column that contains your row ID (eg probably your PK column)
#dbName = 'your_db_name',
#tableName = 'your_table_name',
#schemaName = 'dbo',
#debugMode = 0 -- Set this to 1 if you just want a print out of the SQL used to build the temp table, to 0 if you want the temp table populated
This leaves you with ##ColumnsByValue, on which you can perform whatever search you need, eg:
select * from ##ColumnsByValue WHERE v = 'luxury' and k = 5 --some_row_id
You'd need to re-execute the stored procedure (and if relevant, create/modify the ignore list table prior to it) for each table you want to examine.
A concern with this approach is the nvarchar length might get exceeded in your case. You'd prob. need to use different datatype, reduce the column name lengths etc. Or break it up into sub-steps and union the results together to get the resultset you're after.
Another concern I have is that this is complete overkill for your particular scenario, where a one-off script-to-query-window will give you the basis of what you need, then some clever text editing in eg Notepad++ will get you all the way there... and hence this problem will likely (and quite reasonably) put you off doing it this way! But it is a good general-case question, and so deserves an answer for anyone interested in future ;-)
I've searched here and elsewhere, and haven't found an answer yet. Hope I didn't miss it.
Using SQL Server Management Studio 2008 R2.
I have n specific databases on my server (there are other DBs as well, but I'm only interested in some of them)
Each of these databases has a table within it, which all have the same name. The only difference is the DB name. I want to aggregate these tables together to make one big table on a different database (different to the other DBs).
I can get the db names from the results of a query.
N is unknown.
Is a loop the way to go about this?
I was thinking something along the lines of the following pseudocode:
Set #dbnames = SELECT DISTINCT dbname FROM MyServer.dbo.MyTable
For each #name in #dbnames
INSERT INTO ADifferentDB.dbo.MyOtherTable
SELECT * FROM #name.dbo.table
Next name
(Clearly I'm new to using SQL variable as well, as you can see)
Your first problem is about iterating the databases: you cand do that with a cursor
Then you have another problem, executing a query where part of it is variable (database's name). You can do that with execute function.
All that is something similar to this:
DECLARE #query VARCHAR(max)
DECLARE #dbname VARCHAR(100)
DECLARE my_db_cursor CURSOR
FOR SELECT DISTINCT dbname FROM MyServer.dbo.MyTable
OPEN my_db_cursor
FETCH NEXT FROM my_db_cursor
INTO #dbname
WHILE ##FETCH_STATUS = 0
BEGIN
SET #query = 'INSERT INTO ADifferentDB.dbo.MyOtherTable
SELECT * FROM ' + #dbname + '.dbo.table'
EXECUTE(#query)
FETCH NEXT FROM my_db_cursor
INTO #dbname
END
CLOSE my_db_cursor
DEALLOCATE my_db_cursor
what you want to do is define a CURSOR for row-level operation. here is some doc
I would suggest using sp_MSForEachDB:
EXEC sp_MSForEachDB '
-- Include only the databases you care about.
IF NOT EXISTS (
SELECT *
FROM MySever.dbo.MyTable
WHERE dbname = ''?''
)
-- Exit if the database is not in your table.
RETURN
-- Otherwise, perform your insert.
INSERT INTO ADifferentDB.dbo.MyOtherTable
SELECT * FROM ?.dbo.table
'
In this case, ? is a token that is replaced with each database on the server.