(Working with Microsoft SQL Server Management Studio)
I have a query that should, in theory, return a table containing all of the tables contained within the databases of the server to which I am connected.
However, whenever I run the query, I get the following error:
Msg 137, Level 16, State 1, Line 17
Must declare the scalar variable "#tempFinalTable".
The query itself
DECLARE #tempTableVariable TABLE (id INT NOT NULL IDENTITY(1,1), DB varchar(1000))
DECLARE #tempFinalTable TABLE (id INT NOT NULL IDENTITY(1,1), DB varchar(1000), TABLE_LOC varchar(1000))
DECLARE #DBIndex INT = 1
DECLARE #Query varchar(1000)
DECLARE #MyDB varchar(1000)
DECLARE #RowCount INT = (SELECT COUNT(*) FROM #tempTableVariable)
INSERT INTO #tempTableVariable
SELECT [name]
FROM MASTER.dbo.SYSDATABASES WITH (NOLOCK)
WHERE name NOT IN ('master', 'tempdb', 'model', 'msdb')
WHILE #DBIndex < #RowCount
BEGIN
SET #MyDB = (SELECT DB FROM #tempTableVariable WHERE id = #DBIndex)
SET #Query = 'INSERT INTO'+ #tempFinalTable + ' (DB, TABLE_LOC)
SELECT TABLE_CATALOG, CONCAT(TABLE_CATALOG, ''.'', TABLE_SCHEMA, ''.'', TABLE_NAME)
FROM ' + #MyDB + '.INFORMATION_SCHEMA.TABLES
ORDER BY TABLE_CATALOG'
EXEC(#QUERY)
SET #DBIndex = #DBIndex + 1
END
SELECT *
FROM #tempFinalTable
Any guidance as to where I have made a mistake would be greatly appreciated.
Your primary issue is that you have a syntax error, because you are trying to use the value of the table variable as part of the dynamic SQL string, but only a scalar variable can be used like that.
Even if you put the table variable within the string properly, you would still run into scoping issues, because variables don't pass over to dynamic SQL.
A much better solution is to forgo the table variable, and instead build a UNION ALL query to select it straight from dynamic SQL.
Note also:
Object names are nvarchar(128) you can use the alias sysname
You need to quote names with QUOTENAME
Don't use dbo.sysdatabases and INFORMATION_SCHEMA, they're deprecated. Instead use sys.databases and sys.tables
DECLARE #sql nvarchar(max) = (
SELECT STRING_AGG(CAST('
SELECT
DB = ' + QUOTENAME(d.name, '''') + ',
TABLE_LOC = ' + QUOTENAME(d.name, '''') + ' + ''.'' + s.name + ''.'' + t.name
FROM ' + QUOTENAME(d.name) + '.sys.tables t
JOIN ' + QUOTENAME(d.name) + '.sys.schemas s ON s.schema_id = t.schema_id
'
AS nvarchar(max)), 'UNION ALL')
FROM sys.databases d
WHERE d.database_id > 4 -- not system DB
);
PRINT #sql; --for testing
EXEC sp_executesql #sql;
I can't say whether you need QUOTENAME within the dynamic query, because I don't know what result you want, or what you intend to do with the result. If need be, you can do
TABLE_LOC = QUOTENAME(' + QUOTENAME(d.name, '''') + ') + ''.'' + QUOTENAME(s.name) + ''.'' + QUOTENAME(t.name)
This question already has answers here:
Query to list number of records in each table in a database
(23 answers)
Closed 5 years ago.
How to get a list of all tables with no of records in a particular database in SQL Server.
Thanks
Here's another option - not dependent on INFORMATION_SCHEMA.
This would also allow you to alter your where clause (you may edit your #QUERY accordingly).
DECLARE #QUERY VARCHAR(MAX)
SET #QUERY = ''
/*
* Create a long query with a row count + table name.
* You may alter your where clause here
*/
SELECT #QUERY =
#QUERY + ' SELECT COUNT(*), ''' + QUOTENAME(name)
+ ''' FROM ' + QUOTENAME(name) + CHAR(13)
+ 'UNION ALL'
FROM sys.tables
--Get rid of the last 'UNION ALL'...
SELECT #QUERY = LEFT(#QUERY, LEN(#QUERY) - 10)
--Prepare a temp table - drop if exists and then create it
IF object_id('tempdb..#TableResults') IS NOT NULL
DROP TABLE #TableResults
CREATE TABLE #TableResults(
Count INT,
TableName VARCHAR(MAX)
);
--Insert the main query result into the temp table
INSERT INTO #TableResults
EXEC(#QUERY);
--Select all from the temp table
SELECT * FROM #TableResults
WHERE COUNT = 0
You will need to use Dynamic SQL and check for existance of rows in each table
declare #sql nvarchar(max)
select #sql = isnull(#sql + ' union all ' + char(13) , convert(nvarchar(max), ''))
+ 'select tbl_name = ''' + name + ''' '
+ 'where not exists (select * from ' + quotename(name) + ')'
from sys.tables
print #sql
exec (#sql)
Did you mean this
SELECT COUNT(*) FROM
INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME IN
(
SELECT TABLE_NAME FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_TYPE = 'BASE TABLE')
Saravanan
I just came up with an idea for a piece of code to show all the distinct values for each column, and count how many records for each. I want the code to loop through all columns.
Here's what I have so far... I'm new to SQL so bear with the noobness :)
Hard code:
select [Sales Manager], count(*)
from [BT].[dbo].[test]
group by [Sales Manager]
order by 2 desc
Attempt at dynamic SQL:
Declare #sql varchar(max),
#column as varchar(255)
set #column = '[Sales Manager]'
set #sql = 'select ' + #column + ',count(*) from [BT].[dbo].[test] group by ' + #column + 'order by 2 desc'
exec (#sql)
Both of these work fine. How can I make it loop through all columns? I don't mind if I have to hard code the column names and it works its way through subbing in each one for #column.
Does this make sense?
Thanks all!
You can use dynamic SQL and get all the column names for a table. Then build up the script:
Declare #sql varchar(max) = ''
declare #tablename as varchar(255) = 'test'
select #sql = #sql + 'select [' + c.name + '],count(*) as ''' + c.name + ''' from [' + t.name + '] group by [' + c.name + '] order by 2 desc; '
from sys.columns c
inner join sys.tables t on c.object_id = t.object_id
where t.name = #tablename
EXEC (#sql)
Change #tablename to the name of your table (without the database or schema name).
This is a bit of an XY answer, but if you don't mind hardcoding the column names, I suggest you do just that, and avoid dynamic SQL - and the loop - entirely. Dynamic SQL is generally considered the last resort, opens you up to security issues (SQL injection attacks) if not careful, and can often be slower if queries and execution plans cannot be cached.
If you have a ton of column names you can write a quick piece of code or mail merge in Word to do the substitution for you.
However, as far as how to get column names, assuming this is SQL Server, you can use the following query:
SELECT c.name
FROM sys.columns c
WHERE c.object_id = OBJECT_ID('dbo.test')
Therefore, you can build your dynamic SQL from this query:
SELECT 'select '
+ QUOTENAME(c.name)
+ ',count(*) from [BT].[dbo].[test] group by '
+ QUOTENAME(c.name)
+ 'order by 2 desc'
FROM sys.columns c
WHERE c.object_id = OBJECT_ID('dbo.test')
and loop using a cursor.
Or compile the whole thing together into one batch and execute. Here we use the FOR XML PATH('') trick:
DECLARE #sql VARCHAR(MAX) = (
SELECT ' select ' --note the extra space at the beginning
+ QUOTENAME(c.name)
+ ',count(*) from [BT].[dbo].[test] group by '
+ QUOTENAME(c.name)
+ 'order by 2 desc'
FROM sys.columns c
WHERE c.object_id = OBJECT_ID('dbo.test')
FOR XML PATH('')
)
EXEC(#sql)
Note I am using the built-in QUOTENAME function to escape column names that need escaping.
You want to know the distinct coulmn values in all the columns of the table ? Just replace the table name Employee with your table name in the following code:
declare #SQL nvarchar(max)
set #SQL = ''
;with cols as (
select Table_Schema, Table_Name, Column_Name, Row_Number() over(partition by Table_Schema, Table_Name
order by ORDINAL_POSITION) as RowNum
from INFORMATION_SCHEMA.COLUMNS
)
select #SQL = #SQL + case when RowNum = 1 then '' else ' union all ' end
+ ' select ''' + Column_Name + ''' as Column_Name, count(distinct ' + quotename (Column_Name) + ' ) As DistinctCountValue,
count( '+ quotename (Column_Name) + ') as CountValue FROM ' + quotename (Table_Schema) + '.' + quotename (Table_Name)
from cols
where Table_Name = 'Employee' --print #SQL
execute (#SQL)
I have a DB that has 1000+ tables. 100 of those tables are prefixed with a three letters (let's say 'ABC') Only half of those prefixed tables have MODIFIEDDATETIME column.
I'm trying to do a simple select query to get all the last updated MODIFIEDDATETIME stamp for each Table that actually has a MODIFIEDDATETIME on that table and also begins with the three letter prefix.
I've tried using this function but it doesn't seem to be getting me there. Thoughts?
sp_msforeachtable '
select ''?'', modifieddatetime, count(*)
from ?
where ? like ''%ABC%''
group by modifieddatetime
order by modifieddatetime desc
'
Borrowing from another answer earlier today:
For one, I recommend staying away from undocumented and unsupported
procedures like sp_MSForEachTable. They can be changed or even
removed from SQL Server at any time, and this specific procedure may
have the same symptoms reported by many against sp_MSForEachDb. (See
some background here and here.)
...but also see sp_ineachdb.
Here is how I would do it - most importantly, pull the row count from the metadata which - while not 100% accurate to the millisecond is usually close enough - will not bog down your system performing a scan of every single table:
DECLARE #sql NVARCHAR(MAX);
SELECT #sql = N'';
CREATE TABLE #x
(
[table] NVARCHAR(255),
updated DATETIME,
[rowcount] BIGINT
);
SELECT #sql = #sql + N'INSERT #x SELECT '''
+ QUOTENAME(OBJECT_SCHEMA_NAME([object_id]))
+ '.' + QUOTENAME(OBJECT_NAME([object_id])) + ''',
MAX(MODIFIEDDATETIME), (SELECT SUM(rows) FROM sys.partitions
WHERE [object_id] = ' + CONVERT(VARCHAR(12), [object_id])
+ ') FROM ' + QUOTENAME(OBJECT_SCHEMA_NAME([object_id]))
+ '.' + QUOTENAME(OBJECT_NAME([object_id])) + ';'
FROM sys.columns
WHERE UPPER(name) = 'MODIFIEDDATETIME'
AND UPPER(OBJECT_NAME([object_id])) LIKE 'ABC%';
EXEC sp_executesql #sql;
SELECT [table],updated,[rowcount] FROM #x;
DROP TABLE #x;
That said, I don't know if using MAX(MODIFIEDDATETIME) is appropriate for knowing when a table was touched last. What if a transaction failed? What if the last operation was a delete?
You could do it with dynamic SQL, but this will probably not be very efficient on 1000 tables!
DECLARE #SQL NVARCHAR(MAX) = ''
SELECT #SQL = #SQL + ' UNION SELECT COUNT(' + QUOTENAME(Column_Name) + ') [Rows], MAX(' + QUOTENAME(Column_Name) + ') [MaxModifiedDate], ''' + QUOTENAME(Table_Schema) + '.' + QUOTENAME(Table_Name) + ''' [TableName] FROM ' + QUOTENAME(Table_Schema) + '.' + QUOTENAME(Table_Name)
FROM INFORMATION_SCHEMA.COLUMNS
WHERE Column_Name = 'ModifiedDateTime'
AND Table_Name LIKE 'ABC%'
SET #SQL = 'SELECT MaxModifiedDate, TableName, Rows FROM (' + STUFF(#SQL, 1, 7, '') + ') t ORDER BY MaxModifiedDate DESC'
print #sql
EXEC SP_EXECUTESQL #SQL
It basically builds a query like
SELECT MaxModifiedDate, TableName, Rows
FROM ( SELECT 'Table1' [TableName], MAX(ModifiedDate) [MaxModifedDate], COUNT(ModifiedDate) [Rows]
FROM Table1
UNION
SELECT 'Table2' [TableName], MAX(ModifiedDate) [MaxModifedDate], COUNT(ModifiedDate) [Rows]
FROM Table2
UNION
SELECT 'Table3' [TableName], MAX(ModifiedDate) [MaxModifedDate], COUNT(ModifiedDate) [Rows]
FROM Table3
UNION
...
) c
ORDER BY MaxModifiedDate DESC
I've looked for an answer to this, but all I can find is people asking how to search all columns of ALL tables in a database for a value. I just want to search all columns for a specific table. The code people have come up with for the all tables question is complicated and hard for me to figure out where exactly it's searching a specific table. Can somebody help me out? Thanks
Just use some third party tool. There are several that are 100% free and you can’t go wrong with any of these because they will save you a ton of time.
ApexSQL Search (searches both schema and data), SSMS Toolpack (searches schema and data but not free for SQL Server 2012), SQL Search (searches data only).
Frankly, I don’t really understand why even very experienced DBAs bother writing scripts for this if they can use some tool for free that will do the job.
I have no idea of the column types or data values you're searching for, but I'd guess you're trying to search for a substring among multiple text columns.
This is a job for Full-Text Search.
Don't waste time with LIKE '%' + #SearchStr + '%'. You have to write a lot of complicated code to support it, and that solution won't perform well anyway.
In a similar question I mentioned SQL Workbench/J.
The command that searches the database can also be limited to just one table. So even if that question was PostgreSQL specific, the tool works for SQL Server as well as far as I know.
I modified this stored proc to take a table name as the second parameter and just search that table for the data:
IF EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[dbo].[SearchOneTable]') AND type in (N'P', N'PC'))
DROP PROCEDURE [dbo].[SearchOneTable]
GO
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE PROC [dbo].[SearchOneTable]
(
#SearchStr nvarchar(100) = 'A',
#TableName nvarchar(256) = 'dbo.Alerts'
)
AS
BEGIN
CREATE TABLE #Results (ColumnName nvarchar(370), ColumnValue nvarchar(3630))
--SET NOCOUNT ON
DECLARE #ColumnName nvarchar(128), #SearchStr2 nvarchar(110)
SET #SearchStr2 = QUOTENAME('%' + #SearchStr + '%','''')
--SET #SearchStr2 = QUOTENAME(#SearchStr, '''') --exact match
SET #ColumnName = ' '
WHILE (#TableName IS NOT NULL) AND (#ColumnName IS NOT NULL)
BEGIN
SET #ColumnName =
(
SELECT MIN(QUOTENAME(COLUMN_NAME))
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = PARSENAME(#TableName, 2)
AND TABLE_NAME = PARSENAME(#TableName, 1)
AND DATA_TYPE IN ('char', 'varchar', 'nchar', 'nvarchar')
AND QUOTENAME(COLUMN_NAME) > #ColumnName
)
IF #ColumnName IS NOT NULL
BEGIN
INSERT INTO #Results
EXEC
(
'SELECT ''' + #TableName + '.' + #ColumnName + ''', LEFT(' + #ColumnName + ', 3630)
FROM ' + #TableName + ' (NOLOCK) ' +
' WHERE ' + #ColumnName + ' LIKE ' + #SearchStr2
)
END
END
SELECT ColumnName, ColumnValue FROM #Results
END
GO
Here is a solution that, like #Decker97's approach, figures out from metadata which columns are eligible for string search. Assumes 2005+. Supports text/ntext (though you shouldn't be using those anymore), char/nchar/varchar/nvarchar, and even puts the leading N on the search string where appropriate. Does not support xml columns.
What it does do slightly differently is that it returns a single resultset for each table, not for every single column, so the output is only one row per match no matter how many columns match.
DECLARE #SearchTerm nvarchar(255) = N'foo',
#TableName nvarchar(128) = NULL,
#sql nvarchar(max) = N'';
;WITH tables(obj_name, obj_id, columns) AS
(
SELECT obj_name = QUOTENAME(s.name) + N'.' + QUOTENAME(t.name),
obj_id = [object_id],
columns = (
SELECT N',' + QUOTENAME(c.name)
FROM sys.columns AS c
WHERE c.[object_id] = t.[object_id]
ORDER BY c.column_id FOR XML PATH(N''),
TYPE).value(N'./text()[1]', N'nvarchar(max)')
FROM sys.tables AS t INNER JOIN sys.schemas AS s
ON t.[schema_id] = s.[schema_id]
WHERE (t.name = #TableName OR #TableName IS NULL)
AND EXISTS
(
SELECT 1 FROM sys.columns AS c
WHERE c.[object_id] = t.[object_id]
AND c.system_type_id IN (35,99,167,175,231,239)
)
)
SELECT #sql += N'SELECT N' + char(39)
+ REPLACE(obj_name, char(39), char(39) + char(39))
+ char(39) + columns + N' FROM ' + obj_name + N' WHERE '
+ STUFF((
SELECT N' OR ' + QUOTENAME(name) + N' LIKE ' + CASE
WHEN c.system_type_id IN (99,231,239)
THEN 'N' ELSE N'' END
+ char(39) + N'%' + #SearchTerm + N'%' + char(39)
FROM sys.columns AS c WHERE c.[object_id] = tables.obj_id
AND c.system_type_id IN (35,99,167,175,231,239)
ORDER BY name FOR XML PATH(''), TYPE
).value(N'./text()[1]', N'nvarchar(max)')
+ char(59) + char(13) + char(10), 1, 4, N'')
FROM tables;
PRINT #sql;
--EXEC sys.sp_executeSQL #sql;
Depending on the number of searchable columns in your system, PRINT won't necessarily show you the full command, and you might think there is a bug in the code (or at least a bug in PRINT) that somehow truncates the text. You can increase the size of Results to Text output in SSMS settings, but that still won't be enough. You can use SELECT CONVERT(xml, #sql); instead (see this tip for more info).
If you are on SQL Server 2017 or greater
The new function STRING_AGG() allows you to simplify the code quite a bit, and if you have lots of existing code where you concatenate strings using FOR XML PATH, it can be useful to update those to more modern methods as you revisit them. So here's a version that uses STRING_AGG() in its place:
DECLARE #SearchTerm nvarchar(255) = N'foo',
#TableName nvarchar(128) = NULL,
#sql nvarchar(max) = N'';
;WITH tables(obj_name, obj_id, columns) AS
(
SELECT obj_name = QUOTENAME(s.name) + N'.' + QUOTENAME(t.name),
obj_id = [object_id],
columns = (SELECT STRING_AGG(QUOTENAME(c.name), N',')
WITHIN GROUP (ORDER BY c.column_id)
FROM sys.columns AS c WHERE c.[object_id] = t.[object_id]
AND c.system_type_id IN (35,99,167,175,231,239))
FROM sys.tables AS t INNER JOIN sys.schemas AS s
ON t.[schema_id] = s.[schema_id]
WHERE (t.name = #TableName OR #TableName IS NULL)
)
SELECT #sql += N'SELECT N' + char(39)
+ REPLACE(obj_name, char(39), char(39) + char(39))
+ char(39) + N',' + columns + N' FROM ' + obj_name + N' WHERE '
+ (SELECT STRING_AGG(QUOTENAME(name) + N' LIKE ' + CASE
WHEN c.system_type_id IN (99,231,239)
THEN 'N' ELSE N'' END
+ char(39) + N'%' + #SearchTerm + N'%' + char(39),
N' OR ') + N';' + char(13) + char(10)
FROM sys.columns AS c WHERE c.[object_id] = tables.obj_id
AND c.system_type_id IN (35,99,167,175,231,239))
FROM tables WHERE columns IS NOT NULL;
PRINT #sql;
--EXEC sys.sp_executeSQL #sql;
More dynamic SQL resources
This sounds like you just want to know which table and column some data is stored, not that you want to know that during the execution of your code, or change it. I also had this problem and this solved it:
Download your database in SQL format (using phpmyadmin, for example), open it with a text editor and search for the occurrences you want.
I have come across this issue, normally after uploading data from a CSV file where I had to modify the commas ',' in text fields so the data would load properly & once in SQL Server, the need comes to change the modified character back to a comma & it's helpful to be able to search the entire table. Greg Robidoux at mssqltips has posted a Stored Procedure that does just this, searches the Columns of a specified Table for a particular String value. You can find it along with a SPROC that does not use the cursor & more details here:
https://www.mssqltips.com/sqlservertip/1522/searching-and-finding-a-string-value-in-all-columns-in-a-sql-server-table/
I have posted the original SPROC below:
USE master
GO
CREATE PROCEDURE dbo.sp_FindStringInTable #stringToFind VARCHAR(100), #schema sysname, #table sysname
AS
DECLARE #sqlCommand VARCHAR(8000)
DECLARE #where VARCHAR(8000)
DECLARE #columnName sysname
DECLARE #cursor VARCHAR(8000)
BEGIN TRY
SET #sqlCommand = 'SELECT * FROM [' + #schema + '].[' + #table + '] WHERE'
SET #where = ''
SET #cursor = 'DECLARE col_cursor CURSOR FOR SELECT COLUMN_NAME
FROM ' + DB_NAME() + '.INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = ''' + #schema + '''
AND TABLE_NAME = ''' + #table + '''
AND DATA_TYPE IN (''char'',''nchar'',''ntext'',''nvarchar'',''text'',''varchar'')'
EXEC (#cursor)
OPEN col_cursor
FETCH NEXT FROM col_cursor INTO #columnName
WHILE ##FETCH_STATUS = 0
BEGIN
IF #where <> ''
SET #where = #where + ' OR'
SET #where = #where + ' [' + #columnName + '] LIKE ''' + #stringToFind + ''''
FETCH NEXT FROM col_cursor INTO #columnName
END
CLOSE col_cursor
DEALLOCATE col_cursor
SET #sqlCommand = #sqlCommand + #where
PRINT #sqlCommand
EXEC (#sqlCommand)
END TRY
BEGIN CATCH
PRINT 'There was an error. Check to make sure object exists.'
PRINT error_message()
IF CURSOR_STATUS('variable', 'col_cursor') <> -3
BEGIN
CLOSE col_cursor
DEALLOCATE col_cursor
END
END CATCH
I've found the best answer is just to select * from the table and then copy & paste into Excel and hit Ctrl+F
Cutesie little work-around that involves a bit less copy-paste, since the command can be produced easily using queries.
Invert the IN operator in a WHERE clause as VALUE IN <fields> (as opposed to the more common use case of FIELD IN <values>).
SELECT col_1, col_2, ... , col_n
FROM <table>
WHERE CAST(<value> AS varchar(max)) IN
(
CAST(col_1 AS varchar(max)),
CAST(col_2 AS varchar(max)),
...,
CAST(col_n AS varchar(max))
)
Since varchar is a pretty malleable data type, this becomes pretty foolproof (you can throw ISNULL/NULLIF to modify as needed), and depending on the use case can probably be used across more than one search value.
A more robust solution, using dynamic execution and PL/SQL would be to write a procedure to dynamically build a view of the target table (via reading e.g. MySQL's information_schema schema, Oracle's SYS schema, etc.), constrained to a where clause containing the input string hard-coded into a series of 'OR'-concatenated/IN clauses for filter conditions.