Given a number, how do I discover in what table and column it could be found within?
I don't care if it's fast, it just needs to work.
This might help you. - from Narayana Vyas. It searches all columns of all tables in a given database. I have used it before and it works.
This is the Stored Proc from the above link - the only change I made was substituting the temp table for a table variable so you don't have to remember to drop it each time.
CREATE PROC SearchAllTables
(
#SearchStr nvarchar(100)
)
AS
BEGIN
-- Copyright © 2002 Narayana Vyas Kondreddi. All rights reserved.
-- Purpose: To search all columns of all tables for a given search string
-- Written by: Narayana Vyas Kondreddi
-- Site: http://vyaskn.tripod.com
-- Tested on: SQL Server 7.0 and SQL Server 2000
-- Date modified: 28th July 2002 22:50 GMT
DECLARE #Results TABLE(ColumnName nvarchar(370), ColumnValue nvarchar(3630))
SET NOCOUNT ON
DECLARE #TableName nvarchar(256), #ColumnName nvarchar(128), #SearchStr2 nvarchar(110)
SET #TableName = ''
SET #SearchStr2 = QUOTENAME('%' + #SearchStr + '%','''')
WHILE #TableName IS NOT NULL
BEGIN
SET #ColumnName = ''
SET #TableName =
(
SELECT MIN(QUOTENAME(TABLE_SCHEMA) + '.' + QUOTENAME(TABLE_NAME))
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_TYPE = 'BASE TABLE'
AND QUOTENAME(TABLE_SCHEMA) + '.' + QUOTENAME(TABLE_NAME) > #TableName
AND OBJECTPROPERTY(
OBJECT_ID(
QUOTENAME(TABLE_SCHEMA) + '.' + QUOTENAME(TABLE_NAME)
), 'IsMSShipped'
) = 0
)
WHILE (#TableName IS NOT NULL) AND (#ColumnName IS NOT NULL)
BEGIN
SET #ColumnName =
(
SELECT MIN(QUOTENAME(COLUMN_NAME))
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = PARSENAME(#TableName, 2)
AND TABLE_NAME = PARSENAME(#TableName, 1)
AND DATA_TYPE IN ('char', 'varchar', 'nchar', 'nvarchar')
AND QUOTENAME(COLUMN_NAME) > #ColumnName
)
IF #ColumnName IS NOT NULL
BEGIN
INSERT INTO #Results
EXEC
(
'SELECT ''' + #TableName + '.' + #ColumnName + ''', LEFT(' + #ColumnName + ', 3630)
FROM ' + #TableName +
' WHERE ' + #ColumnName + ' LIKE ' + #SearchStr2
)
END
END
END
SELECT ColumnName, ColumnValue FROM #Results
END
To execute the stored procedure :
EXEC SearchAllTables 'YourStringHere'
If you need to run such search only once then you can probably go with any of the scripts already shown in other answers. But otherwise, I’d recommend using ApexSQL Search for this. It’s a free SSMS addin and it really saved me a lot of time.
Before running any of the scripts you should customize it based on the data type you want to search. If you know you are searching for datetime column then there is no need to search through nvarchar columns. This will speed up all of the queries above.
Based on bnkdev's answer I modified Narayana's Code to search all columns even numeric ones.
It'll run slower, but this version actually finds all matches not just those found in text columns.
I can't thank this guy enough. Saved me days of searching by hand!
CREATE PROC SearchAllTables
(
#SearchStr nvarchar(100)
)
AS
BEGIN
-- Copyright © 2002 Narayana Vyas Kondreddi. All rights reserved.
-- Purpose: To search all columns of all tables for a given search string
-- Written by: Narayana Vyas Kondreddi
-- Site: http://vyaskn.tripod.com
-- Tested on: SQL Server 7.0 and SQL Server 2000
-- Date modified: 28th July 2002 22:50 GMT
CREATE TABLE #Results (ColumnName nvarchar(370), ColumnValue nvarchar(3630))
SET NOCOUNT ON
DECLARE #TableName nvarchar(256), #ColumnName nvarchar(128), #SearchStr2 nvarchar(110)
SET #TableName = ''
SET #SearchStr2 = QUOTENAME('%' + #SearchStr + '%','''')
WHILE #TableName IS NOT NULL
BEGIN
SET #ColumnName = ''
SET #TableName =
(
SELECT MIN(QUOTENAME(TABLE_SCHEMA) + '.' + QUOTENAME(TABLE_NAME))
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_TYPE = 'BASE TABLE'
AND QUOTENAME(TABLE_SCHEMA) + '.' + QUOTENAME(TABLE_NAME) > #TableName
AND OBJECTPROPERTY(
OBJECT_ID(
QUOTENAME(TABLE_SCHEMA) + '.' + QUOTENAME(TABLE_NAME)
), 'IsMSShipped'
) = 0
)
WHILE (#TableName IS NOT NULL) AND (#ColumnName IS NOT NULL)
BEGIN
SET #ColumnName =
(
SELECT MIN(QUOTENAME(COLUMN_NAME))
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = PARSENAME(#TableName, 2)
AND TABLE_NAME = PARSENAME(#TableName, 1)
AND QUOTENAME(COLUMN_NAME) > #ColumnName
)
IF #ColumnName IS NOT NULL
BEGIN
INSERT INTO #Results
EXEC
(
'SELECT ''' + #TableName + '.' + #ColumnName + ''', LEFT(CONVERT(varchar(max), ' + #ColumnName + '), 3630)
FROM ' + #TableName + ' (NOLOCK) ' +
' WHERE CONVERT(varchar(max), ' + #ColumnName + ') LIKE ' + #SearchStr2
)
END
END
END
SELECT ColumnName, ColumnValue FROM #Results
END
This is my independent take on this question that I use for my own work. It works in SQL2000 and greater, allows wildcards, column filtering, and will search most of the normal data types.
A pseudo-code description could be select * from * where any like 'foo'
--------------------------------------------------------------------------------
-- Search all columns in all tables in a database for a string.
-- Does not search: image, sql_variant or user-defined types.
-- Exact search always for money and smallmoney; no wildcards for matching these.
--------------------------------------------------------------------------------
declare #SearchTerm nvarchar(4000) -- Can be max for SQL2005+
declare #ColumnName sysname
--------------------------------------------------------------------------------
-- SET THESE!
--------------------------------------------------------------------------------
set #SearchTerm = N'foo' -- Term to be searched for, wildcards okay
set #ColumnName = N'' -- Use to restrict the search to certain columns, wildcards okay, null or empty string for all cols
--------------------------------------------------------------------------------
-- END SET
--------------------------------------------------------------------------------
set nocount on
declare #TabCols table (
id int not null primary key identity
, table_schema sysname not null
, table_name sysname not null
, column_name sysname not null
, data_type sysname not null
)
insert into #TabCols (table_schema, table_name, column_name, data_type)
select t.TABLE_SCHEMA, c.TABLE_NAME, c.COLUMN_NAME, c.DATA_TYPE
from INFORMATION_SCHEMA.TABLES t
join INFORMATION_SCHEMA.COLUMNS c on t.TABLE_SCHEMA = c.TABLE_SCHEMA
and t.TABLE_NAME = c.TABLE_NAME
where 1 = 1
and t.TABLE_TYPE = 'base table'
and c.DATA_TYPE not in ('image', 'sql_variant')
and c.COLUMN_NAME like case when len(#ColumnName) > 0 then #ColumnName else '%' end
order by c.TABLE_NAME, c.ORDINAL_POSITION
declare
#table_schema sysname
, #table_name sysname
, #column_name sysname
, #data_type sysname
, #exists nvarchar(4000) -- Can be max for SQL2005+
, #sql nvarchar(4000) -- Can be max for SQL2005+
, #where nvarchar(4000) -- Can be max for SQL2005+
, #run nvarchar(4000) -- Can be max for SQL2005+
while exists (select null from #TabCols) begin
select top 1
#table_schema = table_schema
, #table_name = table_name
, #exists = 'select null from [' + table_schema + '].[' + table_name + '] where 1 = 0'
, #sql = 'select ''' + '[' + table_schema + '].[' + table_name + ']' + ''' as TABLE_NAME, * from [' + table_schema + '].[' + table_name + '] where 1 = 0'
, #where = ''
from #TabCols
order by id
while exists (select null from #TabCols where table_schema = #table_schema and table_name = #table_name) begin
select top 1
#column_name = column_name
, #data_type = data_type
from #TabCols
where table_schema = #table_schema
and table_name = #table_name
order by id
-- Special case for money
if #data_type in ('money', 'smallmoney') begin
if isnumeric(#SearchTerm) = 1 begin
set #where = #where + ' or [' + #column_name + '] = cast(''' + #SearchTerm + ''' as ' + #data_type + ')' -- could also cast the column as varchar for wildcards
end
end
-- Special case for xml
else if #data_type = 'xml' begin
set #where = #where + ' or cast([' + #column_name + '] as nvarchar(max)) like ''' + #SearchTerm + ''''
end
-- Special case for date
else if #data_type in ('date', 'datetime', 'datetime2', 'datetimeoffset', 'smalldatetime', 'time') begin
set #where = #where + ' or convert(nvarchar(50), [' + #column_name + '], 121) like ''' + #SearchTerm + ''''
end
-- Search all other types
else begin
set #where = #where + ' or [' + #column_name + '] like ''' + #SearchTerm + ''''
end
delete from #TabCols where table_schema = #table_schema and table_name = #table_name and column_name = #column_name
end
set #run = 'if exists(' + #exists + #where + ') begin ' + #sql + #where + ' print ''' + #table_name + ''' end'
print #run
exec sp_executesql #run
end
set nocount off
I don't put it in proc form since I don't want to maintain it across hundreds of DBs and it's really for ad-hoc work anyway. Please feel free to comment on bug-fixes.
I optimized Allain Lalonde answer (https://stackoverflow.com/a/436676/412368).
Numeric values are still supported. Should be roughly 4-5 times faster (1:03 vs 4:30), tested on a desktop with a 7GB database. http://developer.azurewebsites.net/2015/01/mssql-searchalltables/
IF OBJECT_ID ('dbo.SearchAllTables', 'P') IS NOT NULL
DROP PROCEDURE dbo.SearchAllTables;
GO
CREATE PROC SearchAllTables
(
#SearchStr nvarchar(100)
)
AS
BEGIN
-- Copyright © 2002 Narayana Vyas Kondreddi. All rights reserved.
-- Purpose: To search all columns of all tables for a given search string
-- Written by: Narayana Vyas Kondreddi
-- Site: http://vyaskn.tripod.com
-- Customized and modified: 2014-01-21
-- Tested on: SQL Server 2008 R2
DECLARE #Results TABLE(ColumnName nvarchar(370), ColumnValue nvarchar(3630))
SET NOCOUNT ON
DECLARE #TableName nvarchar(256)
DECLARE #ColumnName nvarchar(128)
DECLARE #DataType nvarchar(128)
DECLARE #SearchStr2 nvarchar(110)
DECLARE #SearchDecimal decimal(38,19)
DECLARE #Query nvarchar(4000)
SET #SearchStr2 = QUOTENAME('%' + #SearchStr + '%', '''')
SET #SearchDecimal = CASE WHEN ISNUMERIC(#SearchStr) = 1 THEN CONVERT(decimal(38,19), #SearchStr) ELSE NULL END
PRINT '#SearchStr2: ' + #SearchStr2
PRINT '#SearchDecimal: ' + CAST(#SearchDecimal AS nvarchar)
SET #TableName = ''
WHILE #TableName IS NOT NULL
BEGIN
SET #ColumnName = ''
SET #TableName =
(
SELECT MIN(QUOTENAME(TABLE_SCHEMA) + '.' + QUOTENAME(TABLE_NAME))
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_TYPE = 'BASE TABLE'
AND QUOTENAME(TABLE_SCHEMA) + '.' + QUOTENAME(TABLE_NAME) > #TableName
AND OBJECTPROPERTY(
OBJECT_ID(
QUOTENAME(TABLE_SCHEMA) + '.' + QUOTENAME(TABLE_NAME)
), 'IsMSShipped'
) = 0
)
WHILE (#TableName IS NOT NULL) AND (#ColumnName IS NOT NULL)
BEGIN
SET #ColumnName =
(
SELECT MIN(QUOTENAME(COLUMN_NAME))
DATA_TYPE
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = PARSENAME(#TableName, 2)
AND TABLE_NAME = PARSENAME(#TableName, 1)
AND DATA_TYPE IN ('char', 'varchar', 'nchar', 'nvarchar',
'int', 'bigint', 'tinyint', 'numeric', 'decimal')
AND QUOTENAME(COLUMN_NAME) > #ColumnName
)
SET #DataType =
(
SELECT DATA_TYPE
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = PARSENAME(#TableName, 2)
AND TABLE_NAME = PARSENAME(#TableName, 1)
AND QUOTENAME(COLUMN_NAME) = #ColumnName
)
PRINT #TableName + '.' + #ColumnName + ' (' + #DataType + ')'
IF #ColumnName IS NOT NULL
BEGIN
IF #DataType IN ('int', 'bigint', 'tinyint', 'numeric', 'decimal')
BEGIN
IF #SearchDecimal IS NOT NULL
BEGIN
SET #Query = 'SELECT ''' + #TableName + '.' + #ColumnName + ''', LEFT(CAST(' + #ColumnName + ' AS nvarchar(110)), 3630) ' +
'FROM ' + #TableName + ' (NOLOCK) ' +
' WHERE ' + #ColumnName + ' = ' + CAST(#SearchDecimal AS nvarchar)
PRINT ' ' + #Query
INSERT INTO #Results
EXEC (#Query)
END
END
ELSE
BEGIN
SET #Query = 'SELECT ''' + #TableName + '.' + #ColumnName + ''', LEFT(' + #ColumnName + ', 3630) ' +
'FROM ' + #TableName + ' (NOLOCK) ' +
' WHERE ' + #ColumnName + ' LIKE ' + #SearchStr2
PRINT ' ' + #Query
INSERT INTO #Results
EXEC (#Query)
END
END
END
END
SELECT ColumnName, ColumnValue FROM #Results
END
I have a solution from a while ago that I kept improving. Also searches within XML columns if told to do so, or searches integer values if providing a integer only string.
/* Reto Egeter, fullparam.wordpress.com */
DECLARE #SearchStrTableName nvarchar(255), #SearchStrColumnName nvarchar(255), #SearchStrColumnValue nvarchar(255), #SearchStrInXML bit, #FullRowResult bit, #FullRowResultRows int
SET #SearchStrColumnValue = '%searchthis%' /* use LIKE syntax */
SET #FullRowResult = 1
SET #FullRowResultRows = 3
SET #SearchStrTableName = NULL /* NULL for all tables, uses LIKE syntax */
SET #SearchStrColumnName = NULL /* NULL for all columns, uses LIKE syntax */
SET #SearchStrInXML = 0 /* Searching XML data may be slow */
IF OBJECT_ID('tempdb..#Results') IS NOT NULL DROP TABLE #Results
CREATE TABLE #Results (TableName nvarchar(128), ColumnName nvarchar(128), ColumnValue nvarchar(max),ColumnType nvarchar(20))
SET NOCOUNT ON
DECLARE #TableName nvarchar(256) = '',#ColumnName nvarchar(128),#ColumnType nvarchar(20), #QuotedSearchStrColumnValue nvarchar(110), #QuotedSearchStrColumnName nvarchar(110)
SET #QuotedSearchStrColumnValue = QUOTENAME(#SearchStrColumnValue,'''')
DECLARE #ColumnNameTable TABLE (COLUMN_NAME nvarchar(128),DATA_TYPE nvarchar(20))
WHILE #TableName IS NOT NULL
BEGIN
SET #TableName =
(
SELECT MIN(QUOTENAME(TABLE_SCHEMA) + '.' + QUOTENAME(TABLE_NAME))
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_TYPE = 'BASE TABLE'
AND TABLE_NAME LIKE COALESCE(#SearchStrTableName,TABLE_NAME)
AND QUOTENAME(TABLE_SCHEMA) + '.' + QUOTENAME(TABLE_NAME) > #TableName
AND OBJECTPROPERTY(OBJECT_ID(QUOTENAME(TABLE_SCHEMA) + '.' + QUOTENAME(TABLE_NAME)), 'IsMSShipped') = 0
)
IF #TableName IS NOT NULL
BEGIN
DECLARE #sql VARCHAR(MAX)
SET #sql = 'SELECT QUOTENAME(COLUMN_NAME),DATA_TYPE
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = PARSENAME(''' + #TableName + ''', 2)
AND TABLE_NAME = PARSENAME(''' + #TableName + ''', 1)
AND DATA_TYPE IN (' + CASE WHEN ISNUMERIC(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(#SearchStrColumnValue,'%',''),'_',''),'[',''),']',''),'-','')) = 1 THEN '''tinyint'',''int'',''smallint'',''bigint'',''numeric'',''decimal'',''smallmoney'',''money'',' ELSE '' END + '''char'',''varchar'',''nchar'',''nvarchar'',''timestamp'',''uniqueidentifier''' + CASE #SearchStrInXML WHEN 1 THEN ',''xml''' ELSE '' END + ')
AND COLUMN_NAME LIKE COALESCE(' + CASE WHEN #SearchStrColumnName IS NULL THEN 'NULL' ELSE '''' + #SearchStrColumnName + '''' END + ',COLUMN_NAME)'
INSERT INTO #ColumnNameTable
EXEC (#sql)
WHILE EXISTS (SELECT TOP 1 COLUMN_NAME FROM #ColumnNameTable)
BEGIN
PRINT #ColumnName
SELECT TOP 1 #ColumnName = COLUMN_NAME,#ColumnType = DATA_TYPE FROM #ColumnNameTable
SET #sql = 'SELECT ''' + #TableName + ''',''' + #ColumnName + ''',' + CASE #ColumnType WHEN 'xml' THEN 'LEFT(CAST(' + #ColumnName + ' AS nvarchar(MAX)), 4096),'''
WHEN 'timestamp' THEN 'master.dbo.fn_varbintohexstr('+ #ColumnName + '),'''
ELSE 'LEFT(' + #ColumnName + ', 4096),''' END + #ColumnType + '''
FROM ' + #TableName + ' (NOLOCK) ' +
' WHERE ' + CASE #ColumnType WHEN 'xml' THEN 'CAST(' + #ColumnName + ' AS nvarchar(MAX))'
WHEN 'timestamp' THEN 'master.dbo.fn_varbintohexstr('+ #ColumnName + ')'
ELSE #ColumnName END + ' LIKE ' + #QuotedSearchStrColumnValue
INSERT INTO #Results
EXEC(#sql)
IF ##ROWCOUNT > 0 IF #FullRowResult = 1
BEGIN
SET #sql = 'SELECT TOP ' + CAST(#FullRowResultRows AS VARCHAR(3)) + ' ''' + #TableName + ''' AS [TableFound],''' + #ColumnName + ''' AS [ColumnFound],''FullRow>'' AS [FullRow>],*' +
' FROM ' + #TableName + ' (NOLOCK) ' +
' WHERE ' + CASE #ColumnType WHEN 'xml' THEN 'CAST(' + #ColumnName + ' AS nvarchar(MAX))'
WHEN 'timestamp' THEN 'master.dbo.fn_varbintohexstr('+ #ColumnName + ')'
ELSE #ColumnName END + ' LIKE ' + #QuotedSearchStrColumnValue
EXEC(#sql)
END
DELETE FROM #ColumnNameTable WHERE COLUMN_NAME = #ColumnName
END
END
END
SET NOCOUNT OFF
SELECT TableName, ColumnName, ColumnValue, ColumnType, COUNT(*) AS Count FROM #Results
GROUP BY TableName, ColumnName, ColumnValue, ColumnType
Source:
http://fullparam.wordpress.com/2012/09/07/fck-it-i-am-going-to-search-all-tables-all-collumns/
It's my way to resolve this question. Tested on SQLServer2008R2
CREATE PROC SearchAllTables
#SearchStr nvarchar(100)
AS
BEGIN
DECLARE #dml nvarchar(max) = N''
IF OBJECT_ID('tempdb.dbo.#Results') IS NOT NULL DROP TABLE dbo.#Results
CREATE TABLE dbo.#Results
([tablename] nvarchar(100),
[ColumnName] nvarchar(100),
[Value] nvarchar(max))
SELECT #dml += ' SELECT ''' + s.name + '.' + t.name + ''' AS [tablename], ''' +
c.name + ''' AS [ColumnName], CAST(' + QUOTENAME(c.name) +
' AS nvarchar(max)) AS [Value] FROM ' + QUOTENAME(s.name) + '.' + QUOTENAME(t.name) +
' (NOLOCK) WHERE CAST(' + QUOTENAME(c.name) + ' AS nvarchar(max)) LIKE ' + '''%' + #SearchStr + '%'''
FROM sys.schemas s JOIN sys.tables t ON s.schema_id = t.schema_id
JOIN sys.columns c ON t.object_id = c.object_id
JOIN sys.types ty ON c.system_type_id = ty.system_type_id AND c .user_type_id = ty .user_type_id
WHERE t.is_ms_shipped = 0 AND ty.name NOT IN ('timestamp', 'image', 'sql_variant')
INSERT dbo.#Results
EXEC sp_executesql #dml
SELECT *
FROM dbo.#Results
END
Thanks for the really useful script.
You may need to add the following modification to the code if your tables have non-convertable fields:
SET #ColumnName =
(
SELECT MIN(QUOTENAME(COLUMN_NAME))
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = PARSENAME(#TableName, 2)
AND TABLE_NAME = PARSENAME(#TableName, 1)
AND DATA_TYPE NOT IN ('text', 'image', 'ntext')
AND QUOTENAME(COLUMN_NAME) > #ColumnName
)
Chris
Here, very sweet and small solution:
1) create a store procedure:
create procedure get_table
#find_str varchar(50)
as
begin
declare #col_name varchar(500), #tab_name varchar(500);
declare #find_tab TABLE(table_name varchar(100), column_name varchar(100));
DECLARE tab_col cursor for
select C.name as 'col_name', T.name as tab_name
from sys.tables as T
left outer join sys.columns as C on C.object_id=T.object_id
left outer join sys.types as TP on C.system_type_id=TP.system_type_id
where type='U'
and TP.name in('text','ntext','varchar','char','nvarchar','nchar');
open tab_col
fetch next from tab_col into #col_name, #tab_name
while ##FETCH_STATUS = 0
begin
insert into #find_tab
exec('select ''' + #tab_name + ''',''' + #col_name + ''' from ' + #tab_name +
' where ' + #col_name + '=''' + #find_str + ''' group by ' +
#col_name + ' having count(*)>0');
fetch next from tab_col into #col_name, #tab_name;
end
CLOSE tab_col;
DEALLOCATE tab_col;
select table_name, column_name from #find_tab;
end
==========================
2) call procedure by calling store procedure:
exec get_table 'serach_string';
If you have phpMyAdmin installed use its Search feature.
Select your DataBase.
Be sure you do have selected DataBase, not a table, otherwise you'll get a completely different search dialog.
Click Search tab
List item Choose the search term you want
Choose the tables to search
Another way using JOIN and CURSOR:
USE My_Database;
-- Store results in a local temp table so that. I'm using a
-- local temp table so that I can access it in SP_EXECUTESQL.
create table #tmp (
tbl nvarchar(max),
col nvarchar(max),
val nvarchar(max)
);
declare #tbl nvarchar(max);
declare #col nvarchar(max);
declare #q nvarchar(max);
declare #search nvarchar(max) = 'my search key';
-- Create a cursor on all columns in the database
declare c cursor for
SELECT tbls.TABLE_NAME, cols.COLUMN_NAME FROM INFORMATION_SCHEMA.TABLES AS tbls
JOIN INFORMATION_SCHEMA.COLUMNS AS cols
ON tbls.TABLE_NAME = cols.TABLE_NAME
-- For each table and column pair, see if the search value exists.
open c
fetch next from c into #tbl, #col
while ##FETCH_STATUS = 0
begin
-- Look for the search key in current table column and if found add it to the results.
SET #q = 'INSERT INTO #tmp SELECT ''' + #tbl + ''', ''' + #col + ''', ' + #col + ' FROM ' + #tbl + ' WHERE ' + #col + ' LIKE ''%' + #search + '%'''
EXEC SP_EXECUTESQL #q
fetch next from c into #tbl, #col
end
close c
deallocate c
-- Get results
select * from #tmp
-- Remove local temp table.
drop table #tmp
You might need to build an inverted index for your database. It is assured to be pretty fast.
-- exec pSearchAllTables 'M54*'
ALTER PROC pSearchAllTables (#SearchStr NVARCHAR(100))
AS
BEGIN
-- A procedure to search all tables in a database for a value
-- Note: Use * or % for wildcard
DECLARE
#Results TABLE([Schema.Table.ColumnName] NVARCHAR(370), ColumnValue NVARCHAR(3630))
SET NOCOUNT ON
DECLARE
#TableName NVARCHAR(256) = ''
, #ColumnName NVARCHAR(128)
, #SearchStr2 NVARCHAR(110) = QUOTENAME(REPLACE(#SearchStr, '*', '%'), '''')
WHILE #TableName IS NOT NULL
BEGIN
SET #ColumnName = ''
SET #TableName =
(
SELECT MIN(QUOTENAME(TABLE_SCHEMA) + '.' + QUOTENAME(TABLE_NAME))
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_TYPE = 'BASE TABLE'
AND QUOTENAME(TABLE_SCHEMA) + '.' + QUOTENAME(TABLE_NAME) > #TableName
AND OBJECTPROPERTY(OBJECT_ID(QUOTENAME(TABLE_SCHEMA) + '.' + QUOTENAME(TABLE_NAME)), 'IsMSShipped') = 0
)
WHILE (#TableName IS NOT NULL) AND (#ColumnName IS NOT NULL)
BEGIN
SET #ColumnName =
(
SELECT MIN(QUOTENAME(COLUMN_NAME))
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = PARSENAME(#TableName, 2)
AND TABLE_NAME = PARSENAME(#TableName, 1)
AND DATA_TYPE IN ('char', 'varchar', 'nchar', 'nvarchar')
AND QUOTENAME(COLUMN_NAME) > #ColumnName
)
IF #ColumnName IS NOT NULL
BEGIN
INSERT INTO #Results
EXEC ('SELECT ''' + #TableName + '.' + #ColumnName + ''', LEFT(' + #ColumnName + ', 3630) FROM ' + #TableName + ' (NOLOCK) WHERE ' + #ColumnName + ' LIKE ' + #SearchStr2)
END
END
END
SELECT
[Schema.Table.ColumnName]
, ColumnValue
FROM #Results
GROUP BY
[Schema.Table.ColumnName]
, ColumnValue
END
For Development purpose you can just export the required tables data into a single HTML and make a direct search on it.
Suppose if you want to get all the table with name a column name contain logintime in the database MyDatabase below is the code sample
use MyDatabase
SELECT t.name AS table_name,
SCHEMA_NAME(schema_id) AS schema_name,
c.name AS column_name
FROM sys.tables AS t
INNER JOIN sys.columns c ON t.OBJECT_ID = c.OBJECT_ID
WHERE c.name LIKE '%logintime%'
ORDER BY schema_name, table_name;
I was looking for a just a numeric value = 6.84 - using the other answers here I was able to limit my search to this
Declare #sourceTable Table(id INT NOT NULL IDENTITY PRIMARY KEY, table_name varchar(1000), column_name varchar(1000))
Declare #resultsTable Table(id INT NOT NULL IDENTITY PRIMARY KEY, table_name varchar(1000))
Insert into #sourceTable(table_name, column_name)
select schema_name(t.schema_id) + '.' + t.name as[table], c.name as column_name
from sys.columns c
join sys.tables t
on t.object_id = c.object_id
where type_name(user_type_id) in ('decimal', 'numeric', 'smallmoney', 'money', 'float', 'real')
order by[table], c.column_id;
DECLARE db_cursor CURSOR FOR
Select table_name, column_name from #sourceTable
DECLARE #mytablename VARCHAR(1000);
DECLARE #mycolumnname VARCHAR(1000);
OPEN db_cursor;
FETCH NEXT FROM db_cursor INTO #mytablename, #mycolumnname
WHILE # #FETCH_STATUS = 0
BEGIN
Insert into #ResultsTable(table_name)
EXEC('SELECT ''' + #mytablename + '.' + #mycolumnname + ''' FROM ' + #mytablename + ' (NOLOCK) ' +
' WHERE ' + #mycolumnname + '=6.84')
FETCH NEXT FROM db_cursor INTO #mytablename, #mycolumnname
END;
CLOSE db_cursor;
DEALLOCATE db_cursor;
Select Distinct(table_name) from #ResultsTable
There are lots of workable answers already. I just wanted to share one I wrote that has additional functionality.
--=======================================================================
-- MSSQL Unified Search
-- Minimum compatibility level = 130 (SQL Server 2016)
-- NOTE: The minimum compatibility level is required by the built-in STRING_SPLIT() function.
-- However, you can create the STRING_SPLIT() function at the bottom of this script for
-- lower versions of MSSQL Server.
--
-- Usage:
-- Set the parameters below and execute this script.
--
/************************ Enter Parameters Here ************************/
/**/
/**/ DECLARE #SearchString VARCHAR(1000) = 'string to search for'; -- Accepts SQL wilcards
/**/
/**/ DECLARE #IncludeUserTables BIT = 1;
/**/ DECLARE #IncludeViews BIT = 0;
/**/ DECLARE #IncludeStoredProcedures BIT = 0;
/**/ DECLARE #IncludeFunctions BIT = 0;
/**/ DECLARE #IncludeTriggers BIT = 0;
/**/
/**/ DECLARE #DebugMode BIT = 0;
/**/ DECLARE #ExcludeColumnTypes NVARCHAR(500) = 'text, ntext, char, nchar, timestamp, bigint, tinyint, smallint, bit, date, time, smalldatetime, datetime, datetime2, real, money, float, decimal, binary, varbinary, image'; -- Comma delimited list
/**/
/***********************************************************************/
SET NOCOUNT ON;
SET #SearchString = QUOTENAME(#SearchString,'''');
DECLARE #Results TABLE ([ObjectType] NVARCHAR(200), [ObjectName] NVARCHAR(200), [ColumnName] NVARCHAR(400), [Value] NVARCHAR(MAX), [SelectStatement] NVARCHAR(1000));
DECLARE #ExcludeColTypes TABLE (system_type_id INT);
INSERT INTO #ExcludeColTypes ([system_type_id])
SELECT [system_type_id]
FROM sys.types WHERE
[name] IN (
SELECT LTRIM(RTRIM([value])) FROM STRING_SPLIT(#ExcludeColumnTypes,',')
);
DECLARE #ObjectType NVARCHAR(200);
DECLARE #ObjectName NVARCHAR(200);
DECLARE #Value NVARCHAR(MAX);
DECLARE #SelectStatement NVARCHAR(1000);
DECLARE #Query NVARCHAR(4000);
/********************* Table Objects *********************/
IF (#IncludeUserTables = 1)
BEGIN
DECLARE #TableObjectId INT = (SELECT MIN([object_id]) FROM sys.tables);
DECLARE #ColumnId INT;
WHILE #TableObjectId IS NOT NULL
BEGIN
SELECT #ObjectType = 'USER TABLE';
SELECT #ObjectName = '[' + SCHEMA_NAME([schema_id]) + '].[' + OBJECT_NAME(#TableObjectId) + ']' FROM sys.tables WHERE [object_id] = #TableObjectId;
SET #ColumnId = (SELECT MIN([column_id]) FROM sys.columns WHERE [system_type_id] NOT IN (SELECT [system_type_id] FROM #ExcludeColTypes) AND [object_id] = #TableObjectId);
WHILE #ColumnId IS NOT NULL
BEGIN
SELECT #Value = '[' + [name] +']' FROM sys.columns WHERE [object_id] = #TableObjectId AND column_id = #ColumnId;
SET #SelectStatement = 'SELECT * FROM ' + #ObjectName + ' WHERE CAST(' + #Value + ' AS NVARCHAR(4000)) LIKE ' + #SearchString + ';';
SET #Query = 'SELECT '
+ QUOTENAME(#ObjectType, '''')
+ ', ' + QUOTENAME(#ObjectName, '''')
+ ', ' + QUOTENAME(#Value, '''')
+ ', ' + #Value
+ ', ''' + REPLACE(#SelectStatement,'''','''''') + ''''
+ ' FROM ' + #ObjectName
+ ' WHERE CAST(' + #Value + ' AS NVARCHAR(4000)) LIKE ' + #SearchString + ';';
IF #DebugMode = 0
BEGIN
INSERT INTO #Results EXEC(#Query);
END;
ELSE
BEGIN
PRINT 'Select Statement: ' + #SelectStatement;
PRINT 'Query: ' + #Query;
END;
SET #ColumnId = (SELECT MIN([column_id]) FROM sys.columns WHERE [system_type_id] NOT IN (SELECT [system_type_id] FROM #ExcludeColTypes) AND [object_id] = #TableObjectId AND [column_id] > #ColumnId);
END;
SET #TableObjectId = (SELECT MIN([object_id]) FROM sys.tables WHERE [object_id] > #TableObjectId);
END;
END;
/********************* Objects Other than Tables *********************/
SET #Query = 'SELECT ' +
'ObjectType = CASE ' +
'WHEN b.[type] = ''V'' THEN ''VIEW'' ' +
'WHEN b.[type] = ''P'' THEN ''STORED PROCEDURE'' ' +
'WHEN b.[type] = ''FN'' THEN ''SCALAR-VALUED FUNCTION'' ' +
'WHEN b.[type] = ''IF'' THEN ''TABLE-VALUED FUNCTION'' ' +
'WHEN b.[type] = ''TR'' THEN ''TRIGGER'' ' +
'END ' +
',[ObjectName] = ''['' + SCHEMA_NAME(b.[schema_id]) + ''].['' + OBJECT_NAME(a.[object_id]) + '']'' ' +
',[ColumnName] = NULL ' +
',[Value] = a.[definition] ' +
',[SelectStatement] = ''SP_HELPTEXT '' + QUOTENAME(''['' + SCHEMA_NAME(b.[schema_id]) + ''].['' + OBJECT_NAME(a.[object_id]) + '']'','''''''') + '';'' ' +
'FROM [sys].[sql_modules] a ' +
'JOIN [sys].[objects] b ON a.[object_id] = b.[object_id] ' +
'WHERE ' +
'( ' +
' a.[definition] LIKE ' + #SearchString +
') ' +
'AND ' +
'( ' +
' ( ' +
CAST(#IncludeViews AS VARCHAR(1)) + ' = 1 ' +
' AND ' +
' b.[type] IN (''V'') ' +
' ) ' +
' OR ' +
' ( ' +
CAST(#IncludeStoredProcedures AS VARCHAR(1)) + ' = 1 ' +
' AND ' +
' b.[type] IN (''P'') ' +
' ) ' +
' OR ' +
' ( ' +
CAST(#IncludeFunctions AS VARCHAR(1)) + ' = 1 ' +
' AND ' +
' b.[type] IN (''FN'',''IF'') ' +
' ) ' +
' OR ' +
' ( ' +
CAST(#IncludeTriggers AS VARCHAR(1)) + ' = 1 ' +
' AND ' +
' b.[type] IN (''TR'') ' +
' ) ' +
'); ';
IF #DebugMode = 0
BEGIN
INSERT INTO #Results EXEC(#Query);
END;
ELSE
BEGIN
PRINT 'Select Statement: ' + #SelectStatement;
PRINT 'Query: ' + #Query;
END;
IF #DebugMode = 0
BEGIN
SELECT
[ObjectType]
,[ObjectName]
,[ColumnName]
,[Value]
,[Count] = CASE
WHEN [ObjectType] IN ('USER TABLE') THEN COUNT(1)
ELSE NULL
END
,[SelectStatement]
FROM #Results
GROUP BY [ObjectType], [ObjectName], [ColumnName], [Value], [SelectStatement]
ORDER BY [Value];
END;
/********************** STRING_SPLIT() FUNCTION **********************
CREATE FUNCTION STRING_SPLIT (
#Expression nvarchar(4000)
,#Delimiter nvarchar(100)
)
RETURNS #Ret TABLE ([value] NVARCHAR(4000))
AS
BEGIN
DECLARE #Start INT = 0, #End INT, #Length INT;
SELECT #End = CHARINDEX(#Delimiter,#Expression), #Length = #End - #Start;
IF #End <= 0
BEGIN
INSERT INTO #Ret ([value]) VALUES (#Expression);
END
ELSE
BEGIN
WHILE #Length >= 0
BEGIN
INSERT INTO #Ret ([value])
SELECT ltrim(rtrim(substring(#Expression,#Start,#Length)));
SELECT #Start = #End + LEN(#Delimiter)
SELECT #End = CHARINDEX(#Delimiter,#Expression,#Start)
IF #End < 1
SELECT #End = LEN(#Expression) + 1;
SELECT #Length = #End - #Start;
END;
END;
RETURN;
END;
*********************************************************************/
By far the best and most universal solution I found is to pipe a dump of the db through to a grep of what you are searching for.
e.g. for Mysql:
mysqldump -pPASSWORD database | grep 'search phrase'
Or if you get too many results, you can then output them to a file:
mysqldump -pPASSWORD database | grep 'search phrase' > results.txt
I am trying to get the total sum from columns of a specific data type(money) for multiple tables in a database. Currently I am able to get the list of columns from specific tables but I am unable to get the sums from those columns.
This is what I have now
use database 1
Select + Column_Name
From information_schema.columns
Where TABLE_NAME = 'claimant'
and data_type = 'money'
The result looks something like below
table_name
column_name
table_1
column_a
table_1
column_b
table_1
column_c
what I would like
table_name
column_name
total_sum
table_1
column_a
66.20
table_1
column_b
300.50
table_1
column_c
5389.42
update for #Squirrel Here is the code I have but it's still giving me issues with truncation.
{
declare #sql nvarchar(max);
select #sql = 'with cte as (' + char(13)
+ 'select' + char(13)
+ string_agg(char(9) + quotename(column_name) + ' = sum(' + quotename(COLUMN_NAME) + ')', ',' + char(13)) + char(13)
+ 'from ' + max(quotename(table_name)) + char(13)
+ ')' + char(13)
+ 'select a.table_name, a.column_name, a.total_sum ' + char(13)
+ 'from cte ' + char(13)
+ 'cross apply (' + char(13)
+ char(9) + 'values' + char(13)
+ string_agg(char(9) + '(''' + table_name + ''',''' + column_name + ''',' + quotename(COLUMN_NAME) + ')', ',' + char(13)) + char(13)
+ ') a (table_name, column_name, total_sum)'
from information_schema.columns AS A
INNER JOIN EDL01.STAGING.TABLE_DETAILS B
ON A.TABLE_NAME = B.DEST_TABLE_NAME
where A.table_name = B.DEST_TABLE_NAME
and data_type = 'money'
print #sql
exec sp_executesql #sql
}
below is the create table
CREATE TABLE [staging].[TABLE_DETAILS](
[SOURCE_TABLE_NAME] [varchar](100) NULL,
[DEST_TABLE_NAME] [varchar](100) NULL,
[TYPE] [varchar](10) NULL,
[PRIORITY] [int] NULL,
[SOURCE_TABLE_DATABASE] [varchar](50) NULL,
[SOURCE_TABLE_SCHEMA] [varchar](50) NULL,
[DEST_TABLE_DATABASE] [varchar](50) NULL,
[DEST_TABLE_SCHEMA] [varchar](50) NULL
) ON [PRIMARY]
GO
Below is part of the results
select a.table_name, a.column_name, a.total_sum
from cte
cross apply (
values
('PAYMENT','BILLEDAMOUNT',[BILLEDAMOUNT]),
('PAYMENT','AMOUNT',[AMOUNT]),
('SIMS_PAYMENT','CHECKAMOUNT',[CHECKAMOUNT]),
('BILLREVIEWHEADER','JURISDICTIONAMOUNT1',[JURISDICTIONAMOUNT1]),
('BILLREVIEWHEADER','JURISDICTIONAMOUNT2',[JURISDICTIONAMOUNT2]),
('BILLREVIE
You need to form the query dynamically and then execute it using sp_executesql or exec()
Note : char(9) is tab, char(13) is carriage return. These are added to format the query so that it is readable when you print it out for verification.
declare #sql nvarchar(max);
select #sql = 'with cte as (' + char(13)
+ 'select' + char(13)
+ string_agg(char(9) + quotename(column_name) + ' = sum(' + quotename(column_name) + ')', ',' + char(13)) + char(13)
+ 'from ' + max(quotename(table_name)) + char(13)
+ ')' + char(13)
+ 'select a.table_name, a.column_name, a.total_sum ' + char(13)
+ 'from cte ' + char(13)
+ 'cross apply (' + char(13)
+ char(9) + 'values' + char(13)
+ string_agg(char(9) + '(''' + table_name + ''', ''' + column_name + ''',' + quotename(column_name) + ')', ',' + char(13)) + char(13)
+ ') a (table_name, column_name, total_sum)'
from information_schema.columns
where table_name = 'table_1'
and data_type = 'money'
print #sql
exec sp_executesql #sql
For your sample table, the generated dynamic query is
with cte as (
select
[column_a] = sum([column_a]),
[column_b] = sum([column_b]),
[column_c] = sum([column_c])
from [table_1]
)
select a.table_name, a.column_name, a.total_sum
from cte
cross apply (
values
('table_1', 'column_a',[column_a]),
('table_1', 'column_b',[column_b]),
('table_1', 'column_c',[column_c])
) a (table_name, column_name, total_sum)
EDIT
using a loop to iterate each table. Basically it execute above query for each of the table and insert the result into a temp table
see db<>fiddle demo
for earlier SQL Server version without string_agg(), use for xml path
select #sql = 'with cte as (' + char(13)
+ 'select' + char(13)
+ stuff
(
(
select ',' + quotename(COLUMN_NAME) + ' = sum(' + quotename(COLUMN_NAME) + ')'
from INFORMATION_SCHEMA.COLUMNS
where TABLE_NAME = #table
and DATA_TYPE = 'money'
for xml path('')
),
1, 1, ''
) + char(13)
+ 'from ' + max(quotename(#table)) + char(13)
+ ')' + char(13)
+ 'select a.table_name, a.column_name, a.total_sum ' + char(13)
+ 'from cte ' + char(13)
+ 'cross apply (' + char(13)
+ char(9) + 'values' + char(13)
+ stuff
(
(
select ',' + '(''' + TABLE_NAME + ''', ''' + COLUMN_NAME + ''',' + quotename(COLUMN_NAME) + ')'
from INFORMATION_SCHEMA.COLUMNS
where TABLE_NAME = #table
and DATA_TYPE = 'money'
for xml path('')
),
1, 1, ''
)
+ ') a (table_name, column_name, total_sum)' + char(13)
I wrote the following query:
IF OBJECT_ID ('tempdb..#ColumnsType') IS NOT NULL DROP TABLE #ColumnsType
DECLARE #vQuery NVARCHAR(MAX) =''
IF OBJECT_ID ('tempdb..#random') IS NOT NULL DROP TABLE #random
CREATE TABLE #random (
ColumnID INT PRIMARY KEY IDENTITY(1,1) NOT NULL
, randomname VARCHAR(50)
, randomvalue INT)
INSERT INTO #random (randomname, randomvalue)
VALUES ('a3', 123)
, ('bla', 4325)
, ('another_bla', 5643)
, ('end_here', 3)
select *
from #random
CREATE TABLE #ColumnsType (
ColumnID INT PRIMARY KEY IDENTITY(1,1) NOT NULL
, ColumnName sysname
, DataType sysname
)
INSERT INTO #ColumnsType (ColumnName, DataType)
SELECT [name],
system_type_id
FROM Tempdb.Sys.Columns
WHERE Object_ID = Object_ID('tempdb..#random')
AND system_type_id = 56
DECLARE #i INT = (SELECT MIN(ColumnID) FROM #random);
DECLARE #maxId INT = (SELECT MAX(ColumnID) FROM #random);
DECLARE #ColumnName VARCHAR(200);
DECLARE #DataType VARCHAR(200);
WHILE #i <= #maxId
BEGIN
SET #ColumnName = (SELECT ColumnName FROM #ColumnsType WHERE ColumnId = #i)
-- SET #DataType = (SELECT DataType FROM #ColumnsType WHERE ColumnId = #i)
SELECT #vQuery =
'SELECT
MIN(TRY_CONVERT(NUMERIC(30, 4), ' +#ColumnName+ ')) AS ' +#ColumnName+ '_MinValue
, MAX(TRY_CONVERT(NUMERIC(30, 4), ' +#ColumnName+ ')) AS ' +#ColumnName+ '_MaxValue
, AVG(TRY_CONVERT(NUMERIC(30, 4), ' +#ColumnName+ ')) AS ' +#ColumnName + '_AvgValue
, STDEV(TRY_CONVERT(NUMERIC(30, 4), ' +#ColumnName+ ')) AS ' +#ColumnName+ '_StandardDeviation
, SUM(TRY_CONVERT(NUMERIC(30, 4), ' +#ColumnName+ ')) AS ' +#ColumnName+ '_TotalSum
FROM tempdb..#random' -- +#Schema+'.'+#Table+ ''
EXEC sp_executesql #vQuery
PRINT #vQuery
SET #i = #i + 1
END
For the sake of demonstration I create temp table with random values. I perform profiling on part of the columns which are consisting only of numeric values. To filter the columns I get their names and filter by type, using Tempdb.Sys.Columns. In normal case with my original data, I use INFORMATION_SCHEMA.COLUMNS but I think this is not that important.
The query returns the following:
The result is presented on two rows. What I'd like to do is to have this result on one row. The idea is to pivot the one row result after and to receive the following result:
As I mentioned, you need to not use a loop, use a set based method and UNION ALL your dynamic statements. I assume here, as well, that you are using a recent version of SQL Server. If not, you'll need to replace STRING_AGG with the old FOR XML PATH (and STUFF) method.
This should be enough to get you started:
USE Sandbox;
GO
CREATE TABLE dbo.YourTable (Col1 int,
Col2 varchar(10));
GO
DECLARE #SchemaName sysname = N'dbo',
#TableName sysname = N'YourTable';
DECLARE #SQL nvarchar(MAX),
#CRLF nchar(2) = NCHAR(13) + NCHAR(10);
DECLARE #Delimiter nvarchar(50) = #CRLF + N'UNION ALL' + #CRLF;
SELECT #SQL = STRING_AGG(CONVERT(nvarchar(MAX),N'SELECT MIN(') + QUOTENAME(c.[name]) + N') AS ' + QUOTENAME(c.[name] + N'_MIN') + N',' + #CRLF +
N' MAX(' + QUOTENAME(c.[name]) + N') AS ' + QUOTENAME(c.[name] + N'_MAX') + #CRLF +
N'FROM ' + QUOTENAME(s.[name]) + N'.' + QUOTENAME(t.[name])
,#Delimiter) WITHIN GROUP (ORDER BY c.column_id)
FROM sys.schemas s
JOIN sys.tables t ON s.schema_id = t.schema_id
JOIN sys.columns c ON t.object_id = c.object_id
WHERE s.[name] = #SchemaName
AND t.[name] = #TableName
PRINT #SQL;
EXEC sys.sp_executesql #SQL;
GO
DROP TABLE dbo.YourTable;
Here is a very clean and nice solution which is what I was looking for:
I choose the columns, from specific table, scan it once and perform multiple calculations on it
I use dynamic query and make it one statement for all columns.
It works really fast. It took a little over 5 minutes to return result for table with 50 mil rows.
The only thing that is left to do is to UNPIVOT in order to insert results in a table that I want.
DECLARE
#q1 NVARCHAR(MAX)
, #q2 NVARCHAR(MAX)
, #q3 NVARCHAR(500)
, #schema VARCHAR(50) = '' -- choose schema
, #table VARCHAR(200) = '' -- choose table
SET #Q1 = 'SELECT ' + '''' + #table + '''' + ' as tableName, '
SET #Q3 = ' FROM ' + #schema + '.' + #table
SELECT #q2 = COALESCE(#q2 + ', ', '')
+ ' max(' + columnName + ') as ' + columnName + '_max, '
+ ' min(' + columnName + ') as ' + columnName + '_min, '
+ ' avg(' + columnName + ') as ' + columnName + '_avg, '
+ ' stdev(' + columnName + ') as ' + columnName + '_stdev, '
+ ' sum(' + columnName + ') as ' + columnName + '_sum '
FROM (
SELECT s.[name] as schemaName, t.[name] as tableName, c.[name] as columnName, st.[name] as typeName
FROM sys.schemas s
INNER JOIN sys.tables t ON s.schema_id = t.schema_id
INNER JOIN sys.columns c ON t.object_id = c.object_id
INNER JOIN sys.types st ON st.user_type_id = c.user_type_id
WHERE 1=1
AND s.[name] = #schema
AND t.[name] = #table
AND st.[name] IN ('') -- choose columns of specific data type, that you want to profile
) data
SELECT #q1 = #q1 + #q2 + #q3
EXEC sys.sp_executesql #Q1
Enjoy!
I have a table which has more than 30 columns(all are varchar). I need to list out all the columns which contains blank i.e.' ' values.
I tried using 'coalesce' but it is only for NULL.
The following query will give you all the columns in a table that might have null or '' values.
It is written so that you can run it for all tables in your database but you can limit it to a single table, as I have done for this specific example, checking a table called testingNulls:
--two variables needed for table name and column name, when looping through all tables
declare #table varchar(255), #col varchar(255), #sql varchar(max)
--this will be used to store the result, to have one result set instead of one row per each cursor cycle
if object_id('tempdb..#nullcolumns') is not null drop table #nullcolumns
create table #nullcolumns (tablename varchar(255), columnname varchar(255))
declare getinfo cursor for
select t.name tablename, c.name
from sys.tables t join sys.columns c on t.object_id = c.object_id
where t.name = 'testingnulls' --here the condition for the table name
open getinfo
fetch next from getinfo into #table, #col
while ##fetch_status = 0
begin
select #sql = 'if exists (select top 1 * from [' + #table + '] where [' + #col + '] is null or [' + #col + '] like '''' ) begin insert into #nullcolumns select ''' + #table + ''' as tablename, ''' + #col + ''' as all_nulls end'
print(#sql)
exec(#sql)
fetch next from getinfo into #table, #col
end
close getinfo
deallocate getinfo
--this should be the result you need:
select * from #nullcolumns
You can see a working example here. I hope this is what you need.
List all columns that contain a blank in some record? You'd use a query per column and collect the results with UNION ALL:
select 'COL1' where exists (select * from mytable where col1 like '% %')
union all
select 'COL2' where exists (select * from mytable where col2 like '% %')
union all
...
union all
select 'COL30' where exists (select * from mytable where col30 like '% %');
If you want like select * from [your_table_name] where [col1] = '' and [col2] = ''....., then use dynamic sql query like below.
Query
declare #sql as varchar(max);
select #sql = 'select * from [your_table_name] where '
+ stuff((
select ' and [' + [column_name] + '] = ' + char(39) + char(39)
from information_schema.columns
where table_name = 'your_table_name'
for xml path('')
)
, 1, 5, ''
);
exec(#sql);
Update
Or else if you want to list the column names which have a blank value, then you can use the below dynamic sql query.
Query
declare #sql as varchar(max);
select #sql = stuff((
select ' union all select ' + [column_name] + ' as [col1], '
+ char(39) + [column_name] + char(39) + ' as [col2]'
+ ' from your_table_name'
from information_schema.columns
where table_name = 'your_table_name'
for xml path('')
)
, 1, 11, ''
);
set #sql = 'select distinct t.col2 as [blank_cols] from(' + #sql
+ ')t
where coalesce(ltrim(rtrim(t.col1)), ' + char(39) + char(39) + ') = '
+ char(39) + char(39) + ';';
exec(#sql);
Find a demo here
But still I'm not sure that this is what you are looking out for.
you have not many choices but to specify all the columns in your where clause
WHERE COL1 = '' AND COL2 = '' AND COL3 = '' AND . . .
or you can use Dynamic SQL to form your query, but that is not an easy path to go
If you want to count number of columns having '' value in a table (not for each row) then use the following
SELECT max(CASE WHEN col1 = '' THEN 1 ELSE 0 END) +
max(CASE WHEN col2 = '' THEN 1 ELSE 0 END) +
max(CASE WHEN col3 = '' THEN 1 ELSE 0 END) +
...
FROM t
demo
I created a dynamic SQL script that you can use by providing the table name only
Here it is
declare #sql nvarchar(max)
declare #table sysname = 'ProductAttributes'
select #sql =
'select * from ' + #table + ' where ' +
string_agg('[' + name + '] = '' '' ', ' and ')
from sys.columns
where object_id = OBJECT_ID(#table)
select #sql
exec sp_executesql #sql
Unfortunately, for SQL string concatenation String_Agg function is new with SQL Server 2017
But it is also possible to use SQL XML Path to concatenate WHERE clause fragments
SELECT #sql = 'select * from ' + #table + ' where ' +
STUFF(
(
SELECT
' and ' + '[' + [name] + '] = '' '' '
from sys.columns
where object_id = OBJECT_ID(#table)
FOR XML PATH(''),TYPE
).value('.','VARCHAR(MAX)'
), 1, 5, ''
)
select #sql as sqlscript
exec sp_executesql #sql
The following query returns the values of the table for each field in terms of null percentage . What I want is to get the sum of those percentages for a specific ProductID. Also, I would like to get a percentage (in an extra column) of the fields do not have value i.e. ="". Any ideas?
use AdventureWorks
DECLARE #TotalCount decimal(10,2), #SQL NVARCHAR(MAX)
SELECT #TotalCount = COUNT(*) FROM [AdventureWorks].[Production].[Product]
SELECT #SQL =
COALESCE(#SQL + ', ','SELECT ') +
'cast(sum (case when ' + QUOTENAME(column_Name) +
' IS NULL then 1 else 0 end)/#TotalCount*100.00 as decimal(10,2)) as [' +
column_Name + ' NULL %]
'
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = 'Product' and TABLE_SCHEMA = 'Production'
SET #SQL = 'set #TotalCount = NULLIF(#TotalCount,0)
' + #SQL + '
FROM [AdventureWorks].Production.Product'
print #SQL
EXECUTE SP_EXECUTESQL #SQL, N'#TotalCount decimal(10,2)', #TotalCount
You can use the following:
use AdventureWorks
DECLARE #colCount int;
DECLARE #nullCheck nvarchar(max) = N'';
DECLARE #emptyCheck nvarchar(max) = N'';
DECLARE #SQL NVARCHAR(MAX);
DECLARE #KeyToCheck int = 123; -- adapt as necessary
SELECT
#nullCheck += '
+ ' + 'count(' + QUOTENAME(column_Name) + ')'
,#emptyCheck += '
+ ' +
CASE
WHEN DATA_TYPE IN('bigint', 'int', 'smallint', 'tinyint', 'bit', 'money', 'smallmoney', 'numeric', 'decimal', 'float', 'real') THEN
-- check numeric data for zero
'sum(case when coalesce(' + QUOTENAME(column_Name) + ', 0) = 0 then 1 else 0 end)'
WHEN DATA_TYPE like '%char' or DATA_TYPE like '%text' THEN
--check character data types for empty string
'sum(case when coalesce(' + QUOTENAME(column_Name) + ', '''') = '''' then 1 else 0 end)'
ELSE -- otherwise, only check for null
'sum(case when ' + QUOTENAME(column_Name) + ' IS NULL then 1 else 0 end)'
END
,#colCount =
count(*) over()
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = 'Product' and TABLE_SCHEMA = 'Production'
;
SET #SQL = 'SELECT case when count(*) > 0 then 100.00 - (' + #nullCheck + '
) * 100.00 / ' + cast(#colCount as nvarchar(max)) + '.00 / count(*) end as null_percent
, case when count(*) > 0 then (' + #emptyCheck + '
) * 100.00 / ' + cast(#colCount as nvarchar(max)) + '.00 / count(*) end as empty_percent
FROM Production.Product
WHERE ProductID = ' + cast(#KeyToCheck as nvarchar(max))
;
print #SQL;
EXECUTE (#SQL)
I simplified one of your expressions: Instead of sum (case when <column> IS NULL then 1 else 0 end), you can just use count(<column>). When using count with an expression instead of *, it counts the rows where this expression is non-null. As this is the opposite from what you need, I added the 100.00 - as the start of the SELECT.
For the "empty check", this would make the logic more complex to understand, hence I left the original logic there and extended it. There, I implemented an check for emptiness for numeric and character/text data types. You can easily extend that for date, binary data etc. with whichever logic you use to determine if a column is empty.
I also found it more simple to leave first + in the two variables #nullCheck and #emptyCheck, as it is valid SQL to start an expression wit this.
I also extended the statement so that if there would potentially be more than one record with ProductId = 123, it shows the average across all records, i. e. the total sum divided by the count of rows. And the outermost case expressions just avoid an division by zero error if count(*) would be zero, i. e. no record with ProductId = 123 found. In that case the return value is null.
You could use AVG function:
SELECT AVG(CASE WHEN value IS NULL THEN 100 ELSE 0 END) AS Percents
FROM Table
UPDATE:
Here is your script:
DECLARE #SQL NVARCHAR(MAX), #TABLE_NAME NVARCHAR(MAX), #TABLE_SCHEMA NVARCHAR(MAX), #PK NVARCHAR(MAX)
SET #TABLE_NAME = 'tblBigTable'
SET #TABLE_SCHEMA = 'dbo'
SET #PK = '8'
SELECT
#SQL = COALESCE(#SQL + ', ', 'SELECT ') +'AVG(CASE WHEN ' + COLUMN_NAME + ' IS NULL THEN 100 ELSE 0 END) AS [' + COLUMN_NAME +' NULL %]'
FROM
INFORMATION_SCHEMA.COLUMNS
WHERE
TABLE_SCHEMA = #TABLE_SCHEMA AND
TABLE_NAME = #TABLE_NAME
SET #SQL = #SQL + ' FROM ' + #TABLE_NAME + ' WHERE pkId = ''' + #PK + ''''
print #SQL
EXECUTE SP_EXECUTESQL #SQL