I'm trying to get data from all tables that I have from another query as follows:
DECLARE #count int
SET #count = (SELECT COUNT(*) FROM (SELECT TABLE_NAME FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME LIKE '%Project%') AS SUBQUERY)
WHILE(#count!=0)
BEGIN
SELECT * from (SELECT TABLE_NAME from (SELECT TABLE_NAME,
ROW_NUMBER() over (order by table_name) as row_number
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_NAME LIKE '%Project%') as sub
WHERE row_number = #count) as another_sub;
SET #count = #count-1
end
What I get with this right now is 5 table names LIKE '%Project%'. I want to get the data from all of these 5 tables, not just their names. Also I don't want to join or union the tables. How can I achieve this?
DECLARE #SQL varchar(max) = '';
SELECT #SQL = #SQL + 'SELECT * FROM ' + QUOTENAME(TABLE_SCHEMA) + '.' + QUOTENAME(TABLE_NAME) + ';' + CHAR(13) + CHAR(10)
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_NAME LIKE '%Project%';
print #SQL;
--EXEC(#SQL);
Related
I need to analyze a large table with hundreds of columns. A lot of columns are unused.
To investigate I could do something like
SELECT DISTINCT Column1
FROM myTable
or
WITH C AS
(
SELECT DISTINCT Column1
FROM MyTable
)
SELECT COUNT(*)
FROM C
Then I do the same for column2 and so on. However these queries only work for one column which is time consuming and does not give overview in one glance.
Any idea how to build such investigation query for all columns in one?
You need only 1 query where you have to list all the columns of the table:
SELECT COUNT(DISTINCT Column1) column1_count,
COUNT(DISTINCT Column2) column2_count,
COUNT(DISTINCT Column3) column3_count
.....................................
FROM MyTable;
For local purposes only, you can make it dynamic like this:
Get the columns of the table
the query is created as the colleagues did and then it is executed with the EXEC()
DECLARE #columns as Table(RowId INT IDENTITY(1,1), ColumnName nVarchar(50))
DECLARE #ii int = 0
DECLARE #max int = 0
DECLARE #sqlQuery nVarchar(MAX)
INSERT INTO #columns
SELECT COLUMN_NAME
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = N'Customer'
SET #sqlQuery = 'SELECT '
SELECT #max = COUNT(*) FROM #columns
WHILE #ii <= #max
BEGIN
SELECT #sqlQuery = CONCAT(#sqlQuery,'COUNT(DISTINCT ',ColumnName,') ',LOWER(ColumnName),'_count, ')
FROM #columns
WHERE RowId = #ii
SET #ii = #ii + 1
END
SELECT #sqlQuery = CONCAT(#sqlQuery,'FROM Customer')
SELECT #sqlQuery = REPLACE(#sqlQuery,', FROM',' FROM')
select #sqlQuery
EXEC (#sqlQuery)
You should flesh out your requirement a bit more. If all you want to know is if a column contains only NULLs, you'll want to check for max(ColumnName) is null
declare #sql table (id int identity(1,1), QueryString nvarchar(max))
create table ##emptyColumns (emptyColumn nvarchar(128))
declare #i int = 0
declare #iMax int
declare #runthis nvarchar(max)
insert #sql
select 'select ''' + QUOTENAME(s.name) + '.' + QUOTENAME(o.name) + quotename(c.name) + ''' as ''column''
from ' + QUOTENAME(s.name) + '.' + QUOTENAME(o.name) + '
having max(' + c.name + ') is null'
from sys.sysobjects o
inner join sys.syscolumns c on c.id = o.id
inner join sys.schemas s on s.schema_id = o.uid
where o.type = 'U'
order by s.name
, o.name
, c.colorder
select #iMax = count(*)
from #sql
print #iMax
while #i < #iMax
begin
set #i = #i + 1
select #runthis = 'insert into ##emptyColumns
' + QueryString
from #sql
where id = #i
execute sp_executesql #runthis
end
select *
from ##emptyColumns
drop table ##emptyColumns
One further option you might consider:
declare #sql nvarchar(max)
select #sql = isnull(#sql + ' union all ', '') + 'select ''' + COLUMN_NAME + ''',
sum(case when ' + COLUMN_NAME + ' is null then 1 else 0 end) as null_values,
count(distinct ' + COLUMN_NAME + ') as count_distinct
from ' + TABLE_SCHEMA + '.' + TABLE_NAME + '
'
from information_schema.columns
where TABLE_SCHEMA = 'MySchema' and TABLE_NAME = 'MyTable'
exec (#sql)
If you had very big tables with large numbers of columns and were only interested in empty columns you could look into something like checksum_agg(checksum(column_name)). It may help improve performance.
You'd need to be wary of column data types, as they are not all compatible with distinct.
I have a set of tables with same prefix and same structure. All I need is to "combine" them as one.
use thisdb
declare #maxrow int, #result nvarchar(4000), #tempname nvarchar(20)
set #maxrow = (select count(*) from information_schema.tables where table_schema = 'dbo' and table_name like N'AAbc%')
set #result = ''
set #tempname = 'mytemp'
select #result = #result + case when [row] = 1 then replace([name],'from',concat('into ##',#tempname, ' from')) when [row] = #maxrow then replace([name],'union all','') else [name] end + ' '
from
(select ['table_name,'] union all') as [name], row_number() over(order by table_name asc) as [Row]
from information_schema.tables
where table_schema = 'dbo' and table_name like N'AAbc%') x
execute sp_executesql #result
Basically I retrieve tables with certain pattern from information_schema.tables, then get rid of the last union all.
Above method works for 20-30 tables, as the #result won't exceed the limit for nvarchar. But I'm curious how to get this job done if number of tables N is very large? Or there's better way to deal with this problem?
This should suffice. "Union all" replaced with "Go".
declare #script nvarchar(max)
set #script = (
SELECT STUFF(( SELECT 'insert #MyTable(sharedColName1, SharedColName2) select sharedColName1, SharedColName2 from ' + Table_Name + ' Go '
FROM ( SELECT DISTINCT
Table_Name
FROM INFORMATION_SCHEMA.Tables
where table_Name Like '<prefix_for_tables_with_identical_names>%'
) x
FOR
XML PATH('')
), 1, 0, '') A
)
sp_executeSql #script
I have this statement:
Declare #sql varchar(max) = ''
declare #tablename as varchar(255) = 'test'
select #sql = #sql + 'select [' + c.name + '],count(*) as ''' + c.name
+ ''' from [' + t.name + ']'
from sys.columns c
inner join sys.tables t on c.object_id = t.object_id
where t.name = #tablename
EXEC (#sql)
But it gives the output comes out in different results windows and when I try to combine it with a union all the text doesn't fit it. I want to try and get the results into a temp table for SQL server is there anyway i can do this?
I'm trying to get:
Column Name Count Distinct Count
a 100 1
b 100 5
c 100 73
d 100 9
The statement above isn't for distinct count but i'm hoping I could replicate the same logic.
I suspect the query you want to construct really looks like:
select 'a' as column_name, count(column_name), count(distinct column_name)
from t
union all
select 'a' as column_name, count(column_name), count(distinct column_name)
from t
union all
. . .
To construct this in SQL Server, you can use logic like this:
declare #q nvarchar(max);
set #q = '
select ''[column_name]'' as column_name, count([column_name]) as cnt, count(distinct [column_name]) as distinct_count
from [table_name]
';
declare #sql nvarchar(max);
select #sql = string_agg(replace(replace(#q,
'[column_name]',
quotename(column_name)
),
'[table_name]',
quotename(table_name)
), ' union all '
)
from information_schema.columns c
where table_name = #name; -- should probably check the schema too!
exec sp_executesql #sql;
I have a SQL database with 60+ tables, almost all of which are populated with a CLIENTID field. I want to count the number of unique client IDs in each table.
The results I'm looking for are:
TABLE_NAME; CLIENTID_COUNT
dbo.HISTORY; 650
dbo.VISITS; 596
dbo.SALES; 1053
...; ...
This seems like it should be so simple but I've been playing around with cursors for hours and can't figure this one out. Please help!
IF OBJECT_ID('tempdb..#temp_RESULTS') IS NOT NULL DROP TABLE #temp_RESULTS
CREATE TABLE #TEMP_RESULTS
(
TABLENAME VARCHAR(MAX),
CLIENTCNT BIGINT
)
DECLARE #TABLENAME VARCHAR(MAX)
DECLARE #command VARCHAR(MAX)
IF OBJECT_ID('tempdb..#temp_PROCESS') IS NOT NULL DROP TABLE #temp_PROCESS
SELECT * INTO #TEMP_PROCESS FROM sys.tables
WHILE EXISTS(SELECT * FROM [#TEMP_PROCESS])
BEGIN
SET #TABLENAME = (SELECT TOP 1 [NAME] FROM [#TEMP_PROCESS])
SET #command = ('SELECT ''' + #TABLENAME + ''', COUNT(DISTINCT CLIENTID) AS CLIENTCNT FROM ' + #TABLENAME)
SELECT #command
INSERT INTO #TEMP_RESULTS
EXEC(#command)
DELETE FROM [#TEMP_PROCESS] WHERE [NAME] = #TABLENAME
END
SELECT * FROM [#TEMP_RESULTS]
Assuming the column is exactly ClientId in every table, you should be able to use this as is:
DROP TABLE IF EXISTS #clientId
CREATE TABLE #clientId
(
TableName nvarchar(1000),
ClientIdCount bigint
)
DECLARE #TableName nvarchar(1000);
DECLARE #CurrentQuery nvarchar(2000);
DECLARE result_cursor CURSOR local fast_forward FOR
SELECT DISTINCT
'['+TABLE_SCHEMA + '].[' + TABLE_NAME + ']'
FROM
INFORMATION_SCHEMA.COLUMNS
WHERE
COLUMN_NAME = 'ClientId'
OPEN result_cursor
FETCH NEXT FROM result_cursor into #TableName
WHILE ##FETCH_STATUS = 0
BEGIN
SET #CurrentQuery = 'SELECT ''' + #TableName + ''', COUNT(DISTINCT ClientId) FROM ' + #TableName
--print #CurrentQuery
INSERT INTO
#clientId
(
TableName,
ClientIdCount
)
EXEC(#CurrentQuery)
FETCH NEXT FROM result_cursor into #TableName
END
--end loop
--clean up
CLOSE result_cursor
DEALLOCATE result_cursor
GO
SELECT
*
FROM
#clientId
You could use dynamic sql.
This will read through your system tables, find those that have a ClientID column, and build the text of a query that's in the general shape of 'Select Count(DISTINCT ClientID)' from each table.
DECLARE #SQLQuery as nvarchar(max) = ''
------------------------------------
-- GET THE TABLES THAT HAVE A CLIENTID FROM SCHEMA
SELECT #SQLQuery = #SQLQuery + qryTxt FROM (
SELECT DISTINCT 'SELECT ''' + tables.name + ''', COUNT(DISTINCT CLIENTID) FROM ' + tables.name + ' UNION ' AS qryTxt
FROM sys.columns left join sys.tables on columns.object_id = tables.object_id where columns.name = CLIENTID AND isnull(tables.name, '') <> '') subquery
------------------------------------
-- REMOVE THE LAST 'UNION' KEYWORD FROM SQLQUERY
SET #SQLQuery = left(#sqlQuery, len(#sqlQuery) - 5)
------------------------------------
-- EXECUTE
execute sp_executesql #SQLQuery
I really dislike cursors and loops. Even though this is not going to be much difference for a performance perspective I like to share how you can leverage the system tables and dynamic sql to avoid using a cursor, while loop or temp tables for something like this.
This code is literally all you need to to do this.
declare #SQL nvarchar(max) = ''
select #SQL = #SQL + 'select TableName = ''' + t.name + ''', ClientID_Count = count(distinct clientID)
from ' + QUOTENAME(t.name) + ' UNION ALL '
from sys.tables t
join sys.columns c on c.object_id = t.object_id
where c.name = 'clientID'
select #SQL = left(#SQL, len(#SQL) - 10) --removes the last UNION ALL
select #SQL
--once your comfortable the dynamic sql is correct just uncomment the line below.
--exec sp_executesql #SQL
A similar pattern to other answers here, but this is how I would tackle it:
IF OBJECT_ID('#Tables', 'U') IS NOT NULL
DROP TABLE #Tables;
SELECT ID = IDENTITY(INT, 1, 1),
SchemaName = OBJECT_SCHEMA_NAME([object_id]),
TableName = OBJECT_NAME([object_id]),
ColumnName = name,
DistinctCount = 0
INTO #Tables
FROM sys.columns
WHERE name = 'CLIENTID';
DECLARE #ID INT = 1,
#MaxID INT = (SELECT MAX(ID) FROM #Tables);
WHILE #ID < #MaxID
BEGIN;
DECLARE #SQLCommand VARCHAR(MAX);
SELECT #SQLCommand = FORMATMESSAGE('
UPDATE #Tables SET DistinctCount = (
SELECT COUNT(DISTINCT %s) FROM %s.%s
)
WHERE ID = %i;',
QUOTENAME(ColumnName), QUOTENAME(SchemaName), QUOTENAME(TableName), ID)
FROM #Tables
WHERE ID = #ID;
EXEC (#SQLCommand);
SET #ID += 1;
END;
SELECT *
FROM #Tables;
I would like to ask for help.
As I have 500 plus tables, and I need to search those table's column having some similar words. Is it possible to search and list those tables?
E.g
Table 1 - Name, age, height
Table 2 - Result, Name, Score
Table 3 - Name, Pic, Parent1, Parent2
I wan to do a query to select all the table that any of the column contain the word "%Name%", is this possible?
Just run following query in your db and replace your search string with string and it will work.
SQL for find particular word/value from all columns and tables in a database
DECLARE #SQL VARCHAR(MAX)
DECLARE #valueToFind VARCHAR(1000)
DECLARE #columnName VARCHAR(1000)
SET #valueToFind = 'string'
SET #columnName = '%%'
CREATE TABLE #TMP
(Clmn VARCHAR(500),
CNT INT)
SELECT #SQL=COALESCE(#SQL,'')+CAST('INSERT INTO #TMP Select ''' + TABLE_SCHEMA + '.' + TABLE_NAME + '.' + COLUMN_NAME + ''' AS Clmn, count(*) CNT FROM '
+ TABLE_SCHEMA + '.[' + TABLE_NAME +
'] WHERE [' + COLUMN_NAME + '] LIKE ''%' + #valueToFind + '%'' ;' AS VARCHAR(8000))
FROM INFORMATION_SCHEMA.COLUMNS
JOIN sysobjects B
ON INFORMATION_SCHEMA.COLUMNS.TABLE_NAME = B.NAME
WHERE COLUMN_NAME LIKE #columnName AND xtype = 'U'
AND DATA_TYPE IN ('char','nchar','ntext','nvarchar','text','varchar')
--PRINT (#SQL)
EXEC(#SQL)
SELECT * FROM #TMP WHERE CNT > 0
DROP TABLE #TMP
-----------------------------------------------------------------------------------------