Most recent datetime column and count for each table - sql

I have a DB that has 1000+ tables. 100 of those tables are prefixed with a three letters (let's say 'ABC') Only half of those prefixed tables have MODIFIEDDATETIME column.
I'm trying to do a simple select query to get all the last updated MODIFIEDDATETIME stamp for each Table that actually has a MODIFIEDDATETIME on that table and also begins with the three letter prefix.
I've tried using this function but it doesn't seem to be getting me there. Thoughts?
sp_msforeachtable '
select ''?'', modifieddatetime, count(*)
from ?
where ? like ''%ABC%''
group by modifieddatetime
order by modifieddatetime desc
'

Borrowing from another answer earlier today:
For one, I recommend staying away from undocumented and unsupported
procedures like sp_MSForEachTable. They can be changed or even
removed from SQL Server at any time, and this specific procedure may
have the same symptoms reported by many against sp_MSForEachDb. (See
some background here and here.)
...but also see sp_ineachdb.
Here is how I would do it - most importantly, pull the row count from the metadata which - while not 100% accurate to the millisecond is usually close enough - will not bog down your system performing a scan of every single table:
DECLARE #sql NVARCHAR(MAX);
SELECT #sql = N'';
CREATE TABLE #x
(
[table] NVARCHAR(255),
updated DATETIME,
[rowcount] BIGINT
);
SELECT #sql = #sql + N'INSERT #x SELECT '''
+ QUOTENAME(OBJECT_SCHEMA_NAME([object_id]))
+ '.' + QUOTENAME(OBJECT_NAME([object_id])) + ''',
MAX(MODIFIEDDATETIME), (SELECT SUM(rows) FROM sys.partitions
WHERE [object_id] = ' + CONVERT(VARCHAR(12), [object_id])
+ ') FROM ' + QUOTENAME(OBJECT_SCHEMA_NAME([object_id]))
+ '.' + QUOTENAME(OBJECT_NAME([object_id])) + ';'
FROM sys.columns
WHERE UPPER(name) = 'MODIFIEDDATETIME'
AND UPPER(OBJECT_NAME([object_id])) LIKE 'ABC%';
EXEC sp_executesql #sql;
SELECT [table],updated,[rowcount] FROM #x;
DROP TABLE #x;
That said, I don't know if using MAX(MODIFIEDDATETIME) is appropriate for knowing when a table was touched last. What if a transaction failed? What if the last operation was a delete?

You could do it with dynamic SQL, but this will probably not be very efficient on 1000 tables!
DECLARE #SQL NVARCHAR(MAX) = ''
SELECT #SQL = #SQL + ' UNION SELECT COUNT(' + QUOTENAME(Column_Name) + ') [Rows], MAX(' + QUOTENAME(Column_Name) + ') [MaxModifiedDate], ''' + QUOTENAME(Table_Schema) + '.' + QUOTENAME(Table_Name) + ''' [TableName] FROM ' + QUOTENAME(Table_Schema) + '.' + QUOTENAME(Table_Name)
FROM INFORMATION_SCHEMA.COLUMNS
WHERE Column_Name = 'ModifiedDateTime'
AND Table_Name LIKE 'ABC%'
SET #SQL = 'SELECT MaxModifiedDate, TableName, Rows FROM (' + STUFF(#SQL, 1, 7, '') + ') t ORDER BY MaxModifiedDate DESC'
print #sql
EXEC SP_EXECUTESQL #SQL
It basically builds a query like
SELECT MaxModifiedDate, TableName, Rows
FROM ( SELECT 'Table1' [TableName], MAX(ModifiedDate) [MaxModifedDate], COUNT(ModifiedDate) [Rows]
FROM Table1
UNION
SELECT 'Table2' [TableName], MAX(ModifiedDate) [MaxModifedDate], COUNT(ModifiedDate) [Rows]
FROM Table2
UNION
SELECT 'Table3' [TableName], MAX(ModifiedDate) [MaxModifedDate], COUNT(ModifiedDate) [Rows]
FROM Table3
UNION
...
) c
ORDER BY MaxModifiedDate DESC

Related

Find count of items from all tables in SQL Server database (filter tables based on column name)

I have to find the total number of distinct items from a particular column (named Ticker) from all tables in the database.
How can I achieve this?. This is what I want:
Table_name | Column | Total_Tickers
------------+---------+---------------
Table_1 | ticker | 500
Table_2 | ticker | 100
Table_3 | ticker | 5000
.
.
I know I've got to use sp_MSForEachTable but I'm not sure how to filter those tables that do not have a Ticker column at all.
This is what I've tried:
create table #counts
(
table_name varchar(255),
ticker_count int
)
EXEC sp_MSForEachTable #command1='INSERT #counts (table_name, ticker_count)
SELECT ''?'', COUNT(ticker) FROM ? ',
#whereand = 'AND ''?'' IN (Select * from information_schema.columns where
column_name = ''%ticker%'')'
SELECT table_name, ticker_count
FROM #counts
ORDER BY table_name, ticker_count DESC
DROP TABLE #counts
It doesn't recognize the COUNT(ticker) on the 7th line since I'm not able to filter the tables!
I'd appreciate any pointers on this. Thanks
Here is a much easier approach
use your_databasename --replace with your database name
go
DECLARE #sql VARCHAR(max)= '',
#column_name SYSNAME = 'ticker'
SET #sql = Stuff((SELECT ' union all select Table_name = '''+ table_name + ''',[Column] = ''' + column_name
+ ''',Total_Tickers = count(distinct '+ column_name + ') from '
+ Quotename(table_catalog) + '.'+ Quotename(table_schema) + '.'+ Quotename(table_name)
FROM information_schema.columns
WHERE column_name = #column_name
FOR xml path('')), 1, 11, '') -- stuff is used to remove the first union all
--SELECT #sql
EXEC (#sql)
Since tables has to be filtered based on column name, I don't think msforeachtable would be helpful here.

get a count of each value from every column in a table SQL Server

So I looked this up and this question is very similar but it's missing a key piece: SQL Server count number of distinct values in each column of a table
So in that question they want the distinct count for each column. What I am looking to do is to get a count of each distinct value for each column in a table (and I'm doing this for all the tables in a particular database which is why I'm looking to try to automate this as much as possible). Currently my code looks like this which I have to run for each column:
select mycol1, COUNT(*) as [Count]
from mytable
group by mycol1
order by [Count] desc
Ideally my output would look like this:
ColumnName1 Count
val1 24457620
val2 17958530
val3 13350
ColumnName2 Count
val1 24457620
val2 17958530
val3 13350
val4 12
and so on for all the columns in the table
This answer below (provided by #beargle) from that previous question is really close to what I'm looking to do but I can't seem to figure out a way to get it to work for what I am trying to do so I would appreciate any help.
DECLARE #Table SYSNAME = 'TableName';
-- REVERSE and STUFF used to remove trailing UNION in string
SELECT REVERSE(STUFF(REVERSE((SELECT 'SELECT ''' + name
+ ''' AS [Column], COUNT(DISTINCT('
+ QUOTENAME(name) + ')) AS [Count] FROM '
+ QUOTENAME(#Table) + ' UNION '
-- get column name from sys.columns
FROM sys.columns
WHERE object_id = Object_id(#Table)
-- concatenate result strings with FOR XML PATH
FOR XML PATH (''))), 1, 7, ';'));
You could use:
DECLARE #Table SYSNAME = 'TableName';
DECLARE #SQL NVARCHAR(MAX) = ''
SELECT #SQL = STUFF((SELECT ' UNION SELECT ''' + name
+ ''' AS [Column], '
+ 'CAST(' + QUOTENAME(Name)
+ ' AS NVARCHAR(MAX)) AS [ColumnValue], COUNT(*) AS [Count] FROM '
+ QUOTENAME(#Table) + ' GROUP BY ' + QUOTENAME(Name)
FROM sys.columns
WHERE object_id = Object_id(#Table)
-- concatenate result strings with FOR XML PATH
FOR XML PATH ('')), 1, 7, '');
EXECUTE sp_executesql #SQL;
Which will produce SQL Like the following for a table with two columns (Column1 and Column2)
SELECT 'Column1' AS [Column],
CAST([Column1] AS NVARCHAR(MAX)) AS [ColumnValue],
COUNT(*) AS [Count]
FROM [TableName]
GROUP BY [Column1]
UNION
SELECT 'Column2' AS [Column],
CAST([Column2] AS NVARCHAR(MAX)) AS [ColumnValue],
COUNT(*) AS [Count]
FROM [TableName]
GROUP BY [Column2]
EDIT
If you want a new result set for each column then use:
DECLARE #Table SYSNAME = 'TableName';
DECLARE #SQL NVARCHAR(MAX) = '';
SELECT #SQL = (SELECT ' SELECT ' + QUOTENAME(Name)
+ ', COUNT(*) AS [Count] FROM '
+ QUOTENAME(#Table) + ' GROUP BY ' + QUOTENAME(Name) + ';'
FROM sys.columns
WHERE object_id = Object_id(#Table)
-- concatenate result strings with FOR XML PATH
FOR XML PATH (''));
EXECUTE sp_executesql #SQL;
Which would produce SQL Like:
SELECT [Column1],
COUNT(*) AS [Count]
FROM [callsupplier]
GROUP BY [Column1];
SELECT [Column2],
COUNT(*) AS [Count]
FROM [callsupplier]
GROUP BY [Column2];
thought i would take a stab at this whilst waiting for a backup to restore
hope this does what you require
create Table #Temp
(tableName varchar(100),
columnName varchar(100),
value varchar(1000),
distinctItems int)
Declare #tabName as varchar(100)
Declare #colName as varchar(100)
Declare #tabid as int
Declare cursorTables Cursor
for
select t.object_id , t.name , c.name from sys.tables t inner join sys.columns c on t.object_id = c.object_id
open cursorTables
Fetch Next from cursorTables into
#tabid,#tabName,#colName
while ##Fetch_Status = 0
Begin
declare #query as nVarchar(1000)
set #query = 'Insert into #Temp SELECT ''' + #tabName + ''' , '''+ #colName +''', ' + #colName + ', COUNT([' + #colName +']) AS Expr1 FROM [' + #tabName+ '] group by [' + #colName + ']'
print #query
exec sp_executesql #query
Fetch Next from cursorTables into
#tabid,#tabName,#colName
End
Close cursorTables
Deallocate cursorTables
select * from #temp
drop table #temp
produces some not very useful results on PK values and i suspect it would not work on columns greater than varchar(1000) but works on a fe of my dbs
This version makes a good snippet:
DECLARE #sql NVARCHAR(MAX) = N'';
SELECT #sql += 'SELECT ''' + t.name + ''', ''' + c.name + ''', ' + c.name + ', COUNT(' + c.name + ') AS C FROM ' + QUOTENAME(s.name) + '.' + QUOTENAME(t.name) + ' GROUP BY ' + c.name + ';' + CHAR(13)
FROM sys.tables AS t
INNER join sys.columns c on t.object_id = c.object_id
INNER JOIN sys.schemas AS s ON t.[schema_id] = s.[schema_id]
WHERE s.name LIKE 'stage' AND t.name LIKE 'table' AND c.name LIKE '%whatever%';
--PRINT #sql;
EXEC sp_executesql #sql

Looping through column names with dynamic SQL

I just came up with an idea for a piece of code to show all the distinct values for each column, and count how many records for each. I want the code to loop through all columns.
Here's what I have so far... I'm new to SQL so bear with the noobness :)
Hard code:
select [Sales Manager], count(*)
from [BT].[dbo].[test]
group by [Sales Manager]
order by 2 desc
Attempt at dynamic SQL:
Declare #sql varchar(max),
#column as varchar(255)
set #column = '[Sales Manager]'
set #sql = 'select ' + #column + ',count(*) from [BT].[dbo].[test] group by ' + #column + 'order by 2 desc'
exec (#sql)
Both of these work fine. How can I make it loop through all columns? I don't mind if I have to hard code the column names and it works its way through subbing in each one for #column.
Does this make sense?
Thanks all!
You can use dynamic SQL and get all the column names for a table. Then build up the script:
Declare #sql varchar(max) = ''
declare #tablename as varchar(255) = 'test'
select #sql = #sql + 'select [' + c.name + '],count(*) as ''' + c.name + ''' from [' + t.name + '] group by [' + c.name + '] order by 2 desc; '
from sys.columns c
inner join sys.tables t on c.object_id = t.object_id
where t.name = #tablename
EXEC (#sql)
Change #tablename to the name of your table (without the database or schema name).
This is a bit of an XY answer, but if you don't mind hardcoding the column names, I suggest you do just that, and avoid dynamic SQL - and the loop - entirely. Dynamic SQL is generally considered the last resort, opens you up to security issues (SQL injection attacks) if not careful, and can often be slower if queries and execution plans cannot be cached.
If you have a ton of column names you can write a quick piece of code or mail merge in Word to do the substitution for you.
However, as far as how to get column names, assuming this is SQL Server, you can use the following query:
SELECT c.name
FROM sys.columns c
WHERE c.object_id = OBJECT_ID('dbo.test')
Therefore, you can build your dynamic SQL from this query:
SELECT 'select '
+ QUOTENAME(c.name)
+ ',count(*) from [BT].[dbo].[test] group by '
+ QUOTENAME(c.name)
+ 'order by 2 desc'
FROM sys.columns c
WHERE c.object_id = OBJECT_ID('dbo.test')
and loop using a cursor.
Or compile the whole thing together into one batch and execute. Here we use the FOR XML PATH('') trick:
DECLARE #sql VARCHAR(MAX) = (
SELECT ' select ' --note the extra space at the beginning
+ QUOTENAME(c.name)
+ ',count(*) from [BT].[dbo].[test] group by '
+ QUOTENAME(c.name)
+ 'order by 2 desc'
FROM sys.columns c
WHERE c.object_id = OBJECT_ID('dbo.test')
FOR XML PATH('')
)
EXEC(#sql)
Note I am using the built-in QUOTENAME function to escape column names that need escaping.
You want to know the distinct coulmn values in all the columns of the table ? Just replace the table name Employee with your table name in the following code:
declare #SQL nvarchar(max)
set #SQL = ''
;with cols as (
select Table_Schema, Table_Name, Column_Name, Row_Number() over(partition by Table_Schema, Table_Name
order by ORDINAL_POSITION) as RowNum
from INFORMATION_SCHEMA.COLUMNS
)
select #SQL = #SQL + case when RowNum = 1 then '' else ' union all ' end
+ ' select ''' + Column_Name + ''' as Column_Name, count(distinct ' + quotename (Column_Name) + ' ) As DistinctCountValue,
count( '+ quotename (Column_Name) + ') as CountValue FROM ' + quotename (Table_Schema) + '.' + quotename (Table_Name)
from cols
where Table_Name = 'Employee' --print #SQL
execute (#SQL)

select the max(date) and table name as columns

I have many tables like:
A_names ( updated_at date )
B_names (updated_at date )
C_names (updated_at date )
I want max of updated_date and table name for all the table like '%names%' in the database
select * into #temp1
from INFORMATION_SCHEMA.TABLES
WHERE TABLE_NAME like '%names%'
How to do it ..? thanks in advance
Try this :-
Select name ,modify_date from sys.tables
where name like '%names'
And avoid using INFORMATION_SCHEMA.TABLES instead use catalog views .
Check this article out The case against INFORMATION_SCHEMA views
Dynamically create a SQL statement and then run that command.
DECLARE #dml nvarchar(max) = N''
SELECT #dml += ' UNION ALL SELECT ''' + name + ''' AS tableName, MAX(updated_at) AS maxUpdated_at FROM '
+ QUOTENAME(name)
FROM sys.tables
WHERE name like '%names%'
PRINT #dml
SET #dml = STUFF(#dml, 1, 10, '')
EXEC sp_executesql #dml
Option without a STUFF function
DECLARE #dml nvarchar(max)
SELECT #dml = ISNULL(#dml + ' UNION ALL', '') + ' SELECT ''' + name + ''' AS tableName, MAX(updated_at) AS maxUpdated_at FROM '
+ QUOTENAME(name)
FROM sys.tables
WHERE name like '%names%'
EXEC sp_executesql #dml
SELECT MAX(updated_at_date)
FROM table1
I'm a bit confused from the description about your table names. Anyways this is the way you select the max date.
Probabaly Praveen's answer is a solution to your problem, but in pseudocode
DECLARE #SqlStmt NVARCHAR(2000)
For Each TableName in 'Select name from sys.tables'
{
if #SqlStmnt <> '' #SqlStmnt = #SqlStmnt + ' UNION '
#sqlStmnt = #sqlStmnt + ' SELECT Max(updated_at_date) AS MAXDATE FROM ' + TableName
}
**EXECUTE ('SELECT MAX(MAXDATE) FROM (' + #SqlStmt + ')')**

Searching for particular value of a column

I want to query for a particular value lets say "AYX" in some particular column of some particular table in a database.I need to get the list of tables and columns basically having value as "AYX"..How do I go for it?Is it possible? I am using SQL SERVER 2008
DECLARE #string NVARCHAR(32) = N'AYX';
CREATE TABLE #results
(
[column] NVARCHAR(768),
[value] NVARCHAR(MAX)
);
DECLARE #sql NVARCHAR(MAX) = N'';
SELECT #sql += 'SELECT '''
+ QUOTENAME(OBJECT_SCHEMA_NAME([object_id]))
+ '.' + QUOTENAME(OBJECT_NAME([object_id]))
+ '.' + QUOTENAME(name) + ''', ' + QUOTENAME(name) + ' FROM '
+ QUOTENAME(OBJECT_SCHEMA_NAME([object_id]))
+ '.' + QUOTENAME(OBJECT_NAME([object_id]))
+ ' WHERE ' + QUOTENAME(name) + ' LIKE ''%' + #string + '%'';
'
FROM sys.columns
WHERE system_type_id IN (35, 99, 167, 175, 231, 239)
AND OBJECTPROPERTY([object_id], 'IsMsShipped') = 0;
INSERT #results EXEC sp_executesql #sql;
SELECT [column],[value] FROM #results;
DROP TABLE #results;
#Aaron Bertrand had a very nice script.
I just want to point out that there is a free tool SSMS Tools Pack can search data in all table/views.
SSMS Tools Pack
You'll need to use dynamic/code generated query.
Have a look at SELECT * FROM INFORMATION_SCHEMA.COLUMNS to get your list of columns in the database.
Restrict to appropriate datatypes with a WHERE clause on that table/view.
Code generate the queries to do the search: SELECT '{TABLE_NAME}' AS TABLE_NAME, '{COLUMN_NAME}' AS COLUMN_NAME, COUNT(*) AS ROW_COUNT FROM {TABLE_NAME} WHERE {COLUMN_NAME} LIKE '%{SEARCH}%'
UNION ALL the resulting queries together (add WHERE ROW_COUNT <> 0 to an outer query)