I support a SQL database for a third party software package. They have a lot of what they call "Shadow Tables", really just audit tables. This is all fine and good but their system does not clean up these tables so it is up to me to do so. They also add new "Shadow Tables" without notice with every upgrade. The old way we were purging the tables was with a long list of DELETE FROM statements but this list has become very long and hard to maintain.
To try to make the purge process easier to maintain and automatically catch new "Shadow Tables" I wrote the following stored procedure. The stored procedure works but I would prefer to figure out a way without using a cursor and dynamic queries since this will be running daily on a lot of different tables. Is there an alternative way of doing this without using a cursor and dynamic queries?
DECLARE #workingTable varchar(128);
DECLARE #sqlText varchar(250);
DECLARE #CheckDate DATETIME = DATEADD(yy, -2, GETDATE());
DECLARE curKey SCROLL CURSOR FOR
SELECT name AS TableName
FROM dataTEST.sys.tables
WHERE (name like '%[_]h' OR name like '%[_]dh')
ORDER BY name
OPEN curKey
WHILE ##fetch_status = 0
BEGIN
FETCH NEXT FROM curKey INTO #workingTable
SET #sqlText = 'DELETE FROM DataTEST.dbo.' + #workingTable + ' WHERE LAST_MOD < ''' + CONVERT(CHAR(10), #CheckDate, 101) + ''';'
--PRINT #sqlText
EXEC (#sqlText)
END
CLOSE curKey
DEALLOCATE curKey
I do not know of anyway to get away from dynamic SQL when you do not know the table names ahead of time. SQL Server has a feature where you can do variable assignment in a select statement, once for each row returned. This can be used to eliminate the cursor and pass one string with all the delete statements to SQL server to execute
DECLARE #sqlText nvarchar(MAX) = ''; -- initialize because NULL + 'x' is NULL
DECLARE #CheckDate DATETIME = DATEADD(YEAR, -2, GETDATE());
SELECT #sqlText = #SqlText + 'DELETE FROM dataTEST.dbo.' + QUOTENAME(name)
+ ' WHERE LAST_MOD < #CheckDate ; '
FROM dataTEST.sys.tables
WHERE (name like '%[_]h' OR name like '%[_]dh')
ORDER BY name
IF ##ROWCOUNT > 0
EXEC sp_executesql #sqlText
, N'#CheckDate DATETIME'
, #CheckDate
I don't think using using cursor and dynamic query here is a bad idea
One way is to append the delete queries and execute it at the end after generating all the delete queries.
Btw, cursor is just used for framing dynamic query so it is not a big deal
DECLARE #workingTable varchar(128);
DECLARE #sqlText nvarchar(max)='';
DECLARE #CheckDate DATETIME = DATEADD(yy, -2, GETDATE());
DECLARE curKey SCROLL CURSOR FOR
SELECT name AS TableName
FROM dataTEST.sys.tables
WHERE (name like '%[_]h' OR name like '%[_]dh')
ORDER BY name
OPEN curKey
WHILE ##fetch_status = 0
BEGIN
FETCH NEXT FROM curKey INTO #workingTable
SET #sqlText += 'DELETE FROM DataTEST.dbo.' + #workingTable + ' WHERE LAST_MOD < ''' + CONVERT(CHAR(10), #CheckDate, 101) + ''';'
END
CLOSE curKey
DEALLOCATE curKey
--PRINT #sqlText
EXEC (#sqlText)
You may get a bit better performance by doing the following:
DECLARE #workingTable SYSNAME;
DECLARE #sqlText nvarchar(MAX);
DECLARE #CheckDate DATETIME = DATEADD(YEAR, -2, GETDATE());
DECLARE curKey CURSOR LOCAL FAST_FORWARD FOR
SELECT name AS TableName
FROM dataTEST.sys.tables
WHERE (name like '%[_]h' OR name like '%[_]dh')
ORDER BY name
OPEN curKey
WHILE ##fetch_status = 0
BEGIN
FETCH NEXT FROM curKey INTO #workingTable
SET #sqlText = 'DELETE FROM DataTEST.dbo.' + QUOTENAME(#workingTable)
+ ' WHERE LAST_MOD < #CheckDate'
Exec sp_executesql #sqlText
,N'#CheckDate DATETIME'
,#CheckDate
END
CLOSE curKey
DEALLOCATE curKey
Improvements:
Use appropriate data type for sql server object names tables (SYSNAME).
Use sp_executesql instead of EXEC(#Sql)
Pass the parameter as date, do not convert it to a string so that sql server can make use of indexes defined on that column.
Use QUOTENAME() function for put square brackets around the table names just in case any of the table name is a reserved key word in sql server, so the query wont error out.
Make your cursor local and fast_forward default settings for cursor are global , you don't need that right?
Related
I have some code to create tables based on a set of dates I define.
Example, I have 5 dates, and they are aren't consecutive. For any of these dates, I want to create a table and I am currently using a Select into.
I am having to do this 5 times, even though the only thing changing is the name of the new table created and the date. Is there a way to do this in an elegant way.
I started writing some code, but I am struggling to get it to loop through all the dates I want. The way I have written it currently, I only works if I edit the date at the start.
DECLARE #MyDate DATE;
SET #MyDate = '2019-01-01';
SET #TableName = 'Table1';
SELECT *
into #TableName
FROM Original_Table
WHERE Query_Date = #MyDate;
Is this a one time thing or do you have to do this on a regular basis?
If it's the first, than I would just do it and get it over with.
If it's the latter, then I suspect something is very wrong with the way that system is designed - but assuming that can't be changed, you can create a stored procedure that will do this using dynamic SQL.
Something like this can get you started:
CREATE PROCEDURE dbo.CreateTableBasedOnDate
(
#MyDate DATE,
-- sysname is a system data type for identifiers: a non-nullable nvarchar(128).
#TableName sysname
)
AS
-- 200 is long enough. Yes, I did the math.
DECLARE #Sql nvarchar(200) =
-- Note: I'm not convinced that quotename is enough to protect you from sql injection.
-- you should be very careful with what user is allowed to execute this procedure.
N'SELECT * into '+ QUOTENAME(#TableName) +N'
FROM Original_Table
WHERE Query_Date = #MyDate;';
-- When dealing with dynamic SQL, Print is your best friend.
-- Remark this row and unremark the next only once you've verified you get the correct SQL
PRINT #SQL;
--EXEC sp_ExecuteSql #Sql, N'#MyDate Date', #MyDate
GO
Usage:
EXEC CreateTableBasedOnDate '2018-01-01', 'zohar';
Use dynamic SQL:
DECLARE #MyDate DATE, #TableName varchar(50);
SET #MyDate = '2019-01-01';
SET #TableName = 'Table1';
DECLARE #sql NVARCHAR(4000);
DECLARE #params NVARCHAR(4000);
SELECT #sql=N'
SELECT *
INTO ' + QUOTENAME(#TableName) + '
FROM Original_Table
WHERE Query_Date = #MyDate;';
SELECT #params = N'#MyDate DATE';
EXEC sys.sp_executesql #sql, #params, #MyDate=#MyDate
Note that dynamic SQL can be dangerous as it opens up a path for SQL injection. Its fine if you are just using it in your own local scripts, but take care if you e.g. wrap this in a procedure that is more widely accessible.
I would use dynamic SQL although I would add another variables for the schema:
DECLARE
#MyDate nVarchar(50) = '2019-01-01',
#Schema nVarchar (50) = 'dbo',
#TableName nVarchar(250) = 'Table1',
#SQL nVarchar(500);
Set #SQL = '
SELECT *
into '+ QUOTENAME(#Schema)+'.'+ QUOTENAME(#TableName) +'
FROM Original_Table
WHERE Query_Date = '+ #MyDate +';
'
--print #SQL
Exec(#SQL)
You can use the print statement to see how the SQL will look before executing this properly. You may also want to look at adding this as a stored procedure.
All,
Trying to set a cursor on a table value inside a table variable, but it does not work. can anyone comment on how I can fix this?
** the code below is called from another stored procedure which provides the value for the tablename variable **
ALTER PROCEDURE [dbo].[usrSetLTDNormDist]
-- Add the parameters for the stored procedure here
#TableName Sysname,
---...
DECLARE #SQLCommand1 NVARCHAR(MAX) = N'
Set #RecCursor1 = Cursor For
Select [Volume], [TRANSDATE] from #TableName'
EXECUTE dbo.sp_executesql #sqlCommand1
-- Open Cursor
Open #RecCursor1
Fetch Next From #RecCursor1
Into #Volume, #TransDate
---...
Add PRINT #SQLCommand1 between the DECLARE and EXECUTE statements to review what is actually being executed. Based on your code snippet, you will see
Set #RecCursor1 = Cursor For
Select [Volume], [TRANSDATE] from #TableName
...that is, the value you set in #TableName is not automagically added to the script. Here's the way I write these things:
DECLARE #SQLCommand1 NVARCHAR(MAX)
SET #SQLCommand1 = replace(N'
Set #RecCursor1 = Cursor For
Select [Volume], [TRANSDATE] from <#TableName>'
,'<#TableName>', #TableName)
PRINT #SQLCommand1
EXECUTE dbo.sp_executesql #sqlCommand1
I use the < > characters to make the replaced values stand out.
This script demonstrates the general technique:
create table T (ID int not null)
go
insert into T(ID) values (99)
go
declare #TableName sysname
declare #ID int
set #TableName = 'T'
declare #SQL nvarchar(max) = N'declare boris cursor for select ID from ' +
QUOTENAME(#TableName)
exec sp_executesql #SQL
open boris
fetch next from boris into #ID
while ##FETCH_STATUS = 0
begin
print #ID
fetch next from boris into #ID
end
close boris
deallocate boris
Producing this output:
(1 row(s) affected)
99
However, I will offer my usual caution - if you're in a situation where you want to operate against multiple tables in the same way, this is usually a sign of a broken data model. Usually there ought to be a single table with additional columns containing data that serves to differentiate the values.
I have a LOT of views in the database.
Each view ofc refers to one or more tables.
There was some work done with those tables (alter, delete columns) and now i need to check all views for any runtime errors.
I went straithforward: got list of all views, iterated over it and launch SELECT TOP 0 * FROM view_name dynamically so any errors should appear in the Messages pane.
This is my code
DECLARE #view_name_template varchar(max) = '%'
DECLARE #columnList varchar(75) = '*'
--------------------------
DECLARE #tmp_views AS TABLE (view_name varchar(max))
DECLARE #view_name varchar(max)
DECLARE #sqlCommand nvarchar(max)
DECLARE #num int = 1
DECLARE #total_count int
SET NOCOUNT ON
INSERT INTO #tmp_views
SELECT name FROM sys.views
WHERE name LIKE #view_name_template
SELECT #total_count = COUNT(*) FROM sys.views WHERE name LIKE #view_name_template
DECLARE db_cursor CURSOR FOR
SELECT view_name FROM #tmp_views ORDER BY LOWER(view_name)
OPEN db_cursor
FETCH NEXT FROM db_cursor INTO #view_name
WHILE ##FETCH_STATUS = 0
BEGIN
SET #sqlCommand = 'SELECT TOP 0 ' + #columnList + ' FROM ' + #view_name
PRINT CAST(#num as varchar(31)) + '/' + CAST(#total_count as varchar(31)) + ' ' + #sqlCommand
EXECUTE sp_executesql #sqlCommand
FETCH NEXT FROM db_cursor INTO #view_name
SET #num = #num + 1
END
CLOSE db_cursor
DEALLOCATE db_cursor
It works fine except it completely freezes on some views (select from those views in other window works extremely fast and fine). I think it is server a memory overflow issue or something similar.
Tell me please: what is the lightweighiest way to check view has errors or not? Maybe SQL Server has a special function or stored procedure?
The code is not "hanging". It is waiting for the view to run, despite the top 0.
SQL Server offers several ways of testing queries. In addition to the top 0, you also have:
`set parseonly1
set noexec on
And then the more recent sp_describe_first_result_set.
Each of these do different things. parseonly checks for syntax errors but doesn't look at table layouts. I believe noexec completely compiles the query, creating the execution plan. top 0 will compile the query and also run it.
In some cases, the optimizer may not recognize that a query that returns no rows might need to do no work. For instance, there might be subqueries that are run despite the top 0, and this is causing the delay.
Two approaches. The first is to use noexec on (documented here). The second, if feasible, would be to create another database with the same structure and no data. You can then test the queries on that database.
I have 3000+ tables in my SQL 2008 database with names like listed below, that all starts with tempBinary_, that I need to delete programmatically, how do I do that?
I don't know if I prefer the solution in a SQL-script or with use of LINQtoSQL, i guess both are fine.
tempBinary_002c90322f4e492795a0b8a14e2f7c99
tempBinary_0039f7db05a9456f96eb3cd6a788225a
tempBinary_0057da9ef0d84017b3d0bbcbfb934fb2
I've used Like before on columns, but I don't know if it good for table names too.
Maybe something like this, where LIKE is used, can do it? I don't know.
Use [dbo].[database_name]
DROP TABLE table_name
WHERE table_name LIKE 'tempBinary_%'
Any ideas?
declare #stmt varchar(max) = ''
declare #tbl_name varchar(255)
DECLARE tbl_cursor CURSOR FORWARD_ONLY READ_ONLY
FOR select name
from sysobjects
where xtype='u' and name like 'tempBinary%'
OPEN tbl_cursor
FETCH NEXT FROM tbl_cursor
INTO #tbl_name;
WHILE ##FETCH_STATUS = 0
BEGIN
set #stmt = #stmt + 'drop table ' + #tbl_name + ';' + CHAR(13)
FETCH NEXT FROM tbl_cursor
INTO #tbl_name
end
CLOSE tbl_cursor;
DEALLOCATE tbl_cursor;
execute sp_sqlexec #stmt
I get the feeling this is pretty basic database work, but it isn't for me. I'm trying to get a list of all of my tombstone tables from system tables and store the results in a cursor. I'm then trying to perform some logic on each of those tables I'm having trouble doing so.
Any help would be greatly appreciated.
Here is the error I get:
Must declare the table variable "#tablename"
Here is the code:
declare tombstonetables cursor for
(select name from sys.objects
where
name like'%tombstone%'
and type = 'U'--for user_table
)
Print 'Begin purging tombstone tables'
declare #tablename varchar(250)
open tombstonetables
fetch next from tombstonetables into #tablename
WHILE ##FETCH_STATUS = 0
begin
select * from #tablename--real logic goes here later
fetch next from tombstonetables into #tablename
end
close tombstonetables
deallocate tombstonetables
Looks like you need to use Dynamic SQL
Here is a reference to a simple walk through http://www.mssqltips.com/tip.asp?tip=1160
You will probably need to make use of sp_executesql
Here is a simple example of using Dynamic SQL with your example
DECLARE #DynamicSQL nvarchar(100)
WHILE ##FETCH_STATUS = 0
begin
SET #DynamicSQL = 'select * from ' + #tablename --real logic goes here later
EXEC #DynamicSQL
fetch next from tombstonetables into #tablename
end