Dynamic SQL DateTime comparison while data archive is not working - sql

I have written this stored procedure for data purge from one of my tables. I have made it configurable to either archive the data or do a hard delete. Since the volume of data is quiet large, i am using loops to do the same.
The delete part of it is working fine However the archive part of it is giving me hard times and i am kind of stuck there with multiple tries.
Here is my SP.
ALTER PROCEDURE [dbo].[spPurgeRecords_new] (
#Age AS INT,
#NumberOfLoops AS BIGINT,
#DeleteSize BIGINT,
#IsArchive BIT
)
AS
SET NOCOUNT ON
SET XACT_ABORT ON
BEGIN
DECLARE #CurrentLoop AS BIGINT;
SET #CurrentLoop = 0
declare #cutoffDate datetime;
declare #maxXDate datetime;
declare #loop varchar(50);
select #cutoffDate = dateadd(year,-#Age,getdate())
select #maxXDate = max(dateCreated) from cbr_audit where dateCreated < #cutoffDate
declare #date varchar(100), #cmd varchar(1000),#archivedate varchar(100)
set #date = (select FORMAT(getdate(), 'yyyyMMdd'));
set #archivedate = (select FORMAT(#maxXDate, 'yyyyMMdd'));
declare #backupTable varchar(100)
set #backupTable = 'cbr_audit_Backup_' + #date;
BEGIN TRY
BEGIN TRANSACTION
WHILE #CurrentLoop < #NumberOfLoops
BEGIN
IF #IsArchive = 1
BEGIN
--Archive the records into a backup table
IF OBJECT_ID (#backupTable, N'U') IS NULL
begin
set #cmd = 'SELECT * INTO [cbr_audit_Backup_'+ #date +'] FROM [cbr_audit] WITH (NOLOCK) where convert(datetime,dateCreated,101) <= CONVERT(DATETIME, ''' + #archivedate + ''', 101)'
exec(#cmd)
end
--Delete the rows from cbr_audit table
DELETE
FROM dbo.cbr_audit
WHERE id IN
(SELECT TOP(#DeleteSize) id
FROM dbo.cbr_audit WITH (NOLOCK)
WHERE dateCreated <= #maxXDate);
END
ELSE
BEGIN
-- Delete the records
DELETE
FROM dbo.cbr_audit
WHERE id IN
(SELECT TOP(#DeleteSize) id
FROM dbo.cbr_audit WITH (NOLOCK)
WHERE dateCreated <= #maxXDate);
END
-- WAITFOR DELAY '00:00:00:500';
SET #CurrentLoop = #CurrentLoop + 1;
set #loop = cast(#currentloop as varchar(50))
RAISERROR (#loop, 0, 1) WITH NOWAIT
END
COMMIT TRANSACTION
END TRY
BEGIN CATCH
--Rollback
RETURN
END CATCH
END
In This SP, in the archive part of code the dynamic sql is not giving any results. DateCreated is of Datetime type. Can someone please help me on this.
Thanks in Advance.

Seems that you are using SQL 2016 or above, try CONCAT, something like:
select concat('SELECT * INTO [cbr_audit_Backup_', #date,'] FROM [cbr_audit] WITH (NOLOCK) where convert(datetime,dateCreated,121) <= ''', CONVERT(varchar(30), #archivedate , 121), '''')
few more things:
I strongly recommend you always to use style 121 (canonical)
or 126 (ISO8601) so you'll SQL will not be confused by mm/dd/yyyy (101) or dd/mm/yyyy (103).
you are deleting by chunks, but the transaction is for all the block. consider doing only transaction for each delete (implicit)
instead insert into and then delete, have a look to OUTPUT clause to DELETE here

Related

Renaming table and view from a different database, works in editor but not from stored procedure

I have to run a monthly job in SQL Server to rename a table and a view in a variety of databases. The database names are stored in a table and this procedure loops through them. The table names change monthly, so I am concatenating the table names based on the current date.
This works well to creating the commands.
If I change my EXEC to PRINT and paste the results into a new query window it works great.
BW_Test.dbo.sp_rename 'BW_Test_DataLog_2018_05','BW_Test_DataLog_2018_06';
BW_Test.dbo.sp_rename 'BW_Test_DataLog','BW_Test_DataLog_2018_05';
However when I run the stored procedure it fails with the following error:
ErrorNumber: 2812 ErrorMessage: Could not find stored procedure 'BW_Test.dbo.sp_rename 'BW_Test_DataLog_2018_05','BW_Test_DataLog_2018_06';'
Here is the stored procedure, thanks in advance!
BEGIN
SET NOCOUNT ON;
-- Find month and year to concatenate with table names
DECLARE #RighNow DATE = GETDATE();
DECLARE #LastMonth DATE = DATEADD(MONTH, -1, GETDATE());
DECLARE #RenameView NVARCHAR(500);
DECLARE #RenameTable NVARCHAR(500);
DECLARE #LastMonthsName NVARCHAR(50);
DECLARE #ThisMonthsName NVARCHAR(50);
DECLARE #COUNTER INT = 0;
DECLARE #MAX INT = (SELECT COUNT(*) FROM DatabaseNames)
DECLARE #Machine VARCHAR(50);
--Start Loop here
WHILE #COUNTER < #MAX
BEGIN
SET #Machine = (SELECT DatabaseName
FROM
(SELECT
(ROW_NUMBER() OVER (ORDER BY (SELECT NULL))) [index],
DatabaseName
FROM
DatabaseNames) R
ORDER BY R.[index]
OFFSET #COUNTER ROWS FETCH NEXT 1 ROWS ONLY);
SET #LastMonthsName = CONCAT(#Machine, '_DataLog', '_', YEAR(#LastMonth), '_', FORMAT(MONTH(#LastMonth), '00'));
SET #ThisMonthsName = CONCAT(#Machine, '_DataLog', '_', YEAR(#RighNow), '_', FORMAT(MONTH(#RighNow), '00'));
SET #RenameView = CONCAT(#Machine, '.dbo.sp_rename ', char(39), #LastMonthsName, char(39), ',', char(39), #ThisMonthsName, char(39), ';');
SET #RenameTable = CONCAT(#Machine, '.dbo.sp_rename ', char(39), #Machine, '_DataLog', char(39), ',', char(39), #LastMonthsName, char(39), ';');
BEGIN TRY
--IMPORTANT - Change the View first or you will have duplicate table names
EXEC #RenameView
EXEC #RenameTable
END TRY
BEGIN CATCH
SELECT
ERROR_NUMBER() AS ErrorNumber,
ERROR_MESSAGE() AS ErrorMessage;
END CATCH;
SET #COUNTER = #COUNTER + 1
END
END
Change
EXEC #RenameView
EXEC #RenameTable
to:
EXEC (#RenameView)
EXEC (#RenameTable)
The problem is that EXEC has actually 2 different implementations, one for dynamic SQL (with parenthesis) and another for procedures (without).

SQL Server trigger locks table

I'm not very versed in writing SQL, but I'm attempting to implement a trigger after INSERT.
The trigger will fire, however, the more complicated this particular SELECT statement gets it seems that it locks the table. If I simplify the statement to only have a couple of argument it runs fine.
If I run the SELECT statement from SSMS, it works, and if I run it from Visual Studio in the VB.net application, it also works. I've tried increasing the command timeout but it still fails. There are only a few rows of data in the table.
Any insight would be greatly appreciated. Please let me know if more detail is required. This is a payroll timeclock table, I'm triggering when a record is inserted and then grabbing the previous days clock out date (so I can compare the previous days payroll hours to the previous days project hours).
I forgot to mention that I'm running the SELECT statement on a view I created that includes two tables from the native application.
SELECT TOP 1
#LastDateClockOut = datEvent
FROM
MyTable.dbo.TimeMTSView
WHERE
datEvent < #Today
AND strUniqueID = #nUser
AND blnEventType = 0
AND lngClassificationID = 0
ORDER BY
datEvent DESC;
Here is the entire trigger:
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
ALTER TRIGGER [dbo].[trgAfterInsert]
ON [dbo].[tblTimes]
AFTER INSERT
AS
DECLARE #trg_lngClassificationID int;
SELECT #trg_lngClassificationID = i.lngClassificationID
FROM inserted i;
IF (#trg_lngClassificationID = 0) --If the date Classification is 0 this means it is NORMAL time as opposed to VACATION or HOLIDAY
DECLARE #trg_lngID int;
DECLARE #trg_lngEmployeeID varchar(20);
DECLARE #trg_blnEventType bit;
DECLARE #blnEventChar nvarchar(1);
DECLARE #trg_datEvent datetime;
DECLARE #nUser varchar(255);
DECLARE #cmd varchar(255);
DECLARE #Today varchar(255);
DECLARE #LastDateClockOut datetime;
DECLARE #strLastClock varchar(255);
SELECT #trg_lngID = i.lngID FROM inserted i;
SELECT #trg_lngEmployeeID = i.lngEmployeeID FROM inserted i;
SELECT #trg_blnEventType = i.blnEventType FROM inserted i;
SELECT #trg_datEvent = i.datEvent FROM inserted i;
SELECT #nUser = strUniqueID
FROM dbo.tblEmployees
WHERE lngID = #trg_lngEmployeeID AND blnDeleted = 0
SELECT #blnEventChar = CONVERT(NVARCHAR, #trg_blnEventType);
SELECT #Today = CONVERT(VARCHAR, GetDate(), 103);
SELECT TOP 1 #LastDateClockOut = datEvent
FROM MyTable.dbo.TimeMTSView
WHERE datEvent < #Today
AND strUniqueID = #nUser
AND blnEventType = 0
AND lngClassificationID = 0
ORDER BY datEvent DESC --This is what fails
SELECT #strLastClock = CONVERT(VARCHAR, #LastDateClockOut, 103);
-- Grab the path to the TimeMTSTrigger.exe written out by the application itself to a log file:
DECLARE #FileContents VARCHAR(MAX)
SELECT #FileContents = BulkColumn
FROM OPENROWSET(BULK'C:\MTSPath\MTSPath.sql', SINGLE_BLOB) x;
SET #cmd = 'Start ' + #FileContents + ' ' + #nUser + ' ' + #blnEventChar + ' ' + #strLastClock
EXEC xp_cmdshell #cmd
I was able to resolve this by not running the SELECT statement on the View I had created in a separate database.
The View had combined two tables from the TIMECLOCKMTS database just to make it easier to grab data. However, the SELECT statement on the View would not run in the SQL Trigger or the exe itself, both would hang and timeout.
By eliminating the View from both the Trigger and EXE and only referencing tables in the Trigger's database this runs perfectly:
USE [TIMECLOCKMTS]
GO
/****** Object: Trigger [dbo].[trgAfterInsert] Script Date: 5/19/2018 2:22:34 PM ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
ALTER TRIGGER [dbo].[trgAfterInsert] ON [dbo].[tblTimes]
AFTER INSERT
AS
declare #trg_lngClassificationID int;
SELECT #trg_lngClassificationID=i.lngClassificationID from inserted i;
IF (#trg_lngClassificationID = 0) --If the date Classification is 0 this means it is NORMAL time as opposed to VACATION or HOLIDAY
declare #trg_lngID int;
declare #trg_lngEmployeeID varchar(20);
declare #trg_blnEventType bit;
declare #blnEventChar nvarchar(1);
declare #trg_datEvent datetime;
declare #nUser varchar(255);
declare #cmd varchar(255);
declare #Today varchar(255);
declare #LastDateClockOut datetime;
SELECT #trg_lngID=i.lngID from inserted i;
SELECT #trg_lngEmployeeID=i.lngEmployeeID from inserted i;
SELECT #trg_blnEventType=i.blnEventType from inserted i;
SELECT #trg_datEvent=i.datEvent from inserted i;
SELECT #nUser = strUniqueID FROM dbo.tblEmployees WHERE lngID = #trg_lngEmployeeID AND blnDeleted = 0
SELECT #blnEventChar = CONVERT(NVARCHAR, #trg_blnEventType);
--Grab the path to the TimeMTSTrigger.exe written out by the application itself to a log file:
DECLARE #FileContents VARCHAR(MAX)
SELECT #FileContents=BulkColumn
FROM OPENROWSET(BULK'C:\MTSPath\MTSPath.sql',SINGLE_BLOB) x;
SET #cmd = 'Start ' + #FileContents + ' ' + #nUser + ' ' + #blnEventChar + ' ' + #trg_lngEmployeeID
EXEC xp_cmdshell #cmd

Arithmetic overflow error converting expression to data type datetime when running dynamic sql

I have seen a couple solutions to this SQL error(8115) but none seems to work for me. I think its because I am using dynamic SQL script. I dynamically pull data from multiple servers that have the same structure and insert into a central table. The issue I have having is that when using dynamic script as seen below I keep getting an error.
DECLARE #SQL NVARCHAR(4000);
DECLARE #counter int = 0;
DECLARE #scriptcnt int;
declare #startdate varchar(8) = '20180304';-- cast(#startdate as date)
declare #enddate varchar(8) = '20180305';--cast(#enddate as date)
select srv.name
INTO #ServerNames
from sys.servers srv
where name like 'Pulse%'
order by 1;
select name,right(name,5) store_no
into #locationCode
from #SERVERNAMES;
With ServerCTE AS (
SELECT Root=NAME + '.[POS].[dbo].'
FROM #SERVERNAMES)
Select
Root + '[Customer]' as Customer
INTO #Tables
from ServerCTE
Begin Transaction
delete from WFLCustomerMaster
--where Location_Code = '16450'--#storeNum
commit;
select
'
Begin TRY
Begin Transaction
Insert into WFLCustomerMaster(.....)
SELECT .....
FROM '+ Customer + '
where [Last_Order_Date] between '+ #startdate+ ' and '+ #enddate+ ' [This line is the issue]
commit;
END TRY
BEGIN CATCH
SELECT
ERROR_PROCEDURE() as nameofprocedure,
ERROR_NUMBER() AS errorcode,
error_line() as errorline,
ERROR_MESSAGE() as errormessage
END CATCH' as Customer into #query
from #Tables a, #locationCode b
where left(a.Customer,13) = b.name
select #scriptcnt = count(*) from #query
while #counter < #scriptcnt
begin
set #counter = #counter + 1
SET #SQL = (SELECT Customer FROM
(SELECT ROW_NUMBER() OVER (ORDER BY Customer) AS ID, Customer FROM #query) MySQL
WHERE ID=#Counter )
EXEC SP_EXECUTESQL #SQL
end
I tried converting #startdate and #enddate to date, datetime and datetime2 but nothing works.
I realize that when I hard code the date it works as per below:
...'where [Last_Order_Date] between ''20180306'' and ''20180306'''....
So I know that this part is the issue. Can someone assist?

How to optimize a slow running query with multiple 'AND' statements?

I'm trying to cleanup old records from a DB (SQL Server) which has only 1 table with millions of records in it & the query I'm using takes forever to run. Is it possible to optimize this one?
DECLARE #Count INT = -15;
DECLARE #CountThreshold INT = -4;
DECLARE #DeletedRows INT = 0 WHILE #Count <= #CountThreshold BEGIN BEGIN TRANSACTION T1
DELETE [Log]
WHERE [Date] <= DATEADD(d, #Count, getdate())
AND (windowsIdentity LIKE 'BTA-SL%'
AND Environment ='')
SET #Count = #Count + 1;
PRINT 'DELETED '+ CONVERT(VARCHAR(10),#DeletedRows) + ' ROWS ON COUNT - '+ CONVERT(VARCHAR(10),#Count)
COMMIT TRANSACTION T1 END
Your while is not util because #CountThreshold < #Count. your can do simply this:
DECLARE #CountThreshold INT = -4;
DELETE [Log]
WHERE [Date] <=DATEADD(d, #CountThreshold , getdate())
AND (windowsIdentity LIKE 'BTA-SL%'
AND Environment ='');
PRINT 'DELETED '+ CAST(##ROWCOUNT AS VARCHAR(10)) + ' rows deleted'
COMMIT TRANSACTION T1 END

SQL Select maxDate in SP

USE [MyDatabase]
GO
Object: StoredProcedure [dbo].[SP_MyProcedure]
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE PROCEDURE [dbo].[SP_MyProcedure]
-- Add the parameters for the stored procedure here
#StartDate NVARCHAR(19),
#EndDate NVARCHAR(19)
AS
BEGIN
SET NOCOUNT ON;
Insert statements for procedure here
DECLARE #FirstQuery nvarchar(1500);
DECLARE #SecondQuery nvarchar(1500);
DECLARE #TSQL nvarchar(4000);
SET #FirstQuery =
'
SELECT * FROM OPENQUERY(LinkedServer,
''
SELECT * FROM Server.Table
WHERE Name IN
(SELECT Tagname COLLATE DATABASE_DEFAULT FROM LocalServer.MyServer.dbo."NameList"
WHERE LOCATION = ''''X'''' AND SOURCE = ''''Y'''') AND TIMESTAMP >= ''''' + #StartDate + ''''' AND TIMESTAMP < ''''' + #EndDate + ''''''')';
My question is: How could I include max(date) in the code, so the #startDate would be compared to the latest available date for a column in the DB instead of todays date.
The code you see works fine, however when I choose a later date than the last existing date in the DB the query runs for a while before it returns an error that no columns were found.
Thanks in advance for any suggetions.
HELP.
The following part of the code doesn't work, but I can't figure out why. Does anyone have any suggestions? Thank you
-- Insert statements for procedure here
DECLARE #FirstQuery nvarchar(1500);
DECLARE #SecondQuery nvarchar(1500);
DECLARE #TSQL nvarchar(4000);
DECLARE #MaxTimeStamp nvarchar(19);
SET#MaxTimeStamp =
'SELECT MAX(TimeStamp) From OPENQUERY(LinkedServer)'
IF #StartDate <= #MaxTimeStamp
BEGIN
SET #FirstQuery =
'
SELECT * FROM OPENQUERY(LinkedServer,
''
SELECT * FROM Server.Table
WHERE Name IN
(SELECT Tagname COLLATE DATABASE_DEFAULT FROM LocalServer.MyServer.dbo."NameList"
WHERE LOCATION = ''''X'''' AND SOURCE = ''''Y'''') AND TIMESTAMP >= ''''' + #StartDate + ''''' AND TIMESTAMP < ''''' + #EndDate + ''''''')';
END
Well, you could replace it with a subquery:
(select max(timestamp) from NameList)
If this is a stored procedure, you might want to put this in a variable, something like:
declare #MaxTimestamp datetime;
select #MaxTimestamp = max(TimeStamp) from NameList
-- in your query, something like
coalesce(#StartTime, maxTimeStamp)
If performance is an issue, try adding an index on NameList(Location, Source, Timestamp).
Try this...
DECLARE #maxDate datetime;
SELECT #maxDate = MAX(timestamp) FROM OPENQUERY(...);
IF #StartDate <= #maxDate
BEGIN
--Your original query here
END