SQL Query, bring data between 2 dates with limitation - sql

Question

In SQL, I am going to write a query which insert all data between 2 dates and also I want to bring then in a 1000 batch but since the number of data between those days are more than my limitation I was going to write a loop which makes the smaller period to bring the data.
here is my code:
DECLARE #StartDate DATETIME = CAST('2021-06-02 01:00:00.000' AS DATETIME)
DECLARE #EndDate DATETIME = CAST('2021-06-23 01:00:00.000' AS DATETIME)
DECLARE #RealRowCount INT = (SELECT DISTINCT SUM(##ROWCOUNT) OVER() FROM GetReport (
#StartDate, #EndDate))
DECLARE #TransactionCount INT = (SELECT DISTINCT TransactionCount FROM GetReport (
#StartDate, #EndDate))
WHILE #RealRowCount < #TransactionCount
BEGIN
DECLARE #DiffDate INT = (SELECT DATEDIFF(DAY, #StartDate, #EndDate))
SET #EndDate = DATEADD(DAY, #DiffDate/2 ,#StartDate)
SELECT *,#StartDate, #EndDate FROM GetReport (#StartDate, #EndDate)
END
PS: I was thinking about find the middle of the period of date and then change them into the new EneDate and StartDate but there is problem here!

Answer 1

Your question is not very clear. Suppose you have 10,000 records between two dates and you do not want to retrieve more than a thousand records at a time. In this case, you can use pagination. Both in the program code and in SQL.
DECLARE #StartDate DATETIME = CAST('2021-06-02 01:00:00.000' AS DATETIME)
DECLARE #EndDate DATETIME = CAST('2021-06-23 01:00:00.000' AS DATETIME)
DECLARE #RealRowCount INT = (SELECT DISTINCT COUNT(*) FROM Products WHERE InsertDate BETWEEN #StartDate AND #EndDate)
DECLARE #Counter INT = 0
WHILE #Counter <= #RealRowCount
BEGIN
SELECT *
FROM Products
WHERE InsertDate BETWEEN #StartDate AND #EndDate
ORDER BY InsertDate
OFFSET #Counter ROWS -- skip #Counter rows
FETCH NEXT 1000 ROWS ONLY -- take 1000 rows
SET #Counter = #Counter + 1000
END
Or you can get the time difference between the two dates and add the start date each time in a specific step and retrieve the data of that date.
For example, the date difference is 20 days. Increase the start date by 5 steps each time to the start date with the end date

Answer 2

I create another table to put Dates and if this table has any rows I can get 'EndDate', but if it has not any records I simply just use the date that I specified.
AccSync is the table that I insert details of my records and AccTransformation is the table wich I want to insert all of my records.
DECLARE #Count INT = (SELECT COUNT(*) FROM [AccTransaction])
DECLARE #Flag BIT = (SELECT IIF(#Count > 1, 1, 0))
DECLARE #End DATETIME = GETDATE();
DECLARE #Start DATETIME
IF(#Flag = 0)
BEGIN
SET #Start = CAST('2021-03-08' AS DATETIME2);
SET #Flag = 1
END
ELSE IF(#Flag = 1)
BEGIN
SET #Start = (SELECT TOP 1 EndDate FROM (SELECT EndDate FROM [AccSync] ORDER BY ActionDate DESC OFFSET 0 ROW) AS TT);
END
DECLARE #RealRowCount INT = (SELECT DISTINCT SUM(##ROWCOUNT) FROM [GetReport] (#Start, #End));
DECLARE #TransactionCount INT = (SELECT DISTINCT TransactionCount FROM [GetReport] (#Start, #End));
----------------------------------------------------------------------------------------------
WHILE (#RealRowCount <> #TransactionCount)
BEGIN
DECLARE #DiffDate INT = (SELECT DATEDIFF(SECOND, #Start, #End))
SET #End = DATEADD(SECOND, (#DiffDate/2), #Start)
SET #RealRowCount = (SELECT DISTINCT SUM(##ROWCOUNT) FROM [GetReport] (#Start, #End))
SET #TransactionCount = (SELECT DISTINCT TransactionCount FROM [GetReport] (#Start, #End))
END
----------------------------------------------------------------------------------------------
INSERT INTO [AccTransaction]
SELECT *
FROM [GetReport](#Start, #End)
----------------------------------------------------------------------------------------------
INSERT INTO [AccSync]
VALUES(NEWID(), GETDATE(), #Start, #End, ISNULL(#TransactionCount,0), DATEDIFF(SECOND, #Start, #End))