I have two tables, both have the same column structure. The first table has around 700 million rows. I want to copy these rows per hour into the second table.
To test my approach I've tried this query, which copies only one row from the first table into the second. The execution time for this query was around 4.5min
DECLARE #startDate DateTime2(0) = '2020-03-29 13:05:18'
DECLARE #endDate DateTime2(0) = '2020-03-29 13:05:18'
WHILE #startDate <= '2020-03-29 13:05:18'
BEGIN
SET #endDate = DATEADD(hh, 1, #startDate)
INSERT INTO "secondtable"
SELECT *
FROM "first table"
WHERE time_position >= #myCounter AND time_position <= '2020-03-29 13:05:18'
SET #startDate = DATEADD(hh, 1, #startDate)
END
For inserting rows per hour into the second table, I've tried this query but it's executing for 20+ minutes and the risk will probably run out of space and stop the query.
DECLARE #startTime DateTime2(0) = (select min(time_position) from "first table" where time_position = '2022-01-01')
DECLARE #endTime datetime2(0) = (select max(time_position) from "first table" where time_position = '2022-01-01')
WHILE #startTime <= #endTime
BEGIN
INSERT INTO "second table"
SELECT *
FROM "first table"
WHERE time_position >= #startTime AND time_position <= #endTime
SET #startTime = DATEADD(HH, 1, #startTime)
END
How can I copy these rows per hour into the second table in an efficient way?
Related
I'm trying to create a monthly Forecasting process in SQL. I've created a query for Jan and wanted to see if it's possible to loop through each additional month and store the results in a Temporary table.
Logically, I would like to set #DMonth variable to 1, run the query, store/append the results into a temp table, loop...
set #DMonth variable to 2, run the query, store/append the results into a temp table, loop...
etc.. setting #DMonth sequentially until it reaches 12 and then exit the loop.
Example variable in query:
Declare #DMonth int;
set #DMonth = 1
I believe I need to set the Loop around the Setting to of the #DMonth variable.
Well, of course, you can loop in sql.The syntax is slightly different but similar and you can easily understand it if you know basic coding.However we can only use the while loop in sql.
The syntax is like shown Below.
while loop. we basically use
While (n < X) = //Condition
BEGIN
It's not clear what do you really want to do inside your loop, but here is how to loop through and save month in a temp table:
CREATE TABLE #temp(mon int);
Declare #StartDate DATETIME = '2016-09-01',
#EndDate DATETIME = '2017-01-31'
-- Now you have 12 months and you can use below query to loop through each month starting from January
WHILE (#StartDate <= #EndDate)
BEGIN
DECLARE #Month INT = Month(#StartDate) -- At the start of loop #Month is equal to 9
-- Save #Month to temp table in each repeat
INSERT INTO #temp (mon)
SELECT #Month;
SET #StartDate = DATEADD(MONTH, 1, #StartDate); -- this line adds 1 value to #Month after first loop #Month will be 10
END
If you want to start your loop from #StartDate (September 2016) and then end it in #EndDate (January 2017) the above query does the exact same process. But if you want to change your #StartDate to 2016-01-01 and #EndDate to 2016-12-31 then use below query:
CREATE TABLE #temp(mon int);
Declare #StartDate DATETIME = DATEADD(MONTH, -8, '2016-09-01'); -- It will set #StartDate to '2016-01-01'
Declare #EndDate DATETIME = DATEADD(MONTH, 11, DATEADD(YEAR, -1, '2017-01-31')); -- It will set #EndDate to '2016-12-31'
-- Now you have 12 months and you can use below query to loop through each month starting from January
WHILE (#StartDate <= #EndDate)
BEGIN
DECLARE #Month INT = Month(#StartDate) -- At the begining #Month is equal to 1
-- Save #Month to temp table in each repeat
INSERT INTO #temp (mon)
SELECT #Month;
SET #StartDate = DATEADD(MONTH, 1, #StartDate); -- this line adds 1 value to #Month after first loop #Month will be 2
END
I'm trying to make multiple edits on a random date to leave a full random datetime within a range of dates (3 months back, 3 months forward) but setting the Hours/Minutes/Seconds/Milliseconds to 0.
I'm doing this so I can then add a random amount of time to create activity start and end times that will always fit within office working hours. The script below sets a variable for the start time of the activity, then carries out 5 separate edits to this variable to zero the time element.
Is there an easier way to carry out multiple DATEADD edits, it seems clunky!
DECLARE #STARTTIME DATETIME
DECLARE #ENDTIME DATETIME
SET #STARTTIME = (SELECT DATEADD(DAY,ABS(CHECKSUM(NEWID()) % 180), (select dateadd(dd, -90, getdate()) )))
SET #STARTTIME = (SELECT DATEADD(HH, - (SELECT DATEPART(HH,#STARTTIME)),#STARTTIME))
SET #STARTTIME = (SELECT DATEADD(MI, - (SELECT DATEPART(MI,#STARTTIME)),#STARTTIME))
SET #STARTTIME = (SELECT DATEADD(SS, - (SELECT DATEPART(SS,#STARTTIME)),#STARTTIME))
SET #STARTTIME = (SELECT DATEADD(MS, - (SELECT DATEPART(MS,#STARTTIME)),#STARTTIME))
SET #STARTTIME = (SELECT DATEADD(hh,(SELECT FLOOR(RAND()*(15-08)+08)), #STARTTIME))
SET #ENDTIME = (SELECT DATEADD(hh,(SELECT FLOOR(RAND()*(3-1)+1)), #STARTTIME))
SELECT
#STARTTIME AS 'STARTTIME',
#ENDTIME AS 'ENDTIME'
Results
STARTIME 2017-04-02 13:00:00.000
ENDTIME 2017-04-02 15:00:00.000
There is not a need to use newid() in T-SQL code. It is only needed within a query to generate multiple random numbers.
So:
set #starttime = cast(datedd(day, floor(rand() * 180 - 90), getdate()) as date);
set #starttime = dateadd(hour, floor(rand()*(15-08)+08), #starttime);
set #enddtime = dateadd(hour, floor(rand()*(3-1)+1), #starttime);
Notes:
This uses cast(. . . as date) to remove the time component of the date.
There is no need to have nested select statements.
For T-SQL code, you can use rand(), rather than the newid() work-around (that is needed within a single query to generate multiple random values).
Don't use date part abbreviations such as "hh". Just spell out the date part. The code is much easier to write and maintain.
You convert it to a Date data type would set the time part to 00:00:000
DECLARE #STARTTIME DATETIME
DECLARE #ENDTIME DATETIME
SET #STARTTIME = cast((SELECT DATEADD(DAY,ABS(CHECKSUM(NEWID()) % 180), (select dateadd(dd, -90, getdate()) ))) as date)
SET #STARTTIME = (SELECT DATEADD(hh,(SELECT FLOOR(RAND()*(15-08)+08)), #STARTTIME))
SET #ENDTIME = (SELECT DATEADD(hh,(SELECT FLOOR(RAND()*(3-1)+1)), #STARTTIME))
SELECT
#STARTTIME AS 'STARTTIME',
#ENDTIME AS 'ENDTIME'
I have a table with results:
C_EventTime L_TID
20130228162022 27200
20130228162059 27200
How would I calculate difference in seconds between the two?
In the end, I need to calculate all the differences to make a total for the month.
I've tried:
declare #startdt '20130228162022'
declare #enddt '20140101000001'
set #startdt = cast('20130101000001' as datetime)
set #enddt = cast('20140101000001' as datetime)
SELECT DATEDIFF(C_EventTime) FROM tTerminalStateLog
WHERE C_EventTime BETWEEN #startDate and #endDate
and L_TID = 27200
but suspect with my limited SQL knowledge I'm way off! Any help appreciated.
Tx
The strings you are using, don't convert nicely to a valid datetime format.
Credit to http://rdineshkumar.wordpress.com/tag/how-to-convert-yyyymmddhhmmss-to-datetimedatetime-in-sql-server/ for the formula here.
First off, DATEDIFF() takes 3 arguments. The return value, be it seconds, days, etc. Then the start/end dates. Ex DATEDIFF(SS,startdate,enddate) Doc.
Here is a sample on how to convert your two start/end values to a datetime, and calculate the difference between them.
declare #startdt datetime
declare #enddt datetime
select #startdt =
Convert(time,Dateadd(SECOND,
Right('20130228162022',2)/1,
Dateadd(MINUTE,
Right('20130228162022',4)/100,
Dateadd(hour,
Right('20130228162022',6)/10000,
'1900-01-01')))) +
convert(datetime,LEFT('20130228162022',8))
select #enddt =
Convert(time,Dateadd(SECOND,
Right('20140101000001',2)/1,
Dateadd(MINUTE,
Right('20140101000001',4)/100,
Dateadd(hour,
Right('20140101000001',6)/10000,
'1900-01-01')))) +
convert(datetime,LEFT('20140101000001',8))
select #startdt, #enddt
select DATEDIFF(ss, #startdt, #enddt)
However your table shows the values split into separate rows... This makes it slightly more complicated.
Assuming you have 2 results per L_Tid (no more, or this won't work) and you always want to compare the oldest-to-newest date (they'll never write backwards), you could do this:
declare #tTerminalStateLog table (C_EventTime varchar(15), L_Tid INT)
insert into #tTerminalStateLog
select '20130228162022',27200 union all
select '20130228162059',27200
declare #startdt varchar(15), #enddt varchar(15)
set #startdt = '20130228162022'
set #enddt = '20140101000001'
; with datesdata as
(
SELECT Convert(time,Dateadd(SECOND,
Right(C_EventTime,2)/1,
Dateadd(MINUTE,
Right(C_EventTime,4)/100,
Dateadd(hour,
Right(C_EventTime,6)/10000,
'1900-01-01')))) +
convert(datetime,LEFT(C_EventTime,8)) myDate,
L_Tid,
ROW_NUMBER() over(order by C_EventTime) as myID
FROM #tTerminalStateLog
WHERE C_EventTime BETWEEN #startdt and #enddt
and L_Tid=27200
)
select d1.myDate, d2.myDate, DATEDIFF(ss, d1.myDate, d2.myDate) [sec_diff]
from datesdata d1
left outer join datesdata d2
on d1.L_Tid=d2.L_Tid
and d2.myID=2
where d1.myID=1
I'm need a help to create a Query. My problem is I have a StartDate and EndDate and need separate this in blocs of 60 minutes.
DECLARE #STARTDATE AS SMALLDATETIME
DECLARE #ENDDATE AS SMALLDATETIME
SET #STARTDATE = '2012-11-21 11:03:00'
SET #ENDDATE = '2012-11-21 13:04:00'
I need the return:
Hour, Time
11 , 57
12 , 60
13 , 04
You could use a recursive CTE. For example:
declare #startDate datetime = '2012-11-21 22:05:00'
declare #endDate datetime = '2012-11-22 01:06:00'
; with TimeList as
(
select #startDate as dt
union all
select dateadd(hour, 1, dateadd(hour, datediff(hour, 0, dt), 0))
from TimeList
where dateadd(hour, 1, dt) < #endDate
)
select dt
from TimeList
union all
select #endDate
The snippet dateadd(hour, datediff(hour, 0, dt), 0) removes the hours and minutes from a date. It does so by calculating the number of hours since date 0 and then adding that number of hours to date 0.
Live example at SQL Fiddle.
I unsure if i understood you but this will return the hour and minute after your start date at 60 min intervals.
DECLARE #STARTDATE AS SMALLDATETIME
DECLARE #ENDDATE AS SMALLDATETIME
DECLARE #time AS TABLE(id int identity(1,1), [hour] int, [time] int)
SET #STARTDATE = '2012-11-21 11:03:00'
SET #ENDDATE = '2012-11-21 13:04:00'
WHILE #STARTDATE < #ENDDATE
BEGIN
SELECT #STARTDATE = DATEADD(MINUTE,60,#STARTDATE)
INSERT INTO #time (hour,time)
VALUES(DATEPART(HOUR,#STARTDATE),DATEPART(MINUTE,#STARTDATE))
END
SELECT * FROM #time
You coan do it in three pieces. First piece is for the first hour, 60 minus the minute value, 2nd piece is time=60 for all hours between start+1 and end, third piece is end minutes
and then insert them into a temp table, as abstractChaos has done.
Insert into temp table like AbstractChaos:
DECLARE #STARTDATE AS SMALLDATETIME
DECLARE #ENDDATE AS SMALLDATETIME
DECLARE #TIME AS TABLE(id INT IDENTITY(1,1), [HOUR] INT, [TIME] INT)
SET #STARTDATE = '2012-11-21 11:03:00'
SET #ENDDATE = '2012-11-21 13:04:00'
INSERT INTO #TIME (HOUR,TIME)
VALUES (datepart(HOUR,#startdate) ,60 - datepart(MINUTE,#startdate) )
WHILE #STARTDATE < #ENDDATE
BEGIN
SELECT #STARTDATE = DATEADD(MINUTE,60,#STARTDATE)
INSERT INTO #TIME (HOUR,TIME)
VALUES(datepart(HOUR,#STARTDATE) , 60)
END
INSERT INTO #TIME (HOUR,TIME)
VALUES(datepart(HOUR,#enddate) , datepart(MINUTE,#startdate))
I am trying to construct an sql query using a while loop that increments a datetime by one minute each iteration and then generates a select statement based on the time:
declare #dt datetime
set #dt = '2011-7-21'
while #dt < '2011-7-22'
begin
select Count(*) From Actions Where Timestamp = #dt
set #dt = DATEADD(mi, 1, #dt)
end
The query works as intended except that every iteration of the while loop seems to produce a new query entirely, rather than simply a new row. Is there a way to construct this so that its one single query and each row is generated by the incrementation of the loop?
I believe this occurs because the select statement is inside the loop, but I'm not sure how to construct it a different way that works.
EDIT - Here is what I came up with using a temporary table, but it is slow. Maybe there is a faster way? If not thats fine, atleast this works:
create table #temp
(
[DT] datetime not null,
[Total] int not null
)
declare #dt datetime
declare #result int
set #dt = '2011-7-21'
while #dt < '2011-7-22'
begin
set #result = Count(*) From Actions Where Timestamp = #dt
insert #temp ([DT],[Total]) values (#dt, #result)
set #dt = DATEADD(mi, 1, #dt)
end
select * from #temp;
drop table #temp;
One way by using a table of numbers
declare #dt datetime
set #dt = '2011-07-21'
select DATEADD(mi, number, #dt)
from master..spt_values
where type = 'P'
and DATEADD(mi, number, #dt) < '2011-07-22'
If you have your own number table, use that
See here for more info http://wiki.lessthandot.com/index.php/Date_Ranges_Without_Loops
you full query would be like
DECLARE #dt DATETIME
SET #dt = '2011-07-21'
SELECT x.SomeTime,y.TheCount FROM
(SELECT DATEADD(mi, number, #dt) as SomeTime FROM master..spt_values
WHERE TYPE = 'P'
AND DATEADD(mi, number, #dt) < '2011-07-22') x
LEFT JOIN (
SELECT TIMESTAMP, COUNT(*) AS TheCount
FROM Actions
GROUP BY TIMESTAMP
) AS y
ON x.SomeTime = dateadd(mi, datediff(mi, 0, y.Timestamp)+0, 0)
If you have a numbers table (from 0 to a million or whatever), this is relatively simple:
SELECT *
FROM Numbers AS n
LEFT JOIN (
SELECT Timestamp, COUNT(*) AS Ct
FROM Actions
GROUP BY Timestamp
) AS ActionSummary
ON ActionSummary.Timestamp = DATEADD(mi, n.Number, '2011-07-21')
WHERE DATEADD(mi, n.Number, '2011-07-21') < '2011-07-22'
ORDER BY DATEADD(mi, n.Number, '2011-07-21')
No need for loops.
There's ways to optimize this, but that should be fairly understandable as it is.
Also note that the timestamps cannot have any seconds or fractions of a second for this to work (your original has this problem as well).