I have a table that stores the start-date and number of the hours. I have also another time table as reference to working days. My main goal is the divide this hours to the working days.
For examle:
ID Date Hour
1 20210504 40
I want it to be structured as
ID Date Hour
1 20210504 8
1 20210505 8
1 20210506 8
1 20210507 8
1 20210510 8
I manage to divide the hours with the given code but couldn't manage to make it in working days.
WITH cte1 AS
(
select 1 AS ID, 20210504 AS Date, 40 AS Hours --just a test case
), working_days AS
(
select date from dateTable
),
cte2 AS
(
select ID, Date, Hours, IIF(Hours<=8, Hours, 8) AS dailyHours FROM cte1
UNION ALL
SELECT
cte2.ID,
cte2.Date + 1
,cte2.Hours - 8
,IIF(Hours<=8, Hours, 8)
FROM cte2
JOIN cte1 t ON cte2.ID = t.ID
WHERE cte2.HOURS > 8 AND cte2.Date + 1 IN (select * from working_days)
When I use it like this it only gives me this output with one day missing
ID Date Hour
1 20210504 8
1 20210505 8
1 20210506 8
1 20210507 8
To solve your problem you need to build your calendar in the right way,
adding also to working_days a ROW_NUMBER to get correct progression.
declare #date_start date = '2021-05-01'
;WITH
cte1 AS (
SELECT * FROM
(VALUES
(1, '20210504', 40),
(2, '20210505', 55),
(3, '20210503', 44)
) X (ID, Date, Hour)
),
numbers as (
SELECT ROW_NUMBER() over (order by o.object_id) N
FROM sys.objects o
),
cal as (
SELECT cast(DATEADD(day, n, #date_start) as date) d, n-1 n
FROM numbers n
where n.n<32
),
working_days as (
select d, ROW_NUMBER() over (order by n) dn
from cal
where DATEPART(weekday, d) < 6 /* monday to friday in italy (country dependent) */
),
base as (
SELECT t.ID, t.Hour, w.d, w.dn
from cte1 t
join working_days w on w.d = t.date
)
SELECT t.ID, w.d, iif((8*n)<=Hour, 8, 8 + Hour - (8*n) ) h
FROM base t
join numbers m on m.n <= (t.Hour / 8.0) + 0.5
join working_days w on w.dn = t.dn + N -1
order by 1,2
You can use a recursive CTE. This should do the trick:
with cte as (
select id, date, 8 as hour, hour as total_hour
from t
union all
select id, dateadd(day, 1, date),
(case when total_hour < 8 then total_hour else 8 end),
total_hour - 8
from cte
where total_hour > 0
)
select *
from cte;
Note: This assumes that total_hour is at least 8, just to avoid a case expression in the anchor part of the CTE. That can trivially be added.
Also, if there might be more than 100 days, you will need option (maxrecursion 0).
Related
I am attempting to understand the progression of my observations that are on time relative to when they were expected, regardless of the date they were expected. Therefore, I want to reindex each observation and generate a list that starts at day 0 (on the expected day) and then calculate forward for 10 more days (arbitrary).
I am testing this in BigQuery:
CREATE TABLE `db.tbl` (
id INTEGER,
expected DATE,
actual DATE
)
INSERT INTO `db.tbl`
( id , expected , actual )
VALUES
( 1 , '2022-01-01' , '2022-01-02' ),
( 2 , '2022-01-11' , '2022-01-20' ),
( 3 , '2022-01-21' , '2022-01-20' )
So, the first row represents an observation that was "missing"/"late"/"not on time" on day 0 (2022-01-01) and then "on time" from day 1 (2022-01-02) until the end of my window of interest (day 10).
The second row represents an observation that was "late" from day 0 (2022-01-11) to day 8 (2022-01-19) and "on time" after that.
The third row represents an observation that was observed early, so it should be "on time" from day 0 through day 10.
I would want the result to be:
day count fraction
0 1 0.33
1 2 0.67
2 2 0.67
3 2 0.67
4 2 0.67
5 2 0.67
6 2 0.67
7 2 0.67
8 2 0.67
9 3 1.00
10 3 1.00
Is this possible with a SELECT statement?
CREATE TEMP TABLE sample (
id INTEGER,
expected DATE,
actual DATE
);
INSERT INTO sample
( id , expected , actual )
VALUES
( 1 , '2022-01-01' , '2022-01-02' ),
( 2 , '2022-01-11' , '2022-01-20' ),
( 3 , '2022-01-21' , '2022-01-20' );
WITH observations AS (
SELECT day, COUNTIF(v = '1') AS count, (SELECT COUNT(id) FROM sample) AS total
FROM sample,
UNNEST([IF(DATE_DIFF(actual, expected, DAY) < 0, 0, DATE_DIFF(actual, expected, DAY))]) diff,
UNNEST(SPLIT(REPEAT('0', diff) || REPEAT('1', 10 - diff), '')) v WITH OFFSET day
GROUP BY 1
)
SELECT day, count, ROUND(count / total, 2) AS fraction
FROM observations;
output:
Consider below
select day, sum(ontime) cnt, round(avg(ontime),2) fraction
from (
select day, if(dt < actual, 0, 1) ontime
from your_table,
unnest(generate_array(0,10)) day
left join unnest(generate_date_array(expected, actual)) dt with offset as day
using(day)
)
group by day
if applied to sample data in your question
with your_table as (
select 1 id, date '2022-01-01' expected, date '2022-01-02' actual union all
select 2, '2022-01-11' , '2022-01-20' union all
select 3, '2022-01-21' , '2022-01-20'
)
output is
I have a table with a column patterns something like '1,2,3,4' and a column name frequency which represents how many times each pattern shall repeat. For ex.
I have a generated a pattern but not able to skip weekends, here is my current code -
;WITH TestCteNew (EmployeeId, ShiftId, StartDate, Enddate)AS (
SELECT
employeeid.n.query('.[1]').value('.', 'INT') EmployeeId,
shiftid.n.query('.[1]').value('.', 'INT') ShiftId
,StartDate, Enddate
FROM
TestCte
CROSS APPLY employeeid.nodes('x') AS employeeid(n)
CROSS APPLY shiftid.nodes('x') AS shiftid(n)
CROSS APPLY (SELECT TOP(2) ROW_NUMBER() OVER(ORDER BY (SELECT NULL))-1 r_num FROM SYS.ALL_OBJECTS A , SYS.ALL_OBJECTS B) X)
,TestCteFinal(EmployeeId, ShiftId, SDate,r_num) AS (
SELECT EmployeeId, ShiftId, StartDate + ROW_NUMBER() OVER (PARTITION BY EmployeeId ORDER BY r_num)-1 AS SD, x.r_num
FROM TestCteNew
CROSS APPLY (SELECT TOP(2) ROW_NUMBER() OVER(ORDER BY (SELECT NULL))-1 r_num FROM SYS.ALL_OBJECTS A , SYS.ALL_OBJECTS B) X)
With the above code I am able to generate something like below
Account DayOfWeek Shifts Shifts
1 20201007 100 1
2 20201107 100 1 (Saturday)
3 20201207 100 2 (Sunday)
4 20201307 100 2
5 20201407 100 3
6 20201507 100 3
7 20201607 100 4
8 20201707 100 4
...Same set of records above once again
Here the issue is my pattern is not skipping weekends, I want something like below.
DECLARE #Pattern VARCHAR(10)= '1,2,3,4', #Frequency INT=2
Account DayOfWeek Shifts Shifts
1 20201007 100 1
2 20201107 100 0 (Saturday)
3 20201207 100 0 (Sunday)
4 20201307 100 1
5 20201407 100 2
6 20201507 100 2
7 20201607 100 3
8 20201707 100 3
9 20201807 100 0 (Saturday)
10 20201907 100 0 (Sunday)
12 20202007 100 4
13 20202107 100 4
14 20202207 100 1
15 20202307 100 1
I want to repeat the pattern in the above defined format.
This is pseudo-code because you the source of the data isn't clear. You refer to a TestCte that isn't defined. The query never uses the #Frequency and #Pattern variables that head the desired outer. The output has an Account column that isn't mentioned anywhere else... But perhaps this approach will work better.
declare #Frequency int = 2;
declare #StartDt date = '20200710';
with num(n) as (
select top (256) row_number() over (order by (select null)) - 1
from sys.all_objects
), dates(n, dt) as (
select row_number() over (order by n), dateadd(day, n, #StartDt)
from num
-- filter weekend dates
where datepart(weekday, dateadd(day, n, #StartDt)) between 1 and 5
)
select p.n, r.n, d.dt
from
patterns as p -- this comes from xml? I'm going to assume these are numbered somehow
inner join num as r /* repetitions */
on r.n <= #Frequency -- I think something like "multiples" might be a better name
inner join dates as d
on d.n = #Frequency * p.n + r.n
I'm not even sure if this can/should be done is SQL but here goes.
I have a table that stores a start date and an end date like so
userPingId createdAt lastUpdatedAt
1 2017-10-17 11:31:52.160 2017-10-18 14:31:52.160
I want to return a result set that groups the results by date and if they were active between different points between the two date.
The different points are
Morning - Before 12pm
Afternoon - Between 12pm and 5pm
Evening - After 5pm
So for example I would get the following results
sessionDate morning afternoon evening
2017-10-17 1 1 1
2017-10-18 1 1 0
Here is what I have so far and I believe that it's quite close but the fact I can't get the results I need make me think that this might not be possible in SQL (btw i'm using a numbers lookup table in my query which I saw on another tutorial)
DECLARE #s DATE = '2017-01-01', #e DATE = '2018-01-01';
;WITH d(sessionDate) AS
(
SELECT TOP (DATEDIFF(DAY, #s, #e) + 1) DATEADD(DAY, n-1, #s)
FROM dbo.Numbers ORDER BY n
)
SELECT
d.sessionDate,
sum(case when
(CONVERT(DATE, createdAt) = d.sessionDate AND datepart(hour, createdAt) < 12)
OR (CONVERT(DATE, lastUpdatedAt) = d.sessionDate AND datepart(hour, lastUpdatedAt) < 12)
then 1 else 0 end) as Morning,
sum(case when
(datepart(hour, createdAt) >= 12 and datepart(hour, createdAt) < 17)
OR (datepart(hour, lastUpdatedAt) >= 12 and datepart(hour, lastUpdatedAt) < 17)
OR (datepart(hour, createdAt) < 12 and datepart(hour, lastUpdatedAt) >= 17)
then 1 else 0 end) as Afternoon,
sum(case when datepart(hour, createdAt) >= 17 OR datepart(hour, lastUpdatedAt) >= 17 then 1 else 0 end) as Evening
FROM d
LEFT OUTER JOIN MYTABLE AS s
ON s.createdAt >= #s AND s.lastUpdatedAt <= #e
AND (CONVERT(DATE, s.createdAt) = d.sessionDate OR CONVERT(DATE, s.lastUpdatedAt) = d.sessionDate)
WHERE d.sessionDate >= #s AND d.sessionDate <= #e
AND userPingId = 49
GROUP BY d.sessionDate
ORDER BY d.sessionDate;
Building on what you started with the numbers table, you can add the time ranges to your adhoc calendar table using another common table expression using cross apply()
and the table value constructor (values (...),(...)).
From there, you can use an inner join based on overlapping date ranges along with conditional aggregation to pivot the results:
declare #s datetime = '2017-01-01', #e datetime = '2018-01-01';
;with n as (select n from (values(0),(1),(2),(3),(4),(5),(6),(7),(8),(9)) t(n))
, d as ( /* adhoc date/numbers table */
select top (datediff(day, #s, #e)+1)
SessionDate=convert(datetime,dateadd(day,row_number() over(order by (select 1))-1,#s))
from n as deka cross join n as hecto cross join n as kilo
cross join n as tenK cross join n as hundredK
order by SessionDate
)
, h as ( /* add time ranges to date table */
select
SessionDate
, StartDateTime = dateadd(hour,v.s,SessionDate)
, EndDateTime = dateadd(hour,v.e,SessionDate)
, v.point
from d
cross apply (values
(0,12,'morning')
,(12,17,'afternoon')
,(17,24,'evening')
) v (s,e,point)
)
select
t.userPingId
, h.SessionDate
, morning = count(case when point = 'morning' then 1 end)
, afternoon = count(case when point = 'afternoon' then 1 end)
, evening = count(case when point = 'evening' then 1 end)
from t
inner join h
on t.lastupdatedat >= h.startdatetime
and h.enddatetime > t.createdat
group by t.userPingId, h.SessionDate
rextester demo: http://rextester.com/MVB77123
returns:
+------------+-------------+---------+-----------+---------+
| userPingId | SessionDate | morning | afternoon | evening |
+------------+-------------+---------+-----------+---------+
| 1 | 2017-10-17 | 1 | 1 | 1 |
| 1 | 2017-10-18 | 1 | 1 | 0 |
+------------+-------------+---------+-----------+---------+
Alternately, you could use pivot() instead of conditional aggregation in the final select:
select UserPingId, SessionDate, Morning, Afternoon, Evening
from (
select
t.userPingId
, h.SessionDate
, h.point
from t
inner join h
on t.lastupdatedat >= h.startdatetime
and h.enddatetime > t.createdat
) t
pivot (count(point) for point in ([Morning], [Afternoon], [Evening])) p
rextester demo: http://rextester.com/SKLRG63092
You can using PIVOT on CTE's to derive solution to this problem.
Below is the test table
select * from ping
Below is the sql query
;with details as
(
select userPingId, createdAt as presenceDate , convert(date, createdAt) as
onlyDate,
datepart(hour, createdAt) as onlyHour
from ping
union all
select userPingId, lastUpdatedAt as presenceDate , convert(date,
lastUpdatedAt) as onlyDate,
datepart(hour, lastUpdatedAt) as onlyHour
from ping
)
, cte as
(
select onlyDate,count(*) as count,
case
when onlyHour between 0 and 12 then 'morning'
when onlyHour between 12 and 17 then 'afternoon'
when onlyHour>17 then 'evening'
end as 'period'
from details
group by onlyDate,onlyHour
)
select onlyDate, coalesce(morning,0) as morning,
coalesce(afternoon,0) as afternoon , coalesce(evening,0) as evening from
(
select onlyDate, count,period
from cte ) src
pivot
(
sum(count)
for period in ([morning],[afternoon],[evening])
) p
Below is the final result
This is a fairly similar answer to the one already posted, I just wanted the practice with PIVOT :)
I use a separate table with the time sections in it. this is then cross joined with the number table to create a date and time range for bucketing. i join this to the data and then pivot it (example: https://data.stackexchange.com/stackoverflow/query/750496/bucketing-data-into-date-am-pm-evening-and-pivoting-results)
SELECT
*
FROM (
SELECT
[userPingId],
dt,
[desc]
FROM (
SELECT
DATEADD(D, number, #s) AS dt,
CAST(DATEADD(D, number, #s) AS datetime) + CAST(s AS datetime) AS s,
CAST(DATEADD(D, number, #s) AS datetime) + CAST(e AS datetime) AS e,
[desc]
FROM #numbers
CROSS JOIN #times
WHERE number < DATEDIFF(D, #s, #e)
) ts
INNER JOIN #mytable AS m
ON m.createdat < ts.e
AND m.[lastUpdatedAt] >= ts.s
) src
PIVOT
(
COUNT([userPingId])
FOR [desc] IN ([am], [pm], [ev])
) piv;
the #times table is just:
s e desc
00:00:00.0000000 12:00:00.0000000 am
12:00:00.0000000 17:00:00.0000000 pm
17:00:00.0000000 23:59:59.0000000 ev
Given a table of intervals, can I efficiently query for the number of currently open intervals at the start of each interval (including the current interval itself)?
For example, given the following table:
start_time end_time
1 10
2 5
3 4
5 6
7 11
19 20
I want the following output:
start_time count
1 1
2 2
3 3
5 3
7 2
19 1
On small datasets, I can solve this problem by joining the dataset against itself:
WITH intervals AS (
SELECT 1 AS start, 10 AS end UNION ALL
SELECT 2, 5 UNION ALL
SELECT 3, 4 UNION ALL
SELECT 5, 6 UNION ALL
SELECT 7, 11 UNION ALL
SELECT 19, 20
)
SELECT
a.start_time,
count(*)
FROM
intervals a CROSS JOIN intervals b
WHERE
a.start_time >= b.start_time AND
a.start_time <= b.end_time
GROUP BY a.start_time
ORDER BY a.start_time
With large datasets the CROSS JOIN is both impractical and unnecessary, since any given answer only depends on a small number of preceding intervals (when sorted by start_time). In fact, on the dataset that I have, it times out. Is there a better way to achieve this?
... CROSS JOIN is both impractical and unnecessary ...
Is there a better way to achieve this?
Try below for BigQuery Standard SQL. No JOINs involved
#standardSQL
SELECT
start_time,
(SELECT COUNT(1) FROM UNNEST(ends) AS e WHERE e >= start_time) AS cnt
FROM (
SELECT
start_time,
ARRAY_AGG(end_time) OVER(ORDER BY start_time) AS ends
FROM intervals
)
-- ORDER BY start_time
You can test/play with it using below example with dummy data from your question
#standardSQL
WITH intervals AS (
SELECT 1 AS start_time, 10 AS end_time UNION ALL
SELECT 2, 5 UNION ALL
SELECT 3, 4 UNION ALL
SELECT 5, 6 UNION ALL
SELECT 7, 11 UNION ALL
SELECT 19, 20
)
SELECT
start_time,
(SELECT COUNT(1) FROM UNNEST(ends) AS e WHERE e >= start_time) AS cnt
FROM (
SELECT
start_time,
ARRAY_AGG(end_time) OVER(ORDER BY start_time) AS ends
FROM intervals
)
-- ORDER BY start_time
I have a query that returns daily data for the last 7 days. I would like to know the syntax for getting weekly data for the last 4 weeks using bigquery
-Week Total
week 1 15
week 2 20
week 3 35
Something along those lines:
SELECT
YEAR(day) AS year,
WEEK(day) AS week,
SUM(metric) AS total
FROM YourTable
WHERE WEEK(CURRENT_DATE()) - WEEK(day) < 4
GROUP BY 1, 2
To test/play - you can use below approach that hopefuly mimics your data
SELECT
YEAR(day) AS year,
WEEK(day) AS week,
SUM(metric) AS total
FROM (
SELECT
DATE(DATE_ADD(TIMESTAMP('2016-01-01'), pos - 1, "DAY")) AS day,
CAST(100 * RAND() AS INTEGER) AS metric
FROM (
SELECT ROW_NUMBER() OVER() AS pos, *
FROM (FLATTEN((
SELECT SPLIT(RPAD('', 1 + DATEDIFF(TIMESTAMP(CURRENT_DATE()), TIMESTAMP('2016-01-01')), '.'),'') AS h
FROM (SELECT NULL)),h
)))
) AS YourTable
WHERE WEEK(CURRENT_DATE()) - WEEK(day) < 4
GROUP BY 1, 2