I'm trying to generate an output that fills in missing counts with 0s.
I'm using Oracle SQL. So far, my solution is based on Grouping records hour by hour or day by day and filling gaps with zero or null with small additions.
WITH TEMP
AS ( SELECT MINDT + ( (LEVEL - 1) / 24) DDD
FROM (SELECT TRUNC (MIN (MY_TIMESTAMP), 'HH24') MINDT,
TRUNC (MAX (MY_TIMESTAMP), 'HH24') MAXDT
FROM MAIN_TABLE.TABLE_VIEW THV
WHERE MY_TIMESTAMP BETWEEN TO_DATE ('08/01/2018:00:00:00',
'MM/DD/YYYY:HH24:MI:SS')
AND TO_DATE (
'08/03/2018:23:59:59',
'MM/DD/YYYY:HH24:MI:SS')) V
CONNECT BY MINDT + ( (LEVEL - 1) / 24) <= MAXDT)
SELECT TO_CHAR (TRUNC (D1, 'HH24'), 'YYYY-MM-DD HH24'), COUNT (D2), ID
FROM (SELECT NVL (MY_TIMESTAMP, DDD) D1,
MY_TIMESTAMP D2,
THV.ID ID
FROM MAIN_TABLE.TABLE_VIEW THV
RIGHT OUTER JOIN
(SELECT DDD FROM TEMP) AD
ON DDD = TRUNC (MY_TIMESTAMP, 'HH24')
WHERE MY_TIMESTAMP BETWEEN TO_DATE ('08/01/2018:00:00:00',
'MM/DD/YYYY:HH24:MI:SS')
AND TO_DATE ('08/03/2018:23:59:59',
'MM/DD/YYYY:HH24:MI:SS'))
GROUP BY ID, TRUNC (D1, 'HH24')
ORDER BY ID, TRUNC (D1, 'HH24')
Right now I'm getting:
CNT ID DT
4 1 2018-08-01 00
1 1 2018-08-01 01
1 1 2018-08-01 04
20 1 2018-08-01 05
76 1 2018-08-01 07
But what I want is:
CNT ID DT
4 1 2018-08-01 00
1 1 2018-08-01 01
0 1 2018-08-01 02
0 1 2018-08-01 03
1 1 2018-08-01 04
20 1 2018-08-01 05
0 1 2018-08-01 06
76 1 2018-08-01 07
Any help would be appreciated.
It works pretty smooth if you have a table to join with that has all the hours you expect to have in the results. For a table to have all the hours, it would just have 24 records.
It can be a temp table, but if it was a real table, it would simplify your report to a standard query. And if this report is used regularly, why not have an extra table? I've seen DBAs have a generic "numbers" table with lots of numbers in it for tricks like this (to get 0-23, query the table where n between 0 and 23). Another example, if you want every individual date for a 90 day period, can use a numbers table for 0-89 and add that value to a start date to be able to join on every possible date in that period.
Related
I have data with a frequency of one minute for 3 years and I would need to put it in one table to make it comparable.
Table1-2019
date_time
v_2020
01.01.2019 01:00:00
50
01.01.2019 01:01:00
49
01.01.2019 01:02:00
56
Table2-2020
date_time
v_2020
01.01.2020 01:00:00
60
01.01.2020 01:01:00
59
01.01.2020 01:02:00
56
Table3-2021
date_time
v_2020
01.01.2021 01:00:00
55
01.01.2021 01:01:00
54
01.01.2021 01:02:00
48
requested table
date_time
v_2019
v_2020
v_2021
01.01. 01:00:00
50
60
55
01.01. 01:01:00
49
59
54
01.01. 01:02:00
56
56
48
Visualisation of tables
I tried several codes, but they didn't work. With functions JOIN and LEFT, I have a problem with the format of date_time column (it is a timestamp without zone). With the SUBSTR I had also a problem with format of date_time.
Finally I tried code below, but it also doesn't work.
CREATE TABLE all AS
SELECT A.date_time, A.v_2019 FROM Table1 AS A
JOIN Table2
WHERE (select datepart(day, month, hour, minute) from A.date_time)=(select datepart(day, month, hour, minute) from Table2.date_time)
JOIN Table3
WHERE (select datepart(day, month, hour, minute) from A.date_time)=(select datepart(day, month, hour, minute) from Table3.date_time)
Once you create your tables run this query. I believe that it is straightforward:
select to_char(t1.date_time, 'mm-dd hh24:mi') date_time,
t1.v_2020 v_2020_2019,
t2.v_2020 v_2020_2020,
t3.v_2020 v_2020_2021
from table1 t1
join table2 t2 on t2.date_time = t1.date_time + interval '1 year'
join table3 t3 on t3.date_time = t1.date_time + interval '2 years';
See DB-fiddle
date_time
v_2020_2019
v_2020_2020
v_2020_2021
01-01 01:00
50
60
55
01-01 01:01
49
59
54
01-01 01:02
56
56
48
While you can do this with an INTERVAL I think you should consider a JOIN condition that uses date manipulating functions.
Keep in mind using something like WHERE DATE_TRUNC(...) or JOIN ... ON DATE_TRUNC(...) will NOT respect indexes on these fields. When passing the field value into a function you're essentially creating a black box that cannot take advantage of an index. You would need to create an index specifically on DATE_TRUNC('DAY', date_time) for example.
Here is another DBFiddle for you to consider
You can do this in a couple ways:
SELECT TO_CHAR(v19.date_time, 'MM-DD HH24:MI') datetime
, v19.v_2019
, v20.v_2020
, v21.v_2021
FROM t_2019 v19
FULL JOIN t_2020 v20
ON DATE_PART('MONTH', v19.date_time) = DATE_PART('MONTH', v20.date_time)
AND DATE_PART('DAY', v19.date_time) = DATE_PART('DAY', v20.date_time)
AND v19.date_time::TIME = v20.date_time::TIME
FULL JOIN t_2021 v21
ON DATE_PART('MONTH', v20.date_time) = DATE_PART('MONTH', v21.date_time)
AND DATE_PART('DAY', v20.date_time) = DATE_PART('DAY', v21.date_time)
AND v20.date_time::TIME = v21.date_time::TIME
;
SELECT TO_CHAR(v19.date_time, 'MM-DD HH24:MI') datetime
, v19.v_2019
, v20.v_2020
, v21.v_2021
FROM t_2019 v19
FULL JOIN t_2020 v20
ON TO_CHAR(v19.date_time, 'MM-DD HH24:MI') = TO_CHAR(v20.date_time, 'MM.DD HH24:MI')
FULL JOIN t_2021 v21
ON TO_CHAR(v20.date_time, 'MM-DD HH24:MI') = TO_CHAR(v21.date_time, 'MM.DD HH24:MI')
;
Both of these result in the following:
datetime
v_2019
v_2020
v_2021
01-01 01:00
50
60
55
01-01 01:01
49
59
54
01-01 01:02
56
56
48
My o/p of the query is like below
Date Hour Orders
2018-02-22 00 22
2018-02-22 03 12
2018-02-22 04 12
2018-02-22 08 12
But I want it like this
Date Hour Orders
2018-02-22 00 22
2018-02-22 01 0
2018-02-22 02 0
2018-02-22 03 12
2018-02-22 04 12
2018-02-22 05 0
2018-02-22 06 0
2018-02-22 07 0
2018-02-22 08 12
Even though there is no order placed during that hour, that hour should be displayed and it should show order placed as 0.
Another way to do that is to use oracle "data-densification" method. The key thing is to add partition by clause after outer join like below.
With Needed_Hours (hr) as (
select lpad(level-1, 2, '0')
from dual
connect by level <= (
select max(to_number(Hour)) - min(to_number(Hour)) + 1
from your_table
)
)
select t."Date", h.hr hour, nvl(t.orders, 0)orders
from Needed_Hours h
left join your_table t partition by (t."Date")
on h.hr = t.hour;
Assuming you have timestamp data(namely dt), split into two columns as date and hour and apply outer join among the data set derived through row generation by difference of extremum hours and the original table(namely t) such as
WITH t1 AS
(
SELECT level-1 AS lvl
FROM dual
CONNECT BY level <= ( SELECT EXTRACT(HOUR FROM MAX(dt) - MIN(dt))+1 FROM t )
), t2 AS
(
SELECT TRUNC(dt) AS "Date", EXTRACT( HOUR FROM dt ) AS hr, orders FROM t
)
SELECT MAX("Date") OVER (ORDER BY lvl ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW ) AS "Date",
lvl AS "Hour", NVL(orders,0) AS "Orders"
FROM t1
LEFT JOIN t2
ON t2.hr = t1.lvl
ORDER BY t1.lvl
Demo
I have table records as -
date n_count
2020-02-19 00:00:00 4
2020-07-14 00:00:00 1
2020-07-17 00:00:00 1
2020-07-30 00:00:00 2
2020-08-03 00:00:00 1
2020-08-04 00:00:00 2
2020-08-25 00:00:00 2
2020-09-23 00:00:00 2
2020-09-30 00:00:00 3
2020-10-01 00:00:00 11
2020-10-05 00:00:00 12
2020-10-19 00:00:00 1
2020-10-20 00:00:00 1
2020-10-22 00:00:00 1
2020-11-02 00:00:00 376
2020-11-04 00:00:00 72
2020-11-11 00:00:00 1
I want to be grouped all the records into months for finding month total count which is working, but there is a missing of month. how to fill this gap.
time month_count
"2020-02-01" 4
"2020-07-01" 4
"2020-08-01" 5
"2020-09-01" 5
"2020-10-01" 26
"2020-11-01" 449
This is what I have tried.
SELECT (date_trunc('month', date))::date AS time,
sum(n_count) as month_count
FROM table1
group by time
order by time asc
You can use generate_series() to generate all starts of months between the earliest and latest date available in the table, then bring the table with a left join:
select d.dt, coalesce(sum(t.n_count), 0) as month_count
from (
select generate_series(date_trunc('month', min(date)), date_trunc('month', max(date)), '1 month') as dt
from table1
) as d(dt)
left join table1 t on t.date >= d.dt and t.date < d.dt + interval '1 month'
group by d.dt
order by d.dt
I would simply UNION a date series, generated from MIN and MAX date:
demo:db<>fiddle
WITH cte AS ( -- 1
SELECT
*,
date_trunc('month', date)::date AS time
FROM
t
)
SELECT
time,
SUM(n_count) as month_count --3
FROM (
SELECT
time,
n_count
FROM cte
UNION
SELECT -- 2
generate_series(
(SELECT MIN(time) FROM cte),
(SELECT MAX(time) FROM cte),
interval '1 month'
)::date,
0
) s
GROUP BY time
ORDER BY time
Use CTE to calculate date_trunc only once. Could be left out if you like to call your table twice in the UNION below
Generate monthly date series from MIN to MAX date containing your n_count value = 0. Add it to the table
Do your calculation
This question already has answers here:
Row for each date from start date to end date
(2 answers)
Generate series of months for every row in Oracle
(1 answer)
Create all months list from a date column in ORACLE SQL
(3 answers)
Closed 2 years ago.
What I'm trying to do is take a record that looks like this:
Start_DT End_DT ID
4/5/2013 10/9/2013 1
and change it to look like this:
DT ID
4/1/2013 1
5/1/2013 1
6/1/2013 1
7/1/2013 1
8/1/2013 1
9/1/2013 1
10/1/2013 1
I am having difficult time making this work. Any help would be appreciated.
Thanks
You can use a recursive CTE for this:
with dates (dte, end_mon, id) as (
select trunc(start_dt, 'MON') as dte, trunc(end_dt, 'MON') as end_mon, id
from t
union all
select dte + interval '1' month, end_mon, id
from dates
where dte < end_mon
)
select *
from dates;
Here is a db<>fiddle.
Alternatively, but similarly - row generator technique is what you're looking for.
(My date format is DD.MM.YYYY; can't tell for yours so I'm just guessing as both of your dates can be read in two ways, e.g. 4/5/2013 - is it 4th of May or 5th of April?).
SQL> with dates (start_dt, end_dt, id) as
2 (select date '2013-05-04', date '2013-09-10', 1 from dual)
3 select start_dt + level - 1 dt, id
4 from dates
5 connect by level <= end_dt - start_dt + 1
6 order by dt;
DT ID
---------- ----------
04.05.2013 1
05.05.2013 1
06.05.2013 1
07.05.2013 1
08.05.2013 1
<snip>
06.09.2013 1
07.09.2013 1
08.09.2013 1
09.09.2013 1
10.09.2013 1
130 rows selected.
SQL>
This can be achieved by using CONNECT BY to generate the months between each date.
Query
WITH
dates (start_date, end_date, id)
AS
(SELECT TO_DATE ('4/5/2013', 'MM/DD/YYYY'), TO_DATE ('10/9/2013', 'MM/DD/YYYY'), 1
FROM DUAL)
SELECT TO_CHAR (ADD_MONTHS (TRUNC (start_date, 'MM'), LEVEL - 1), 'MM/DD/YYYY') AS dt, id
FROM dates
CONNECT BY ADD_MONTHS (TRUNC (start_date, 'MM'), LEVEL - 1) <= TRUNC (end_date, 'MM');
Result
DT ID
_____________ _____
04/01/2013 1
05/01/2013 1
06/01/2013 1
07/01/2013 1
08/01/2013 1
09/01/2013 1
10/01/2013 1
I am using this code to calculate the difference between two dates ignoring weekends:
SELECT To_date(SYSDATE) -
To_date('01.07.2014', 'DD.MM.YYYY')
- 2 * ( TRUNC(Next_day(To_date(SYSDATE) - 1, 'FRI'))
- TRUNC( Next_day(To_date('01.07.2014' , 'DD.MM.YYYY')
- 1, 'FRI')) ) / 7 AS DAYS_BETWEEN
FROM dual
I have another table called table1 in which the column "date" exists (its type is "DATE") in which all dates where a holiday is are written down.
Example table 1:
DATES
12.06.2011
19.06.2014
09.05.2013
...
I am trying to make my code check this table and that if one date is between the two dates above it makes -1 day in the output.
It should be easy if you divide it into following tasks:
Generate all the dates between the two given dates using Row Generator method as shown here.
Ignore the dates which are weekend, i.e. Saturdays and Sundays
Check whether the dates in the range are having any match in the holiday table.
The following row generator query will give you the total count of weekdays, i.e. not including Saturdays and Sundays:
SQL> WITH dates AS
2 (SELECT to_date('01/01/2014', 'DD/MM/YYYY') date1,
3 to_date('31/12/2014', 'DD/MM/YYYY') date2
4 FROM dual
5 )
6 SELECT SUM(weekday) weekday_count
7 FROM
8 (SELECT
9 CASE
10 WHEN TO_CHAR(date1+LEVEL-1, 'DY','NLS_DATE_LANGUAGE=AMERICAN')
11 NOT IN ('SAT', 'SUN')
12 THEN 1
13 ELSE 0
14 END weekday
15 FROM dates
16 CONNECT BY LEVEL <= date2-date1+1
17 )
18 /
WEEKDAY_COUNT
-------------
261
SQL>
Now, based on above row generator query, let's see a test case.
The following query will calculate the count of working days between 1st Jan 2014 and 31st Dec 2014 excluding the holidays as mentioned in the table.
The WITH clause is only to use it as tables, in your case you can simply use your holiday table.
SQL> WITH dates
2 AS (SELECT To_date('01/01/2014', 'DD/MM/YYYY') date1,
3 To_date('31/12/2014', 'DD/MM/YYYY') date2
4 FROM dual),
5 holidays
6 AS (SELECT To_date('12.06.2011', 'DD.MM.YYYY') holiday FROM dual UNION ALL
7 SELECT To_date('19.06.2014', 'DD.MM.YYYY') holiday FROM dual UNION ALL
8 SELECT To_date('09.05.2013', 'DD.MM.YYYY') holiday FROM dual),
9 count_of_weekdays
10 AS (SELECT SUM(weekday) weekday_count
11 FROM (SELECT CASE
12 WHEN To_char(date1 + LEVEL - 1, 'DY',
13 'NLS_DATE_LANGUAGE=AMERICAN')
14 NOT IN (
15 'SAT',
16 'SUN' ) THEN 1
17 ELSE 0
18 END weekday
19 FROM dates
20 CONNECT BY LEVEL <= date2 - date1 + 1)),
21 count_of_holidays
22 AS (SELECT Count(*) holiday_count
23 FROM holidays
24 WHERE holiday NOT BETWEEN To_date('01/01/2015', 'DD/MM/YYYY') AND
25 To_date('31/03/2015', 'DD/MM/YYYY'))
26 SELECT weekday_count - holiday_count as working_day_count
27 FROM count_of_weekdays,
28 count_of_holidays
29 /
WORKING_DAY_COUNT
-----------------
258
SQL>
There were total 261 weekdays, out of which there were 3 holidays in holiday table. So, total count of working days in the output is 261 - 3 = 258.
SELECT To_date(sysdate)- To_date('01.07.2014','DD.MM.YYYY')
- (2 * (to_char(To_date(sysdate), 'WW') - to_char(To_date('01.07.2014','DD.MM.YYYY'), 'WW'))) AS DAYS_BETWEEN
FROM dual