Value for missing data in SQL - sql

Question for the SQL gurus. I have a table with 3 columns. [Date, Meter, Quality], where there will only be one line per date for each meter. As an example:
SELECT * FROM MyDB WHERE Meter = 'MeterX' AND Date > '1-AUG-2022' AND Date <= '5-AUG-2022' ORDER BY Date;
I would query much larger date ranges so would usually miss if there is a date missing. Is there a way that I can have a value returned in the Quality column like "Missing" if that partiqular day is missing from the database? This means that I also need the missing date in the Date column. I also only have read access, so no creating temp tables to join with.
Thank you.

Use a PARTITIONed OUTER JOIN to a row-generator:
SELECT c.day,
m.meter,
COALESCE(m.quality, 0) AS quality
FROM (
SELECT DATE '2022-08-01' + (LEVEL - 1) AS day
FROM DUAL
CONNECT BY DATE '2022-08-01' + (LEVEL - 1) <= DATE '2022-08-05'
) c
LEFT OUTER JOIN MyDB m
PARTITION BY (m.meter)
ON (c.day <= m."DATE" and m."DATE" < c.day + 1)
WHERE m.Meter = 'MeterX'
ORDER BY c.day;
Which, for the sample data:
CREATE TABLE mydb ("DATE", meter, quality) AS
SELECT DATE '2022-08-01', 'MeterX', 42 FROM DUAL UNION ALL
SELECT DATE '2022-08-02', 'MeterX', 23 FROM DUAL UNION ALL
SELECT DATE '2022-08-04', 'MeterX', 7 FROM DUAL UNION ALL
SELECT DATE '2022-08-05', 'MeterX', 99 FROM DUAL;
Outputs:
DAY
METER
QUALITY
01-AUG-22
MeterX
42
02-AUG-22
MeterX
23
03-AUG-22
MeterX
0
04-AUG-22
MeterX
7
05-AUG-22
MeterX
99
db<>fiddle here

for postgres this could work for you
with date_dimension as (
SELECT dd::DATE
FROM generate_series
( '2022-08-01'::timestamp
, '2022-08-05'::timestamp
, '1 day'::interval) dd
)
select *
from my_table
left join date_dimension on date_dimension.dd = my_table.Date
where Meter = 'MeterX'
and Date > '2022-08-01'
and Date <= '2022-08-05'
order by Date;

Related

Get the last time before the call - SQL Query

I have here a set of 2 tables that I need to bash.
First table is the list of time and date the customer contacted us, its not unique.
The next table is the escalated call they made to us.
What I need to do is to show the date and time before the escalated call.
I can do simple left join based on customer ID, but having issue on the getting the last call.
Hope that I can get answers + explanation that I can use moving forward.
Here's my code so far:
Select a.customer id, a.contact_time, b.date of contact time as last_contact
from escalated_call a
left join all calls b on a.customer id = b.customer ID
Just Use a Where Clause
Select a.customerid,
a.contact_time,
b.DateOfContactTime as last_contact
from escalated_call AS a LEFT JOIN Calls AS b on a.customerID = b.customerID
WHERE a.contact_time < b.DateOfContactTime
You just need an aggregate max here, you can also do it with a correlated subquery but it’s probably not worth it.
You may need to correct your column names, I’ve just guessed you have underscored instead of the spaces
Select a.customer_id
,a.contact_time
,max(b.date_of_contact_time) as last_contact
from escalated_call a
left join all_calls b
on a.customer_id = b.customer_ID
Group by a.customer_id, a.contact_time
From Oracle 12, you can use a LATERAL join and return the FIRST ROW ONLY:
SELECT ec.*, ac.dt
FROM escalated_calls ec
LEFT OUTER JOIN LATERAL (
SELECT ac.dt
FROM all_calls ac
WHERE ac.customer_id = ec.customer_id
AND ac.dt <= ec.ct
ORDER BY ac.dt DESC
FETCH FIRST ROW ONLY
) ac
ON (1 = 1)
Which, for the sample data:
CREATE TABLE all_calls(customer_id, dt) AS
SELECT 1, DATE '2019-12-24' + INTERVAL '00:00' HOUR TO MINUTE FROM DUAL UNION ALL
SELECT 1, DATE '2019-12-24' + INTERVAL '00:15' HOUR TO MINUTE FROM DUAL UNION ALL
SELECT 1, DATE '2019-12-24' + INTERVAL '00:35' HOUR TO MINUTE FROM DUAL UNION ALL
SELECT 1, DATE '2019-12-24' + INTERVAL '01:00' HOUR TO MINUTE FROM DUAL UNION ALL
SELECT 2, DATE '2019-12-24' + INTERVAL '00:00' HOUR TO MINUTE FROM DUAL UNION ALL
SELECT 2, DATE '2019-12-24' + INTERVAL '00:15' HOUR TO MINUTE FROM DUAL UNION ALL
SELECT 2, DATE '2019-12-24' + INTERVAL '00:35' HOUR TO MINUTE FROM DUAL UNION ALL
SELECT 2, DATE '2019-12-24' + INTERVAL '01:00' HOUR TO MINUTE FROM DUAL;
CREATE TABLE escalated_calls (customer_id, ct) AS
SELECT 1, DATE '2019-12-24' + INTERVAL '00:45' HOUR TO MINUTE FROM DUAL UNION ALL
SELECT 2, DATE '2019-12-24' + INTERVAL '00:05' HOUR TO MINUTE FROM DUAL;
Outputs:
CUSTOMER_ID
CT
DT
1
2019-12-24 00:45:00
2019-12-24 00:35:00
2
2019-12-24 00:05:00
2019-12-24 00:00:00
db<>fiddle here
You can also use a subquery in the select clause to solve this problem.
SELECT e.*
, ( SELECT max(a.Date_Of_Contact_Time)
FROM all_calls a
WHERE a.customer_id = e.customer_id
AND a.Date_Of_Contact_Time <= e.contact_time
) AS correct_answer
FROM escalated_calls e
;

Generate Dates in oracle with gaps during Time Period

I Need To Generate Dates During Specific Time Period ( Using Oracle SQL Developer 12& 18).
/
I Found Bellow Solution:
SELECT TO_DATE('01/01/2020','DD/MM/YYYY') + (ROWNUM - 1) DATEGEN
FROM ALL_OBJECTS
WHERE TO_DATE('01/01/2020','DD/MM/YYYY') + (ROWNUM -1) <= TO_DATE('01/11/2020','DD/MM/YYYY')
I Get Daily Dates (As Bellow):
01-JAN-20 12.00.00 AM
02-JAN-20 12.00.00 AM
03-JAN-20 12.00.00 AM
04-JAN-20 12.00.00 AM
05-JAN-20 12.00.00 AM
06-JAN-20 12.00.00 AM
I Need To Get Dates With Gap 5 days or 3 days (Example):
01-JAN-20 12.00.00 AM
05-JAN-20 12.00.00 AM
10-JAN-20 12.00.00 AM
15-JAN-20 12.00.00 AM
Any Suggestions?!
Thanks in Advance.
You can use the hiearchy query with simple logic as follows:
SELECT DATE '2020-01-01' + ( LEVEL * 5 ) - 1
FROM DUAL CONNECT BY
LEVEL <= ( DATE '2020-11-01' - DATE '2020-01-01' ) / 5;
Please note that in your question first two dates have 4 days of difference and in my solution, the First date will be 05-01-2020
You could use a standad recursive query:
with cte (dt) as (
select date '2020-01-01' from dual
union all
select dt + interval '5' day from cte where dt < date '2020-01-15'
)
select * from cte order by dt
You can adjust the boundaries as needed, or move them to another cte:
with
params (startdt, enddt) as (
select date '2020-01-01', date '2020-01-15' from dual
),
cte (dt, enddt) as (
select startdt dt, enddt from params
union all
select dt + interval '5' day, enddt from cte where dt < enddt
)
select dt from cte order by dt
Note that the intervals between the dates are not constant in your sample data (there are 4 days between the first two dates, then 5 days.
You might want to adjust the where clause of the recursive member depending on your actual requirement. Here, the last day may be greater the January 15th. If you don't want that, then change the where clause to:
where dt + interval '5' day < enddt

How to avoid using limit query in sql redshift to get last x weeks of data

I have a query below which gives me data for the past 8 weeks and it works fine -
WITH dates
AS (
SELECT (
date_trunc('week', getdate() + INTERVAL '1 day')::DATE - 7 * (
row_number() OVER (
ORDER BY true
) - 1
) - INTERVAL '1 day'
)::DATE AS week_info
FROM data.process LIMIT 8
)
SELECT dates.week_info
,'W' || ceiling(date_part('week', dates.week_info + INTERVAL '1 day')) AS week_number
,COUNT(DISTINCT zeus.client_id) AS PROC
FROM data.active_values zeus
JOIN dates ON zeus.updated_timestamp <= dates.week_info
WHERE zeus.kites_version = (
SELECT MAX(kites_version)
FROM data.active_values f2
WHERE zeus.client_id = f2.client_id
AND zeus.type = f2.type
AND f2.updated_timestamp <= dates.week_info
)
AND zeus.type = 'hello-world'
AND zeus.STATUS = 'CURRENT'
GROUP BY dates.week_info
ORDER BY dates.week_info DESC LIMIT 8
But the problem I have is I am using limit 8 to get the above query working. I am trying to see if there is any way by which we can avoid using limit 8 and just use zeus.updated_timestamp value to get the past 8 weeks data in a similar output format as my current query is giving?
Output is coming like this from above query and I want it to be in this format only:
week_info week_number PROC
--------------------------------
2020-10-25 W44 100
2020-10-18 W43 101
2020-10-11 W42 109
2020-10-04 W41 134
2020-09-27 W40 982
2020-09-20 W39 187
2020-09-13 W38 765
2020-09-06 W37 234
Note:-
updated_timestamp column has full date in it like 2020-10-28 18:56:25:17
So 2 removals of LIMIT requested. The first, in the CTE, can be replaced by adding a WHERE clause in the outer select - "WHERE dates.week_info > 8 weeks ago" (I'll leave it to you to define 8 weeks ago. Also there are more efficient ways to make 8 dates than using a window function and scanning an unneeded table but that is your choice. Changing this will remove the LIMIT / WHERE need all together. Your CTE then looks something like:
select date_trunc('week', getdate() + INTERVAL '1 day')::DATE - (t.num * 7) - 1 as week_info
from (select 0 union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7) as t (num)
The second LIMIT is coming about because of the inequality in the JOIN clause which is causing a lot of row replication - I hope this is really what you need. There will only be 8 dates coming from the CTE AND having a GROUP BY on this date means that there will only be 8 rows of output. If there are only 8 possible rows there is no reason to have a LIMIT.
EDIT - merged code (untested):
WITH dates
AS (
select date_trunc('week', getdate() + INTERVAL '1 day')::DATE - (t.num * 7) - 1 as week_info
from (select 0 union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7) as t (num)
)
SELECT dates.week_info
,'W' || ceiling(date_part('week', dates.week_info + INTERVAL '1 day')) AS week_number
,COUNT(DISTINCT zeus.client_id) AS PROC
FROM data.active_values zeus
JOIN dates ON zeus.updated_timestamp <= dates.week_info
WHERE zeus.kites_version = (
SELECT MAX(kites_version)
FROM data.active_values f2
WHERE zeus.client_id = f2.client_id
AND zeus.type = f2.type
AND f2.updated_timestamp <= dates.week_info
)
AND zeus.type = 'hello-world'
AND zeus.STATUS = 'CURRENT'
GROUP BY dates.week_info
ORDER BY dates.week_info DESC
EDIT 2 - attempt to address correlated subquery issue:
If I understand correctly the where clause in question is just trying to ensure that only client_ids with values that match on kite_version are counted. A more direct (and less error prone) way to get this is to calculate the subgroup max directly. The below code attempts to do this but I don't have your data nor your business intent so this is an example of a better way to attack this type of requirement.
WITH dates
AS (
select date_trunc('week', getdate() + INTERVAL '1 day')::DATE - (t.num * 7) - 1 as week_info
from (select 0 union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7) as t (num)
),
active_values_plus AS (
SELECT client_id, updated_timestamp, type, status, kites_version, MAX(kites_version) OVER (PARTITION BY client_id, type) AS max_kites_version
FROM data.active_values
)
SELECT dates.week_info
,'W' || ceiling(date_part('week', dates.week_info + INTERVAL '1 day')) AS week_number
,COUNT(DISTINCT zeus.client_id) AS PROC
FROM active_values_plus zeus
JOIN dates ON zeus.updated_timestamp <= dates.week_info
WHERE zeus.kites_version = zeus.max_kites_version
AND zeus.type = 'hello-world'
AND zeus.STATUS = 'CURRENT'
GROUP BY dates.week_info
ORDER BY dates.week_info DESC

Query to find the missing dates of a month in Oracle

Here are the dates available in my Record_Date (Date) column in Attendance table in Oracle 10g.You can find the dates 04/06/2016 08/06/2016 16/06/2016 23/06/2016 29/06/2016 are missing in the sequence.
**Record_Date**
01/06/2016
02/06/2016
03/06/2016
05/06/2016
06/06/2016
07/06/2016
09/06/2016
10/06/2016
12/06/2016
13/06/2016
14/06/2016
15/06/2016
17/06/2016
18/06/2016
19/06/2016
20/06/2016
21/06/2016
22/06/2016
24/06/2016
25/06/2016
26/06/2016
27/06/2016
28/06/2016
30/06/2016
01/07/2016
I just need a query to find the missing dates in the specific month (and later also in the Year).
Kindly show me an approach
You can use this one:
WITH all_days AS
(SELECT DATE '2016-06-01' + LEVEL-1 AS the_day
FROM dual
CONNECT BY DATE '2016-06-01' + LEVEL-1 <= DATE '2016-06-30')
SELECT the_day
FROM all_days
WHERE the_day <>ALL (SELECT Record_Date FROM Attendance);
Or, if you like to have it more dynamically:
WITH all_days AS
(SELECT START_DATE + LEVEL AS the_day
FROM dual
CROSS JOIN
(SELECT
TRUNC(MIN(Record_Date), 'MM') -1 AS START_DATE,
TRUNC(LAST_DAY(MAX(Record_Date))) AS END_DATE
FROM Attendance)
CONNECT BY START_DATE + LEVEL <= END_DATE)
SELECT the_day
FROM all_days
WHERE the_day <>ALL (SELECT Record_Date FROM Attendance);
Note, <>ALL is the same as NOT IN - it's just my personal preference.
with
nums(num )
as
(select 0 from dual
union all
select num + 1 from nums
where num < (select max(col) from qtable)- (select min(col) from qtable)
),
date_btwn(dat)
as(select num + (select min(col) from qtable) from nums)
select dat from date_btwn
minus
select col from qtable;
Inline views nums will generate all numbers to add from start date. date_btwn contains all dates between start date and end date in table. We are excluding the dates in our table using minus.
You need to generate all dates and you have to find missing ones. Below with cte i have done it
Using CTE and not inQuery :
with calendar as (
select rownum - 1 as daynum,
to_date('1-jun-2016') + (rownum - 1) as monthdate
from dual )
select monthdate as monthdate
from calendar
where monthdate not in (
SELECT
Record_Date
FROM Attendance )
and monthdate >= to_date('1-jun-2016') /* CHANGE as per need */
and monthdate < to_date('1-jul-2016') /* CHANGE as per need */
Or Using CTE and left join Query :
with calendar as (
select rownum - 1 as daynum,
to_date('1-jun-2016') + (rownum - 1) as monthdate
from dual )
select monthdate as monthdate
from calendar C left join Attendance A on A.Record_Date=c.monthdate
where A.Record_Date is null
and monthdate >= to_date('1-jun-2016') /* CHANGE as per need */
and monthdate < to_date('1-jul-2016') /* CHANGE as per need */
as per datecolumn format change it to validformat into selection query first then use this query.

SQL: Return '0' for a row if it doesn't exist

I have a SQL query which displays count, date, and time.
This is what the output looks like:
And this is my SQL query:
select
count(*),
to_char(timestamp, 'MM/DD/YYYY'),
to_char(timestamp, 'HH24')
from
MY_TABLE
where
timestamp >= to_timestamp('03/01/2016','MM/DD/YYYY')
group by
to_char(timestamp, 'MM/DD/YYYY'), to_char(timestamp, 'HH24')
Now, in COUNT column, I want to display 0 if the count doesn't exist for that hour. So on 3/2/2016 at 8am, the count was 6. Then at 9am the count was 0 so that row didn't get displayed. I want to display that row. And at 10am & 11am, the counts are displayed then it just goes to next day.
So how do I display count of 0? I want to display 0 count for each day every hour doesn't matter if it's 0 or 6 or whatever. Thanks :)
Use a partition outer join:
SELECT m.day,
h.hr,
COALESCE( freq, 0 ) AS freq
FROM ( SELECT LEVEL - 1 AS hr
FROM DUAL
CONNECT BY LEVEL <= 24
) h
LEFT OUTER JOIN
( SELECT COUNT(*) AS freq,
TO_CHAR( "timestamp", 'mm/dd/yyyy' ) AS day,
EXTRACT( HOUR FROM "timestamp" ) AS hr
FROM MY_TABLE
WHERE "timestamp" >= TIMESTAMP '2016-03-01 00:00:00'
GROUP BY
TO_CHAR( "timestamp", 'mm/dd/yyyy' ),
EXTRACT( HOUR FROM "timestamp" )
) m
PARTITION BY ( m.day, m.hr )
ON ( m.hr = h.hr );
Use a cte to generate numbers for all the hours in a day. Then cross join the result with all the possible dates from the table. Then left join on the cte which has all date and hour combinations, to get a 0 count when a row is absent for a particular hour.
with nums(n) as (select 1 from dual
union all
select n+1 from nums where n < 24)
,dateshrscomb as (select n,dt
from nums
cross join (select distinct trunc(timestamp) dt from my_table
where timestamp >= to_timestamp('03/01/2016','MM/DD/YYYY')
) alldates
)
select count(trunc(m.timestamp)), d.dt, d.n
from dateshrscomb d
left join MY_TABLE m on to_char(m.timestamp, 'HH24') = d.n
and trunc(m.timestamp) = d.dt
and m.timestamp >= to_timestamp('03/01/2016','MM/DD/YYYY')
group by d.dt, d.n
with cteHours(h) as (select 0 from dual
union all
select h+1 from cteHours where h < 24)
, cteDates(d) AS (
SELECT
trunc(MIN(timestamp)) as d
FROM
My_Table
WHERE
timestamp >= to_timestamp('03/01/2016','MM/DD/YYYY')
UNION ALL
SELECT
d + 1 as d
FROM
cteDates
WHERE
d + 1 <= (SELECT trunc(MAX(timestamp)) FROM MY_TABLE)
)
, datesNumsCross (d,h) AS (
SELECT
d, h
FROM
cteDates
CROSS JOIN cteHours
)
select count(*), to_char(d.d, 'MM/DD/YYYY'), d.h
from datesNumsCross d
LEFT JOIN MY_TABLE m
ON d.d = trunc(m.timestamp)
AND d.h = to_char(m.timestamp, 'HH24')
group by d.d, d.h
#VPK is doing a good job at answering, I just happened to be writing this at the same time as his last edit to generate a date hour cross join. This solution differs from his in that it will get all dates between your desired max and min. Where as his will get only the dates within the table so if you have a day missing completely it would not be represented in his but would in this one. Plus I did a little clean up on the joins.
Here is one way to do that.
Using Oracle's hierarchical query feature and level psuedo column, generate the dates and hours.
Then do an outer join of above with your data.
Need to adjust the value of level depending upon your desired range (This example uses 120). Start date needs to be set as well. It is ( trunc(sysdate, 'hh24')-2/24 ) in this example.
select nvl(c1.cnt, 0), d1.date_part, d1.hour_part
from
(
select
to_char(s.dt - (c.lev)/24, 'mm/dd/yyyy') date_part,
to_char(s.dt - (c.lev)/24, 'hh24') hour_part
from
(select level lev from dual connect by level <= 120) c,
(select trunc(sysdate, 'hh24')-2/24 dt from dual) s
where (s.dt - (c.lev)/24) < trunc(sysdate, 'hh24')-2/24
) d1
full outer join
(
select
count(*) cnt,
to_char(timestamp, 'MM/DD/YYYY') date_part,
to_char(timestamp, 'HH24') hour_part
from
MY_TABLE
where
timestamp >= to_timestamp('03/01/2016','MM/DD/YYYY')
group by
to_char(timestamp, 'MM/DD/YYYY'), to_char(timestamp, 'HH24')
) c1
on d1.date_part = c1.date_part
and d1.hour_part = c1.hour_part