SQL difference in counter between two dates - sql

I have a table like the below:
ID, MachineID Customer TimeStamp Counter type
1 A ABC 2017-10-25 3:08PM 1952 1
2 A ABC 2017-10-25 3:00PM 1940 1
3 A ABC 2017-10-25 12:05PM 1920 1
4 A ABC 2017-10-25 9:00AM 1900 1
5 B BCD 2017-10-25 3:11PM 1452 1
6 B BCD 2017-10-25 3:10PM 1440 1
7 B BCD 2017-10-25 12:15PM 1420 1
8 B BCD 2017-10-25 9:30AM 1400 1
9 A ABC 2017-10-23 3:08PM 1900 1
10 A ABC 2017-10-23 3:00PM 1840 1
11 A ABC 2017-10-23 12:05PM 1820 1
12 A ABC 2017-10-23 9:00AM 1800 1
13 B BCD 2017-10-23 3:11PM 1399 1
14 B BCD 2017-10-23 3:10PM 1340 1
15 B BCD 2017-10-23 12:15PM 1320 1
16 B BCD 2017-10-23 9:30AM 1300 1
The counter value increases whenever there is a click. I am trying to calculate number of clicks for each day by taking maximum counter value at the end of day and subtract the previous day maximum counter value and so on.
How do I do this in SQL server. Have to repeat this for each customer and Machine

Try this. I am using LAG function in order to achieve this. You can use where clause to filter out specific date you want :
Create table #counter(ID int, timeStamp datetime, Counter int, type int)
insert into #counter values
(1, '20171024 3:08PM' ,1952, 1),
(1, '20171025 3:00PM' ,1964, 1)
Select iq.*, (iq."counter" - iq.yesterday_counter) as today_count
from
(select id,
cast("timestamp" as date) as today_date,
"counter",
LAG("counter") over (order by cast("timestamp" as date)) yesterday_counter
from #counter
) iq
output:
id today_date counter yesterday_counter today_count
----------- ---------- ----------- ----------------- -----------
1 2017-10-24 1952 NULL NULL
1 2017-10-25 1964 1952 12

A SQL query to get the max counter for each day is:
SELECT CAST(timeStamp as date) AS [dateval]
,MAX(Counter) AS [maxCounter]
FROM YOURDATASET
GROUP BY CAST(timeStamp as date)
This is converting the datetime to date- cutting out the time, then taking the max(Counter).
One method to get the difference is to save the result in a temp datastructure, then query it to get the difference.
The question is whether your previous date is exactly the previous day, or if you're skipping days between counts, or taking the weekend off, etc. In that case you have to select the greatest previous date to the date being examined.
ex.
DECLARE #temp TABLE (dateval date, maxCounter int)
INSERT INTO #temp(dateval, maxCounter)
SELECT CAST(timeStamp as date) AS [dateval]
,MAX(Counter)
FROM YOURDATASET
GROUP BY CAST(timeStamp as date)
SELECT T.dateval
,T.dateval
-
(SELECT maxCounter
FROM #temp T2
WHERE T2.dateVal = (SELECT MAX(dateVal)
FROM #temp T3
WHERE T3.dateVal < T1.dateVal
)
) AS [Difference]
FROM #temp T
ORDER BY T.dateval

Related

How to generate series using start and end date and quarters on postgres

I have a table like shown below where I want to use the start and end date to evenly distribute the value for each row to the 3 months in each quarter to all of the quarters in between start and end date (last two columns).
I am familiar with generate series and intervals in Postgres but I am having hard time to get what I want.
My table has and ID column that groups rows together, a quarter column that indicates which quarter the row references for the ID, a value column that is the value for the whole quarter (and every quarter in the date range), and start_date and end_date columns indicating the date range. Here is a sample:
ID quarter value start_date end_date
1 2 152 2019-11-07 2050-12-30
1 1 785 2019-11-07 2050-12-30
2 2 152 2019-03-05 2050-12-30
2 1 785 2019-03-05 2050-12-30
3 4 41 2018-06-12 2050-12-30
3 3 50 2018-06-12 2050-12-30
3 2 88 2018-06-12 2050-12-30
3 1 29 2018-06-12 2050-12-30
4 2 1607 2018-12-17 2050-12-30
4 1 4803 2018-12-17 2050-12-30
Here is my desired output (for ID 1):
ID quarter value start_date end_date
1 2 152/3 2020-04-01 2020-07-01
1 1 785/3 2020-01-01 2020-04-01
1 2 152/3 2021-04-01 2021-07-01
1 1 785/3 2021-01-01 2021-04-01
start_date in the output will be the next quarter on first table. I need the series to be generated from the start_date to the end_date of the first table.
You can do this by using the GENERATE_SERIES function and passing in the start and end date for each unique (by ID) row and setting the interval to 3 months. Then join the result back with your original table on both ID and quarter.
Here's an example (note original_data is what I've called your first table):
WITH
quarters_table AS (
SELECT
t.ID,
(EXTRACT('month' FROM t.quarter_date) - 1)::INT / 3 + 1 AS quarter,
t.quarter_date::DATE AS start_date,
COALESCE(
LEAD(t.quarter_date) OVER (),
DATE_TRUNC('quarter', t.original_end_date) + INTERVAL '3 months'
)::DATE AS end_date
FROM (
SELECT
original_record.ID,
original_record.end_date AS original_end_date,
GENERATE_SERIES(
DATE_TRUNC('quarter', original_record.start_date),
DATE_TRUNC('quarter', original_record.end_date),
INTERVAL '3 months'
) AS quarter_date
FROM (
SELECT DISTINCT ON (original_data.ID)
original_data.ID,
original_data.start_date,
original_data.end_date
FROM
original_data
ORDER BY
original_data.ID
) AS original_record
) AS t
)
SELECT
quarters_table.ID,
quarters_table.quarter,
original_data.value::DOUBLE PRECISION / 3 AS value,
quarters_table.start_date,
quarters_table.end_date
FROM
quarters_table
INNER JOIN
original_data
ON
quarters_table.ID = original_data.ID
AND quarters_table.quarter = original_data.quarter;
Sample output:
id | quarter | value | start_date | end_date
----+---------+------------------+------------+------------
1 | 1 | 261.666666666667 | 2020-01-01 | 2020-04-01
1 | 2 | 50.6666666666667 | 2020-04-01 | 2020-07-01
1 | 1 | 261.666666666667 | 2021-01-01 | 2021-04-01
1 | 2 | 50.6666666666667 | 2021-04-01 | 2021-07-01
For completeness, here's the original_data table I've used in testing:
WITH
original_data AS (
SELECT
1 AS ID,
2 AS quarter,
152 AS value,
'2019-11-07'::DATE AS start_date,
'2050-12-30'::DATE AS end_date
UNION ALL
SELECT
1 AS ID,
1 AS quarter,
785 AS value,
'2019-11-07'::DATE AS start_date,
'2050-12-30'::DATE AS end_date
UNION ALL
SELECT
2 AS ID,
2 AS quarter,
152 AS value,
'2019-03-05'::DATE AS start_date,
'2050-12-30'::DATE AS end_date
-- ...
)
This is one way to go about it. Showing an example based on the output you've outlined. You can then add more conditions to the CASE/WHEN for additional quarters.
SELECT
ID,
Quarter,
Value/3 AS "Value",
CASE
WHEN Quarter = 1 THEN '2020-01-01'
WHEN Quarter = 2 THEN '2020-04-01'
END AS "Start_Date",
CASE
WHEN Quarter = 1 THEN '2020-04-01'
WHEN Quarter = 2 THEN '2020-07-01'
END AS "End_Date"
FROM
Table

How duplicate a rows in SQL base on difference between date columns and divided aggregated column per duplicate row?

I have a table with some records about fuel consumption. The important columns in the table are: CONSUME_DATE_FROM and CONSUM_DATE_TO.
I want to calculate average fuel consumption per cars on a monthly basis but some rows are not in the same month. For example some have a three month difference between them and the total of gas per litre is aggregated in a single row.
Now I should find records that have difference more than a month between CONSUME_DATE_FROM and CONSUM_DATE_TO, and duplicate them in current or second table per count of month and divide the total gas per litre between related rows.
I've this table with the following data:
ID VehicleId CONSUME_DATE_FROM CONSUM_DATE_TO GAS_PER_LITER
1 100 2018-10-25 2018-12-01 600
2 101 2018-07-19 2018-07-24 100
3 102 2018-12-31 2019-01-01 400
4 103 2018-03-29 2018-05-29 200
5 104 2018-02-05 2018-02-09 50
The expected output table should be as below
ID VehicleId CONSUME_DATE_FROM CONSUM_DATE_TO GAS_PER_LITER
1 100 2018-10-25 2018-12-01 200
1 100 2018-10-25 2018-12-01 200
1 100 2018-10-25 2018-12-01 200
2 101 2018-07-19 2018-07-24 100
3 102 2018-12-31 2019-01-01 200
3 102 2018-12-31 2019-01-01 200
4 103 2018-03-29 2018-05-29 66.66
4 103 2018-03-29 2018-05-29 66.66
4 103 2018-03-29 2018-05-29 66.66
5 104 2018-02-05 2018-02-09 50
Or as below
ID VehicleId CONSUME_DATE_FROM CONSUM_DATE_TO GAS_PER_LITER DATE_RELOAD_GAS
1 100 2018-10-25 2018-12-01 200 2018-10-01
1 100 2018-10-25 2018-12-01 200 2018-11-01
1 100 2018-10-25 2018-12-01 200 2018-12-01
2 101 2018-07-19 2018-07-24 100 2018-07-01
3 102 2018-12-31 2019-01-01 200 2018-12-01
3 102 2018-12-31 2019-01-01 200 2019-01-01
4 103 2018-03-29 2018-05-29 66.66 2018-03-01
4 103 2018-03-29 2018-05-29 66.66 2018-04-01
4 103 2018-03-29 2018-05-29 66.66 2018-05-01
5 104 2018-02-05 2018-02-09 50 2018-02-01
Can someone please help me out with this query?
I'm using oracle database
Your business rule treats the difference between CONSUME_DATE_FROM and CONSUM_DATE_TO as absolute months. So you expect the difference between 2018-10-25 and 2018-12-01 to be three months whereas the difference in days actually equates to about 1.1 months. So we can't use simple date arithmetic to get your desired output, we need to do some additional massaging of the dates.
The query below implements your desired logic by deriving the first day of the month for CONSUME_DATE_FROM and the last day of the month for CONSUME_DATE_TO, then using ceil() to round the difference up to the nearest whole number of months.
This is calculated in a subquery which is used in the main query with the old connect by level trick to multiply a record by level number of times:
with cte as (
select f.*
, ceil(months_between(last_day(CONSUM_DATE_TO)
, trunc(CONSUME_DATE_FROM,'mm'))) as diff
from fuel_consumption f
)
select cte.id
, cte.VehicleId
, cte.CONSUME_DATE_FROM
, cte.CONSUM_DATE_TO
, cte.GAS_PER_LITER/cte.diff as GAS_PER_LITER
, add_months(trunc(cte.CONSUME_DATE_FROM, 'mm'), level-1) as DATE_RELOAD_GAS
from cte
connect by level <= cte.diff
and prior cte.id = cte.id
and prior sys_guid() is not null
;
"what about if add a additional column "DATE_RELOAD_GAS" that display difference date for similar rows"
From your posted sample it seems like DATE_RELOAD_GAS is the first day of the month for each month bounded by CONSUME_DATE_FROM and CONSUM_DATE_TO. I have amended my solution to implement this rule.
By using connect by level structure with considering to_char(c.CONSUME_DATE_FROM + level - 1,'yyyymm') as month I was able to resolve as below :
select ID, VehicleId, myMonth, CONSUME_DATE_FROM, CONSUM_DATE_TO,
trunc(GAS_PER_LITER/max(rn) over (partition by ID order by ID),2) as GAS_PER_LITER,
'01.'||substr(myMonth,5,2)||'.'||substr(myMonth,1,4) as DATE_RELOAD_GAS
from
(
with consumption( ID, VehicleId, CONSUME_DATE_FROM, CONSUM_DATE_TO, GAS_PER_LITER ) as
(
select 1,100,date'2018-10-25',date'2018-12-01',600 from dual union all
select 2,101,date'2018-07-19',date'2018-07-24',100 from dual union all
select 3,102,date'2018-12-31',date'2019-01-01',400 from dual union all
select 4,103,date'2018-03-29',date'2018-05-29',200 from dual union all
select 5,104,date'2018-02-05',date'2018-02-09', 50 from dual
)
select ID, to_char(c.CONSUME_DATE_FROM + level - 1,'yyyymm') myMonth,
VehicleId, c.CONSUME_DATE_FROM, c.CONSUM_DATE_TO, GAS_PER_LITER,
row_number() over (partition by ID order by ID) as rn
from dual join consumption c
on c.ID >= 2
group by ID, to_char(c.CONSUME_DATE_FROM + level - 1,'yyyymm'), VehicleId,
c.CONSUME_DATE_FROM, c.CONSUM_DATE_TO, c.GAS_PER_LITER
connect by level <= c.CONSUM_DATE_TO - c.CONSUME_DATE_FROM + 1
union all
select ID, to_char(c.CONSUME_DATE_FROM + level - 1,'yyyymm') myMonth,
VehicleId, c.CONSUME_DATE_FROM, c.CONSUM_DATE_TO, GAS_PER_LITER,
row_number() over (partition by ID order by ID) as rn
from dual join consumption c
on c.ID = 1
group by ID, to_char(c.CONSUME_DATE_FROM + level - 1,'yyyymm'), VehicleId,
c.CONSUME_DATE_FROM, c.CONSUM_DATE_TO, c.GAS_PER_LITER
connect by level <= c.CONSUM_DATE_TO - c.CONSUME_DATE_FROM + 1
) q
group by ID, VehicleId, myMonth, CONSUME_DATE_FROM, CONSUM_DATE_TO, GAS_PER_LITER, rn
order by ID, myMonth;
I met an interesting issue that if I consider the join condition in the subquery as c.ID >= 1 query hangs on for huge period of time, so splitted into two parts by union all
as c.ID >= 2 and c.ID = 1
Rextester Demo

SQL : Sum by criteria

I'm working with Oracle and cannot achieve the query I need for the moment.
Suppose I have the following table :
- ID Date Type Value
- 1 01/12/2016 prod 1
- 2 01/01/2017 test 10
- 3 01/06/2017 test 20
- 4 01/12/2017 prod 30
- 5 15/12/2017 test 40
- 6 01/01/2018 test 50
- 7 01/06/2018 test 60
- 8 01/12/2018 prod 70
I need to sum the VALUES between the "prod" TYPES + the last "prod" VALUE.
The results should be :
- 1 01/01/2016 - 1
- 2 01/01/2017 - 60
- 3 01/06/2017 - 60
- 4 01/12/2017 - 60
- 5 15/12/2017 - 220
- 6 01/01/2018 - 220
- 7 01/06/2018 - 220
- 8 01/12/2018 - 220
I first had to sum VALUES by YEAR without taking TYPES into account.
The need changed and I don't see how to start to identify, for each line, which is the previous "prod" DATE and sum each VALUE including the last "prod" TYPE.
Thanks
You can define the groups using a cumulative sum on type = 'PROD' -- in reverse, then use a window function for the final summation:
select t.*,
sum(value) over (partition by grp) as total
from (select t.*,
sum(case when type = 'PROD' then 1 else 0 end) over (order by id desc) as grp
from t
) t
order by id;
To see the grouping logic, look at:
ID Date Type Value Grp
1 01/12/2016 prod 1 3
2 01/01/2017 test 10 2
3 01/06/2017 test 20 2
4 01/12/2017 prod 30 2
5 15/12/2017 test 40 1
6 01/01/2018 test 50 1
7 01/06/2018 test 60 1
8 01/12/2018 prod 70 1
This identifies the groups that need to be summed. The DESC is because "prod" ends a group. If "prod" started a group (i.e. was included with the sum on the next row), then ASC would be used.
Rextester Demo
Gordon Linoff's answer is great.
This below is just for a bit of a different flavor(12c+)
Setup:
ALTER SESSION SET NLS_DATE_FORMAT = 'DD/MM/YYYY';
CREATE TABLE TEST_TABLE(
THE_ID INTEGER,
THE_DATE DATE,
THE_TYPE CHAR(4),
THE_VALUE INTEGER);
INSERT INTO TEST_TABLE VALUES (1,TO_DATE('01/12/2016'),'prod',1);
INSERT INTO TEST_TABLE VALUES (2,TO_DATE('01/01/2017'),'test',10);
INSERT INTO TEST_TABLE VALUES (3,TO_DATE('01/06/2017'),'test',20);
INSERT INTO TEST_TABLE VALUES (4,TO_DATE('01/12/2017'),'prod',30);
INSERT INTO TEST_TABLE VALUES (5,TO_DATE('15/12/2017'),'test',40);
INSERT INTO TEST_TABLE VALUES (6,TO_DATE('01/01/2018'),'test',50);
INSERT INTO TEST_TABLE VALUES (7,TO_DATE('01/06/2018'),'test',70);
INSERT INTO TEST_TABLE VALUES (8,TO_DATE('01/12/2018'),'prod',60);
COMMIT;
Query:
SELECT
THE_ID, THE_DATE, MAX(RUNNING_GROUP_SUM) OVER (PARTITION BY THE_MATCH_NUMBER) AS GROUP_SUM
FROM TEST_TABLE
MATCH_RECOGNIZE (
ORDER BY THE_ID
MEASURES
MATCH_NUMBER() AS THE_MATCH_NUMBER,
RUNNING SUM(THE_VALUE) AS RUNNING_GROUP_SUM
ALL ROWS PER MATCH
AFTER MATCH SKIP PAST LAST ROW
PATTERN (TEST_TARGET{0,} PROD_TARGET)
DEFINE TEST_TARGET AS THE_TYPE = 'test',
PROD_TARGET AS THE_TYPE = 'prod')
ORDER BY THE_ID ASC;
Result:
THE_ID THE_DATE GROUP_SUM
---------- ---------- ----------
1 01/12/2016 1
2 01/01/2017 60
3 01/06/2017 60
4 01/12/2017 60
5 15/12/2017 220
6 01/01/2018 220
7 01/06/2018 220
8 01/12/2018 220

Use Calendar table to generate historical view of the data

I have a created_date (timestamp) on 1 of my tables, that also has the duration column of a project, and I need to join with another table that only has first_day_of_month column that has the first day of each month, and other relevant information.
Table 1
id project_id created_date duration
1 12345 01/01/2015 10
2 12345 20/10/2015 11
3 12345 10/04/2016 13
4 12345 10/08/2016 15
Table 2
project_id month_start_date
12345 01/01/2015
12345 01/02/2015
12345 01/03/2015
12345 01/04/2015
...
12345 01/08/2016
Expected result
project_id month_start_date duration
12345 01/01/2015 10
12345 01/02/2015 10
...
12345 01/10/2015 11
12345 01/11/2015 11
...
12345 01/04/2016 13
12345 01/05/2016 13
12345 01/06/2016 13
...
12345 01/08/2016 15
I want to be able to present the data listed in my second table historically. So, basically I want the query to return the same duration related to the month_start_date, so that values will repeat until another dateadd(month,datediff(month,0,created_date),0) = first_day_of_month is met... and so forth.
This is my query:
select table2.project_name,
table2.month_start_date,
table1.duration,
table1.created_date
from table1 left outer join table2
on table1.project_id=table2.project_id
where dateadd(month,datediff(month,0,table1.created_date),0)<=table2.month_start_date
group by table2.project_name,table2.month_start_date,table1.duration,table1.created_date
order by table2.month_start_date asc
but I get repeated records on this:
Result I'm getting
project_id month_start_date duration
12345 01/01/2015 10
12345 01/02/2015 10
...
12345 01/10/2015 10
12345 01/10/2015 11
...
12345 01/04/2016 10
12345 01/04/2016 11
12345 01/04/2016 13
...
12345 01/08/2016 10
12345 01/08/2016 11
12345 01/08/2016 13
12345 01/08/2016 15
Can anyone help?
Thank you!
I'd use CROSS/OUTER APPLY operator.
Here is one possible variant. For each row in your calendar table Table2 (for each month) the inner correlated subquery inside the CROSS APPLY finds one row from Table1. It will be the row with the same project_id and the first row with created_date before the month_start_date plus 1 month.
SELECT
Table2.project_id
,Table2.month_start_date
,Durations.duration
FROM
Table2
CROSS APPLY
(
SELECT TOP(1) Table1.duration
FROM Table1
WHERE
Table1.project_id = Table2.project_id
AND Table1.created_date < DATEADD(month, 1, Table2.month_start_date)
ORDER BY Table1.created_date DESC
) AS Durations
;
Make sure that Table1 has index on (project_id, created_date) include (duration). Otherwise, performance would be poor.

Update the list of dates to have the same day

I have this in my table
TempTable
Id Date
1 1-15-2010
2 2-14-2010
3 3-14-2010
4 4-15-2010
i would like to change every record so that they have all same day, that is the 15th
like this
TempTable
Id Date
1 1-15-2010
2 2-15-2010 <--change to 15
3 3-15-2010 <--change to 15
4 4-15-2010
what if i like on the 30th?
the records should be
TempTable
Id Date
1 1-30-2010
2 2-28-2010 <--change to 28 because feb has 28 days only
3 3-30-2010 <--change to 30
4 4-30-2010
thanks
You can play some fun tricks with DATEADD/DATEDIFF:
create table T (
ID int not null,
DT date not null
)
insert into T (ID,DT)
select 1,'20100115' union all
select 2,'20100214' union all
select 3,'20100314' union all
select 4,'20100415'
SELECT ID,DATEADD(month,DATEDIFF(month,'20100101',DT),'20100115')
from T
SELECT ID,DATEADD(month,DATEDIFF(month,'20100101',DT),'20100130')
from T
Results:
ID
----------- -----------------------
1 2010-01-15 00:00:00.000
2 2010-02-15 00:00:00.000
3 2010-03-15 00:00:00.000
4 2010-04-15 00:00:00.000
ID
----------- -----------------------
1 2010-01-30 00:00:00.000
2 2010-02-28 00:00:00.000
3 2010-03-30 00:00:00.000
4 2010-04-30 00:00:00.000
Basically, in the DATEADD/DATEDIFF, you specify the same component to both (i.e. month). Then, the second date constant (i.e. '20100130') specifies the "offset" you wish to apply from the first date (i.e. '20100101'), which will "overwrite" the portion of the date your not keeping. My usual example is when wishing to remove the time portion from a datetime value:
SELECT DATEADD(day,DATEDIFF(day,'20010101',<date column>),'20100101')
You can also try something like
UPDATE TempTable
SET [Date] = DATEADD(dd,15-day([Date]), DATEDIFF(dd,0,[Date]))
We have a function that calculates the first day of a month, so I just addepted it to calculate the 15 instead...