Related
I would like to calculate targets for opened rates and clicked rates based on actuals of the last month and the same month last year.
My table is aggregated at daily level and I have grouped it by month and year to get the monthly averages. I have then created a self-join to join my current dates on the results of the previous months. This works fine for all months except for January because SQL can't know that it's supposed to join 1 on 12. Is there a way to specify this in my join clause?
Essentially, the results for January 2021 shouldn't be null because I have December 2020 data.
This is my data and my query:
CREATE TABLE exasol_last_year_avg(
date_col date,
country text,
brand text,
category text,
delivered integer,
opened integer,
clicked integer
)
INSERT INTO exasol_last_year_avg
(date_col,country,brand,category,delivered,opened,clicked) VALUES
(2021-01-01,'AT','brand1','cat1',100,60,23),
(2021-01-01,'AT','brand1','cat2',200,50,45),
(2021-01-01,'AT','brand2','cat1',300,49,35),
(2021-01-01,'AT','brand2','cat2',400,79,57),
(2021-02-02,'AT','brand1','cat1',130,78,30),
(2021-02-02,'AT','brand1','cat2',260,65,59),
(2021-02-02,'AT','brand2','cat1',390,64,46),
(2021-02-02,'AT','brand2','cat2',520,103,74),
(2020-12-02,'AT','brand1','cat1',130,78,30),
(2020-12-02,'AT','brand1','cat2',260,65,59),
(2020-12-02,'AT','brand2','cat1',390,64,46),
(2020-12-02,'AT','brand2','cat2',520,103,74),
(2020-02-02,'AT','brand1','cat2',236,59,53),
(2020-02-02,'AT','brand2','cat1',355,58,41),
(2020-02-02,'AT','brand2','cat2',473,93,67),
(2020-02-02,'AT','brand1','cat1',118,71,27)
This is written in PostgresSQL because I think it's more accessible to most people, but my production database is Exasol!
select *
from
(Select month_col,
year_col,
t_campaign_cmcategory,
t_country,
t_brand,
(t2_clicktoopenrate + t3_clicktoopenrate)/2 as target_clicktoopenrate,
(t2_openrate + t3_openrate)/2 as target_openrate
from (
with CTE as (
select extract(month from date_col) as month_col,
extract(year from date_col) as year_col,
category as t_campaign_cmcategory,
country as t_country,
brand as t_brand,
round(sum(opened)/nullif(sum(delivered),0),3) as OpenRate,
round(sum(clicked)/nullif(sum(opened),0),3) as ClickToOpenRate
from public.exasol_last_year_avg
group by 1, 2, 3, 4, 5)
select t1.month_col,
t1.year_col,
t2.month_col as t2_month_col,
t2.year_col as t2_year_col,
t3.month_col as t3_month_col,
t3.year_col as t3_year_col,
t1.t_campaign_cmcategory,
t1.t_country,
t1.t_brand,
t1.OpenRate,
t1.ClickToOpenRate,
t2.OpenRate as t2_OpenRate,
t2.ClickToOpenRate as t2_ClickToOpenRate,
t3.OpenRate as t3_OpenRate,
t3.ClickToOpenRate as t3_ClickToOpenRate
from CTE t1
left join CTE t2
on t1.month_col = t2.month_col + 1
and t1.year_col = t2.year_col
and t1.t_campaign_cmcategory = t2.t_campaign_cmcategory
and t1.t_country = t2.t_country
and t1.t_brand = t2.t_brand
left join CTE t3
on t1.month_col = t3.month_col
and t1.year_col = t3.year_col + 1
and t1.t_campaign_cmcategory = t3.t_campaign_cmcategory
and t1.t_country = t3.t_country
and t1.t_brand = t3.t_brand) as target_base) as final_tbl
Start with an aggregation query:
select date_trunc('month', date_col), country, brand,
sum(opened) * 1.0 / nullif(sum(delivered), 0) as OpenRate,
sum(clicked) * 1.0 / nullif(sum(opened), 0) as ClickToOpenRate
from exasol_last_year_avg
group by 1, 2, 3;
Then, use window functions. Assuming you have a value for every month (with no gaps). you can just use lag(). I'm not sure what your final calculation is, but this brings in the data:
with mcb as (
select date_trunc('month', date_col) as yyyymm, country, brand,
sum(opened) * 1.0 / nullif(sum(delivered), 0) as OpenRate,
sum(clicked) * 1.0 / nullif(sum(opened), 0) as ClickToOpenRate
from exasol_last_year_avg
group by 1, 2, 3
)
select mcb.*,
lag(openrate, 1) over (partition by country, brand order by yyyymm) as prev_month_openrate,
lag(ClickToOpenRate, 1) over (partition by country, brand order by yyyymm) as prev_month_ClickToOpenRate,
lag(openrate, 12) over (partition by country, brand order by yyyymm) as prev_year_openrate,
lag(ClickToOpenRate, 12) over (partition by country, brand order by yyyymm) as prev_year_ClickToOpenRate
from mcb;
This works with a different join condition:
select *
from
(Select month_col,
year_col,
t_campaign_cmcategory,
t_country,
t_brand,
(t2_clicktoopenrate + t3_clicktoopenrate)/2 as target_clicktoopenrate,
(t2_openrate + t3_openrate)/2 as target_openrate
from (
with CTE as (
select extract(month from date_col) as month_col,
extract(year from date_col) as year_col,
category as t_campaign_cmcategory,
country as t_country,
brand as t_brand,
round(sum(opened)/nullif(sum(delivered),0),3) as OpenRate,
round(sum(clicked)/nullif(sum(opened),0),3) as ClickToOpenRate
from public.exasol_last_year_avg
group by 1, 2, 3, 4, 5)
select t1.month_col,
t1.year_col,
t2.month_col as t2_month_col,
t2.year_col as t2_year_col,
t3.month_col as t3_month_col,
t3.year_col as t3_year_col,
t1.t_campaign_cmcategory,
t1.t_country,
t1.t_brand,
t1.OpenRate,
t1.ClickToOpenRate,
t2.OpenRate as t2_OpenRate,
t2.ClickToOpenRate as t2_ClickToOpenRate,
t3.OpenRate as t3_OpenRate,
t3.ClickToOpenRate as t3_ClickToOpenRate
from CTE t1
left join CTE t2
-- adjusted join condition
on ((t1.month_col = (CASE WHEN t1.month_col = 1 then t2.month_col - 11 END) and t1.year_col = t2.year_col + 1)
or (t1.month_col = (CASE WHEN t1.month_col != 1 then t2.month_col + 1 END) and t1.year_col = t2.year_col))
and t1.t_campaign_cmcategory = t2.t_campaign_cmcategory
and t1.t_country = t2.t_country
and t1.t_brand = t2.t_brand
left join CTE t3
on t1.month_col = t3.month_col
and t1.year_col = t3.year_col + 1
and t1.t_campaign_cmcategory = t3.t_campaign_cmcategory
and t1.t_country = t3.t_country
and t1.t_brand = t3.t_brand) as target_base) as final_tbl
I use the following query in ordre to filling missing months
Declare #Sample Table(year int, month int,product as nvarchar(50), qty_ytd int);
Insert #Sample(year, month, qty_ytd) Values
(2017, 01,'book', 20),
(2017, 02, 'pc', 30),
(2018, 01, 'book', 50);
;With Months As
(Select 1 As month
Union All
Select month + 1 From Months Where month < 12)
, YearsAndMonths As
(Select distinct year,m.month from #Sample cross join Months m)
select ym.*, coalesce(s.qty_ytd, s2.qty_ytd) qty_ytd, coalesce(s.qty_ytd, 0) QTY from YearsAndMonths ym
left join #sample s on ym.year = s.year and ym.month = s.month
left join (select qty_ytd, year,
row_number() over (partition by year order by month desc) rn
from #Sample) s2 on ym.year = s2.year and rn = 1
How could I add 'product ' as well ?
Firstly, I would recommend creating a calendar table since this pops up as a use case every once in a while. A quick example can be found here
Now, once you have the calendar table (let's call it static.calendar) ready, the code is fairly simple as follows:
with Products
as
(
SELECT distinct product
FROM #Sample
),
TimeRange
as
(
SELECT DISTINCT year,
month
FROM static.calendar
)
ProductTimeRange
as
(
SELECT p.products,
tr.year,
tr.month
FROM Products as p
CROSS JOIN TimeRange as tr
)
SELECT ptr.products,
ptr.year,
ptr.month,
s.qty_ytd
FROM ProductTimeRange as ptr
LEFT JOIN #sample as s
ON ptr.products = s.products
AND ptr.year = s.year
AND ptr.month = s.month
ORDER BY ptr.products,
ptr.year,
ptr.month
Use a cross join to generate the rows that you want -- all the years, months, and products.
Then use left join to bring in the data you want:
With Months As (
Select 1 As month
Union All
Select month + 1
From Months
Where month < 12
)
select y.year, m.month, s.product, coalesce(qty_ytd, 0) as qty_ytd
from (select distinct year from #sample) y cross join
months m cross join
(select distinct product from #sample) p left join
#sample s
on s.year = y.year and s.month = m.month and s.product = p.product;
Here is a db<>fiddle.
I have the following table:
PERSON
ID
Name
date_created
date_left
What I want is a list of all months and the amount of users joined and the amount of users that left.
I already have the following query: it returns the amount of new users that joined in the month that I pass:
select MONTH(date_created) 'Month', YEAR(date_created) 'Year', count(*) as 'New Users'
from person p
where YEAR(date_created) = 2018 and MONTH( p.date_created) = 5
group by MONTH(date_created), YEAR(date_created)
It returns what I want:
How would I edit this to include a year report and add the column 'Users left' next to the 'new users' one?
My result would be:
MONTH YEAR NEW USERS USERS LEFT
1 2019 10 5
I would "unpivot" the data using cross apply:
select v.[year], v.[month], sum(v.iscreated) as num_created,
sum(v.isleft) as num_left
from person p cross apply
(values (year(p.date_created), month(p.date_created), 1, 0),
(year(p.date_left), month(p.date_left), 0, 1)
) v([year], [month], iscreated, isleft)
group by v.[year], v.[month]
order by v.[year], v.[month];
The straight-forward approach is probably to full outer join all entries and all leaves. SQL Server makes this a bit awkward by not featuring USING, so we must use ON and COALESCE on month and year instead.
select
coalesce(pin.year, pout.year) as year,
coalesce(pin.month, pout.month) as month,
coalesce(pin.cnt, 0) as count_in,
coalesce(pout.cnt, 0) as count_out
from
(
select year(date_created) as year, month(date_created) as month, count(*) as cnt
from person
group by year(date_created), month(date_created)
) pin
full outer join
(
select year(date_left) as year, month(date_left) as month, count(*) as cnt
from person
group by month(date_left), year(date_left)
) pout on pout.year = pin.year and pout.month = pin.month
order by year, month;
Maybe you could do it with a SubSelect? Tried it right now with ORACLE Syntax, I'm not sure if it works in SQL-Server.
SELECT * FROM
(
select MONTH(date_created) 'Month_C', YEAR(date_created) 'Year_C', count(*) as 'New Users'
from person p
where YEAR(date_created) = 2018 and MONTH( p.date_created) = 5
group by MONTH(date_created), YEAR(date_created)
)created_user,
(
select MONTH(date_left) 'Month_L', YEAR(date_left) 'Year_L', count(*) as 'New Users'
from person p
where YEAR(date_left) = 2018 and MONTH( p.date_left) = 5
group by MONTH(date_left), YEAR(date_left)
) left_user
where created_user.Year_C = left_user.Year_L
and created_user.Month_C = left_user.Month_L
I have the following query:
SELECT DISTINCT
YEAR(DateRegistered) as Years,
Months.[MonthName],
COUNT(UserID)as totalReg
FROM
Months WITH(NOLOCK)
LEFT OUTER JOIN
UserProfile WITH(NOLOCK)
ON
Months.MonthID = MONTH(DateRegistered)
AND
DateRegistered > DATEADD(MONTH, -12,GETDATE())
GROUP BY YEAR(DateRegistered), Months.[MonthName]
ORDER BY Months.[MonthName]
As you can tell this will always bring back 12 months worth of data. As such it is working, although there is a bug with this method.
It creates Null values in months where there is no data, now the record should exist(whole point of the query) but Year field is bringing Nulls which is something I dont want.
Now I understand the problem is because there is no data, how is it supposed to know what year?
So my question is - is there any way to sort this out and replace the nulls? I suspect I will have to completely change my methodology.
**YEAR** **MONTH** **TOTAL**
2013 April 1
2013 August 1
NULL December 0
2013 February 8
2013 January 1
2013 July 1
NULL June 0
2013 March 4
NULL May 0
NULL November 0
NULL October 0
2012 September 3
If you want 12 months of data, then construct a list of numbers from 1 to 12 and use these as offsets with getdate():
with nums as (
select 12 as level union all
select level - 1
from nums
where level > 1
)
select YEAR(thedate) as Years,
Months.[MonthName],
COUNT(UserID) as totalReg
FROM (select DATEADD(MONTH, - nums.level, GETDATE()) as thedate
from nums
) mon12 left outer join
Months WITH (NOLOCK)
on month(mon12.thedate) = months.monthid left outer join
UserProfile WITH (NOLOCK)
ON Months.MonthID = MONTH(DateRegistered) and
DateRegistered > DATEADD(MONTH, -12, GETDATE())
GROUP BY YEAR(thedate), Months.[MonthName]
ORDER BY Months.[MonthName];
I find something strange about the query though. You are defining the span from the current date. However, you seem to be splitting the months themselves on calendar boundaries. I also find the table months to be awkward. Why aren't you just using the datename() and month() functions?
Try this out:
;With dates as (
Select DateName(Month, getdate()) as [Month],
DatePart(Year, getdate()) as [Year],
1 as Iteration
Union All
Select DateName(Month,DATEADD(MONTH, -Iteration, getdate())),
DatePart(Year,DATEADD(MONTH, -Iteration, getdate())),
Iteration + 1
from dates
where Iteration < 12
)
SELECT DISTINCT
d.Year,
d.Month as [MonthName],
COUNT(up.UserID)as totalReg
FROM dates d
LEFT OUTER JOIN UserProfile up ON d.Month = DateName(DateRegistered)
And d.Year = DatePart(Year, DateRegistered)
GROUP BY d.Year, d.Month
ORDER BY d.Year, d.Month
Here's my attempt at a solution:
declare #UserProfile table
(
id bigint not null identity(1,1) primary key clustered
, name nvarchar(32) not null
, dateRegistered datetime not null default(getutcdate())
)
insert #UserProfile
select 'person 1', '2011-01-23'
union select 'person 2', '2013-01-01'
union select 'person 3', '2013-05-27'
declare #yearMin int, #yearMax int
select #yearMin = year(MIN(dateRegistered))
, #yearMax= year(MAX(dateRegistered))
from #UserProfile
;with monthCte as
(
select 1 monthNo, DATENAME(month, '1900-01-01') Name
union all
select monthNo + 1, DATENAME(month, dateadd(month,monthNo,'1900-01-01'))
from monthCte
where monthNo < 12
)
, yearCte as
(
select #yearMin yearNo
union all
select yearNo + 1
from yearCte
where yearNo < #yearMax
)
select y.yearNo, m.Name, COUNT(up.id) UsersRegisteredThisPeriod
from yearCte y
cross join monthCte m
left outer join #UserProfile up
on year(up.dateRegistered) = y.yearNo
and month(up.dateRegistered) = m.monthNo
group by y.yearNo, m.monthNo, m.Name
order by y.yearNo, m.monthNo
SQL Fiddle Version: http://sqlfiddle.com/#!6/d41d8/6640
You have to calculate the counts in a Derived Table (or a CTE) first and then join
untested:
SELECT
COALESCE(dt.Years, YEAR(DATEADD(MONTH, -Months.MonthID, GETDATE()))),
Months.[MonthName],
COALESCE(dt.totalReg, 0)
FROM
Months WITH(NOLOCK)
LEFT OUTER JOIN
(
SELECT
YEAR(DateRegistered) AS Years,
MONTH(DateRegistered) AS Mon,
COUNT(UserID)AS totalReg
FROM UserProfile WITH(NOLOCK)
WHERE DateRegistered > DATEADD(MONTH, -12,GETDATE())
GROUP BY
YEAR(DateRegistered),
MONTH(DateRegistered)
) AS dt
ON Months.MonthID = dt.mon
ORDER BY 1, Months.MonthID
I changed the order to Months.MonthID instead of MonthName and i added year because you might have august 2012 and 2013 in your result.
Here is my sql code:
SELECT Month = datename(month,dateadd(mm,datediff(mm,0,StartTime),0)),
Cast ((ROUND(((CONVERT(DECIMAL(10,2),SUM(Duration)/60)/60)),2)*55) AS numeric(36,2))
FROM VExecutionGlobalHistory Where Tester <> 'dit2988' AND TestTypeID = 1
group by dateadd(mm,datediff(mm,0,StartTime),0), year(dateadd(mm,datediff(mm,0,StartTime),0))
order by dateadd(mm,datediff(mm,0,StartTime),0), year(dateadd(mm,datediff(mm,0,StartTime),0))
This query returns:
April|123
May|3456
June|856
July|345
I want it to return the following:
May|3456
June|856
July|345
Does anyone have any ideas? Im using Microsoft SQL Server Management Studio 2010
The following will return the first day of the month, 2 months back from today:
SELECT CAST(DATEADD(month, DATEDIFF(month, 0, GETDATE())-2, 0)AS DATE)
So you can add to your WHERE criteria:
SELECT Month = datename(month,dateadd(mm,datediff(mm,0,StartTime),0)),
Cast ((ROUND(((CONVERT(DECIMAL(10,2),SUM(Duration)/60)/60)),2)*55) AS numeric(36,2))
FROM VExecutionGlobalHistory
Where Tester <> 'dit2988' AND TestTypeID = 1
AND StartTime >= CAST(DATEADD(month, DATEDIFF(month, 0, GETDATE())-2, 0) AS DATE)
group by dateadd(mm,datediff(mm,0,StartTime),0), year(dateadd(mm,datediff(mm,0,StartTime),0))
order by dateadd(mm,datediff(mm,0,StartTime),0), year(dateadd(mm,datediff(mm,0,StartTime),0))
SELECT TOP 12 b.Month, b.Amount
FROM
(
SELECT TOP 12 a.Year, a.SortMonth, a.Month, a.Amount
FROM
(
SELECT
Year = datename(Year,StartTime),
SortMonth = month(StartTime),
Month = datename(month,dateadd(mm,datediff(mm,0,StartTime),0)),
Amount = Cast ((ROUND(((CONVERT(DECIMAL(10,2),SUM(Duration)/60)/60)),2)*55) AS numeric(36,2))
FROM VExecutionGlobalHistory
Where Tester <> 'dit2988'
AND TestTypeID = 1
group by
datename(Year,StartTime),
month(StartTime),
dateadd(mm,datediff(mm,0,StartTime),0),
year(dateadd(mm,datediff(mm,0,StartTime),0))
) a
ORDER BY a.Year, a.SortMonth DESC
) b
ORDER BY b.Year, b.SortMonth