Year over Year by Month Comparison and Month to Date in BigQuery - sql

Edit: #shawnt00 has the correct answer. Thank you very much!
I am having trouble accurately doing a year over year comparison by month but at any point during the month. For example for August 2022 vs 2021, I want to compare August 1 - August 25, rather than full month of August 2021.
I am also using a daily date field.
I want the final result to basically be:
Product_ID, Year, Month, PY_Sales, CY_Sales
Edit: I have daily totals. Some products do have not sales on certain days though:
product_id
sale_date
units
1
2021-01-01
5
2
2021-01-02
4
...
...
...
1
2021-06-05
2
2
2022-01-06
1
2
2022-08-15
9
This is the code I have, but it doesn't do MTD. So 2021 August is the entire month of August and I want it the same dates for 2022. I used this code because some products do not have sales on certain months.
WITH cte AS
(
SELECT
PRODUCT_ID,
EXTRACT(YEAR FROM SALE_DATE) AS Year,
EXTRACT(MONTH FROM SALE_DATE) AS Month,
CONCAT(EXTRACT(YEAR FROM SALE_DATE), '-',EXTRACT(MONTH FROM SALE_DATE)) AS Year_Month,
SUM(Units) AS Units
FROM data
WHERE Product_ID = 1
AND DATE(SALE_DATE) >= '2019-01-01'
GROUP BY 1, 2, 3
),
diff AS
(
SELECT
COALESCE(c.PRODUCT_ID, p.PRODUCT_ID) AS Product_ID,
COALESCE(c.Year, p.Year + 1) AS Year,
COALESCE(c.Month, p.Month) AS Month,
IFNULL(c.Units, 0) AS Current_Units,
IFNULL(p.Units, 0) AS Previous_Units,
NULLIF(((IFNULL(c.Units, 0) - IFNULL(p.Units,0)) / p.Units),0) * 100 AS Percent_Change
FROM CTE c
FULL OUTER JOIN CTE p ON c.PRODUCT_ID = p.PRODUCT_ID AND c.Year = p.Year + 1 AND c.Month = p.Month
WHERE c.Year <= EXTRACT(YEAR FROM CURRENT_DATE())
ORDER BY 2, c.Year, c.Month
)
SELECT *
FROM diff
--This is to avoid dividing by 0
WHERE diff.Previous_Units > 0
--AND Percent_Change <= -.5

You could just roll up two different monthly totals and then switch for the current month comparison:
with agg as (
select
PRODUCT_ID,
extract(year from SALE_DATE) as yr,
extract(month from SALE_DATE) as mth,
sum(Units) as Units,
sum(case when extract(day from SALE_DATE) <= extract(day from current_date())
then Units end) as UnitsMTD
from data
where date(SALE_DATE) >= '2019-01-01' -- one year before report output
group by 1, 2, 3
)
select c.Yr, c.Mth, c.PRODUCT_ID,
case when Yr = extract(year from current_date())
and Mth = extract(month from current_date())
then (c.UnitsMTD - p.UnitsMTD) / p.UnitsMTD
else (c.Units - p.Units ) / p.Units
end as Percent_Change
from agg c left outer join agg p
on p.Product_ID = c.Product_ID and p.Yr = c.Yr - 1 and p.Mth = c.Mth
order by c.Yr, c.Mth, c.PRODUCT_ID;
Note my earlier comment about leap years. This will treat February 28 of the year following a leap year as an "MTD" month. You might need to handle that differently inside the case expression.

Related

BigQuery - Year over Year Comparison with Month to Date

I am having trouble accurately doing a year over year comparison by month but at any point during the month. For example for August 2022 vs 2021, I want to compare August 1 to today, rather than full month of August 2021.
My data has a date field.
I want the final result to basically be:
Product_ID, Year, Month, PY_Sales, CY_Sales
I have daily totals. Some products do have not sales on certain days though. Here's an example:
product_id
sale_date
units
1
2021-01-01
5
2
2021-01-02
4
...
...
...
1
2021-06-05
2
2
2021-08-01
1
2
2021-08-31
6
2
2022-01-06
1
2
2022-08-15
9
The final result for August should be:
product_id
Year
Month
PY_Sales
CY_Sales
2
2022
8
1
9
Right now my code will show 7 for August for product_id = 2 because 6 sales happened on August 31st but that day hasn't happened yet in 2022.
This is the code I have, but it doesn't do MTD. Right now, PY_Sales for August 2022 is showing the entire August of 2021, but I want it to show the MTD of August 2021. I used this code because some products do not have sales on certain months.
WITH cte AS
(
SELECT
PRODUCT_ID,
EXTRACT(YEAR FROM SALE_DATE) AS Year,
EXTRACT(MONTH FROM SALE_DATE) AS Month,
CONCAT(EXTRACT(YEAR FROM SALE_DATE), '-',EXTRACT(MONTH FROM SALE_DATE)) AS Year_Month,
SUM(Units) AS Units
FROM data
WHERE Product_ID = 1
AND DATE(SALE_DATE) >= '2019-01-01'
GROUP BY 1, 2, 3
),
diff AS
(
SELECT
COALESCE(c.PRODUCT_ID, p.PRODUCT_ID) AS Product_ID,
COALESCE(c.Year, p.Year + 1) AS Year,
COALESCE(c.Month, p.Month) AS Month,
IFNULL(c.Units, 0) AS Current_Units,
IFNULL(p.Units, 0) AS Previous_Units,
NULLIF(((IFNULL(c.Units, 0) - IFNULL(p.Units,0)) / p.Units),0) * 100 AS Percent_Change
FROM CTE c
FULL OUTER JOIN CTE p ON c.PRODUCT_ID = p.PRODUCT_ID AND c.Year = p.Year + 1 AND c.Month = p.Month
WHERE c.Year <= EXTRACT(YEAR FROM CURRENT_DATE())
ORDER BY 2, c.Year, c.Month
)
SELECT *
FROM diff
--This is to avoid dividing by 0
WHERE diff.Previous_Units > 0
--AND Percent_Change <= -.5
I'm being a little repetitive but I hope this is clear! Thank you so much!
In the cte table you summarize the sold units by month and year.
Your question can be solved by adding here a column units_last_year. This contains the units, which are sold up to the day one year ago. Today is the 27th of August 2022, therefore the units on the 31th of August 2021 will be set to zero.
SUM(Units) AS Units,
SUM(IF(SALE_DATE< date_sub(current_Date(),interval 1 year), Units, 0 )) as units_last_year
Please use the safe_divide command, if there is any chance of diving by zero
Here is the full query with example data.
You given an example of fixed dates, which are compared to the current date. Therefore, the query would not show the desired effect after 30th of August 2022.
The product_id three is made up values related to the current date, thus the following query yields results after August 2022.
with data as (
select *,date(sale_date_) as sale_date
from (
Select 1 product_id, "2021-01-01" sale_date_, 5 units
union all select 2,"2021-01-02", 4
union all select 1,"2021-06-05", 2
union all select 2,"2021-08-01", 1
union all select 2,"2021-08-31", 6
union all select 2,"2022-01-06", 1
union all select 2,"2022-08-15", 9
union all select 3, current_date(), 10
union all select 3, date_sub(current_date(),interval 1 year), 9
union all select 3, date_sub( date_trunc(current_date(),month),interval 1 year), 1
)
),
cte AS
(
SELECT
PRODUCT_ID,
EXTRACT(YEAR FROM SALE_DATE) AS Year,
EXTRACT(MONTH FROM SALE_DATE) AS Month,
CONCAT(EXTRACT(YEAR FROM SALE_DATE), '-',EXTRACT(MONTH FROM SALE_DATE)) AS Year_Month,
SUM(Units) AS Units,
sum(if(SALE_DATE< date_sub(current_Date(),interval 1 year), units, 0 )) as units_last_year
FROM data
WHERE # Product_ID = 1 AND
DATE(SALE_DATE) >= '2019-01-01'
GROUP BY 1, 2, 3, 4
),
diff AS
(
SELECT
COALESCE(c.PRODUCT_ID, p.PRODUCT_ID) AS Product_ID,
COALESCE(c.Year, p.Year + 1) AS Year,
COALESCE(c.Month, p.Month) AS Month,
IFNULL(c.Units, 0) AS Current_Units,
IFNULL(p.Units, 0) AS Previous_Units,
IFNULL(p.Units_last_Year, 0) AS Previous_Units_ok,
NULLIF(((IFNULL(c.Units, 0) - IFNULL(p.Units,0)) / p.Units),0) * 100 AS Percent_Change,
NULLIF(safe_divide((IFNULL(c.Units, 0) - IFNULL(p.Units_last_Year,0)) , p.Units_last_Year),0) * 100 AS Percent_Change_ok,
FROM CTE c
FULL OUTER JOIN CTE p ON c.PRODUCT_ID = p.PRODUCT_ID AND c.Year = p.Year + 1 AND c.Month = p.Month
WHERE c.Year <= EXTRACT(YEAR FROM CURRENT_DATE())
ORDER BY 2, c.Year, c.Month
)
SELECT *
FROM diff

Count active users if they have made a buy in the last three months (Historical) SQL

This is my query
with months (Date,Familia) as (
select cast(eomonth(datefromparts(year(date_var),
month(date_var),01)) as datetime) as Fecha, p.family as 'Familia'
from sales v
left join products p on p.id_product=v.id_product
where date_var >= '2016-08-01'
group by date_var, p.family
)
select m.Date, m.Family, (
select count(distinct v.user_id)
from sales v
where datediff(month, m.Date, v.date_var) between -2 and 0 and
v.date_var >= '2016-08-01'
) as 'Active Users'
from months m
group by m.family, m.Date
order by m.Date
I want to obtain the number of active users, taking into account that a user counts as active if they have made a purchase in the last three months.
For instance
family
year
month
#
Nubrenza
2017
1
2500
Keppra
2017
1
350
Nubrenza
2017
2
2400
Keppra
2017
2
357
Active users of January 2017 would be count( DISTINCT users) who have made a transaction in January 2017, Dec 2016 and / or Nov 2016 and so on...
Update my query is now showing the distinct count of users grouping them by month but it's returning the same value for all my families, how can I fix that?
You can generate the months and use a subquery:
with months as (
select convert(date, '2017-01-01') as month
union all
select dateadd(month, 1, month)
from months
where month < '2018-01-01'
)
select m.month,
(select count(*)
from mytable t
where datediff(month, date_var, m.month) between 0 and 2
)
from months m;

SQL Count Entries for each Month of the last 6 Months

I got a problem while trying to count the entries that were created in a month for the last 6 months.
The table looks like this:
A B C D
Year Month Startingdate Identifier
-----------------------------------------
2019 3 2019-03-12 OAM_1903121
2019 2 2019-03-21 OAM_1902211
And the result should look like:
A B C
Year Month Amount of orders
---------------------------------
2019 3 26
2019 2 34
This is what I have so far, but it doesn't get me the proper results:
SELECT year, month, COUNT(Startingdate) as Amount
FROM table
WHERE Startingdate > ((TRUNC(add_months(sysdate,-3) , 'MM'))-1)
GROUP BY year, month
I have not tested it, but it should work:
select year, month, count(Stringdate) as Amount_of_order
from table
where Stringdate between add_months(sysdate, -6) and sysdate
group by year, month;
Let me know.
Try that :
SELECT YEAR(Startingdate) AS [Year], MONTH(Startingdate) AS [Month], COUNT(*) AS Amount
FROM table
WHERE Startingdate > DATEADD(MONTH, -6, GETDATE())
GROUP BY YEAR(Startingdate), MONTH(Startingdate)
ORDER BY YEAR(Startingdate), MONTH(Startingdate) DESC
I think your issue is the filtering. If so, this should handle the most recent six full months:
SELECT year, month, COUNT(*) as num_orders
FROM table
WHERE Startingdate >= TRUNC(add_months(sysdate, -6) , 'MM')
GROUP BY year, month;

postgreSQL- Count for value between previous month start date and end date

I have a table as follows
user_id date month year visiting_id
123 11-04-2017 APRIL 2017 4500
123 12-05-2017 MAY 2017 4567
123 13-05-2017 MAY 2017 4568
123 17-05-2017 MAY 2017 4569
123 22-05-2017 MAY 2017 4570
123 11-06-2017 JUNE 2017 4571
123 12-06-2017 JUNE 2017 4572
I want to calculate the visiting count for the current month and last month at the monthly level as follows:
user_id month year visit_count_this_month visit_count_last_month
123 APRIL 2017 1 0
123 MAY 2017 4 1
123 JUNE 2017 2 4
I was able to calculate visit_count_this_month using the following query
SELECT v.user_id, v.month, v.year,
SUM(is_visit_this_month) as visit_count_this_month
FROM
(SELECT user_id, date, month, year,
CASE WHEN TO_CHAR(date, 'MM/YYYY') = TO_CHAR(date, 'MM/YYYY')
THEN 1 ELSE 0
END as is_visit_this_month
FROM visits
GROUP BY user_id, date, month, year
HAVING user_id = 123) v
GROUP BY v.user_id, v.month, v.year
However, I'm stuck with calculating visit_count_last_month. Similar to this, I also want to calculate visit_count_last_2months.
Can somebody help?
You can use a LATERAL JOIN like this:
SELECT user_id, month, year, COUNT(*) as visit_count_this_month, visit_count_last_month
FROM visits v
CROSS JOIN LATERAL (
SELECT COUNT(*) as visit_count_last_month
FROM visits
WHERE user_id = v.user_id
AND date = (CAST(v.date AS date) - interval '1 month')
) l
GROUP BY user_id, month, year, visit_count_last_month;
SQLFiddle - http://sqlfiddle.com/#!15/393c8/2
Assuming there are values for every month, you can get the counts per month first and use lag to get the previous month's values per user.
SELECT T.*
,COALESCE(LAG(visits,1) OVER(PARTITION BY USER_ID ORDER BY year,mth),0) as last_month_visits
,COALESCE(LAG(visits,2) OVER(PARTITION BY USER_ID ORDER BY year,mth),0) as last_2_month_visits
FROM (
SELECT user_id, extract(month from date) as mth, year, COUNT(*) as visits
FROM visits
GROUP BY user_id, extract(month from date), year
) T
If there can be missing months, it is best to generate all months within a specified timeframe and left join ing the table on to that. (This example shows it for all the months in 2017).
select user_id,yr,mth,visits
,coalesce(lag(visits,1) over(PARTITION BY USER_ID ORDER BY yr,mth),0) as last_month_visits
,coalesce(lag(visits,2) OVER(PARTITION BY USER_ID ORDER BY yr,mth),0) as last_2_month_visits
from (select u.user_id,extract(year from d.dt) as yr, extract(month from d.dt) as mth,count(v.visiting_id) as visits
from generate_series(date '2017-01-01', date '2017-12-31',interval '1 month') d(dt)
cross join (select distinct user_id from visits) u
left join visits v on extract(month from v.dt)=extract(month from d.dt) and extract(year from v.dt)=extract(year from d.dt) and u.user_id=v.user_id
group by u.user_id,extract(year from d.dt), extract(month from d.dt)
) t

SQL spread month value into weeks

I have a table where I have values by month and I want to spread these values by week, taking into account that weeks that spread into two month need to take part of the value of each of the month and weight on the number of days that correspond to each month.
For example I have the table with a different price of steel by month
Product Month Price
------------------------------------
Steel 1/Jan/2014 100
Steel 1/Feb/2014 200
Steel 1/Mar/2014 300
I need to convert it into weeks as follows
Product Week Price
-------------------------------------------
Steel 06-Jan-14 100
Steel 13-Jan-14 100
Steel 20-Jan-14 100
Steel 27-Jan-14 128.57
Steel 03-Feb-14 200
Steel 10-Feb-14 200
Steel 17-Feb-14 200
As you see above, the week that overlaps between Jan and Feb needs to be calculated as follows
(100*5/7)+(200*2/7)
This takes into account tha the week of the 27th has 5 days that fall into Jan and 2 into Feb.
Is there any possible way to create a query in SQL that would achieve this?
I tried the following
First attempt:
select
WD.week,
PM.PRICE,
DATEADD(m,1,PM.Month),
SUM(PM.PRICE/7) * COUNT(*)
from
( select '2014-1-1' as Month, 100 as PRICE
union
select '2014-2-1' as Month, 200 as PRICE
)PM
join
( select '2014-1-20' as week
union
select '2014-1-27' as week
union
select '2014-2-3' as week
)WD
ON WD.week>=PM.Month
AND WD.week < DATEADD(m,1,PM.Month)
group by
WD.week,PM.PRICE, DATEADD(m,1,PM.Month)
This gives me the following
week PRICE
2014-1-20 100 2014-02-01 00:00:00.000 14
2014-1-27 100 2014-02-01 00:00:00.000 14
2014-2-3 200 2014-03-01 00:00:00.000 28
I tried also the following
;with x as (
select price,
datepart(week,dateadd(day, n.n-2, t1.month)) wk,
dateadd(day, n.n-1, t1.month) dt
from
(select '2014-1-1' as Month, 100 as PRICE
union
select '2014-2-1' as Month, 200 as PRICE) t1
cross apply (
select datediff(day, t.month, dateadd(month, 1, t.month)) nd
from
(select '2014-1-1' as Month, 100 as PRICE
union
select '2014-2-1' as Month, 200 as PRICE)
t
where t1.month = t.month) ndm
inner join
(SELECT (a.Number * 256) + b.Number AS N FROM
(SELECT number FROM master..spt_values WHERE type = 'P' AND number <= 255) a (Number),
(SELECT number FROM master..spt_values WHERE type = 'P' AND number <= 255) b (Number)) n --numbers
on n.n <= ndm.nd
)
select min(dt) as week, cast(sum(price)/count(*) as decimal(9,2)) as price
from x
group by wk
having count(*) = 7
order by wk
This gimes me the following
week price
2014-01-07 00:00:00.000 100.00
2014-01-14 00:00:00.000 100.00
2014-01-21 00:00:00.000 100.00
2014-02-04 00:00:00.000 200.00
2014-02-11 00:00:00.000 200.00
2014-02-18 00:00:00.000 200.00
Thanks
If you have a calendar table it's a simple join:
SELECT
product,
calendar_date - (day_of_week-1) AS week,
SUM(price/7) * COUNT(*)
FROM prices AS p
JOIN calendar AS c
ON c.calendar_date >= month
AND c.calendar_date < DATEADD(m,1,month)
GROUP BY product,
calendar_date - (day_of_week-1)
This could be further simplified to join only to mondays and then do some more date arithmetic in a CASE to get 7 or less days.
Edit:
Your last query returned jan 31st two times, you need to remove the =from on n.n < ndm.nd. And as you seem to work with ISO weeks you better change the DATEPART to avoid problems with different DATEFIRST settings.
Based on your last query I created a fiddle.
;with x as (
select price,
datepart(isowk,dateadd(day, n.n, t1.month)) wk,
dateadd(day, n.n-1, t1.month) dt
from
(select '2014-1-1' as Month, 100.00 as PRICE
union
select '2014-2-1' as Month, 200.00 as PRICE) t1
cross apply (
select datediff(day, t.month, dateadd(month, 1, t.month)) nd
from
(select '2014-1-1' as Month, 100.00 as PRICE
union
select '2014-2-1' as Month, 200.00 as PRICE)
t
where t1.month = t.month) ndm
inner join
(SELECT (a.Number * 256) + b.Number AS N FROM
(SELECT number FROM master..spt_values WHERE type = 'P' AND number <= 255) a (Number),
(SELECT number FROM master..spt_values WHERE type = 'P' AND number <= 255) b (Number)) n --numbers
on n.n < ndm.nd
) select min(dt) as week, cast(sum(price)/count(*) as decimal(9,2)) as price
from x
group by wk
having count(*) = 7
order by wk
Of course the dates might be from multiple years, so you need to GROUP BY by the year, too.
Actually, you need to spred it over days, and then get the averages by week. To get the days we'll use the Numbers table.
;with x as (
select product, price,
datepart(week,dateadd(day, n.n-2, t1.month)) wk,
dateadd(day, n.n-1, t1.month) dt
from #t t1
cross apply (
select datediff(day, t.month, dateadd(month, 1, t.month)) nd
from #t t
where t1.month = t.month and t1.product = t.product) ndm
inner join numbers n on n.n <= ndm.nd
)
select product, min(dt) as week, cast(sum(price)/count(*) as decimal(9,2)) as price
from x
group by product, wk
having count(*) = 7
order by product, wk
The result of datepart(week,dateadd(day, n.n-2, t1.month)) expression depends on SET DATEFIRST so you might need to adjust accordingly.