SQL Aggregates OVER and PARTITION - sql

All,
This is my first post on Stackoverflow, so go easy...
I am using SQL Server 2008.
I am fairly new to writing SQL queries, and I have a problem that I thought was pretty simple, but I've been fighting for 2 days. I have a set of data that looks like this:
UserId Duration(Seconds) Month
1 45 January
1 90 January
1 50 February
1 42 February
2 80 January
2 110 February
3 45 January
3 62 January
3 56 January
3 60 February
Now, what I want is to write a single query that gives me the average for a particular user and compares it against all user's average for that month. So the resulting dataset after a query for user #1 would look like this:
UserId Duration(seconds) OrganizationDuration(Seconds) Month
1 67.5 63 January
1 46 65.5 February
I've been batting around different subqueries and group by scenarios and nothing ever seems to work. Lately, I've been trying OVER and PARTITION BY, but with no success there either. My latest query looks like this:
select Userid,
AVG(duration) OVER () as OrgAverage,
AVG(duration) as UserAverage,
DATENAME(mm,MONTH(StartDate)) as Month
from table.name
where YEAR(StartDate)=2014
AND userid=119
GROUP BY MONTH(StartDate), UserId
This query bombs out with a "Duration' is invalid in the select list because it is not contained in either an aggregate function or the GROUP BY clause" error.
Please keep in mind I'm dealing with a very large amount of data. I think I can make it work with CASE statements, but I'm looking for a cleaner, more efficient way to write the query if possible.
Thank you!

You are joining two queries together here:
Per-User average per month
All Organisation average per month
If you are only going to return data for one user at a time then an inline select may give you joy:
SELECT AVG(a.duration) AS UserAvergage,
(SELECT AVG(b.Duration) FROM tbl b WHERE MONTH(b.StartDate) = MONTH(a.StartDate)) AS OrgAverage
...
FROM tbl a
WHERE userid = 119
GROUP BY MONTH(StartDate), UserId
Note - using comparison on MONTH may be slow - you may be better off having a CTE (Common Table Expression)

missing partition clause in Average function
OVER ( Partition by MONTH(StartDate))

Please try this. It works fine to me.
WITH C1
AS
(
SELECT
AVG(Duration) AS TotalAvg,
[Month]
FROM [dbo].[Test]
GROUP BY [Month]
),
C2
AS
(
SELECT Distinct UserID,
AVG(Duration) OVER(PARTITION BY UserID, [Month] ORDER BY UserID) AS DetailedAvg,
[Month]
FROM [dbo].[Test]
)
SELECT C2.*, C1.TotalAvg
FROM C2 c2
INNER JOIN C1 c1 ON c1.[Month] = c2.[Month]
ORDER BY c2.UserID, c2.[Month] desc;

I was able to get it done using a self join, There's probably a better way.
Select UserId, AVG(t1.Duration) as Duration, t2.duration as OrgDur, t1.Month
from #temp t1
inner join (Select Distinct MONTH, AVG(Duration) over (partition by Month) as duration
from #temp) t2 on t2.Month = t1.Month
group by t1.Month, t1.UserId, t2.Duration
order by t1.UserId, Month desc
Here's using a CTE which is probably a better solution and definitely easier to read
With MonthlyAverage
as
(
Select MONTH, AVG(Duration) as OrgDur
from #temp
group by Month
)
Select UserId, AVG(t1.Duration) as Duration, m.duration as OrgDur , t1.Month
from #temp t1
inner join MonthlyAverage m on m.Month = t1.Month
group by UserId, t1.Month, m.duration

You can try below with less code.
SELECT Distinct UserID,
AVG(Duration) OVER(PARTITION BY [Month]) AS TotalAvg,
AVG(Duration) OVER(PARTITION BY UserID, [Month] ORDER BY UserID) AS DetailedAvg,
[Month]
FROM [dbo].[Test]

Related

Past 7 days running amounts average as progress per each date

So, the query is simple but i am facing issues in implementing the Sql logic. Heres the query suppose i have records like
Phoneno Company Date Amount
83838 xyz 20210901 100
87337 abc 20210902 500
47473 cde 20210903 600
Output expected is past 7 days progress as running avg of amount for each date (current date n 6 days before)
Date amount avg
20210901 100 100
20210902 500 300
20210903 600 400
I tried
Select date, amount, select
avg(lg) from (
Select case when lag(amount)
Over (order by NULL) IS NULL
THEN AMOUNT
ELSE
lag(amount)
Over (order by NULL) END AS LG)
From table
WHERE DATE>=t.date-7) as avg
From table t;
But i am getting wrong avg values. Could anyone please help?
Note: Ive tried without lag too it results the wrong avgs too
You could use a self join to group the dates
select distinct
a.dt,
b.dt as preceding_dt, --just for QA purpose
a.amt,
b.amt as preceding_amt,--just for QA purpose
avg(b.amt) over (partition by a.dt) as avg_amt
from t a
join t b on a.dt-b.dt between 0 and 6
group by a.dt, b.dt, a.amt, b.amt; --to dedupe the data after the join
If you want to make your correlated subquery approach work, you don't really need the lag.
select dt,
amt,
(select avg(b.amt) from t b where a.dt-b.dt between 0 and 6) as avg_lg
from t a;
If you don't have multiple rows per date, this gets even simpler
select dt,
amt,
avg(amt) over (order by dt rows between 6 preceding and current row) as avg_lg
from t;
Also the condition DATE>=t.date-7 you used is left open on one side meaning it will qualify a lot of dates that shouldn't have been qualified.
DEMO
You can use analytical function with the windowing clause to get your results:
SELECT DISTINCT BillingDate,
AVG(amount) OVER (ORDER BY BillingDate
RANGE BETWEEN TO_DSINTERVAL('7 00:00:00') PRECEDING
AND TO_DSINTERVAL('0 00:00:00') FOLLOWING) AS RUNNING_AVG
FROM accounts
ORDER BY BillingDate;
Here is a DBFiddle showing the query in action (LINK)

DISTINCT ON to find min and max times

I have tried using DISTINCT ON with posrgresql to achieve the following:
Lets say I have a table that looks like this:
id time price
1 12:00 10
1 13:00 20
1 14:00 30
And my goal is to create a table with only 1 row per id, that shows a column of the minimum time price and the maximum time price. Something that looks like this:
id min_time_price max_time_price
1 10 30
I tried using DISTINCT ON (id) but can't really get it.
Would love some help, Thank you!
Here is one method:
select t.id, tmin.price, tmax.price
from (select t.id, min(time) as min_time, max(time) as max_time
from t
) t join
t tmin
on t.id = tmin.id and t.min_time = tmin.time join
t tmax
on t.id = tmax.id and t.max_time = tmax.time;
You can also use aggregation. Postgres doesn't have first()/last() aggregation functions, but arrays are handy:
select t.id,
array_agg(price order by time asc)[1] as min_time_price,
array_agg(price order by time desc)[1] as max_time_price
from t
group by id;
Or using first_value() and last_value():
select distinct t.id,
first_value(price) over (partition by time order by time) as min_time_price,
first_value(price) over (partition by time order by time desc) as max_time_price
from t

how to find number of active users for say 1 day,2 days, 3 days.....postgreSQL

A distribution of # days active within a week: I am trying to find how many members are active for 1 day, 2days, 3days,…7days during a specific week 3/1-3/7.
Is there any way to use aggregate function on top of partition by?
If not what can be used to achieve this?
select distinct memberID,count(date) over(partition by memberID) as no_of_days_active
from visitor
where date between '"2019-01-01 00:00:00"' and '"2019-01-07 00:00:00"'
order by no_of_days_active
result should look something like this
#Days Active Count
1 20
2 32
3 678
4 34
5 3
6 678
7 2345
I think you want two levels of aggregation to count the number of days during the week:
select num_days_active, count(*) as num_members
from (select memberID, count(distinct date::date) as num_days_active
from visitor
where date >= '2019-01-01'::date and
date < '2019-01-08'::date
group by memberID
) v
group by num_days_active
order by num_days_active;
Note that I changed the date comparisons. If you have a time component, then between does not work. And, because you included time in the constant, I added an explicit conversion to date for the count(distinct). That might not be necessary, if date is really a date with no time component.
Piggybacking off of #Gordon's answer, I personally like using a with statement for the subqueries:
with dat as (
select distinct
memberID,
count(date) over(partition by memberID) as no_of_days_active
from visitor
where 1=1
and date between '2019-01-01'::date and '2019-01-07'::date
order by no_of_days_active
)
select
no_of_days_active,
count(no_of_days_active) no_of_days_active_cnt
from dat
group by no_of_days_active
order by no_of_days_active

How to calculate daily average from aggregate results with SQL?

I'm working on outputting some data and I want to pull the daily average of some numbers.
As you can see, what I want to do is count the amount of rows received/results(think the row ID) and then divide it against the day value to make the daily average.(30/1) , (64/2) etc I've tried everything, but I keep running into a wall with this.
As it stands, I'm guessing to make this work a sub query of some sort is needed. I just don't know how to get the day(Row id 1,2,3,4 etc) to use for the division.
SELECT calendar_date, SUM(NY_dayscore * cAttendance)
FROM vw_Appointments
WHERE status = 'Confirmed'
Group by calendar_date
Attempted count with distinct, to no avail
SUM(NY_dayscore * cAttendance) ) / count(distinct calendar_date)
My original code is long and cba to post it all. So just attempting to post a small sample code to get guidance on the issue.
In SQL Server 2012+, you would use the cumulative average:
select calendar_date, sum(NY_dayscore * cAttendance),
avg(sum(NY_dayscore * cAttendance)) over (order by calendar_date) as running_average
from vw_appointments a
where status = 'Confirmed'
group by calendar_date
order by calendar_date;
In SQL Server 2008, this is more difficult:
with a as (
select calendar_date, sum(NY_dayscore * cAttendance) as showed
from vw_appointments a
where status = 'Confirmed'
group by calendar_date
)
select a.*, a2.running_average
from a outer apply
(select avg(showed) as running_average
from a a2
where a2.calendar_date <= a.calendar_date
) a2
order by calendar_date;
Is it ROW_NUMBER() that you are missing?
SELECT
calendar_date,
SUM(NY_dayscore * cAttendance) / (ROW_NUMBER() OVER (ORDER BY calendar_date ASC)) AS average
FROM vw_Appointments
WHERE status = 'Confirmed'
GROUP BY calendar_date
ORDER BY calendar_date
I think you need sum(showed) over (..)/row_number() over (..)
WITH Table1(date, showed) AS
(
SELECT '2019-01-02', 30 UNION ALL
SELECT '2019-01-03', 34 UNION ALL
SELECT '2019-01-03', 41 UNION ALL
SELECT '2019-01-04', 48
)
SELECT date,
sum(showed) over (order by date) /
row_number() over (order by date)
as daily_average
FROM Table1
GROUP BY showed, date;
date daily_average
2019-01-02 30
2019-01-03 52
2019-01-03 35
2019-01-04 38
Demo

sql return 1st day of each month in table

I have a sql table like so with two columns...
3/1/17 100
3/2/17 200
3/3/17 300
4/3/17 600
4/4/17 700
4/5/17 800
I am trying to run a query that returns the 1st day of each month in that above table, and grab the corresponding value.
results should be
3/1/17 100
4/3/17 600
then once I have these results... do something with each one.
any ideas how I can get started?
In standard SQL, you would use row_number():
select t.*
from (select t.*,
row_number() over (partition by extract(year from dte), extract(month from dte)
order by dte asc) as seqnum
from t
) t
where seqnum = 1;
Most databases support this functionality, but the exact functions (particularly for dates) may differ depending on the database.
An alternative (SQL Server flavour):
SELECT t.*
FROM YourTable t
JOIN (
select MIN(DateColumn) as MinimumDate
from YourTable
group by FORMAT(DateColumn,'yyyyMM')
) q on (t.DateColumn = q.MinimumDate)
ORDER BY t.DateColumn;
For the GROUP BY this will also be fine:
group by YEAR(DateColumn), MONTH(DateColumn)
or
group by DATEPART(YEAR,DateColumn), DATEPART(MONTH,DateColumn)