SQL/Presto: how to rank within a subgroup of each group - sql

I have a table like the following:
group_id sub_group_id user_id score time
1 a1 ann 1 2019
1 a1 bob 1 2020
1 a2 cat 0 2020
2 b1 dan 0 2019
2 b1 eva 0 2019
2 b1 ed 1 2020
2 b2 liz 1 2020
i want to rank user_id within subgroup of each group by the score and then by time (earlier better) each user_id gets. so the desired output is
group_id sub_group_id user_id score time rank
1 a1 ann 1 2019 1
1 a1 bob 1 2020 2
1 a2 cat 0 2020 1
2 b1 dan 0 2019 1
2 b1 eva 0 2019 1
2 b1 ed 1 2020 2
2 b2 liz 1 2020 1

Use rank():
select t.*,
rank() over (partition by group_id, sub_group_id order by score desc, time) as ranking
from t;
Actually, I'm not sure if higher scores are better than lower ones, so you might want score asc.

Related

Non-repeated values in Big Query

I am fairly new to SQL, so this might be an easy solution for most, but I am having an issue with joins in Big Query. I have two tables:
TABLE A
id name purchases
1 alex 2
2 jane 7
3 peter 8
4 mario 1
5 luigi 6
TABLE B
id name visited
1 alex jan
2 jane jan
2 jane feb
3 peter jan
3 peter feb
3 peter mar
4 mario feb
5 luigi mar
I want my end result to have unique number of purchases per name/id, so the following:
TABLE C
id name visited purchases
1 alex jan 2
2 jane jan 7
2 jane feb 0
3 peter jan 8
3 peter feb 0
3 peter mar 0
4 mario feb 1
5 luigi mar 6
However, no matter what joins I perform, I end up with number of purchases per user matched every time, like the following:
id name visited purchases
1 alex jan 2
2 jane jan 7
2 jane feb 7
3 peter jan 8
3 peter feb 8
3 peter mar 8
4 mario feb 1
5 luigi mar 6
What would be the query to have Table C from Tables A and B?
Thank you.
One method is using row_number()
select b.*, coalesce(a.purchases, 0) purchases
from (
select *, row_number() over(partition by id order by visited) rn
from b ) b
left join a on a.id = b.id and b.rn=1
You may wish to decode visited to an ordinal depending on ordering requirements, for example
.. order by case visited when 'jan' then 1 when .. end ..

R - get a vector that tells me if a value of another vector is the first appearence or not

I have a data frame of sales with three columns: the code of the customer, the month the customer bought that item, and the year.
A customer can buy something in september and then in december make another purchase, so appear two times. But I'm interested in knowing the absolutely new customoers by month and year.
So I have thought in make an iteration and some checks and use the %in% function and build a boolean vector that tells me if a customer is new or not and then count by month and year with SQL using this new vector.
But I'm wondering if there's a specific function or a better way to do that.
This is an example of the data I would like to have:
date cust month new_customer
1 14975 25 1 TRUE
2 14976 30 1 TRUE
3 14977 22 1 TRUE
4 14978 4 1 TRUE
5 14979 25 1 FALSE
6 14980 11 1 TRUE
7 14981 17 1 TRUE
8 14982 17 1 FALSE
9 14983 18 1 TRUE
10 14984 7 1 TRUE
11 14985 24 1 TRUE
12 14986 22 1 FALSE
So put it more simple: the data frame is sorted by date, and I'm interested in a vector (new_customer) that tells me if the customer purchased something for the first time or not. For example customer 25 bought something the first day, and then four days later bought something again, so is not a new customer. The same can be seen with customer 17 and 22.
I create dummy data my self with id, month of numeric format, and year
dat <-data.frame(
id = c(1,2,3,4,5,6,7,8,1,3,4,5,1,2,2),
month = c(1,6,7,8,2,3,4,8,11,1,10,9,1,12,2),
year = c(2019,2019,2019,2019,2019,2020,2020,2020,2020,2020,2021,2021,2021,2021,2021)
)
id month year
1 1 1 2019
2 2 6 2019
3 3 7 2019
4 4 8 2019
5 5 2 2019
6 6 3 2020
7 7 4 2020
8 8 8 2020
9 1 11 2020
10 3 1 2020
11 4 10 2021
12 5 9 2021
13 1 1 2021
14 2 12 2021
15 2 2 2021
Then, group by id and arrange by year and month (order is meaningful). Then use filter and row_number().
dat %>%
group_by(id) %>%
arrange(year, month) %>%
filter(row_number() == 1)
id month year
<dbl> <dbl> <dbl>
1 1 1 2019
2 5 2 2019
3 2 6 2019
4 3 7 2019
5 4 8 2019
6 6 3 2020
7 7 4 2020
8 8 8 2020
Sample Code
You can change in your code according to this logic:-
Create Table:-
CREATE TABLE PURCHASE(Posting_Date DATE,Customer_Id INT,Customer_Name VARCHAR(15));
Insert Data Into Table
Posting_Date Customer_Id Customer_Name
2018-01-01 C_01 Jack
2018-02-01 C_01 Jack
2018-03-01 C_01 Jack
2018-04-01 C_02 James
2019-04-01 C_01 Jack
2019-05-01 C_01 Jack
2019-05-01 C_03 Gill
2020-01-01 C_02 James
2020-01-01 C_04 Jones
Code
WITH Date_CTE (PostingDate,CustomerID,FirstYear)
AS
(
SELECT MIN(Posting_Date) as [Date],
Customer_Id,
YEAR(MIN(Posting_Date)) as [F_Purchase_Year]
FROM PURCHASE
GROUP BY Customer_Id
)
SELECT T.[ActualYear],(CASE WHEN T.[Customer Status] = 'new' THEN COUNT(T.[Customer Status]) END) AS [New Customer]
FROM (
SELECT DISTINCT YEAR(T2.Posting_Date) AS [ActualYear],
T2.Customer_Id,
(CASE WHEN T1.FirstYear = YEAR(T2.Posting_Date) THEN 'new' ELSE 'old' END) AS [Customer Status]
FROM Date_CTE AS T1
left outer join PURCHASE AS T2 ON T1.CustomerID = T2.Customer_Id
) AS T
GROUP BY T.[ActualYear],T.[Customer Status]
Final Result
ActualYear New Customer
2018 2
2019 1
2020 1
2019 NULL
2020 NULL

Compare data from for specific column grouping and Update based on criteria

I have a table with the following structure:
Employee Project Task Accomplishment Score Year
John A 1 5 60 2016
John A 1 6 40 2018
John A 2 3 30 2016
Simon B 2 0 30 2017
Simon B 2 4 30 2019
David C 1 3 20 2015
David C 1 2 40 2016
David C 3 0 25 2017
David C 3 5 35 2017
I want to create a view with Oracle SQLout of the above table which looks like as follows:
Employee Project Task Accomplishment Score Year UpdateScore Comment
John A 1 5 60 2016 60
John A 1 6 40 2018 100 (=60+40)
John A 2 3 30 2016 30
Simon B 2 0 30 2017 30
Simon B 2 4 40 2019 40 (no update because Accomplishement was 0)
David C 1 3 20 2015 20
David C 1 2 40 2016 60 (=20+40)
David C 3 0 25 2017 25
David C 3 5 35 2017 35 (no update because Accomplishement was 0)
The Grouping is: Employee-Project-Task.
The Rule of the UpdateScore column:
If for a specific Employee-Project-Task group Accomplishment column value is greater than 0 for the previous year, add the previous year's score to the latest year for the same Employee-Project-Task group.
For example: John-A-1 is a group which is different from John-A-2. So as we can see for John-A-1 the Accomplishment is 5 (which is greater than 0) in 2016, so we add the Score from 2016 with the score of 2018 for the John-A-1 and the updated score becomes 100.
For Simon-B-2, the accomplishment was 0, so there will be no update for 2019 for Simon-B-2.
Note: I don't need the Comment field, it is there just for more clarification.
Use analytic functions to determine if there was a score for the previous year, and if so, add it to the UpdatedScore.
select Employee, Project, Task, Accomplishment, Score, Year,
case when lag(Year) over (partition by Employee, Project order by Year) = Year - 1
then lag(Score) over (partition by Employee, Project order by Year)
else 0
end + Score as UpdatedScore
from EmployeeScore;
This is a bit strange -- you are counting the accomplishment of 0 in one year but not the next. Okay.
Use analytic functions:
select t.*,
(case when lag(accomplishment) over (partition by Employee, Project, Task order by year) > 0
then lag(score) over (partition by Employee, Project, Task order by year)
else 0
end) + score as update_score
from t;
from t

Creating Pivots based for a rowcount in SQL

I have 2 tables T1 and T2.
T1 has the following columns:
Id, EntityId, TypeofValue, Year, value
EntityId can have 7 values:
1,2,3,4,5,100,101
Typeofvalue can have 2 values
1 indicates Actual
2 indicates Target
T2 has the following columns:
NoOfRecordsToDisplay
I need to fetch the number of records (if existing) for Target corresponding to an Id. The record count (1 if any record is present 0 if none) for Target needs to be grouped into two categories: First having Entityid 1 and Maximum records under Second group with entityids 2,3,4,100,101 [Not 5]
However, the catches are:
Sometimes Target value might not be present for a year
I need to get only last records for targets on the basis of NoOfRecordsToDisplay (The number of records to display comes from T2) for actual
Example1:
NoOfRecordsToDisplay =3, ID =123
The data below should return
CountGroup1: 1, Countgroup2: 1
as Entityid 1 has least one value for target for last 3 years -2015, 2014,2013 in this case
as Entityid 2 or 3 has at least 1 value for years -2015, 2014
Id EntityId TypeofValue Year Value
123 1 1 2015 55
123 1 1 2014 56
123 1 1 2013 57
123 1 1 2012 58
123 1 2 2015 50
123 1 2 2014 50
123 1 2 2013 50
123 1 2 2012 50
123 2 1 2015 55
123 2 1 2014 56
123 3 1 2015 57
123 3 1 2014 58
123 2 2 2015 55
123 2 2 2014 56
123 3 2 2015 57
124 1 1 2015 55
124 1 1 2014 56
124 2 1 2013 57
124 2 1 2012 58
124 1 2 2015 50
124 1 2 2014 50
124 2 2 2013 50
124 2 2 2012 50
Another dataset
NoOfRecordsToDisplay =3, ID =123
The data below should return:
CountGroup1: 0, Countgroup2: 1
as Entityid 1 has no target value for last 3 years (entityid 1 has a target value but for 2012)
as Entityid 2 has at one value for target years -2015 (entityid 3 has a target value but for 2010)
Id EntityId TypeofValue Year Value
123 1 1 2015 55
123 1 1 2014 56
123 1 1 2013 57
123 1 1 2012 58
123 1 2 2012 58
123 2 1 2015 55
123 2 1 2014 56
123 2 2 2015 55
123 2 2 2011 56
123 3 2 2010 57
Thank you so much for your help.
I have been trying to find this solution for a long time, I am not sure if Pivot will help
The question is different from the other one that I had posted as I am trying to create a group count based on entity groups.

return the last row that meets a condition in sql

I have two tables:
Meter
ID SerialNumber
=======================
1 ABC1
2 ABC2
3 ABC3
4 ABC4
5 ABC5
6 ABC6
RegisterLevelInformation
ID MeterID ReadValue Consumption PreviousReadDate ReadType
============================================================================
1 1 250 250 1 jan 2015 EST
2 1 550 300 1 feb 2015 ACT
3 1 1000 450 1 apr 2015 EST
4 2 350 350 1 jan 2015 EST
5 2 850 500 1 feb 2015 ACT
6 2 1000 150 1 apr 2015 ACT
7 3 1500 1500 1 jan 2015 EST
8 3 2500 1000 1 mar 2015 EST
9 3 5000 2500 4 apr 2015 EST
10 4 250 250 1 jan 2015 EST
11 4 550 300 1 feb 2015 ACT
12 4 1000 450 1 apr 2015 EST
13 5 350 350 1 jan 2015 ACT
14 5 850 500 1 feb 2015 ACT
15 5 1000 150 1 apr 2015 ACT
16 6 1500 1500 1 jan 2015 EST
17 6 2500 1000 1 mar 2015 EST
18 6 5000 2500 4 apr 2015 EST
I am trying to group by meter serial and return the last actual read date for each of the meters but I am unsure as to how to accomplish this. Here is the sql I have thus far:
select a.SerialNumber, ReadTypeCode, MAX(PreviousReadDate) from Meter as a
left join RegisterLevelInformation as b on a.MeterID = b.MeterID
where ReadType = 'ACT'
group by a.SerialNumber,b.ReadTypeCode, PreviousReadDate
order by a.SerialNumber
I can't seem to get the MAX function to take effect in returning only the latest actual reading row and it returns all dates and the same meter serial is displayed several times.
If I use the following sql:
select a.SerialNumber, count(*) from Meter as a
left join RegisterLevelInformation as b on a.MeterID = b.MeterID
group by a.SerialNumber
order by a.SerialNumber
then each serial is shown only once. Any help would be greatly appreciated.
Like #PaulGriffin said in his comment you need to remove PreviousReadDate column from your GROUP BY clause.
Why are you experiencing this behaviour?
Basically the partition you have chosen - (SerialNumber,ReadTypeCode,PreviousReadDate) for each distinct pair of those values prints you SerialNumber, ReadTypeCode, MAX(PreviousReadDate). Since you are applying a MAX() function to each row of the partition that includes this column you are simply using an aggregate function on one value - so the output of MAX() will be equal to the one without it.
What you wanted to achieve
Get MAX value of PreviousReadDate for every pair of (SerialNumber,ReadTypeCode). So this is what your GROUP BY clause should include.
select a.SerialNumber, ReadTypeCode, MAX(PreviousReadDate) from Meter as a
left join RegisterLevelInformation as b on a.MeterID = b.MeterID
where ReadType = 'ACT'
group by a.SerialNumber,b.ReadTypeCode
order by a.SerialNumber
Is the correct SQL query for what you want.
Difference example
ID MeterID ReadValue Consumption PreviousReadDate ReadType
============================================================================
1 1 250 250 1 jan 2015 EST
2 1 550 300 1 feb 2015 ACT
3 1 1000 450 1 apr 2015 EST
Here if you apply the query with grouping by 3 columns you would get result:
SerialNumber | ReadTypeCode | PreviousReadDate
ABC1 | EST | 1 jan 2015 -- which is MAX of 1 value (1 jan 2015)
ABC1 | ACT | 1 feb 2015
ABC1 | EST | 1 apr 2015
But instead when you only group by SerialNumber,ReadTypeCode it would yield result (considering the sample data that I posted):
SerialNumber | ReadTypeCode | PreviousReadDate
ABC1 | EST | 1 apr 2015 -- which is MAX of 2 values (1 jan 2015, 1 apr 2015)
ABC1 | ACT | 1 feb 2015 -- which is MAX of 1 value (because ReadTypeCode is different from the row above
Explanation of your second query
In this query - you are right indeed - each serial is shown only once.
select a.SerialNumber, count(*) from Meter as a
left join RegisterLevelInformation as b on a.MeterID = b.MeterID
group by a.SerialNumber
order by a.SerialNumber
But this query would produce you odd results you don't expect if you add grouping by more columns (which you have done in your first query - try it yourself).
You need to remove PreviousReadDate from your Group By clause.
This is what your query should look like:
select a.SerialNumber, ReadTypeCode, MAX(PreviousReadDate) from Meter as a
left join RegisterLevelInformation as b on a.MeterID = b.MeterID
where ReadType = 'ACT'
group by a.SerialNumber,b.ReadTypeCode
order by a.SerialNumber
To understand how the group by clause works when you mention multiple columns, follow this link: Using group by on multiple columns
You will understand what was wrong with your query and why it returns all dates and the same meter serial is displayed several times.
Good luck!
Kudos! :)