pandas filtering after a groupby with groupby-specifc filter conditions? - pandas

Question

I've seen a number of great solutions to "filter after a groupby" where the filter condition is fixed ("hey, group by name and then look for everyone over the age of 21", wherein 21 is fixed. I'm instead looking for a way to filter based on the results of a groupby.
example:
df = pd.DataFrame({'person':['Sue', 'Sue', 'Sue', 'Bill', 'Alfonso'],
'date': ['2019-01-01','2019-01-02', '2019-01-03','2019-02-01', '2019-03-01'],
'my_value': [5,10,20,10,5],
'my_other_value': [3,2,9,6,8]})
I want to be able to ask a question along the lines of:
"starting from the first time a person has a my_value of 10, tell me the mean of my_other_value for all successive records".
In the example, the first date that Sue has a my_value == 10 is 2019-01-02, so the her mean for my_other_value is (2+9)/2 = 5.5, which comes from 2019-01-02 and 2019-01-03. Bill only has one entry but it does have a my_value of 10 so his mean of my_other_value is 6. Alfonso, sadly, never has a my_value of 10, so he's not even included in the final tally
So, I started off with
df2 = df.query('my_value == 10').groupby('person').first().reset_index()
which gets my the first time a person has a my_value of 10. From this I know the person and the date it happened. So in English, I want to now filter those results for that person, so that I can do a .mean() but only including rows for that person >= the date I learned from the call to first(). I'm stuck, of course.
I kinda sorta was hoping something like this would work:
df3 = df.groupby('person').apply( lambda x: x['date'] >= df2['date']).mean()
but I know that can't really work because how does the lambda know to match up the correct person in the df.groupby() with the same person in the df2 grouping?
Another option was thinking "hey maybe there's a version of expanding() that can start with something other than the very first record"
Crossing my fingers that one of the above approaches is directionally correct and some hero shows up to say "oh, you are so close, just add in this little extra part!"

Answer 1

"oh, you are so close, just add in this little extra part!"
See below for little extra part.
df = pd.DataFrame({'person':['Sue', 'Sue', 'Sue', 'Bill', 'Alfonso'],
'date': ['2019-01-01','2019-01-02', '2019-01-03','2019-02-01', '2019-03-01'],
'my_value': [5,10,20,10,5],
'my_other_value': [3,2,9,6,8]})
df = df.sort_values(['person', 'date']).reset_index(drop=True)
>>> df
person date my_value my_other_value
0 Alfonso 2019-03-01 5 8
1 Bill 2019-02-01 10 6
2 Sue 2019-01-01 5 3
3 Sue 2019-01-02 10 2
4 Sue 2019-01-03 20 9
Find first date of my_value == 10
df2 = df.query('my_value == 10').groupby('person').first()['date'].reset_index()
df2 = df2.rename(columns={'date': 'first_date'})
>>> df2
person first_date
0 Bill 2019-02-01
1 Sue 2019-01-02
Merge the DataFrames
df_merged = pd.merge(df, df2, how='left', on=['person'])
>>> df_merged
person date my_value my_other_value first_date
0 Alfonso 2019-03-01 5 8 NaN
1 Bill 2019-02-01 10 6 2019-02-01
2 Sue 2019-01-01 5 3 2019-01-02
3 Sue 2019-01-02 10 2 2019-01-02
4 Sue 2019-01-03 20 9 2019-01-02
Calculate mean of my_other_value
grouped = df_merged[df_merged['date'] >= df_merged['first_date']].groupby('person')
>>> grouped['my_other_value'].mean()
person
Bill 6.0
Sue 5.5
Name: my_other_value, dtype: float64