difference between rows in a multi-level dataframe - pandas

Question

I am looking to find difference between two rows in a multi-level data by iterating values within certain class and been trying different techniques by reading tutorials as I am still new to python/pandas power.
what I am trying to do is to find out difference between scores of teacher and each student in certain class.
dataframe:
Class, Name ,Reference, stats
X ,SHE ,student, 30
X ,GHE ,student, 20
X ,GMK ,student ,10
X ,JKO ,teacher ,50
Y ,HHH ,student ,20
Y ,KLP ,teacher ,30
Output:
Class,teacher, student, difference
X, JKO, SHE,20
X, JKO,GHE, 30
X, JKO, GMK, 40
Y, KLP, HHH, 10
Can anyone help me by guiding me towards the right direction? there can be more than 1 teachers in a class.
Thank you

Answer 1

Just split your dataset into two data frames, one for students one for teachers. Then merge.
students = df[df.Reference == 'student'][['Class','Name','stats']]
teachers = df[df.Reference == 'teacher'][['Class','Name','stats']]
new_df = students.merge(teachers, on='Class', suffixes=('_student','_teacher'))
new_df['difference'] = new_df.stats_teacher - new_df.stats_student
print(new_df)
Class Name_student stats_student Name_teacher stats_teacher difference
0 X SHE 30 JKO 50 20
1 X GHE 20 JKO 50 30
2 X GMK 10 JKO 50 40
3 Y HHH 20 KLP 30 10

Answer 2

Use:
print (df)
Class Name Reference stats
0 X SHE student 30
1 X GHE student 20
2 X GMK student 10
3 X JKO teacher 50
4 X ABC teacher 100 <-added one new row for general data
5 Y HHH student 20
6 Y KLP teacher 30
df = (df.query("Reference == 'teacher'")
.merge(df.query("Reference == 'student'"), on='Class', suffixes=('_t','_s'))
.assign(difference=lambda x: x['stats_t'] - x['stats_s'])
.drop(['Reference_s','Reference_t','stats_s','stats_t'], axis=1)
.rename(columns={'Name_s':'student','Name_t':'teacher'})
)
print (df)
Class teacher student difference
0 X JKO SHE 20
1 X JKO GHE 30
2 X JKO GMK 40
3 X ABC SHE 70
4 X ABC GHE 80
5 X ABC GMK 90
6 Y KLP HHH 10
Explanation:
Filter DataFrame by query with student and teacher rows
Then merge by column Class for all combinations in per groups
Then assign new column with subtract
Remove unnecessary columns by drop
Last rename columns

Answer 3

Below is the code with many for loops. So there should an optimized solution than this. (Later i will try to update this solution in a better way)
import pandas as pd
df = pd.read_csv("student.csv")
ref = df[df['Reference'] == 'teacher'].index.values.astype(int)
df['TeacherName'] = 'NA'
df['Difference'] = 0
for i in range(len(ref)):
if(i == 0):
for j in range(ref[i]+1):
df['TeacherName'][j] = df['Name'][ref[i]]
df['Difference'][j] = df['stats'][ref[i]] - df['stats'][j]
else:
for j in range(ref[i-1]+1, ref[i]):
df['TeacherName'][j] = df['Name'][ref[i]]
df['Difference'][j] = df['stats'][ref[i]] - df['stats'][j]
df[~ df.index.isin(ref)]
I'm getting the index of row for every occurrence of df['Reference'] == 'teacher' into a list named ref which will be dropped from df after loop statements.