Vlookup from the same pandas dataframe - pandas

Question

I have a hierarchical dataset that looks like this:
emp_id
emp_name
emp_manager
emp_org_lvl
1
John S
Bob A
1
2
Bob A
Paul P
2
3
Paul P
Charles Y
3
What I want to do is extend this table to have the emp_name for each manager going up the org chart. E.g.
emp_id
emp_name
emp_manager
emp_org_lvl
lvl2_name
lvl3_name
1
John S
Bob A
1
Paul P
Charles Y
In Excel, I would do a vlookup in column lvl2_name to see who Bob A's manager is e.g. something like this vlookup(c2,B:C,2,False). Using pandas, the direction seems to be to use Merge. The problem with this is that Merge seems to require two separate dataframes and you can't specify what column to return. Is there a better way than having a separate dataframe for each emp_org_lvl?
# Code to create table:
header = ['emp_id','emp_name','emp_manager','emp_org_lvl']
data = [[ 1,'John S' ,'Bob A', 1],[2, 'Bob A', 'Paul P', 2],[3, 'Paul P', 'Charles Y', 3]]
df = pd.DataFrame(data, columns=header)

Answer 1

You can try this:
# provide a lookup for employee to manager
manager_dict = dict(zip(df.emp_name, df.emp_manager))
# initialize the loop
levels_to_go_up = 3
employee_column_name = 'emp_manager'
# loop and keep adding columns to the dataframe
for i in range(2, levels_to_go_up + 1):
new_col_name = f'lvl{i}_name'
# create a new column by looking up employee_column_name's manager
df[new_col_name] = df[employee_column_name].map(manager_dict)
employee_column_name = new_col_name
>>>df
Out[67]:
emp_id emp_name emp_manager emp_org_lvl lvl2_name lvl3_name
0 1 John S Bob A 1 Paul P Charles Y
1 2 Bob A Paul P 2 Charles Y NaN
2 3 Paul P Charles Y 3 NaN NaN
Alternatively if you wanted to retrieve ALL managers in the tree, you could use a recursive function, and return the results as a list:
def retrieve_managers(name, manager_dict, manager_list=None):
if not manager_list:
manager_list = []
manager = manager_dict.get(name)
if manager:
manager_list.append(manager)
return retrieve_managers(manager, manager_dict, manager_list)
return manager_list
df['manager_list'] = df.emp_name.apply(lambda x: retrieve_managers(x, manager_dict))
>>> df
Out[71]:
emp_id emp_name emp_manager emp_org_lvl manager_list
0 1 John S Bob A 1 [Bob A, Paul P, Charles Y]
1 2 Bob A Paul P 2 [Paul P, Charles Y]
2 3 Paul P Charles Y 3 [Charles Y]
Finally, you can in fact self-join a dataframe while subselecting columns.
df = df.merge(df[['emp_name', 'emp_manager']], left_on='emp_manager', right_on='emp_name', suffixes=("", f"_joined"), how='left')
>>> df
Out[82]:
emp_id emp_name emp_manager emp_org_lvl emp_name_joined emp_manager_joined
0 1 John S Bob A 1 Bob A Paul P
1 2 Bob A Paul P 2 Paul P Charles Y
2 3 Paul P Charles Y 3 NaN NaN