Compare multiple records in pandas - pandas

Question

Trying to compare df1 and df2 for 'Cntr No' AND either the value in any of the column of df2 ['Labour Cost', Material Cost', 'Amount in Estimate Currency'] must be matched with df1's Total.
For example df1 OOLU 3868088 is matched with df2 OOLU 3868088 AND the Total value of df1 "28" is matched with df2's "Labour Cost" value of '28'.
df:
df1 = pd.DataFrame({'Cntr No': ['OOLU 3868088','OOLU 3868088','OOLU 3868088','TRIU 0625840','TRIU 0625840','TRIU 0625840','TRIU 1234567','OOLU 6232016','OOLU 0981231','OOLU 1212444'],
'Total': [12,28,48,119,82.5,11.0,18.0,11.0,13.0,10.0]})
df2 = pd.DataFrame({'Cntr No': ['OOLU 3868088','OOLU 3868088','OOLU 3868088','TRIU 0625840','TRIU 0625840','TRIU 0625840','TRIU 1234567'],
'Labour Cost': [0.0,0.0,28.0,0.0,54.0,0.0,0.0],
'Material Cost':[0.00,12.0,58.91,82.5,54.0,0.0,16.0],
'Amount in Estimate Currency':[48.00,12.00,87.81,82.5,119.0,12.0,16.0]})
Expected output:
Cntr No Total Tally_with_df2
0 OOLU 3868088 12.0 Yes
1 OOLU 3868088 28.0 Yes
2 OOLU 3868088 48.0 Yes
3 TRIU 0625840 119.0 Yes
4 TRIU 0625840 82.5 Yes
5 TRIU 0625840 11.0 No
6 TRIU 1234567 18.0 No
Used code: this is the below code I tried on but doesn't able to achieve my requirements
cols = ['Labour Cost', 'Material Cost', 'Amount in Estimate Currency']
d = {k: set(v.values()) for k, v in \
df_co.set_index('Cntr No')[cols].to_dict(orient='index').items()}
df['Tally'] = [j in d.get(i, set()) for i, j in zip(df['Cntr No'], df['Total'])]
df['Tally'] = df['Tally'].map({True: 'Yes', False: 'No'})
df1:
Cntr No object
Serviced By object
Location object
WO No object
WASH - CHEMICAL float64
PTI - CHILL float64
WASHING CONTAINER AGENT float64
WASH - CHEMICAL AGENT float64
WASHING CONTAINER -AGENT float64
BUNDLING/UNBUNDLING OF FR float64
PTI - AUTO float64
PTI float64
Struct Repair - Labour float64
Struct Repair - Material float64
Machy Repair - Labour float64
Total float64
Vendor object
Sz object
Ty object
CO object
WO Date object
WO ID object
df2:
Cntr No object
Equipment Size/type Group Code object
Labour Cost float64
Material Cost float64
Amount in Estimate Currency float64
Remarks object

Answer 1

IIUC, we can create a groupby data from df2 for every unique Cntr No.
## this is grouped data
to_remove = df2.select_dtypes(['object']).columns.values.tolist()
df3 = (df2
.groupby('Cntr No')
.apply(lambda df: set(np.concatenate(df.loc[:, df.columns.difference(to_remove)].values))))
## df3 looks like this - using set for faster speed
print(df3)
Cntr No
OOLU 3868088 {0.0, 12.0, 48.0, 87.81, 58.91, 28.0}
TRIU 0625840 {0.0, 12.0, 82.5, 54.0, 119.0}
TRIU 1234567 {16.0, 0.0}
## this function ensures all cases are handles
def get_value(x, data):
if x['Cntr No'] not in data.index:
return 'Not Found'
else:
if x['Total'] in data[x['Cntr No']]:
return 'Yes'
else:
return 'No'
## next we do a simple look-up
df1['Tally_with_df2'] = df1.apply(lambda x: get_value(x, df3), axis=1)
print(df1)
Cntr No Total Tally_with_df2
0 OOLU 3868088 12.0 Yes
1 OOLU 3868088 28.0 Yes
2 OOLU 3868088 48.0 Yes
3 TRIU 0625840 119.0 Yes
4 TRIU 0625840 82.5 Yes
5 TRIU 0625840 11.0 No
6 TRIU 1234567 18.0 No