Custom metrics with keras using oop - oop

Question

I am trying to pass custom metrics to keras.compile. I am also learning OOP and trying to apply it to machine learning. What I want to do is also track f1, precision and recall per epoch.
I can pass for example f1, recall and precision to the function as separate functions but not as a class with an init method.
Here is what I have been trying to do:
class Metrics:
def __init__(self, y_true, y_pred):
self.y_true = y_true
self.y_pred = y_pred
self.tp = K.sum(K.cast(y_true * y_pred, 'float'), axis=0)
self.fp = K.sum(K.cast((1 - y_true) * y_pred, 'float'), axis=0)
self.fn = K.sum(K.cast(y_true*(1 - y_pred), 'float'), axis=0)
def precision_score(self):
precision = self.tp / (self.tp + self.fp + K.epsilon())
return precision
def recall_score(self):
recall = self.tp / (self.tp + self.fn + K.epsilon())
return recall
def f1_score(self):
precision = precision_score(self.y_true, self.y_pred)
recall = recall_score(self.y_true, self.y_pred)
f1 = 2 * precision * recall / (precision + recall + K.epsilon())
f1 = tf.where(tf.is_nan(f1), tf.zeros_like(f1), f1)
f1 = K.mean(f1)
return f1
if __name__ == '__main__':
# Some images
train_generator = DataGenerator().create_data()
validation_generator = DataGenerator().create_data()
model = create_model(
input_shape = INPUT_SHAPE,
n_out = N_CLASSES)
model.compile(
loss = 'binary_crossentropy',
optimizer = Adam(0.03),
# This is the part in question:
metrics = ['acc', Metrics.f1_score, Metrics.recall_score, Metrics.precision_score]
)
history = model.fit_generator(
train_generator,
steps_per_epoch = 5,
epochs = 5,
validation_data = next(validation_generator),
validation_steps = 7,
verbose = 1
)
It also works without the def init part by passing in Metrics.f1_score but why is it not working with initializing?
If I pass in Metrics.f1_score I get:
TypeError: f1_score() takes 1 positional argument but 2 were given
If I pass in Metrics.f1_score() I get:
TypeError: f1_score() missing 1 required positional argument: 'self'
If I pass in Metrics().f1_score I get:
TypeError: __init__() missing 2 required positional arguments: 'y_true' and 'y_pred'
If I pass in Metrics().f1_score() I get:
TypeError: __init__() missing 2 required positional arguments: 'y_true' and 'y_pred'

Answer 1

I'm afraid you can't do that. Keras is expecting a function that takes 2 arguments (y_true, y_pred). You are passing a function that takes 1 argument (self), so it's never going to be compatible. You can't change this behaviour because it's keras that gets to define this interface. This is why you get all the errors:
TypeError: f1_score() takes 1 positional argument but 2 were given
You passed a function that takes 1 argument (self) but Keras passed 2 (y_true,y_pred).
TypeError: f1_score() missing 1 required positional argument: 'self'
By passing it with () you are not really passing the function but calling it. You called it without arguments, but it expects 1 (self).
TypeError: __init__() missing 2 required positional arguments: 'y_true' and 'y_pred'
You are instantiating a Metrics object with 0 arguments, but your constructor (init) expects 2: y_true and y_pred.
If you wanted to group all your custom metrics in a Class, they would have to be static methods. A static method cannot access instance variables, because it does not receive a self argument. It would look like this:
class Metrics:
#staticmethod
def precision_score(tp, fp):
precision = tp / (tp + fp + K.epsilon())
return precision
#staticmethod
def recall_score(tp, fn):
recall = tp / (tp + fn + K.epsilon())
return recall
#staticmethod
def f1_score(y_true,y_pred):
tp = K.sum(K.cast(y_true * y_pred, 'float'), axis=0)
fp = K.sum(K.cast((1 - y_true) * y_pred, 'float'), axis=0)
fn = K.sum(K.cast(y_true*(1 - y_pred), 'float'), axis=0)
precision = Metrics.precision_score(tp,fp)
recall = Metrics.recall_score(tp, fn)
f1 = 2 * precision * recall / (precision + recall + K.epsilon())
f1 = tf.where(tf.is_nan(f1), tf.zeros_like(f1), f1)
f1 = K.mean(f1)
return f1
This way you can pass Metrics.f1_score to Keras. There is almost no difference between this Metrics class and having all these 3 static methods as module level functions, it's just a different way to group related functionality together. There is even a third way: use nested functions and drop the class altogether:
def f1_score(y_true,y_pred):
def precision_score(tp, fp):
precision = tp / (tp + fp + K.epsilon())
return precision
def recall_score(tp, fn):
recall = tp / (tp + fn + K.epsilon())
return recall
tp = K.sum(K.cast(y_true * y_pred, 'float'), axis=0)
fp = K.sum(K.cast((1 - y_true) * y_pred, 'float'), axis=0)
fn = K.sum(K.cast(y_true*(1 - y_pred), 'float'), axis=0)
precision = precision_score(tp,fp)
recall = recall_score(tp, fn)
f1 = 2 * precision * recall / (precision + recall + K.epsilon())
f1 = tf.where(tf.is_nan(f1), tf.zeros_like(f1), f1)
f1 = K.mean(f1)
return f1