Custom loss function for sequential output in Keras - tensorflow

Question

I need to write a custom loss for my keras model. As I need to write the function using Keras functions for auto-backpropagation, I am not sure how I will implement this, as this might require some looping operations -
Target[1*300] - [...0 0 0 1 0 0 0 0 0 1 0 0 0...]
Output[1*300] - [...0 0 1 0 0 0 0 0 0 0 1 0 0...]
What I need is that while calculating loss I don't need a exact match.
Even if my output has a discrepancy of +/- three places. I want it to mark it to consider this as a correct prediction.
For example, both of these should be considered as the right predictions -
Output[1*300] - [...0 0 1 0 0 0 0 0 0 0 1 0 0...]
Output[1*300] - [...0 1 0 0 0 0 0 0 0 0 0 1 0...]
The code which I have written till now -
import tensorflow as tf
tar = tf.placeholder(tf.float32, shape=(1, 10))
tar_unpacked = tf.unstack(tar)
pred = tf.placeholder(tf.float32, shape=(1, 10))
pred_unpacked = tf.unstack(pred)
for t in tar_unpacked:
result_tensor = tf.equal(t,1)
tar_ind = tf.where(result_tensor)
with tf.Session() as sess:
print(sess.run([tar_ind], feed_dict={tar:np.asarray([[0, 0,1, 0,0,0,1,0,0,0]]),pred:np.asarray([[0, 0,1, 0,0,0,1,0,0,0]])}))
Now what I want to do next is generate valid indexes by adding each from
[-3,-2,-1,0,1,2,3]
to elements in tar_ind and then compare the indexes with pred_unstacked.
My naive loss would be 1 - (NUM_MATCHED/TOTAL)
But the problem is that tar_ind is a variably sized tensor, and I cannot loop over it for the next operation.
Update-1.
As suggested by #user36624, I tried the alternate approach of having tf.py_func which gives the updated y_pred and then I used the updated ones for binary cross-entropy.
As I have implemented the function using py_func, It is giving me error as ValueError: An operation hasNonefor the gradient. Please make sure that all of your ops have a gradient defined (i.e., are differentiable). Common ops without gradient: K.argmax, K.round, K.eval.
Also as he suggested that I need to manually stop gradients which I don't know how to do?
def specificity_loss_wrapper():
def specificity_loss(y_true, y_pred):
y_pred = tf.py_func(best_match,[y_true,y_pred],(tf.float32))
y_pred = tf.stop_gradient(y_pred)
y_pred.set_shape(y_true.get_shape())
return K.binary_crossentropy(y_true, y_pred)
return specificity_loss
spec_loss = specificity_loss_wrapper()
and
...
model.compile(loss=spec_loss, optimizer='adam', metrics=['accuracy'])
...
In my understanding, binary_crossentropy should be differentiable.
Thanks

Answer 1

What you are suggesting is to compute
1. offsets = compute_index_offsets( y_true, y_pred )
2. loss = 1 - num(offsets <= 3)/total
I suggest to solve it in an alternative way.
1. y_true_mod = DP_best_match( y_true, y_pred )
2. loss = 1 - num(y_true_mod==y_pred)/total
The advantage of modifying y_true is that it is equivalent to providing a new target value, and thus it is not a part of the model graph optimization or the loss computation.
What DP_best_match( y_true, y_pred ) should do is to modify y_true according to y_pred,
e.g. given
y_true[1*300] - [...0 0 0 1 0 0 0 0 0 1 0 0 0...]
y_pred[1*300] - [...0 0 1 0 0 0 0 0 0 0 1 0 0...]
then DP_best_match( y_true, y_pred ) should give the new target
y_true_mod[1*300] - [...0 0 1 0 0 0 0 0 0 0 1 0 0...]
Note, DP_best_match( y_true, y_pred ) is aiming to modify y_true to best match y_pred, so it is deterministic and nothing to optimize. Thus, no need to have backpropagation. This means you need to manually stop gradients if you implement DP_best_match( y_true, y_pred ) in tf. Otherwise, you can implement it in numpy and wrap the function via tf.py_func, which might be easier to implement.
Final remark, you should make sure the proposed loss function makes sense. For me, it makes more sense to use binary_crossentropy or mse after finding the best y_true_mod.