I am training a U-Net image segmentation on whole slide pathology images. I was wondering how can I handle un-annotated areas? I am working with huge tissues and it’s impossible to annotate all or the vast majority of the tissue, so I have annotations from a pathologist who has annotated selected tissue structures of interest to us. That means that in many tiles I’m generating, there is a segment that’s not annotated.
Would it affect the U-Net negatively by indirectly indicating that the un-annotated area is negative to one category or another, although it’s not negative? How do I handle this important case? Does it make sense to mask the image to only the annotated parts, such that un-annotated regions are black?
Thanks
One way to deal with this is to use a weighted loss function where you simply assign a weight of zero to the class that you don't want to include. Essentially, you're treating the unannotated area as an additional class that doesn't contribute to the loss function. You can find the GitHub repo to a fully functional Keras implementation here.
Specifically, I would use a weighted categorical cross-entropy loss function. You can find an implementation for Keras here:
from keras import backend as K
def weighted_categorical_crossentropy(weights):
"""
A weighted version of keras.objectives.categorical_crossentropy
Variables:
weights: numpy array of shape (C,) where C is the number of classes
Usage:
weights = np.array([0.5,2,10]) # Class one at 0.5, class 2 twice the normal weights, class 3 10x.
loss = weighted_categorical_crossentropy(weights)
model.compile(loss=loss,optimizer='adam')
"""
weights = K.variable(weights)
def loss(y_true, y_pred):
# scale predictions so that the class probas of each sample sum to 1
y_pred /= K.sum(y_pred, axis=-1, keepdims=True)
# clip to prevent NaN's and Inf's
y_pred = K.clip(y_pred, K.epsilon(), 1 - K.epsilon())
# calc
loss = y_true * K.log(y_pred) * weights
loss = -K.sum(loss, -1)
return loss
return loss
And you can then compile your model for training like this:
model.compile(optimizer='adam', loss=weighted_categorical_crossentropy(np.array([background_weight, foreground_weight, 0])), metrics='accuracy')
Related
I am trying to implement a GAN called the SimGAN proposed by Apple researchers. The SimGAN is used to refine labelled synthetic images so that they look more like the unlabelled real images.
The link to the paper can be found on arXiv here.
In the paper, the loss function of the combined model, which comprises the generator and the discriminator, has a self-regularization component in the form of an L1 loss that penalizes too great a difference between the synthetic images and the images after refinement. In other words, the refinement should not be too drastic.
I would like to know how I can implement this self-regularization loss in Keras. Here is what I tried:
def self_regularization_loss(refined_images, syn_images):
def l1loss(y_true, y_pred):
return keras.metrics.mean_absolute_error(refined_images, syn_images)
return l1loss
However, I do not think I can compile the model in the way below as the batches of refined and synthetic images change during training time.
model.compile(loss=[self_regularization_loss(current_batch_of_refined, current_batch_of_synthetic),
local_adversarial_loss],
optimizer=opt)
What is the way to implement this loss?
Trying using the tf.function decorator and tf.GradientTape():
#tf.function
def train_step(model, batch):
with tf.GradientTape() as tape:
refined_images, syn_images = batch
loss = self_regularization_loss(model, refined_images, syn_images)
gradients = tape.gradient(loss, model.trainable_variables)
self.optimizer.apply_gradients(zip(gradients, model.trainable_variables))
your training loop can look something like:
for image_batch in dataset:
train_step(model, image_batch)
Here it is assumed that model is of type tf.keras.Model. More details to the model class can be found here. Note that model is also passed to self_regularization_loss. In this function your model recieves both images as inputs and then gives you the respective output. Then you calculate your loss.
I want to train a Neural Network for a classification task in Keras using a TensorFlow backend with a custom loss function. In my loss, I want to give different weights to different training examples. I have some datapoints I consider important and some I do not consider as important. I want my loss function to take this into account and punish errors in important examples more than in less important ones.
I have already built my model:
input = tf.keras.Input(shape=(16,))
hidden_layer_1 = tf.keras.layers.Dense(5, kernel_initializer='glorot_uniform', activation='relu')(input)
output = tf.keras.layers.Dense(1, kernel_initializer='normal', activation='softmax')(hidden_layer_1)
model = tf.keras.Model(input, output)
model.compile(loss=custom_loss(input), optimizer='adam', run_eagerly=True, metrics = [tf.keras.metrics.Accuracy(), 'acc'])
and the currrent state of my loss function is:
def custom_loss(input):
def loss(y_true, y_pred):
return ...
return loss
I'm struggling with implementing the loss function in the way I explained above, mainly because I don't exactly know what input, y_pred and y_true are (KerasTensors, I know - but what is the content? And is it for one training example only or for the whole batch?). I'd appreciate help with
printing out the values of input, y_true and y_pred
converting the input value to a numpy ndarray ([1,3,7] for example) so I can use the array to look up my weight for this specific training data point
once I have my weigth as a number (0.5 for example), how do I implement the computation of the loss function in Keras? My loss for one training exaple should be 0 if the classification was correct and weight if it was incorrect.
I have written my custom training loop using tf.GradientTape(). My data has 2 classes. The classes are not balanced; class1 data contributes almost 80% and class2 contributes remaining 20%. Therefore in order to remove this imbalance I was trying to write custom loss function which will take into account this imbalance and apply the corresponding class weights and calculate the loss. i.e. I want to use the class_weights = [0.2, 0.8]. I am not able to find similar examples.
However all the examples I am seeing are using model.fit approach where its easier to pass the class_weights. I am not able to find out the example which uses class_weights with custom training loop using tf.GradientTape.
I did go through the suggestions of using sample_weight, however I don't have the data where in I can specify the weights for samples, therefore my preference is to use class weight.
I am using BinaryCrossentropy loss as loss function but I want to change the loss based on the class_weights. That's where I am stuck, how to tell BinaryCrossentropy to consider the class_weights.
Is my approach of using custom loss function correct or there is better way to make use of class_weights while training with custom training loop (not using model.fit)?
you can write your own loss function. in that loss function call BinaryCrossentropy and then multiply the result in the weight you want and return that
Here's an implementation that should work for n classes instead of just 2.
For your example of 80:20 split, calculate weights as below (assuming 100 samples in total).
Weight calculation (ref: Handling Class Imbalance: TensorFlow):
weight_class_0 = (1/count_for_class_0) * (total_samples / num_classes) # (80%) 0.625
weight_class_1 = (1/count_for_class_1) * (total_samples / num_classes) # (20%) 2.5
class_wts = tf.constant([weight_class_0, weight_class_1])
Loss function: Requires labels to be sparse and logits unscaled (no activations applied).
# Example logits=[[-3.2, 2.0], [1.2, 0.5], ...], (sparse)labels=[0, 1, ...]
def weighted_sparse_categorical_crossentropy(labels, logits, weights):
loss = tf.nn.sparse_softmax_cross_entropy_with_logits(labels, logits)
class_weights = tf.gather(weights, labels)
return tf.reduce_mean(class_weights * loss)
You can supply this loss function to custom training loops.
I classify MNIST digits and I want to sample the probabilities (not the latent function) for each class on multiple many times. However, gp.predict_y gives the probabilities just for one case.
Thus I take f_samples = gp.predict_f_samples which returns numerous examples from the underlying latent function.
Now, how to 'squeeze' the f_samples through the robust_max likelihood?
Code for my gp:
kernel = gpflow.kernels.Matern52(input_dim=128, ARD=ARD, active_dims=np.arange(128))\
+ gpflow.kernels.White(input_dim=128, active_dims=np.arange(128))
# Robustmax Multiclass Likelihood
invlink = gpflow.likelihoods.RobustMax(10) # Robustmax inverse link function
likelihood = gpflow.likelihoods.MultiClass(10, invlink=invlink) # Multiclass likelihood
Z = x_train[::5].copy() # inducing inputs
gp = gpflow.models.SVGP(x_train, y_train, num_latent=10,
kern=kernel, Z=Z, likelihood=likelihood,
whiten=True, q_diag=True)
GPflow version: 1.5.1
Once you've sampled you're no longer working with a probability distribution - you have actual values for each of your 10 latent functions. To convert a sample to probabilities over the classes you can just apply the RobustMax function (probability 1-epsilon for the largest latent function, epsilon/9 for all the others) to the 10 values you get. E.g.
eps = 0.001
f_samples = gp.predict_f_samples(x_test, num_samples)
largests = np.argmax(f_samples , axis = 2)
prob_samples = (np.eye(10)[largests]*(1-eps-eps/9)+eps/9)
Note that the probabilities you get will all be 0.999 on one class and 0.0001 on all the others - that's what RobustMax is. If you're intending to average over your samples, you probably just want to call gp.predict_y(), which actually integrates the RobustMax over the probability distribution and can give you some smoother class probabilities if the latent means are close.
I'm currently working on using tensorflow to adress a multi-class segmentation problem on a SegNet Architecture
My classes are heavily unbalanced and thus I need to integrate the median frequency balancing (using weights on classes on loss calculation). I use the following tip (based on this post) to apply Softmax. I need help to extend it in order to add the weights, I'm not sure how to do it. Current implementation:
reshaped_logits = tf.reshape(logits, [-1, nClass])
reshaped_labels = tf.reshape(labels, [-1])
loss = tf.nn.sparse_softmax_cross_entropy_with_logits(reshaped_logits, reshaped_labels)
My idea would be:
To split the logits tensor in nClass tensor,
Apply the softmax on each independently,
Weight them with median frequency balancing
Finally, summing the weighted losses.
Would that be the right approach?
Thanks
You can find the code to do that here
def _compute_cross_entropy_mean(class_weights, labels, softmax):
cross_entropy = -tf.reduce_sum(tf.multiply(labels * tf.log(softmax), class_weights),
reduction_indices=[1])
cross_entropy_mean = tf.reduce_mean(cross_entropy,
name='xentropy_mean')
return cross_entropy_mean
where head is your class weighing matrix.