What do each for the following losses mean? (in the TensorFlow Object detection API, while training FasterRCNN based models)
Loss/BoxClassifierLoss/classification_loss/mul_1
Loss/BoxClassifierLoss/localization_loss/mul_1
Loss/RPNLoss/localization_loss/mul_1
Loss/RPNLoss/objectness_loss/mul_1
clone_loss_1
The losses for the Region Proposal Network:
Loss/RPNLoss/localization_loss/mul_1: Localization Loss or the Loss of the Bounding Box regressor for the RPN
Loss/RPNLoss/objectness_loss/mul_1: Loss of the Classifier that classifies if a bounding box is an object of interest or background
The losses for the Final Classifier:
Loss/BoxClassifierLoss/classification_loss/mul_1: Loss for the classification of detected objects into various classes: Cat, Dog, Airplane etc
Loss/BoxClassifierLoss/localization_loss/mul_1: Localization Loss or the Loss of the Bounding Box regressor
clone_loss_1 is relevant only if you train on multiple GPUs: Tensorflow would create a clone of the model to train on each GPU and report the loss on each clone. If you are training the model on a single GPU/CPU, then you will just see clone_loss_1, which is the same as TotalLoss.
The other losses are as described in Rohit's answer.
There are four losses that you will encounter if you are using the faster rcnn network
1.RPN LOSS/LOCALIZATION LOSS
If we see the architecture of faster rcnn we will be having the cnn for getting the regoin proposals. For getting the region proposals from the feature map we have the loss functions . This is the localization loss for bounding boxes for the anchors generated.'
2.RPN LOSS/OBJECTNESS LOSS
This is also when we are extracting the region proposals whether the object is present in the anchorbox or not.
3.BOX_CLASSIFIERLOSS/CLASSIFICATION_LOSS
This is at the final layer to which class the object belongs to whether dog or cat??
4.BOX_CLASSIFIERLOSS/LOCALIZATION_LOSS
This is also at the final layer for the bounding boxes of the object. (coordinates for dog and cat)
Related
I have dataset of 3 types of cars: CarA, CarB, CarC.
All cars have the same shape but different color and/or logo.
I have trained an object detection model using tensor flow object detection api's with the base model as SSD ResNet50 V1 FPN 640x640 (RetinaNet50). Training was performed for 10000 steps and the training was stopped when loss reached 0.15
When I test the model, it classifies any car with the same shape as all 3 CarA, CarB, CarC. Is the model not able to distinguish based on color/logo and only works based on shape? I want to ask whether this color aspect can be handled better using any specific base model?
I am using Keras with a tensorflow backend to train some CNNs for semantic segmentation of biomedical images. I am trying to weight every pixel in my input images during training and believe I am doing so with the data generator I am passing to model.fit.
However, I am a little confused about the meaning of 'sample_weights' vs. 'sample_weight' in the documentation for model.fit.
'sample_weights' is the third optional output from your dataset or image generator - i.e. the output of the generator can either be the tuple (inputs, targets) or the tuple (inputs, targets, sample_weights). I believe this lets me create a mask that weights my samples pixel-by-pixels, but this isn't super clear from the documentation.
'sample_weight' is a separate field that seems to be pretty clearly defined as a weight you can give to every sample. If I understand, this would allow me to give more or less weight to particular images in my training set.
Do I have this right? Thanks.
The labels for YOLO is like [class, x , y, width, height] . Since the dataset is very large, is there any shortcut to generate the labels for YOLO, or we have to hardcode them through measurement?
Method 1: Using Pre-trained YOLOv4 models.
YOLOv4 models were pre-trained on COCO dataset. So, if your object(s) can be found in this list, then, you can use the pre-trained weights to pseudo-label your object(s).
To process a list of images data/new_train.txt and save results of detection in Yolo training format for each image as label <image_name>.txt, use: darknet.exe detector test cfg/coco.data cfg/yolov4.cfg yolov4.weights -thresh 0.25 -dont_show -save_labels < data/new_train.txt
Method 2: Using Other Pre-trained Models. It's the same concept. Use other pre-trained models to detect your object (as long as they have trained their models on your object), then export/convert the labels to YOLO format.
Method 3: Use hand-crafted feature descriptors. Examples are shape detection, color-based detection, etc.
Method 4: Manual labelling. If everything else fails, do the labelling yourself or hire some data labelling services. Here's a list of tools that you can use if you want to label them yourself.
In a segmentation task I wanted to have my model to have two outputs because I implemented weight maps as suggested in the original U-net paper https://arxiv.org/pdf/1505.04597.pdf.
As per the suggestion I created weightmaps concentrating some of the ground truth mask to have higher weights. Now I have a model with.
weightmap=layers.Lambda(lambda x:x)(weight_map) # A non trainable layer to output this as tensor for loss function
Model=model(inputs=[input,weight_map], outputs=[output,weightmap]
Now I need to compute binary cross entropy loss for the following model
def custom_loss(target,outputs):
loss=K.binary_crossentropy(target,outputs[0]) #ouputs[0] should be the model output
loss=loss*outputs[1] #outputs[1] should be weightmaps
return loss
This output[0] and output[1] slicing of output tensor from model doesnt work.
Is there anything I can do to implement the following with both outputs of model in a single loss function?
I am trying to convert my CNN written with tensorflow layers to use the keras api in tensorflow (I am using the keras api provided by TF 1.x), and am having issue writing a custom loss function, to train the model.
According to this guide, when defining a loss function it expects the arguments (y_true, y_pred)
https://www.tensorflow.org/guide/keras/train_and_evaluate#custom_losses
def basic_loss_function(y_true, y_pred):
return ...
However, in every example I have seen, y_true is somehow directly related to the model (in the simple case it is the output of the network). In my problem, this is not the case. How do implement this if my loss function depends on some training data that is unrelated to the tensors of the model?
To be concrete, here is my problem:
I am trying to learn an image embedding trained on pairs of images. My training data includes image pairs and annotations of matching points between the image pairs (image coordinates). The input feature is only the image pairs, and the network is trained in a siamese configuration.
I am able to implement this successfully with tensorflow layers and train it sucesfully with tensorflow estimators.
My current implementations builds a tf Dataset from a large database of tf Records, where the features is a dictionary containing the images and arrays of matching points. Before I could easily feed these arrays of image coordinates to the loss function, but here it is unclear how to do so.
There is a hack I often use that is to calculate the loss within the model, by means of Lambda layers. (When the loss is independent from the true data, for instance, and the model doesn't really have an output to be compared)
In a functional API model:
def loss_calc(x):
loss_input_1, loss_input_2 = x #arbirtray inputs, you choose
#according to what you gave to the Lambda layer
#here you use some external data that doesn't relate to the samples
externalData = K.constant(external_numpy_data)
#calculate the loss
return the loss
Using the outputs of the model itself (the tensor(s) that are used in your loss)
loss = Lambda(loss_calc)([model_output_1, model_output_2])
Create the model outputting the loss instead of the outputs:
model = Model(inputs, loss)
Create a dummy keras loss function for compilation:
def dummy_loss(y_true, y_pred):
return y_pred #where y_pred is the loss itself, the output of the model above
model.compile(loss = dummy_loss, ....)
Use any dummy array correctly sized regarding number of samples for training, it will be ignored:
model.fit(your_inputs, np.zeros((number_of_samples,)), ...)
Another way of doing it, is using a custom training loop.
This is much more work, though.
Although you're using TF1, you can still turn eager execution on at the very beginning of your code and do stuff like it's done in TF2. (tf.enable_eager_execution())
Follow the tutorial for custom training loops: https://www.tensorflow.org/tutorials/customization/custom_training_walkthrough
Here, you calculate the gradients yourself, of any result regarding whatever you want. This means you don't need to follow Keras standards of training.
Finally, you can use the approach you suggested of model.add_loss.
In this case, you calculate the loss exaclty the same way I did in the first answer. And pass this loss tensor to add_loss.
You can probably compile a model with loss=None then (not sure), because you're going to use other losses, not the standard one.
In this case, your model's output will probably be None too, and you should fit with y=None.