I was reading the this blog about focal loss. In the section Focal Loss Trick it says:
Facebook AI Research used is to initialize the bias term of the last
layer to some non-zero value such that the pt of positive samples is
small and the pt of negative samples is large. Concretely, they set
the bias term b=−log((1−π)/π). Here π is simply are variable instead
of the ordinary π. In their case, they set π=0.01, therefore b≫wx.
I want to do the same using tensorflow object detection api. Here, the focal loss is given by the following line in config file:
loss {
classification_loss {
weighted_sigmoid_focal {
alpha: 0.25
gamma: 2.0
}
} }
But I don't know how to set the bias term of the last layer to some non-zero value. How to achieve it in tensorflow ?
It's given by class_prediction_bias_init in the box_predictor. So, the config file will look something like this:
box_predictor {
weight_shared_convolutional_box_predictor {
class_prediction_bias_init: -1.99
}
}
Related
I have used PCA with the 'Sphereize data' option on the following page successfully: https://projector.tensorflow.org/
I wonder how to run the same computation locally using the TensorFlow API. I found the PCA documentation in the API documentation, but I am not sure if sphereizing the data is available somewhere in the API too?
The "sphereize data" option normalizes the data by shifting each point by the centroid and making unit norm.
Here is the code used in Tensorboard (in typescript):
normalize() {
// Compute the centroid of all data points.
let centroid = vector.centroid(this.points, (a) => a.vector);
if (centroid == null) {
throw Error('centroid should not be null');
}
// Shift all points by the centroid and make them unit norm.
for (let id = 0; id < this.points.length; ++id) {
let dataPoint = this.points[id];
dataPoint.vector = vector.sub(dataPoint.vector, centroid);
if (vector.norm2(dataPoint.vector) > 0) {
// If we take the unit norm of a vector of all 0s, we get a vector of
// all NaNs. We prevent that with a guard.
vector.unit(dataPoint.vector);
}
}
}
You can reproduce that normalization using the following python function:
def sphereize_data(x):
"""
x is a 2D Tensor of shape :(num_vectors, dim_vectors)
"""
centroids = tf.reduce_mean(x, axis=0, keepdims=True)
return tf.math.div_no_nan((x - centroids), tf.norm(x - centroids, axis=0, keepdims=True))
I've tried to search the answer in the documentation, the code and here but I had no luck.
I'd like to know what is the final number of images that are generated by the data augmentation using the object detection API in Tensorflow.
For the sake of clarity I'd put an example: let's say that I have a dataset with 2 classes, each one of then with 50 images originally. Then I apply this config:
data_augmentation_options {
ssd_random_crop {
}
}
data_augmentation_options {
random_rgb_to_gray {
}
}
data_augmentation_options {
random_distort_color {
}
}
data_augmentation_options {
ssd_random_crop_pad_fixed_aspect_ratio {
}
}
How can I know the final number of images generated to train my model? (if there is a way). BTW, I'm using model_main.py to train my model.
Thanks in advance.
In file inputs.py, it can be seen in function augment_input_fn that all data augmentation options are passed to preprocessor.preprocess method.
The details are all in file preprocessor.py, specifically in function preprocess:
for option in preprocess_options:
func, params = option
if func not in func_arg_map:
raise ValueError('The function %s does not exist in func_arg_map' %
(func.__name__))
arg_names = func_arg_map[func]
for a in arg_names:
if a is not None and a not in tensor_dict:
raise ValueError('The function %s requires argument %s' %
(func.__name__, a))
def get_arg(key):
return tensor_dict[key] if key is not None else None
args = [get_arg(a) for a in arg_names]
if (preprocess_vars_cache is not None and
'preprocess_vars_cache' in inspect.getargspec(func).args):
params['preprocess_vars_cache'] = preprocess_vars_cache
results = func(*args, **params)
if not isinstance(results, (list, tuple)):
results = (results,)
# Removes None args since the return values will not contain those.
arg_names = [arg_name for arg_name in arg_names if arg_name is not None]
for res, arg_name in zip(results, arg_names):
tensor_dict[arg_name] = res
Note that in the above code, arg_names contain all the original image names, that means each augmentation option will only be performed on the original images (not on those obtained after previous augmentation options).
Also in preprocessor.py, we can see each augmentation option will produce only an image of the same shape as the original image.
So as a result, in your case, four options and 100 original images, 400 augmented images will be added to tensor_dict.
I am trying to run TF object detection with mask rcnn, but it keeps dying on a node with 500GB of memory.
I updated the models/research/object_detection/trainer.py ConfigProto to
session_config = tf.ConfigProto(allow_soft_placement=True,
intra_op_parallelism_threads=1,
inter_op_parallelism_threads=1,
device_count = {'CPU': 1},
log_device_placement=False)
I updated the mask_rcnn_inception_resnet_v2_atrous_coco.config to
train_config: {
batch_queue_capacity: 500
num_batch_queue_threads: 8
prefetch_queue_capacity: 10
Updating the ConfigProto has had the best effect so far. I got it all the way to 30 steps before it died instead of 1. I'm reducing the values in the train_config by half for this run. I have also reduced the number of images and objects significantly.
Any other ideas?
500GB is a good amount of memory. I have had issues with running out of GPU memory, which is a separate constraint.
For TensorFlow v2, I have found the following useful:
1. Reduce batch_size to a small value
In the config file, set:
train_config: {
batch_size: 4
...
}
batch_size can be as low as 1.
2. Reduce the dimensions of resized images
In the config file, set the resizer height and width to a value lower than the default of 1024x1024.
model {
faster_rcnn {
number_of_stages: 3
num_classes: 1
image_resizer {
fixed_shape_resizer {
height: 256
width: 256
}
}
3. Don't train the Feature Detector
This only applies to Mask R-CNN, and is the most difficult change to implement. In the file research/object_detection/model_lib_v2.py, change the following code:
Current:
def eager_train_step(detection_model,
...
trainable_variables = detection_model.trainable_variables
gradients = tape.gradient(total_loss, trainable_variables)
if clip_gradients_value:
gradients, _ = tf.clip_by_global_norm(gradients, clip_gradients_value)
optimizer.apply_gradients(zip(gradients, trainable_variables))
New:
def eager_train_step(detection_model,
...
# Mask R-CNN variables to train -- not feature detector
trainable_variables = detection_model.trainable_variables
to_fine_tune = []
prefixes_to_train = ['FirstStageBoxPredictor',
'mask_rcnn_keras_box_predictor',
'RPNConv'
]
for var in trainable_variables:
if any([var.name.startswith(prefix) for prefix in prefixes_to_train]):
to_fine_tune.append(var)
gradients = tape.gradient(total_loss, to_fine_tune)
if clip_gradients_value:
gradients, _ = tf.clip_by_global_norm(gradients, clip_gradients_value)
optimizer.apply_gradients(zip(gradients, to_fine_tune))
There are implications to each of these changes. However, they may allow for a "good enough" result using scarce resources.
I had a similar issue. I managed to reduce memory consumption by another factor of 2.5x by setting the following values:
prefetch_size: 4
num_readers: 4
min_after_dequeue: 1
I am not sure which of them (maybe all?) are responsible for reducing the memory, (i did not test that) or how much their exact values influence the memory consumption, but you can easily try that out.
Some of the options that previously worked to reduce memory usage have been deprecated. From object_detection/protos/input_reader.proto:
optional uint32 queue_capacity = 3 [default=2000, deprecated=true];
optional uint32 min_after_dequeue = 4 [default=1000, deprecated=true];
optional uint32 prefetch_size = 13 [default = 512, deprecated=true];
optional uint32 num_parallel_map_calls = 14 [default = 64, deprecated=true];
As of today, num_parallel_batches appears to be the larges memory hog.
The *_input_reader messages my config file now looks like this:
train_input_reader: {
tf_record_input_reader {
input_path: "<DATASET_DIR>/tfrecords/train*.tfrecord"
}
label_map_path: "<DATASET_DIR>/label_map.pbtxt"
load_instance_masks: true
mask_type: PNG_MASKS
num_parallel_batches: 1
}
Mask RCNN training now uses ~50% less CPU memory than before (training on 775 x 522 images).
I was trying to use the object detection API of Tensorflow to train a model.
And I was using the sample config of faster rcnn resnet101 (https://github.com/tensorflow/models/blob/master/object_detection/samples/configs/faster_rcnn_resnet101_voc07.config).
The following code was part of the config file I didn't quite understand:
image_resizer {
keep_aspect_ratio_resizer {
min_dimension: 600
max_dimension: 1024
}
}
My questions were:
What was the exact meaning of min_dimension and max_dimension? Did it mean the size of input image would be resized to 600x1024 or 1024x600?
If I had different size of image and maybe some of them are relatively larger than 600x1024 (or 1024x600), could/should I increase the value of min_dimension and max_dimension?
The reason why I had such question was from this post:
TensorFlow Object Detection API Weird Behaviour
In this post, the author itself gave an answer to the question:
Then I decided to crop the input image and provide that as an input. Just to see if the results improve and it did!
It turns out that the dimensions of the input image were much larger than the 600 x 1024 that is accepted by the model. So, it was scaling down these images to 600 x 1024 which meant that the cigarette boxes were losing their details :)
It used the same config as I used.
And I was not sure if I could change these parameters if they were default or recommended setting to this special model, faster_rcnn_resnet101.
After some tests, I guess I find the answer. Please correct me if there is anything wrong.
In .config file:
image_resizer {
keep_aspect_ratio_resizer {
min_dimension: 600
max_dimension: 1024
}
}
According to the image resizer setting of 'object_detection/builders/image_resizer_builder.py'
if image_resizer_config.WhichOneof(
'image_resizer_oneof') == 'keep_aspect_ratio_resizer':
keep_aspect_ratio_config = image_resizer_config.keep_aspect_ratio_resizer
if not (keep_aspect_ratio_config.min_dimension
<= keep_aspect_ratio_config.max_dimension):
raise ValueError('min_dimension > max_dimension')
return functools.partial(
preprocessor.resize_to_range,
min_dimension=keep_aspect_ratio_config.min_dimension,
max_dimension=keep_aspect_ratio_config.max_dimension)
Then it tries to use 'resize_to_range' function of 'object_detection/core/preprocessor.py'
with tf.name_scope('ResizeToRange', values=[image, min_dimension]):
image_shape = tf.shape(image)
orig_height = tf.to_float(image_shape[0])
orig_width = tf.to_float(image_shape[1])
orig_min_dim = tf.minimum(orig_height, orig_width)
# Calculates the larger of the possible sizes
min_dimension = tf.constant(min_dimension, dtype=tf.float32)
large_scale_factor = min_dimension / orig_min_dim
# Scaling orig_(height|width) by large_scale_factor will make the smaller
# dimension equal to min_dimension, save for floating point rounding errors.
# For reasonably-sized images, taking the nearest integer will reliably
# eliminate this error.
large_height = tf.to_int32(tf.round(orig_height * large_scale_factor))
large_width = tf.to_int32(tf.round(orig_width * large_scale_factor))
large_size = tf.stack([large_height, large_width])
if max_dimension:
# Calculates the smaller of the possible sizes, use that if the larger
# is too big.
orig_max_dim = tf.maximum(orig_height, orig_width)
max_dimension = tf.constant(max_dimension, dtype=tf.float32)
small_scale_factor = max_dimension / orig_max_dim
# Scaling orig_(height|width) by small_scale_factor will make the larger
# dimension equal to max_dimension, save for floating point rounding
# errors. For reasonably-sized images, taking the nearest integer will
# reliably eliminate this error.
small_height = tf.to_int32(tf.round(orig_height * small_scale_factor))
small_width = tf.to_int32(tf.round(orig_width * small_scale_factor))
small_size = tf.stack([small_height, small_width])
new_size = tf.cond(
tf.to_float(tf.reduce_max(large_size)) > max_dimension,
lambda: small_size, lambda: large_size)
else:
new_size = large_size
new_image = tf.image.resize_images(image, new_size,
align_corners=align_corners)
From the above code, we can know if we have an image whose size is 800*1000. The size of final output image will be 600*750.
That is, this image resizer will always resize your input image according to the setting of 'min_dimension' and 'max_dimension'.
Reading the source code of the tensorflow I found the Sigmoid Gradient Computation is defined below.
Status SigmoidGrad(const AttrSlice& attrs, FunctionDef* g) {
// clang-format off
return GradForUnaryCwise(g, {
{{"y"}, "Sigmoid", {"x"}},
FDH::Const("const", 1.0f),
{{"one"}, "Cast", {"const"}, {{"SrcT", DT_FLOAT}, {"DstT", "$T"}}},
{{"a"}, "Sub", {"one", "y"}, {}, {"dy"}},
{{"b"}, "Mul", {"y", "a"}}, // y * (1 - y)
{{"dx"}, "Mul", {"dy", "b"}}, // dy * y * (1 - y)
});
// clang-format on
}
My question is, why do tensorflow recompute the Sigmoid's output for computing it's gradient. Isn't it stored in the op's context??
The code piece comes from github
Derivative of sigmoid can be calculated in terms of sigmoid:
So TF can reuse some of its previous notes to calculate the result.
Why do tensorflow recompute the Sigmoid's output for its gradient?
It does not — that's the beauty of frameworks such as tensorflow. Tensorflow will reuse nodes that can be reused. So the sigmoid node in the gradient will automagically reuse the sigmoid computation.
Note that node reuse is much more powerful than a per-op optimization, because the optimization can happen anywhere in the graph, between different operations.