model_main.py faster-rcnn CUDA_ERROR_OUT_OF_MEMORY - tensorflow

Question

Description:
I am able to train faster-rcnn model with legacy/train.py, but it runs into problem as below when I try to use model_main.py to train with the same config setting.
Image resolution: 1920x1080
tensorflow/stream_executor/cuda/cuda_driver.cc:890] failed to alloc 8589934592 bytes on host: CUDA_ERROR_OUT_OF_MEMORY: out of memory
.\tensorflow/core/common_runtime/gpu/gpu_host_allocator.h:44] could not allocate pinned host memory of size: 8589934592
tensorflow/core/common_runtime/bfc_allocator.cc:764] Bin (256): Total Chunks: 4753, Chunks in use: 4753. 1.16MiB allocated for chunks. 1.16MiB in use in bin. 144.3KiB client-requested in use in bin.
tensorflow/core/common_runtime/bfc_allocator.cc:800] InUse at 0000000203800000 next 1 of size 256
What I have tried:
Set batch size to 1
use memory growing
config = tf.ConfigProto()
config.gpu_options.allow_growth = True
session = tf.Session(config=config)
or
session_config = tf.ConfigProto()
session_config.gpu_options.allow_growth = True
config = tf.estimator.RunConfig(model_dir=FLAGS.model_dir, session_config=session_config, log_step_count_steps=10, save_summary_steps=20, keep_checkpoint_max=20, save_checkpoints_steps=100)
don't allocate whole of your GPU memory
config = tf.ConfigProto()
config.gpu_options.per_process_gpu_memory_fraction = 0.6
session = tf.Session(config=config)
or
session_config = tf.ConfigProto()
session_config.gpu_options.per_process_gpu_memory_fraction = 0.6
config = tf.estimator.RunConfig(model_dir=FLAGS.model_dir,
session_config=session_config, log_step_count_steps=10,
save_summary_steps=20, keep_checkpoint_max=20,
save_checkpoints_steps=100)
TensorFlow CUDA_ERROR_OUT_OF_MEMORY
Setting of queue_capacity, min_after_dequeue, num_readers, batch_queue_capacity, num_batch_queue_threads, prefetch_queue_capacity
Out Of Memory when training on Big Images
reduce min_dimension, max_dimension to 270, 480
None of these work for me.
Environment:
OS Platform and Distribution: Win 10 pro version: 1909
TensorFlow installed from: pip tensorflow-gpu
TensorFlow version 1.14
object-detection: 0.1 CUDA/cuDNN version: Cuda 10.0, Cudnn 10.0
GPU model and memory: NVIDIA GeForce RTX 2070 SUPER, Memory 8 G
system memory: 32G
My config:
# Faster R-CNN with Inception v2, configured for Oxford-IIIT Pets Dataset.
# Users should configure the fine_tune_checkpoint field in the train config as
# well as the label_map_path and input_path fields in the train_input_reader and
# eval_input_reader. Search for "PATH_TO_BE_CONFIGURED" to find the fields that
# should be configured.
model {
faster_rcnn {
num_classes: 2
image_resizer {
keep_aspect_ratio_resizer {
min_dimension: 1080
max_dimension: 1920
}
}
feature_extractor {
type: 'faster_rcnn_inception_v2'
first_stage_features_stride: 16
}
first_stage_anchor_generator {
grid_anchor_generator {
scales: [0.25, 0.5, 1.0, 2.0]
aspect_ratios: [0.5, 1.0, 2.0]
height_stride: 16
width_stride: 16
}
}
first_stage_box_predictor_conv_hyperparams {
op: CONV
regularizer {
l2_regularizer {
weight: 0.0
}
}
initializer {
truncated_normal_initializer {
stddev: 0.01
}
}
}
first_stage_nms_score_threshold: 0.0
first_stage_nms_iou_threshold: 0.7
first_stage_max_proposals: 300
first_stage_localization_loss_weight: 2.0
first_stage_objectness_loss_weight: 1.0
initial_crop_size: 14
maxpool_kernel_size: 2
maxpool_stride: 2
second_stage_box_predictor {
mask_rcnn_box_predictor {
use_dropout: false
dropout_keep_probability: 1.0
fc_hyperparams {
op: FC
regularizer {
l2_regularizer {
weight: 0.0
}
}
initializer {
variance_scaling_initializer {
factor: 1.0
uniform: true
mode: FAN_AVG
}
}
}
}
}
second_stage_post_processing {
batch_non_max_suppression {
score_threshold: 0.0
iou_threshold: 0.6
max_detections_per_class: 100
max_total_detections: 300
}
score_converter: SOFTMAX
}
second_stage_localization_loss_weight: 2.0
second_stage_classification_loss_weight: 1.0
}
}
train_config: {
batch_size: 1
optimizer {
momentum_optimizer: {
learning_rate: {
manual_step_learning_rate {
initial_learning_rate: 0.0002
schedule {
step: 900000
learning_rate: .00002
}
schedule {
step: 1200000
learning_rate: .000002
}
}
}
momentum_optimizer_value: 0.9
}
use_moving_average: false
}
gradient_clipping_by_norm: 10.0
fine_tune_checkpoint: ""
from_detection_checkpoint: true
load_all_detection_checkpoint_vars: true
# Note: The below line limits the training process to 200K steps, which we
# empirically found to be sufficient enough to train the pets dataset. This
# effectively bypasses the learning rate schedule (the learning rate will
# never decay). Remove the below line to train indefinitely.
num_steps: 200000
data_augmentation_options {
random_horizontal_flip {
}
}
batch_queue_capacity: 60
num_batch_queue_threads: 30
prefetch_queue_capacity: 40
}
train_input_reader: {
tf_record_input_reader {
input_path: "D:\\object_detection\\train_data\\train.record"
}
label_map_path: "D:\\object_detection\\pascal_label_map.pbtxt"
queue_capacity: 2
min_after_dequeue: 1
num_readers: 1
}
eval_config: {
metrics_set: "coco_detection_metrics"
num_examples: 1101
}
eval_input_reader: {
tf_record_input_reader {
input_path: "D:\\object_detection\\eval_data\\eval.record"
}
label_map_path: "D:\\object_detection\\pascal_label_map.pbtxt"
shuffle: false
num_readers: 1
}
If there are other solutions, I will be very grateful to you.

Answer 1

Object detection models consume a lot of memory. This is because how they work and the large amount of anchors that they generate to find the boxes.
You are doing all fine, but your GPU is not enough for training these kind of models.
Things you can do:
Reduce the image size, lets say something like 720x512
Use SGD as optimizer, instead of others optimizers such Adam. SGD consumes approximately 3 times less memory than Adam.
Also is worth to mention that you are doing well with small batches of 1 instances. If I am not wrong, FasterRCNN is trained with only 2 images per batch

Answer 2

I just found that if I set batch_size to 3, then it works normally. When I set batch_size back to 1, it encounters OOM problem.
It is weird and I still don't know why, since it should always save memory with lower batch size.
If you encounter the same situation, can try to increase the batch size slightly, but I cannot guarantee it will work.