Variables of tensorflow generate error in a loop - tensorflow

Question

I have a similar problem to TensorFlow: varscope.reuse_variables().
I am doing cross-validation on a dataset.
Each time I call a function e.g. myFunctionInFile1()) with new data (currently due to limited space, I am omitting the data assigning details). This function is not in same python file. Because of Which I import this function from that file in my main python file (file2). This function creates a complete CNN and train and test a model on the given training and testing data with newly initialized and trained parameters.
From the main file (file2), at first validation, myFunctionInFile1 is called and the CNN model train and test it and returns results to the main file (file2). However, in the second iteration with new data, following code:
def myFunctionInFile1():
# Nodes for the input variables
x = tf.placeholder("float", shape=[None, D], name='Input_data')
y_ = tf.placeholder(tf.int64, shape=[None], name='Ground_truth')
keep_prob = tf.placeholder("float")
bn_train = tf.placeholder(tf.bool) # Boolean value to guide batchnorm
def bias_variable(shape, name):
initial = tf.constant(0.1, shape=shape)
return tf.Variable(initial, name=name)
def conv2d(x, W):
return tf.nn.conv2d(x, W, strides=[1, 1, 1, 1], padding='SAME')
def max_pool_2x2(x):
return tf.nn.max_pool(x, ksize=[1, 2, 2, 1],
strides=[1, 2, 2, 1], padding='SAME')
with tf.name_scope("Reshaping_data") as scope:
x_image = tf.reshape(x, [-1, D, 1, 1])
initializer = tf.contrib.layers.xavier_initializer()
"""Build the graph"""
# ewma is the decay for which we update the moving average of the
# mean and variance in the batch-norm layers
with tf.name_scope("Conv1") as scope:
# reuse = tf.AUTO_REUSE
W_conv1 = tf.get_variable("Conv_Layer_1", shape=[5, 1, 1, num_filt_1], initializer=initializer)
b_conv1 = bias_variable([num_filt_1], 'bias_for_Conv_Layer_1')
a_conv1 = conv2d(x_image, W_conv1) + b_conv1
with tf.name_scope('Batch_norm_conv1') as scope:
a_conv1 = tf.contrib.layers.batch_norm(a_conv1,is_training=bn_train,updates_collections=None)
give me following error:
ValueError: Variable BatchNorm_2/beta does not exist, or was not created with tf.get_variable(). Did you mean to set reuse=tf.AUTO_REUSE in VarScope?
What is the problem, as generally in programming in C/C++/Java if you exit a function, the local variables in that called function are deleted automatically at return. And at each time new call it should create a new set of parameters. Than why it gives this error. How can I fix this?

Answer 1

TensorFlow layers like batch_norm are implemented using tf.get_variable. tf.get_variable has a reuse argument (which it can also get from a variable_scope), defaulting to False, and when called with reuse=False it always creates variables. You can call it with reuse=True which means it will reuse existing variables or fail if the variables do not exist.
In your case you're calling batch norm with reuse=True for the first time, so it's having a hard time creating the variables. Try setting reuse=False in your variable scope or using, as the error message suggests, tf.AUTO_REUSE.