Can't restore model with tf.contrib.layers.convolution2d_transpose applied - tensorflow

Question

I'm trying to restore a model using following code:
new_saver = tf.train.import_meta_graph(model_path+'.meta')
new_saver.restore(sess, model_path)
g=tf.get_default_graph()
And for each weight or bias in original graph, I did g.get_tensrr_by_name().
But when I tried to do this on a deconv2d layer, which is something like below:
def deconv2d(self,inputs, num_outputs, kernel_shape, g,scope,strides=[1, 1]):
with tf.variable_scope(scope) as scope:
weights_initializer = g.get_tensor_by_name("prsr/conditioning/deconv/Conv2d_transpose/weights:0")
biases_initializer = g.get_tensor_by_name("prsr/conditioning/deconv/Conv2d_transpose/biases:0")
return tf.contrib.layers.convolution2d_transpose(inputs=inputs, num_outputs=num_outputs,kernel_size=kernel_shape,stride=strides, \
padding='SAME', weights_initializer=weights_initializer,biases_initializer=biases_initializer)
it failed and showed following error:
File "restore.py", line 41, in deconv2d
padding='SAME', weights_initializer=weights_initializer,biases_initializer=biases_initializer)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/framework/python/ops/arg_scope.py", line 177, in func_with_args
return func(*args, **current_args)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/layers/python/layers/layers.py", line 1126, in convolution2d_transpose
outputs = layer.apply(inputs)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/layers/base.py", line 323, in apply
return self.__call__(inputs, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/layers/base.py", line 289, in __call__
self.build(input_shapes[0])
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/layers/convolutional.py", line 1043, in build
dtype=self.dtype)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py", line 1033, in get_variable
use_resource=use_resource, custom_getter=custom_getter)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py", line 932, in get_variable
use_resource=use_resource, custom_getter=custom_getter)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py", line 349, in get_variable
validate_shape=validate_shape, use_resource=use_resource)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/layers/base.py", line 278, in variable_getter
variable_getter=functools.partial(getter, **kwargs))
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/layers/base.py", line 228, in _add_variable
trainable=trainable and self.trainable)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/layers/python/layers/layers.py", line 1327, in layer_variable_getter
return _model_variable_getter(getter, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/layers/python/layers/layers.py", line 1316, in _model_variable_getter
custom_getter=getter, use_resource=use_resource)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/framework/python/ops/arg_scope.py", line 177, in func_with_args
return func(*args, **current_args)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/framework/python/ops/variables.py", line 259, in model_variable
use_resource=use_resource)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/framework/python/ops/arg_scope.py", line 177, in func_with_args
return func(*args, **current_args)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/framework/python/ops/variables.py", line 214, in variable
use_resource=use_resource)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py", line 341, in _true_getter
use_resource=use_resource)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py", line 638, in _get_single_variable
raise ValueError("If initializer is a constant, do not specify shape.")
ValueError: If initializer is a constant, do not specify shape.
I don't know which shape does this refer to, and I don't think weights_initializer and biases_initializer are constants, they are tensors, right? By the way, I'm very sure that those two tensors, prsr/conditioning/deconv/Conv2d_transpose/weights and prsr/conditioning/deconv/Conv2d_transpose/biasess exist in the original graph, since I checked this using print_tensors_in_checkpoint_file, and I can actually see the values.
So how can I restore model which applies this tf.contrib.layers.convolution2d_transpose() layer? I searched a lot on both stackoverflow and github, but nothing worked. Any help would be appreciated.

Answer 1

weights_initializer and bias_initializer are not what you think they are. You probably think of those two tensors as the initial values for the weights used in the deconvolution, right? However, the initializer argument is a function not a tensor that should look something like this:
def my_initializer(shape, dtype=tf.float32, partition_info=None):
# do some computation to build up a tensor of the given shape
return that_tensor
You could then use this initializer like so:
tf.contrib.layers.convolution2d_transpose(inputs=inputs, ..., weights_initializer=my_initializer)
So, as a solution to your problem, I think the following should work:
def weights_initializer(shape, dtype=tf.float32, partition_info=None):
weights = tf.get_default_graph().get_tensor_by_name("prsr/conditioning/deconv/Conv2d_transpose/weights:0")
return weights
This feels a bit hacky in my opinion, though. Why do you want to load a graph and then use pre-trained weights in a new operation? Why are those weights not associated with this operation before already when you set up the initial model?
PS: When dealing with variables you might find tf.get_variable come in handy. If you create variables using tf.get_variable you can later retrieve those variables with tf.get_variable again, without having to call the cumbersome get_tensor_by_name. Check this for more info.