tensorflow slim inception_v3 model error - tensorflow

I am trying to use the tensorflow inception_v3 model for a transfer learning project.I get the following error on building the model.
TypeError: Expected int32, got list containing Tensors of type '_Message' instead.
The same error does not arise on using the same script for inception_v1 model.
The models are imported from slim.nets
Running on CPU
Tensorflow version : 0.12.1
Script
import tensorflow as tf
slim = tf.contrib.slim
import models.inception_v3 as inception_v3
print("initializing model")
inputs=tf.placeholder(tf.float32, shape=[32,299,299,3])
with slim.arg_scope(inception_v3.inception_v3_arg_scope()):
logits,endpoints = inception_v3.inception_v3(inputs, num_classes=1001, is_training=False)
trainable_vars=tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES)
for tvars in trainable_vars:
print tvars.name
Full Error Message
Traceback (most recent call last):
File "test.py", line 8, in <module>
logits,endpoints = inception_v3.inception_v3(inputs, num_classes=1001, is_training=False)
File "/home/ashish/projects/python/fashion-language/models/inception_v3.py", line 576, in inception_v3
depth_multiplier=depth_multiplier)
File "/home/ashish/projects/python/fashion-language/models/inception_v3.py", line 181, in inception_v3_base
net = array_ops.concat([branch_0, branch_1, branch_2, branch_3], 3)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/array_ops.py", line 1075, in concat
dtype=dtypes.int32).get_shape(
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 669, in convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/constant_op.py", line 176, in _constant_tensor_conversion_function
return constant(v, dtype=dtype, name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/constant_op.py", line 165, in constant
tensor_util.make_tensor_proto(value, dtype=dtype, shape=shape, verify_shape=verify_shape))
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/tensor_util.py", line 367, in make_tensor_proto
_AssertCompatible(values, dtype)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/tensor_util.py", line 302, in _AssertCompatible
(dtype.name, repr(mismatch), type(mismatch).__name__))
TypeError: Expected int32, got list containing Tensors of type '_Message' instead.

Found my mistake, i was importing the model from https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/slim/python/slim/nets whereas the updated models are at https://github.com/tensorflow/models/tree/master/slim/nets.
Still haven't understood why there are two different repositories for the same classes.Must be a valid reason.

Related

How to finalize a model in Keras

I have a straightforward model developed in Keras:
....
model = Model(input, output)
model.compile(optimizer='adam', loss='categorical_crossentropy')
graph = tf.compat.v1.get_default_graph()
graph.finalize()
history = model.fit(X, y, epochs=30)
Since I'm dealing with some memory leak problems, it seemed like a good idea to finalize the graph to prevent the mentioned issue. But when I do, I get an exception RuntimeError: Graph is finalized and cannot be modified.:
Traceback (most recent call last):
File "./train.py", line 43, in <module>
history = model.fit(X, y, epochs=30)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py", line 780, in fit
steps_name='steps_per_epoch')
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training_arrays.py", line 157, in model_iteration
f = _make_execution_function(model, mode)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training_arrays.py", line 532, in _make_execution_function
return model._make_execution_function(mode)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py", line 2276, in _make_execution_function
self._make_train_function()
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py", line 2212, in _make_train_function
if not isinstance(K.symbolic_learning_phase(), int):
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/backend.py", line 299, in symbolic_learning_phase
False, shape=(), name='keras_learning_phase')
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/array_ops.py", line 2159, in placeholder_with_default
return gen_array_ops.placeholder_with_default(input, shape, name)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/gen_array_ops.py", line 6406, in placeholder_with_default
"PlaceholderWithDefault", input=input, shape=shape, name=name)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/op_def_library.py", line 527, in _apply_op_helper
preferred_dtype=default_dtype)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py", line 1224, in internal_convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/constant_op.py", line 305, in _constant_tensor_conversion_function
return constant(v, dtype=dtype, name=name)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/constant_op.py", line 246, in constant
allow_broadcast=True)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/constant_op.py", line 290, in _constant_impl
name=name).outputs[0]
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py", line 3588, in create_op
self._check_not_finalized()
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py", line 3225, in _check_not_finalized
raise RuntimeError("Graph is finalized and cannot be modified.")
RuntimeError: Graph is finalized and cannot be modified.
There's nothing custom in this model, all the layers are from Keras library. And I'm using Tensorflow 1.14 and the Keras that comes with it (tensorflow.keras).
My question is, what are my options here? How can I pinpoint the reason for graph change? Or maybe I'm finalizing the graph wrong!?
[UPDATE]
To make sure that the problem is not with my setup and model, I followed the example provided in Tensorflow docs (follow the Colab link). I just added the two lines of code to finalize the graph just before calling the fit method. And I faced the same error. So my question stands, how do you finalize a model in Keras?

Save Model for Serving but "ValueError: Both labels and logits must be provided." when trying to export model

I wanted to save a model to do some predictions on specific pictures. Here is my serving function:
def _serving_input_receiver_fn():
# Note: only handles one image at a time
feat = tf.placeholder(tf.float32, shape=[None, 120, 50, 1])
return tf.estimator.export.TensorServingInputReceiver(features=feat, receiver_tensors=feat)
and here is where I export the model:
export_dir_base = os.path.join(FLAGS.model_dir, 'export')
export_dir = estimator.export_savedmodel(
export_dir_base, _serving_input_receiver_fn)
But I get the following error:
ValueError: Both labels and logits must be provided.
Now this Error I don't understand since the Serving stuff should just create a placeholder so I can later put some images through the placeholder to make predictions on the saved model?
Here is the whole traceback:
Traceback (most recent call last):
File "/home/cezary/models/official/mnist/mnist_tpu.py", line 222, in <module>
tf.app.run()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 125, in run
_sys.exit(main(argv))
File "/home/cezary/models/official/mnist/mnist_tpu.py", line 206, in main
export_dir_base, _serving_input_receiver_fn)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/estimator/estimator.py", line 650, in export_savedmodel
mode=model_fn_lib.ModeKeys.PREDICT)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/estimator/estimator.py", line 703, in _export_saved_model_for_mode
strip_default_attrs=strip_default_attrs)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/estimator/estimator.py", line 811, in _export_all_saved_models
mode=model_fn_lib.ModeKeys.PREDICT)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/tpu/python/tpu/tpu_estimator.py", line 1971, in _add_meta_graph_for_mode
mode=mode)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/estimator/estimator.py", line 879, in _add_meta_graph_for_mode
config=self.config)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/tpu/python/tpu/tpu_estimator.py", line 1992, in _call_model_fn
features, labels, mode, config)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/estimator/estimator.py", line 1107, in _call_model_fn
model_fn_results = self._model_fn(features=features, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/tpu/python/tpu/tpu_estimator.py", line 2203, in _model_fn
features, labels, is_export_mode=is_export_mode)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/tpu/python/tpu/tpu_estimator.py", line 1131, in call_without_tpu
return self._call_model_fn(features, labels, is_export_mode=is_export_mode)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/tpu/python/tpu/tpu_estimator.py", line 1337, in _call_model_fn
estimator_spec = self._model_fn(features=features, **kwargs)
File "/home/cezary/models/official/mnist/mnist_tpu.py", line 95, in model_fn
cross_entropy = tf.nn.sigmoid_cross_entropy_with_logits(labels=labels, logits=logits)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/nn_impl.py", line 156, in sigmoid_cross_entropy_with_logits
labels, logits)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/nn_ops.py", line 1777, in _ensure_xent_args
raise ValueError("Both labels and logits must be provided.")
ValueError: Both labels and logits must be provided.
Nevermind the mnist naming, I just used the structure of the code, but didn't rename it.
Thanks for any help!
(I can't comment with a brand new account.) I was able to replicate your error by setting features and receiver_tensors to have the same value, but I don't think that your __serving_input_receiver_fn is implemented correctly. Can you follow the example here?

Tensorflow error "feed a value for placeholder tensor" occurs on the second RBM in a deep belief net

I first constructed an RBM and tested it on a set of data, it worked well. Then I wrote a DBN with stacked RBM and trained it with the same set of data. The program stopped with the following error when it tried to train the second RBM.
Traceback (most recent call last):
File "D:\Python\DL_DG\analysis\debug\debug_01_ppi.py", line 44, in <module>
ppi_dbn.fit(ppi_in)
File "D:/Python/DL_DG/Model\dbn_test.py", line 95, in fit
rbm.fit(input_data)
File "D:/Python/DL_DG/Model\rbm_test.py", line 295, in fit
self.partial_fit(batch_x, b, e)
File "D:/Python/DL_DG/Model\rbm_test.py", line 188, in partial_fit
feed_dict={self.x: batch_x})
File "C:\Users\pil562\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\client\session.py", line 895, in run
run_metadata_ptr)
File "C:\Users\pil562\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1124, in _run
feed_dict_tensor, options, run_metadata)
File "C:\Users\pil562\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1321, in _do_run
options, run_metadata)
File "C:\Users\pil562\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1340, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: You must feed a value for placeholder tensor 'input/x' with dtype float and shape [?,128]
[[Node: input/x = Placeholder[dtype=DT_FLOAT, shape=[?,128], _device="/job:localhost/replica:0/task:0/cpu:0"]()]]
Caused by op 'input/x', defined at:
File "<string>", line 1, in <module>
File "C:\Users\pil562\AppData\Local\Programs\Python\Python36\lib\idlelib\run.py", line 142, in main
ret = method(*args, **kwargs)
File "C:\Users\pil562\AppData\Local\Programs\Python\Python36\lib\idlelib\run.py", line 460, in runcode
exec(code, self.locals)
File "D:\Python\DL_DG\analysis\debug\debug_01_ppi.py", line 42, in <module>
learning_rate_rbm=[0.001,0.01],rbm_gauss_visible=True)
File "D:/Python/DL_DG/Model\dbn_test.py", line 52, in __init__
sample_gauss_visible=self.sample_gauss_visible, sigma=self.sigma))
File "D:/Python/DL_DG/Model\rbm_test.py", line 358, in __init__
xavier_const,err_function,use_tqdm,tqdm)
File "D:/Python/DL_DG/Model\rbm_test.py", line 46, in __init__
self.x = tf.placeholder(tf.float32, [None, self.n_visible],name='x')
File "C:\Users\pil562\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\ops\array_ops.py", line 1548, in placeholder
return gen_array_ops._placeholder(dtype=dtype, shape=shape, name=name)
File "C:\Users\pil562\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\ops\gen_array_ops.py", line 2094, in _placeholder
name=name)
File "C:\Users\pil562\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 767, in apply_op
op_def=op_def)
File "C:\Users\pil562\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\ops.py", line 2630, in create_op
original_op=self._default_original_op, op_def=op_def)
File "C:\Users\pil562\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\ops.py", line 1204, in __init__
self._traceback = self._graph._extract_stack() # pylint: disable=protected-access
InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'input/x' with dtype float and shape [?,128]
[[Node: input/x = Placeholder[dtype=DT_FLOAT, shape=[?,128], _device="/job:localhost/replica:0/task:0/cpu:0"]()]]
The error occurs at the following function:
def partial_fit(self, batch_x, k, j):
print(batch_x.dtype, batch_x.shape)
summary, _ = self.sess.run([self.merged, self.update_weights + self.update_deltas],
feed_dict={self.x: batch_x})
self.train_writer.add_summary(summary, k*self.batch_size+j)
I output the type and shape of batch_x. The shape is the same during the whole training process. The type is float64 when training the first rbm, and float32 when training the second rbm. That's where it stopped and throw out the error.
The DBN worked well when I didn't compute the summary and just used the following code:
self.sess.run(self.update_weights + self.update_deltas,feed_dict={self.x: batch_x})
It also worked well if I only train a single RBM (with or without the summary).
The batch_x used to train the second RBM is probabilities of the hidden layer in the first RBM.
Could somebody help me solve this problem? I'm not sure if the float64 is the problem.
I guess it's hard for anyone to solve the problem only with the two pieces of code I give. lol. The full code is too long to post here.
I save the output of the first RBM and use it as input to train another RBM. It works well. Thus, I think the problem is not the type or shape of the feeded batch_x, but the structure of the DBN, or the way I collected summaries.
Hope my situation can help others with similar problems.

Do the Tensorflow Silm assume TF version 1.4 for NASNet?

I try to train NASNet-A_Mobile_224 for two class classification by using train_image_classifier.py from slim with nasnet_mobile, However I get error
TypeError: separable_convolution2d() got an unexpected keyword argument 'data_format'
I suspect that the new NASNet requires TF version 1.4. Can somebody confirm this? I'm using Tensorflow 1.3.
More extensive error is given below:
Traceback (most recent call last):
File "train_image_classifier.py", line 574, in <module>
tf.app.run()
File "/home/sami/virenv/tensorflow_vanilla/local/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "train_image_classifier.py", line 474, in main
clones = model_deploy.create_clones(deploy_config, clone_fn, [batch_queue])
File "/home/sami/projects/Tools/models/research/slim/deployment/model_deploy.py", line 193, in create_clones
outputs = model_fn(*args, **kwargs)
File "train_image_classifier.py", line 457, in clone_fn
logits, end_points = network_fn(images)
File "/home/sami/projects/Tools/models/research/slim/nets/nets_factory.py", line 135, in network_fn
return func(images, num_classes, is_training=is_training, **kwargs)
File "/home/sami/projects/Tools/models/research/slim/nets/nasnet/nasnet.py", line 371, in build_nasnet_mobile
final_endpoint=final_endpoint)
File "/home/sami/projects/Tools/models/research/slim/nets/nasnet/nasnet.py", line 450, in _build_nasnet_base
net, cell_outputs = stem()
File "/home/sami/projects/Tools/models/research/slim/nets/nasnet/nasnet.py", line 445, in <lambda>
stem = lambda: _imagenet_stem(images, hparams, stem_cell)
File "/home/sami/projects/Tools/models/research/slim/nets/nasnet/nasnet.py", line 264, in _imagenet_stem
cell_num=cell_num)
File "/home/sami/projects/Tools/models/research/slim/nets/nasnet/nasnet_utils.py", line 326, in __call__
stride, original_input_left)
File "/home/sami/projects/Tools/models/research/slim/nets/nasnet/nasnet_utils.py", line 352, in _apply_conv_operation
net = _stacked_separable_conv(net, stride, operation, filter_size)
File "/home/sami/projects/Tools/models/research/slim/nets/nasnet/nasnet_utils.py", line 183, in _stacked_separable_conv
stride=stride)
File "/home/sami/virenv/tensorflow_vanilla/local/lib/python2.7/site-packages/tensorflow/contrib/framework/python/ops/arg_scope.py", line 181, in func_with_args
return func(*args, **current_args)
TypeError: separable_convolution2d() got an unexpected keyword argument 'data_format'
YES,it must be tensorflow 1.4.0

Tensorflow: Tried SVHN by editing label 0-9, still not working

I am trying to classify SVHN data set by following this tutorial:
https://www.tensorflow.org/versions/0.6.0/tutorials/deep_cnn/index.html
I am using train_32x32.mat file. In order to use it with CNN code (mentioned above), I converted this .mat file to several .bin files using this simple code:
import numpy as np
import scipy.io
from array import array
read_input = scipy.io.loadmat('data/train_32x32.mat')
j=0
output_file = open('data_batch_%d.bin' % j, 'ab')
for i in range(0, 64000):
# create new bin file
if i>0 and i % 12800 == 0:
output_file.close()
j=j+1
output_file = open('data_batch_%d.bin' % j, 'ab')
# Write to bin file
if read_input['y'][i] == 10:
read_input['y'][i] = 0
read_input['y'][i].astype('uint8').tofile(output_file)
read_input['X'][:,:,:,i].astype('uint32').tofile(output_file)
output_file.close()
But when I tried to classify SVHN using these customized .bin files I'm getting stuck with error "Invalid argument: Indices are not valid (out of bounds)" listed below:
Filling queue with 20000 CIFAR images before starting to train. This will take a few minutes.
I tensorflow/core/common_runtime/local_device.cc:25] Local device intra op parallelism threads: 4
I tensorflow/core/common_runtime/local_session.cc:45] Local session inter op parallelism threads: 4
W tensorflow/core/common_runtime/executor.cc:1027] 0x1a53160 Compute status: Invalid argument: Indices are not valid (out of bounds). Shape: dim { size: 128 } dim { size: 10 }
[[Node: SparseToDense = SparseToDense[T=DT_FLOAT, Tindices=DT_INT32, _device="/job:localhost/replica:0/task:0/cpu:0"](concat, SparseToDense/output_shape, SparseToDense/sparse_values, SparseToDense/default_value)]]
Traceback (most recent call last):
File "cifar10_train.py", line 138, in <module>
tf.app.run()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/default/_app.py", line 11, in run
sys.exit(main(sys.argv))
File "cifar10_train.py", line 134, in main
train()
File "cifar10_train.py", line 104, in train
_, loss_value = sess.run([train_op, loss])
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 345, in run
results = self._do_run(target_list, unique_fetch_targets, feed_dict_string)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 419, in _do_run
e.code)
tensorflow.python.framework.errors.InvalidArgumentError: Indices are not valid (out of bounds). Shape: dim { size: 128 } dim { size: 10 }
[[Node: SparseToDense = SparseToDense[T=DT_FLOAT, Tindices=DT_INT32, _device="/job:localhost/replica:0/task:0/cpu:0"](concat, SparseToDense/output_shape, SparseToDense/sparse_values, SparseToDense/default_value)]]
Caused by op u'SparseToDense', defined at:
File "cifar10_train.py", line 138, in <module>
tf.app.run()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/default/_app.py", line 11, in run
sys.exit(main(sys.argv))
File "cifar10_train.py", line 134, in main
train()
File "cifar10_train.py", line 76, in train
loss = cifar10.loss(logits, labels)
File "/home/sarah/Documents/SVHN/cifar10.py", line 364, in loss
dense_labels = tf.sparse_to_dense(concated,[FLAGS.batch_size, NUM_CLASSES],1.0, 0.0)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gen_sparse_ops.py", line 153, in sparse_to_dense
default_value=default_value, name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/op_def_library.py", line 633, in apply_op
op_def=op_def)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1710, in create_op
original_op=self._default_original_op, op_def=op_def)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 988, in __init__
self._traceback = _extract_stack()
I found TensorFlow CIFAR10 Example , similar issue in stackoverflow. But even if I change the label, it's still not working.
Please let me know if I did something wrong or not understanding any logic.
Thanks
Sarah
Something was wrong with my installed version of Tensorflow (might be a bug). Upgrading to new version solved the issue.
Thanks