need help deciphering tensorflow feed InvalidArgumentError - tensorflow

I don't think I understand this feed error from tensorflow
Debug: [[ 0. 0.]]
Debug: (1, 2)
Debug: float64
2018-05-09 09:56:34.615561: W tensorflow/core/kernels/queue_base.cc:295] _0_input_producer: Skipping cancelled enqueue attempt with queue not closed
Traceback (most recent call last):
File "/home/kiran/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1323, in _do_call
return fn(*args)
File "/home/kiran/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1302, in _run_fn
status, run_metadata)
File "/home/kiran/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py", line 473, in __exit__
c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: You must feed a value for placeholder tensor 'seqModel/a_prev' with dtype double and shape [1,2]
[[Node: seqModel/a_prev = Placeholder[dtype=DT_DOUBLE, shape=[1,2], _device="/job:localhost/replica:0/task:0/device:GPU:0"]()]]
the way I feed the placeholder is:
self.a_prev = tf.placeholder(tf.float64, shape=[1,2], name='a_prev')
batch = tf.train.batch([self.x_acc, self.y_acc,
self.prev_pos],
batch_size=1, capacity=20000, num_threads=1)
x_acc, y_acc, prev_pos = sess.run(batch)
test = np.array([[ x_acc[0,0], y_acc[0,0] ]])
print("Debug: ",test)
print("Debug:",test.shape)
print("Debug:",test.dtype)
_,X_hat_val,loss_val, X_val = sess.run([train,X_hat,loss, self.X],
feed_dict={self.a_prev : np.array([[x_acc[0,0],y_acc[0,0] ]]),
self.pos1 : np.array([[ prev_pos[0,0] ]])
})
The error does not make sense because I am feeding values to the placeholder but it says that there are no values. What does that mean?

NB: I didn't run your code, as it depends on unavailable data.
However, it's probable your error is caused by reassigning the self.a_prev attribute, line 173. With this line, self.a_prev doesn't point to the tf.placeholder(..., name='a_prev') anymore, but to a different Tensor (from self.new_evidence) - so the actual placeholder doesn't get fed when running.
Toy example for this supposition
import tensorflow as tf
import numpy as np
x_acc = np.random.rand(2, 2)
y_acc = np.random.rand(2, 2)
a_prev = tf.placeholder(tf.float64, shape=[1,2], name='a_prev')
some_results = tf.add(a_prev, 1.)
a_prev = tf.constant([[-1, -1]])
# ... now "a_prev" the python variable isn't pointing to the placeholder anymore,
# so "a_prev" the placeholder exists in the graph with no python pointer to it.
with tf.Session() as sess:
res = sess.run(some_results, feed_dict={a_prev : np.array([[x_acc[0,0],y_acc[0,0] ]])})
# "a_prev" the constant is assigned the values, not "a_prev" the placeholder,
# hence an error.
InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'a_prev' with dtype double and shape [1,2]
[[Node: a_prev = Placeholderdtype=DT_DOUBLE, shape=[1,2],
_device="/job:localhost/replica:0/task:0/device:GPU:0"]] [[Node: Add/_1 = _Recvclient_terminated=false,
recv_device="/job:localhost/replica:0/task:0/device:CPU:0",
send_device="/job:localhost/replica:0/task:0/device:GPU:0",
send_device_incarnation=1, tensor_name="edge_8_Add",
tensor_type=DT_DOUBLE,
_device="/job:localhost/replica:0/task:0/device:CPU:0"]]

Related

Tensorflow Object Detection API dictionary error

I am using the object_detection_tutorial.ipynb to run an inference.
After I run the inference, I got this error on the output_dict:
# Run inference
output_dict = model(input_tensor)
# All outputs are batches tensors.
# Convert to numpy arrays, and take index [0] to remove the batch dimension.
# We're only interested in the first num_detections.
num_detections = int(output_dict.pop('num_detections'))
output_dict = {key:value[0, :num_detections].numpy()
for key,value in output_dict.items()}
output_dict['num_detections'] = num_detections
Error log:
Traceback (most recent call last): File "detect.py", line 140, in
show_inference(detection_model, image_path) File "detect.py", line 109, in show_inference
output_dict = run_inference_for_single_image(model, image_np) File "detect.py", line 94, in run_inference_for_single_image
num_detections = int(output_dict.pop('num_detections')) TypeError: int() argument must be a string, a bytes-like object or a number, not
'Tensor'
Solved upgrading to tensorflow 2.0

TensorFlow program sometimes works, sometimes throws different errors related to reshape nodes at different points in training using same random seed

I'm having difficulty with a particularly spooky-looking TensorFlow issue. (TensorFlow 1.4.1 and Python 2.7)
Errors
I'm seeing a couple different errors when running the same program. Here is an example of one:
W tensorflow/core/framework/op_kernel.cc:1192] Invalid argument: Input to reshape is a tensor with 122496 values, but the requested shape has 0
[[Node: optimizer/gradients/energy_2/map/while/Gather_grad/Reshape = Reshape[T=DT_FLOAT, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/device:GPU:0"](optimizer/gradients/energy_2/map/while/mul_grad/tuple/control_dependency_1, optimizer/gradients/energy_2/map/while/Gather_grad/concat)]]
2018-03-15 18:52:25.377745: W tensorflow/core/framework/op_kernel.cc:1192] Invalid argument: Input to reshape is a tensor with 122496 values, but the requested shape has 0
[[Node: optimizer/gradients/energy_2/map/while/Gather_grad/Reshape = Reshape[T=DT_FLOAT, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/device:GPU:0"](optimizer/gradients/energy_2/map/while/mul_grad/tuple/control_dependency_1, optimizer/gradients/energy_2/map/while/Gather_grad/concat)]]
2018-03-15 18:52:25.378256: W tensorflow/core/framework/op_kernel.cc:1192] Invalid argument: Input to reshape is a tensor with 122496 values, but the requested shape has 0
[[Node: optimizer/gradients/energy_2/map/while/Gather_grad/Reshape = Reshape[T=DT_FLOAT, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/device:GPU:0"](optimizer/gradients/energy_2/map/while/mul_grad/tuple/control_dependency_1, optimizer/gradients/energy_2/map/while/Gather_grad/concat)]]
2018-03-15 18:52:25.378753: W tensorflow/core/framework/op_kernel.cc:1192] Invalid argument: Input to reshape is a tensor with 122496 values, but the requested shape has 0
[[Node: optimizer/gradients/energy_2/map/while/Gather_grad/Reshape = Reshape[T=DT_FLOAT, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/device:GPU:0"](optimizer/gradients/energy_2/map/while/mul_grad/tuple/control_dependency_1, optimizer/gradients/energy_2/map/while/Gather_grad/concat)]]
2018-03-15 18:52:25.379193: W tensorflow/core/framework/op_kernel.cc:1192] Invalid argument: Input to reshape is a tensor with 122496 values, but the requested shape has 0
[[Node: optimizer/gradients/energy_2/map/while/Gather_grad/Reshape = Reshape[T=DT_FLOAT, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/device:GPU:0"](optimizer/gradients/energy_2/map/while/mul_grad/tuple/control_dependency_1, optimizer/gradients/energy_2/map/while/Gather_grad/concat)]]
2018-03-15 18:52:25.379692: W tensorflow/core/framework/op_kernel.cc:1192] Invalid argument: Input to reshape is a tensor with 122496 values, but the requested shape has 0
[[Node: optimizer/gradients/energy_2/map/while/Gather_grad/Reshape = Reshape[T=DT_FLOAT, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/device:GPU:0"](optimizer/gradients/energy_2/map/while/mul_grad/tuple/control_dependency_1, optimizer/gradients/energy_2/map/while/Gather_grad/concat)]]
2018-03-15 18:52:25.380208: W tensorflow/core/framework/op_kernel.cc:1192] Invalid argument: Input to reshape is a tensor with 122496 values, but the requested shape has 0
[[Node: optimizer/gradients/energy_2/map/while/Gather_grad/Reshape = Reshape[T=DT_FLOAT, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/device:GPU:0"](optimizer/gradients/energy_2/map/while/mul_grad/tuple/control_dependency_1, optimizer/gradients/energy_2/map/while/Gather_grad/concat)]]
2018-03-15 18:52:25.380709: W tensorflow/core/framework/op_kernel.cc:1192] Invalid argument: Input to reshape is a tensor with 122496 values, but the requested shape has 0
[[Node: optimizer/gradients/energy_2/map/while/Gather_grad/Reshape = Reshape[T=DT_FLOAT, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/device:GPU:0"](optimizer/gradients/energy_2/map/while/mul_grad/tuple/control_dependency_1, optimizer/gradients/energy_2/map/while/Gather_grad/concat)]]
2018-03-15 18:52:25.381166: W tensorflow/core/framework/op_kernel.cc:1192] Invalid argument: Input to reshape is a tensor with 122496 values, but the requested shape has 0
[[Node: optimizer/gradients/energy_2/map/while/Gather_grad/Reshape = Reshape[T=DT_FLOAT, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/device:GPU:0"](optimizer/gradients/energy_2/map/while/mul_grad/tuple/control_dependency_1, optimizer/gradients/energy_2/map/while/Gather_grad/concat)]]
2018-03-15 18:52:25.381654: W tensorflow/core/framework/op_kernel.cc:1192] Invalid argument: Input to reshape is a tensor with 122496 values, but the requested shape has 0
[[Node: optimizer/gradients/energy_2/map/while/Gather_grad/Reshape = Reshape[T=DT_FLOAT, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/device:GPU:0"](optimizer/gradients/energy_2/map/while/mul_grad/tuple/control_dependency_1, optimizer/gradients/energy_2/map/while/Gather_grad/concat)]]
2018-03-15 18:52:25.382138: W tensorflow/core/framework/op_kernel.cc:1192] Invalid argument: Input to reshape is a tensor with 122496 values, but the requested shape has 0
[[Node: optimizer/gradients/energy_2/map/while/Gather_grad/Reshape = Reshape[T=DT_FLOAT, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/device:GPU:0"](optimizer/gradients/energy_2/map/while/mul_grad/tuple/control_dependency_1, optimizer/gradients/energy_2/map/while/Gather_grad/concat)]]
2018-03-15 18:52:25.382601: W tensorflow/core/framework/op_kernel.cc:1192] Invalid argument: Input to reshape is a tensor with 122496 values, but the requested shape has 0
[[Node: optimizer/gradients/energy_2/map/while/Gather_grad/Reshape = Reshape[T=DT_FLOAT, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/device:GPU:0"](optimizer/gradients/energy_2/map/while/mul_grad/tuple/control_dependency_1, optimizer/gradients/energy_2/map/while/Gather_grad/concat)]]
2018-03-15 18:52:25.383111: W tensorflow/core/framework/op_kernel.cc:1192] Invalid argument: Input to reshape is a tensor with 122496 values, but the requested shape has 0
[[Node: optimizer/gradients/energy_2/map/while/Gather_grad/Reshape = Reshape[T=DT_FLOAT, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/device:GPU:0"](optimizer/gradients/energy_2/map/while/mul_grad/tuple/control_dependency_1, optimizer/gradients/energy_2/map/while/Gather_grad/concat)]]
2018-03-15 18:52:25.383601: W tensorflow/core/framework/op_kernel.cc:1192] Invalid argument: Input to reshape is a tensor with 122496 values, but the requested shape has 0
[[Node: optimizer/gradients/energy_2/map/while/Gather_grad/Reshape = Reshape[T=DT_FLOAT, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/device:GPU:0"](optimizer/gradients/energy_2/map/while/mul_grad/tuple/control_dependency_1, optimizer/gradients/energy_2/map/while/Gather_grad/concat)]]
2018-03-15 18:52:25.384107: W tensorflow/core/framework/op_kernel.cc:1192] Invalid argument: Input to reshape is a tensor with 122496 values, but the requested shape has 0
[[Node: optimizer/gradients/energy_2/map/while/Gather_grad/Reshape = Reshape[T=DT_FLOAT, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/device:GPU:0"](optimizer/gradients/energy_2/map/while/mul_grad/tuple/control_dependency_1, optimizer/gradients/energy_2/map/while/Gather_grad/concat)]]
2018-03-15 18:52:25.384551: W tensorflow/core/framework/op_kernel.cc:1192] Invalid argument: Input to reshape is a tensor with 122496 values, but the requested shape has 0
[[Node: optimizer/gradients/energy_2/map/while/Gather_grad/Reshape = Reshape[T=DT_FLOAT, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/device:GPU:0"](optimizer/gradients/energy_2/map/while/mul_grad/tuple/control_dependency_1, optimizer/gradients/energy_2/map/while/Gather_grad/concat)]]
2018-03-15 18:52:25.384792: W tensorflow/core/framework/op_kernel.cc:1192] Invalid argument: Input to reshape is a tensor with 122496 values, but the requested shape has 0
[[Node: optimizer/gradients/energy_2/map/while/Gather_grad/Reshape = Reshape[T=DT_FLOAT, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/device:GPU:0"](optimizer/gradients/energy_2/map/while/mul_grad/tuple/control_dependency_1, optimizer/gradients/energy_2/map/while/Gather_grad/concat)]]
2018/03/15 18:52:25 ERROR|--|Traceback (most recent call last):
File "experiment_runner.py", line 140, in experimentset
results = e.run()
File "experiment_runner.py", line 69, in run
results = run_fn()
File "experiment_runner.py", line 100, in traintest
return tt.run(self.exp_specs, self.data, model)
File "/s/chopin/a/grad/jonbyrd/protqa/protqa/experiment/train_test.py", line 149, in run
return self._fit_model(exp_specs, data, model)
File "/s/chopin/a/grad/jonbyrd/protqa/protqa/experiment/train_test.py", line 56, in _fit_model
self.train_proteins_epoch(data["train"], model, exp_specs["args"]["minibatch_size"])
File "/s/chopin/a/grad/jonbyrd/protqa/protqa/experiment/train_test.py", line 186, in train_proteins_epoch
model.train(minibatch)
File "/s/chopin/a/grad/jonbyrd/protqa/protqa/modeling/models/tf_model.py", line 169, in train
results = self._train(data, options=run_options, run_metadata=run_metadata, **kwargs)
File "/s/chopin/a/grad/jonbyrd/protqa/protqa/modeling/models/tf_model.py", line 113, in _train
results = self.run_graph([self.train_op, self.loss], data, "train", **kwargs)
File "/s/chopin/a/grad/jonbyrd/protqa/protqa/modeling/models/protnet.py", line 135, in run_graph
return self.sess.run(outputs, feed_dict=feed_dict, options=options, run_metadata=run_metadata)
File "/s/jawar/j/nobackup/protein_learning/virtualenv/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 889, in run
run_metadata_ptr)
File "/s/jawar/j/nobackup/protein_learning/virtualenv/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1120, in _run
feed_dict_tensor, options, run_metadata)
File "/s/jawar/j/nobackup/protein_learning/virtualenv/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1317, in _do_run
options, run_metadata)
File "/s/jawar/j/nobackup/protein_learning/virtualenv/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1336, in _do_call
raise type(e)(node_def, op, message)
InvalidArgumentError: Input to reshape is a tensor with 122496 values, but the requested shape has 0
[[Node: optimizer/gradients/energy_2/map/while/Gather_grad/Reshape = Reshape[T=DT_FLOAT, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/device:GPU:0"](optimizer/gradients/energy_2/map/while/mul_grad/tuple/control_dependency_1, optimizer/gradients/energy_2/map/while/Gather_grad/concat)]]
Caused by op u'optimizer/gradients/energy_2/map/while/Gather_grad/Reshape', defined at:
File "experiment_runner.py", line 332, in <module>
main()
File "experiment_runner.py", line 328, in main
e.run()
File "experiment_runner.py", line 69, in run
results = run_fn()
File "experiment_runner.py", line 140, in experimentset
results = e.run()
File "experiment_runner.py", line 69, in run
results = run_fn()
File "experiment_runner.py", line 99, in traintest
model = tt.build_model(self.exp_specs, self.data)
File "/s/chopin/a/grad/jonbyrd/protqa/protqa/experiment/train_test.py", line 141, in build_model
model = eval(hparams["name"] + "(exp_specs, data['train'])")
File "<string>", line 1, in <module>
File "/s/chopin/a/grad/jonbyrd/protqa/protqa/modeling/models/protnet.py", line 110, in __init__
self.setup_loss()
File "/s/chopin/a/grad/jonbyrd/protqa/protqa/modeling/models/tf_model.py", line 90, in setup_loss
self.train_op = self.hparams["optimizer"](self.loss, **self.hparams["optimizer_args"])
File "/s/chopin/a/grad/jonbyrd/protqa/protqa/modeling/optimizers.py", line 9, in tf_sgd
return tf.train.GradientDescentOptimizer(learning_rate).minimize(loss)
File "/s/jawar/j/nobackup/protein_learning/virtualenv/lib/python2.7/site-packages/tensorflow/python/training/optimizer.py", line 343, in minimize
grad_loss=grad_loss)
File "/s/jawar/j/nobackup/protein_learning/virtualenv/lib/python2.7/site-packages/tensorflow/python/training/optimizer.py", line 414, in compute_gradients
colocate_gradients_with_ops=colocate_gradients_with_ops)
File "/s/jawar/j/nobackup/protein_learning/virtualenv/lib/python2.7/site-packages/tensorflow/python/ops/gradients_impl.py", line 581, in gradients
grad_scope, op, func_call, lambda: grad_fn(op, *out_grads))
File "/s/jawar/j/nobackup/protein_learning/virtualenv/lib/python2.7/site-packages/tensorflow/python/ops/gradients_impl.py", line 353, in _MaybeCompile
return grad_fn() # Exit early
File "/s/jawar/j/nobackup/protein_learning/virtualenv/lib/python2.7/site-packages/tensorflow/python/ops/gradients_impl.py", line 581, in <lambda>
grad_scope, op, func_call, lambda: grad_fn(op, *out_grads))
File "/s/jawar/j/nobackup/protein_learning/virtualenv/lib/python2.7/site-packages/tensorflow/python/ops/array_grad.py", line 373, in _GatherGrad
values = array_ops.reshape(grad, values_shape)
File "/s/jawar/j/nobackup/protein_learning/virtualenv/lib/python2.7/site-packages/tensorflow/python/ops/gen_array_ops.py", line 3938, in reshape
"Reshape", tensor=tensor, shape=shape, name=name)
File "/s/jawar/j/nobackup/protein_learning/virtualenv/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "/s/jawar/j/nobackup/protein_learning/virtualenv/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2956, in create_op
op_def=op_def)
File "/s/jawar/j/nobackup/protein_learning/virtualenv/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1470, in __init__
self._traceback = self._graph._extract_stack() # pylint: disable=protected-access
...which was originally created as op u'energy_2/map/while/Gather', defined at:
File "experiment_runner.py", line 332, in <module>
main()
[elided 6 identical lines from previous traceback]
File "<string>", line 1, in <module>
File "/s/chopin/a/grad/jonbyrd/protqa/protqa/modeling/models/protnet.py", line 77, in __init__
dtype=tf.float32, parallel_iterations=32)
File "/s/jawar/j/nobackup/protein_learning/virtualenv/lib/python2.7/site-packages/tensorflow/python/ops/functional_ops.py", line 389, in map_fn
swap_memory=swap_memory)
File "/s/jawar/j/nobackup/protein_learning/virtualenv/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2816, in while_loop
result = loop_context.BuildLoop(cond, body, loop_vars, shape_invariants)
File "/s/jawar/j/nobackup/protein_learning/virtualenv/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2640, in BuildLoop
pred, body, original_loop_vars, loop_vars, shape_invariants)
File "/s/jawar/j/nobackup/protein_learning/virtualenv/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2590, in _BuildLoop
body_result = body(*packed_vars_for_body)
File "/s/jawar/j/nobackup/protein_learning/virtualenv/lib/python2.7/site-packages/tensorflow/python/ops/functional_ops.py", line 379, in compute
packed_fn_values = fn(packed_values)
File "/s/chopin/a/grad/jonbyrd/protqa/protqa/modeling/models/protnet.py", line 75, in <lambda>
], None, in_dims=nv, in_dists=self.in_dists, **args)[0],
File "/s/chopin/a/grad/jonbyrd/protqa/protqa/modeling/models/nn_components.py", line 447, in energy
return tf.reshape(tf.reduce_mean(tf.einsum('abi,abj->abij', (tf.expand_dims(verts, axis=1) * tf.gather(verts, hood_indices)), dists), axis=[0,1]), [in_dims*in_dists]), None
File "/s/jawar/j/nobackup/protein_learning/virtualenv/lib/python2.7/site-packages/tensorflow/python/ops/array_ops.py", line 2486, in gather
params, indices, validate_indices=validate_indices, name=name)
File "/s/jawar/j/nobackup/protein_learning/virtualenv/lib/python2.7/site-packages/tensorflow/python/ops/gen_array_ops.py", line 1834, in gather
validate_indices=validate_indices, name=name)
File "/s/jawar/j/nobackup/protein_learning/virtualenv/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
InvalidArgumentError (see above for traceback): Input to reshape is a tensor with 122496 values, but the requested shape has 0
[[Node: optimizer/gradients/energy_2/map/while/Gather_grad/Reshape = Reshape[T=DT_FLOAT, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/device:GPU:0"](optimizer/gradients/energy_2/map/while/mul_grad/tuple/control_dependency_1, optimizer/gradients/energy_2/map/while/Gather_grad/concat)]]
However, I'm getting several different errors. For this node in my graph:
[[Node: optimizer/gradients/energy_1/map/while/Gather_grad/Reshape = Reshape[T=DT_FLOAT, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/device:GPU:0"](optimizer/gradients/energy_1/map/while/mul_grad/tuple/control_dependency_1, optimizer/gradients/energy_1/map/while/Gather_grad/concat)]]
Here are some of the errors I've seen:
Size 1 must be non-negative, not -1231271574
Size 1 must be non-negative, not -1225669337
Input to reshape is a tensor with 122496 values, but the requested shape has 0
Input to reshape is a tensor with 122496 values, but the requested shape has 1715491170492
Input to reshape is a tensor with 122496 values, but the requested shape has 1693172050944
Input to reshape is a tensor with 122496 values, but the requested shape has 1706639062128
For this node in my graph:
[[Node: optimizer/gradients/energy_1/map/while/Mean_grad/Reshape = Reshape[T=DT_FLOAT, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/device:GPU:0"](optimizer/gradients/energy_1/map/while/TensorArrayWrite/TensorArrayWriteV3_grad/tuple/control_dependency, optimizer/gradients/energy_1/map/while/Mean_grad/DynamicStitch/_203)]]
I've seen these errors for example:
Size 0 must be non-negative, not -1237175937
Input to reshape is a tensor with 512 values, but the requested shape has 0
I also had a "Nan in summary histogram" error, but I'm going to assume that was due to model divergence.
I don't understand why I'm getting these reshape-related errors at runtime, halfway through the training process. I also don't understand why the values in these errors change each run.
Circumstances
When running the same program with the same hyperparameters on the same data with the same numpy and tensorflow random seeds, the program will sometimes run without issues, but usually throw one of these errors at different points in the training process. Sometimes this happens at the very first epoch, sometimes after many training epoch (even after 40+ epochs, shortly before the training finishes).
The strange thing is that this seems to be very dependent on the number of latent features/convolution filters in the layer before the one that throws the error. Smaller numbers of filters like 16, 32, 64, and 128 will almost always get errors related to the first computation graph node I mentioned, while a 512 filters will mostly get errors related to the second node. These hyperparameter numbers failed 7-10/10 runs.
However, running the program with that number of filters being either 1 or 1024 was successful 10/10 runs, which confuses me.
Program
The program is part of a research framework for deep learning on protein structures. The part that is giving me errors is part of a graph convolutional/message-passing network that downsamples a variable-size/shape graph to a single latent representation. The number of filters in the previous section corresponds to the number of latent features per node in the graph.
Here is the downsampling method:
def energy(input, _, in_dims, in_dists, **kwargs):
'''Params:
input: a tuple representing a single graph containing:
a 2d tensor of vertex representations(vertices x features)
a 3d tensor of distance metrics between nodes (vertices x neighbors x distances)
a 2d tensor containing indices of the neighbors of each vertex in the first tensor(vertices x neighbor indices)
in_dims: number of incoming features for each vertex
in_dists: number of distance metrics
Returns: a 1d tensor of size [in_dims*in_dists] which is the sum over all pairs of neighboring vertices of
(the outer product of (the elementwise product of the two vertices) and the distances).
'''
verts, dists, hood_indices = input
return tf.reshape(tf.reduce_mean(tf.einsum('abi,abj->abij', (tf.expand_dims(verts, axis=1) * tf.gather(verts, hood_indices)), dists), axis=[0,1]), [in_dims*in_dists]), None
Here is the map_fn where that method is called, where layer_fn is the above method:
# downsample each graph using layer_fn
input = tf.map_fn(
lambda ind, data=input[0], merge_fn=layer_fn, nv=input[0].get_shape().as_list()[-1], args=args: merge_fn(
[tf.slice(data, [tf.squeeze(tf.slice(ind, [0], [1])), 0],
[tf.squeeze(tf.slice(ind, [1], [1])), nv], name="merge_vertex_slice"),
tf.slice(self.distances, [tf.squeeze(tf.slice(ind, [0], [1])), 0, 0],
[tf.squeeze(tf.slice(ind, [1], [1])), self.in_nhood_size, self.in_dists], name="merge_distance_slice"),
tf.slice(tf.squeeze(self.in_hood_indices), [tf.squeeze(tf.slice(ind, [0], [1])), 0],
[tf.squeeze(tf.slice(ind, [1], [1])), self.in_nhood_size], name="merge_index_slice"),
], None, in_dims=nv, in_dists=self.in_dists, **args)[0],
tf.stack([tf.cumsum(self.graph_orders, exclusive=True), self.graph_orders], axis=-1),
dtype=tf.float32, parallel_iterations=32)
Stack Overflow will not allow me to post the class that builds the computation graph as it brings my post over the character limit.
The program runs without errors when using a top_k method to downsample the graph.
Confusion
I don't understand why I would get these reshape errors after successful training epochs, or why the number of filters is affecting the issue in such a way. I also don't understand why I'm getting different values in the reshape errors each time. The tensor dimensions should all be fixed except for the number of examples in a minibatch (which I handle via the map_fn) and the number of vertices in each example graph.
I'm having considerable difficulty figuring this out and would be very grateful for outside input. Thank you!
My issue ended up being that I was passing indices to tf.gather() that were larger than the size of the tensor I was trying to gather from (the hood_indices within the energy function). I'm not exactly sure how that led to the errors I was seeing, but it fixed my problem.
I had a similar error "tensorflow.python.framework.errors_impl.InvalidArgumentError: Size 0 must be non-negative, not -1610612736 [Op:Reshape]"
It turns out to be related to tf.repeat. Basically, the following simple code will get you an OOM error:
a = tf.range(5000000)
b = tf.concat([tf.zeros(5000000-1, dtype=tf.int32),tf.constant([5000000], dtype=tf.int32)], axis=0)
c = tf.repeat(a,b)
The solution is to tune args of tf.repeat, e.g. make input smaller (discard useless values), remove zeros from repeats
For more details: https://github.com/tensorflow/tensorflow/issues/46648#issuecomment-876168035

Using tf.py_func to generate input data

Python version = 3.6.3
Tensorflow version = 1.3.0
I have worked in Keras but am now trying to work directly in TensorFlow. I am trying to implement the equivelent of Keras's fit_generator whereby I don't have to have all training data loaded into memory at the start but can feed it into the network as needed for training. The code below represents my attempt to start something like that, but if I am going about this all wrong I would love to know where in the docs I should look and what keywords I should use to search for this.
My system is currently based on a generator that reads through sqlite database files to extract np.arrays and then converts them into my desired data shape (a time series with one prediction for forward). I am trying to migrate that system now to work with Tensorflow Datasets and getting stuck at applying tf.py_func. Here is how I am trying to work right now
import tensorflow as tf
import os
from tensorflow.contrib.data import Dataset, Iterator
import sqlite3
import pandas as pd
import numpy as np
LOOKBACK_ROWS = 600
DATA_DIR = '/mnt/derived_data/processedData'
files = os.listdir(DATA_DIR)
def data_from_files(f):
with sqlite3.connect(DATA_DIR + f) as conn:
results = conn.execute("SELECT col1, col2, FROM tbl")
col_names = [d[0] for d in results.description]
arr = np.array(results.fetchall())
num_obs = arr.shape[0] - LOOKBACK_ROWS + 1
X = np.zeros((num_obs, LOOKBACK_ROWS, 1), dtype = np.float32)
Y = np.zeros((num_obs, 1), dtype = np.float32)
for i in range(num_obs):
idx = i + LOOKBACK_ROWS - 1
X[i , :, 0] = arr[(idx - LOOKBACK_ROWS + 1):(idx + 1), 0]
Y[i, 0] = arr[idx, 1]
return tf.convert_to_tensor(X, name = 'X'), tf.convert_to_tensor(Y, name = 'Y')
filenames = tf.constant(files)
dataset = Dataset.from_tensor_slices((filenames))
dataset = dataset.map(lambda filename: tuple(tf.py_func(
data_from_files,
[filename],
[tf.float32, tf.float32])))
iterator = Iterator.from_structure(dataset.output_types, dataset.output_shapes)
next_element = iterator.get_next()
dataset_init_op = iterator.make_initializer(dataset)
with tf.Session() as sess:
sess.run(dataset_init_op)
while True:
try:
elem = sess.run(next_element)
print('Success')
except tf.errors.OutOfRangeError:
print('End of dataset.')
break
The initializations run fine, but then when I start the session and run I get the following errors:
2017-10-16 16:58:45.227612: I tensorflow/core/common_runtime/gpu/gpu_device.cc:976] DMA: 0
2017-10-16 16:58:45.227615: I tensorflow/core/common_runtime/gpu/gpu_device.cc:986] 0: Y
2017-10-16 16:58:45.227620: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:65:00.0)
2017-10-16 16:58:45.276138: W tensorflow/core/framework/op_kernel.cc:1192] Invalid argument: TypeError: must be str, not bytes
2017-10-16 16:58:45.276306: W tensorflow/core/framework/op_kernel.cc:1192] Invalid argument: TypeError: must be str, not bytes
[[Node: PyFunc = PyFunc[Tin=[DT_STRING], Tout=[DT_FLOAT, DT_FLOAT], token="pyfunc_0"](arg0)]]
Traceback (most recent call last):
File "/opt/python/3.6.3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1327, in _do_call
return fn(*args)
File "/opt/python/3.6.3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1306, in _run_fn
status, run_metadata)
File "/opt/python/3.6.3/lib/python3.6/contextlib.py", line 88, in __exit__
next(self.gen)
File "/opt/python/3.6.3/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py", line 466, in raise_exception_on_not_ok_status
pywrap_tensorflow.TF_GetCode(status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: TypeError: must be str, not bytes
[[Node: PyFunc = PyFunc[Tin=[DT_STRING], Tout=[DT_FLOAT, DT_FLOAT], token="pyfunc_0"](arg0)]]
[[Node: IteratorGetNext = IteratorGetNext[output_shapes=[<unknown>, <unknown>], output_types=[DT_FLOAT, DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](Iterator)]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/usr/code/nn/data_folder/pipeline.py", line 51, in <module>
elem = sess.run(next_element)
File "/opt/python/3.6.3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 895, in run
run_metadata_ptr)
File "/opt/python/3.6.3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1124, in _run
feed_dict_tensor, options, run_metadata)
File "/opt/python/3.6.3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1321, in _do_run
options, run_metadata)
File "/opt/python/3.6.3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1340, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: TypeError: must be str, not bytes
[[Node: PyFunc = PyFunc[Tin=[DT_STRING], Tout=[DT_FLOAT, DT_FLOAT], token="pyfunc_0"](arg0)]]
[[Node: IteratorGetNext = IteratorGetNext[output_shapes=[<unknown>, <unknown>], output_types=[DT_FLOAT, DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](Iterator)]]
>>> python.el: native completion setup loaded
>>>
Questions
(1) This seems like exactly a use case for py_func but am I wrong about that? If not, can anyone point me to some resources that show more in depth than the Tensorflow docs? (I did notice one potential related issue on git: https://github.com/tensorflow/tensorflow/issues/12396 but the fix of wrapping everything with a tuple did not help me).
(2) What is the general flow I should be following, particularly where I want to start with something like a bunch of filenames and output more than one training Example per file name?
Thank you.
Below I rewrote my script so that it can be a self-contained runnable example. I believe the issue is still the same as in the code above, but I am repasting the error as well to confirm.
Self contained runnable code example incorporating changes from #mrry's answer:
import tensorflow as tf
import os
import numpy as np
LOOKBACK_ROWS = 600
arr = np.random.random_sample((2000, 2))
np.save("npfile.npy", arr)
def data_from_files(f):
arr = np.load(f)
num_obs = arr.shape[0] - LOOKBACK_ROWS + 1
X = np.zeros((num_obs, LOOKBACK_ROWS, 1), dtype = np.float32)
Y = np.zeros((num_obs, 1), dtype = np.float32)
for i in range(num_obs):
idx = i + LOOKBACK_ROWS - 1
X[i , :, 0] = arr[(idx - LOOKBACK_ROWS + 1):(idx + 1), 0]
Y[i, 0] = arr[idx, 1]
return X, Y
files = ["npfile.npy"]
filenames = tf.constant(files)
# NOTE: In TensorFlow 1.4, `tf.contrib.data` is now `tf.data`.
dataset = tf.contrib.data.Dataset.from_tensor_slices(filenames)
# NOTE: In TensorFlow 1.4, the `tuple` is no longer needed.
dataset = dataset.map(lambda filename: tuple(tf.py_func(
data_from_files,
[filename],
[tf.float32, tf.float32])))
# NOTE: If you only have one `Dataset`, you do not need to use
# `Iterator.from_structure()`.
iterator = dataset.make_initializable_iterator()
next_element = iterator.get_next()
with tf.Session() as sess:
sess.run(iterator.initializer)
while True:
try:
elem = sess.run(next_element)
print('Success')
except tf.errors.OutOfRangeError:
print('End of dataset.')
break
Error:
2017-10-16 18:30:44.143668: I tensorflow/core/common_runtime/gpu/gpu_device.cc:976] DMA: 0
2017-10-16 18:30:44.143672: I tensorflow/core/common_runtime/gpu/gpu_device.cc:986] 0: Y
2017-10-16 18:30:44.143679: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:65:00.0)
2017-10-16 18:30:44.190852: W tensorflow/core/framework/op_kernel.cc:1192] Unknown: AttributeError: 'bytes' object has no attribute 'read'
2017-10-16 18:30:44.190959: W tensorflow/core/framework/op_kernel.cc:1192] Unknown: AttributeError: 'bytes' object has no attribute 'read'
[[Node: PyFunc = PyFunc[Tin=[DT_STRING], Tout=[DT_FLOAT, DT_FLOAT], token="pyfunc_0"](arg0)]]
Traceback (most recent call last):
File "/opt/python/3.6.3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1327, in _do_call
return fn(*args)
File "/opt/python/3.6.3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1306, in _run_fn
status, run_metadata)
File "/opt/python/3.6.3/lib/python3.6/contextlib.py", line 88, in __exit__
next(self.gen)
File "/opt/python/3.6.3/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py", line 466, in raise_exception_on_not_ok_status
pywrap_tensorflow.TF_GetCode(status))
tensorflow.python.framework.errors_impl.UnknownError: AttributeError: 'bytes' object has no attribute 'read'
[[Node: PyFunc = PyFunc[Tin=[DT_STRING], Tout=[DT_FLOAT, DT_FLOAT], token="pyfunc_0"](arg0)]]
[[Node: IteratorGetNext = IteratorGetNext[output_shapes=[<unknown>, <unknown>], output_types=[DT_FLOAT, DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](Iterator)]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "demo.py", line 48, in <module>
elem = sess.run(next_element)
File "/opt/python/3.6.3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 895, in run
run_metadata_ptr)
File "/opt/python/3.6.3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1124, in _run
feed_dict_tensor, options, run_metadata)
File "/opt/python/3.6.3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1321, in _do_run
options, run_metadata)
File "/opt/python/3.6.3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1340, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.UnknownError: AttributeError: 'bytes' object has no attribute 'read'
[[Node: PyFunc = PyFunc[Tin=[DT_STRING], Tout=[DT_FLOAT, DT_FLOAT], token="pyfunc_0"](arg0)]]
[[Node: IteratorGetNext = IteratorGetNext[output_shapes=[<unknown>, <unknown>], output_types=[DT_FLOAT, DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](Iterator)]]
Taking your questions in reverse order:
What is the general flow I should be following, particularly where I want to start with something like a bunch of filenames and output more than one training Example per file name?
To transform one element into many, use the Dataset.flat_map(f) transformation. This transformation lets you define a function f(x) that maps a single element x to a nested Dataset object, and then it takes care of flattening the nested datasets.
This seems like exactly a use case for py_func but am I wrong about that?
This is a use case for tf.py_func() but your program has a slight error: the tf.py_func() op expects that your function (data_from_files()) returns NumPy arrays, and not tf.Tensor objects. Simply returning X and Y should work.
With those two points answered, let's take a look at how you can rewrite your code:
import tensorflow as tf
import os
import sqlite3
import pandas as pd
import numpy as np
LOOKBACK_ROWS = 600
DATA_DIR = '/mnt/derived_data/processedData'
files = os.listdir(DATA_DIR)
def data_from_files(f):
with sqlite3.connect(DATA_DIR + f) as conn:
results = conn.execute("SELECT col1, col2, FROM tbl")
col_names = [d[0] for d in results.description]
arr = np.array(results.fetchall())
num_obs = arr.shape[0] - LOOKBACK_ROWS + 1
X = np.zeros((num_obs, LOOKBACK_ROWS, 1), dtype = np.float32)
Y = np.zeros((num_obs, 1), dtype = np.float32)
for i in range(num_obs):
idx = i + LOOKBACK_ROWS - 1
X[i , :, 0] = arr[(idx - LOOKBACK_ROWS + 1):(idx + 1), 0]
Y[i, 0] = arr[idx, 1]
return X, Y
filenames = tf.constant(files)
# NOTE: In TensorFlow 1.4, `tf.contrib.data` is now `tf.data`.
dataset = tf.contrib.data.Dataset.from_tensor_slices(filenames)
# NOTE: In TensorFlow 1.4, the `tuple` is no longer needed.
dataset = dataset.map(lambda filename: tuple(tf.py_func(
data_from_files,
[filename],
[tf.float32, tf.float32])))
# NOTE: If you only have one `Dataset`, you do not need to use
# `Iterator.from_structure()`.
iterator = dataset.make_initializable_iterator()
next_element = iterator.get_next()
with tf.Session() as sess:
sess.run(iterator.initializer)
while True:
try:
elem = sess.run(next_element)
print('Success')
except tf.errors.OutOfRangeError:
print('End of dataset.')
break
I have the same error (AttributeError: 'bytes' object has no attribute 'read') when i use tensorflow. I did these things: uninstall numpy, delete files about numpy in 'Lib\site-packages', reinstall numpy. And the error is gone. Maybe some mistakes Occurred where i updated numpy.

How to use tensorflow tf.metrics.mean_iou?

I am trying to use the inbuilt mean_iou function of tensorflow to compute the IoU score for semantic segmentation.
My code is:
#y_mask.shape == [batch_size, h * w, n_classes]
#y_mask.shape == [batch_size, h * w, n_classes]
iou = tf.metrics.mean_iou(tf.argmax(y_mask,2), tf.argmax(mask_,2), n_classes)
However I am getting the following error trace:
tensorflow.python.framework.errors_impl.FailedPreconditionError:
Attempting to use uninitialized value mean_iou/total_confusion
_matrix
[[Node: mean_iou/AssignAdd = AssignAdd[T=DT_DOUBLE, _class=["loc:#mean_iou/total_confusion_matrix"], use_locking=false
, _device="/job:localhost/replica:0/task:0/cpu:0"](mean_iou/total_confusion_matrix, mean_iou/confusion_matrix/SparseTensorDense
Add)]]
Caused by op u'mean_iou/AssignAdd', defined at:
File "sample_tf_ynet.py", line 207, in <module>
trainSeg()
File "sample_tf_ynet.py", line 166, in trainSeg
iou, cm_op = tf.metrics.mean_iou(tf.argmax(y_mask,2), tf.argmax(mask_,2), n_classes)
File "/home/meetshah1995/anaconda2/lib/python2.7/site-packages/tensorflow/python/ops/metrics_impl.py", line 782, in mean_iou
update_op = state_ops.assign_add(total_cm, current_cm)
File "/home/meetshah1995/anaconda2/lib/python2.7/site-packages/tensorflow/python/ops/gen_state_ops.py", line 75, in assign_ad
d
use_locking=use_locking, name=name)
File "/home/meetshah1995/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 763, in a
pply_op
op_def=op_def)
File "/home/meetshah1995/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2395, in create_op
original_op=self._default_original_op, op_def=op_def)
File "/home/meetshah1995/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1264, in __init__
self._traceback = _extract_stack()
FailedPreconditionError (see above for traceback): Attempting to use uninitialized value mean_iou/total_confusion_matrix
[[Node: mean_iou/AssignAdd = AssignAdd[T=DT_DOUBLE, _class=["loc:#mean_iou/total_confusion_matrix"], use_locking=false
, _device="/job:localhost/replica:0/task:0/cpu:0"](mean_iou/total_confusion_matrix, mean_iou/confusion_matrix/SparseTensorDense
Add)]]
Please guide me on the correct usage of this for semantic segmentation.
I solved it by calling
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
sess.run(tf.local_variables_initializer())
Simplest form I could come up with (3 classes):
# y_pred and y_true are np.arrays of shape [1, size, channels]
with tf.Session() as sess:
ypredT = tf.constant(np.argmax(y_pred, axis=-1))
ytrueT = tf.constant(np.argmax(y_true, axis=-1))
iou,conf_mat = tf.metrics.mean_iou(ytrueT, ypredT, num_classes=3)
sess.run(tf.local_variables_initializer())
sess.run([conf_mat])
miou = sess.run([iou])
print(miou)
prints:
[0.6127908]

Trying to implement recurrent network with tf.scan()

I am trying to implement a recurrent state tensor using tf.scan. The code I have at the moment is this:
import tensorflow as tf
import math
import numpy as np
INPUTS = 10
HIDDEN_1 = 20
BATCH_SIZE = 3
def iterate_state(prev_state_tuple, input):
with tf.name_scope('h1'):
weights = tf.get_variable('W', shape=[INPUTS, HIDDEN_1], initializer=tf.truncated_normal_initializer(stddev=1.0 / math.sqrt(float(INPUTS))))
biases = tf.get_variable('bias', shape=[HIDDEN_1], initializer=tf.constant_initializer(0.0))
matmuladd = tf.matmul(inputs, weights) + biases
unpacked_state, unpacked_out = tf.split(0,2,prev_state_tuple)
prev_state = unpacked_state
state = 0.9* prev_state + 0.1*matmuladd
output = tf.nn.relu(state)
return tf.concat(0,[state, output])
def data_iter():
while True:
idxs = np.random.rand(BATCH_SIZE, INPUTS)
yield idxs
with tf.Graph().as_default():
inputs = tf.placeholder(tf.float32, shape=(BATCH_SIZE, INPUTS))
with tf.variable_scope('states'):
initial_state = tf.zeros([HIDDEN_1],
name='initial_state')
initial_out = tf.zeros([HIDDEN_1],
name='initial_out')
concat_tensor = tf.concat(0,[initial_state, initial_out])
states, output = tf.scan(iterate_state, inputs,
initializer=concat_tensor, name='states')
sess = tf.Session()
# Run the Op to initialize the variables.
sess.run(tf.initialize_all_variables())
iter_ = data_iter()
for i in xrange(0, 2):
print ("iteration: ",i)
input_data = iter_.next()
out,st = sess.run([output,states], feed_dict={ inputs: input_data})
However, I get this error when running this:
Traceback (most recent call last):
File "cycles_in_graphs_with_scan.py", line 37, in <module>
initializer=concat_tensor, name='states')
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 442, in __iter__
raise TypeError("'Tensor' object is not iterable.")
TypeError: 'Tensor' object is not iterable.
(tensorflow)charlesq#Leviathan ~/projects/stuff $ python cycles_in_graphs_with_scan.py
Traceback (most recent call last):
File "cycles_in_graphs_with_scan.py", line 37, in <module>
initializer=concat_tensor, name='states')
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 442, in __iter__
raise TypeError("'Tensor' object is not iterable.")
TypeError: 'Tensor' object is not iterable.
I've already tried with pack/unpack and concat/split but I get this same error.
Any ideas how to solve this problem?
You're getting an error because tf.scan() returns a single tf.Tensor, so the line:
states, output = tf.scan(...)
...cannot destructure (unpack) the tensor returned from tf.scan() into two values (states and outputs). Effectively, the code is trying to treat the result of tf.scan() as a list of length 2, and assign the first element to states and the second element to output, but—unlike a Python list or tuple—tf.Tensor does not support this.
Instead you need to extract the values from the result of tf.scan() manually. For example, using tf.split():
scan_result = tf.scan(...)
# Assumes values are packed together along `split_dim`.
states, output = tf.split(split_dim, 2, scan_result)
Alternatively, you could use tf.slice() or tf.unpack() to extract the relevant states and output values.