I am trying to train a model using Tensorflow's SVM canned estimator. My input data is a queue sourced from Big Query using Tensorflow's BigQueryReader which then feeds a shuffle_batch queue. My features are Rank 2 and my example keys are Rank 1. The actual shape of the key and feature tensors aren't fully defined since the number of training/eval examples can vary. I am getting the following error during training:
File "/root/.local/lib/python2.7/site-
packages/tensorflow/contrib/linear_optimizer/python/ops/sdca_ops.py",
line 313, in minimize example_state_data =self._hashtable.lookup(example_ids_hashed)
File "/root/.local/lib/python2.7/site-packages/tensorflow/contrib/linear_optimizer/python/ops/sharded_mutable_dense_hashtable.py",
line 108, in lookup self._check_keys(keys)
File "/root/.local/lib/python2.7/site-packages/tensorflow/contrib/linear_optimizer/python/ops/sharded_mutable_dense_hashtable.py",
line 99, in _check_keys keys.get_shape())
ValueError: Key shape must be fully defined, got (?, 2)
The issue seems to be tied specifically to the processing of the example key tensor. Has anyone else run into this issue? I have tried very similar code with the other canned estimators and haven't gotten a complaint about the shape not being fully defined. However, this is the first estimator that required keys. When the data is parsed by parse_example after being read from BigQuery, the output are Rank-2 tensors with shape (?,1). In order to create a rank-1 tensor for the keys I am using the reshape function which results in shape (?,). I'm not sure if this could be contributing to my problem:
features = tf.parse_example(examples_serialized, features=FEATURES)
features[key_column] = tf.reshape(features[key_column],[-1])
Related
I'm trying to apply federated learning to an existing keras model that takes two inputs. When I call tff.learning.from_compiled_keras_model and include a dummy batch, I get this error: ValueError: Layer model_1 expects 2 inputs, but it received 1 input tensors. Inputs received: [<tf.Tensor 'packed:0' shape=(2, 20) dtype=int64>].
The model accepts two numpy arrays as inputs, so I defined my dummy_batch as:
x = tf.constant(np.random.randint(1,100, size=[20]))
collections.OrderedDict([('x', [x, x]), ('y', x)])
I dug around a little bit and saw that eventually, tf.convert_to_tensor_or_sparse_tensor gets called on the input list (in the __init__ for _KerasModel), and that returns a single tensor of shape (2,20), instead of two separate arrays or tensors. Is there some other way I can represent the list of inputs to avoid this issue?
The TFF team just pushed a commit that should contain this bugfix; this commit should be what you want. See in particular the change in tensorflow_federated/python/learning/model_utils_test.py--the added test case should have been a repro of your issue, and it now passes.
You were right to call out our call to tf.convert_to_tensor_or_sparse_tensor; now we use tf.nest.map_structure to map this function call to the leaves of the passed-in data structure. Note that Keras does some extra input normalization as well; we decided not to duplicate that logic here.
This change won't be in the pip package until the next release, but if you build from source, it will be available now.
Thanks for this catch, and pointing to the right place!
I am trying to predict() the output for a single data point d, using my trained Keras model loaded from a file. But I get a ValueError If predicting from data tensors, you should specify the 'step' argument. What does that mean?
I tried setting step=1, but then I get a different error ValueError: Cannot feed value of shape () for Tensor u'input_1:0', which has shape '(?, 600)'.
Here is my code:
d = np.concatenate((hidden[p[i]], hidden[x[i]])).resize((1,600))
hidden[p[i]] = autoencoder.predict(d,steps=)
The model is expecting (?,600) as input. I have concatenated two numpy arrays of shape (300,) each to get (600,), which is resized to (1,600). This (1,600) is my input to predict().
In my case, the input to predict was None (because I had a bug in another part of the code).
In official doc, steps refer to the total number of steps before stopping. So steps=1 means make predictions on one batch instead of making prediction on one record (single data point).
https://keras.io/models/sequential/
-> Define value of steps argument,
d = np.concatenate((hidden[p[i]],
hidden[x[i]])).resize((1,600))
hidden[p[i]] = autoencoder.predict(d,steps=1)
If you are using a test data generator, it is good practice to define the steps, as mentioned in the documentation.
If you are predicting a single instance, no need to define the steps. Just make sure the argument (i.e. instance 'd') is not None, otherwise that error will show up. Some reshaping may also be necessary.
in my case i got the same error, i just reshaped the data to predict with numpy function reshape() to the shape of the data originally used to train the model.
I've a Keras model that when I fit fails with this error
> kerasInput = Input(shape=(None, 47))
> LSTM(..)(kerasInput)
...
> model.fit(realInput, ...)
ValueError: Error when checking input: expected input_1 to have 3 dimensions, but got array with shape (10842, 1)
When looking at my input I found it has a shape of (10842, 1) but for each row it's actually a list of list. I can verify with
> pd.DataFrame(realInput[0]).shape
(260, 47)
How I could correct my input shape?
When trying with keras Reshape layer, the creation of the model fails with:
Model inputs must come from `keras.layers.Input` (thus holding past layer metadata), they cannot be the output of a previous non-Input layer. Here, a tensor specified as input to your model was not an Input tensor, it was generated by layer reshape_8.
Note that input tensors are instantiated via `tensor = keras.layers.Input(shape)`.
The tensor that caused the issue was: reshape_8/Reshape:0
You can use numpy.expand_dims method to convert the shape to 3D.
import numpy as np
np.expand_dims(realInput,axis=0)
Reshape layer keras
https://keras.io/layers/core/#reshape
Use the third parameter as 1
# Something Similar to this
X_train = np.reshape(X_train,(X_train.shape[0],X_train.shape[1],1))
Edit: Added np.reshape method
Refer this repository: https://github.com/NilanshBansal/Stock_Price_Prediction/blob/master/Stock_Price_Prediction_20_days_later_4_LSTM.ipynb
As I said before in the comments. You will need to make sure to reshape your data to match what LSTM expects to receive and also make sure the input_shape is correctly set.
I found this post quite helpful when I struggled with inputting to an LSTM layer. I hope it helps you too : Reshape input for LSTM
I am trying to experiment with a multi-layer encoder-decoder type of network. The screenshot of the last several layers of network architecture is as follows. This is how I setup model compiling and training process.
optimizer = SGD(lr=0.001, momentum=0.9, decay=0.0005, nesterov=False)
autoencoder.compile(loss="sparse_categorical_crossentropy", optimizer=optimizer, metrics=['accuracy'])
model.fit(imgs_train, imgs_mask_train, batch_size=batch_size, nb_epoch=nb_epoch, verbose=1,callbacks=[model_checkpoint])
imgs_train and imgs_mask_train are of shape (2000, 1, 128, 128). imgs_train represent the raw image and imgs_mask_train represents the mask image. I am trying to solve a semantic segmentation problem. However, running the program generates the following error message, (I only keep the main related part).
tensorflow.python.pywrap_tensorflow.StatusNotOK: Invalid argument: logits first dimension must match labels size. logits shape=[4096,128] labels shape=[524288]
[[Node: SparseSoftmaxCrossEntropyWithLogits = SparseSoftmaxCrossEntropyWithLogits[T=DT_FLOAT, Tlabels=DT_INT64, _device="/job:localhost/replica:0/task:0/cpu:0"](Reshape_364, Cast_158)]]
It seems to me that the loss function of sparse_categorical_crossentropy causes the problem for the current (imgs_train, imgs_mask_train) shape setting. The Keras API does not include the detail about how to setup the target tensor. Any suggestions are highly appreciated!
I am currently trying to figure the same problem and as far as I can tell it takes a sparse representation of the target category. That means integers as the target label instead of the one-hot encoded binary class matrix.
Concerning your problem, do you have categories in your masking or do you just have information about the outline of an object? With outline information it becomes a pixel wise binary loss instead of a categorical one. If you have categories, the output of your decoder should have dimensionality (None, number_of_classes, 128, 128). On that you should be able to use a sparse target mask but I haven't tried this myself...
Hope that helps
I am very new to tensorflow. Now I am implementing a pairwise cnn model. While I got the error:
tensorflow.python.pywrap_tensorflow.StatusNotOK: Not found:
FeedInputs: unable to find feed output Placeholder:0
The error is attributed to the code:
_,train_loss,train_feature1,train_feature2=sess.run([model.train_op,model.cost,model.feature1,model.feature2],feed)
Specifically in sess.run function, feed is a dictionary containing input data and corresponding labels. feature1 and feature2 are the output of the same neural networks and finally I would like to compare feature1 and feature2.
The gist codes are:
with self.graph.device('/gpu:1'):
self.inputs1=tf.placeholder(tf.float32,[self.batchSize,self.inputH,self.inputW,hp.channel])
self.inputs2=tf.placeholder(tf.float32,[self.batchSize,self.inputH,self.inputW,hp.channel])
self.labels=tf.placeholder(tf.float32,[self.batchSize])
self.feature1=forward(self.inputs1)
self.feature2=forward(self.inputs2)
self.cost=tf.reduce_sum( tf.mul( -1.0*self.labels, tf.sub( self.feature1,self.feature2 ) ) )
self.tvars=tf.trainable_variables()
grads=tf.gradients(self.cost,self.tvars)
optimizer=tf.train.MomentumOptimizer(learning_rate=self.learning_rate,momentum=self.momentum)
self.train_op=optimizer.apply_gradients(zip(grads,self.tvars))
Here I create three placeholders, and do some operations like train_op. Forward is the neural networks used in my implementation. Then in the training part, I load text data and labels with data_loader.next_batch(), finally run the optimization operation with sess.run(). The codes is available below.
for b in xrange(data_loader.batch_num):
pbar.update(int((b/data_loader.batch_num)*100) )
time.sleep(0.01)
maps1,maps2,labels=data_loader.next_batch()
feed={model.labels:labels,model.inputs1:maps1,model.inputs2:maps2}
_,train_loss,train_feature1,train_feature2=sess.run([model.train_op,model.cost,model.feature1,model.feature2],feed)
NEW ERROR
I tried to change the maps1 as Tensors instead of numpy ndarray, the error above disappeared. Unfortunately new errors raised.
File "Train.py", line 55, in train _,train_loss,train_feature1,train_feature2=sess.run([model.train_op,model.cost,model.feature1,model.feature2],feed)
File "/usr/lib/python2.7/site-packages/tensorflow/python/client/session.py",
line 334, in run
np_val = np.array(subfeed_val, dtype=subfeed_t.dtype.as_numpy_dtype)
ValueError: setting an array element with a sequence.