I have a pandas dataframe containing np.arrays in each cell (see the photo to understand better). Each array is 1000 samples long. However, when trying to use this as a training data in LSTM, it won't go through. enter image description here
model.fit(x_train, y_train, epochs = 15)
ValueError: Failed to convert a NumPy array to a Tensor (Unsupported object type numpy.ndarray).
How do I tackle this? Can't find an answer elsewhere, tried this:
x_train=np.asarray(x_train).astype(np.float32)
but it failed due ValueError: setting an array element with a sequence.
Is there another way to use this sort of numpy arrays as input?
I was trying to train LSTM with my pandas dataframe data containing 1000 sample long np.arrays
Related
I have a class 'tensorflow.python.framework.ops.Tensor as output and need to convert this to a numpy array.
.numpy() doesn't work because it isn't a eagerTensor.
.eval doesn't work as well, because i'm using tensorflow >2.0
Is there any other way to fix this?
img_height=330
img_width=600
img_depth=23
save_model="saved_Models/wheatModel"
prediction_data_path=["data/stacked/MOD13Q1.A2017.2738.tif","data/stacked/MOD13Q1.A2017.889.tif","data/stacked/MOD13Q1.A2017.923.tif"]
prediction_data=dataConv.preparePredictionData(prediction_data_path)
prediction_reshaped=dataConv.reshapeFiles(prediction_data,img_width,img_height,img_depth)
x_ds =tf.stack(prediction_reshaped)
model = tf.keras.models.load_model(save_model)
model.predict(x_ds)
image=model.get_layer(name='prediction_image').output
n,output_width,output_height,output_depth,output_channels=image.shape
print(type(image))
image=tf.reshape(image,(output_width,output_height,output_depth))
print(type(image))
image.numpy()
So in the code above.
I load my trained model
predict the given images
get the output from the next to last layer
reshape this data
Now i want to convert this tensor to an numpyarray
Unexpected array shape of the dot product result from a 2D numpy array and a constant tensor in tensorflow.
I am working on a problem that the input X needs to be preprocessed before feeding into my neural network model. The preprocess involves a lot of numpy array operations, and I found some unexpected results.
test1 = tf.constant(np.ones((3,1)),dtype=tf.float32)
test2 = np.ones((8,3,3))
test3 = np.dot(test2[1],test1)
test3.shape
I expect the result is (3,1), but the result is (3,3). In fact, no matter what the shape I put in the tf.constant, the result is always (3,3). Besides, I don't want to convert all my numpy array in the proprocess code to tensors. Any comments or ideas would be greatly appreciated!
I have an image tensor of the following dimensions
TensorShape([Dimension(1), Dimension(96), Dimension(96), Dimension(3)])
I want this tensor to be in following channel first dimension
TensorShape([Dimension(1), Dimension(2), Dimension(96), Dimension(96)])
I have tried
tf.transpose (image, perm = [0,3,1,2])
but it did not work It is returning in the same as previous.
Since, this is the requirement of Facenet algorithm , please suggest the way to do it.
You can try to convert the tensor into NumPy array and then use np.rollaxis and convert back to Tensorflow tensor.
I was trying to build a LSTM neural net with Keras to predict tags for words in a set of sentences.
The implementation is all pretty straight forward, but the surprising thing was that
given the exactly same and otherwise correctly implemented code and
using Tensorflow 1.4.0 with Keras running on Tensorflow Backend,
on some people's computers, it returned tensors with wrong dimensions, while for others it worked perfectly.
The problem occured in the following context:
First, we turned the list of training sentences (sentences as a list of word indeces) into a 2-D matrix using the pad_sequences method from Keras (https://keras.io/preprocessing/sequence/):
def do_padding(sequences, length, padding_value):
return pad_sequences(sequences, maxlen=length, padding='post',
truncating='post', value=padding_value)
train_sents_padded = do_padding(train_sents, MAX_LENGTH,
word_to_id[PAD_TOKEN])
Next, we used our do_padding method on the corresponding training labels to turn them into a padded matrix. At the same time, we used the Keras to_categorical method (https://keras.io/utils/#to_categorical) to add a one-hot encoded vector to the created label matrix (one one-hot vector for each cell in the matrix, that means for word in each training sentence):
train_labels_padded = to_categorical(do_padding(train_labels, MAX_LENGTH,
label_to_id["O"]), NUM_LABELS)
We expected the resulting shape to be 3-D: (len(train_labels), MAX_LENGTH, NUM_LABELS). Yet, we found that the resulting shape was 2-D and basically looked like this: ((len(train_labels) x MAX_LENGTH), NUM_LABELS), meaning the numbers on the two expected dimensions len(train_labels) and MAX_LENGTH were multiplied and flattened into one dimension.
Interestingly, this problem as said before only occured for about 50% of the people, using Tensorflow 1.4.0 and Keras running on Tensorflow Backend.
We managed to solve the problem by reshaping the label matrix manually:
train_labels_padded = np.reshape(train_labels_padded, (len(train_labels),
MAX_LENGTH, NUM_LABELS))
I was just wondering if any of you have experienced a similar problem and have figured out the reason why this happens.
If the input data is in numpy array format, then we can declare a placeholder in the graph and feed the placeholder with the numpy array data. However, if the input data is already in Tensor format (this is the case when we load jpg files using tf.image.decode_jpeg), then we can't feed a Tensor to a placeholder. In this case, should we use non trainable TF Variables as placeholders, and feed the Tensor to these Variables by tf.assign?
Figured it out. You can simply feed batches of Tensors to the model. The model probably has a line that looks similar to op = optimizer.minimize(loss). Then, each time sess.run(op) is called, the model will be trained on the batch provided to it. Also, each time sess.run(op) is called, we should have a different batch if we use tf.train.batch to provide the batch.