Incompatible input shape - tensorflow

I have a CNN-LSTM model where the CNN model takes as input data with shape (None, 301,4,1) and outputs data with shape (None, 606). To adapt the cnn output to the input of the LSTM, I added a TimeDistributed Layer where it call the CNN model each window-size=100, so the input shape of this layer=(None, 100,301,4,1), and then we have some stacked LSTM layers.
This is the architecture of the CNN model:
This is the architecture of the LSTM model:
The code for this architecture is the following :
input_layer1=Input(shape=(301,4,1))
...
merge_layer=Concatenate(axis=1)([global_max_pooling, lambda_14])
cnn_model = Model(inputs= input_layer1, outputs=merge_layer) cnn_model.compile(optimizer=RMSprop(),loss="mean_squared_error",metrics=['mse', 'mae'])
input_lstm = Input(shape=(100,301,4,1))
cnn_output = TimeDistributed(cnn_model)(input_lstm)
...
output_layer=Dense(1,activation="linear")(lstm3)
cnn_lstm_model = Model(inputs= input_lstm, outputs=output_layer)
cnn_lstm_model.compile(optimizer=RMSprop(),loss="mean_squared_error",metrics=['mse', 'mae'])
Then saved only the cnn_lstm_model model.
For the training, this is my code:
batchsize=100
epoch=20
cnn_lstm_model.fit(train_data_force_temp_X,data_Y,
batch_size=batchsize,
epochs=epoch,
verbose=1,
shuffle=True,
validation_data=(test_data_force_temp_X,test_Y),
callbacks=[TensorBoard(log_dir="./CNN_LSTM")])
Where train_data_force_temp_X.shape = (1960, 301, 4, 1) , PS: 1960 is number of samples.
But I have this issue :
ValueError: Input 0 is incompatible with layer model_1: expected
shape=(None, 100, 301, 4, 1), found shape=(None, 301, 4, 1)
I understood that passed the wrong shape to the cnn_lstm_model but I thought that it will pass the data first to the cnn model which have the shape=(None, 301, 4, 1) and then for each 100 CNN outputs it will call the time distributed layer and continue the process, It seems I have not understood the process correctly.
So my question is :
do I have to run the data first into the cnn model, do prediction and
then used those outputs as an input for the cnn_lstm model ?
How can I fix the training process ?
Thank you in advance for the help.

Related

LSTM: return_sequences has to be set "true" or validation is not working

I'm working on timeseries prediction LSTM.
My problem is model compiles and fit only if
model.add(LSTM(lstmUnits, activation=activation_, return_sequences=True))
model.add(Dense(features, activation=activationOutput_))
model.fit(bx,by,epochs=maxEpoch,validation_split=0.2)
but it does not fit if return_sequences is set to False.
model.add(LSTM(lstmUnits, activation=activation_, return_sequences=False))
model.add(Dense(features, activation=activationOutput_))
model.fit(bx,by,epochs=maxEpoch,validation_split=0.2)
In that case I have got an error like
ValueError: Shapes (None, 30, 2) and (None, 2) are incompatible
pointing fix function line.
Is this possible that fix() build-in validation require LSTM output to be "sequenced" ?
So how can I validate time prediction LSTM?

Efficientnet inputshape error with tf.data.Dataset

when feeding a tf.data.Dataset to train EfficientnetB0 model I get the following error:
ValueError: in converted code:
C:\Users\fconrad\AppData\Local\Continuum\anaconda3\envs\venv_spielereien\lib\site-packages\tensorflow_core\python\keras\engine\training_v2.py:677 map_fn
batch_size=None)
C:\Users\fconrad\AppData\Local\Continuum\anaconda3\envs\venv_spielereien\lib\site-packages\tensorflow_core\python\keras\engine\training.py:2410 _standardize_tensors
exception_prefix='input')
C:\Users\fconrad\AppData\Local\Continuum\anaconda3\envs\venv_spielereien\lib\site-packages\tensorflow_core\python\keras\engine\training_utils.py:573 standardize_input_data
'with shape ' + str(data_shape))
ValueError: Error when checking input: expected efficientnet-b0_input to have 4 dimensions, but got array with shape (224, 224, 3)
I realy wonder why this happens, since when I create a batch from my Dataset:
train_generator = (tf.data.Dataset
.from_tensor_slices((train_imgs, train_labels))
.map(read_img)
.map(flip_img)
.map(brightness)
.map(blur)
.map(noise)
.map(rotate_90)
.repeat()
.shuffle(512)
.batch(BATCH_SIZE)
.prefetch(True))
validation_generator = (tf.data.Dataset
.from_tensor_slices((validation_imgs, validation_labels))
.map(read_img)
)
print(train_generator.__iter__().__next__()[0].shape)
I get the expected result (64, 224, 224, 3).
But after creating the model the error above raises when I start training:
effn = tfkeras.EfficientNetB0(include_top=False, input_shape=img_shape, classes=4)
effn_model = tf.keras.Sequential()
effn_model.add(effn)
effn_model.add(tf.keras.layers.GlobalAveragePooling2D())
effn_model.add(tf.keras.layers.Dense(4, 'softmax'))
effn_model.compile(optimizer= 'adam', loss='categorical_crossentropy', metrics= ['categorical_accuracy'])
effn_model.fit(train_generator,
epochs=20,
steps_per_epoch=train_imgs.shape[0] // BATCH_SIZE,
validation_data= validation_generator)
Does anyone know why the slices from dataset have shape (64,224,224,3) but the model doesnt recognize the batch dimension? when I try to train a keras.application model, everything works fine.
I use tensorflow 2.1 and the pip install of efficientnet. Thanks
as explained here keras.io/api/applications/efficientnet/
input_shape: Optional shape tuple, only to be specified if include_top is False. It should have exactly 3 inputs channels.
as so try this->
from tensorflow.keras.applications.efficientnet import EfficientNetB0, EfficientNetB5
mm = EfficientNetB0(include_top=True, weights=None, input_tensor=None, input_shape=(128, 128, 3), pooling=None, classes=2, classifier_activation="sigmoid")
mm.summary()
note the input_shape=(128, 128, 3) It has 3 channels.

Issues with Keras Conv1D and VGG

I trying to build a deep learning model with VGG16 on top. I have implemented it in Keras using following code:
image_input = Input(shape=(224, 224, 3))
model = VGG16(input_tensor=image_input, include_top=True,weights='imagenet')
model.summary()
fc7 = model.get_layer('fc2').output
conv1d = Conv1D(1,5,activation='relu', name="conv1d",input_shape=(1,4096)) (fc7) #error appears here
# flat = Flatten()(conv1d)
fc8 = Dense(512, activation='relu', name="fc8")(conv1d)
#x= Flatten(name='flatten')(last_layer)
out = Dense(num_classes, activation='softmax', name='output')(fc8)
custom_vgg_model = Model(image_input, out)
custom_vgg_model.summary()
I am getting the following error:
ValueError: Input 0 is incompatible with layer conv1d: expected ndim=3, found ndim=2
Why can't we do the consecutive feature vectors 1d convolution like in the image below?
enter link description here
A fully connected layer in a VGG is 2D, and a 1D convolutional layer expects 3D data.
At the point where VGG adds a Dense layer, it destroys the image format (4D) with a flatten or a global pooling, transforming it into plain data (2D). You no longer have dimensions to use convolutions.
If you try to explain why you want a Conv1D, what do you expect from it, then we could think of an alternative.
Example model:
movie_data = any_data_with_shape((number_of_videos, frames, 224, 224, 3))
movie_input = Input((None,224,224,3)) #None means any number of frames
vgg = VGG16(include_top=True,weights='imagenet')
This part is only necessary if you're getting intermediary outputs from vgg:
vgg_in = vgg.input
vgg_out = vgg.get_layer('fc2').output #make sure this layer exists
vgg = Model(vgg_in, vgg_out)
Continue:
vgg_outs = TimeDistributed(vgg)(movie_input) #out shape (None, frames, fc2_units)
outs = Conv1D(.....)(vgg_outs)
outs = GlobalAveragePooling1D()(outs)
outs = Dense(....)(outs)
.....
your_model = model(move_input, outs)

TFLearn LSTM Time Series Classification

I am trying to build an LSTM network which takes a sequence and classifies the last time step in each sequence.
This is what I have so far:
#build
net = tf.input_data(shape=[None, 64, 17])
net = tf.lstm(net, 128, dropout=[.2,.8], return_seq=True)
net = tf.lstm(net, 128, dropout=[.2,.8], return_seq=True)
net = tf.lstm(net, 128, dropout=[.2,.8])
net = tf.fully_connected(net, 3, activation='softmax')
net = tf.regression(net, optimizer='adam', learning_rate=0.01, loss='categorical_crossentropy')
#train
model = tf.DNN(net, tensorboard_verbose=0)
model.fit(trainX, trainY, validation_set=(testX,testY), show_metric=True, batch_size=None)
My data has been shaped into a large number of sequences with each being 64 timesteps long. each timestep has 17 features. The first sequence being timesteps 0 to 63, the second being timesteps 1 to 64, etc.
The network builds just fine, but in the fit method I get this error:
'ValueError: Cannot feed value of shape (64,17) for Tensor
'InputData/X:0', which has shape (?,64,17)
Anyone has a suggestion as to my problem?
It's not in your snippet, but it looks like trainX has the shape (64, 17). If so, you should reshape it o a batch of size 1:
trainX = np.expand_dims(trainX, 0) # now it's [1, 64, 17]
The same for testX.

How to load MobileNet weights with an input tensor in Keras

I'm trying to apply transfer learning to MNIST using MobileNet weights in Keras. Keras documentation to use MobileNet https://keras.io/applications/#mobilenet
Mobilenet accepts 224x224x3 as input but MNIST is 28x28x1. I'm creating a Lambda layer which can convert 28x28x1 image into 224x224x3 and send it as input to MobileNet. The following code causes
TypeError: Input layers to a Model must be InputLayer objects. Received inputs: Tensor("lambda_2/ResizeNearestNeighbor:0", shape=(?, 224, 224, 3), dtype=float32). Input 0 (0-based) originates from layer type Lambda.
height = 28
width = 28
input_image = Input(shape=(height,width,1))
def resize_image_to_inception(x):
x = K.repeat_elements(x, 3, axis=3)
x = K.resize_images(x, 8, 8, data_format="channels_last")
return x
input_image_ = Lambda(resize_image_to_inception, output_shape=(224, 224, 3))(input_image)
print(type(input_image_))
base_model = MobileNet(input_tensor=input_image_, weights='imagenet', include_top=False)