Why the input size ot shape is not valid? - tensorflow

I know that similar questions were asked before, but the solutions didn't helped me.
I have the following model:
model = Sequential()
# CNN
model.add(Conv2D(filters=16, kernel_size=2, input_shape=(40, 2000, 1), activation='relu'))
model.add(MaxPooling2D(pool_size=2))
model.add(Dropout(0.2))
# CNN
model.add(Conv2D(filters=32, kernel_size=2, activation='relu'))
model.add(MaxPooling2D(pool_size=2))
model.add(Dropout(0.2))
# CNN
model.add(Conv2D(filters=64, kernel_size=2, activation='relu'))
model.add(MaxPooling2D(pool_size=2))
model.add(Dropout(0.2))
# CNN
model.add(Conv2D(filters=128, kernel_size=2, activation='relu'))
model.add(MaxPooling2D(pool_size=2))
model.add(Dropout(0.2))
model.add(GlobalAveragePooling2D())
model.add(Dense(num_labels, activation='softmax'))
optimizer = optimizers.SGD(lr=0.002, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(loss='categorical_crossentropy', metrics=['accuracy'], optimizer='adam')
I'm trying to fit the model:
model.fit(X_train, y_train_hot, batch_size=10, epochs=50,
validation_data=(X_test, y_test_hot))
where
X_train.shape = {tuple:3} (246, 40, 2000)
from other post I read (Keras input_shape for conv2d and manually loaded images) it seems that my input is right.
But I'm getting the following error:
ValueError: Input 0 of layer sequential is incompatible with the layer: : expected min_ndim=4, found ndim=3. Full shape received: [None, 40, 2000]
What am I missing ? and how can I fix it ?

As you saw from the link you posted you must reshape your data to be 4dim.
X_train = X_train.reshape(246, 40, 2000, 1)
or
X_train = X_train.reshape(-1, 40, 2000, 1)
4D: [batch_size, img_height, img_width, number_of_channels]

The error is that you did not include the 4-th axis(i.e. axis=3 or axis -1 for that matter).
Here, you can see the following:
expected min_ndim=4, found ndim=3. Full shape received: [None, 40,
2000]
The None translates to the batch size, which of course is variable and set to None.
Then, you have 40, 2000, which should correspond to height and width respectively.
At the same time, remember that you wrote in your code that the input shape your network expects is input_shape=(40, 2000, 1) not (40,2000).
You need to explicitly add the "color"/"number of channels" axis, the 3-rd channel in this case, so you need to either use reshape or expand_dims to achieve that.
For demonstrative purposes, suppose that X is of shape (40,2000), then reshape X to X = X.reshape(40,2000,1)

Related

Wrong accuracy validation set

I have a model for identification object by spectrum
model = Sequential()
model.add(Conv1D(filters = 64, input_shape=(train_generator.get_half_spec_size(160000), 1), kernel_size = 20, activation='relu'))
model.add(BatchNormalization())
model.add(Conv1D(filters = 64, kernel_size = 16, activation='relu', kernel_regularizer=regularizers.l1_l2(l1=1e-5, l2=1e-4),
bias_regularizer=regularizers.l2(1e-4),
activity_regularizer=regularizers.l2(1e-5)))
model.add(BatchNormalization())
model.add(MaxPool1D(strides=5))
model.add(Flatten())
model.add(Dense(512, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(256, activation='relu'))
#model.add(BatchNormalization())
model.add(Dense(2, activation='softmax'))
model.compile(optimizer=Adam(), loss='sparse_categorical_crossentropy', metrics=['accuracy'])
It seems that I have strange thing with 'accuracy' metric. First of all, almost in all epoches I have 0.66 accuracy for train set and 0.99 for validation set.
Okay. Than when model has learned - I just took one sample and check if validation is correct on sample. The validation was not correct (there were 1 but predicted 0), but answer of evaluate was
[2.060739278793335, 1.0]
so big loss ( expectingly ) and wrong accuracy (1.0)
Then I took another sample and predicted again. answer of 'evaluate' was
[0.18439120054244995, 1.0]
so correct loss and accuracy now (and in fact the prediction of model was correct).
My suggestion is that 'accuracy' metrics works wrong there. Or where there is a mistake?

Neural network with 3d numpys as input and linear regression (one output)

I have already make some linear regression with NN and tensorflow but my input was a pandas dataframe (X_train).
Now I would like to create a neural network with coordinates. I will have X1(x1,y1) and X2(x2,y2) as input. X1.shape: 75, 2, 120 and X2.shape: 75, 2, 120 and y.shape: 75,1
What should be the architecture of the NN network ??
I have tried this:
print('Shape X1:', np.shape(X1))
Shape X1: (75, 2, 120)
model = keras.Sequential()
model.add(layers.Dense(50, input_dim=len(X1[1]), kernel_initializer='normal', activation='relu'))
model.add(layers.Dense(50, activation='relu'))
model.add(layers.Dense(1, activation="linear"))
model.summary()
loss = 'mse'
metric = ['mse']
model.compile(loss=loss,
optimizer= tf.keras.optimizers.Adam(learning_rate=0.001),
metrics=metric)
history = model.fit(X1, y, epochs=50, validation_split=0.3, verbose=1)
Here is the error i have got:
ValueError: Input 0 of layer sequential_6 is incompatible with the layer: expected axis -1 of input shape to have value 2 but received input with shape [None, 2, 120]
Change input shape. And you have to remove one dimension:
import tensorflow as tf
X1 = tf.random.uniform((75, 2, 120))
model = tf.keras.Sequential()
model.add(tf.keras.layers.Dense(50, input_shape=(X1.shape[1], X1.shape[2]), kernel_initializer='normal', activation='relu'))
model.add(tf.keras.layers.Flatten())
model.add(tf.keras.layers.Dense(50, activation='relu'))
model.add(tf.keras.layers.Dense(1, activation="linear"))
model.summary()

Merging tensors based on a key

I am dealing with a problem in which network design is such that it requires merging output of one part of the network with a tabular input(other input) data based on a key and training the network further with the merged data. It appeared that there is no way two tensors can be merged based on a key. Hence though of converting tensor to numpy to pandas data and them merging. The merged data would be converted back to tensor and used further in the network. Below is the code for it:
def build_convnet(shape=(112, 112, 1)):
from keras.layers import Conv2D, BatchNormalization, MaxPool2D, GlobalMaxPool2D
momentum = .9
model = keras.Sequential()
model.add(Conv2D(64, (3,3), input_shape=shape,
padding='same', activation='relu'))
model.add(Conv2D(64, (3,3), padding='same', activation='relu'))
model.add(BatchNormalization(momentum=momentum))
model.add(MaxPool2D())
model.add(Conv2D(128, (3,3), padding='same', activation='relu'))
model.add(Conv2D(128, (3,3), padding='same', activation='relu'))
model.add(BatchNormalization(momentum=momentum))
model.add(MaxPool2D())
model.add(Conv2D(256, (3,3), padding='same', activation='relu'))
model.add(Conv2D(256, (3,3), padding='same', activation='relu'))
model.add(BatchNormalization(momentum=momentum))
model.add(MaxPool2D())
model.add(Conv2D(512, (3,3), padding='same', activation='relu'))
model.add(Conv2D(512, (3,3), padding='same', activation='relu'))
model.add(BatchNormalization(momentum=momentum))
# flatten...
model.add(GlobalMaxPool2D())
return model
def action_model(shape=(3, 112, 112, 1)):
from keras.layers import TimeDistributed, GRU, Dense, Dropout, Concatenate
# Create our convnet with (224, 224, 3) input shape
convnet = build_convnet(shape[1:])
# then create our final model
model = keras.Sequential()
# add the convnet with (5, 224, 224, 3) shape
model.add(TimeDistributed(convnet, input_shape=shape))
# here, you can also use GRU or LSTM
model.add(GRU(64))
# and finally, we make a decision network
model.add(Dense(1024, activation='relu'))
model.add(Dropout(.5))
model.add(Dense(512, activation='relu'))
model.add(Dropout(.5))
model.add(Dense(128, activation='relu'))
model.add(Dropout(.5))
model.add(Dense(64, activation='relu'))
model.add(Dense(4, activation='relu'))
return model
# create the tab_data and cnn_gru models
tab_dt = keras.Input(shape=(trainX.shape[1],))
cnn_gru = action_model(X_train.shape[1:])
# converting tensor to numpy array and merging with a tabular data on a key(Patient)
cnn_gru_np = cnn_gru.output.eval()
cnn_gru_pd = pd.Dataframe(cnn_gru_np, names = ["V1", "V2", "V3", "V4"])
cnn_gru_pd["Patient"] = train_p
tab_dt_np = tab_dt.eval()
tab_dt_pd = pd.Dataframe(tab_dt_np, names = ["Weeks", "Percent", "Age", "Sex_Male", "SmokingStatus_Ex-smoker", "SmokingStatus_Never smoked"])
tab_dt_pd["Patient"] = train_p.numpy()
combinedInput_pd = pd.merge(tab_dt_pd, cnn_gru_pd, on = ["Patient"], how = "left")
combinedInput_pd.drop(["Patient"], axis = 1, inplace = True)
combinedInput_np = np.array(combinedInput_pd)
combinedInput = tf.convert_to_tensor(combinedInput_np)
# being our regression head
x = Dense(8, activation="relu")(combinedInput)
x = Dense(1, activation="relu")(x)
model = Model(inputs=[tab_dt, cnn_gru.input], outputs=x)
I am getting the below error for eval function in the line "cnn_gru_np = cnn_gru.output.eval()"
ValueError: Cannot evaluate tensor u`enter code here`sing `eval()`: No default session is registered. Use `with sess.as_default()` or pass an explicit session to `eval(session=sess)`
Please help with suggesting what is going wrong here.
The reason you're getting a ValueError is that the output of a keras model isn't an eager tensor, and thus does not support eval like that.
Just try
some_model = keras.Sequential([keras.layers.Dense(10, input_shape=(5,))])
print(type(some_model.output))
print(type(tf.zeros((2,))))
some_model.output.eval()
# <class 'tensorflow.python.framework.ops.Tensor'>
# <class 'tensorflow.python.framework.ops.EagerTensor'>
# ValueError
However, there is a bigger problem with your approach: there is no connected computation graph from your models inputs to your models outputs because none of the pandas stuff are tensorflow ops. I.E. even if you were able to use eager tensors, you still wouldn't be able to train your model with automatic differentiation.
You're going to have to specify your entire model in tf I'm afraid.
Maybe you could do the data processing before giving it as input to the model? Then you only need split concat ops to put everything together?

Keras input shape mismatch error for multi-feature CNN classification model

Here is my code:
model = Sequential()
model.add(Conv1D(32, kernel_size=3,
activation='relu',
input_shape=(14,1)))
model.add(MaxPooling1D(pool_size=1))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(128, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(1, activation='softmax'))
model.summary()
model.compile(loss=keras.losses.categorical_crossentropy,
optimizer=keras.optimizers.Adadelta(),
metrics=['accuracy'])
model.fit(X_train.values, y_train.values,
batch_size=4,
epochs=1,
verbose=2,
validation_data=(X_test.values,y_test.values))
And error is:
Error when checking input: expected conv1d_35_input to have 3 dimensions, but got array with shape (13166, 14)
As suggested by other posts, I tweaked with flatten layer before output layer but that did not work.
My X_train.values.shape gives (13166, 14)
Any suggestion how should I fix this?
You need to reshape the X_train.values from (13166, 14) to (13166, 14, 1) as your input shape of CNN network is (None, 14, 1).
This may solve your problem:
X_train.values.reshape([-1,14,1])

Error when checking target: expected dense_18 to have shape (1,) but got array with shape (10,)

Y_train = to_categorical(Y_train, num_classes = 10)#
random_seed = 2
X_train,X_val,Y_train,Y_val = train_test_split(X_train, Y_train, test_size = 0.1, random_state=random_seed)
Y_train.shape
model = Sequential()
model.add(Flatten())
model.add(Dense(64, activation='relu'))
model.add(Dense(128, activation='relu'))
model.add(Dense(10, activation='softmax'))
model.compile(optimizer = 'adam', loss = 'sparse_categorical_crossentropy',metrics=['accuracy'])
model.fit(X_train, Y_train, batch_size = 86, epochs = 3,validation_data = (X_val, Y_val), verbose =2)
I have to classify the MNIST data into 10 classes. I am converting the Y_train into one hot encoded array. I have gone through a number of answers but none have helped. Kindly guide me in this regard as I am a novice in ML and neural network.
It seems there is no need to use model.add(Flatten()) in your first layer. Instead of doing so, you can use a dense layer with a specific input size like: model.add(Dense(64, input_shape=your_input_shape, activation="relu").
To ensure this issue happens because of the layers, you can check whether to_categorical() function works alone with jupyter notebook.
Updated Answer
Before the model, you should reshape your model. In that case 28*28 to 784.
train_images = train_images.reshape((-1, 784))
test_images = test_images.reshape((-1, 784))
I also suggest to normalize the data that could be done by simply dividing the images to 255
After that step you should create your model.
model = Sequential([
Dense(64, activation='relu', input_shape=(784,)),
Dense(64, activation='relu'),
Dense(10, activation='softmax'),
])
Have you noticed input_shape=(784,) That is the shape of your flattened input.
Last step, compiling and fitting.
model.compile(
optimizer='adam',
loss='categorical_crossentropy',
metrics=['accuracy'],
)
model.fit(
train_images,
train_labels,
epochs=10,
batch_size=16,
)
What you do is you have just flattened the input layer without feeding the network with an input. That's why you experience an issue. The point is you should manually reshape your inputs and feed forward to the Dense() layers with parameter input_shape