Multiclass classification label error using TensoFlow - tensorflow

I am doing a multiclass classification using TensorFlow. Target has 4 values[0, 1, 13, 14]. That is why I took 4 in the last Dense Layer. I am using sparse_categorical_crossentropy in my loss function. My code is as follows:-
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import LabelEncoder
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Flatten, Conv1D, MaxPool1D, Dropout, Conv2D
import matplotlib.pyplot as plt
x_train, x_test, y_train, y_test = train_test_split(X, data['seg_type'], test_size=0.33, random_state=1)
#The known number of output classes.
labels = [0, 1, 13, 14]
num_classes = 4
# # label encoding
# encoder = LabelEncoder()
# y_train = encoder.fit_transform(y_train)
# y_test = encoder.fit_transform(y_test)
# # one hot encoding
# y_train = tensorflow.keras.utils.to_categorical(y_train, num_classes)
# y_test = tensorflow.keras.utils.to_categorical(y_test, num_classes)
y_train = np.array(y_train)
y_test = np.array(y_test)
# build CNN model
model = Sequential()
model.add(Conv2D(32, (1, 1) , input_shape = (1, 3, 1), activation='relu'))
model.add(Conv2D(64, (1, 1) , input_shape = (1, 3, 1), activation='relu'))
model.add(Conv2D(128, (1, 1) ,activation='relu'))
model.add(Flatten()) # flatten
model.add(Dense(32, activation='relu')) # fc
model.add(Dense(64, activation='relu'))
model.add(Dense(128, activation='relu'))
model.add(Dense(4, activation='softmax'))
# model compile
model.compile(loss='sparse_categorical_crossentropy',
optimizer='adam',
metrics=['accuracy'])
model.summary()
batch_size = 32
epochs = 20
history = model.fit(x_train, y_train,
batch_size=batch_size,
epochs=epochs,
verbose=2)
But after fitting the model it shows the error: -
InvalidArgumentError: Received a label value of 14 which is outside the valid range of [0, 4). Label values: 14 14 13 13 13 14 13 13 14 1 13 13 14 14 14 1 14 13 1 14 13 14 14 13 1 14 13 14 1 14 14 14
[[node sparse_categorical_crossentropy/SparseSoftmaxCrossEntropyWithLogits/SparseSoftmaxCrossEntropyWithLogits (defined at <ipython-input-81-33f0652484e5>:6) ]] [Op:__inference_train_function_574154]
Function call stack:
train_function
Full code - My notebook
Data - Access the data here
Thanks in advance!!!!!!!!!!

As you have used four labels in your code here,
#The known number of output classes.
labels = [0, 1, 13, 14]
num_classes = 4
The valid range of your labels is 0,1,2 & 3. So the values while passing for prediction should always come from this range itself, the model does not recognizes 13 and 14 values.
You need to use any encoder to convert your input labels to valid range and do reverse while predicting for them.
You can just change your labels to range with in [0,3] for workaround.

change the loss function from sparse_categorical_crossentropy to categorical crossentroy and convert data into the categorical data
from keras.utils import to_categorical
y_data = to_categorical(y_data, classes = 4)
Now it will work, the present situation is the classes should be like [0, 1, 2, 3] but you are using [0, 1, 13, 14]. the solution for this is to categorical function

Related

Why 1D CNN give me error of shape mismatched after finish it first epoch

I am currently using a 1D CNN to classify raw signal data with 5 different class. I am now at the traning step but I have problem that after the first epoch finish it throw me an error like below:
Epoch 1/10
1406/1407 [============================>.] - ETA: 0s - loss: 1.6070 - accuracy: 0.2161
WARNING:tensorflow:Model was constructed with shape (None, 300, 1) for input KerasTensor(type_spec=TensorSpec(shape=(None, 300, 1), dtype=tf.float32, name='conv1d_11_input'), name='conv1d_11_input', description="created by layer 'conv1d_11_input'"), but it was called on an input with incompatible shape (32, 1, 1). ValueError: Exception encountered when calling layer 'conv1d_11' (type Conv1D). Negative dimension size caused by subtracting 2 from 1 for '{{node sequential_5/conv1d_11/Conv1D}} = Conv2D[T=DT_FLOAT, data_format="NHWC", dilations=[1, 1, 1, 1], explicit_paddings=[], padding="VALID", strides=[1, 1, 1, 1], use_cudnn_on_gpu=true](sequential_5/conv1d_11/Conv1D/ExpandDims, sequential_5/conv1d_11/Conv1D/ExpandDims_1)' with input shapes: [32,1,1,1], [1,2,1,16].
Call arguments received by layer 'conv1d_11' (type Conv1D):
• inputs=tf.Tensor(shape=(32, 1, 1), dtype=float32)
The model I use has structure like below:
from tensorflow.keras import layers
from tensorflow.keras import models
from keras.layers import Dense, Dropout, Flatten, Conv1D, MaxPooling1D
model = tf.keras.Sequential()
model.add(tf.keras.layers.Conv1D(16, 2, activation='relu', input_shape = input_shape))
model.add(tf.keras.layers.MaxPooling1D(2))
model.add(tf.keras.layers.Conv1D(32, 2, activation='relu',padding='valid'))
model.add(tf.keras.layers.MaxPooling1D(2))
model.add(Dropout(0.5))
model.add(tf.keras.layers.Conv1D(64, 2, activation='relu',padding='valid'))
model.add(tf.keras.layers.MaxPooling1D(2))
model.add(Dropout(0.5))
model.add(tf.keras.layers.Flatten())
model.add(tf.keras.layers.Dense(32, activation='relu'))
model.add(tf.keras.layers.Dense(16, activation='relu'))
model.add(tf.keras.layers.Dense(5, activation='softmax'))
And this is how I prepare data for the model:
time_steps = 300
num_features = 1
num_samples = 45000
input_shape = (time_steps, num_features)
batch_size =32
X_train = X_train.reshape(num_samples, time_steps, num_features)
#convert to numpy array
X_train = np.array(X_train)
y_train = y_train[:num_samples]
print(X_train.shape)
print(y_train.shape)
(45000, 300, 1)
(45000, 5)
I've tried but have no idea where I miss. Pleas help me out. If you need anymore details please leave comment.Thanks

ValueError in categorical_crossentropy loss shape

I am building a multi-class CNN model but I am unable to compile the model due to loss shape error.
Both output layer and labels should have correct shapes; labels being (m, 1, 3) and final dense layer containing 3 perceptions with softmax activation
loss='categorical_crossentropy'
import numpy as np
import pandas as pd
from preprocess import DataLoader
import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Flatten, Conv3D, Dropout, MaxPooling3D
from tensorflow.keras.metrics import categorical_crossentropy
from tensorflow.keras import optimizers
target_width = 160
target_height = 192
target_depth = 192
num_classes = 3
batch_size = 4
data_loader = DataLoader(target_shape=(target_width, target_height, target_depth))
train, test = data_loader.Get_Data_List()
print("Train size: " + str(len(train)))
print("Test size: " + str(len(test)))
def custom_one_hot(labels):
label_dict = {"stableAD":np.array([0,0,1]),
"stableMCI":np.array([0,1,0]),
"stableNL":np.array([1,0,0])}
encoded_labels = []
for label in labels:
encoded_labels.append(label_dict[label].reshape(1,3))
return np.asarray(encoded_labels)
def additional_data_prep(train, test):
# Extract data from tuples
train_labels, train_data = zip(*train)
test_labels, test_data = zip(*test)
X_train = np.asarray(train_data)
X_test = np.asarray(test_data)
y_train = custom_one_hot(train_labels)
y_test = custom_one_hot(test_labels)
return X_train, y_train, X_test, y_test
X, y, X_test, y_test = additional_data_prep(train, test)
X = np.expand_dims(X, axis=-1).reshape((X.shape[0],target_width,target_height,target_depth,1))
X_test = np.expand_dims(X_test, axis=-1).reshape((X_test.shape[0],target_width,target_height,target_depth,1))
model = Sequential()
model.add(Conv3D(24, kernel_size=(13, 11, 11), activation='relu', input_shape=(target_width,target_height,target_depth,1), padding='same', strides=4))
model.add(MaxPooling3D(pool_size=(3, 3, 3), strides=2))
model.add(Dropout(0.1))
model.add(Conv3D(48, kernel_size=(6, 5, 5), activation='relu', padding='same'))
model.add(MaxPooling3D(pool_size=(3, 3, 3), strides=2))
model.add(Dropout(0.1))
model.add(Conv3D(24, kernel_size=(4, 3, 3), activation='relu'))
model.add(MaxPooling3D(pool_size=(3, 3, 3), strides=2))
model.add(Dropout(0.1))
model.add(Conv3D(8, kernel_size=(2, 2, 2), activation='relu'))
model.add(MaxPooling3D(pool_size=(1, 1, 1), strides=2))
model.add(Dropout(0.1))
model.add(Flatten())
model.add(Dense(num_classes, activation='softmax'))
model.compile(loss='categorical_crossentropy',
optimizer=optimizers.Adam(learning_rate=0.0015),
metrics=['accuracy','categorical_crossentropy'])
model.fit(X, y, batch_size=batch_size, epochs=10, verbose=2, use_multiprocessing=True)
model.evaluate(X_test, y_test, verbose=2, use_multiprocessing=True)
Results in this error message:
Traceback (most recent call last):
File "train.py", line 70, in <module>
model.fit(X, y, batch_size=batch_size, epochs=10, verbose=2, use_multiprocessing=True)
File "/home/554282/.local/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training.py", line 819, in fit
use_multiprocessing=use_multiprocessing)
File "/home/554282/.local/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training_v2.py", line 235, in fit
use_multiprocessing=use_multiprocessing)
File "/home/554282/.local/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training_v2.py", line 593, in _process_training_inputs
use_multiprocessing=use_multiprocessing)
File "/home/554282/.local/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training_v2.py", line 646, in _process_inputs
x, y, sample_weight=sample_weights)
File "/home/554282/.local/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training.py", line 2383, in _standardize_user_data
batch_size=batch_size)
File "/home/554282/.local/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training.py", line 2489, in _standardize_tensors
y, self._feed_loss_fns, feed_output_shapes)
File "/home/554282/.local/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training_utils.py", line 810, in check_loss_and_target_compatibility
' while using as loss `' + loss_name + '`. '
ValueError: A target array with shape (8, 1, 3) was passed for an output of shape (None, 3) while using as loss `categorical_crossentropy`. This loss expects targets to have the same shape as the output.
The custom_one_hot function returns a [M, 1, 3] array. You should reshape that to [M, 3] since the output of the CNN is [M, 3]. M here is the batch size.

Value error: Shape mismatch - when using tensorflow

When running the code below I get an error, saying there is a shape mismatch, after searching on google I found this is commonly caused by using cross_entropy as a loss function when the target is not one-hot encoded. Since mine isn't one-hot encoded I used sparse_categorical_crossentropy instead, but the error is still there.
Here is the error I recieved :
Train on 60000 samples
Epoch 1/100
32/60000 [..............................] - ETA: 1:18
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-123-4ce1d4d047d4> in <module>()
----> 1 model.fit(x_train, y_train, batch_size=32, epochs=100, verbose=1)
25 frames
/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/nn_ops.py in sparse_softmax_cross_entropy_with_logits(_sentinel, labels, logits, name)
3391 "should equal the shape of logits except for the last "
3392 "dimension (received %s)." % (labels_static_shape,
-> 3393 logits.get_shape()))
3394 # Check if no reshapes are required.
3395 if logits.get_shape().ndims == 2:
ValueError: Shape mismatch: The shape of labels (received (32,)) should equal the shape of logits except
for the last dimension (received (5408, 10)).
This is my code:
import numpy as np
import tensorflow as tf
from tensorflow.keras.datasets import fashion_mnist
(x_train, y_train), (x_test, y_test) = fashion_mnist.load_data()
x_train = x_train / 255.0
x_test = x_test / 255.0
x_train = np.reshape(x_train, (-1,28,28,1))
model = tf.keras.models.Sequential()
model.add(tf.keras.layers.Conv2D(filters=64, kernel_size=3, strides=2, input_shape=[28,28,1], activation='relu'))
model.add(tf.keras.layers.Dense(units = 10, activation='softmax'))
model.compile(optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['sparse_categorical_accuracy'])
model.fit(x_train, y_train, batch_size=32, epochs=100, verbose=1)
I only just realized I had forgotten to add 'flatten' layer to the model:
model.add(tf.keras.layers.Flatten())
which has resolved the issue.
model = tf.keras.models.Sequential()
model.add(tf.keras.layers.Conv2D(filters=64, kernel_size=3, strides=2, input_shape= [28,28,1], activation='relu'))
model.add(tf.keras.layers.Flatten())
model.add(tf.keras.layers.Dense(units = 10, activation='softmax'))

ValueError: Error when checking target: expected dense_2 to have shape (None, 2) but got array with shape (321, 3)

I want to create an image classifier using keras, and train it with a few example images. Then, i will be using pre-trained models and adding a few layers at the end, but first, i want to understand keras and CNNs.
My console prints the following error:
ValueError: Error when checking target: expected dense_2 to have shape
(None, 2) but got array with shape (321, 3)
Here is my code:
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
import argparse
import sys
import time
import numpy as np
import cv2
import time
from PIL import Image
import keras
import glob
from keras.models import Sequential
from keras.models import load_model
from keras.layers import Dense, Dropout, Flatten
from keras.layers import Conv2D, MaxPooling2D
from keras.optimizers import SGD
from sklearn.preprocessing import LabelBinarizer
labels = ['buena', 'mala', 'otro']
def to_one_hot(labels, ys):
result = np.zeros((len(ys),len(labels)))
for i in range(result.shape[0]):
for j in range(result.shape[1]):
result[i,j] = int(ys[i] == labels[j])
return result
def build_dataset(labels):
num_classes = len(labels)
x = []
y = []
for label in labels:
for filename in (glob.glob('./tf_files/papas_fotos/'+label+'/*.jpg')):
img = cv2.imread(filename)
img = np.resize(img,(100,100, 3))
x.append(img)
y.append(label)
y = to_one_hot(labels, y)
# y = keras.utils.to_categorical(y, num_classes=3)
x = np.array(x)
x_train = x[20:]
y_train = y[20:]
x_test = x[:19]
y_test = y[:19]
print (x.shape, y.shape)
return x_train, y_train, x_test, y_test
model = Sequential()
# input: 100x100 images with 3 channels -> (100, 100, 3) tensors.
# this applies 32 convolution filters of size 3x3 each.
model.add(Conv2D(32, (3, 3), activation='relu', input_shape=(100, 100, 3)))
model.add(Conv2D(32, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(256, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(3, activation='softmax'))
sgd = SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(loss='categorical_crossentropy', optimizer=sgd)
x_train, y_train, x_test, y_test = build_dataset(labels)
model = load_model('thebestmodel.h5')
print (model)
model.fit(x_train, y_train, batch_size=32, epochs=20)
score = model.evaluate(x_test, y_test, batch_size=32)
model.save('thebestmodel.h5')
print (score)
What mistake am I making? I think that may be the size of my one hot encoded labels, but i can't make it work.
Thanks!
Although your code was fixed for this specific error, you're loading a saved model: model = load_model('thebestmodel.h5')
This is undoing everything before this line.

input_shape parameter mismatch error in Convolution1D in keras

I want to classify a dataset using Convulation1D in keras.
DataSet Description:
train dataset size = [340,30] ; no of sample = 340 , sample dimension = 30
test dataset size = [230,30] ; no of sample = 230 , sample dimension = 30
label size = 2
Fist I try by the following code using the information from keras site https://keras.io/layers/convolutional/
batch_size=1
nb_epoch = 10
sizeX=340
sizeY=30
model = Sequential()
model.add(Convolution1D(64, 3, border_mode='same', input_shape=(sizeX,sizeY)))
model.add(Convolution1D(32, 3, border_mode='same'))
model.add(Convolution1D(16, 3, border_mode='same'))
model.add(Dense(1))
model.add(Activation('sigmoid'))
model.compile(loss='binary_crossentropy',
optimizer='adam',
metrics=['accuracy'])
print('Train...')
model.fit(X_train_transformed, y_train, batch_size=batch_size, nb_epoch=nb_epoch,
validation_data=(X_test, y_test))
score, acc = model.evaluate(X_test_transformed, y_test, batch_size=batch_size)
print('Test score:', score)
print('Test accuracy:', acc)
it gives the following error ,
ValueError: Error when checking model input: expected convolution1d_input_1 to have 3 dimensions, but got array with shape (340, 30)
Then I have transformed the Train and Test data into 3 dimension from 2 dimension by using the following code ,
X_train = np.reshape(X_train_transformed, (X_train_transformed.shape[0], X_train_transformed.shape[1], 1))
X_test = np.reshape(X_test_transformed, (X_test_transformed.shape[0], X_test_transformed.shape[1], 1))
Then I run the modified following code ,
batch_size=1
nb_epoch = 10
sizeX=340
sizeY=30
model = Sequential()
model.add(Convolution1D(64, 3, border_mode='same', input_shape=(sizeX,sizeY)))
model.add(Convolution1D(32, 3, border_mode='same'))
model.add(Convolution1D(16, 3, border_mode='same'))
model.add(Dense(1))
model.add(Activation('sigmoid'))
model.compile(loss='binary_crossentropy',
optimizer='adam',
metrics=['accuracy'])
print('Train...')
model.fit(X_train, y_train, batch_size=batch_size, nb_epoch=nb_epoch,
validation_data=(X_test, y_test))
score, acc = model.evaluate(X_test, y_test, batch_size=batch_size)
print('Test score:', score)
print('Test accuracy:', acc)
But it shows the error ,
ValueError: Error when checking model input: expected convolution1d_input_1 to have shape (None, 340, 30) but got array with shape (340, 30, 1)
I am unable to find the dimension mismatch error here.
With the release of TF 2.0 and tf.keras, you can fairly easily update your model to work with these new versions. This can be done with the following code:
# import tensorflow 2.0
# keras doesn't need to be imported because it is built into tensorflow
from __future__ import absolute_import, division, print_function, unicode_literals
try:
%tensorflow_version 2.x
except Exception:
pass
import tensorflow as tf
batch_size = 1
nb_epoch = 10
# the model only needs the size of the sample as input, explained further below
size = 30
# reshape as you had before
X_train = np.reshape(X_train_transformed, (X_train_transformed.shape[0],
X_train_transformed.shape[1], 1))
X_test = np.reshape(X_test_transformed, (X_test_transformed.shape[0],
X_test_transformed.shape[1], 1))
# define the sequential model using tf.keras
model = tf.keras.Sequential([
# the 1d convolution layers can be defined as shown with the same
# number of filters and kernel size
# instead of border_mode, the parameter is padding
# the input_shape is (the size of each sample, 1), explained below
tf.keras.layers.Conv1D(64, 3, padding='same', input_shape=(size, 1)),
tf.keras.layers.Conv1D(32, 3, padding='same'),
tf.keras.layers.Conv1D(16, 3, padding='same'),
# Dense and Activation can be combined into one layer
# where the dense layer has 1 neuron and a sigmoid activation
tf.keras.layers.Dense(1, activation='sigmoid')
])
# the model can be compiled, fit, and evaluated in the same way
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
print('Train...')
model.fit(X_train, y_train, batch_size=batch_size, nb_epoch=nb_epoch,
validation_data=(X_test, y_test))
score, acc = model.evaluate(X_test, y_test, batch_size=batch_size)
print('Test score:', score)
print('Test accuracy:', acc)
The problem you are having comes from the input shape of your model. According to the keras documentation the input shape of the model has to be (batch, step, channels). This means that the first dimension is the number of instances you have. The second dimension is the size of each sample. The third dimension is the number of channels which in your case would only be one. Overall, your input shape would be (340, 30, 1). When you actually define the input shape in the model, you only need to specify the the second and third dimension which means your input shape would be (size, 1). The model already expects the first dimension, the number of instances you have, as input so you do not need to specify that dimension.
Can you try this?
X_train = np.reshape(X_train_transformed, (1, X_train_transformed.shape[0], X_train_transformed.shape[1]))
X_test = np.reshape(X_test_transformed, (1, X_test_transformed.shape[0], X_test_transformed.shape[1]))