I want to write the model using a custom loss, something like loss = sum(abs(y_true-y_pred)*returns), where returns is a vector containing different weights for each prediction (different vector for each dataset, and so different for training and test sets).
Here is my network using categorical_crossentropy loss. How is possible to change it?
model = Sequential()
model.add(Dense(2, activation='relu',input_shape=(X_train.shape[1],)) )
model.add(Dropout(0.1))
model.add(Dense(2, activation='relu', input_shape=(X_train.shape[1],)))
model.add(Dense(3, activation='softmax'))
callback = tf.keras.callbacks.EarlyStopping(monitor='loss', patience=50)
model.compile(loss='categorical_crossentropy', optimizer='RMSprop', metrics=['AUC'])
epochs=1000
history = model.fit(X_train, y_train, epochs=epochs, batch_size=32,validation_data=(X_test, y_test),callbacks=[callback])
You need to declare a custom loss function:
def my_loss(y_true,y_pred):
return someFunc(y_true, y_pred)
And then pass it to model.compile:
model.compile(loss=my_loss, optimizer='RMSprop', metrics=['AUC'])
Related
I am training my first ML model. I am working on a 10-class classification problem. From what I can see, the model is overfitting since there is a significant difference between the training and validation accuracy.
This is the relevant code for the model
model = keras.Sequential()
model.add(keras.Input(shape=(x_train[0].shape)))
model.add(tf.keras.layers.Conv2D(filters=32,kernel_size=3, strides = (3, 3), padding = "same", activation = "relu", kernel_regularizer=tf.keras.regularizers.l1_l2(0.01)))
model.add(tf.keras.layers.MaxPool2D(strides=2))
model.add(tf.keras.layers.Conv2D(filters=32, kernel_size=(3,3), padding='valid', activation='relu', kernel_regularizer=tf.keras.regularizers.l1_l2(0.01)))
model.add(tf.keras.layers.MaxPool2D(strides=2))
model.add(tf.keras.layers.Dropout(0.5))
model.add(tf.keras.layers.Flatten())
model.add(tf.keras.layers.Dense(128, activation='relu'))
model.add(tf.keras.layers.Dropout(0.5))
model.add(tf.keras.layers.BatchNormalization())
model.add(tf.keras.layers.Dense(10))
optimizer = tf.keras.optimizers.Adam(learning_rate=0.001/2)
model.summary()
model.compile(optimizer=optimizer,
loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
metrics=['accuracy'])
history = model.fit(x_train, y_train, epochs = 30, validation_data = (x_val, y_val), callbacks=tf.keras.callbacks.EarlyStopping(verbose=1, patience=4))
There are large fluctuations in the validation accuracy and I am not sure why.
I have tried augmenting the data and have also injected noise into the training data. (This is an audio classification problem with 10 different classes)
https://i.stack.imgur.com/TXe50.png
I am building two ANN models (ANN_1 and ANN_2). I want to use the output of the ANN_1 as input to ANN_2.
The structure of ANN_1 goes as follows:
# define the Keras model
model_1 = Sequential()
model_1.add(Dense(46, input_dim=46, activation='relu'))
model_1.add(Dense(31, input_dim=46, activation='relu'))
model_1.add(Dense(1, activation='sigmoid'))
#Training the ANN_1 model
model_1.compile(loss="mean_absolute_error", optimizer='adam')
model_1.summary()
#Training the ANN on the Training set
history = model_1.fit(X_train, y_train, epochs=800, batch_size=15, validation_data=(X_test,
y_test))
The structure of ANN_2 goes as follows:
# define the Keras model
model_2 = Sequential()
model_2.add(Dense(52, input_dim=52, activation='relu'))
model_2.add(Dense(40, activation='relu'))
model_2.add(Dense(1, activation='sigmoid'))
#Training the ANN_2 model
model_2.compile(loss="mean_absolute_error", optimizer='Adam')
model_2.summary()
#Training the ANN_2 on the Training set
history = model_2.fit(X_train_2, y_train_2, epochs=1000, batch_size=15, validation_data=
(X_test_2, y_test_2))
I have a simple MLP built in Keras. The shapes of my inputs are:
X_train.shape - (6, 5)
Y_train.shape - 6
Create the model
model = Sequential()
model.add(Dense(32, input_shape=(X_train.shape[0],), activation='relu'))
model.add(Dense(Y_train.shape[0], activation='softmax'))
# Compile and fit
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
model.fit(X_train, Y_train, epochs=10, batch_size=1, verbose=1, validation_split=0.2)
# Get output vector from softmax
output = model.layers[-1].output
This gives me the error:
ValueError: Error when checking input: expected dense_1_input to have shape (6,) but got array with shape (5,).
I have two questions:
Why do I get the above error and how can I solve it?
Is output = model.layers[-1].output the way to return the softmax vector for a given input vector? I haven't ever done this in Keras.
in the input layer use input_shape=(X_train.shape[1],) while your last layer has to be a dimension equal to the number of classes to predict
the way to return the softmax vector is model.predict(X)
here a complete example
n_sample = 5
n_class = 2
X = np.random.uniform(0,1, (n_sample,6))
y = np.random.randint(0,n_class, n_sample)
model = Sequential()
model.add(Dense(32, input_shape=(X.shape[1],), activation='relu'))
model.add(Dense(n_class, activation='softmax'))
# Compile and fit
model.compile(loss='sparse_categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
model.fit(X, y, epochs=10, batch_size=1, verbose=1)
# Get output vector from softmax
model.predict(X)
Y_train = to_categorical(Y_train, num_classes = 10)#
random_seed = 2
X_train,X_val,Y_train,Y_val = train_test_split(X_train, Y_train, test_size = 0.1, random_state=random_seed)
Y_train.shape
model = Sequential()
model.add(Flatten())
model.add(Dense(64, activation='relu'))
model.add(Dense(128, activation='relu'))
model.add(Dense(10, activation='softmax'))
model.compile(optimizer = 'adam', loss = 'sparse_categorical_crossentropy',metrics=['accuracy'])
model.fit(X_train, Y_train, batch_size = 86, epochs = 3,validation_data = (X_val, Y_val), verbose =2)
I have to classify the MNIST data into 10 classes. I am converting the Y_train into one hot encoded array. I have gone through a number of answers but none have helped. Kindly guide me in this regard as I am a novice in ML and neural network.
It seems there is no need to use model.add(Flatten()) in your first layer. Instead of doing so, you can use a dense layer with a specific input size like: model.add(Dense(64, input_shape=your_input_shape, activation="relu").
To ensure this issue happens because of the layers, you can check whether to_categorical() function works alone with jupyter notebook.
Updated Answer
Before the model, you should reshape your model. In that case 28*28 to 784.
train_images = train_images.reshape((-1, 784))
test_images = test_images.reshape((-1, 784))
I also suggest to normalize the data that could be done by simply dividing the images to 255
After that step you should create your model.
model = Sequential([
Dense(64, activation='relu', input_shape=(784,)),
Dense(64, activation='relu'),
Dense(10, activation='softmax'),
])
Have you noticed input_shape=(784,) That is the shape of your flattened input.
Last step, compiling and fitting.
model.compile(
optimizer='adam',
loss='categorical_crossentropy',
metrics=['accuracy'],
)
model.fit(
train_images,
train_labels,
epochs=10,
batch_size=16,
)
What you do is you have just flattened the input layer without feeding the network with an input. That's why you experience an issue. The point is you should manually reshape your inputs and feed forward to the Dense() layers with parameter input_shape
I'm implementing this simple neural network, with these inputs data:
x_train = np.asarray(x_train)
y_train = np.asarray(y_train)
x_test = np.asarray(x_test)
y_test = np.asarray(y_test)
After have defined the network's structure:
model = Sequential()
model.add(Dense(20, input_dim=5, init='normal', activation='sigmoid'))
model.add(Dense(1, init='normal', activation='sigmoid'))
I run this to train and evaluate the NN:
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
history = model.fit(x_train, y_train, nb_epoch=10, validation_split=0.2)
and I always get the same result from model.fit( ... ) :
32/143 [=====>........................] - ETA: 0s.
It seems that doesn't work at all, despite I obtain consistent results on training and validation. How does I have to interpreter this stationary result about model. fit output?