I'm trying to train a CNN that predicts if an image is an image of a cat or a dog using keras with tensorflow on my GPU, but it's taking a lot of time per epoch.
I followed a tutorial to build this CNN from scratch, so i've installed CUDA 10.0, Visual Studio community 2017, tensorflow on GPU and Keras (all of this using Spyder and Anaconda). But when i started training the CNN i opened the task manager and saw that CUDA is being used by 6-7%. It happens the same when i scan the GPU usage with NVSMI.
My GPU is an NVIDIA RTX 2060.
This is the code i'm running:
from keras.models import Sequential
from keras.layers import Convolution2D
from keras.layers import MaxPooling2D
from keras.layers import Flatten
from keras.layers import Dense
from keras.preprocessing.image import ImageDataGenerator
classifier = Sequential()
classifier.add(Convolution2D(32, (3, 3), padding = 'same', input_shape = (64, 64, 3), activation = 'relu'))
classifier.add(MaxPooling2D(pool_size = (2, 2)))
classifier.add(Convolution2D(32, (3, 3), activation = 'relu'))
classifier.add(MaxPooling2D(pool_size = (2, 2)))
classifier.add(Flatten())
classifier.add(Dense(128, activation = 'relu'))
classifier.add(Dense(1, activation = 'sigmoid'))
classifier.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['accuracy'])
train_datagen = ImageDataGenerator(
rescale=1./255,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True)
test_datagen = ImageDataGenerator(rescale=1./255)
training_set = train_datagen.flow_from_directory(
'dataset/training_set',
target_size=(64, 64),
batch_size=32,
class_mode='binary')
test_set = test_datagen.flow_from_directory(
'dataset/test_set',
target_size=(64, 64),
batch_size=32,
class_mode='binary')
classifier.fit_generator(
training_set,
steps_per_epoch=8000,
epochs=10,
validation_data=test_set,
validation_steps=2000)
I want to know if there's any chance to set an specific value for the usage of the GPU or at least to make it grow more than 6%.
Related
so I'm new right here and in Python also. I'm trying to make my own network. I found some pictures of docs and cats 15x15 and unfortunatly couldn't make this basic network...
So, these are libraries which I'm using
from tensorflow.keras.models import Sequential
from tensorflow.keras import utils
from tensorflow.keras.datasets import mnist
from tensorflow.keras.layers import Dense
import numpy as np
import matplotlib.pyplot as plt
import tensorflow as tf
import keras
from tensorflow.keras.layers import Conv2D
from tensorflow.keras.layers import MaxPooling2D
from tensorflow.keras.layers import GlobalMaxPooling2D
Body
train_dataset = tf.keras.preprocessing.image_dataset_from_directory(
'drive/MyDrive/cats vs dogs/cats vs dogs/training',
color_mode="rgb",
batch_size=32,
image_size=(150, 150),
shuffle=True,
seed=42,
validation_split=0.1,
subset='training',
interpolation="bilinear",
follow_links=False,
)
validation_dataset = tf.keras.preprocessing.image_dataset_from_directory(
'drive/MyDrive/cats vs dogs/cats vs dogs/training',
color_mode="rgb",
batch_size=32,
image_size=(150, 150),
shuffle=True,
seed=42,
validation_split=0.1,
subset='validation',
interpolation="bilinear",
follow_links=False,
)
test_dataset = tf.keras.preprocessing.image_dataset_from_directory(
'drive/MyDrive/cats vs dogs/cats vs dogs/test',
batch_size = 32,
image_size = (150, 150),
interpolation="bilinear"
)
model = Sequential()
model.add(keras.Input(shape=(150, 150, 3)))
model.add(Conv2D(32, 5, strides=2, activation="relu"))
model.add(Conv2D(32, 3, activation="relu"))
model.add(MaxPooling2D(3))
model.add(Dense(250, activation='sigmoid'))
model.add(Dense(100))
model.add(MaxPooling2D(3))
model.add(Dense(2))
model.summary()
model.compile(loss='binary_crossentropy',
optimizer='adam',
metrics=['accuracy'])
history = model.fit(train_dataset, validation_data=validation_dataset, epochs=5, verbose=2)
And I get this error
Incompatible shapes: [29] vs. [29,7,7,2]
[[node gradient_tape/binary_crossentropy/mul_1/BroadcastGradientArgs
(defined at /usr/local/lib/python3.7/dist-packages/keras/optimizer_v2/optimizer_v2.py:464)
]] [Op:__inference_train_function_4364]
Errors may have originated from an input operation.
Input Source operations connected to node
gradient_tape/binary_crossentropy/mul_1/BroadcastGradientArgs:
In[0] gradient_tape/binary_crossentropy/mul_1/Shape:
In[1] gradient_tape/binary_crossentropy/mul_1/Shape_1
I was trying to change from binary_crossentropy to categorical_crossentrapy but it didn't help, I suppose my mistake is in datasets or inputs but I don't know how to solve it :(
Really hope to find help here!
[my architecture][1]
[1]: https://i.stack.imgur.com/w4Y9N.png
You need to flatten your prediction somewhere, otherwise you are outputing an image (29 samples of size 7x7 with 2 channels), while you simply want a flat 2 dimensional logits (so shape 29x2). The architecture you are using is somewhat odd, did you mean to have flattening operation before first Dense layer, and then no "maxpooling2d" (as it makes no sense for flattened signal)? Mixing relu and sigmoid activations is also quite non standard, I would encourage you to start with established architectures rather than try to compose your own to get some intuitions.
model = Sequential()
model.add(keras.Input(shape=(150, 150, 3)))
model.add(Conv2D(32, 5, strides=2, activation="relu"))
model.add(Conv2D(32, 3, activation="relu"))
model.add(MaxPooling2D(3))
model.add(Flatten())
model.add(Dense(250, activation="relu"))
model.add(Dense(100, activation="relu"))
model.add(Dense(2))
model.summary()
I'm using a PDF to build my first Convolutional Nerual Network using cats and dogs and am encountering a consistent error. The text is: WARNING:tensorflow:sample_weight modes were
coerced from
...
to
['...']
WARNING:tensorflow:sample_weight modes were coerced from
...
to
['...']
The relevant code is pasted in two sections below. Any help would be appreciated because I'm hitting a wall in regards to this.
This top bit is working but may be relevant:
#Build the network
#Import needed layers and models from tensorflow.keras
import tensorflow as tf
from tensorflow.keras.layers import Dense, Activation, Flatten, Conv2D, MaxPooling2D, Dropout
from tensorflow.keras.models import Sequential
#Build model--Use sequential value--Most common
model = models.Sequential()
#Input layer
model.add(layers.Conv2D(32, (3,3), activation = 'relu',
input_shape = (150, 150, 3)))
model.add(layers.MaxPooling2D(2,2))
#First hidden layer
model.add(layers.Conv2D(64, (3,3), activation = 'relu'))
model.add(layers.MaxPooling2D(2,2))
#Second hidden layer
model.add(layers.Conv2D(128, (3,3), activation = 'relu'))
model.add(layers.MaxPooling2D(2,2))
#Third hidden layer
model.add(layers.Conv2D(128, (3,3), activation = 'relu'))
model.add(layers.MaxPooling2D(2,2))
#Fourth hidden layer
model.add(layers.Flatten())
model.add(layers.Dense(512, activation = 'relu'))
#Output layer
model.add(layers.Dense(1, activation = 'sigmoid'))
#
from tensorflow.keras import optimizers
#Compilation step
model.compile(loss = 'binary_crossentropy',
optimizer= 'adam',
metrics=['acc'])
#Read images from directories
from tensorflow.keras.preprocessing.image import ImageDataGenerator
train_datagen = ImageDataGenerator(rescale = 1./255)
test_datagen = ImageDataGenerator(rescale = 1./255)
train_generator = test_datagen.flow_from_directory(
train_dir,
target_size = (150, 150),
batch_size = 20,
class_mode = 'binary')
validation_generator = test_datagen.flow_from_directory(
validation_dir,
target_size = (150, 150),
batch_size = 20,
class_mode = 'binary')
Fit model with a batch generator
This part of the code is what causes the error
history = model.fit_generator(
train_generator,
steps_per_epoch = 100,
epochs = 30,
validation_data = validation_generator,
validation_steps = 50)
As a final note, this code is in Python 3 and uses the kagglecatsanddogs database from Microsoft
The below Warning is fixed in the Nightly Version of Tensorflow and will be included in the Next Stable Version, Tensorflow 2.2
WARNING:tensorflow:sample_weight modes were coerced from ... to
['...'] WARNING:tensorflow:sample_weight modes were coerced from ... to
['...']
Currently, to make if work, please install Tensorflow Nightly Version as shown below:
!pip install tf-nightly
For more details, please refer this Github Issue.
I am following a course on deep learning and I am doing right now the CNN networks the train set is 8000 photos 4000 cats and 4000 dogs the training set is 2000/2000 the size I am using for images is 64x64 with RGB. I am using Keras with 2 conv2d/maxpool layers of 32 filters a flatten layer and two dense layers of 128 and 1 output. My problem is that this setup is performing at 15 minutes per epoch and for 25 epochs that means 6 Hours of training at least plus sometimes on some epochs is freezing for sometimes at 7999/8000 I am running this on windows 10 and anaconda with python 3.7 and TensorFlow 1.13. Is this a good performance or I can improve it? I was expecting from the new Turing architecture better performances.
# -*- coding: utf-8 -*-
# Part 1 - Building the convolutional neural network
import tensorflow as tf
from keras import backend as K
config = tf.ConfigProto(intra_op_parallelism_threads=6,
inter_op_parallelism_threads=6,
allow_soft_placement=True,
device_count = {'CPU' : 1,
'GPU' : 1}
)
session = tf.Session(config=config)
K.set_session(session)
from keras.models import Sequential
from keras.layers import Conv2D
from keras.layers import MaxPooling2D
from keras.layers import Flatten
from keras.layers import Dense
# Initialising the CNN
classifier = Sequential()
# Step 1 - Convolution
classifier.add(Conv2D(32, (3, 3), input_shape = (64, 64, 3), activation = 'relu'))
# Step 2 - Pooling
classifier.add(MaxPooling2D(pool_size = (2, 2)))
# Adding a second convolutional layer
classifier.add(Conv2D(32, (3, 3), activation = 'relu'))
classifier.add(MaxPooling2D(pool_size = (2, 2)))
# Step 3 - Flattening
classifier.add(Flatten())
# Step 4 - Full connection
classifier.add(Dense(units = 128, activation = 'relu'))
classifier.add(Dense(units = 1, activation = 'sigmoid'))
# Compiling the CNN
classifier.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['accuracy'])
weights = classifier.get_weights()
#Part 2 - Fiting the CNN to the images
from keras.preprocessing.image import ImageDataGenerator
train_datagen = ImageDataGenerator(rescale = 1./255,
shear_range = 0.2,
zoom_range = 0.2,
horizontal_flip = True)
test_datagen = ImageDataGenerator(rescale = 1./255)
training_set = train_datagen.flow_from_directory('dataset/training_set',
target_size = (64, 64),
batch_size = 32,
class_mode = 'binary')
test_set = test_datagen.flow_from_directory('dataset/test_set',
target_size = (64, 64),
batch_size = 32,
class_mode = 'binary')
classifier.fit_generator(training_set,
steps_per_epoch = 8000,
epochs = 25,
validation_data = test_set,
validation_steps = 2000)
classifier.save("my first model")
Thank you
I have a model that I've trained for 75 epochs. I saved the model with model.save(). The code for training is
from keras.preprocessing.image import ImageDataGenerator
from keras.models import Sequential, load_model
from keras.layers import Conv2D, MaxPooling2D
from keras.layers import Activation, Dropout, Flatten, Dense
from keras import backend as K
# dimensions of our images.
img_width, img_height = 320, 240
train_data_dir = 'dataset/Training_set'
validation_data_dir = 'dataset/Test_set'
nb_train_samples = 4000 #total
nb_validation_samples = 1000 # total
epochs = 25
batch_size = 10
if K.image_data_format() == 'channels_first':
input_shape = (3, img_width, img_height)
else:
input_shape = (img_width, img_height, 3)
model = Sequential()
model.add(Conv2D(32, (3, 3), input_shape=input_shape))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(32, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(64, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Flatten())
model.add(Dense(64))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(1))
model.add(Activation('sigmoid'))
model.compile(loss='binary_crossentropy',
optimizer='rmsprop',
metrics=['accuracy'])
# this is the augmentation configuration we will use for training
train_datagen = ImageDataGenerator(
rescale=1. / 255,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True)
# this is the augmentation configuration we will use for testing:
# only rescaling
test_datagen = ImageDataGenerator(rescale=1. / 255)
train_generator = train_datagen.flow_from_directory(
train_data_dir,
target_size=(img_width, img_height),
batch_size=batch_size,
class_mode='binary')
validation_generator = test_datagen.flow_from_directory(
validation_data_dir,
target_size=(img_width, img_height),
batch_size=batch_size,
class_mode='binary')
model.fit_generator(
train_generator,
steps_per_epoch=nb_train_samples // batch_size,
epochs=epochs,
validation_data=validation_generator,
validation_steps=5)
model.save('model1.h5')
How do I restart training? Do I just run this code again? Or do I need to make changes and what are those changes?
I read that post and tried to understand some. I read this here: Loading a trained Keras model and continue training
You can simply load your model with
from keras.models import load_model
model = load_model('model1.h5')
Using Tensorflow, I build a binary classification model:
from keras.preprocessing.image import ImageDataGenerator, array_to_img, img_to_array, load_img
from keras.models import Sequential
from keras.layers import Conv2D, MaxPooling2D
from keras.layers import Activation, Dropout, Flatten, Dense
from keras import backend as K
import tensorflow
import glob
from PIL import Image
import numpy as np
img_width, img_height = 28, 28#all MNIST images are of size (28*28)
train_data_dir = '/Binary Classifier/data/train'#train directory generated by train_cla
validation_data_dir = '/Binary Classifier/data/val'#validation directory generated by val_cla
train_samples = 40000
validation_samples = 10000
epochs = 2
batch_size = 512
if K.image_data_format() == 'channels_first':
input_shape = (1, img_width, img_height)
else:
input_shape = (img_width, img_height, 1)
#build a sequential model to train data
model = Sequential()
model.add(Conv2D(32, (3, 3), input_shape=input_shape))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(32, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(64, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Flatten())
model.add(Dense(64))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(1))
model.add(Activation('sigmoid'))
model.compile(loss='binary_crossentropy',
optimizer='rmsprop',
metrics=['accuracy'])
train_datagen = ImageDataGenerator(#train data generator
rescale=1. / 255,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True)
val_datagen = ImageDataGenerator(rescale=1. / 255)#validation data generator
train_generator = train_datagen.flow_from_directory(#train generator
train_data_dir,
target_size=(img_width, img_height),
batch_size=batch_size,
class_mode='binary')
validation_generator = val_datagen.flow_from_directory(#validation generator
validation_data_dir,
target_size=(img_width, img_height),
batch_size=batch_size,
class_mode='binary')
model.fit_generator(#fit the generator to train and validate the model
train_generator,
steps_per_epoch=train_samples // batch_size,
epochs=epochs,
validation_data=validation_generator,
validation_steps=validation_samples // batch_size)
But I got an error saying "ValueError: Error when checking input: expected conv2d_1_input to have shape (28, 28, 1) but got array with shape (28, 28, 3)", and I don't understand where this error comes from. I specifically defines the input shape to be either (28,28,1) or (28,28,1), and all my input data are MNIST digits which should also be size of (28,28,1). How does the generator receive a (28,28,3) array? Any help is appreciated!
The default in ImageDataGenerator's flow_from_directory is to load color images in RGB format, which implies three channels. You want to load images as grayscale (one channel), and you can do this by setting the color_mode parameter in flow_from_directory to grayscale.
train_generator = train_datagen.flow_from_directory(
train_data_dir,
target_size=(img_width, img_height),
batch_size=batch_size,
class_mode='binary', color_mode = 'grayscale')
validation_generator = val_datagen.flow_from_directory(
validation_data_dir,
target_size=(img_width, img_height),
batch_size=batch_size,
class_mode='binary', color_mode = 'grayscale')