I am trying to use plt.imshow to display pictures from the Cifar dataset. I am using the following simple command:
plt.imshow(X_test[7])
where X_test is the default data you get when you run:
(X_train, y_train), (X_test, y_test) = keras.datasets.cifar10.load_data()
The images I am getting look like the following: my image
While in reality they should look like: real image
Does anyone know how to fix this?
Thank you :)
Related
My trained model accepts images of int32 datatype. However, my upstream sends float32 data. Do I have to convert the training images to float32 and train the model again? My code looks something like this:
(x_train, y_train), (x_test, y_test) = dataset.load_data()
print(x_train.dtype)
# 'int32'
I have built a model on Tensorflow 2 (using Keras) to train and evaluate on MNIST data. It can properly run the commands:
model.prepare_training()
model.compile(loss=tf.keras.losses.CategoricalCrossentropy(), metrics=[tf.keras.metrics.Accuracy()])
history = model.fit(x=x_train, y=y_train, callbacks=[model.MS, model.WS])
pred = model.predict(x_test)
Yet, when I try to evaluate it against the test data using
score = model.evaluate(x_test, y_test)
I get a large error message specifically for the model.evaluate command, whose last part reads:
TypeError: Can not convert a NoneType into a Tensor or Operation.
I have seen similar questions which were 'solved' by the OP updating their Tensorflow and/or Keras to the most recent version, but I've done that already and it didn't work. The model also works, since I've taken the prediction and found a rough estimate of the accuracy, which is 88%. So, the problem is probably not a logical one, but a technical one.
Any idea as to what might be happening?
EDIT:
As per request, here's how I load the data:
(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()
I have since then figured out that this is a bug with TensorFlow 2.
Im trying to make a handwritten digit recognition program with machine learning. As the title says, i want to simple resize mnist dataset with cv2 library. And i want the resize to be done before normalizing the data, so that its the resized data that gets through my ANN(Artificial Neural Network) model. But im not really sure how to do that, this is the code i have:
model.add = modell
train = keras.utils.normalize(train)
test = keras.utils.normalize(test)
model=Sequential()
modell(Flatten())
modell(Dense(200, activation='sigmoid'))
modell(Dense(200, activation='sigmoid'))
modell(Dense(10, activation="sigmoid"))
model.fit(img, lab, epochs=1)
print(model.evaluate(test, test))
pred = model.predict(test[:1])
p=np.argmax(pred, axis=1)
print("Machine", p)
print("Correct", test[:1])
I have tried to put this in to my code, right before normalizing my data:
for i in train_img:
photo = cv2.reshape(train_img[i] 19 19)
img.append(photo)
But it gives me this error:
TypeError: 'tuple' object is not callable
How do i do it?
in your code i is in train_img. Therefore i is an image and can not be used as an index
so try this
new_train_img = []
for img in train_img:
new_train.append(cv2.resize(img,(14,14)))
I am creating a machine learning model for classifying images of numbers. I have trained the model using Tensorflow and Keras using the inbuilt tf.keras.datasets.mnist dataset. The model works quite well with the test images from the mnist dataset itself but I would like to feed it images of my own. The images that I am feeding this model is extracted from a Captcha so they will follow a similar pattern. I have included some examples of the images in this public google drive folder. When I feed these images, I noticed that the model is not very accurate and I have some guesses as to why.
The background of the image creates too much noise in the picture.
The number is not centered.
The image is not striclty in the color format of MNIST training set (Black background white text).
I wanted to ask how can I remove the background and centre it so that the noise in the image is reduced allowing for better classifications.
Here is the model I am using:
import tensorflow as tf
from tensorflow import keras
mnist = keras.datasets.mnist
(x_train, y_train), (x_test, y_test) = mnist.load_data()
class Stopper(keras.callbacks.Callback):
def on_epoch_end(self, epoch, log={}):
if log.get('acc') >= 0.99:
self.model.stop_training = True
print('\nReached 99% Accuracy. Stopping Training...')
model = keras.Sequential([
keras.layers.Flatten(),
keras.layers.Dense(1024, activation=tf.nn.relu),
keras.layers.Dense(10, activation=tf.nn.softmax)])
model.compile(
optimizer=tf.train.AdamOptimizer(),
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
x_train, x_test = x_train / 255, x_test / 255
model.fit(x_train, y_train, epochs=10, callbacks=[Stopper()])
And here is my method of importing the image into tensorflow:
from PIL import Image
img = Image.open("image_file_path").convert('L').resize((28, 28), Image.ANTIALIAS)
img = np.array(img)
model.predict(img[None,:,:])
I have also included some examples from the MNIST dataset here. I would like a script to convert my images as closely to the MNIST dataset format as possible. Also, since I would have to do this for an indefinite number of images, I would appreciate if you could provide a fully automated method for this conversion. Thank you very much.
You need to train with a dataset similar to the images you're testing. The MNIST data is hand-written numbers, which is not going to be similar to the computer generated fonts for Captcha data.
What you need to do is gain a catalog of Captcha data similar to what you're predicting on (preferably from the same source you will be inputting to the final model). It's a painstaking task to capture the data, and you'll probably need around 300-400 images for each label before you start to get something useful.
A key note: your model will only ever be as good as the training data you supplied to the model. Trying to make a good model with bad training data is an effort in pure frustration
To address some of your thoughts:
[the model is not very accurate because] the background of the image creates too much noise in the picture.
This is true. If the image data has noise and the neural net was not trained using any noise in the images, then it will not recognize a strong pattern when it encounters this type of distortion. One possible way to combat this is to take clean images and progamatically add noise to the image (noise similar to what you see in the real Captcha) before sending it to be trained.
[the model is not very accurate because] The number is not centered.
Also true for the same reasons. If all the training data is centered, the model will be overtuned for this property and make incorrect guesses. Follow a similar pattern to the one above if you don't have the capacity to manually capture and catalog a good sampling of data.
[the model is not very accurate because] The image is not striclty in the color format of MNIST training set (Black background white text).
You can get around this by applying a binary threshold to the data before processing/ normalize the color input before training. Depending on the amount of noise in the captcha you may have better results allowing the number and noise to retain some of it's color information (still put in greyscale and normalize, just don't apply the threshold).
Additionally I'd recommend using a convolution net rather than the linear network as it is better at distinguishing 2D features like edges and corners. i.e. use keras.layers.Conv2D layers before flattening with keras.layers.Flatten
See the great example found here: Trains a simple convnet on the MNIST dataset.
model = tf.keras.models.Sequential(
[
tf.keras.layers.Conv2D(
32,
kernel_size=(3, 3),
activation=tf.nn.relu,
input_shape=input_shape,
),
tf.keras.layers.Conv2D(64, (3, 3), activation=tf.nn.relu),
tf.keras.layers.MaxPooling2D(pool_size=(2, 2)),
tf.keras.layers.Dropout(0.25),
tf.keras.layers.Flatten(),
tf.keras.layers.Dense(128, activation=tf.nn.relu),
tf.keras.layers.Dropout(0.5),
tf.keras.layers.Dense(
num_classes, activation=tf.nn.softmax
),
]
)
I've used this setup for reading fonts in video gameplay footage, and with a test set of 10,000 images I'm achieving 99.98% accuracy, using a random sampling of half the dataset in training, and calculating accuracy using the total set.
I use Keras.
(X_train, y_train), (X_test, y_test) = mnist.load_data()
X_train' shape is (number_of_training_sample,224,224,3)
Y_train's shape is (number_of_training_sample, 10)
Features and labels are separated in different ndarray.
BUT I want to change these ndarrays to 'mnist_train.csv' format.
(https://raw.githubusercontent.com/sjwhitworth/golearn/master/examples/datasets/mnist_train.csv)
My own dataset is totally same format with the return value of 'mnist.load_data()'. BUT only 'mnist_train.csv' format is worked in the code that I want to use.
Could you let me know how to change the format????
(Sorry for really simple question.. I am a newbie of numpy and python.)
(X_train, y_train), (X_test, y_test) = mnist.load_data()
loads data as numpy arrays
'mnist_train.csv' is a .csv file store on a hard disc which we usually read with pandas library
import pandas as pd
X_train = pd.read_csv('filename.csv')
Pandas reads a .csv as a dataframe (with column names if there are such)
Using keras you need numpy arrays, so the
(X_train, y_train), (X_test, y_test) = mnist.load_data()
should work fine
Post the code you are using and the errors that is produced for further explaination