so I'm new right here and in Python also. I'm trying to make my own network. I found some pictures of docs and cats 15x15 and unfortunatly couldn't make this basic network...
So, these are libraries which I'm using
from tensorflow.keras.models import Sequential
from tensorflow.keras import utils
from tensorflow.keras.datasets import mnist
from tensorflow.keras.layers import Dense
import numpy as np
import matplotlib.pyplot as plt
import tensorflow as tf
import keras
from tensorflow.keras.layers import Conv2D
from tensorflow.keras.layers import MaxPooling2D
from tensorflow.keras.layers import GlobalMaxPooling2D
Body
train_dataset = tf.keras.preprocessing.image_dataset_from_directory(
'drive/MyDrive/cats vs dogs/cats vs dogs/training',
color_mode="rgb",
batch_size=32,
image_size=(150, 150),
shuffle=True,
seed=42,
validation_split=0.1,
subset='training',
interpolation="bilinear",
follow_links=False,
)
validation_dataset = tf.keras.preprocessing.image_dataset_from_directory(
'drive/MyDrive/cats vs dogs/cats vs dogs/training',
color_mode="rgb",
batch_size=32,
image_size=(150, 150),
shuffle=True,
seed=42,
validation_split=0.1,
subset='validation',
interpolation="bilinear",
follow_links=False,
)
test_dataset = tf.keras.preprocessing.image_dataset_from_directory(
'drive/MyDrive/cats vs dogs/cats vs dogs/test',
batch_size = 32,
image_size = (150, 150),
interpolation="bilinear"
)
model = Sequential()
model.add(keras.Input(shape=(150, 150, 3)))
model.add(Conv2D(32, 5, strides=2, activation="relu"))
model.add(Conv2D(32, 3, activation="relu"))
model.add(MaxPooling2D(3))
model.add(Dense(250, activation='sigmoid'))
model.add(Dense(100))
model.add(MaxPooling2D(3))
model.add(Dense(2))
model.summary()
model.compile(loss='binary_crossentropy',
optimizer='adam',
metrics=['accuracy'])
history = model.fit(train_dataset, validation_data=validation_dataset, epochs=5, verbose=2)
And I get this error
Incompatible shapes: [29] vs. [29,7,7,2]
[[node gradient_tape/binary_crossentropy/mul_1/BroadcastGradientArgs
(defined at /usr/local/lib/python3.7/dist-packages/keras/optimizer_v2/optimizer_v2.py:464)
]] [Op:__inference_train_function_4364]
Errors may have originated from an input operation.
Input Source operations connected to node
gradient_tape/binary_crossentropy/mul_1/BroadcastGradientArgs:
In[0] gradient_tape/binary_crossentropy/mul_1/Shape:
In[1] gradient_tape/binary_crossentropy/mul_1/Shape_1
I was trying to change from binary_crossentropy to categorical_crossentrapy but it didn't help, I suppose my mistake is in datasets or inputs but I don't know how to solve it :(
Really hope to find help here!
[my architecture][1]
[1]: https://i.stack.imgur.com/w4Y9N.png
You need to flatten your prediction somewhere, otherwise you are outputing an image (29 samples of size 7x7 with 2 channels), while you simply want a flat 2 dimensional logits (so shape 29x2). The architecture you are using is somewhat odd, did you mean to have flattening operation before first Dense layer, and then no "maxpooling2d" (as it makes no sense for flattened signal)? Mixing relu and sigmoid activations is also quite non standard, I would encourage you to start with established architectures rather than try to compose your own to get some intuitions.
model = Sequential()
model.add(keras.Input(shape=(150, 150, 3)))
model.add(Conv2D(32, 5, strides=2, activation="relu"))
model.add(Conv2D(32, 3, activation="relu"))
model.add(MaxPooling2D(3))
model.add(Flatten())
model.add(Dense(250, activation="relu"))
model.add(Dense(100, activation="relu"))
model.add(Dense(2))
model.summary()
Related
I believed that, if I have a binary-classification problem then I should always have only 1 node in the last layer, since the last layer has to decide about the classification. However, in the following code it is not true.
Let's download the pizza/steak datasets (image dataset) and prepare the data using the ImageDataGenerator:
import zipfile
import tensorflow as tf
from tensorflow.keras.layers import Input, Dense, Dropout, Conv2D, MaxPooling2D, Flatten
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.preprocessing import image_dataset_from_directory
from tensorflow.keras.applications import EfficientNetB0, resnet50
from tensorflow.keras.models import Sequential
import numpy as np
import pandas as pd
!wget https://storage.googleapis.com/ztm_tf_course/food_vision/pizza_steak.zip
zip_ref = zipfile.ZipFile("pizza_steak.zip", "r")
zip_ref.extractall()
zip_ref.close()
train_directory = './pizza_steak/train/'
test_directory = './pizza_steak/test/'
IMAGE_SIZE = (224, 224)
image_data_generator = ImageDataGenerator(rescale=1. / 255,
zoom_range=0.2,
shear_range=0.2,
rotation_range=0.2)
train_dt = image_data_generator.flow_from_directory(directory=train_directory,
class_mode='categorical',
batch_size=32,
target_size=IMAGE_SIZE)
test_dt = image_data_generator.flow_from_directory(directory=test_directory,
class_mode='categorical',
batch_size=32,
target_size=IMAGE_SIZE)
and then build, compile a neural-network and fit the data on it:
model = Sequential()
model.add(Conv2D(filters=16, kernel_size=3, activation='relu'))
model.add(Conv2D(filters=16, kernel_size=3, activation='relu'))
model.add(MaxPooling2D())
model.add(Conv2D(filters=16, kernel_size=3, activation='relu'))
model.add(Conv2D(filters=16, kernel_size=3, activation='relu'))
model.add(MaxPooling2D())
model.add(Flatten())
model.add(Dense(1, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
model.fit(train_dt,
epochs=5,
validation_data=test_dt,
validation_steps=len(test_dt)
As you can see the val_accuracy is not better than 0.5000, which is very bad!
And now if you just change the last layer to model.add(Dense(2, activation='sigmoid')) and run the same model with 2 nodes in the last layer, you will end up with a far better result, such as val_accuracy: 0.8680.
How should know, how many nodes should I have in the last layer when I have a binary-classification model?
Thanks to #Dr.Snoopy, i add an answer here jut to complete the question.
The point is how do we label our data using the image_data_generator.flow_from_directory().
If we set the class_mode='categorical' then the target is ONE_HOT and the number of nodes in the last layer is equal to "number of classes of target feature". In my case, it is a binary feature, so i need to have 2 nodes in the last layer.
However, if we use class_mode='binary' then the target is indexed and we can have only one node in the last layer.
I've created sound classifier build using Keras from some tutorials in the internet. Here is my model code
import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, InputLayer, Dropout, Conv1D, Conv2D, Flatten, Reshape, MaxPooling1D, MaxPooling2D, BatchNormalization, TimeDistributed
from tensorflow.keras.optimizers import Adam
model = Sequential()
model.add(Reshape((int(input_length / 40), 40), input_shape=(input_length, )))
model.add(Conv1D(8, kernel_size=3, activation='relu', padding='same'))
model.add(MaxPooling1D(pool_size=2, strides=2, padding='same'))
model.add(Dropout(0.25))
model.add(Conv1D(16, kernel_size=3, activation='relu', padding='same'))
model.add(MaxPooling1D(pool_size=2, strides=2, padding='same'))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(classes, activation='softmax', name='y_pred'))
opt = Adam(lr=0.005, beta_1=0.9, beta_2=0.999)
# this controls the batch size, or you can manipulate the tf.data.Dataset objects yourself
BATCH_SIZE = 32
train_dataset = train_dataset.batch(BATCH_SIZE, drop_remainder=False)
validation_dataset = validation_dataset.batch(BATCH_SIZE, drop_remainder=False)
model.compile(loss='categorical_crossentropy', optimizer=opt, metrics=['accuracy'])
model.fit(train_dataset, epochs=1000, validation_data=validation_dataset, verbose=2, callbacks=callbacks)
My teacher ask me what is algorithm I use for classifying (he said something like K-NN, Naive Bayes, SVM or something like that), and I don't know what I'm using.
You're using a Convolutional Neural Network (CNN)
I just started with deep learning and i want to get the input/output of each layer in real-time. I am using google colab with tensorflow 2 and python 3. I tried to get the layers like this but for some reason that i don't understand is not working. Any help will be appreciated.
# Here are imports
from __future__ import absolute_import, division, print_function, unicode_literals
try:
# %tensorflow_version only exists in Colab.
%tensorflow_version 2.x
except Exception:
pass
import tensorflow as tf
from tensorflow.keras import datasets, layers, models
import matplotlib.pyplot as plt
from tensorflow.keras import backend as K
# I am using CIFAR10 dataset
(train_images, train_labels), (test_images, test_labels) =
datasets.cifar10.load_data()
Normalize pixel values to be between 0 and 1
train_images, test_images = train_images / 255.0, test_images / 255.0
# Here is the model
model = models.Sequential()
model.add(layers.Conv2D(32, (3, 3), activation='relu', input_shape=(32, 32, 3)))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(64, (3, 3), activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(64, (3, 3), activation='relu'))
model.add(layers.Flatten())
model.add(layers.Dense(64, activation='relu'))
model.add(layers.Dense(10, activation='softmax'))
# Compilation of the model
model.compile(optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
history = model.fit(train_images, train_labels, epochs=10,
validation_data=(test_images, test_labels))
# Based on
https://stackoverflow.com/questions/41711190/keras-how-to-get-the-output-of-each-layer
# I tried this
tf.compat.v1.disable_eager_execution()
inp = model.input # input placeholder
outputs = [layer.output for layer in model.layers] # all layer outputs
functors = [K.function([inp, K.learning_phase()], [out]) for out in outputs] # evaluation functions
Testing
test = np.random.random(input_shape)[np.newaxis,...]
layer_outs = [func([test, 1.]) for func in functors]
print(layer_outs)
#The error appear at line
functors = [K.function([inp, K.learning_phase()], [out]) for out in outputs]
#I got this error message
Tensor Tensor("conv2d/Identity:0", shape=(None, 30, 30, 32), dtype=float32) is not an element of this graph.
This error basically tells you that you want to change the graph after compiling it. When you call compile, TF will statically define all operations. You have to move the code snippet where you define functors above the compile method. Just swap the last lines with these ones:
tf.compat.v1.disable_eager_execution()
inp = model.input # input placeholder
outputs = [layer.output for layer in model.layers] # all layer outputs
functors = [K.function([inp, K.learning_phase()], [out]) for out in outputs] # evaluation functions
model.compile(optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
history = model.fit(train_images, train_labels, epochs=1,
validation_data=(test_images, test_labels))
#Testing
input_shape = [1] + list(model.input_shape[1:])
test = np.random.random(input_shape)
layer_outs = [func([test, 1.]) for func in functors]
print(layer_outs)
I have a data base with this shape: (1400000, 44)
which the 44th column is output.
all numbers are float and between 0 and 1. I used a Tensorflow like below but the loss function is non and the acc is zero.
# Create network with Keras
import tensorflow as tf
from keras.models import Sequential
from keras.layers import Dense
import numpy as np
dataset=np.loadtxt("Dataset5.txt")
s=dataset.size
tr_size=int( 0.7*s)
X = dataset[0:tr_size,0:43]
Y = dataset[0:tr_size,43]
# create model
model = Sequential()
model.add(Dense(64, input_dim=43, init='uniform', activation='relu'))
model.add(Dense(16, init='uniform', activation='relu'))
model.add(Dense(4, init='uniform', activation='sigmoid'))
model.add(Dense(1, init='uniform', activation='relu'))
# Compile model
model.compile(loss='mean_squared_error', optimizer='adam', metrics=['accuracy'])
# Fit the model
model.fit(X, Y, epochs=150, batch_size=1000, verbose=2)
# calculate predictions
predictions = model.predict(X)
I am attempting to 'flow' my data into a neural network with Keras. I am using the .flow_from_directory method and the process is giving me fits. I am using the basic example from the keras documentation (I am using tensorflow):
ROWS = 64
COLS = 64
CHANNELS = 3
from keras.preprocessing.image import ImageDataGenerator
train_datagen = ImageDataGenerator(
rescale=1./255)
test_datagen = ImageDataGenerator(rescale=1./255)
train_generator = train_datagen.flow_from_directory(
'train',
target_size=(64, 64),
batch_size=32,
class_mode='binary')
validation_generator = test_datagen.flow_from_directory(
'../tutorial/l1/kaggle_solutions/dogs_vs_cats/valid',
target_size=(64, 64),
batch_size=1,
class_mode='binary')
from keras.models import Sequential
from keras.layers import Convolution2D
from keras.layers import Dense, Activation, Flatten, Dropout, MaxPooling2D
from keras.regularizers import l2
model = Sequential()
model.add(Convolution2D(4, 4, 4, border_mode='same', input_shape=(64, 64,3), activation='relu'))
from keras.utils.np_utils import to_categorical
from keras.optimizers import SGD, RMSprop
model.compile(loss='binary_crossentropy', optimizer=RMSprop(lr=1e-4), metrics=['accuracy'])
model.fit_generator(
train_generator,
samples_per_epoch=2500,
nb_epoch=20,
validation_data=validation_generator,
nb_val_samples=3100)
Running this i get the following error:
Exception: Error when checking model target: expected convolution2d_84 to have 4 dimensions, but got array with shape (32, 1)
I have been tinkering around for a long time and found the following--switching the 'model.add' to grayscale input
model.add(Convolution2D(4, 4, 4, border_mode='same', input_shape=(64, 64,3), activation='relu'))
gives me the following error (as expected--but appears to confirm my original input was correct):
Error when checking model input: expected convolution2d_input_49 to have shape (None, 64, 64, 1) but got array with shape (32, 64, 64, 3)
So I am passing (in the original) a 4-d array of 32,64,64,3 with the original, but I am getting the error that I THINK means
Expected (1,64,64,3) and got (32,64,64,3)
As I am sending data in batches of 32. Curiously enough if I set the batch to zero (to give a 0,64,64,3 input) I get:
Exception: Error when checking model target: expected convolution2d_87 to have 4 dimensions, but got array with shape (0, 1)
Based on the documentation, I cannot figure out the proper way to flow the data into the model--i cannot pass the batch size to the model when using fit_generator, and it appears that the batch_size (num of samples) is the problem.
Any help would be greatly appreciated.
There is no problem with your ImageDataGenerator. As stated in the error message there is a mismatch between the shape of your model output and the shape of its targets. You use class_mode = 'binary', so expected output of your model is a single value, but instead it yields output of shape (batch_size, 64, 64, 4) since you have one convolutional layer and nothing else in your model.
Try something like this:
model.add(Convolution2D(4, 4, 4, border_mode='same', input_shape=(64, 64,3), activation='relu'))
model.add(Flatten())
model.add(Dense(1))
model.add(Activation('sigmoid'))