I am trying to implement a dense layer in keras. The input is EEG recording using 2 channels, each of them consist of a vector of 8 points and the total number of training points is 17. The y is also 17 points.
I used
x=x.reshape(17,2,8,1)
y=y.reshape(17,1,1,1)
model.add(Dense(1, input_shape=(2,8,1), activation='relu'))
print(model.summary())
model.compile(loss='sparse_categorical_crossentropy', optimizer='adam')
print(model.compile)
model.fit(x, y, batch_size = 17,epochs=500, verbose=1)
but i get the following error
Error when checking target: expected dense_57 to have shape (2, 8, 1) but got array with shape (17, 1, 1)
Since the Dense layer has output dimension 1, it would expect y to be of the shape (2, 8, 1). An easy fix would be to do the following
x = x.reshape(17, 16)
y = y.reshape(17, 1)
model.add(Dense(1, input_shape=(16,), activation='relu'))
Related
If I wanted to make a model that would take a single number and then just output a single number (not a linear relationship, not sure what kind), how would I shape the input and output layers, and what kind of loss/optimizer/activation functions should I use? Thanks.
Your question includes many things. What i will highly recommand you to
understand
Regression based problem
Classification based problem
Based on that you need to figure out which activation function or loss function or optimizer you need to use because for regression and classification those are different. Try to figure out things one after another.
For input/ouput see THIS
You have only one feature as input then the model based on,
Classification based Problem,
Loss Function - categorical_crossentropy || sparse_categorical_crossentropy
optimizer - Adam
output layer - number of class need to predict
output activation - softmax
model = tf.keras.Sequential()
model.add(layers.Dense(8, activation='relu', input_shape = (1, ))) #input shape as 1
model.add(layers.Dense(3, activation='softmax')) #3 is number of class
model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.001), loss='categorical_crossentropy', metrics=['accuracy'])
Regression based Problem,
Loss Function - mean_square_error
optimizer - Adam
output layer - 1
output activation - default (relu)
model = tf.keras.Sequential()
model.add(layers.Dense(8, activation='relu', input_shape = (1, ))) #input shape as 1
model.add(layers.Dense(1)) #1 is number of output
model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.001), loss='mean_square_error', metrics=['accuracy'])
Binary based Problem (0 or 1),
Loss Function - binary_crossentropy
optimizer - Adam
output activation - sigmoid
output layer - 1
model = tf.keras.Sequential()
model.add(layers.Dense(8, activation='relu', input_shape = (1, ))) #input shape as 1
model.add(layers.Dense(1, activation='sigmoid')) #1 is number of output
model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.001), loss='binary_crossentropy', metrics=['accuracy'])
This is my signal data
The length of each sample data is = 64.
The sum of train data is =49572
length=len(x_train)
model = tf.keras.models.Sequential([
tf.keras.layers.Conv1D(32, 3, activation='relu', input_shape=(length,64)),
tf.keras.layers.MaxPooling1D(3),
tf.keras.layers.Conv1D(64, 3, activation='relu'),
tf.keras.layers.MaxPooling1D(3),
tf.keras.layers.Conv1D(128, 3, activation='relu'),
tf.keras.layers.MaxPooling1D(3),
tf.keras.layers.Conv1D(128, 3, activation='relu'),
tf.keras.layers.MaxPooling1D(3),
tf.keras.layers.Flatten(),
tf.keras.layers.Dense(512, activation='relu'),
tf.keras.layers.Dense(512, activation='relu'),
tf.keras.layers.Dense(512, activation='relu'),
tf.keras.layers.Dense(29, activation='softmax')
])
I want to make a CNN model for signal data. So, I use Conv1d.
How to know the input_shape from my data?
From the keras Conv1D documentation:
When using this layer as the first layer in a model, provide an
input_shape argument (tuple of integers or None, e.g. (10, 128) for
sequences of 10 vectors of 128-dimensional vectors, or (None, 128) for
variable-length sequences of 128-dimensional vectors.
From your image, it seems like your data is simple 1-dimensional hat means length should equal to 1 in your case.
Think of the length dimension as the color channel of an image in the 2D convolution case. Black-white images have only a single color dimension, therefore width x height x 1, whereas RGB images have 3 color channels, hence width x height x 3.
Similar, if you work with time series and 1D convolutions you may have more then one signal, e.g. temperature + atmospheric pressure + humidity measured throughout the day for each minute. Then your signal would be of shape 1440 x 3
When using CNN with tensorflow, what the convulsion matrix looks like (what are the kernel values) ?
Look on this basic example of CNN:
model.add(layers.Conv2D(32, (3, 3), activation='relu', input_shape=(32, 32, 3)))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(64, (3, 3), activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(64, (3, 3), activation='relu'))
what the convolution matrix looks like ?
what are the values of the 3x3 matrix ?
In the example above, we use 3 Conv2D layers (each layer use 3x3 convultion matrix).
Does those 3 matrixes are the same ? or they will have different values ?
Each convolution layer will have a weight and bias which can be inspected using
# For 1 layer <conv> (weight)
model.layers[0].get_weights()[0]
# For 1 layer <conv> (bias)
model.layers[0].get_weights()[1]
# For 2 layer <pool> (no weight and bias term) <so empty list is returned>
model.layers[1].get_weights()
#and so on....
conv matrix is a 4D tensor (in_channel × filter_size × filter_size × out_channel) and for your case: (3, 3, 3, 32).
Each filter will have different value. Nothing is common.
When taking the one dimensional convolution of a one dimensional array, I receive an error which suggests my second dimension is not big enough.
Here is the overview of the relevant code:
inputs_ = tf.placeholder(tf.float32 ,(None, 45), name='inputs')
x1 = tf.expand_dims(inputs_, axis=1)
x1 = tf.layers.conv1d(x1, filters=64, kernel_size=1, strides=1, padding='valid')
I am hoping to increase the kernel size to 3 such that neighbouring points also influence the output of each input node, however I get the following error:
ValueError: Negative dimension size caused by subtracting 3 from 1 for
'conv1d_4/convolution/Conv2D' (op: 'Conv2D') with input shapes:
[?,1,1,45], [1,3,45,64].
My guess is that tensorflow is expecting me to reshape my input into two dimensions so that some depth can be used to do the kernel multiplication. Question is why is this the case and what to expect for the layer behaviour based on the input dimensions
You need to add a Channel dimension as last dimension even if you only have one channel.
So this code works:
inputs_ = tf.placeholder(tf.float32 ,(None, 45), name='inputs')
x1 = tf.expand_dims(inputs_, axis=-1)
x1 = tf.layers.conv1d(x1, filters=64, kernel_size=3, strides=1, padding='valid')
So basically the error was caused because your tensor looked like having a width of 1, with 45 channels. TensorFlow was trying to convolve with a kernel size 3 along a size 1 dimension.
I am trying to use LSTM to model multi-sample time series data. My input data has shape (100, 93, 6) - 100 independent time series (from the same/similar process), 93 time steps, 6 dimensions at each observation. Output shape is (100, 93) - one bool output per time step for each independent time series. (This is a small sample of real data, of course). However, I can't figure out how to construct such a Network in Keras:
from keras.models import Sequential
from keras.layers import LSTM, core, Activation, Dense
import numpy as np
data = np.load('sample.npz')
X = data['x']
y = data['y']
print('X shape: ',X.shape)
print('{} samples, {} time steps, {} observations at each time step, per sample\n'.format(*X.shape))
print('y shape: ',y.shape)
print('{} samples, {} time steps, boolean outcome per observation\n'.format(*y.shape))
print(X[0][2], X[0][55])
print(y[0][2], y[0][92])
X shape: (100, 93, 6) 100 samples, 93 time steps, 6 observations at
each time step, per sample
y shape: (100, 93) 100 samples, 93 time steps, boolean outcome per
observation
[ 1.80000000e+01 1.56000000e+05 2.00000000e+03 1.00000000e+04
3.00000000e+00 5.94000000e+04] [ 0. 0. 0. 0. 0. 0.]
1.0 0.0
model = Sequential()
model.add(LSTM(output_dim=4, input_shape=(93, 6), return_sequences=False))
model.add(Dense(1, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='adam')
model.fit(X, y, verbose=2)
Exception: Error when checking model target: expected dense_2 to have
shape (None, 1) but got array with shape (100, 93)
I believe Keras assumes that I have one output (Y) per timeseries, while I have one output per time step per time series. How do I make it work in Keras?
I was missing TimeDistributed Layer..
This works:
model = Sequential()
model.add(LSTM(output_dim=4, input_shape=(93, 6), return_sequences=True))
model.add(TimeDistributed(Dense(1, activation='sigmoid')))
model.compile(loss='binary_crossentropy', optimizer='adam')