Time series CNN, trying to use 1,1 input shape - tensorflow

Question

I'm trying to do a CNN 1D for time series.
First issue:
When trying to use an input shape of [1,1] I get an error:
Error: Negative dimension size caused by adding layer average_pooling1d_AveragePooling1D1 with input shape [,0,128]
2nd issue
I have 2 different arrays (1d) for my data: first array is the input data containing the time series and the 2nd array contains the output data with closed values for a stock.
Something that got me to a few more results was to set the input shape to [6,1].
Model summary:
_________________________________________________________________
Layer (type) Output shape Param #
=================================================================
conv1d_Conv1D1 (Conv1D) [null,5,128] 384
_________________________________________________________________
average_pooling1d_AveragePoo [null,4,128] 0
_________________________________________________________________
conv1d_Conv1D2 (Conv1D) [null,3,64] 16448
_________________________________________________________________
average_pooling1d_AveragePoo [null,2,64] 0
_________________________________________________________________
conv1d_Conv1D3 (Conv1D) [null,1,16] 2064
_________________________________________________________________
average_pooling1d_AveragePoo [null,0,16] 0
_________________________________________________________________
flatten_Flatten1 (Flatten) [null,0] 0
_________________________________________________________________
dense_Dense1 (Dense) [null,1] 1
=================================================================
Here training the model got me into issues:
const trainX = tf.tensor1d(data.inTime).reshape([100, 6, 1])
100 - size of my array
6 - features
1 - 1 unit as output
Error: Size(100) must match the product of shape 100,6,1
I'm stuck at the training step because I don't know how to train it.
I would prefere to have a [1,1] input shape, to give only 1 time series and to have 1 output from it.
The model
async function buildModel() {
const model = tf.sequential()
// settings
const kernelSize = 2
const poolSize = [2]
// tf layers
model.add(tf.layers.conv1d({
inputShape: [6, 1],
kernelSize: kernelSize,
filters: 128,
strides: 1,
useBias: true,
activation: 'relu',
kernelInitializer: 'varianceScaling'
}))
model.add(tf.layers.averagePooling1d({poolSize: poolSize, strides: [1]}))
// 2nd layer
model.add(tf.layers.conv1d({
kernelSize: kernelSize,
filters: 64,
strides: 1,
useBias: true,
activation: 'relu',
kernelInitializer: 'varianceScaling'
}))
model.add(tf.layers.averagePooling1d({poolSize: poolSize, strides: [1]}))
model.add(tf.layers.conv1d({
kernelSize: kernelSize,
filters: 16,
strides: 1,
useBias: true,
activation: 'relu',
kernelInitializer: 'varianceScaling'
}))
model.add(tf.layers.averagePooling1d({poolSize: poolSize, strides: [1]}))
model.add(tf.layers.flatten())
model.add(tf.layers.dense({
units: 1,
kernelInitializer: 'VarianceScaling',
activation: 'linear'
}))
// optimizer + learning rate
const optimizer = tf.train.adam(0.0001)
model.compile({
optimizer: optimizer,
loss: 'meanSquaredError',
metrics: ['accuracy'],
})
return model
}
Training where the error is occurring
async function train(model, data) {
console.log(`MODEL SUMMARY:`)
model.summary()
// Train the model
const epochs = 2
// train data size, 28, 28, 1
const trainX = tf.tensor1d(data.inTime).reshape([100, 6, 1])
const trainY = tf.tensor([data.outClosed], [1, data.size, 1])
let result = await model.fit(trainX, trainY, {
epochs: epochs
})
print("Loss after last Epoch (" + result.epoch.length + ") is: " + result.history.loss[result.epoch.length-1])
return result
}
Any ideas into how to fix it will be much appreciated!

Answer 1

Time series is a sequence taken at successive equally spaced points in time according to wikipedia. The goal of the neural network NN used on time series is to find the pattern between the series of data. Convolutiona Neural Networks CNN are rarely if not never used on this kind of data. Other NN often used are RNN and LSTM. If we are interested in finding a pattern in a series of data, the inputShape can't be [1, 1]; otherwise it will mean finding a pattern on a unique point. It can be done theoretically, but in reality it does not capture the essence of the time series.
The model used here is using CNN with average pooling layer. Of course, a pooling layer cannot be applied on a layer with a pooling size bigger than the shape of the layer thus throwing the error:
Error: Negative dimension size caused by adding layer average_pooling1d_AveragePooling1D1 with input shape [,0,128]
The last error:
Error: Size(100) must match the product of shape 100,6,1
indicates a mismatch of the size of the tensors.
100 * 6 * 1 = 600 elements in the tensor (size =600) whereas the input tensor has 100 elements resulting in the error.