the impact of flatten layer and its correct usage - tensorflow

Question

I am working on simple 1D convolution model, which is built as follows
model1= Sequential()
model1.add(Conv1D(60,32, strides=1, activation='relu',padding='causal',input_shape=(64,1)))
model1.add(Conv1D(80,10, strides=1, activation='relu',padding='causal'))
model1.add(Conv1D(100,5, strides=1, activation='relu',padding='causal'))
model1.add(MaxPooling1D(2))
model1.add(Dense(300,activation='relu'))
model1.add(Flatten())
model1.add(Dense(1,activation='relu'))
print(model1.summary())
Its model summary is as follows
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv1d_1 (Conv1D) (None, 64, 60) 1980
_________________________________________________________________
conv1d_2 (Conv1D) (None, 64, 80) 48080
_________________________________________________________________
conv1d_3 (Conv1D) (None, 64, 100) 40100
_________________________________________________________________
max_pooling1d_1 (MaxPooling1 (None, 32, 100) 0
_________________________________________________________________
dense_1 (Dense) (None, 32, 300) 30300
_________________________________________________________________
flatten_1 (Flatten) (None, 9600) 0
_________________________________________________________________
dense_2 (Dense) (None, 1) 9601
=================================================================
Total params: 130,061
Trainable params: 130,061
Non-trainable params: 0
_________________________________________________________________
If I change move the flatten layer before the first dense layer, as follows, I got the following model architecture. it seems that the number of model parameters of this one is much larger than the previous one. Why the placement of flatten layer has such a larger impact? What's the correct way to place the flatten layer.
model1= Sequential()
model1.add(Conv1D(60,32, strides=1, activation='relu',padding='causal',input_shape=(64,1)))
model1.add(Conv1D(80,10, strides=1, activation='relu',padding='causal'))
model1.add(Conv1D(100,5, strides=1, activation='relu',padding='causal'))
model1.add(MaxPooling1D(2))
model1.add(Flatten())
model1.add(Dense(300,activation='relu'))
model1.add(Dense(1,activation='relu'))

Answer 1

The difference is that in the first case you have a channel-wise dense layer. The layer will map 100 inputs to 300 outputs, using 100 x 300 = 30,000 weights and 300 biases, for a total of 30,300 parameters. The same operation will be repeated for all 32 channels of its input from max_pooling1d_1.
In the second case you flatten the input first, so now you have 3,200 inputs and map it to 300 outputs, requiring 300 x 3,200 + 300 = 960,300 parameters.
Which one is the correct one is up to you. In the first case the network is much smaller, will learn quicker, will be much less prone to overfitting, but might not have the expressivity necessary to give a usable performance on your dataset. But does it make sense to force the dense layer to treat all channels the same way? Only experiments can tell. You have to try both ways and see which one yields better results.