Combining a Pre-trained Model with a Custom Model in TF - tensorflow

Question

I have simple network that I would like to increase its complexity by combining it with a pre-trained model such as InceptionV3. However, once I join them together with the following command:
snn_model = Model(inputs=baseModel.input, outputs=model, name = 'snn')
I face this error:
ValueError: Output tensors of a Functional model must be the output of a TensorFlow `Layer` (thus holding past layer metadata). Found: <tensorflow.python.keras.engine.functional.Functional object at 0x7f82d1804c10>
My Network is as follows:
def build_siamese_model(inputShape, embeddingDim=48):
# increase model complexity by adding Inception
# make the network itself generate the embediings
# specify the inputs for the feature extractor network
inputs = Input(inputShape)
# define the first set of CONV => RELU => POOL => DROPOUT layers
x = Conv2D(64,(2,2), padding='same', activation='relu')(inputs)
x = MaxPooling2D(pool_size=2)(x)
x = Dropout(0.3)(x)
# second set of CONV => RELU => POOL => DROPOUT layers
x = Conv2D(64,(2,2), padding='same', activation='relu')(x)
x = MaxPooling2D(pool_size=2)(x)
x = Dropout(0.3)(x)
# prepare the final outputs
pooledOutput = GlobalAveragePooling2D()(x)
outputs = Dense(embeddingDim)(pooledOutput)
# build the model
model = Model(inputs, outputs)
# return the model to the calling function
return model
I am combining my network with InceptionV3 as follows:
baseModel = InceptionV3(weights="imagenet", include_top=False, input_shape=(160, 160,3), input_tensor=Input(shape=(160, 160,3)))
snn_model = Model(inputs=baseModel.input, outputs=model, name = 'snn')
Even if I try to switch between these models by giving the InceptionV3 output as an input to my custom network I got another error:
ValueError: Negative dimension size caused by subtracting 2 from 1 for '{{node max_pooling2d_62/MaxPool}} = MaxPool[T=DT_FLOAT, data_format="NHWC", explicit_paddings=[], ksize=[1, 2, 2, 1], padding="VALID", strides=[1, 2, 2, 1]](Placeholder)' with input shapes: [?,1,1,64].
So, my idea is to combine a custom model with a pre-trained model to increase complexity and achieve a better performance.

Answer 1

A simple google search for transfer learning will have Transfer learning and fine-tuning as the first result. I suggest that you read it first as it has exactly what you are trying to do.
Basically you will use InceptionV3 as you would do a normal layer inside your build_siamese_model function that returns your entire model. Something like this will do:
# specify the inputs for the feature extractor network
inputs = Input(inputShape)
# define the first set of CONV => RELU => POOL => DROPOUT layers
x = baseModel(inputs) # initialized from InceptionV3
x = Conv2D(64,(2,2), padding='same', activation='relu')(x)
x = MaxPooling2D(pool_size=2)(x)
x = Dropout(0.3)(x)
# the rest of your code ...
Again, you should read the documentation to understand how to properly instantiate a pre-trained model and handle batch normalization layers

Answer 2

The first error is due to specifying output incorrectly. You should define output of the model, a output of a layer. So in order to resolve the first error, modified code, should be like this:
snn_model = Model(inputs=baseModel.input, outputs=model.output, name = 'snn')
The second error, indicates, you are shrinking input tensor, through your model layers, until it reaches to (1,1) size. Then it can not go further, since each Conv2D or MaxPooling layer could makes the input size smaller, and it will gets negative, somewhere in your middle layers. So, try to remove some layers or architecture, or try to feed a bigger input size to avoid reaching a (1,1) tensor in middle layers. You can use model.summary() to see each layers output shape, in order to figure out the journey of input tensor throughout your layers.