Using a model from TF Hub as a convolutional feature extractor - tensorflow

Question

I would like to build a custom model on top of ResNet-50 extractor (its intermediate layer).
Here how I am trying to do that:
def get_model(img_shape, num_classes):
inputs = Input(shape=img_shape)
backbone = hub.load("https://tfhub.dev/google/imagenet/resnet_v2_50/feature_vector/3").signatures["image_feature_vector"]
x = backbone(inputs)['resnet_v2_50/block3/unit_1/bottleneck_v2/shortcut']
# Add a per-pixel classification layer
outputs = Conv2D(num_classes + 1, 3, activation="softmax", padding="same")(x)
model = tf.keras.Model(inputs=inputs, outputs=outputs)
return model
This code works. However, it runs very long (7 min in Colab) and the model summary I get look like this:
input_14 (InputLayer) [(None, 224, 224, 3)] 0
tf_op_layer_StatefulPartiti [(None, 2048), 0
onedCall_16 (TensorFlowOpLa (None, 28, 28, 256),
yer) (None, 56, 56, 256),
(None, 56, 56, 64),
(None, 56, 56, 64),
(None, 56, 56, 256),
(None, 56, 56, 256),
(None, 56, 56, 256),
(None, 56, 56, 64),
(None, 56, 56, 64),
(None, 56, 56, 256),
(None, 28, 28, 256),
(None, 56, 56, 64),
(None, 28, 28, 64),
(None, 28, 28, 256),
(None, 14, 14, 512),
(None, 28, 28, 512),
(None, 28, 28, 128),
(None, 28, 28, 128),
(None, 28, 28, 512),
(None, 28, 28, 512),
(None, 28, 28, 512),
(None, 28, 28, 128),
(None, 28, 28, 128),
(None, 28, 28, 512),
(None, 28, 28, 512),
(None, 28, 28, 128),
(None, 28, 28, 128),
(None, 28, 28, 512),
(None, 14, 14, 512),
(None, 28, 28, 128),
(None, 14, 14, 128),
(None, 14, 14, 512),
(None, 7, 7, 1024),
(None, 14, 14, 1024),
(None, 14, 14, 256),
(None, 14, 14, 256),
(None, 14, 14, 1024),
(None, 14, 14, 1024),
(None, 14, 14, 1024),
(None, 14, 14, 256),
(None, 14, 14, 256),
(None, 14, 14, 1024),
(None, 14, 14, 1024),
(None, 14, 14, 256),
(None, 14, 14, 256),
(None, 14, 14, 1024),
(None, 14, 14, 1024),
(None, 14, 14, 256),
(None, 14, 14, 256),
(None, 14, 14, 1024),
(None, 14, 14, 1024),
(None, 14, 14, 256),
(None, 14, 14, 256),
(None, 14, 14, 1024),
(None, 7, 7, 1024),
(None, 14, 14, 256),
(None, 7, 7, 256),
(None, 7, 7, 1024),
(None, 7, 7, 2048),
(None, 7, 7, 2048),
(None, 7, 7, 512),
(None, 7, 7, 512),
(None, 7, 7, 2048),
(None, 7, 7, 2048),
(None, 7, 7, 2048),
(None, 7, 7, 512),
(None, 7, 7, 512),
(None, 7, 7, 2048),
(None, 7, 7, 2048),
(None, 7, 7, 512),
(None, 7, 7, 512),
(None, 7, 7, 2048),
(None, 112, 112, 64),
(None, 1, 1, 2048)]
conv2d_3 (Conv2D) (None, 14, 14, 2) 18434
To me that looks like a hanging outputs from all intermediate layers of ResNet. Also, the execution time is somewhat too long for instantiation such a simple model.
How to avoid getting list of tensors as TensorFlowOpLayer and just link a single output to the rest of the model?