Keras Layer Concatenation - tensorflow

Question

I'm trying to see how I can create a model in Keras with multiple Embedding Layers and other inputs. Here's how my model is structured(E=Embedding Layer, [....]=Input Layer):
E E [V V V]
\ | /
\ | /
Dense
|
Dense
Here is my code so far:
model_a = Sequential()
model_a.add(Embedding(...))
model_b = Sequential()
model_b.add(Embedding(...))
model_c = Sequential()
model_c.add(Embedding(...))
model_values = Sequential()
model_values.add(Input(...))
classification_model = Sequential()
classification_layers = [
Concatenate([model_a,model_b,model_c, model_values]),
Dense(...),
Dense(...),
Dense(2, activation='softmax')
]
for layer in classification_layers:
classification_model.add(layer)
classification_model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
classification_model.fit(train_data,one_hot_labels, epochs=1, validation_split=0.2)
However I get the following error:
ValueError: A `Concatenate` layer should be called on a list of at least 2 inputs
I am at a loss at what I'm doing wrong here. Here's the a little more detail for the error log:
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-37-d5ab23b17e9d> in <module>()
----> 1 classification_model.fit(train_data,one_hot_labels, epochs=1, validation_split=0.2)
/usr/local/lib/python3.5/dist-packages/keras/engine/training.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, **kwargs)
953 sample_weight=sample_weight,
954 class_weight=class_weight,
--> 955 batch_size=batch_size)
956 # Prepare validation data.
957 do_validation = False
/usr/local/lib/python3.5/dist-packages/keras/engine/training.py in _standardize_user_data(self, x, y, sample_weight, class_weight, check_array_lengths, batch_size)
674 # to match the value shapes.
675 if not self.inputs:
--> 676 self._set_inputs(x)
677
678 if y is not None:
/usr/local/lib/python3.5/dist-packages/keras/engine/training.py in _set_inputs(self, inputs, outputs, training)
574 assert len(inputs) == 1
575 inputs = inputs[0]
--> 576 self.build(input_shape=(None,) + inputs.shape[1:])
577 return
578
/usr/local/lib/python3.5/dist-packages/keras/engine/sequential.py in build(self, input_shape)
225 self.inputs = [x]
226 for layer in self._layers:
--> 227 x = layer(x)
228 self.outputs = [x]
229
/usr/local/lib/python3.5/dist-packages/keras/engine/base_layer.py in __call__(self, inputs, **kwargs)
430 '`layer.build(batch_input_shape)`')
431 if len(input_shapes) == 1:
--> 432 self.build(input_shapes[0])
433 else:
434 self.build(input_shapes)
/usr/local/lib/python3.5/dist-packages/keras/layers/merge.py in build(self, input_shape)
339 # Used purely for shape validation.
340 if not isinstance(input_shape, list) or len(input_shape) < 2:
--> 341 raise ValueError('A `Concatenate` layer should be called '
342 'on a list of at least 2 inputs')
343 if all([shape is None for shape in input_shape]):
ValueError: A `Concatenate` layer should be called on a list of at least 2 inputs

Answer 1

input1 = Input(input_shape=...)
input2 = Input(...)
input3 = Input(...)
values = Input(...)
out1 = Embedding(...)(input1)
out2 = Embedding(...)(input2)
out3 = Embedding(...)(input3)
#make sure values has a shape compatible with the embedding outputs.
#usually it should have shape (equal_samples, equal_length, features)
joinedInput = Concatenate()([out1,out2,out3,values])
out = Dense(...)(joinedInput)
out = Dense(...)(out)
out = Dense(2, activation='softmax')(out)
model = Model([input1,input2,input3,values], out)

Answer 2

You are missing the 'axis' parameter in the call. If you are concatenating on the last dimension (it's unclear what the dimensions of these embeddings and input tensors are), use:
concatenate([model_a,model_b,model_c, model_values], axis=-1)