I am trying to create a Keras model with three inputs. Only one of them goes through the first few layers and the other two are concatenated at a dense layer. How would I achieve this without disconnecting the graph? The code is shown below
import keras
input_img = Input(shape=(784,))
input_1 = Input(shape=(1,))
input_2 = Input(shape=(1,))
x = (Dense(48,kernel_initializer='normal',activation="relu"))(input_img)
x = (Dropout(0.2))(x)
x = (Dense(24,activation="tanh"))(x)
x = (Dropout(0.3))(x)
x = (Dense(1))(x)
x = keras.layers.concatenate([x, input_1, input_2])
x = (Activation("sigmoid"))(x)
x = Model(input_img, x)
x.compile(loss="binary_crossentropy", optimizer='adam')
To give a more general overview of what I'm attempting, I am essentially trying to create a Convolutional Neural Network with additional features added to the dense layer for classification.
Since your model has three inputs, i.e. input_img, input_1 and input_2 you need to pass a list of these three inputs while defining your model as follows:
x = Model([input_img, input_1, input_2], x)
Hope this helps.
Related
I would like to use a neural network in Keras that takes 2 inputs of different sizes (a vector v and a matrix A) and outputs a vector u, which is v after acted upon by A.
I have managed to input the matrix and vector. The problem is, when I try to use the vector u as the target when fitting the model, it complains:
ValueError: Data cardinality is ambiguous:
x sizes: 70, 312
y sizes: 70
Make sure all arrays contain the same number of samples.
The best option in your case zero padding or padding up would likely be the best decision in your situation. Inputs are only zeroed out in these situations to account for the absence of data. It is frequently used for CNN pictures' borders.
An RNN, which is a simpler option and can easily accommodate your variable-length inputs, is another option.
Here is the example code. I think it can help you.
input_layer = Input(shape=(None, None, channels))
x = Conv2D(16,(4,4), activation = 'relu')(input_layer)
x = Conv2D(32,(4,4), activation = 'relu')(x)
x = Dropout(0.2)(x)
x = Conv2D(64,(4,4), activation = 'relu')(x)
x = Dropout(0.5)(x)
x = Conv2D(128, (1,1))(x)
x = GlobalAveragePooling2D()(x)
output_layer = Dense(5, activation = "softmax")(x)
model = Model(inputs = input_layer, outputs=output_layer)
model.compile(optimizer = "adam", loss = "categorical_crossentropy",
metrics=["accuracy"])
I have created four 3D-CNN models and each was trained on a different (but related) set of images, such that each set has images of a different perspective of the same objects. (i.e: n objects have images from 4 different perspectives, each model is associated to a single perspective).
def get_model(width=128, height=128, depth=4):
inputs = Input((width, height, depth, 3))
x = Conv3D(filters=64, kernel_size=8,padding='same', activation="relu")(inputs)
x = MaxPool3D(pool_size=2,data_format= "channels_first", padding='same')(x)
x = BatchNormalization()(x)
x = Conv3D(filters=256, kernel_size=3,padding='same', activation="relu")(x)
x = MaxPool3D(pool_size=2,data_format= "channels_first", padding='same')(x)
x = BatchNormalization()(x)
x = GlobalAveragePooling3D()(x)
x = Dense(units=512, activation="relu")(x)
x = Dropout(0.3)(x)
outputs = Dense(units=2, activation="sigmoid")(x)
# Define the model.
model = keras.Model(inputs, outputs)
return model
I have now four pre-trained models, and I would like to combine them by removing the last dense layer (sigmoid) and instead, concatenating the dense layers of all the four models followed by an activation function (i.e: sigmoid). I would like to keep four input layers such that each will take an image of an object from one perspective. I have seen examples of concatenating an output layer of model_1 to the input layer of model_2, however, I am not sure how to deal with four separate input layers and concatenating towards the end of the model.
Let's assume that you have your pretrained model files named "A.h5" and "B.h5". You can simply load them in TensorFlow, access the layer that interrest you with the layers attribute, and merge them with the Functional API. One example could be the following :
import tensorflow as tf
pretrainedmodel_files = ["A.h5", "B.h5"]
A,B = [tf.keras.models.load_model(filename) for filename in pretrainedmodel_files]
# Skipping the last dense layer and the dropout means accessing the layer at the index -3
concat = tf.keras.layers.Concatenate()([A.layers[-3].output, B.layers[-3].output])
out = tf.keras.layers.Dense(2,activation="sigmoid")(concat)
model = tf.keras.Model(inputs=[A.input, B.input], outputs=out)
I've created two simple model with the following code:
tf.keras.Sequential(
[
tf.keras.layers.Dense(10, activation="relu", input_shape=(5,)),
tf.keras.layers.Dropout(0.3),
tf.keras.layers.Dense(2, activation="sigmoid")
]
)
And then merged them together with my sample code.
A and B have the following architecture (visualization with netron):
And the merged network:
So I am trying to build an LSTM based autoencoder, which I want to use for the time series data. These are spitted up to sequences of different lengths. Input to the model has thus shape [None, None, n_features], where the first None stands for number of samples and the second for time_steps of the sequence. The sequences are processed by LSTM with argument return_sequences = False, coded dimension is then recreated by function RepeatVector and ran through LSTM again. In the end I would like to use the TimeDistributed layer, but how to tell python that the time_steps dimension is dynamic? See my code:
from keras import backend as K
.... other dependencies .....
input_ae = Input(shape=(None, 2)) # shape: time_steps, n_features
LSTM1 = LSTM(units=128, return_sequences=False)(input_ae)
code = RepeatVector(n=K.shape(input_ae)[1])(LSTM1) # bottleneck layer
LSTM2 = LSTM(units=128, return_sequences=True)(code)
output = TimeDistributed(Dense(units=2))(LSTM2) # ??????? HOW TO ????
# no problem here so far:
model = Model(input_ae, outputs=output)
model.compile(optimizer='adam', loss='mse')
this function seems to do the trick
def repeat(x_inp):
x, inp = x_inp
x = tf.expand_dims(x, 1)
x = tf.repeat(x, [tf.shape(inp)[1]], axis=1)
return x
example
input_ae = Input(shape=(None, 2))
LSTM1 = LSTM(units=128, return_sequences=False)(input_ae)
code = Lambda(repeat)([LSTM1, input_ae])
LSTM2 = LSTM(units=128, return_sequences=True)(code)
output = TimeDistributed(Dense(units=2))(LSTM2)
model = Model(input_ae, output)
model.compile(optimizer='adam', loss='mse')
X = np.random.uniform(0,1, (100,30,2))
model.fit(X, X, epochs=5)
I'm using tf.keras with TF 2.2
Greatly appreciate it if someone could help me out here:
I'm trying to do some finetuning on a regression task --- my inputs are 200X200 RGB images and my prediction output/label is a set of real values (let's say, within [0,10], though scaling is not a big deal here...?) --- on top of InceptionV3 architecture. Here are my functions that take a pretrained Inception model, remove the last layer and add a a new layer, set up for finetuning...
"""
Fine-tuning functions
"""
IM_WIDTH, IM_HEIGHT = 299, 299 #fixed size for InceptionV3
NB_EPOCHS = 3
BAT_SIZE = 32
FC_SIZE = 1024
NB_IV3_LAYERS_TO_FREEZE = 172
def eucl_dist(inputs):
x, y = inputs
return ((x - y)**2).sum(axis=-1)
def add_new_last_continuous_layer(base_model):
"""Add last layer to the convnet
Args:
base_model: keras model excluding top, for instance:
base_model = InceptionV3(weights='imagenet',include_top=False)
Returns:
new keras model with last layer
"""
x = base_model.output
x = GlobalAveragePooling2D()(x)
x = Dense(FC_SIZE, activation='relu')(x)
predictions = Lambda(eucl_dist, output_shape=(1,))(x)
model = Model(input=base_model.input, output=predictions)
return model
def setup_to_finetune_continuous(model):
"""Freeze the bottom NB_IV3_LAYERS and retrain the remaining top
layers.
note: NB_IV3_LAYERS corresponds to the top 2 inception blocks in
the inceptionv3 architecture
Args:
model: keras model
"""
for layer in model.layers[:NB_IV3_LAYERS_TO_FREEZE]:
layer.trainable = False
for layer in model.layers[NB_IV3_LAYERS_TO_FREEZE:]:
layer.trainable = True
model.compile(optimizer=SGD(lr=0.0001, momentum=0.9),
loss='eucl_dist')
Here are my implementations:
base_model = InceptionV3(weights = "imagenet",
include_top=False, input_shape=(3,200,200))
model0 = add_new_last_continuous_layer(base_model)
setup_to_finetune_continuous(model0)
history=model0.fit(train_x, train_y, validation_data = (test_x, test_y), nb_epoch=epochs, batch_size=32)
scores = model0.evaluate(test_x, test_y, verbose = 0)
features = model0.predict(X_train)
where train_x is a (168435, 3, 200, 200) numpy array and train_y is a (168435,) numpy array. The same goes for test_x and test_y except the number of observations is 42509.
I got the TypeError: Tensor object is not iterable bug which occurred at predictions = Lambda(eucl_dist, output_shape=(1,))(x)'' when going through theadd_new_last_continuous_layer()`` function. Could you anyone kindly give me some guidance to get around that and what the problem is? Greatly appreciated and happy holidays!
EDIT:
Changed the functions to:
def eucl_dist(inputs):
x, y = inputs
return ((x - y)**2).sum(axis=-1)
def add_new_last_continuous_layer(base_model):
"""Add last layer to the convnet
Args:
base_model: keras model excluding top, for instance:
base_model = InceptionV3(weights='imagenet',include_top=False)
Returns:
new keras model with last layer
"""
x = base_model.output
x = GlobalAveragePooling2D()(x)
x1 = Dense(FC_SIZE, activation='relu')(x)
x2 = Dense(FC_SIZE, activation='relu')(x)
predictions = Lambda(eucl_dist, output_shape=eucl_dist_shape)([x1,x2])
model = Model(input=base_model.input, output=predictions)
return model
Your output shape for the lambda layer is wrong. Define your functions like this:
from keras import backend as K
def euclidean_distance(vects):
x, y = vects
return K.sqrt(K.maximum(K.sum(K.square(x - y), axis=1, keepdims=True), K.epsilon()))
def eucl_dist_output_shape(shapes):
shape1, shape2 = shapes
return (shape1[0], 1)
predictions = Lambda(euclidean_distance, output_shape=eucl_dist_output_shape)([input1, input2])
what I have is the following, which I believe is a network with one hidden LSTM layer:
# Parameters
learning rate = 0.001
training_iters = 100000
batch_size = 128
display_step = 10
# Network Parameters
n_input = 13
n_steps = 10
n_hidden = 512
n_classes = 13
# tf Graph input
x = tf.placeholder("float", [None, n_steps, n_input])
y = tf.placeholder("float", [None, n_classes])
# Define weights
weights = {
'out' : tf.Variable(tf.random_normal([n_hidden, n_classes]))
}
biases = {
'out' : tf.Variable(tf.random_normal([n_classes]))
}
However, I am trying to build an LSTM network using TensorFlow to predict power consumption. I have been looking around to find a good example, but I could not find any model with 2 hidden LSTM layers. Here's the model that I would like to build:
1 input layer,
1 output layer,
2 hidden LSTM layers(with 512 neurons in each),
time step(sequence length): 10
Could anyone guide me to build this using TensorFlow? ( from defining weights, building input shape, training, predicting, use of optimizer or cost function, etc), any help would be much appreciated.
Thank you so much in advance!
Here is how I do it in a translation model with GRU cells. You can just replace the GRU with an LSTM. It is really easy just use tf.nn.rnn_cell.MultiRNNCell with a list of the multiple cells it should wrap. In the code bellow I am manually unrolling it but you can pass it to tf.nn.dynamic_rnn or tf.nn.rnn as well.
y = input_tensor
with tf.variable_scope('encoder') as scope:
rnn_cell = rnn.MultiRNNCell([rnn.GRUCell(1024) for _ in range(3)])
state = tf.zeros((BATCH_SIZE, rnn_cell.state_size))
output = [None] * TIME_STEPS
for t in reversed(range(TIME_STEPS)):
y_t = tf.reshape(y[:, t, :], (BATCH_SIZE, -1))
output[t], state = rnn_cell(y_t, state)
scope.reuse_variables()
y = tf.pack(output, 1)
First you need some placeholders to put your training data (one batch)
x_input = tf.placeholder(tf.float32, [batch_size, truncated_series_length, 1])
y_output = tf.placeholder(tf.float32, [batch_size, truncated_series_length, 1])
A LSTM need a state, which consists of two components, the hidden state and the cell state, very good guide here: https://arxiv.org/pdf/1506.00019.pdf. For every layer in the LSTM you have one cell state and one hidden state.
The problem is that Tensorflow stores this in a LSTMStateTuple which you can not send into placeholder. So you need to store it in a Tensor, and then unpack it into a tuple:
state_placeholder = tf.placeholder(tf.float32, [num_layers, 2, batch_size, state_size])
l = tf.unpack(state_placeholder, axis=0)
rnn_tuple_state = tuple(
[tf.nn.rnn_cell.LSTMStateTuple(l[idx][0], l[idx][1])
for idx in range(num_layers)]
)
Then you can use the built-in Tensorflow API to create the stacked LSTM layer.
cell = tf.nn.rnn_cell.LSTMCell(state_size, state_is_tuple=True)
cell = tf.nn.rnn_cell.MultiRNNCell([cell]*num_layers, state_is_tuple=True)
outputs, state = tf.nn.dynamic_rnn(cell, x_input, initial_state=rnn_tuple_state)
From here you continue with the outputs to calculate logits and then a loss with respect to the y_inputs.
Then you run each batch with the sess.run-command, with truncated backpropagation (good explanation here http://r2rt.com/styles-of-truncated-backpropagation.html)
init_state = np.zeros((num_layers, 2, batch_size, state_size))
...current_state... = sess.run([...state...], feed_dict={x_input:batch_in, state_placeholder:current_state ...})
current_state = np.array(current_state)
You will have to convert the state to a numpy array before feeding it again.
Perhaps it is better to use a librarly like Tflearn or Keras instead?