I am trying to rewrite a piece of tflearn code using Keras.
The goals is to combine two inputs where one input skips the first layer. The following code works in tflearn:
# Two different inputs.
inputs = tflearn.input_data(shape=[None, 10])
action = tflearn.input_data(shape=[None, 10])
#First layer used only by the inputs
net = tflearn.fully_connected(inputs, 400, activation='relu')
# Add the action tensor in the 2nd hidden layer
# Use two temp layers to get the corresponding weights and biases
t1 = tflearn.fully_connected(net, 300)
t2 = tflearn.fully_connected(action, 300)
# Combine the two layers using the weights from t1 and t2 and the bias from t2
net = tflearn.activation(tf.matmul(net,t1.W) + tf.matmul(action, t2.W) + t2.b, activation='relu')
I am trying to replicate this code in Keras using the following code:
# Two different inputs.
inputs = tf.placeholder(tf.float32, [None, 10])
action = tf.placeholder(tf.float32, [None, 10])
#First layer used only by the inputs
t1 = Sequential()
t1.add(Dense(400, activation='relu', input_shape=(1,10)))
# Add the action tensor in the 2nd hidden layer
# Use two temp layers to get the corresponding weights and biases
t1.add(Dense(300))
t2 = Sequential()
t2.add(Dense(300, input_shape=(1,10)))
# Combine the two layers
critnet = Sequential()
critnet.add(Merge([t1, t2], mode='sum'))
critnet.add(Activation('relu'))
# Create the net using the inputs and action placeholder
net = critnet([inputs, action])
The code in keras behaves differently. How to combine two layers in keras in order to get the same result as in tflearn?
You could use a Lambda layer take takes your 2 layers as input and using keras.backend to merge them the same way. I think there is K.dot for matmul.
Related
I came across this code for tuning the topology of the neural network. However I am unsure of how I can instantiate the first layer without flatening the input.
My input is like this:
With M features (the rows) and N samples (the columns).
How can I create the first (input) layer?
# Initialize sequential API and start building model.
model = keras.Sequential()
model.add(keras.layers.Flatten(input_shape=(28,28)))
# Tune the number of hidden layers and units in each.
# Number of hidden layers: 1 - 5
# Number of Units: 32 - 512 with stepsize of 32
for i in range(1, hp.Int("num_layers", 2, 6)):
model.add(
keras.layers.Dense(
units=hp.Int("units_" + str(i), min_value=32, max_value=512, step=32),
activation="relu")
)
# Tune dropout layer with values from 0 - 0.3 with stepsize of 0.1.
model.add(keras.layers.Dropout(hp.Float("dropout_" + str(i), 0, 0.3, step=0.1)))
# Add output layer.
model.add(keras.layers.Dense(units=10, activation="softmax"))
I know that Keras usually instantiates the first hidden layer along with the input layer, but I don't see how I can do it in this framework. Below is the code for instantiating input + first hidden layer at once.
model.add(Dense(100, input_shape=(CpG_num,), kernel_initializer='normal', activation='relu')
If you have multiple inputs and want to set your input shape, let's suppose you have a dataframe with m-> rows, n-> columns... then simply do this...
m = no_of_rows #1000
n = no_of_columns #10
no_of_layers = 64
#we will not write m because m will be taken as a batch here.
_input = tf.keras.layers.Input(shape=(n))
dense = tf.keras.layers.Dense(no_of_layers)(_input)
output = tf.keras.backend.function(_input , dense)
#Now, I can see that it is working or not...!
x = np.random.randn(1000 , 10)
print(output(x).shape)
How to combine two Keras models using functional API. I mean, I have two models (a, which is pretrained with freezer weights, and b). I want to create a c model by adding the b model to the bottom of the frozen model.
In detail, I have the following two models:
def define_neural_network_model_1st(input_shape, initializer, outputs = 1):
........
return model
def define_neural_network_model_2st(input_shape, initializer, outputs = 1):
........
return model
Since the first one is trained I am loading the weights and freezing the model.
neural_network_model_1st.load_weights('./../some_path.hdf5')
neural_network_model_1st.trainable = False
When I am trying to merge both blocks in the following way
merge_interpretation = Model(inputs=[neural_network_model_1st.inputs], outputs=neural_network_model_2st(neural_network_model_1st.inputs))
I am receiving:
What I am doing wrong? I am waiting to have 1 layer from the frozen model plus all layers in the second one.
Let suppose I have two models,
resnet_50 = tf.keras.applications.ResNet50(weights=None,
input_shape=(224 , 224 , 3),
classes = 2)
vgg_16 = tf.keras.applications.VGG16(
weights=None,
input_shape=(224,224,3),
classes=2,
include_top=False)
Now I want to merge these two models, first of all, I will make sure the output of the first model should be the same shape as the input of the second model, so for that first I have to do some pre-processing.
model = tf.keras.Model(vgg_16.inputs , vgg_16.layers[-2].output)
model2 = tf.keras.Model(resnet_50.get_layer('conv4_block6_out').input , resnet_50.output)
input = tf.keras.layers.Input(shape=(224 , 224 , 3))
out1 = model(input)
intermediate_layer = tf.keras.layers.Conv2D(model2.inputs[0][0].shape[2] , 1 , 1 , padding='same')(out1)
out2 = model2(intermediate_layer)
f_model = tf.keras.Model(input , out2)
tf.keras.utils.plot_model(
f_model,
show_shapes=True)
Now this is the output shape of the two models
[The Architecture of the combined two models][1]
[1]: https://i.stack.imgur.com/F7H8d.png
You can see the individual summary of the models by doing this
f_model.layers[1].summary() #This will show the summary of the first model VGG16
f_model.layers[3].summary() #This will show the summary of the second model Resnet18
But if you run the f_model.summary() this will not show the summary of the combined two models as one, because in the backend Keras take model one as a functional graph Node so it acts as a Node of the graph.
I want to feed in a 1-D CNN a sequence of fixed length and want it to make a prediction (regression), but I want to have a variable batch size during training. The tutorials are not really helpful.
In my input layer I have something like this:
input = tf.placeholder(tf.float32, [None, sequence_length], name="input")
y = tf.placeholder(tf.float32, [None, 1], name="y")
so I assume the None dimension, can be the a variable batch size of any number, so the current input dimension is batch_size * sequence_length and I am supposed to feed the network a 2d np array with dimensions any * sequence_length
tf.nn.conv1d expects 3-D, since my input is a single channel that is 1 np array of sequence_length observations the input I will need to feed to the cnn should be 1*batch_size * sequence_length, if I had on the other hand 2 different sequences that I combine to predict a single value in the end it would have been 2*batch_size * sequence_length and I would also need to concatenate the 2 different channels. So in my case I need
input = tf.expand_dims(input, -1)
and then the filter also follow the same:
filter_size = 5
channel_size = 1
num_filters = 10
filter_shape = [filter_size, channel_size, num_filters]
filters = tf.Variable(tf.truncated_normal(filter_shape, stddev=0.1), name="filters")
tf.nn.conv1d(value=input, filters=filters, stride=1)
After that I add a FC layer, but the network isn't able to learn anything, even the a basic function such as sin(x), does the code above look correct?
Also how can I do a maxpooling?
I don't fundamentally understand the shapes of arrays or how to determine the epochs and batch sizes of training data. My data has 6 columns, column 0 is the independent variable - a string, columns 1-4 are the Deep Neural Network inputs and column 5 is the binary outcome due to the inputs. I have 99 rows of data.
I want to understand how to get rid of this error.
#Importing Datasets
dataset=pd.read_csv('TestDNN.csv')
x = dataset.iloc[:,[1,5]].values # lower bound independent variable to upper bound in a matrix (in this case up to not including column5)
y = dataset.iloc[:,5].values # dependent variable vector
#Splitting data into Training and Test Data
from sklearn.model_selection import train_test_split
x_train, x_test, y_train, y_test = train_test_split(x,y, test_size=0.2, random_state=0)
#Feature Scaling
from sklearn.preprocessing import StandardScaler
sc = StandardScaler()
x_train = sc.fit_transform(x_train)
x_test=sc.transform(x_test)
# PART2 - Making ANN, deep neural network
#Importing the Keras libraries and packages
import keras
from keras.models import Sequential
from keras.layers import Dense
#Initialising ANN
classifier = Sequential()
#Adding the input layer and first hidden layer
classifier.add(Dense(activation= 'relu', input_dim =4, units=2,
kernel_initializer="uniform"))#rectifier activation function
#Adding second hidden layer
classifier.add(Dense(activation= 'relu', units=2,
kernel_initializer="uniform")) #rectifier activation function
#Adding the Output Layer
classifier.add(Dense(activation= 'sigmoid', units=1,
kernel_initializer="uniform"))
#Compiling ANN - stochastic gradient descent
classifier.compile(optimizer='adam', loss='binary_crossentropy',metrics=
['accuracy'])
#Fit ANN to training set
#PART 3 - Making predictions and evaluating the model
#Fitting classifier to the training set
classifier.fit(x_train, y_train, batch_size=32, epochs=5)#original batch is
10 and epoch is 100
The problem is with x definition. This line:
x = dataset.iloc[:,[1,5]].values
... tells pandas to take the columns 1 and 5 only, so it has shape [78, 2]. You probably meant taking all columns before the 5-th:
x = dataset.iloc[:,:5].values
Keras Dense layer needs an input_dim or input_shape to be specified. What value do I put in there?
My input is a matrix of 1,000,000 rows and only 3 columns. My output is 1,600 classes.
What do I put there?
dimensionality of the inputs (1000000, 1600)
2 because it's a 2D matrix
input_dim is the number of dimensions of the features, in your case that is just 3. The equivalent notation for input_shape, which is an actual dimensional shape, is (3,)
In your case
lets assume x and y=target variable and are look like as follows after feature engineering
x.shape
(1000000, 3)
y.shape
((1000000, 1600)
# as first layer in a sequential model:
model = Sequential()
model.add(Dense(32, input_shape=x.shape[1])) # Input layer
# now the model will take as input arrays of shape (*, 3)
# and output arrays of shape (*, 32)
...
...
model.add(Dense(y.shape[1],activation='softmax')) # Output layer
y.shape[1]= 1600, the number of output which is the number of classes you have, since you are dealing with Classification.
X = dataset.iloc[:, 3:13]
meaning the X parameter having all the rows and 3rd column till 12th column inclusive and 13th column exclusive.
We will also have a X0 parameter to be given to the neural network, so total
input layers becomes 10+1 = 11.
Dense(input_dim = 11, activation = 'relu', kernel_initializer = 'he_uniform')