Environment info
Operating System:
Rocks OS (Centos 6.5)
I installed from sources, and here is my version:
https://github.com/shiyemin/tensorflow/
Nothing changed but to make it compile successfully on our server.
ERROR
I use caffe-tensorflow to convert caffe model to tensorflow and the GoogLeNet is selected to construct our network.
I add a LSTM layer to this code as follows:
#layer
def lstm(self, input, lstm_type, n_steps, initial_state, num_units, name):
# with tf.variable_scope(name) as scope:
input_shape = input.get_shape()
dim = 1
for d in input_shape[1:].as_list():
dim *= d
input = tf.reshape(input, [input_shape[0].value, -1])
# select LSTM type, Define a lstm cell with tensorflow
if lstm_type == 'basic':
lstm_cell = rnn_cell.BasicLSTMCell(num_units, input_size=dim)
elif lstm_type == 'lstm':
lstm_cell = rnn_cell.LSTMCell(num_units, input_size=dim)
elif lstm_type == 'GRU':
lstm_cell = rnn_cell.GRUCell(num_units, input_size=dim)
else:
raise ValueError("LSTM type %s error."%lstm_type)
# Split data because rnn cell needs a list of inputs for the RNN inner loop
input = tf.split(0, n_steps, input) # n_steps * (batch_size, n_hidden)
# Get lstm cell output
outputs, states = rnn.rnn(lstm_cell, input, initial_state=initial_state) # , scope=scope)
outputs = tf.concat(0, outputs)
return outputs #, states
When i add this LSTM layer to GoogLeNet, the following error occurs.
Failed precondition: Attempting to use uninitialized value RNN/GRUCell/Gates/Linear/Bias
But when i using the code from:
https://github.com/aymericdamien/TensorFlow-Examples/blob/master/examples/3%20-%20Neural%20Networks/recurrent_network.py,
everything works well.
Anyone knows what happened? I don't know how to debug this error.
Related
I am creating a language model with a bidirecitonal LSTM, seq2seq model.
I have created the model and trained it successfully:
lstm_units = 100
# Set up embedding layer using pretrained weights
embedding_layer = Embedding(total_words+1, emb_dimension, input_length=max_input_len, weights=[embedding_matrix], name="Embedding")
# Encoder
encoder_input_x = Input(shape=(None,), name="Enc_x_Input")
encoder_embedding_x = embedding_layer(encoder_input_x)
encoder_lstm_x, enc_state_h_fwd, enc_state_c_fwd, enc_state_h_bwd, enc_state_c_bwd = Bidirectional(LSTM(lstm_units, dropout=0.5, return_state=True, name="Enc_LSTM1"), name="Enc_Bi1")(encoder_embedding_x) # pass hidden activation and memory cell states forward
encoder_state_h = Concatenate()([enc_state_h_fwd, enc_state_h_bwd])
encoder_state_c = Concatenate()([enc_state_c_fwd, enc_state_c_bwd])
encoder_states = [encoder_state_h, encoder_state_c] # package states to pass to decoder
# Decoder
decoder_input_x = Input(shape=(None,), name="Dec_x_Input")
decoder_embedding_x = embedding_layer(decoder_input_x)
decoder_lstm_layer = LSTM(lstm_units*2, return_state=True, return_sequences=True, dropout=0.5, name="Dec_LSTM1") # We define an LSTM layer without passing anything in here, as we will need to use this LSTM later.
decoder_lstm_x, _, _ = decoder_lstm_layer(decoder_embedding_x, initial_state=encoder_states) # we pass in encoder states
decoder_dense_layer = TimeDistributed(Dense(total_words+1, activation="softmax", name="Dec_Softmax")) # we set this dense to a variable so we can use it later, as above with the LSTM
decoder_output_x = decoder_dense_layer(decoder_lstm_x)
model = Model(inputs=[encoder_input_x, decoder_input_x], outputs=decoder_output_x)
model.compile(loss='sparse_categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
I then set up the inference model:
# Inference Encoder
inf_encoder_model = Model(encoder_input_x, encoder_states) # Here we are creating a model using layers from the model we built earlier.
# The encoder model outputs the encoder_states, ie. the concatenated h and c values from the BLSTM
# Inference Decoder
# Create new inputs for decoder state
inf_dec_state_h_input = Input(shape=(2*lstm_units,), name="Dec_h_state_input") # The must be sized to fit both FWD and BWD h values from the BLSTM
inf_dec_state_c_input = Input(shape=(2*lstm_units,), name="Dec_c_state_input")
inf_dec_state_input = [inf_dec_state_h_input, inf_dec_state_c_input] # package states to pass to decoder
# Decoder LSTM + Dense
inf_decoder_lstm_x, inf_dec_state_h, inf_dec_state_c = decoder_lstm_layer(decoder_embedding_x, initial_state=inf_dec_state_input) # reuse embedding layer from training. We pass in encoder states
inf_decoder_states = [inf_dec_state_h, inf_dec_state_c] # I think we we loop inference, we'll pass these states back in to the input instead of the encoder states
inf_decoder_output = decoder_dense_layer(inf_decoder_lstm_x)
decoder_model = Model([decoder_input_x] + inf_dec_state_input, [inf_decoder_output] + inf_decoder_states) # we reuse the decoder_input_x from the training model
The decoder model for inference is set up to take the decoder inputs + the c and h states which are output from the encoder.
When running the inference loop using this code:
states = inf_encoder_model.predict(x_inputs[700])
# Generate empty target sequence of length 1.
target_seq = np.zeros((max_output_len, 1), dtype=int)
# Populate the first character of target sequence with the start character.
target_seq[0, 0] = 4 # 4 is the start of sequence token used during training
# Get prediction
prediction, h, c = decoder_model.predict([target_seq] + states)
it gives me a long error that ends with:
ValueError: Layer Dec_LSTM1 expects 3 input(s), but it received 1 input tensors. Inputs received: [<tf.Tensor 'model_16/Embedding/embedding_lookup/Identity_1:0' shape=(None, 1, 100) dtype=float32>]
The encoder states seem to be fine; a list containing 2 arrays, the h and c values, each with shape (60, 200). The target_seq is an array of shape (1, 60). x_inputs[700] is training data, also of shape (1, 60).
Why is the model.predict line suggesting I am giving it 1 input tensor when I am giving it a list containing 3 arrays?
I am using Tensorflow 2.4.0 and Ubuntu 16.04
Given this model
base_model = tf.keras.applications.EfficientNetB0(weights="imagenet", include_top=False)
base_model.trainable = base_model_trainable
inputs = tf.keras.Input(shape=(IMG_SIZE,IMG_SIZE,3), name="input")
x = tf.keras.applications.efficientnet.preprocess_input(inputs)
# more details - https://www.tensorflow.org/tutorials/images/transfer_learning#important_note_about_batchnormalization_layers
x = base_model(x,
training=False) # training=training is needed only if there are layers with different behavior during training versus inference (e.g. Dropout)
x = tf.keras.layers.GlobalAveragePooling2D()(x)
x = tf.keras.layers.Dropout(0.2)(x)
outputs = tf.keras.layers.Dense(len(class_names), name="output")(x)
model = tf.keras.Model(inputs, outputs)
My objective is given the input to get the value of model.get_layer("efficientnetb0").output,
and model.output
if I do (a) (it does not give model.get_layer("efficientnetb0").output)
layer_name="efficientnetb0"
grad_model = tf.keras.models.Model([model.inputs], [model.output])
or (b) (it requires an additional input after preprocessing)
grad_model = tf.keras.models.Model([model.inputs, model.get_layer(layer_name).input], [model.get_layer(layer_name).output, model.output])
both works
but if I do
grad_model = tf.keras.models.Model([model.inputs], [model.get_layer(layer_name).output, model.output])
It gives exception:
ValueError: Graph disconnected: cannot obtain value for tensor KerasTensor(type_spec=TensorSpec(shape=(None, None, None, 3), dtype=tf.float32, name='input_1'), name='input_1', description="created by layer 'input_1'") at layer "rescaling". The following previous layers were accessed without issue: []
What is the reason, and is there a way for me to pass in just an input and get two outputs?
I am trying to get to run a bit of sample code from github in order to learn Working with Tensorflow 2 and the YOLO Framework. My Laptop has a M1000M Graphics Card and I installed the CUDA Platform from NVIDIA from here.
So the Code in question is this bit:
tf.compat.v1.disable_eager_execution()
_MODEL_SIZE = (416, 416)
_CLASS_NAMES_FILE = './data/labels/coco.names'
_MAX_OUTPUT_SIZE = 20
def main(type, iou_threshold, confidence_threshold, input_names):
class_names = load_class_names(_CLASS_NAMES_FILE)
n_classes = len(class_names)
model = Yolo_v3(n_classes=n_classes, model_size=_MODEL_SIZE,
max_output_size=_MAX_OUTPUT_SIZE,
iou_threshold=iou_threshold,
confidence_threshold=confidence_threshold)
if type == 'images':
batch_size = len(input_names)
batch = load_images(input_names, model_size=_MODEL_SIZE)
inputs = tf.compat.v1.placeholder(tf.float32, [batch_size, *_MODEL_SIZE, 3])
detections = model(inputs, training=False)
saver = tf.compat.v1.train.Saver(tf.compat.v1.global_variables(scope='yolo_v3_model'))
with tf.compat.v1.Session() as sess:
saver.restore(sess, './weights/model.ckpt')
detection_result = sess.run(detections, feed_dict={inputs: batch})
draw_boxes(input_names, detection_result, class_names, _MODEL_SIZE)
print('Detections have been saved successfully.')
While executing this (also wondering why starting the detection.py doesnt use GPU in the first place), I get the Error Message:
File "C:\SDKs etc\Python 3.8\lib\site-packages\tensorflow\python\client\session.py", line 1451, in _call_tf_sessionrun
return tf_session.TF_SessionRun_wrapper(self._session, options, feed_dict,
tensorflow.python.framework.errors_impl.UnimplementedError: The Conv2D op currently only supports the NHWC tensor format on the CPU. The op was given the format: NCHW
[[{{node yolo_v3_model/conv2d/Conv2D}}]]
Full Log see here.
If I am understanding this correctly, the format of inputs = tf.compat.v1.placeholder(tf.float32, [batch_size, *_MODEL_SIZE, 3]) is already NHWC (Model Size is a tuple of 2 Numbers) and I don't know how I need to change things in Code to get this running on CPU.
If I am understanding this correctly, the format of inputs =
tf.compat.v1.placeholder(tf.float32, [batch_size, *_MODEL_SIZE, 3]) is
already NHWC (Model Size is a tuple of 2 Numbers) and I don't know how
I need to change things in Code to get this running on CPU.
Yes you are. But look here:
def __init__(self, n_classes, model_size, max_output_size, iou_threshold,
confidence_threshold, data_format=None):
"""Creates the model.
Args:
n_classes: Number of class labels.
model_size: The input size of the model.
max_output_size: Max number of boxes to be selected for each class.
iou_threshold: Threshold for the IOU.
confidence_threshold: Threshold for the confidence score.
data_format: The input format.
Returns:
None.
"""
if not data_format:
if tf.test.is_built_with_cuda():
data_format = 'channels_first'
else:
data_format = 'channels_last'
And later:
def __call__(self, inputs, training):
"""Add operations to detect boxes for a batch of input images.
Args:
inputs: A Tensor representing a batch of input images.
training: A boolean, whether to use in training or inference mode.
Returns:
A list containing class-to-boxes dictionaries
for each sample in the batch.
"""
with tf.compat.v1.variable_scope('yolo_v3_model'):
if self.data_format == 'channels_first':
inputs = tf.transpose(inputs, [0, 3, 1, 2])
Solution:
check tf.test.is_built_with_cuda() work as expected
if not - set order manually when create model:
model = Yolo_v3(n_classes=n_classes, model_size=_MODEL_SIZE,
max_output_size=_MAX_OUTPUT_SIZE,
iou_threshold=iou_threshold,
confidence_threshold=confidence_threshold,
data_format = 'channels_last')
When saving a model, in my case, HRNet, i met the following error:
ValueError: Unable to save the object ListWrapper([ListWrapper([None])])
Some of my layer looks like(Full version: https://github.com/zheLim/auto-face-parsing/blob/master/lib/model/hrnet_blocks.py):
class MultiResolutionLayer(layers.Layer):
def __init__(self, n_channels_list, bn_momentum=0.01, activation='relu'):
"""
fuse feature from different branch with adding
:param n_branches:
:param n_channels:
:param multi_scale_output:
"""
super(MultiResolutionLayer, self).__init__()
self.n_branches = len(n_channels_list)
self.fuse_layers = [[] for branch_i in range(self.n_branches)]
for branch_i in range(self.n_branches):
layer = []
for branch_j in range(self.n_branches):
if branch_i < branch_j:
# resolution of branch i is greater than branch_j
# branch_j will be upsample with nearest resize
layer.append(keras.Sequential(
[layers.Conv2D(filters=n_channels_list[branch_i], kernel_size=1, strides=1, padding='same',
use_bias=False, activation=activation),
layers.BatchNormalization(momentum=bn_momentum)]))
elif branch_i == branch_j:
# branch i is branch_j
layer.append(None)
The error message suggest that "If you don't need this list checkpointed, wrap it in a tf.contrib.checkpoint.NoDependency object; it will be automatically un-wrapped and subsequently ignored." However, tf 2.0 has no contri module.
Any one knows how to solve this problem?
I have been working on LSTM for timeseries forecasting by using tensorflow. Now, i want to try sequence to sequence (seq2seq). In the official site there is a tutorial which shows NMT with embeddings . So, how can I use this new seq2seq module without embeddings? (directly using time series "sequences").
# 1. Encoder
encoder_cell = tf.contrib.rnn.BasicLSTMCell(LSTM_SIZE)
encoder_outputs, encoder_state = tf.nn.static_rnn(
encoder_cell,
x,
dtype=tf.float32)
# Decoder
decoder_cell = tf.nn.rnn_cell.BasicLSTMCell(LSTM_SIZE)
helper = tf.contrib.seq2seq.TrainingHelper(
decoder_emb_inp, decoder_lengths, time_major=True)
decoder = tf.contrib.seq2seq.BasicDecoder(
decoder_cell, helper, encoder_state)
# Dynamic decoding
outputs, _ = tf.contrib.seq2seq.dynamic_decode(decoder)
outputs = outputs[-1]
# output is result of linear activation of last layer of RNN
weight = tf.Variable(tf.random_normal([LSTM_SIZE, N_OUTPUTS]))
bias = tf.Variable(tf.random_normal([N_OUTPUTS]))
predictions = tf.matmul(outputs, weight) + bias
What should be the args for TrainingHelper() if I use input_seq=x and output_seq=label?
decoder_emb_inp ???
decoder_lengths ???
Where input_seq are the first 8 point of the sequence, and output_seq are the last 2 point of the sequence.
Thanks on advance!
I got it to work for no embedding using a very rudimentary InferenceHelper:
inference_helper = tf.contrib.seq2seq.InferenceHelper(
sample_fn=lambda outputs: outputs,
sample_shape=[dim],
sample_dtype=dtypes.float32,
start_inputs=start_tokens,
end_fn=lambda sample_ids: False)
My inputs are floats with the shape [batch_size, time, dim]. For the example below dim would be 1, but this can easily be extended to more dimensions. Here's the relevant part of the code:
projection_layer = tf.layers.Dense(
units=1, # = dim
kernel_initializer=tf.truncated_normal_initializer(
mean=0.0, stddev=0.1))
# Training Decoder
training_decoder_output = None
with tf.variable_scope("decode"):
# output_data doesn't exist during prediction phase.
if output_data is not None:
# Prepend the "go" token
go_tokens = tf.constant(go_token, shape=[batch_size, 1, 1])
dec_input = tf.concat([go_tokens, target_data], axis=1)
# Helper for the training process.
training_helper = tf.contrib.seq2seq.TrainingHelper(
inputs=dec_input,
sequence_length=[output_size] * batch_size)
# Basic decoder
training_decoder = tf.contrib.seq2seq.BasicDecoder(
dec_cell, training_helper, enc_state, projection_layer)
# Perform dynamic decoding using the decoder
training_decoder_output = tf.contrib.seq2seq.dynamic_decode(
training_decoder, impute_finished=True,
maximum_iterations=output_size)[0]
# Inference Decoder
# Reuses the same parameters trained by the training process.
with tf.variable_scope("decode", reuse=tf.AUTO_REUSE):
start_tokens = tf.constant(
go_token, shape=[batch_size, 1])
# The sample_ids are the actual output in this case (not dealing with any logits here).
# My end_fn is always False because I'm working with a generator that will stop giving
# more data. You may extend the end_fn as you wish. E.g. you can append end_tokens
# and make end_fn be true when the sample_id is the end token.
inference_helper = tf.contrib.seq2seq.InferenceHelper(
sample_fn=lambda outputs: outputs,
sample_shape=[1], # again because dim=1
sample_dtype=dtypes.float32,
start_inputs=start_tokens,
end_fn=lambda sample_ids: False)
# Basic decoder
inference_decoder = tf.contrib.seq2seq.BasicDecoder(dec_cell,
inference_helper,
enc_state,
projection_layer)
# Perform dynamic decoding using the decoder
inference_decoder_output = tf.contrib.seq2seq.dynamic_decode(
inference_decoder, impute_finished=True,
maximum_iterations=output_size)[0]
Have a look at this question. Also I found this tutorial to be very useful to understand seq2seq models, although it does use embeddings. So replace their GreedyEmbeddingHelper by an InferenceHelper like the one I posted above.
P.s. I posted the full code at https://github.com/Andreea-G/tensorflow_examples