Export Model For Serving But Tensor Type Dismatch - tensorflow

What related GitHub issues or StackOverflow threads have you found by searching the web for your problem?
I am trying to export the model for serving , but it's report type error about inputs tensor.
but in the export and predict part , the inputs are the same type.
If possible, provide a minimal reproducible example (We usually don't have time to read hundreds of lines of your code)
here is a sample code for my exporting
```
named_graph_signature = {
'inputs': exporter.generic_signature({
'sparse_index': tf.placeholder(tf.int64, name="feature_index")
'sparse_ids': tf.placeholder(tf.int64,name = "feature_ids"),
'sparse_values':tf.placeholder(tf.int64, name ="feature_values"),
'sparse_shape':tf.placeholder(tf.int64, name="feature_shape")
}),
'outputs': exporter.generic_signature({
'prob': inference_softmax
})}
model_exporter.init(
sess.graph.as_graph_def(),
#default_graph_signature=named_graph_signature,
named_graph_signatures=named_graph_signature,
init_op=init_op)
model_exporter.export(export_path, tf.constant(export_version), sess)
print('Done exporting!')
```
here is my code for predicting
```
ins = "0 142635:1 250810:1 335229:1 375278:1 392970:1 506983:1 554566:1 631968:1 647823:1 658803:1 733446:1 856305:1 868202:1"
FEATURE_SIZE = 1000000
tokens = ins.split(" ")
feature_num = 0
feature_ids = []
feature_values = []
feature_index = []
for feature in tokens[1:]:
feature_id, feature_value = feature.split(":")
feature_ids.append(int(feature_id))
feature_values.append(float(feature_value))
feature_index.append([1, feature_num])
feature_num += 1
feature_shape = [1, FEATURE_SIZE]
sparse_index = tf.contrib.util.make_tensor_proto(numpy.asarray(feature_index), dtype=tf.int64)
sparse_ids = tf.contrib.util.make_tensor_proto(numpy.asarray(feature_ids), dtype=tf.int64)
sparse_values = tf.contrib.util.make_tensor_proto(numpy.asarray(feature_values), dtype=tf.float32)
sparse_shape= tf.contrib.util.make_tensor_proto(numpy.asarray(feature_shape), dtype=tf.int64)
channel = implementations.insecure_channel(host, port)
stub = prediction_service_pb2.beta_create_PredictionService_stub(channel)
request = predict_pb2.PredictRequest()
request.model_spec.name = model_name
request.model_spec.version.value = model_version
print model_name,model_version
request.inputs['sparse_index'].CopyFrom(sparse_index)
request.inputs['sparse_ids'].CopyFrom(sparse_ids)
request.inputs['sparse_values'].CopyFrom(sparse_values)
request.inputs['sparse_shape'].CopyFrom(sparse_shape)
# Send request
result = stub.Predict(request, request_timeout)
```
Logs or other output that would be helpful
(If logs are large, please upload as attachment or provide link).
```
Traceback (most recent call last):
File "run.py", line 63, in <module>
main()
File "run.py", line 59, in main
result = stub.Predict(request, request_timeout)
File "/home/serving/.local/lib/python2.7/site-packages/grpc/beta/_client_adaptations.py", line 305, in __call__
self._request_serializer, self._response_deserializer)
File "/home/serving/.local/lib/python2.7/site-packages/grpc/beta/_client_adaptations.py", line 203, in _blocking_unary_unary
raise _abortion_error(rpc_error_call)
grpc.framework.interfaces.face.face.AbortionError: AbortionError(code=StatusCode.INVALID_ARGUMENT, details="input size does not match signature")
```

Related

Skopt gp_minimize eroor while looking for optimal point

I have a problem with gp_minimize from skopt. I'm doing hyperparameter tunning for transfer learning (base model vgg19). Everything works fine until gp_minimize starts to evaluate optimal point (evaluate random point works fine). I have this error, i spend few days on this and i still don't know what to do:
Traceback (most recent call last):
File "C:/Users/mea/Train_models/04_VGG19_train_model.py", line 144, in <module>
search_result = gp_minimize(func=fitness,
File "C:\Users\me\anaconda3\envs\Train_models\lib\site-packages\skopt\optimizer\gp.py", line 259, in gp_minimize
return base_minimize(
File "C:\Users\Wme\anaconda3\envs\Train_models\lib\site-packages\skopt\optimizer\base.py", line 300, in base_minimize
result = optimizer.tell(next_x, next_y)
File "C:\Users\mea\anaconda3\envs\Train_models\lib\site-packages\skopt\optimizer\optimizer.py", line 493, in tell
return self._tell(x, y, fit=fit)
File "C:\Users\me\anaconda3\envs\Train_models\lib\site-packages\skopt\optimizer\optimizer.py", line 552, in _tell
X = self.space.transform(self.space.rvs(
File "C:\Users\me\anaconda3\envs\Train_models\lib\site-packages\skopt\space\space.py", line 900, in rvs
columns.append(dim.rvs(n_samples=n_samples, random_state=rng))
File "C:\Users\Weronika Gramacka\anaconda3\envs\Train_models\lib\site-packages\skopt\space\space.py", line 698, in rvs
return self.inverse_transform(list(choices))
File "C:\Users\me\anaconda3\envs\Train_models\lib\site-packages\skopt\space\space.py", line 685, in inverse_transform
inv_transform = super(Categorical, self).inverse_transform(Xt)
File "C:\Users\me\anaconda3\envs\Train_models\lib\site-packages\skopt\space\space.py", line 168, in inverse_transform
return self.transformer.inverse_transform(Xt)
File "C:\Users\me\anaconda3\envs\Train_models\lib\site-packages\skopt\space\transformers.py", line 309, in inverse_transform
X = transformer.inverse_transform(X)
File "C:\Users\me\anaconda3\envs\Train_models\lib\site-packages\skopt\space\transformers.py", line 216, in inverse_transform
return [
File "C:\Users\me\anaconda3\envs\Train_models\lib\site-packages\skopt\space\transformers.py", line 217, in <listcomp>
self.inverse_mapping_[int(np.round(i))] for i in Xt
KeyError: 6
Process finished with exit code 1
Code is from tutorial and looks like this (only important parts):
dim_num_dense_layers = Integer(low=1, high=3, name='num_dense_layers')
dim_num_dense_nodes = Integer(low=60, high=1500, name='num_dense_nodes')
dim_activation = Categorical(categories=[ 'sigmoid', 'tanh', 'relu', 'softmax'],
name='activation')
dim_dropout = Real(low = 0.01, high = 0.4, name = 'dropout')
dim_init = Categorical(categories = ['uniform', 'lecun_uniform', 'normal', 'zero', 'glorot_normal', 'glorot_uniform', 'he_normal', 'he_uniform'], name = 'kernel_initializer')
dim_loss = Categorical(categories = ['categorical_crossentropy', 'categorical_hinge', 'mean_squared_error', 'huber_loss'], name = 'loss')
dimensions = [dim_num_dense_layers,
dim_num_dense_nodes,
dim_activation,
dim_dropout,
dim_init,
dim_loss]
default_parameters = [2, 600, 'relu', 0.2, 'uniform', 'huber_loss']
#use_named_args(dimensions=dimensions)
def fitness(num_dense_layers, num_dense_nodes, activation, dropout, kernel_initializer, loss):
# Print the hyper-parameters.
keras.backend.clear_session()
model = create_model(num_dense_layers, num_dense_nodes, activation, dropout, kernel_initializer, loss)
log_dir = "Tensor_board/04/" + datetime.datetime.now().strftime("%Y%m%d-%H%M%S")
tensorboard_callback = TensorBoard(
log_dir='log_dir',
histogram_freq=1,
write_graph=True,
write_grads=False,
write_images=False)
history = model.fit(train_data,
epochs=1,
batch_size=32,
validation_data=val_data,
callbacks=[tensorboard_callback])
accuracy = max(history.history['val_accuracy'])
global best_accuracy
if accuracy > best_accuracy:
model.save("Models/vgg19_flat.h5")
best_accuracy = accuracy
del model
gc.collect()
keras.backend.clear_session()
return -accuracy
checkpoint_saver = CheckpointSaver("./checkpoint.pkl", compress=9)
search_result = gp_minimize(func=fitness,
dimensions=dimensions,
acq_func='EI',
n_calls=30,
n_initial_points=1,
x0=default_parameters, verbose=True, callback=[checkpoint_saver])

DecodeError: Truncated message.while running Graph.ParseFromString()

I downloaded the code from tensorflow/tensorflow/image_retraining/retrain.py and modified a little to retrain the model based on my need(like path to find training images folder and location to save the model and labels etc). While running the retrain.py file I get the following message towards the end of the execution
An exception has occurred, use %tb to see the full traceback.
SystemExit
On viewing the stack trace using %tb I get
Traceback (most recent call last):
File "<ipython-input-11-06ad74d82e7c>", line 1, in <module>
runfile('C:/Users/Srikanth1.R/Desktop/Desktop/My_Folder/Inage analytics/hub-master/examples/image_retraining/retrain.py', wdir='C:/Users/Srikanth1.R/Desktop/Desktop/My_Folder/Inage analytics/hub-master/examples/image_retraining')
File "C:\Users\Srikanth1.R\AppData\Local\Continuum\anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py", line 705, in runfile
execfile(filename, namespace)
File "C:\Users\Srikanth1.R\AppData\Local\Continuum\anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py", line 102, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)
File "C:/Users/Srikanth1.R/Desktop/Desktop/My_Folder/Inage analytics/hub-master/examples/image_retraining/retrain.py", line 2424, in <module>
tf.app.run(main=main, argv=[sys.argv[0]] + unparsed)
File "C:\Users\Srikanth1.R\AppData\Local\Continuum\anaconda3\lib\site-packages\tensorflow\python\platform\app.py", line 134, in run
SystemExit
But although I get a model.pb file.
I used this file in my prediction of new images. But while Parsing the model.pb file the console is throwing the following error
File "C:\Users\Srikanth1.R\AppData\Local\Continuum\anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py", line 705, in runfile
execfile(filename, namespace)
File "C:\Users\Srikanth1.R\AppData\Local\Continuum\anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py", line 102, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)
File "C:/Users/Srikanth1.R/Desktop/Desktop/My_Folder/Car_damage_prediction/tensorflow-master/tensorflow-master/tensorflow/examples/label_image/label_image.py", line 117, in <module>
graph = load_graph(model_file)
File "C:/Users/Srikanth1.R/Desktop/Desktop/My_Folder/Car_damage_prediction/tensorflow-master/tensorflow-master/tensorflow/examples/label_image/label_image.py", line 31, in load_graph
graph_def.ParseFromString(f.read())
File "C:\Users\Srikanth1.R\AppData\Local\Continuum\anaconda3\lib\site-packages\google\protobuf\message.py", line 185, in ParseFromString
self.MergeFromString(serialized)
File "C:\Users\Srikanth1.R\AppData\Local\Continuum\anaconda3\lib\site-packages\google\protobuf\internal\python_message.py", line 1083, in MergeFromString
if self._InternalParse(serialized, 0, length) != length:
File "C:\Users\Srikanth1.R\AppData\Local\Continuum\anaconda3\lib\site-packages\google\protobuf\internal\python_message.py", line 1120, in InternalParse
pos = field_decoder(buffer, new_pos, end, self, field_dict)
File "C:\Users\Srikanth1.R\AppData\Local\Continuum\anaconda3\lib\site-packages\google\protobuf\internal\decoder.py", line 633, in DecodeField
if value._InternalParse(buffer, pos, new_pos) != new_pos:
File "C:\Users\Srikanth1.R\AppData\Local\Continuum\anaconda3\lib\site-packages\google\protobuf\internal\python_message.py", line 1120, in InternalParse
pos = field_decoder(buffer, new_pos, end, self, field_dict)
File "C:\Users\Srikanth1.R\AppData\Local\Continuum\anaconda3\lib\site-packages\google\protobuf\internal\decoder.py", line 612, in DecodeRepeatedField
if value.add()._InternalParse(buffer, pos, new_pos) != new_pos:
File "C:\Users\Srikanth1.R\AppData\Local\Continuum\anaconda3\lib\site-packages\google\protobuf\internal\python_message.py", line 1120, in InternalParse
pos = field_decoder(buffer, new_pos, end, self, field_dict)
File "C:\Users\Srikanth1.R\AppData\Local\Continuum\anaconda3\lib\site-packages\google\protobuf\internal\decoder.py", line 743, in DecodeMap
if submsg._InternalParse(buffer, pos, new_pos) != new_pos:
File "C:\Users\Srikanth1.R\AppData\Local\Continuum\anaconda3\lib\site-packages\google\protobuf\internal\python_message.py", line 1109, in InternalParse
new_pos = local_SkipField(buffer, new_pos, end, tag_bytes)
File "C:\Users\Srikanth1.R\AppData\Local\Continuum\anaconda3\lib\site-packages\google\protobuf\internal\decoder.py", line 850, in SkipField
return WIRETYPE_TO_SKIPPER[wire_type](buffer, pos, end)
File "C:\Users\Srikanth1.R\AppData\Local\Continuum\anaconda3\lib\site-packages\google\protobuf\internal\decoder.py", line 799, in _SkipGroup
new_pos = SkipField(buffer, pos, end, tag_bytes)
File "C:\Users\Srikanth1.R\AppData\Local\Continuum\anaconda3\lib\site-packages\google\protobuf\internal\decoder.py", line 850, in SkipField
return WIRETYPE_TO_SKIPPER[wire_type](buffer, pos, end)
File "C:\Users\Srikanth1.R\AppData\Local\Continuum\anaconda3\lib\site-packages\google\protobuf\internal\decoder.py", line 814, in _SkipFixed32
raise _DecodeError('Truncated message.')
DecodeError: Truncated message.
This is my code for prediction on test images
def load_graph(model_file):
graph = tf.Graph()
graph_def = tf.GraphDef()
with open(model_file, "rb") as f:
graph_def.ParseFromString(f.read())
with graph.as_default():
tf.import_graph_def(graph_def)
return graph
def read_tensor_from_image_file(file_name,
input_height=299,
input_width=299,
input_mean=0,
input_std=255):
input_name = "file_reader"
output_name = "normalized"
file_reader = tf.read_file(file_name, input_name)
if file_name.endswith(".png"):
image_reader = tf.image.decode_png(
file_reader, channels=3, name="png_reader")
elif file_name.endswith(".gif"):
image_reader = tf.squeeze(
tf.image.decode_gif(file_reader, name="gif_reader"))
elif file_name.endswith(".bmp"):
image_reader = tf.image.decode_bmp(file_reader, name="bmp_reader")
else:
image_reader = tf.image.decode_jpeg(
file_reader, channels=3, name="jpeg_reader")
float_caster = tf.cast(image_reader, tf.float32)
dims_expander = tf.expand_dims(float_caster, 0)
resized = tf.image.resize_bilinear(dims_expander, [input_height, input_width])
normalized = tf.divide(tf.subtract(resized, [input_mean]), [input_std])
sess = tf.Session()
result = sess.run(normalized)
return result
def load_labels(label_file):
label = []
proto_as_ascii_lines = tf.gfile.GFile(label_file).readlines()
for l in proto_as_ascii_lines:
label.append(l.rstrip())
return label
if __name__ == "__main__":
file_name = "C:\\Users\\Srikanth1.R\\Desktop\\Car Images\\car.jpg"
model_file = "C:\\Users\\Srikanth1.R\\Desktop\\Desktop\\My_Folder\\Inage analytics\\hub-master\\examples\\image_retraining\\tmp\\saved_model\\saved_model.pb"
label_file = "C:\\Users\\Srikanth1.R\\Desktop\\Desktop\\My_Folder\\Inage analytics\\hub-master\\examples\\image_retraining\\tmp\\output_labels.txt"
input_height = 299
input_width = 299
input_mean = 0
input_std = 255
input_layer = "input"
output_layer = "InceptionV3/Predictions/Reshape_1"
parser = argparse.ArgumentParser()
parser.add_argument("--image", help="image to be processed")
parser.add_argument("--graph", help="graph/model to be executed")
parser.add_argument("--labels", help="name of file containing labels")
parser.add_argument("--input_height", type=int, help="input height")
parser.add_argument("--input_width", type=int, help="input width")
parser.add_argument("--input_mean", type=int, help="input mean")
parser.add_argument("--input_std", type=int, help="input std")
parser.add_argument("--input_layer", help="name of input layer")
parser.add_argument("--output_layer", help="name of output layer")
args = parser.parse_args()
if args.graph:
model_file = args.graph
if args.image:
file_name = args.image
if args.labels:
label_file = args.labels
if args.input_height:
input_height = args.input_height
if args.input_width:
input_width = args.input_width
if args.input_mean:
input_mean = args.input_mean
if args.input_std:
input_std = args.input_std
if args.input_layer:
input_layer = args.input_layer
if args.output_layer:
output_layer = args.output_layer
graph = load_graph(model_file)
t = read_tensor_from_image_file(
file_name,
input_height=input_height,
input_width=input_width,
input_mean=input_mean,
input_std=input_std)
input_name = "import/" + input_layer
output_name = "import/" + output_layer
input_operation = graph.get_operation_by_name(input_name)
output_operation = graph.get_operation_by_name(output_name)
with tf.Session(graph=graph) as sess:
results = sess.run(output_operation.outputs[0], {
input_operation.outputs[0]: t
})
results = np.squeeze(results)
top_k = results.argsort()[-5:][::-1]
labels = load_labels(label_file)
for i in top_k:
print(labels[i], results[i])
Is the above mentioned DecodeError in some way related to the error I get while running retrain.py ?
Or are both the errors independent ?
Can anyone please tell me how to solve the above errors?
Thank you in advance ?
You can also use below mentioned approach for prediction,
with tf.Session(graph=tf.Graph()) as sess:
tf.saved_model.loader.load(
sess, [tf.saved_model.tag_constants.SERVING], <path for .pb file>)
sess.run(...)
If you still want to use,
def load_graph(model_file):
graph = tf.Graph()
graph_def = tf.GraphDef()
with open(model_file, "rb") as f:
graph_def.ParseFromString(f.read())
with graph.as_default():
tf.import_graph_def(graph_def)
return graph
make sure model_file, should be a frozen graph.
Please refer link, Frozen Graph for more details.

Error when using viterbi_decode of tensorflow

I am using this github.com/Determined22/zh-NER-TF
I just used another train_data of the same format.
Nothing is wrong with the code because it's okay when I run with the original train_data. What can cause this?
Traceback (most recent call last):
File "main.py", line 83, in <module>
model.train(train=train_data, dev=dev_data)
File "/home/mengyuguang/NER/model.py", line 161, in train
self.run_one_epoch(sess, train, dev, self.tag2label, epoch, saver)
File "/home/mengyuguang/NER/model.py", line 221, in run_one_epoch
label_list_dev, seq_len_list_dev = self.dev_one_epoch(sess, dev)
File "/home/mengyuguang/NER/model.py", line 256, in dev_one_epoch
label_list_, seq_len_list_ = self.predict_one_batch(sess, seqs)
File "/home/mengyuguang/NER/model.py", line 277, in predict_one_batch
viterbi_seq, _ = viterbi_decode(logit[:seq_len], transition_params)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/crf/python/ops/crf.py", line 333, in viterbi_decode
trellis[0] = score[0]
IndexError: index 0 is out of bounds for axis 0 with size 0
def read_corpus(self, corpus_path):
data = []
with open(corpus_path, 'r') as r_file:
sent_, tag_ = [], []
for line in r_file:
line = line.strip()
if len(line) != 0 and line != '-DOCSTART-':
ls = line.split('\t')
char, tag = ls[0], ls[-1]
sent_.append(char)
tag_.append(tag)
else:
data.append((sent_, tag_))
sent_, tag_ = [], []
# Bug-fix
# Here, since the last tuple (sent_, tag_) will be added into data
# It will case IndexError in viterbi_decode since the sequence_length is 0
if sent_ and tag_:
data.append((sent_, tag_))
self.data = data
The code should be changed to the following:
def read_corpus(corpus_path):
"""
read corpus and return the list of samples
:param corpus_path:
:return: data
"""
data = []
with open(corpus_path, encoding='utf-8') as fr:
lines = fr.readlines()
sent_, tag_ = [], []
for line in lines:
if line != '\n' and line != '\t\n': #
[char, label] = line.strip().split()
sent_.append(char)
tag_.append(label)
#else:
elif sent_ !=[] and tag_ !=[]: #
data.append((sent_, tag_))
sent_, tag_ = [], []
return data

Tensorflow serving error when connecting with client "input size does not match signature"

I have gone about exporting the textsum model using the export_textsum.py file shown below and when I connect using the textsumclient.py file below I receive the error:
Traceback (most recent call last): File "textsum_client.py", line
90, in
tf.app.run() File "/usr/local/lib/python2.7/site-packages/tensorflow/python/platform/app.py",
line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough)) File "textsum_client.py", line 83, in main
FLAGS.concurrency, FLAGS.num_tests) File "textsum_client.py", line 72, in do_singleDecode
result = stub.Predict(request, 5.0) # 5 seconds File "/usr/local/lib/python2.7/site-packages/grpc/beta/_client_adaptations.py",
line 324, in call
self._request_serializer, self._response_deserializer) File "/usr/local/lib/python2.7/site-packages/grpc/beta/_client_adaptations.py",
line 210, in _blocking_unary_unary
raise _abortion_error(rpc_error_call) grpc.framework.interfaces.face.face.AbortionError:
AbortionError(code=StatusCode.INVALID_ARGUMENT, details="input size
does not match signature")
I believe that it may have something to do with the building of tf_example in my export_textsum file but I honestly have not had luck figuring this out as of yet. Anyone with a bit more experience know what I am doing wrong here? If there are any ideas to help me narrow down exactly what is going on here I am open to any advice. Thanks.
textsumclient.py
from __future__ import print_function
import sys
import threading
# This is a placeholder for a Google-internal import.
from grpc.beta import implementations
import numpy
import tensorflow as tf
from datetime import datetime
from tensorflow_serving.apis import predict_pb2
from tensorflow_serving.apis import prediction_service_pb2
#from tensorflow_serving.example import mnist_input_data
tf.app.flags.DEFINE_integer('concurrency', 1,
'maximum number of concurrent inference requests')
tf.app.flags.DEFINE_integer('num_tests', 10, 'Number of test images')
tf.app.flags.DEFINE_string('server', '172.17.0.2:9000', 'PredictionService host:port')
tf.app.flags.DEFINE_string('work_dir', '/tmp', 'Working directory. ')
FLAGS = tf.app.flags.FLAGS
def do_singleDecode(hostport, work_dir, concurrency, num_tests):
#Connect to server
host, port = hostport.split(':')
channel = implementations.insecure_channel(host, int(port))
stub = prediction_service_pb2.beta_create_PredictionService_stub(channel)
#Prepare our request object
request = predict_pb2.PredictRequest()
request.model_spec.name = 'textsum_model'
request.model_spec.signature_name = 'predict'
#Make some test data
test_data_set = ['This is a test','This is a sample']
#Lets test her out
now = datetime.now()
article, abstract = test_data_set
#***** POPULATE REQUEST INPUTS *****
request.inputs['article'].CopyFrom(
tf.contrib.util.make_tensor_proto(test_data_set[0], shape=[len(test_data_set[0])]))
request.inputs['abstract'].CopyFrom(
tf.contrib.util.make_tensor_proto(test_data_set[1], shape=[len(test_data_set[1])]))
result = stub.Predict(request, 5.0) # 5 seconds
waiting = datetime.now() - now
return result, waiting.microseconds
def main(_):
if not FLAGS.server:
print('please specify server host:port')
return
result, waiting = do_singleDecode(FLAGS.server, FLAGS.work_dir,
FLAGS.concurrency, FLAGS.num_tests)
print('\nTextsum result: %s%%' % result)
print('Waiting time is: ', waiting, 'microseconds.')
if __name__ == '__main__':
tf.app.run()
export_textsum.py
decode_mdl_hps = hps
# Only need to restore the 1st step and reuse it since
# we keep and feed in state for each step's output.
decode_mdl_hps = hps._replace(dec_timesteps=1)
model = seq2seq_attention_model.Seq2SeqAttentionModel(
decode_mdl_hps, vocab, num_gpus=FLAGS.num_gpus)
decoder = seq2seq_attention_decode.BSDecoder(model, batcher, hps, vocab)
serialized_output = tf.placeholder(tf.string, name='tf_output')
serialized_tf_example = tf.placeholder(tf.string, name='tf_example')
feature_configs = {
'article': tf.FixedLenFeature(shape=[1], dtype=tf.string),
'abstract': tf.FixedLenFeature(shape=[1], dtype=tf.string),
}
tf_example = tf.parse_example(serialized_tf_example, feature_configs)
saver = tf.train.Saver()
config = tf.ConfigProto(allow_soft_placement = True)
with tf.Session(config = config) as sess:
# Restore variables from training checkpoints.
ckpt = tf.train.get_checkpoint_state(FLAGS.checkpoint_dir)
if ckpt and ckpt.model_checkpoint_path:
saver.restore(sess, ckpt.model_checkpoint_path)
global_step = ckpt.model_checkpoint_path.split('/')[-1].split('-')[-1]
print('Successfully loaded model from %s at step=%s.' %
(ckpt.model_checkpoint_path, global_step))
else:
print('No checkpoint file found at %s' % FLAGS.checkpoint_dir)
return
# ************** EXPORT MODEL ***************
export_path = os.path.join(FLAGS.export_dir,str(FLAGS.export_version))
print('Exporting trained model to %s' % export_path)
#-------------------------------------------
tensor_info_inputs = tf.saved_model.utils.build_tensor_info(serialized_tf_example)
tensor_info_outputs = tf.saved_model.utils.build_tensor_info(serialized_output)
prediction_signature = (
tf.saved_model.signature_def_utils.build_signature_def(
inputs={ tf.saved_model.signature_constants.PREDICT_INPUTS: tensor_info_inputs},
outputs={tf.saved_model.signature_constants.PREDICT_OUTPUTS:tensor_info_outputs},
method_name=tf.saved_model.signature_constants.PREDICT_METHOD_NAME
))
#----------------------------------
legacy_init_op = tf.group(tf.tables_initializer(), name='legacy_init_op')
builder = saved_model_builder.SavedModelBuilder(export_path)
builder.add_meta_graph_and_variables(
sess=sess,
tags=[tf.saved_model.tag_constants.SERVING],
signature_def_map={
'predict':prediction_signature,
},
legacy_init_op=legacy_init_op)
builder.save()
print('Successfully exported model to %s' % export_path)
except:
traceback.print_exc()
pass
def main(_):
Export()
if __name__ == "__main__":
tf.app.run()
It looks like you should specify a shape of [1] both in your client and graph definition.
export_textsum.py
feature_configs = {
'article': tf.FixedLenFeature(shape=[1], dtype=tf.string),
'abstract': tf.FixedLenFeature(shape=[1], dtype=tf.string),
}
textsumclient.py
request.inputs['article'].CopyFrom(
tf.contrib.util.make_tensor_proto([test_data_set[0]], shape=[1]))
request.inputs['abstract'].CopyFrom(
tf.contrib.util.make_tensor_proto([test_data_set[1]], shape=[1]))
Or perhaps using shape=[len(test_data_set[0])] would be more appropriate
QuantumLicht I again just want to thank you for your assistance here as it was one part of my issue. It seemed to have something to do with the keys used in the feature config. I am still using TF 1.2 and I remember reading sometime back that there were some fixes performed for proper key names being able to be used now in newer versions. That said, as I debugged I noticed that it was expecting a single input named "inputs". So I removed "abstract" and set article to inputs. I then had to modify the output of decode and the final issue was related to the fact that I was only loading the model but never running the function against the model to get back the output that I needed to then send into tensor_info_outputs.

No op named GatherTree when using BeamSearchDecoder

I'm implementing a Seq2Seq model with TensorFlow. My code works using the Greedy Decoder, but when I was using BeamSearchDecoder to improve the performance, I encountered this error:
Traceback (most recent call last):
File "/Users/MichaelChen/Projects/CN-isA-Relation-Extraction/isa_seq2seq/predict.py", line 83, in <module>
out_file='result/result_wc_4.out', checkpoint=checkpoint)
File "/Users/MichaelChen/Projects/CN-isA-Relation-Extraction/isa_seq2seq/predict.py", line 48, in predict
loader = tf.train.import_meta_graph(checkpoint + '.meta')
File "/usr/local/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1686, in import_meta_graph
**kwargs)
File "/usr/local/lib/python3.6/site-packages/tensorflow/python/framework/meta_graph.py", line 504, in import_scoped_meta_graph
producer_op_list=producer_op_list)
File "/usr/local/lib/python3.6/site-packages/tensorflow/python/framework/importer.py", line 283, in import_graph_def
raise ValueError('No op named %s in defined operations.' % node.op)
ValueError: No op named GatherTree in defined operations.
This error occurred when I used the infer module to generate
outputs:
with tf.Session(graph=loaded_graph) as sess:
loader = tf.train.import_meta_graph(checkpoint + '.meta')
loader.restore(sess, checkpoint)
input_data = loaded_graph.get_tensor_by_name('inputs:0')
logits = loaded_graph.get_tensor_by_name('predictions:0')
src_seq_len = loaded_graph.get_tensor_by_name('source_sequence_length:0')
tgt_seq_len = loaded_graph.get_tensor_by_name('target_sequence_length:0')
for i in range(len(text)):
if len(text[i].strip()) < 1:
continue
text_seq = src2seq_word(text[i], True)
answer_logits = sess.run(logits, {input_data: [text_seq] * batch_size,
tgt_seq_len: [len(text_seq)] * batch_size,
src_seq_len: [len(text_seq)] * batch_size}
)[0]
pred_res = "".join([pp.id2c[i] for i in answer_logits if i != pad and i != eos])
Program failed at loader = tf.train.import_meta_graph(checkpoint + '.meta')
I don't know if I handle the outputs of the decoder right, so here is the corresponding code:
# 5. Predicting decoder
# Share params with Training Deocder
tiled_dec_start_state = tf.contrib.seq2seq.tile_batch(encoder_state, beam_width)
tiled_encoder_outputs = tf.contrib.seq2seq.tile_batch(encoder_outputs, beam_width)
tiled_src_seq_len = tf.contrib.seq2seq.tile_batch(src_seq_len, beam_width)
with tf.variable_scope('decode', reuse=True):
batch_size_tensor = tf.constant(batch_size)
beam_decoder_cell = get_decoder_cell(tiled_encoder_outputs, tiled_src_seq_len, 2 * num_units)
beam_initial_state = beam_decoder_cell.zero_state(batch_size_tensor * beam_width, tf.float32)
beam_initial_state = beam_initial_state.clone(cell_state=tiled_dec_start_state)
start_tokens = tf.tile(tf.constant([c2id['<GO>']], dtype=tf.int32), [batch_size], name='start_tokens')
predicting_decoder = tf.contrib.seq2seq.BeamSearchDecoder(
cell=beam_decoder_cell,
embedding=decoder_embeddings,
start_tokens=start_tokens,
end_token=c2id['<EOS>'],
initial_state=beam_initial_state,
beam_width=beam_width,
output_layer=output_layer
)
predicting_decoder_output, _, _ = tf.contrib.seq2seq.dynamic_decode(decoder=predicting_decoder, maximum_iterations=max_tgt_seq_len)
Handling outputs:
training_decoder_output, predicting_decoder_output = seq2seq_model(params...)
training_logits = tf.identity(training_decoder_output.rnn_output, name='logits')
predicting_logits = tf.identity(predicting_decoder_output.predicted_ids[:,:,0], name='predictions')
Also, I found something in the nmt model in
https://github.com/tensorflow/nmt/blob/77e6c55052ba31a8d733c94bb820d091c8156d35/nmt/model.py (line 391)
if beam_width > 0:
logits = tf.no_op()
sample_id = outputs.predicted_ids
else:
logits = outputs.rnn_output
sample_id = outputs.sample_id
Is this has something to do with my error?
Thanks in advance!