I recently built a tensorflow model and trained it; Here is the input.
data = tf.placeholder(tf.float32, [None,20,1]) #Number of examples, number of input, dimension of each input
After a while of computation through LSTM cells, and dense layers, the final prediction is made here:
logits = tf.layers.dense(inputs=den3, units=1)
After training it and saving it, like so:
for i in range(epoch):
for j in range(batchSize):
loss = sess.run([step,error],feed_dict={data:np.array([tx[j]]),target:np.array([ty[j]])})
saver.save(sess,'model')
This in turn saves 4 files which are
here
I then copy then over to the Pi, but i don't know how to just open them and pass some inputs and read the output, without building the whole graph and assigning each variable.
You should just be able to copy the files over to your Pi and then run saver.restore() to load them again. Here's an example of doing that:
from tensorflow.python.training import saver as saver_lib
saver = saver_lib.Saver()
saver.restore(sess, 'model')
This is based on https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/tools/freeze_graph.py
Related
I had lost my dataset by a careless mistake. I have only my tflite file left in my hand. Is there any solution to reverse back h5 file. I have been done decent research in this but no solutions found.
The conversion from a TensorFlow SaveModel or tf.keras H5 model to .tflite is an irreversible process. Specifically, the original model topology is optimized during the compilation by the TFLite converter, which leads to some loss of information. Also, the original tf.keras model's loss and optimizer configurations are discarded, because those aren't required for inference.
However, the .tflite file still contains some information that can help you restore the original trained model. Most importantly, the weight values are available, although they might be quantized, which could lead to some loss in precision.
The code example below shows you how to read weight values from a .tflite file after it's created from a simple trained tf.keras.Model.
import numpy as np
import tensorflow as tf
# First, create and train a dummy model for demonstration purposes.
model = tf.keras.Sequential([
tf.keras.layers.Dense(10, input_shape=[5], activation="relu"),
tf.keras.layers.Dense(1, activation="sigmoid")])
model.compile(loss="binary_crossentropy", optimizer="sgd")
xs = np.ones([8, 5])
ys = np.zeros([8, 1])
model.fit(xs, ys, epochs=1)
# Convert it to a TFLite model file.
converter = tf.lite.TFLiteConverter.from_keras_model(model)
tflite_model = converter.convert()
open("converted.tflite", "wb").write(tflite_model)
# Use `tf.lite.Interpreter` to load the written .tflite back from the file system.
interpreter = tf.lite.Interpreter(model_path="converted.tflite")
all_tensor_details = interpreter.get_tensor_details()
interpreter.allocate_tensors()
for tensor_item in all_tensor_details:
print("Weight %s:" % tensor_item["name"])
print(interpreter.tensor(tensor_item["index"])())
These weight values loaded back from the .tflite file can be used with tf.keras.Model.set_weights() method, which will allow you to re-inject the weight values into a new instance of trainable Model that you have in Python. Obviously, this requires you to still have access to the code that defines the model's architecture.
Everyone!
I have a question releate in trained model reusing( tensorflow ).
I have train model
I want predict new data used trained model.
I use DNNClassifier.
I have a model.ckpt-200000.meta, model.ckpt-200000.index, checkpoint, and eval folder.
but I don't know reuse this model..
plz help me.
First, you need to import your graph,
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
new_saver = tf.train.import_meta_graph('model.ckpt-200000.meta')
new_saver.restore(sess, tf.train.latest_checkpoint('./'))
Then you can give input to the graph and get the output.
graph = tf.get_default_graph()
input = graph.get_tensor_by_name("input:0")#input tensor by name
feed_dict ={input:} #input to the model
#Now, access theoutput operation.
op_to_restore = graph.get_tensor_by_name("y_:0") #output tensor
print sess.run(op_to_restore,feed_dict) #get output here
Few things to note,
You can replace the above code with your training part of the graph
(i.e you can get the output without training).
However, you still have to construct your graph as previously and
only replace the training part.
Above method only loading the weights for the constructed graph. Therefore, you have to construct the graph first.
A good tutorial on this can be found here, http://cv-tricks.com/tensorflow-tutorial/save-restore-tensorflow-models-quick-complete-tutorial/
If you don't want to construct the graph again you can follow this tutorial, https://blog.metaflow.fr/tensorflow-how-to-freeze-a-model-and-serve-it-with-a-python-api-d4f3596b3adc
In text processing there is embedding to show up (if I understood it correctly) the database words as vector (after dimension reduction).
now, I am wondering, is there any method like this to show extracted features via CNN?
for example: consider we have a CNN and train and test sets. we want to train the CNN with train set and meanwhile see the extracted features (from dense layer) corresponding class labels via CNN in the embedding section of tensorboard.
the purpose of this work is seeing the features of input data in every batch and understand how close or far are they from together. and finally, in the trained model, we can find out accuracy of our classifier (like softmax or etc.).
thank you in advance for your help.
I have taken help of Tensorflow documentation.
For in depth information on how to run TensorBoard and make sure you are logging all the necessary information, see TensorBoard: Visualizing Learning.
To visualize your embeddings, there are 3 things you need to do:
1) Setup a 2D tensor that holds your embedding(s).
embedding_var = tf.get_variable(....)
2) Periodically save your model variables in a checkpoint in LOG_DIR.
saver = tf.train.Saver()
saver.save(session, os.path.join(LOG_DIR, "model.ckpt"), step)
3) (Optional) Associate metadata with your embedding.
If you have any metadata (labels, images) associated with your embedding, you can tell TensorBoard about it either by directly storing a projector_config.pbtxt in the LOG_DIR, or use our python API.
For instance, the following projector_config.ptxt associates the word_embedding tensor with metadata stored in $LOG_DIR/metadata.tsv:
embeddings {
tensor_name: 'word_embedding'
metadata_path: '$LOG_DIR/metadata.tsv'
}
The same config can be produced programmatically using the following code snippet:
from tensorflow.contrib.tensorboard.plugins import projector
# Create randomly initialized embedding weights which will be trained.
vocabulary_size = 10000
embedding_size = 200
embedding_var = tf.get_variable('word_embedding', [vocabulary_size,
embedding_size])
# Format: tensorflow/tensorboard/plugins/projector/projector_config.proto
config = projector.ProjectorConfig()
# You can add multiple embeddings. Here we add only one.
embedding = config.embeddings.add()
embedding.tensor_name = embedding_var.name
# Link this tensor to its metadata file (e.g. labels).
embedding.metadata_path = os.path.join(LOG_DIR, 'metadata.tsv')
#Use the same LOG_DIR where you stored your checkpoint.
summary_writer = tf.summary.FileWriter(LOG_DIR)
# The next line writes a projector_config.pbtxt in the LOG_DIR. TensorBoard will
# read this file during startup.
projector.visualize_embeddings(summary_writer, config)
In one of my project of computer vision, I use public pre-trained inception-v3 available here: http://download.tensorflow.org/models/image/imagenet/inception-2015-12-05.tgz. This network is at the beginning of my classification chain (a lot of stuff is performed on logits produced by the network). I would like to feed this network with a batch of images (instead of sequentially processing images) in order to make it faster.
However, the provided network had been "frozen", and it can only process one image at a time.
Is there any solution to "unfreeze" a graph and adapt it so that I can use it on batch of images?
(N.B : I found related topics on the internet, but they all suggest to take a more recent network available for instance here :
http://download.tensorflow.org/models/image/imagenet/inception-v3-2016-03-01.tar.gz. This is not what I would like to do since a lot of stuff has been tuned on the output of the frozen model.)
Not sure if this is too late, but here is the code snippet that I used:
# First loan model into and old graph
proto_file = ... # downloaded inception protofile
graph_def = tf.GraphDef.FromString(open(proto_file, 'rb').read())
to_delete = {“DecodeJpeg", "Cast", "ExpandDims", "pool_3/_reshape", "softmax"}
graph_def = delete_ops_from_graph(graph_def, to_delete)
new_graph = tf.Graph()
with new_graph.as_default():
x = tf.placeholder(tf.uint8, [None, None, None, 3], name="batched_inputs")
x_cast = tf.cast(x, dtype=tf.float32)
y = tf.import_graph_def(graph_def, input_map={"ExpandDims:0": x_cast}, return_elements=["pool_3:0"],name="")
...
Now new_graph is the graph that has batch dimension (takes in 4d tensor NHWC). Note that this is good if you want to use inception-2015-12-05.tgz as a feature extractor. You would need to take the output from output = new_graph.get_tensor_by_name("pool_3:0")
For the definition of delete_ops_from_graph, see Tensorflow: delete nodes from graph
I'd like to pass the parameters of the trained model (weights and bias for convolution and fully connected layers) to other frameworks or languages including iOS and Torch by parsing the saved file.
I tried tf.train.write_graph(session.graph_def, '', 'graph.pb'), but it seems it only includes the graph architecture without weights and bias. If so, to create checkpoint file (saver.save(session, "model.ckpt")) is the best way? Is it easy to parse ckpt file type in Swift or other languages?
Please let me know if you have any suggestions.
Instead of parsing a .ckpt file, you can just try evaluating the tensor (in your case the weights of a convolutional layer) and getting the values as a numpy array. Here is a quick toy example (tested on r0.10 - there might some small API changes in newer versions):
import tensorflow as tf
import numpy as np
x = tf.placeholder(np.float32, [2,1])
w = tf.Variable(tf.truncated_normal([2,2], stddev=0.1))
b = tf.Variable(tf.constant(1.0, shape=[2,1]))
z = tf.matmul(w, x) + b
with tf.Session() as sess:
sess.run(tf.initialize_all_variables())
w_val, z_val = sess.run([w, z], feed_dict={x: np.arange(2).reshape(2,1)})
print(w_val)
print(z_val)
Output:
[[-0.02913031 0.13549708]
[ 0.13807134 0.03763327]]
[[ 1.13549709]
[ 1.0376333 ]]
If you have trouble getting a reference to your tensor (say it is in nested into a higher-level "layer" operation), try finding by name. More info here: Tensorflow: How to get a tensor by name?
If you want to see the how the weights change during training, you can also try to save all the values you are interested into tf.Summary objects and parse them later: Parsing `summary_str` byte string evaluated on tensorflow summary object