I try to describe the situations completely. But due to my ability of language, there will be possible some unclear statements. Please let me know. I will try to explain my meaning.
Recently, I want to apply facenet (I mean davisking's project on github) to my project. Therefore, I wrote a class
class FacenetEmbedding:
def __init__(self, model_path):
self.sess = tf.InteractiveSession()
self.sess.run(tf.global_variables_initializer())
# Load the model
facenet.load_model(model_path)
# Get input and output tensors
self.images_placeholder = tf.get_default_graph().get_tensor_by_name("input:0")
self.tf_embeddings = tf.get_default_graph().get_tensor_by_name("embeddings:0")
self.phase_train_placeholder = tf.get_default_graph().get_tensor_by_name("phase_train:0")
def get_embedding(self, images):
feed_dict = {self.images_placeholder: images, self.phase_train_placeholder: False}
embedding = self.sess.run(self.tf_embeddings, feed_dict=feed_dict)
return embedding
def free(self):
self.sess.close()
I can use this class independent in flask.
model_path = "models/20191025-223514/"
fe = FacenetEmbedding(model_path)
But I have different demands later. I train two models by using keras. I want to use them (.h5 model) with the above facenet model to predict. I load them first.
modelPic = load_model('models/pp.h5')
lePic = pickle.loads(open('models/pp.pickle', "rb").read())
print(modelPic.predict(np.zeros((1, 128, 128, 3))))
modelM = load_model('models/pv.h5')
leM = pickle.loads(open('models/pv.pickle', "rb").read())
print(modelM.predict(np.zeros((1, 128, 128, 3))))
I print the fake image to test the models. It seems to work normally. But when I run flask server and try to post an image to this api, the message pop up and the prediction doesn't work.
Tensor input_1_3:0, specified in either feed_devices or fetch_devices was not found in the Graph
Exception ignored in: <bound method BaseSession._Callable.__del__ of <tensorflow.python.client.session.BaseSession._Callable object at 0x7ff27d0f0dd8>>
Traceback (most recent call last):
File "/home/idgate/.virtualenvs/Line_POC/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1455, in __del__
self._session._session, self._handle, status)
File "/home/idgate/.virtualenvs/Line_POC/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py", line 528, in __exit__
c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: No such callable handle: 140675571821088
I try to use these two keras model without loading facenet model in flask server. It works normally. I think that it must collide with something (maybe about session?) to make these three models cannot work simultaneously. But I don't know how to solve this problem. Please help me! Thanks in advance.
Related
So I’m working on a perceptual experiment for which I have designed a moderately complex stimulus which requires a good amount of computation to create. I had it implemented in Python; it works fine, but it’s slow to generate.
Quick brainstorm: hey, try implementing the stimulus computation in Tensorflow! A little bit of work paid off, cutting stimulus generation time in half (actually fast enough for real-time display), and the tensorflow model proves a more compact design. Great!
But then I started dreaming. I’ve wanted to be able to experiment via an app, but didn’t want to redo all my code in Swift or Java or whatever the language du jour for mobile/web is. But if all the heavy lifting code is embedded in a Tensorflow lite model, then a small wrapper for iOS/Android/(javascript maybe) would be doable in a reasonable time frame.
Here’s where my question comes in. In the off-line stimulus generation model, I configure my parameters and let it generate a video file, which can then be viewed by my subjects. If my theoretical app just takes a tensorflow model instead of a video file, then I’m really just shortening download time. What I’d really like to be able to do is to adjust the stimulus parameters within the app, instead of guessing, generating, and uploading again.
So (and from here on I’m just winging it as far as my TF skills) I turned my configuration parameters into tf.Variables, stuck them into the model, and voila, I can now adjust my stimulus on the fly, from within the Python CLI. Great! Now I just save the model…
Oops.
How does saving tf.Variables work? Here's a simple subset of my code to demonstrate the problem. Start with a layer that computes the sine of a temporal input, with a configurable phase, frequency, and amplitude:
class Sine(tf.keras.layers.Layer):
def __init__(self, *args, **kwargs):
super(Sine, self).__init__(*args, **kwargs)
self._twopi = tf.constant(np.pi * 2.0)
def call(self, parameters):
time = parameters[0]
scale = parameters[1]
frequency = parameters[2]
base = parameters[3]
phase = parameters[4]
# def call(self, time, scale, frequency, base, phase):
time = tf.cast(time, tf.float32)
return scale*tf.sin(self._twopi * frequency * time + phase) + base
Note here that I tried compressing the five arguments into one list to see what that would do.
Here's a stupid model that uses it:
class StupidModel:
def __init__(self, frequency, amplitude, base, phase):
self._frequency = tf.Variable(frequency, name="frequency", dtype=tf.float32)
self._amplitude = tf.Variable(amplitude, name="amplitude", dtype=tf.float32)
self._base = tf.Variable(base, name="base", dtype=tf.float32)
self._phase = tf.Variable(phase, name="phase", dtype=tf.float32)
self._model = self._build_model()
def _build_model(self):
input = tf.keras.layers.Input(1)
out = Sine()([input, self._frequency, self._amplitude, self._base, self._phase])
model = tf.keras.Model(inputs=[input], outputs = out)
model._myfrequency = self._frequency
model._myamplitude = self._amplitude
model._mybase = self._base
model._myphase = self._phase
return model
def __call__(self, time):
return self._model.predict(time)
And here's what happens when I try to save it:
>>> sm._model.save("foo.tf")
WARNING:tensorflow:Compiled the loaded model, but the compiled metrics have yet to be built. `model.compile_metrics` will be empty until you train or evaluate the model.
INFO:tensorflow:Assets written to: foo.tf/assets
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/j/.miniforge3/lib/python3.9/site-packages/keras/utils/traceback_utils.py", line 70, in error_handler
raise e.with_traceback(filtered_tb) from None
File "/Users/j/.miniforge3/lib/python3.9/json/encoder.py", line 199, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/Users/j/.miniforge3/lib/python3.9/json/encoder.py", line 257, in iterencode
return _iterencode(o, 0)
TypeError: Unable to serialize <tf.Variable 'frequency:0' shape=() dtype=float32, numpy=0.5> to JSON. Unrecognized type <class 'tensorflow.python.ops.resource_variable_ops.ResourceVariable'>.
I believe that were I using TF1.X, I'd be looking at placeholders and feed dicts. But what can I do now? I know what I'm doing is probably outside of the normal tensorflow usage, e.g. no training, in-flight tuning, etc., but it works so well right up until saving...
Further exploration has yielded fruit. It seems complicated, and I shudder to think of what I'll need to do to convert to Tensorflow Lite, but as of now, I can:
Add arbitrary configuration parameters
Save the model
Load the model
Retrieve configuration parameters names
Set configuration parameters
The key I found was to make my StupidModel inherit from tf.Module. I added an intervening class to encapsulate the parameter logic:
class Parameterized(tf.Module):
def __init__(self, *args, **kwargs):
super(Parameterized, self).__init__()
def _save(self, name, variable):
self.__setattr__(name, variable)
def _initialize(self):
for key in self.variables().keys():
self.set(key, self.get(key))
def save(self, name):
tf.saved_model.save(self, name)
#tf.function
def variables(self):
return {k:self._trackable_children()[k] for k in self._trackable_children() if issubclass(type(self._trackable_children()[k]), tf.Variable)}
#tf.function
def set(self, variable, value):
self.__getattribute__(variable).assign(value)
#tf.function
def get(self, variable):
return self.__getattribute__(variable)
and then I updated the StupidModel to inherit from Parameterized, making sure to add the appropriate #tf.function decorator:
class StupidModel(Parameterized):
def __init__(self, frequency, amplitude, base, phase):
super().__init__(name="stupid")
self._save("frequency", tf.Variable(frequency, name="frequency", dtype=tf.float32))
self._save("amplitude", tf.Variable(amplitude, name="amplitude", dtype=tf.float32))
self._save("base", tf.Variable(base, name="base", dtype=tf.float32))
self._save("phase", tf.Variable(phase, name="phase", dtype=tf.float32))
self._sine = Sine()
self._initialize()
def _make_key(self, var):
return var.name.split(":")[0]
#tf.function(input_signature=(tf.TensorSpec(shape=[None], dtype=tf.float32),))
def __call__(self, time):
return self._sine(time, self.frequency, self.amplitude, self.base, self.phase)
Now, when I reload:
>>> import tensorflow as tf
>>> sm = tf.saved_model.load("foo.tf")
[...]
>>> sm([1, 2, 3])
<tf.Tensor: shape=(3,), dtype=float32, numpy=array([0.2602872 , 0.73971283, 0.26028723], dtype=float32)>
And I can check out my parameters (by name), set them, and see the change:
>>> sm.variables().keys()
dict_keys(['phase', 'amplitude', 'base', 'frequency'])
>>> sm.set('base', 5.0)
>>> sm([1, 2, 3])
<tf.Tensor: shape=(3,), dtype=float32, numpy=array([4.7602873, 5.2397127, 4.7602873], dtype=float32)>
And one final important note: the tf.function argument provided allows me to use arbitrary inputs. (Without it, I can only call __call__ with an argument signature that exactly matches calls made before the save, e.g. if I called sm([1, 2, 3]) and then saved, I'd get an error back if I called sm([10, 20, 30]) after reloading. Specifying the signature in #tf.function prevents that.):
>>> sm([10, 20, 30])
<tf.Tensor: shape=(3,), dtype=float32, numpy=array([5.239713 , 5.239714 , 5.2397127], dtype=float32)>
So that's what I've learned. It seems complicated, and I'm definitely open to easier approaches.
When I create a Keras model with one or more custom layers, I can use the model.save() method to persist the Keras model using the TensorFlow SavedModel format.
I can load this model from the filesystem using tf.keras.models.load_model() function and save it to the filesystem again.
But when I load the SavedModel from the filesystem a second time, it fails with this exception:
TypeError: f(inputs, training, training, training, training, *, training, training) missing 1 required argument: training
You can try replicating this issue with the following code:
import tensorflow as tf
class CustomLayer(tf.keras.layers.Layer):
def call(self, inputs, *args, **kwargs):
return inputs
model1 = tf.keras.Sequential([
CustomLayer()
])
model1.build((None, 1))
model1.compile()
model1.save("model1")
model2 = tf.keras.models.load_model("model1")
model2.save("model2")
# This line should raise a TypeError.
model3 = tf.keras.models.load_model("model2")
Why the problem exists
The problem is that the TensorFlow SavedModel format does not actually serialize custom Python code. It only saves the TensorFlow graph generated by custom Keras layers and other Python objects.
The tf.keras.models.load_model() function--by default--does not return the Python layer. Instead, it returns a placeholder layer containing the same part of the TensorFlow computation graph. We can see this in the example in my question:
>>> model1.layers
[<__main__.CustomLayer at 0x7ff04c14ee20>]
>>> model2.layers
[<keras.saving.saved_model.load.CustomLayer at 0x7ff114fd7be0>]
When model2 is saved and loaded from the filesystem, TensorFlow cannot correctly parse the *args and **kwargs arguments in CustomLayer.call().
I don't know whether the actual bug is within the saving code, the loading code, or both.
The real fix needs to happen within TensorFlow/Keras, but in the meantime, there are
Workarounds
You can choose any ONE of the below workarounds to avoid serialization errors with custom Keras layers.
Change the signature on Layer.call()
Currently, the official method signature on Layer.call() is def call(self, inputs, *args, **kwargs):
But TensorFlow will throw a TypeError when trying to load a model with a custom layer with this signature. To fix the error, write all of your custom layers with a signature of def call(self, inputs):. If your layer behaves differently during training or inference, then you can use the method signature def call(self, inputs, training=None):
This makes it easier for TensorFlow to generate placeholder layers generated in the keras.saving.saved_model.load module. But this placeholder layer is still not exactly the same as the original Python code.
Use the custom_objects parameter on tf.keras.models.load_model()
It is possible to load a model with its original Python layers instead of the placeholder layers. Just pass a dictionary mapping layer names to Python layer class objects. This requires your code to be able to import the original Python layer. The example in my question can be fixed as follows:
model3 = tf.keras.models.load_model(
"model2",
custom_objects=dict(
CustomLayer=CustomLayer,
),
)
Make sure that your layer implements Layer.get_config() and returns a dictionary with all of the parameters needed to recreate the layer from scratch. The layer must be able to be recreated with Layer.from_config().
Import the Python layer and add it to Keras's global registry
Keras maintains a global registry of custom Python classes and other objects to refer to when loading SavedModels. You can register your custom Keras layer with the #tf.keras.utils.register_keras_serializable() decorator. For example:
#tf.keras.utils.register_keras_serializable(
package="my_python_package"
)
class CustomLayer(tf.keras.layers.Layer):
def call(self, inputs, *args, **kwargs):
return inputs
This method also requires that your layer properly implement Layer.get_config().
Install the Python layer object with tf.keras.utils.custom_object_scope()
Much like the above two solutions, the tf.keras.utils.custom_object_scope() context manager can specify which custom layers to use when deserialization.
I am running into an error after upgrading to TFF 0.17.0. The same code works perfectly in TFF 0.16.1. The training works just fine in both versions however when I try to copy weights from the FL state to model to evaluate it on test dataset, I get the following error:
File "fl/main_fl.py", line 166, in keras_evaluate
loss, accuracy = self.model.evaluate(test_dataset, verbose=0)
File "/home/ubuntu/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow/python/keras/engine/training_v1.py", line 905, in evaluate
self._assert_built_as_v1()
File "/home/ubuntu/anaconda3/envs/tensorflow2_p36/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer_v1.py", line 852, i$ _assert_built_as_v1
(type(self),))
ValueError: Your Layer or Model is in an invalid state. This can happen for the following cases:
1. You might be interleaving estimator/non-estimator models or interleaving models/layers made in tf.compat.v1.Graph.as_default() with model$/layers created outside of it. Converting a model to an estimator (via model_to_estimator) invalidates all models/layers made before the conv$rsion (even if they were not the model converted to an estimator). Similarly, making a layer or a model inside a a tf.compat.v1.Graph invalid$tes all layers/models you previously made outside of the graph.
2. You might be using a custom keras layer implementation with custom __init__ which didn't call super().__init__. Please check the impleme$tation of <class 'tensorflow.python.keras.engine.functional.Functional'> and its bases.
Below is my keras_evaluate method:
def keras_evaluate(self, test_dataset, mode='test', step=0):
self.state.model.assign_weights_to(self.model)
loss, accuracy = self.model.evaluate(test_dataset, verbose=0)
print('Mode={}, Loss={}, Accuracy={}'.format(mode, loss, accuracy))
self.state is the state returned by tff.learning.build_federated_averaging_process i.e tff.templates.IterativeProcess, test_dataset is of type tf.data.Dataset and self.model is tf.keras.Model type i.e keras functional model. I have one custom layer however it does have super() method so point 2 in the error is misleading me.
Any help will be appreciated.
The code for this problem is quite complex because I'm trying to implement fractalNet but changing the convolution base block to just a dense layer. I'm trying to separately build two fractalNets (one after the other so I don't think they should be interfering). One for the policy and one for the value function.
There are also a number of issues I have seen so far that may or may not be related. One is that I can't import numpy as np and use np which is why I've been forced to use numpy(). The other is that my code seems to trying to be working on tensors tf.Tensor[stuff] as well as Tensor[stuff] in different sections at the same time. The build_model function below outputs Tensor[stuff] from the Input call whereas the neural network builder code uses tf.Tensor[stuff]. I tried but to no avail to stick to type.
Here is the complete error that keeps killing the code:
/home/ryan/.local/lib/python3.6/site-packages/keras/engine/network.py:190: UserWarning: Model inputs must come from `keras.layers.Input` (thus holding past layer metadata), they cannot be the output of a previous non-Input layer. Here, a tensor specified as input to your model was not an Input tensor, it was generated by layer activation_1.
Note that input tensors are instantiated via `tensor = keras.layers.Input(shape)`.
The tensor that caused the issue was: activation_1/Relu:0
str(x.name))
Traceback (most recent call last):
File "train.py", line 355, in <module>
main(**vars(args))
File "train.py", line 302, in main
val_func = NNValueFunction(bl,c,layersizes,dropout,deepest,obs_dim) # Initialize the value function
File "/home/ryan/trpo_fractalNN/trpo/value.py", line 37, in __init__
self.model = self._build_model()
File "/home/ryan/trpo_fractalNN/trpo/value.py", line 56, in _build_model
model = Model(inputs=obs_input, outputs=outputs)
File "/home/ryan/.local/lib/python3.6/site-packages/keras/legacy/interfaces.py", line 91, in wrapper
return func(*args, **kwargs)
File "/home/ryan/.local/lib/python3.6/site-packages/keras/engine/network.py", line 94, in __init__
self._init_graph_network(*args, **kwargs)
File "/home/ryan/.local/lib/python3.6/site-packages/keras/engine/network.py", line 241, in _init_graph_network
self.inputs, self.outputs)
File "/home/ryan/.local/lib/python3.6/site-packages/keras/engine/network.py", line 1511, in _map_graph_network
str(layers_with_complete_input))
ValueError: Graph disconnected: cannot obtain value for tensor Tensor("input_1:0", shape=(None, 29), dtype=float32) at layer "input_1". The following previous layers were accessed without issue: []
So here is the part of the code that I'm suspicious of at the moment because of the fact that somehow it is breaking at the very beginning on the value function's neural net.
def _build_model(self):
""" Construct TensorFlow graph, including loss function, init op and train op """
# hid1 layer size is 10x obs_dim, hid3 size is 10, and hid2 is geometric mean
# hid3_units = 5 # 5 chosen empirically on 'Hopper-v1'
# hid2_units = int(np.sqrt(hid1_units * hid3_units))
# heuristic to set learning rate based on NN size (tuned on 'Hopper-v1')
obs = keras.layers.Input(shape=(self.obs_dim,))
# I'm not sure why it won't work with np??????????????????????????????????????????????????????????????????????????????????
obs_input = Dense(int(self.layersizes[0][0].numpy()))(obs) # Initial fully-connected layer that brings obs number up to a len that will work with fractal architecture
obs_input = Activation('relu')(obs_input)
self.lr = 1e-2 / np.sqrt(self.layersizes[2][0]) # 1e-2 empirically determined
print('Value Params -- lr: {:.3g}'
.format(self.lr))
outputs = fractal_net(self,bl=self.bl,c=self.c,layersizes=self.layersizes,
drop_path=0.15,dropout=self.dropout,
deepest=self.deepest)(obs_input)
model = Model(inputs=obs_input, outputs=outputs)
optimizer = Adam(self.lr)
model.compile(optimizer=optimizer, loss='mse')
return model
I found out the issue. The problem was that since I was trying to combine multiple files, I had a 'Dense' call to bring the obs_len to the desired size and then took that and plugged it into the fractalNet code. However, I didn't realize that this would break things. I solved the issue by removing the initial Dense call and placing it inside the fractalNet code itself.
So moral of the story, don't try to break up different parts of the NN layers into separate files. Just as a side comment, In the current fractalNN code, it calls fractal_net and then a Dense layer afterwards and apparently this still works. But I think it breaks things to try to reverse this order. I hope this helps someone else.
I'm trying to run Albert Tensorflow hub version on multiple GPUs in the same machine. The model works perfectly on single GPU.
This is the structure of my code:
strategy = tf.distribute.MirroredStrategy()
print('Number of devices: {}'.format(strategy.num_replicas_in_sync)) # it prints 2 .. correct
if __name__ == "__main__":
with strategy.scope():
run()
Where in run() function, I read the data, build the model, and fit it.
I'm getting this error:
Traceback (most recent call last):
File "Albert.py", line 130, in <module>
run()
File "Albert.py", line 88, in run
model = build_model(bert_max_seq_length)
File "Albert.py", line 55, in build_model
model.compile(loss="categorical_crossentropy", optimizer=optimizer, metrics=["accuracy"])
File "/home/****/py_transformers/lib/python3.5/site-packages/tensorflow_core/python/training/tracking/base.py", line 457, in _method_wrapper
result = method(self, *args, **kwargs)
File "/home/bighanem/py_transformers/lib/python3.5/site-packages/tensorflow_core/python/keras/engine/training.py", line 471, in compile
' model.compile(...)'% (v, strategy))
ValueError: Variable (<tf.Variable 'bert/embeddings/word_embeddings:0' shape=(30000, 128) dtype=float32>) was not created in the distribution strategy scope of (<tensorflow.python.distribute.mirrored_strategy.MirroredStrategy object at 0x7f62e399df60>). It is most likely due to not all layers or the model or optimizer being created outside the distribution strategy scope. Try to make sure your code looks similar to the following.
with strategy.scope():
model=_create_model()
model.compile(...)
Is it possible that this error occures because Albert model was prepared before by tensorflow team (built and compiled)?
Edited:
To be precise, Tensorflow version is 2.1.
Also, this is the way I load Albert pretrained model:
features = {"input_ids": in_id, "input_mask": in_mask, "segment_ids": in_segment, }
albert = hub.KerasLayer(
"https://tfhub.dev/google/albert_xxlarge/3",
trainable=False, signature="tokens", output_key="pooled_output",
)
x = albert(features)
Following this tutorial: SavedModels from TF Hub in TensorFlow 2
Two-part answer:
1) TF Hub hosts two versions of ALBERT (each in several sizes):
https://tfhub.dev/google/albert_base/3 etc. from the Google research team that originally developed ALBERT comes in the hub.Module format for TF1. This will likely not work with a TF2 distribution strategy.
https://tfhub.dev/tensorflow/albert_en_base/1 etc. from the TensorFlow Model Garden comes in the revised TF2 SavedModel format. Please try this one for use in TF2 with a distribution strategy.
2) That said, the immediate problem appears to be what is explained in the error message (abridged):
Variable 'bert/embeddings/word_embeddings' was not created in the distribution strategy scope ... Try to make sure your code looks similar to the following.
with strategy.scope():
model = _create_model()
model.compile(...)
For a SavedModel (from TF Hub or otherwise), it's the loading that needs to happen under the distribution strategy scope, because that's what's re-creating the tf.Variable objects in the current program. Specifically, any of the following ways to load a TF2 SavedModel from TF Hub have to occur under the distribution strategy scope for distribution to work:
tf.saved_model.load();
hub.load(), which just calls tf.saved_model.load() (after downloading if necessary);
hub.KerasLayer when used with a string-valued model handle, on which it then calls hub.load().