I am going to train my model quantization aware. However, when i use it , the tensorflow_model_optimization cannot quantize tf.reshape function , and throws an error.
tensorflow version : '2.4.0-dev20200903'
python version : 3.6.9
the code:
import os
os.environ['CUDA_VISIBLE_DEVICES'] = '3'
from tensorflow.keras.applications import VGG16
import tensorflow_model_optimization as tfmot
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers
quantize_model = tfmot.quantization.keras.quantize_model
inputs = keras.Input(shape=(784,))
# img_inputs = keras.Input(shape=(32, 32, 3))
dense = layers.Dense(64, activation="relu")
x = dense(inputs)
x = layers.Dense(64, activation="relu")(x)
outputs = layers.Dense(10)(x)
outputs = tf.reshape(outputs, [-1, 2, 5])
model = keras.Model(inputs=inputs, outputs=outputs, name="mnist_model")
# keras.utils.plot_model(model, "my_first_model.png")
q_aware_model = quantize_model(model)
and the output:
Traceback (most recent call last):
File "<ipython-input-39-af601b78c010>", line 14, in <module>
q_aware_model = quantize_model(model)
File "/home/essys/.local/lib/python3.6/site-packages/tensorflow_model_optimization/python/core/quantization/keras/quantize.py", line 137, in quantize_model
annotated_model = quantize_annotate_model(to_quantize)
File "/home/essys/.local/lib/python3.6/site-packages/tensorflow_model_optimization/python/core/quantization/keras/quantize.py", line 210, in quantize_annotate_model
to_annotate, input_tensors=None, clone_function=_add_quant_wrapper)
...
File "/home/essys/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/tensorflow/python/autograph/impl/api.py", line 667, in wrapper
raise e.ag_error_metadata.to_exception(e)
TypeError: in user code:
TypeError: tf__call() got an unexpected keyword argument 'shape'
If somebody know, please help ?
The reason behind is because your layer is not yet support for QAT at the moment. If you want to quantize it, you have to self writing your quantization by quantize_annotate_layer and pass it through quantize_scope and apply to your model by quantize_apply as describe in here: https://www.tensorflow.org/model_optimization/guide/quantization/training_comprehensive_guide?hl=en#quantize_custom_keras_layer
I have create a batch_norm_layer in here as an example
Tensorflow 2.x is not complete for QAT layer, pls consider using tf1.x by adding FakeQuant after operators.
Related
I wanted to apply Bert on a sequence of sentences in the following manner, but I am getting a NotImplementedError
How to reproduce :
import tensorflow as tf
from transformers import BertTokenizer, TFBertModel
inputs = tf.keras.Input(shape=(50, 64), dtype='int32')
model = TFBertModel.from_pretrained('bert-base-uncased')
outputs = tf.keras.layers.TimeDistributed(model)(inputs)
NotImplementedError Traceback (most recent call last)
<ipython-input-5-631f3cd2e8b2> in <module>
----> 1 outputs = tf.keras.layers.TimeDistributed(model)(inputs)
Whereas the code would work fine for
inputs = tf.keras.Input(shape=(10, 128, 128, 3))
conv_2d_layer = tf.keras.layers.Conv2D(64, (3, 3))
outputs = tf.keras.layers.TimeDistributed(conv_2d_layer)(inputs)
Is there anything I am missing here?
import numpy as np
from keras.applications.vgg19 import decode_predictions
from prettytable import PrettyTable
import time
from keras import backend as K
from tensorflow import keras
from tensorflow.python import keras
from keras import models, layers, Model, Input
import tensorflow as tf
model_2=keras.models.load_model('model_2.h5',compile=False)
model_2.summary()
predictions1= np.load('D:/predictions_result.npy')
def profiler(model, test_input):
data_input=test_input
for layer in model.layers:
start = time.time()
im_imput=keras.layers.Input(batch_shape=model.get_layer(layer.name).get_input_shape_at(0))
im_out = layer(im_imput)
new_model = keras.models.Model(inputs=im_imput,outputs=im_out)
data_input = new_model.predict(data_input)
end = time.time() - start
print(end)
result=1
profiler(model_2,predictions1)
tmp=np.zeros((1,224,224,64))
for i in range(0,1):
tmp[i,:,:,:]=predictions1[i,:]
predictions2 = model_2.predict(tmp)
label_vgg19 = decode_predictions(predictions2)
print ('label_vgg19 =', label_vgg19)
When I try to run the above code I get below error. My question is how to remove the first layer of the model after loading. I initially split VGG model into sub-models and then load the submodel. I tried different approaches but none is working. Help is highly appreciated.
Traceback (most recent call last):
File "C:/Users/40227422/PycharmProjects/model_partititon/model_2_sock.py", line 42, in <module>
profiler(model_2,predictions1)
File "C:/Users/40227422/PycharmProjects/model_partititon/model_2_sock.py", line 28, in the
profiler
data_input = new_model.predict(data_input)
File "C:\Users\40227422\AppData\Local\Continuum\miniconda3\envs\tensorflow\lib\site-
packages\tensorflow\python\keras\engine\training_utils.py", line 332, in standardize_input_data
' but got array with shape ' + str(data_shape))
ValueError: Error when checking input: expected input_1 to have shape (224, 224, 3) but got array
with shape (224, 224, 64)
when I try to use kerassurgeon to delete a layer using below code I get error
ValueError: not enough values to unpack (expected 2, got 0)
from kerassurgeon import Surgeon
surgeon = Surgeon(model_2)
layer_1 = model_2.layers[0] # selecting 2nd layer
surgeon.add_job('delete_layer', layer_1)
new_model = surgeon.operate()
I was not able to recreate the error you were facing. May be you can share the reproducible code. Below options I have tried to delete a layer and it worked.
from kerassurgeon.operations import delete_layer
# delete layer_1 from a model
model = delete_layer(model_2, layer_1)
OR
# delete layer_1 from a model
from kerassurgeon import Surgeon
surgeon = Surgeon(model_2)
surgeon.add_job('delete_layer', layer_1)
new_model = surgeon.operate()
I am trying to train a model with new data samples in each iteration in a loop in keras (using tensorflow backend). Due to GPU memory error after some iterations, I appended K.clear_session(). However, after one iteration, the code throws the error:
'Cannot interpret feed_dict key as Tensor: ' + e.args[0])
TypeError: Cannot interpret feed_dict key as Tensor: Tensor Tensor("Placeholder:0", shape=(7, 7, 3, 64), dtype=float32) is not an element of this graph.
If I remove K.clear_session() at end, there is no error. Is there anyone who can explain why this error comes in second iteration?
I tried other methods (for gpu release) but none of them worked and this is my last option. But it throws error. I have pasted an example code which can produce the error. Please NOTE that this is not the actual code, I just made an example to reproduce the error which I am facing in actual code.
from __future__ import absolute_import, division, print_function, unicode_literals
import numpy as np
import tensorflow as tf
import random
seed_value= 0
import os
import keras
os.environ['PYTHONHASHSEED']=str(seed_value)
random.seed(0)
np.random.seed(0)
from keras import backend as K
from keras.datasets import cifar10
(x_train, y_train), (x_test, y_test) = cifar10.load_data()
for i in range(3):
base_model = tf.keras.applications.resnet50.ResNet50(weights='imagenet', input_shape=(32, 32, 3),
include_top=False)
x = base_model.output
x = tf.keras.layers.GlobalAveragePooling2D()(x)
output = tf.keras.layers.Dense(10, activation='softmax',
kernel_initializer=tf.keras.initializers.RandomNormal(seed=4))(x)
model = tf.keras.Model(inputs=base_model.input, outputs=output)
y_train = keras.utils.to_categorical(y_train, 10)
y_test = keras.utils.to_categorical(y_test, 10)
for layer in base_model.layers:
layer.trainable = False
optimizer = tf.train.AdamOptimizer(learning_rate=0.0001)
model.compile(optimizer=optimizer, loss='categorical_crossentropy',
metrics=['accuracy'])
model.fit(x_train,y_train,batch_size=1024,epochs=1,verbose=1)
K.clear_session()
Traceback (most recent call last):
File "C:\Users\sirshad\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1092, in _run
subfeed, allow_tensor=True, allow_operation=False)
File "C:\Users\sirshad\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\ops.py", line 3490, in as_graph_element
return self._as_graph_element_locked(obj, allow_tensor, allow_operation)
File "C:\Users\sirshad\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\ops.py", line 3569, in _as_graph_element_locked
raise ValueError("Tensor %s is not an element of this graph." % obj)
ValueError: Tensor Tensor("Placeholder:0", shape=(7, 7, 3, 64), dtype=float32) is not an element of this graph.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "E:/codes/experiments-AL/breakhis/40X-M-B/codes-AL/error_debug.py", line 22, in <module>
include_top=False)
File "C:\Users\sirshad\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\keras\applications\__init__.py", line 70, in wrapper
return base_fun(*args, **kwargs)
File "C:\Users\sirshad\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\keras\applications\resnet50.py", line 32, in ResNet50
return resnet50.ResNet50(*args, **kwargs)
File "C:\Users\sirshad\AppData\Local\Programs\Python\Python36\lib\site-packages\keras_applications\resnet50.py", line 291, in ResNet50
model.load_weights(weights_path)
File "C:\Users\sirshad\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\keras\engine\network.py", line 1544, in load_weights
saving.load_weights_from_hdf5_group(f, self.layers)
File "C:\Users\sirshad\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\keras\engine\saving.py", line 806, in load_weights_from_hdf5_group
K.batch_set_value(weight_value_tuples)
File "C:\Users\sirshad\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\keras\backend.py", line 2784, in batch_set_value
get_session().run(assign_ops, feed_dict=feed_dict)
File "C:\Users\sirshad\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\client\session.py", line 929, in run
run_metadata_ptr)
File "C:\Users\sirshad\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1095, in _run
'Cannot interpret feed_dict key as Tensor: ' + e.args[0])
TypeError: Cannot interpret feed_dict key as Tensor: Tensor Tensor("Placeholder:0", shape=(7, 7, 3, 64), dtype=float32) is not an element of this graph.
Process finished with exit code 1
I was able to overcome this issue by saving the imagenet pre-trained model to disk and then loading everytime in loop after I call tf.keras.backend.clear_session(). So saving the base model to file and then loading works. But I am still confused why it did not work before with
base_model = tf.keras.applications.resnet50.ResNet50
I am trying to create an image classifier that utilizes the pre-trained ResNet V2 model provided in the slim documentation.
Here is the code so far:
import tensorflow as tf
slim = tf.contrib.slim
from PIL import Image
from inception_resnet_v2 import *
import numpy as np
checkpoint_file = 'inception_resnet_v2_2016_08_30.ckpt'
sample_images = ['carrot.jpg']
input_tensor = tf.placeholder(tf.float32, shape=(None,299,299,3), name='input_image')
scaled_input_tensor = tf.scalar_mul((1.0/255), input_tensor)
scaled_input_tensor = tf.subtract(scaled_input_tensor, 0.5)
scaled_input_tensor = tf.multiply(scaled_input_tensor, 2.0)
variables_to_restore = slim.get_model_variables()
print(variables_to_restore)
init_fn = slim.assign_from_checkpoint_fn(
checkpoint_file,
slim.get_model_variables('InceptionResnetV2'))
sess = tf.Session()
init_fn(sess)
arg_scope = inception_resnet_v2_arg_scope()
with slim.arg_scope(arg_scope):
logits, end_points = inception_resnet_v2(scaled_input_tensor, is_training=False)
for image in sample_images:
im = Image.open(image).resize((299,299))
im = np.array(im)
im = im.reshape(-1,299,299,3)
predict_values, logit_values = sess.run([end_points['Predictions'], logits], feed_dict={input_tensor: im})
print (np.max(predict_values), np.max(logit_values))
print (np.argmax(predict_values), np.argmax(logit_values))
The problem is I keep getting this error:
Traceback (most recent call last):
File "./classify.py", line 21, in <module>
slim.get_model_variables('InceptionResnetV2'))
File "/home/ubuntu/tensorflow/local/lib/python2.7/site-packages/tensorflow/contrib/framework/python/ops/variables.py", line 584, in assign_from_checkpoint_fn
saver = tf_saver.Saver(var_list, reshape=reshape_variables)
File "/home/ubuntu/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 1040, in __init__
self.build()
File "/home/ubuntu/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 1061, in build
raise ValueError("No variables to save")
ValueError: No variables to save
So it seems TF/Slim is unable to find any variables and this is made clear when I call:
variables_to_restore = slim.get_model_variables()
print(variables_to_restore)
As it outputs an empty array.
How can I go about using the pre-trained model?
This happens because you haven't constructed the model in your graph yet to have any variables starting with the name "InceptionResnetV2" to be captured and restored by the saver.
I believe you should put the model construction before using slim.get_variables_to_restore().
For instance:
with slim.arg_scope(arg_scope):
logits, end_points = inception_resnet_v2(scaled_input_tensor, is_training=False)
variables_to_restore = slim.get_model_variables()
This way, the Tensor variables will be constructed and you should see variables_to_restore is no longer empty.
You need to manually add the model variables.
Try this
with slim.arg_scope(arg_scope):
logits, end_points = inception_resnet_v2(scaled_input_tensor, is_training=False)
# Add model variables
for var in tf.global_variables(scope='inception_resnet_v2'):
slim.add_model_variable(var)
With reference to this post:
Using pre-trained inception_resnet_v2 with Tensorflow
i am trying to use the inception_resnet_v2 model to get predictions of images also. Therefore i looked at the snippet and tried to get it running, but it says "input_tensor" is not defined. Is there anything missing in the code mentioned or can anyone get me some hint to get it running / how to define the input_tensor variable?
Here is the snippet again:
import tensorflow as tf
slim = tf.contrib.slim
from PIL import Image
from inception_resnet_v2 import *
import numpy as np
checkpoint_file = 'inception_resnet_v2_2016_08_30.ckpt'
sample_images = ['dog.jpg', 'panda.jpg']
#Load the model
sess = tf.Session()
arg_scope = inception_resnet_v2_arg_scope()
with slim.arg_scope(arg_scope):
logits, end_points = inception_resnet_v2(input_tensor, is_training=False)
saver = tf.train.Saver()
saver.restore(sess, checkpoint_file)
for image in sample_images:
im = Image.open(image).resize((299,299))
im = np.array(im)
im = im.reshape(-1,299,299,3)
predict_values, logit_values = sess.run([end_points['Predictions'],logits], feed_dict={input_tensor: im})
print (np.max(predict_values), np.max(logit_values))
print (np.argmax(predict_values), np.argmax(logit_values))
Thanks
The code snippet appears to lack any definition for input_tensor. Looking at the definition of the inception_resnet_v2() function, the fact that the tensor is used in a feed_dict, and the fact that the size of your image is 299 x 299, you could define input_tensor as follows:
input_tensor = tf.placeholder(tf.float32, [None, 299, 299, 3])