The version I use is tensorflow-gpu version 2.1.0, installed from pip.
import tensorflow as tf
import tensorflow_hub as hub
tf.keras.backend.set_learning_phase(True)
module_url = "https://tfhub.dev/tensorflow/efficientnet/lite0/classification/2"
module2 = tf.keras.Sequential([
hub.KerasLayer(module_url, trainable=False, input_shape=(224,224,3))])
output1 = module2(tf.ones(shape=(1,224,224,3)))
print(module2.summary())
When I set trainable = True, the operation will give an error.
So, can't I retrain it on tf2.1 version?
The EfficientNet-Lite models on TFHub are based on TensorFlow 1, and thus are subject to many restrictions on TF2 including fine-tuning as you've discovered. The EfficientNet models were updated to TF2 but we're still waiting for their lite counterparts.
https://www.tensorflow.org/hub/model_compatibility
https://github.com/tensorflow/hub/issues/751
UPDATE: Beginning October 5, 2021, the EfficientNet-Lite models on TFHub are available for TensorFlow 2.
Related
My tensorflow recognizes my gpu
However, when I call model.fit() on my data it shows:
epoch(1/2) and then the kernel dies immediately
If I run this in a separate virtual environment with no GPU it works fine:
I have simplified the model architecture and number of training points to only ten as a quick test and it still fails
Simple example
from numpy import loadtxt
from keras.models import Sequential
from keras.layers import Dense
model = keras.Sequential()
model.add(Dense(4,
activation='relu'))
model.add(Dense(1, activation='sigmoid'))
opt = keras.optimizers.Adam(learning_rate=.001)
model.compile(loss = 'binary_crossentropy' , optimizer = opt, metrics = ['accuracy'] )
info = model.fit(X_train, y_train, epochs=2, batch_size=2,shuffle=True, verbose=1)
versions:
Python 3.8.8
Num GPUs Available 1
2.5.0-dev20210227
2.4.3
cuda v11.2
I am going to answer my own question rather than deleting this because maybe someone else will be making the same simple mistake I was.
The main mistake I made was having the incorrect CUDA download. you can refer to the what versions are correct at this link:
https://www.tensorflow.org/install/source#gpu
TLDR: Just follow this video:
https://www.youtube.com/watch?v=hHWkvEcDBO0
This also highlighted the importance of a virtual environment where you control the package versions to prevent incompatibilities.
I had the same problem. I transferred the code into a python file and found the root cause. In my case it was copying cudnn dll files into C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.6\bin. Check the following link as well:
Could not load dynamic library 'cudnn64_8.dll'; dlerror: cudnn64_8.dll not found
I have trained a multi_gpu_model using tensorflow 1.13/1.14 and saved them with keras.model.save('<.hdf5>').
Now, after migrating to tensorflow 2.4.1, in which Keras is integrated as tensorflow.keras, I cannot tensorflow.keras.models.load_model as I did before, due to the following error:
AttributeError: module 'tensorflow.python.keras.backend' has no attribute 'slice'
After trying to import keras.models.load_model, and trying different versions of keras (2.2.4 -> 2.4.1) and tensorflow (2.2 -> 2.4.1), I cannot load_model from my .hdf5 file using my TF 2.2+ code.
I do know that in TF 2.X + we can train using distributed machines by implementing the "strategy" scope, and it does work, but I have a lot of "old" models that I need to work on the same code base which is now being migrated to TF 2.4.1
Apparently the problem was not the TF versions, but the way I was saving my models on my TF 1.X code versions.
I used the keras.multi_gpu_model class for both training and saving, while this practice is wrong, as clearly stated on Keras documentation:
"To save the multi-gpu model, use .save(fname) or .save_weights(fname)
with the template model (the argument you passed to multi_gpu_model),
rather than the model returned by multi_gpu_model."
So, after figuring this out a method for model conversion, using TF 1.X code, was adopted:
build you model from scratch, namely new_model
load your pre-trained weights from the multi_gpu_model, namely 'old_model'
copy your old_model's weights, which is old_model.layers[3] (due to the wrong usage of multi_gpu_model) to your new_model
save new_model as .hdf5 file
use new_model.hdf5 everywhere - TF 1.X and TF 2.X
After downloading an EfficientNet model from tensorflow.keras.applications.efficientnet, and retraining it on our own data, I've noticed that the results are not reproducible. The results are reproducible for other architectures like VGG16, ResNet101, InceptionV3, and InceptionResNetV2, but not for any of the EfficientNetBx models.
Layer by layer analysis shows that the DepthwiseConv2D layer is producing different gradients. I also tried the MobileNetV2 which has the same type of layers, and it is NOT reproducible either. I wonder if anyone else has encountered this issue and how they have solved it.
Please note that I've set all the following seeds, and even have tensorflow-determinism:
random.seed(1)
np.random.seed(1)
tf.random.set_seed(1)
os.environ['TF_CUDNN_DETERMINISTIC'] = TRUE
os.environ['TF_DETERMINISTIC_OPS'] = TRUE
TensorFlow Version: tensorflow-gpu==2.3
Opened this issue on Tensorflow GitHub:
https://github.com/tensorflow/tensorflow/issues/47174
I am trying to build a NER model in Keras using ELMO Embeddings. SO I stumped across this tutorial and started implementing. I got lots of errors and some of them are as:
import tensorflow as tf
import tensorflow_hub as hub
from keras import backend as K
sess = tf.Session()
K.set_session(sess)
elmo_model = hub.Module("https://tfhub.dev/google/elmo/2", trainable=True)
sess.run(tf.global_variables_initializer())
sess.run(tf.tables_initializer())
def ElmoEmbedding(x):
return elmo_model(inputs={"tokens": tf.squeeze(tf.cast(x, tf.string)),
"sequence_len": tf.constant(batch_size*[max_len])},signature="tokens",as_dict=True)["elmo"]
input_text = Input(shape=(max_len,), dtype=tf.string)
embedding = Lambda(ElmoEmbedding, output_shape=(None, 1024))(input_text)
It gives me AttributeError: module 'tensorflow' has no attribute 'Session' . So if I comment out sess= code and run, it gives me AttributeError: module 'keras.backend' has no attribute 'set_session'.
Then again, Elmo code line is giving me RuntimeError: Exporting/importing meta graphs is not supported when eager execution is enabled. No graph exists when eager execution is enabled..
I have the following configurations:
tf.__version__
'2.3.1'
keras.__version__
'2.4.3'
import sys
sys.version
'3.8.3 (default, Jul 2 2020, 17:30:36) [MSC v.1916 64 bit (AMD64)]'
How can I use ELMO Embeddings in Keras Model?
You are using the old Tensorflow 1.x syntax but you have tensorflow 2 installed.
This is the new way to do elmo in TF2
Extracting ELMo features using tensorflow and convert them to numpy
I am trying to load the Tf hub model and predict the output using #tf.function decorator. It is throwing tensorflow.python.framework.errors_impl.FailedPreconditionError: Table not initialized. error.
TF version - 2.1.0
TF hub Version - 0.8.0
Note: It is working without using #tf.function decorator
import tensorflow as tf
import tensorflow_hub as hub
image_tensor = tf.constant(2.0, shape=[1, 298, 298, 3])
#tf.function
def run_function(method, args):
return method(args)
detector = hub.KerasLayer("https://tfhub.dev/google/openimages_v4/ssd/mobilenet_v2/1",
signature_outputs_as_dict=True)
detector_output = run_function(detector, image_tensor)
class_names = detector_output["detection_class_entities"]
print(class_names)
Can anyone know the reason why it is not working with #tf.function?
You are using a TensorFlow V1 hub model in hub.KerasLayer which is to be used for tf2.0 models
In TensorFlow hub, you can find a toggle button to view tf hub models for specific TensorFlow versions.
To make it work using hub.KeralLayer, change the URL to either of the following tf2.0 mobilenet versions
https://tfhub.dev/google/tf2-preview/mobilenet_v2/classification/4
https://tfhub.dev/google/imagenet/mobilenet_v2_050_96/classification/4
or if you have to use the exact URL as in your example. Use hub.Module instead of hub.KeralLayer