I'm trying to use a model from tensorflow hub on Kaggle.
Like so:
m = tf.keras.Sequential([
hub.KerasLayer("https://tfhub.dev/google/tf2-preview/mobilenet_v2/feature_vector/4", output_shape=[1280],
trainable=False), # Can be True, see below.
tf.keras.layers.Dense(num_classes, activation='softmax')
])
m.build([None, 224, 224, 3]) # Batch input shape.
It works well with GPU, but as soon as I switch to TPU with TF records I get the following error:
InvalidArgumentError: Unsuccessful TensorSliceReader constructor: Failed to get matching files on /tmp/tfhub_modules/87fb99f72aec02d017e12c0a3d86c5c182ec22ca/variables/variables: Unimplemented: File system scheme '[local]' not implemented (file: '/tmp/tfhub_modules/87fb99f72aec02d017e12c0a3d86c5c182ec22ca/variables/variables')
However the set up and tfrecords dataset are all correct as it works with a switching the pretrained model to a keras application of the same model (i.e. for example above using the mobilenet keras application).
I tried caching but I have been unsuccessful, is there something I have to beware when following this guide:
https://www.tensorflow.org/hub/caching
Thanks in advance!
The failure happens because the TPU is trying to load the TFHub model from /tmp/ which it doesn't have access to. You should be able to get this to work by:
with strategy.scope():
load_locally = tf.saved_model.LoadOptions(experimental_io_device='/job:localhost')
m = tf.keras.Sequential([
hub.KerasLayer(
"https://tfhub.dev/google/tf2-preview/mobilenet_v2/feature_vector/4",
output_shape=[1280],
load_options=load_locally,
trainable=False), # Can be True, see below.
tf.keras.layers.Dense(num_classes, activation='softmax')
])
Source: EfficientNetB7 on 100+ flowers.
Related
I'm using the following code to load an imagenet pre-trained VGG19 model and fit to my custom dataset.
from keras.applications.vgg19 import VGG19
optim = tf.keras.optimizers.RMSprop(momentum=0.9)
vgg19 = VGG19(include_top=False, weights='imagenet', input_tensor=tf.keras.layers.Input(shape=(224, 224, 3)))
vgg19.trainable = False
# x = keras.layers.GlobalAveragePooling2D()(model_vgg19_pt.output)
x = keras.layers.Flatten()(vgg19.output)
output = keras.layers.Dense(n_classes, activation='softmax')(x)
model_vgg19_pt = keras.models.Model(inputs=[vgg19.input], outputs=[output])
model_vgg19_pt.compile(optimizer=optim,
loss='categorical_crossentropy', metrics=['categorical_accuracy'])
callback = tf.keras.callbacks.LearningRateScheduler(scheduler)
model_vgg19_pt.fit(x_train, y_train, batch_size=20,
epochs=50, callbacks=[callback]
)
on model.fit() line, I get the following error
KeyError: 'The optimizer cannot recognize variable dense_1/kernel:0. This usually means you are trying to call the optimizer to update different parts of the model separately. Please call optimizer.build(variables) with the full list of trainable variables before the training loop or use legacy optimizer `tf.keras.optimizers.legacy.{self.class.name}.'
What does it mean and how can I fix it?
I get the same errors for
keras.applications.inception_v3
too, when using the same implementation method.
Additionally, this was working with jupyter notebook file on tensorflow cpu, but when running on a remote machine with tensorflow-gpu installed, I'm getting these errors.
This works fine with optimizer SGD, but not with RMSprop. why?
Additional
Using this:
model_vgg19_pt.compile(optimizer=tf.keras.optimizers.RMSprop(momentum=0.9),
loss='categorical_crossentropy', metrics=['categorical_accuracy'])
instead as used above works. But can somebody explain why....
Which version of Tensorflow GPU have you installed? TensorFlow 2.10 was the last TensorFlow release that supported GPU on native-Windows. Please check the link to install TensorFlow by following all the Hardware/Software requirements for the GPU support.
The LearningRateScheduler arguments in callback is not defined which you are passing while model compilation.
I was able to train the model after removing the callback from model.fit(). (Attaching the gist here for your reference)
I am trying to download the VGG19 model via TensorFlow
base_model = VGG19(input_shape = [256,256,3],
include_top = False,
weights = 'imagenet')
However the download always gets stuck before it finishes downloading. I've tried with different models too like InceptionV3 and the same happens there.
Fortunately, the prompt makes the link available where the model can be downloaded manually
Downloading data from https://storage.googleapis.com/tensorflow/keras-applications/vgg19/vgg19_weights_tf_dim_ordering_tf_kernels_notop.h5
19546112/80134624 [======>.......................] - ETA: 11s
After downloading the model from the given link I try to import the model using
base_model = load_model('vgg19_weights_tf_dim_ordering_tf_kernels_notop.h5')
but I get this error
ValueError: No model found in config file.
How do I load in the downloaded .h5 model manually?
You're using load_model on weights, instead of a model. You need to have a defined model first, then load the weights.
weights = "path/to/weights"
model = VGG19 # the defined model
model.load_weights(weights) # the weights
Got the same problem when learning on tensorflow tutorial, too.
Transfer learning and fine-tuning: Create the base model from the pre-trained convnets
# Create the base model from the pre-trained model MobileNet V2
IMG_SIZE = (160, 160)
IMG_SHAPE = IMG_SIZE + (3,)
base_model = tf.keras.applications.MobileNetV2(input_shape=IMG_SHAPE, include_top=False, weights=None)
# load model weights manually
weights = 'mobilenet_v2_weights_tf_dim_ordering_tf_kernels_1.0_160_no_top.h5'
base_model.load_weights(weights)
I tried download the model.h5, and load manually. It works.
`
Created a simple dummy sequential model in tf.keras as shown below:
model = tf.keras.Sequential()
model.add(layers.Dense(10, input_shape=(100, 100)))
model.add(layers.Conv1D(3, 2))
model.add(layers.Flatten())
model.add(layers.Dense(10, activation='softmax', name='predict_10'))
Trained the model and saved it using tf.keras.models.saved_model.
To get the input input and output node names used saved_model_cli.
saved_model_cli show --dir "path/to/SavedModel" --all
Froze the saved model with freeze_graph.py utility.
python freeze_graph.py --input_saved_model_dir=<path/to/SavedModel> --output_graph=<path/freeze.pb> --input_binary=True --output_node_names=StatefulPartitionedCall
Model is frozen.
Now Here's the main issue:
To load the frozen graph I've used this guide Migrate tf1.x to tf2.x (wrap_frozen_graph)
Used
with tf.io.gfile.GFile("patf/to/freeze.pb", 'rb') as f:
graph_def = tf.compat.v1.GraphDef()
graph_def.ParseFromString(f.read())
load_frozen = wrap_frozen_graph(graph_def, inputs='dense_3_input:0', outputs='predict_10:0')
Output error
ValueError: Input 1 of node StatefulPartitionedCall was passed float from dense_3/kernel:0 incompatible with expected resource.
I'm getting same error when converting .pb to .dlc (Qualcomm).
Actually I want to run original model on Qualcomm's Hexagon DSP or GPU.
I have a tensorflow python model using Keras that works well which I'd like to replicate and use in the browser for both training and predictions completely in the browser. Here's the python model:
model = tf.keras.Sequential([
layers.BatchNormalization(input_shape=(199,)),
layers.Dense(128, activation='relu'),
layers.Dropout(0.25),
layers.Dense(8, activation='sigmoid')
])
When I export the model and import it into tfjs using tfjs.converters.save_keras_model(model, tfjs_export_file_name) it works fine in the browser - the same input data generates the same results.
However when I create a new tfjs model with the same topology as in python and train it in the browser on the same data I get very different results.
const model = tf.sequential({
layers: [
tf.layers.batchNormalization({inputShape:[199]}),
tf.layers.dense({units: 128, inputShape:[199],useBias:true,activation: 'relu'}),
tf.layers.dropout({rate:0.25}),
tf.layers.dense({units: 8, activation: 'sigmoid'}),
]
});
Training, label, and input data have been triple-checked. They are the same.
My goal is to create and train a model in tfjs that matches the Python output.
A simplified version that recreates the problem is here: https://github.com/blairwheadon/tftest/tree/master
Thanks for your insights!
I'm building image processing network in tensorflow and I want to make use of texture loss. Texture loss seems simple to implement if you have pretrained model loaded.
I'm using TF to build the computational graph for my model and I want to incorporate Keras.application.VGG19 model to get output from layer 'block4_conv4'.
The problem is: I have two TF tensors target and result from my main model, how to feed them into keras VGG19 in the same session to compute their diff and use it in main loss for my model?
It seems following code does the trick
with tf.variable_scope("") as scope:
phi_func = VGG19(include_top=False, weights=None, input_shape=(128, 128, 3))
text_1 = phi_func(predicted)
scope.reuse_variables()
text_2 = phi_func(x)
text_loss = tf.reduce_mean((text_1 - text_2)**2)
right after session created I call phi_func.load_weights(path) to initiate weights