Hi I am using tensorflow and model optimisations.
This is an overview of the process:
from tensorflow_model_optimization.quantization.keras import quantise_model
model = define_model()
qat_model = quantize_model(model)
qat_model.fit(...)
qat_model.save_weights("qat_weights.h5")
... Finish for Now ...
On another run
model = define_model()
qat_model = quantize_model(model)
qat_model.load_weights("qat_weights.h5")
But when I go to qat_model.fit(...) it has to start training again from 0%
So there must be a problem with either saving or loading of the weights
Related
I am trying to download the VGG19 model via TensorFlow
base_model = VGG19(input_shape = [256,256,3],
include_top = False,
weights = 'imagenet')
However the download always gets stuck before it finishes downloading. I've tried with different models too like InceptionV3 and the same happens there.
Fortunately, the prompt makes the link available where the model can be downloaded manually
Downloading data from https://storage.googleapis.com/tensorflow/keras-applications/vgg19/vgg19_weights_tf_dim_ordering_tf_kernels_notop.h5
19546112/80134624 [======>.......................] - ETA: 11s
After downloading the model from the given link I try to import the model using
base_model = load_model('vgg19_weights_tf_dim_ordering_tf_kernels_notop.h5')
but I get this error
ValueError: No model found in config file.
How do I load in the downloaded .h5 model manually?
You're using load_model on weights, instead of a model. You need to have a defined model first, then load the weights.
weights = "path/to/weights"
model = VGG19 # the defined model
model.load_weights(weights) # the weights
Got the same problem when learning on tensorflow tutorial, too.
Transfer learning and fine-tuning: Create the base model from the pre-trained convnets
# Create the base model from the pre-trained model MobileNet V2
IMG_SIZE = (160, 160)
IMG_SHAPE = IMG_SIZE + (3,)
base_model = tf.keras.applications.MobileNetV2(input_shape=IMG_SHAPE, include_top=False, weights=None)
# load model weights manually
weights = 'mobilenet_v2_weights_tf_dim_ordering_tf_kernels_1.0_160_no_top.h5'
base_model.load_weights(weights)
I tried download the model.h5, and load manually. It works.
`
my situation is that saving model is extremely slow under Colab TPU environment.
I first encountered this issue when using checkpoint callback, which causes the training stuck at the end of the 1st epoch.
Then, I tried taking out callback and just save the model using model.save_weights(), but nothing has changed. By using Colab terminal, I found that the saving speed is about ~100k for 5 minutes.
The version of Tensorflow = 2.3
My code of model fitting is here:
with tpu_strategy.scope(): # creating the model in the TPUStrategy scope means we will train the model on the TPU
Baseline = create_model()
checkpoint = keras.callbacks.ModelCheckpoint('baseline_{epoch:03d}.h5',
save_weights_only=True, save_freq="epoch")
hist = model.fit(get_train_ds().repeat(),
steps_per_epoch = 100,
epochs = 5,
verbose = 1,
callbacks = [checkpoint])
model.save_weights("epoch-test.h5", overwrite=True)
I found the issue happened because I explicitly switched to graph mode by writing
from tensorflow.python.framework.ops import disable_eager_execution
disable_eager_execution()
Before
with tpu_strategy.scope():
model.fit(...)
Though I still don't understand the cause, remove disable_eager_execution solved the issue.
Recently, I've decided to apply some reinforcement learning and deep Q learning I've learned to the LunarLander environment by OpenAI.
My algorithm is just Deep Q-learning with experience replay and I wanted to be able to save the model/agent and then load up the model on its own and make it just interact with the environment without any fitting/training of its weights. I had chosen to save a few models using q_network.save(directory+"lunar_model_score{}.h5".format(accum_reward)) at the end of each iteration/episode with the highest consecutive scores and low epsilon value (so that the model is doing more predicting than exploring) during training.
However when I try to load the model again elsewhere and try to run the model just in the environment without training, it performs very poorly as if it had not been trained, code for testing:
import gym
import tensorflow as tf
import matplotlib.pyplot as plt
import numpy as np
env = gym.make('LunarLander-v2')
action_space = env.action_space.n
state_space = env.observation_space.shape[0]
lunar_agent = tf.keras.models.load_model('C:/Users/haora/gymEnv/LunarLand/models/lunar_model_score215.65755254109038.h5')
file_name = 'lunarLand_test_data.txt'
datafile = open(file_name,"w+")
episodes = 10
lunar_agent.summary()
#print(lunar_agent.get_weights())
for e in range(episodes):
state = env.reset()
accum_reward = 0
while True:
env.render()
state = np.reshape(state,[1,state_space])
prediction = lunar_agent.predict(state)
action = np.argmax(prediction[0])
next_state, reward, done, _ = env.step(action)
accum_reward += reward
if done:
break
print("episode:{}/{} | score:{}".format(e,episodes,accum_reward))
datafile.write(str(e)+','+str(accum_reward)+'\n')
env.close()
datafile.close()
I've verified that the weight values and architecture I saved in the training file is the same as the weights i got when I called print(lunar_agent.get_weights()), so I was wondering why there is such a big discrepency between the model when training and the model when only interacting with the environment and how to fix it so that I can run different models at different iterations of training and make the agent perform accordingly when only interacting with the environment.
I have repurposed an Inception V3 network using the transfer learning method, following this article.
For that, I removed the final network layer, and fed hundreds of images of my face into the network.
A new model was then sucessfully generated: inceptionv3-ft.model
Now I would like to load this model and use its fixed weights to apply my face as a 'theme' on a input image, like google-dream.
For that I am using a keras program, which loads models like so:
from keras.applications import inception_v3
# Build the InceptionV3 network with our placeholder.
# The model will be loaded with pre-trained ImageNet weights.
model = inception_v3.InceptionV3(weights='imagenet',
include_top=False)
dream = model.input
Full code here: https://github.com/keras-team/keras/blob/master/examples/deep_dream.py
So, how do I load and pass not a pre-trained but rather my RE-trained model weights?
simply:
from keras.models import load_model
model = load_model('inceptionv3-ft.model')
dream = model.input
I'm working on facial expression recognition using CNN. I'm using Keras and Tensorflow as backend. My model is saved to h5 format.
I want to retrain my network, and fine-tune my model with the VGG model.
How can I do that with keras ?
Save your models architecture and weights:
json_string = model.to_json()
model.save_weights('model_weights.h5')
Load model architecture and weights:
from keras.models import model_from_json
model = model_from_json(json_string)
model.load_weights('model_weights.h5')
Start training again from here for finetuning. I hope this helps.
You can use the Keras model.save(filepath) function.
Details for the various Keras saving and loading techniques are discussed with examples in this YouTube video: Save and load a Keras model
model.save(filepath)saves:
The architecture of the model, allowing to re-create the model.
The weights of the model.
The training configuration (loss, optimizer).
The state of the optimizer, allowing to resume training exactly where you left off.
To load this saved model, you would use the following:
from keras.models import load_model
new_model = load_model(filepath)
If you used model.to_json(), you would only be saving the architecture of the model. Additionally, if you used model.save_weights(), you would only be saving the weights of the model. With both of these alternative saving techniques, you would not be saving the training configuration (loss, optimizer), nor would you be saving the state of the optimizer.