Is there a way to import a tensorflow checkpoint to a keras model before starting the training, in order to do some changes to the model?
Related
If I have the saved description and weights of the model in json and .h5 files respectively, how can i continue/transfer training the SSD model in Keras on additional data? Which Keras SSD implementation would be simple to do this? https://github.com/pierluigiferrari/ssd_keras?
from keras.models import model_from_json
with open("path_to_json_file.json") as json_file:
model = model_from_json(json_file.read())
model.load_weights("path_to_weights_file.h5")
Suppose I have a pre-trained model stored in a Tensorflow checkpoint. I'd like to convert it into a Keras model. I can load the checkpoint into a TF session alright but that's where I get stuck.
I think it's impossible to create a Keras model using TF checkpoint, but you can copy it's weights to the already created Keras model.
Checkout this. https://github.com/yuyang-huang/keras-inception-resnet-v2
The extract_weights.py is to save the TF weights to numpy array, while load_weights.py is for load the npy file to the Keras model.
For more reference, this is how I implement it https://github.com/DableUTeeF/keras-efficientnet/tree/master/keras_efficientnet.
I am training to train and distribute my keras functional-model onto the different GPUs on my system. This works in the first step, after checking my GPU-utilization (command: watch -n0.5 nvidia-smi). After restoring the functional again, the parallelization for the different branches of the model does not work after all in the training.
import tensorflow as tf
import numpy as np
from tensorflow.keras.layers import Input, Dense, Concatenate
from tensorflow.keras.models import Model, load_model, Sequential
with tf.device('/gpu:0'):
Input_1=Input(shape=(256,),name="Input_1")
Dense_1=Dense(128)(Input_1)
with tf.device('/gpu:1'):
Input_2=Input(shape=(256,),name="Input_2")
Dense_2=Dense(128)(Input_2)
with tf.device('/gpu:2'):
Input_3=Input(shape=(256,),name="Input_3")
Dense_3=Dense(128)(Input_3)
with tf.device('/cpu:0'):
Concatenate_=Concatenate()([Dense_1,Dense_2,Dense_3])
output=Dense(1)(Concatenate_)
model=Model(inputs=[Input_1,Input_2,Input_3],outputs=[output])
model=tf.keras.utils.multi_gpu_model(model,gpus=3)
model.compile(optimizer="sgd",loss="mean_squared_error")
model.save("./test.h5")
input_dict={}
for k in ["Input_1","Input_2","Input_3"]:
input_dict.update({k: np.random.standard_normal((10000,256))})
output_dict={model.layers[-1].name: np.random.standard_normal((10000,1))}
test_0=model.fit(x=input_dict,y=output_dict,batch_size=128,epochs=20)
del model
model=load_model("./test.h5")
test_1=model.fit(x=input_dict,y=output_dict,batch_size=128,epochs=20)
After restoration model in the last part of the code, the model uses only one GPU. Removing the line "model=tf.keras.utils.multi_gpu_model(model,gpus=3)" does not help after all.
My 3 GPUs are GEFORCE RTX 2080 Ti and I am using Anacondaa on Ubuntu. How do I make the model work with GPU-parallelization also when I restore it through "load_model"?
Is it possible to convert a keras model (h5 file of network architecture and weights) into a tensorflow model? Or is there an equivalent function to model.save of keras in tensorflow?
Yes, it is possible, because Keras, since it uses Tensorflow as backend, also builds computational graph. You just need to get this graph from your Keras model.
"Keras only uses one graph and one session. You can access the session
via: K.get_session(). The graph associated with it would then be:
K.get_session().graph."
(from fchollet: https://github.com/keras-team/keras/issues/3223#issuecomment-232745857)
Or you can save this graph in checkpoint format (https://www.tensorflow.org/api_docs/python/tf/train/Saver):
import tensorflow as tf
from keras import backend as K
saver = tf.train.Saver()
sess = K.get_session()
retval = saver.save(sess, ckpt_model_name)
By the way, since tensorflow 13 you can use keras right from it:
from tensorflow.python.keras import models, layers
In tensorflow I can do something like this when creating a session:
tf.GPUOptions(per_process_gpu_memory_fraction=0.333,allow_growth=True)
Is there a way to do the same in keras with the tensorflow backend?
You can set the Keras global tensorflow session with keras.backend.tensorflow_backend.set_session():
import tensorflow as tf
import keras.backend.tensorflow_backend as ktf
def get_session(gpu_fraction=0.333):
gpu_options = tf.GPUOptions(per_process_gpu_memory_fraction=gpu_fraction,
allow_growth=True)
return tf.Session(config=tf.ConfigProto(gpu_options=gpu_options))
ktf.set_session(get_session())