I have a CNN model and I want to save it and load it for prediction in different tab. But I am confused whether the model.evulotion part is included in the part I will save. And I don't know if it would be better to use Model.checkpoint or model.save to save and load. Is there anyone have an idea ? Thank you in advance
I'm in dilemma about using both of them so I've use it.
Using model.eval() just tells the PyTorch model to use mean values for batch normalisation, and deactivates the dropout layers. You can save your model without using model.eval() as it will not affect the performance.
While saving model, saving model's state dictionary is preferred. This can be done as shown:
#declare class of model here
model = NeuralNetwork()
#add training code below
...
#saving model, the model will be saved at the intermediateWeightPath location
intermediateWeightPath = "./bestmodel.pth"
torch.save(model.state_dict(), intermediateWeightPath)
Related
I am using a custom keras model in Databricks environment.
For a custom keras model, model.save(model.h5) does not work, because custom model is not serializable. Instead it is recommended to use model.save_weights(path) as an alternate.
model.save_weights(pathDirectory) works. This yields 3 files checkpoint,.data-00000-of-00001,.index in the pathDirectory
For loading weights, Following mechanism is working fine.
model = Model()
model.load_weights(path)
But I want to train my model on pretrained weights I just saved. Like I saved model weights, and continue training on these saved weights afterwards.
So, when I load model weights and apply training loop, I get this error, TypeError: 'CheckpointLoadStatus' object is not callable
After much research, I have found a workaround,
we can also save model using
model.save("model.hpy5") and read it the saved model in databricks.
model.h5 not work for customized models, but it works for standard models.
It appears that
model.layers[n].rate
can be viewed and changed, but does not reach the back end and actually change training behavior. What's the easiest way to change it for real? I'm hoping not to have to make a whole new model and transfer the weights.
The easiest way to achieve this would be:
Change the rates in the layers
model.layers[i].rate = 0.04 #layer[i] is the dropout layer
Clone this model to a new model using
model = keras.models.clone(model) #weights would be reinitialized
Compile the new model
model.compile(optimizer=..., loss=...) #optimizer state would be reset
Set the original weights to the new clone model
model.load_weights(file_weights) #load weights
Discussion on this problem can be found here.
i got a model converted from caffe by using MMDNN tool, it converted the caffe model into a saved_model tensorflow style. it's a resnet18 model, and i just strip out several layers in the last, i wish i could load this architecture in the model_fn in a tf.estimator, and manually add some extra layers to do my job.
As the tutorial recommended that I could use loader.load method to load the saved_model. But i just want to use it in a estimator, and i need to define the architecture in the model_fn function. I searched out the SO and github but there isn't a very specific workflow to do that thing, somebody could help me out?
Here is one way of fine tuning using tf.Estimator:
Define your model using the SAME variable names/scopes as in your saved model
Use tf.estimator's warm start functions to initialize your new model with the saved weights. Here is a code snippet :
if fine_tuning:
ws = tf.estimator.WarmStartSettings(ckpt_to_initialize_from=path_saved_model,
vars_to_warm_start='.*')
else:
ws = None
estimator = tf.estimator.Estimator(model_fn=model_function,
warm_start_from=ws,
...
)
This will initialize any variable that share names between your currently defined graph and the saved model.
I just finished training a categorizer model exactly the way described in https://github.com/GoogleCloudPlatform/MiniCat but I am not sure how to use the model to make predictions.
Trained model in the direction Train
Data in the directory Data
I'm really new to this and I don't know where to start. I read something about deploying model in https://cloud.google.com/ml-engine/docs/tensorflow/deploying-models but how do I even create a SavedModel.
Any answers will be appreciated.
So in the folder where you got the trained model, you just need to load that model in your session. First create a saver (you can also use it for laoding)
train_saver = tf.train.Saver()
Now inside your session:
train_saver.restore(sess, 'path/to/model/doc_classifier_cnn_model.ckpt')
Then just feed the tensors with feed_dict.
Other option is to create a protobuf file (.pb) but in doing so you will have to load the model as I said.
I was wondering why do we need to save the model while training ?
isn't enough to save it once at the beginning of the train and then only save the weights during the train ?
I mean , The model isn't changing during the train , why this boolean is need for ?
class ModelCheckpoint(Callback):
...
save_weights_only: if True, then only the model's weights will be saved.
...
Thanks !
Its not a need or requirement, its just convenience. In a typical DL/DS workflow, you train a lot of models with different configurations and it is quite easy to get lost. Maybe you now have saved the weights for the best model but you don't remember which model configuration it was used. That information is not part of the weights and has to be recorded separately.
Then Keras provides a simple solution, to store the mode (which takes less than 10 KB) along with the weights, so in the case that you lose the original model configuration, it is still saved in the same HDF5 file.
Also consider the case where you send the model weights to someone else without the model configuration, how can you load the weights without a model? Again its just convenience.