I trained, checked with my validation set that it learned pretty well, and saved the weigths (ckpt) of my DNN and now I would like to "visualize my classes" (like here : https://www.auduno.com/2015/07/29/visualizing-googlenet-classes/.) by "Instead of using backpropagation to optimize the weights, which we do during training, we keep the weights fixed and instead optimize the input pixels."
Is it possible to restore my model as non trainable ?
If not, is there a way to circumvent this and use my model to it anyway ?
Thank you :)
Related
I have a CNN model and I want to save it and load it for prediction in different tab. But I am confused whether the model.evulotion part is included in the part I will save. And I don't know if it would be better to use Model.checkpoint or model.save to save and load. Is there anyone have an idea ? Thank you in advance
I'm in dilemma about using both of them so I've use it.
Using model.eval() just tells the PyTorch model to use mean values for batch normalisation, and deactivates the dropout layers. You can save your model without using model.eval() as it will not affect the performance.
While saving model, saving model's state dictionary is preferred. This can be done as shown:
#declare class of model here
model = NeuralNetwork()
#add training code below
...
#saving model, the model will be saved at the intermediateWeightPath location
intermediateWeightPath = "./bestmodel.pth"
torch.save(model.state_dict(), intermediateWeightPath)
I'm reading the official tutorial on save-load in Keras and it seems whether I used save or save_weights methods, then the optimizer parameters are going to be saved at any rate. How can save model's weights only?
model.save('./savedmodel.h5', save_format='h5', include_optimizer=False)
If save_format='tf', whether include_optimizer=False or True, it's useless as I tried.
In Keras, to save model weights, do:
model.save_weights('my_model_weights.h5')
To load model weights:
model.load_weights('my_model_weights.h5')
Also see additional example on saving/loading weights by layer name from here.
It appears that
model.layers[n].rate
can be viewed and changed, but does not reach the back end and actually change training behavior. What's the easiest way to change it for real? I'm hoping not to have to make a whole new model and transfer the weights.
The easiest way to achieve this would be:
Change the rates in the layers
model.layers[i].rate = 0.04 #layer[i] is the dropout layer
Clone this model to a new model using
model = keras.models.clone(model) #weights would be reinitialized
Compile the new model
model.compile(optimizer=..., loss=...) #optimizer state would be reset
Set the original weights to the new clone model
model.load_weights(file_weights) #load weights
Discussion on this problem can be found here.
I was wondering why do we need to save the model while training ?
isn't enough to save it once at the beginning of the train and then only save the weights during the train ?
I mean , The model isn't changing during the train , why this boolean is need for ?
class ModelCheckpoint(Callback):
...
save_weights_only: if True, then only the model's weights will be saved.
...
Thanks !
Its not a need or requirement, its just convenience. In a typical DL/DS workflow, you train a lot of models with different configurations and it is quite easy to get lost. Maybe you now have saved the weights for the best model but you don't remember which model configuration it was used. That information is not part of the weights and has to be recorded separately.
Then Keras provides a simple solution, to store the mode (which takes less than 10 KB) along with the weights, so in the case that you lose the original model configuration, it is still saved in the same HDF5 file.
Also consider the case where you send the model weights to someone else without the model configuration, how can you load the weights without a model? Again its just convenience.
When I start training a model, there is no model saved previously. I can use model.compile() safely. I have now saved the model in a h5 file for further training using checkpoint.
Say, I want to train the model further. I am confused at this point: can I use model.compile() here? And should it be placed before or after the model = load_model() statement? If model.compile() reinitializes all the weights and biases, I should place it before model = load_model() statement.
After discovering some discussions, it seems to me that model.compile() is only needed when I have no model saved previously. Once I have saved the model, there is no need to use model.compile(). Is it true or false? And when I want to predict using the trained model, should I use model.compile() before predicting?
When to use?
If you're using compile, surely it must be after load_model(). After all, you need a model to compile. (PS: load_model automatically compiles the model with the optimizer that was saved along with the model)
What does compile do?
Compile defines the loss function, the optimizer and the metrics. That's all.
It has nothing to do with the weights and you can compile a model as many times as you want without causing any problem to pretrained weights.
You need a compiled model to train (because training uses the loss function and the optimizer). But it's not necessary to compile a model for predicting.
Do you need to use compile more than once?
Only if:
You want to change one of these:
Loss function
Optimizer / Learning rate
Metrics
The trainable property of some layer
You loaded (or created) a model that is not compiled yet. Or your load/save method didn't consider the previous compilation.
Consequences of compiling again:
If you compile a model again, you will lose the optimizer states.
This means that your training will suffer a little at the beginning until it adjusts the learning rate, the momentums, etc. But there is absolutely no damage to the weights (unless, of course, your initial learning rate is so big that the first training step wildly changes the fine tuned weights).
Don't forget that you also need to compile the model after changing the trainable flag of a layer, e.g. when you want to fine-tune a model like this:
load VGG model without top classifier
freeze all the layers (i.e. trainable = False)
add some layers to the top
compile and train the model on some data
un-freeze some of the layers of VGG by setting trainable = True
compile the model again (DON'T FORGET THIS STEP!)
train the model on some data