I want to know if there is any way to see the parameters of models in tensorflow. there is a command in pytorch i.e. torch.load('/filepath').
For a prediction context, you can do a
model = tf.keras.models.load_model(PATH, compile=True)
That works with both .h5 keras models and models on SavedModel format. Otherwise you might have to provide custom metrics and training code you may not have on the prediction context.
For references, check it here: https://www.tensorflow.org/api_docs/python/tf/keras/models/load_model
Provided that you already have a model saved at MODEL_PATH, this should do the trick:
model = tf.keras.models.load_model(MODEL_PATH)
model.summary()
Check this out for more info on saving and loading models.
Related
I am using a custom keras model in Databricks environment.
For a custom keras model, model.save(model.h5) does not work, because custom model is not serializable. Instead it is recommended to use model.save_weights(path) as an alternate.
model.save_weights(pathDirectory) works. This yields 3 files checkpoint,.data-00000-of-00001,.index in the pathDirectory
For loading weights, Following mechanism is working fine.
model = Model()
model.load_weights(path)
But I want to train my model on pretrained weights I just saved. Like I saved model weights, and continue training on these saved weights afterwards.
So, when I load model weights and apply training loop, I get this error, TypeError: 'CheckpointLoadStatus' object is not callable
After much research, I have found a workaround,
we can also save model using
model.save("model.hpy5") and read it the saved model in databricks.
model.h5 not work for customized models, but it works for standard models.
I'm reading the official tutorial on save-load in Keras and it seems whether I used save or save_weights methods, then the optimizer parameters are going to be saved at any rate. How can save model's weights only?
model.save('./savedmodel.h5', save_format='h5', include_optimizer=False)
If save_format='tf', whether include_optimizer=False or True, it's useless as I tried.
In Keras, to save model weights, do:
model.save_weights('my_model_weights.h5')
To load model weights:
model.load_weights('my_model_weights.h5')
Also see additional example on saving/loading weights by layer name from here.
I have keras pretrained model(model.h5). And I want to prune that model with tensorflow Magnitude-based weight pruning with Keras. One curious things is that my pretrained model is built with original keras model > I mean that is not from tensorflow.keras. Inside tensorflow Magnitude-based weight pruning with Keras example, they show how to do with tensorflow.keras model. I want to ask is that can I use their tool to prune my original keras pretrained model?
inside their weight pruning toolkit ,there is two way. one is pruned the model layer by layer while training and second is pruned the whole model. I tried the second way to prune the whole pretrained model. below is my code.
inside their weight pruning toolkit ,there is two way. one is pruned the model layer by layer while training and second is pruned the whole model. I tried the second way to prune the whole pretrained model. below is my code.
For my original pretrained model, I load the weight from model.h5 and can call model.summary() after I apply prune_low_magnitude() none of the method from model cannot call including model.summary() method. And show the error like AttributeError: 'NoneType' object has no attribute 'summary'
model = get_training_model(weight_decay)
model.load_weights('model/keras/model.h5')
model.summary()
epochs = 1
end_step = np.ceil(1.0 * 100 / 2).astype(np.int32) * epochs
print(end_step)
new_pruning_params = {
'pruning_schedule': tfm.sparsity.keras.PolynomialDecay(initial_sparsity=0.1,
final_sparsity=0.90,
begin_step=40,
end_step=end_step,
frequency=30)
}
new_pruned_model = tfm.sparsity.keras.prune_low_magnitude(model, **new_pruning_params)
print(new_pruned_model.summary())
Inside their weight pruning toolkit
enter link description here ,there is two way. one is pruned the model layer by layer while training and second is pruned the whole model. I tried the second way to prune the whole pretrained model. below is my code.
For my original pretrained model, I load the weight from model.h5 and can call model.summary() after I apply prune_low_magnitude() none of the method from model cannot call including model.summary() method. And show the error like
AttributeError: 'NoneType' object has no attribute 'summary'
I hope this answer still helps, but I recently had the same issue that prune_low_magnitude() returns an object of type 'None'. Also new_pruned_model.compile() would not work.
The model I had been using was a pretrained model that could be imported from tensorflow.python.keras.applications.
For me this worked:
(0) Import the libraries:
from tensorflow_model_optimization.python.core.api.sparsity import keras as sparsity
from tensorflow.python.keras.applications.<network_type> import <network_type>
(1) Define the pretrained model architecture
# define model architecture
loaded_model = <model_type>()
loaded_model.summary()
(2) Compile the model architecture and load the pretrained weights
# compile model
opt = SGD(lr=learn_rate, momentum=momentum)
loaded_model.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])
loaded_model.load_weights('weight_file.h5')
(3) set pruning parameters and assign pruning schedule
# set pruning parameters
pruning_params = {
'pruning_schedule': sparsity.PolynomialDecay(...)
}
# assign pruning schedule
model_pruned = sparsity.prune_low_magnitude(loaded_model, **pruning_params)
(4) compile model and show summary
# compile model
model_pruned.compile(
loss=tf.keras.losses.categorical_crossentropy,
optimizer='SGD',
metrics=['accuracy'])
model_pruned.summary()
It was important to import the libraries specifically from tensorflow.python.keras and use this keras model from the TensorFlow library.
Also, it was important to use the TensorFlow Beta Release (pip install tensorflow==2.0.0b1), otherwise still an object with type 'None' would be returned by prune_low_magnitude.
I am using PyCharm 2019.1.3 (x64) as IDE. Here is the link that led me to this solution: https://github.com/tensorflow/model-optimization/issues/12#issuecomment-526338458
I just finished training a categorizer model exactly the way described in https://github.com/GoogleCloudPlatform/MiniCat but I am not sure how to use the model to make predictions.
Trained model in the direction Train
Data in the directory Data
I'm really new to this and I don't know where to start. I read something about deploying model in https://cloud.google.com/ml-engine/docs/tensorflow/deploying-models but how do I even create a SavedModel.
Any answers will be appreciated.
So in the folder where you got the trained model, you just need to load that model in your session. First create a saver (you can also use it for laoding)
train_saver = tf.train.Saver()
Now inside your session:
train_saver.restore(sess, 'path/to/model/doc_classifier_cnn_model.ckpt')
Then just feed the tensors with feed_dict.
Other option is to create a protobuf file (.pb) but in doing so you will have to load the model as I said.
I am using keras to build a multi-output classification model. My dataset is such as
[x1,x2,x3,x4,y1,y2,y3]
x1,x2,x3 are the features, and y1,y2,y3 are the labels, the y1,y2,y3 are multi-classes.
And I already built a model (I ingore some hidden layers):
def baseline_model(input_dim=23,output_dim=3):
model_in = Input(shape=(input_dim,))
model = Dense(input_dim*5,kernel_initializer='uniform',input_dim=input_dim)(model_in)
model = Activation(activation='relu')(model)
model = Dropout(0.5)(model)
...................
model = Dense(output_dim,kernel_initializer='uniform')(model)
model = Activation(activation='sigmoid')(model)
model = Model(model_in,model)
model.compile(optimizer='adam',loss='binary_crossentropy', metrics=['accuracy'])
return model
And then I try to use the method of keras to make it support classification:
estimator = KerasClassifier(build_fn=baseline_model)
estimator.fit()
estimator.predict(df[0:10])
But I found that the result is not multi-output, only one dimension is output.
[0,0,0,0,0,0,0,0,0,0]
So for the multi-output classification problem, we can not use KerasClassifier function to learn it?
You do not need to wrap the model in KerasClassifier. That wrapper is so that you can use the Keras model with Scikit-Learn. The type of model (classifier, regression, multiclass classifier, etc) is ultimately determined by the shape and activation of the final layer of your model.
You can simply use model.fit() function that is part of Keras. Make sure that you pass the data into the function. You can see more info on the fit function here: https://keras.io/models/model/#fit
Also your loss is setup as binary_crossentropy. For a multi-class problem you will want to use categorical_crossentropy.
model.compile(optimizer='adam',loss='categorical_crossentropy', metrics=['accuracy'])
This model isn't really what Keras refers to as multi-output as far as I can tell. With multi-output you are trying to get the output from several different layers and possibly apply different loss functions to them.
Base on the setup in your question you would be able to use the Keras Sequential model instead of the Functional model if you wanted. Keras recommends using the Sequential model if you can because its simpler. https://keras.io/getting-started/sequential-model-guide/