I am trying to convert a tf2 BERT checkpoint to a pytorch model in order to upload it on Huggingface hub. The tf2 BERT checkpoint was obtained using the official tensorflow scripts.
Everything goes well in the training routine (performances and graphs are looking good), but now I am not able to convert this tf2 checkpoint a pytorch model. I know that there are 2 scripts for doing this type of conversion (one for tf1 checkpoints and one for tf2 checkpoints), but the script for tf2 checkpoints performs only the encoder conversion.
I would like to export a tf2 BERT checkpoint to a pytorch model, exporting the MLM head alongside the encoder.
Does anybody have an idea? Thank you!
Related
What I tried so far:
pre-train a model using unsupervised method in PyTorch, and save off the checkpoint file (using torch.save(state, filename))
convert the checkpoint file to onnx format (using torch.onnx.export)
convert the onnx to tensorflow saved model (using onnx-tf)
trying to load the variables in saved_model folder as checkpoint in my tensorflow training code (using tf.train.init_from_checkpoint) for fine-tuning
But now I am getting stuck at step 4 because I notice that variables.index and variables.data#1 files are basically empty (probably because of this: https://github.com/onnx/onnx-tensorflow/issues/994)
Also, specifically, if I try to use tf.train.NewCheckpointReader to load the files and call ckpt_reader.get_variable_to_shape_map(), _CHECKPOINTABLE_OBJECT_GRAPH is empty
Any suggestions/experience are appreciated :-)
What would be the easiest way to convert a .pb or a .h5 model to a .meta file.
I have 2 programs, one generating a model a keras and one loading the same model in tensorflow 1.0. Can anyone help me with converting the keras model to a metagraphdef and get all trainable variables with weights,
I trained an ANN model as saved it as .h5 file.Then I converted the model into tensorflow model and got 'savedmodel.pb' and 'variables' folder.
Then I used model optimiser openvino to generate IR files using:
python3 mo_tf.py --input_model saved_model.pb
But I get the following error:
[ FRAMEWORK ERROR ] Error parsing message
TensorFlow cannot read the model file: "/home/user/Downloads/OpenVino/dldt-2019/model-optimizer/saved_model.pb" is incorrect TensorFlow model file
Can openVINO be used to convert ANN models in the first place?
Unfortunately, OpenVINO does not support Keras model format.
I guess you got an error from model optimizer because your model is not frozen.
There are a lot of scripts which can be used to convert a Keras model to a frozen Tensorflow graph. I can recommend this one for example.
Hope it will help.
Suppose I have a pre-trained model stored in a Tensorflow checkpoint. I'd like to convert it into a Keras model. I can load the checkpoint into a TF session alright but that's where I get stuck.
I think it's impossible to create a Keras model using TF checkpoint, but you can copy it's weights to the already created Keras model.
Checkout this. https://github.com/yuyang-huang/keras-inception-resnet-v2
The extract_weights.py is to save the TF weights to numpy array, while load_weights.py is for load the npy file to the Keras model.
For more reference, this is how I implement it https://github.com/DableUTeeF/keras-efficientnet/tree/master/keras_efficientnet.
Anyone knows how to save tensorflow keras checkpoint file into .pb binary file for serving?
I am not 100% sure how to implement this.
Do I need to declare tf.keras.backend.get_session() to save graph and pb file?
In sum,
1. Method to save pb file via tf.keras
or
2. Any approaches using tf.keras.estimator.model_to_estimator(keras_model=model)
Thanks
Here is a link on how to use keras with tensorflow. Basically you should get the tensorflow model from keras (K.get_session() and K.get_graph().graph) and use the tensorflow methotd to save it.