Can I change the weights in a CNN like tensorflow? Or the weights of the images are fixed?
In Tensorflow it is possible, depending on the version and on the layer you use. It is even possible for class weights in the fitting method. Best is you read it in the Documentation for your used Version:-)
Related
I would like to train a MLP(Multi Layer Perceptron) with MNIST dataset. I use a validation set so I can save the weights of the best model. Then I want to load these weights back into the same architecture and use them to initialize and train with another dataset. I would like to know if this is possible with Tensorflow 1.x or 2.x. Right now I am trying to write a custom function to do it but it is getting complicated. I am using tf 1.x.
I suggest you take a look at tensorflow's documentation, here a link of a tutorial to save your weights and load them afterwards:
https://www.tensorflow.org/tutorials/keras/save_and_load
I am looking to train a large face identification network. Resnet or VGG-16/19. TensorFlow 1.14
My question is - if I run out of GPU memory - is it valid strategy to train sets of layers one by one?
For example train 2 cnn and maxpooling layer as one set, then "freeze the weights" somehow and train next set etc..
I know I can train on multi-gpu in tensorflow but what if I want to stick to just one GPU..
The usual approach is to use transfer learning: use a pretrained model and fine-tune it for the task.
For fine-tuning in computer vision, a known approach is re-training only the last couple of layers. See for example:
https://www.learnopencv.com/keras-tutorial-fine-tuning-using-pre-trained-models/
I may be wrong but, even if you freeze your weights, they still need to be loaded into the memory (you need to do whole forward pass in order to compute the loss).
Comments on this are appreciated.
I have searched this for a while, but it seems Keras only has quantization feature after the model is trained. I wish to add Tensorflow fake quantization to my Keras sequential model. According to Tensorflow's doc, I need these two functions to do fake quantization: tf.contrib.quantize.create_training_graph() and tf.contrib.quantize.create_eval_graph().
My question is has anyone managed to add these two functions in a Keras model? If yes, where should these two function be added? For example, before model.compile or after model.fit or somewhere else? Thanks in advance.
I worked around by post-training quantization. Since my final goal is to train a mdoel for mobile device, instead of fake quantization during training, I exported keras .h5 file and converted to Tenforflow lite .tflite file directly (with post_training_quantize flag set to true). I tested this on a simple cifar-10 model. The original keras model and the quantized tflite model have very close accuracy (the quantized one a bit lower).
Post-training quantization: https://www.tensorflow.org/performance/post_training_quantization
Convert Keras model to tensorflow lite: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/lite/toco/g3doc/python_api.md
Used the tf-nightly tensorflow here: https://pypi.org/project/tf-nightly/
If you still want to do fake quantization (because for some model, post-training quantization may give poor accuracy according to Google), the original webpage is down last week. But you can find it from github: https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/quantize
Update: Turns out post-quantization does not really quantize the model. During inference, it still uses float32 kernels to do calculations. Thus, I've switched to quantization-aware training. The accuracy is pretty good for my cifar10 model.
I have a resnet model trained in caffe. I want to try that with show_and_tell instead of inception_v3 image model.
What is the best way to convert the model form caffe to tensorflow
model ?
How would I go about switching the inception model to resnet
?
I'm not sure about the "best" way, but there are some tools that seem to address this, like https://github.com/ethereon/caffe-tensorflow
https://github.com/guerzh/tf_weights
I have a reference model, (a TensorFlow implementation of AlexNet with pretrained weights) that I wanted to test on my own personal data set of images. Do you guys know what would be the next steps to doing this?