I make a densenet network from gluon.vision
densenet = vision.densenet121(pretrained=True, ctx=mx.cpu())
I want to get the outputs of each convolutionnal layer (after a prediction), to plot them afterwards (features maps).
I can't do densenet.get_internals() (as I saw on internet and Github), as my network is not a Symbol but a HybridBlock.
I find a solution in mxnet forum :
Gluon, get features maps of a CNN
Actually, you have to transform the gluon model to a symbol using methods export() to save parameters (method from HybridBlock), and mx.sym.load() to load them.
function get_interals()["name_of_the_layer"] get all the layers from begining to this layer, so you can do feat_maps = net(image) to get all the features maps for this layer.
Then you can do a SummaryWriter in mxBoard to export it to Tensorboard.
Related
To set the context, if i go to : https://huggingface.co/ProsusAI/finbert and input the following sentence on their hosted API form
Stocks rallied and the British pound gained.
I get the sentiment as 89.8% positive,6.7% neutral and the rest negative, which is as one would expect.
However if I download the tensorflow version of the model from :https://huggingface.co/ProsusAI/finbert/tree/main along with the respective Json files, and it run it locally I get the output as
array([[0.2945392 , 0.4717328 , 0.23372805]] which corresponds to a ~ 30% positive sentiment.
The code i am using locally is as follows ( modfin is the local folder where i have stored the t5_model.h5 alongwith the other files)
model = TFAutoModelForSequenceClassification.from_pretrained("C:/Users/Downloads/modfin",config="C:/Users/Downloads/modfin/config.json",num_labels=3)
tokenizer = AutoTokenizer.from_pretrained("C:/Users/Downloads/modfin",config="C:/Users/Downloads/modfin/tokenizer_config.json")
inputs = tokenizer(sentences, padding = True, truncation = True, return_tensors='tf')
outputs = model(**inputs)
nn.softmax(outputs[0]).numpy()
for the model I also get a warning as follows
All model checkpoint layers were used when initializing TFBertForSequenceClassification.
Some layers of TFBertForSequenceClassification were not initialized from the model checkpoint at C:/Users/Downloads/modfin and are newly initialized: ['classifier']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
which is strange since I would assume a pre-trained model such as finbert should already be fine-tuned. when i replace TFAutoModelForSequenceClassification with TFAutoModel i see that the transofrmer that is chosen automatically is 'TFBertModel' whose output is 10 X768 tensor which i am not able to interpret into sentiment classes. any help here would be greatly appreciated
I am currently trying to create embeddings of images by passing them through pre-trained Neural Networks and getting the values obtained at the last layer just before the fully-connected ones. I did not have much problem doing it with Pytorch implementations of other Neural Networks. However, I am stuck with the Tensorflow implementation of SimCLR v2 and do not know how to proceed.
The official repo of SimCLR v2 is this one: https://github.com/google-research/simclr
And the paper is here: https://arxiv.org/abs/2006.10029v2
If I understood correctly the paper and the code, this architecture is composed of a backbone ResNet as well as a projection head. In my case, I am not interested in the projection head and just want to obtain the results of the output of the ResNet model.
Looking at the code in the colabs, I have managed to import pre-trained SimCLR models:
model_path = 'gs://simclr-checkpoints-tf2/simclrv2/pretrained/r50_1x_sk0/saved_model'
saved_model = tf.saved_model.load(model_path)
However, I do not know what to do to get the outputs of the ResNet. In all the colabs, they only get the outputs of the projection head which I am uninterested in.
for x in ds.take(1):
image = x['image']
labels = x['label']
logits = saved_model(image, trainable=False)['logits_sup']
pred = tf.argmax(logits, -1)
Moreover, the way the model is imported makes it difficult to get the variables and layers. For instance if I try obtain a summary of the model, I have this error:
'_UserObject' object has no attribute 'summary'
I also do not want to convert the weights of Tensorflow into Pytorch and import them into a pytorch ResNet.
What then would be the best way to isolate the ResNet from the overall SimCLR v2 architecture in order to get the outputs of the final layer ?
Feature visualizing in tensor flow or keras is easy and can be found here. https://machinelearningmastery.com/how-to-visualize-filters-and-feature-maps-in-convolutional-neural-networks/ or Convolutional Neural Network visualization - weights or activations?
how to do this in pytorch?
I am using PyTorch with pretrained resnet18 model. All i need to input the image and get activation for specific layer(e.g. Layer2.0.conv2). Layer2.0.conv2 is specified in the pretrained model.
In simple words; how to convert link one code to PyTorch? how to get the specific layers in resnet18 PyTorch and how to get the activation for input image.
I tried this in tensorflow and it worked but not PyTorch.
You would have to register PyTorch's hooks on specific layer. See this tutorial for intro about hooks.
Basically, it allows to capture input/output of forward/backward going into the torch.nn.Module. Whole thing could be a bit complicated, there exists a library with similar goal to your (disclaimer I'm the author), called torchfunc. Especially torchfunc.hooks.recorder allows you to do what you want, see code snippet and comments below:
import torchvision
import torchfunc
my_network = torchvision.resnet18(pretrained=True)
# Recorder saving inputs to all submodules
recorder = torchfunc.hooks.recorders.ForwardPre()
# Will register hook for all submodules of resnet18
# You could specify some submodules by index or by layer type, see docs
recorder.modules(my_networks)
# Push example image through network
my_network(torch.randn(1, 3, 224, 224))
You could register recorder only for some layers (submodule) specified by index or layer type, to get necessary info, run:
# Zero image before going into the third submodule of this network
recorder.data[3][0]
# You can see all submodules and their positions by running this:
for i, submodule in enumerate(my_network.modules()):
print(i, submodule)
# Or you can just print the network to get this info
print(my_network)
i got a model converted from caffe by using MMDNN tool, it converted the caffe model into a saved_model tensorflow style. it's a resnet18 model, and i just strip out several layers in the last, i wish i could load this architecture in the model_fn in a tf.estimator, and manually add some extra layers to do my job.
As the tutorial recommended that I could use loader.load method to load the saved_model. But i just want to use it in a estimator, and i need to define the architecture in the model_fn function. I searched out the SO and github but there isn't a very specific workflow to do that thing, somebody could help me out?
Here is one way of fine tuning using tf.Estimator:
Define your model using the SAME variable names/scopes as in your saved model
Use tf.estimator's warm start functions to initialize your new model with the saved weights. Here is a code snippet :
if fine_tuning:
ws = tf.estimator.WarmStartSettings(ckpt_to_initialize_from=path_saved_model,
vars_to_warm_start='.*')
else:
ws = None
estimator = tf.estimator.Estimator(model_fn=model_function,
warm_start_from=ws,
...
)
This will initialize any variable that share names between your currently defined graph and the saved model.
I am using tensorflow 1.3.0 to train a CNN classification model. However I need to get access to the prelogits layer to evaluate my method (i.e. while this is casted as a classification problem, the method is not a classification problem but is used to extract CNN features, i.e. to produce a point in an N-dimensional vector space for an input image test)
I am using both the dataset API (with TFRecord files) and the estimator API to train the model. However, I don't see how I can get access/return the prelogits value using the Estimator API, i.e. estimator.train(), .evaluate() or .predict() since model_fn() needs to return a specific tf.estimator.EstimatorSpec object.
Previously (i.e. using the standard sess=tf.Session() method) I could train the model and get access to the prelogits layer while training (or by loading the model after training) and feed the network with a specific input to get the specific layer output with a sess.run(specific_layer) as long as the layer was named specific_layer.
I have tried to use the prediction output of EstimatorSpec but it did not work. Any ideas/suggestions?