I am trying to learn Trax. I have previous exprience with Tensorflow, and I prefer writing neural networks with functional api(https://www.tensorflow.org/guide/keras/functional). I was wondering if it is possible to do the same with Trax, the resources I checked are in Serial(Sequential in Tenforflow/keras).
example code:
input = (placeholder)
layer1 = (placeholder)(input)
output = (placeholder)(layer1)
Related
I am currently trying to create embeddings of images by passing them through pre-trained Neural Networks and getting the values obtained at the last layer just before the fully-connected ones. I did not have much problem doing it with Pytorch implementations of other Neural Networks. However, I am stuck with the Tensorflow implementation of SimCLR v2 and do not know how to proceed.
The official repo of SimCLR v2 is this one: https://github.com/google-research/simclr
And the paper is here: https://arxiv.org/abs/2006.10029v2
If I understood correctly the paper and the code, this architecture is composed of a backbone ResNet as well as a projection head. In my case, I am not interested in the projection head and just want to obtain the results of the output of the ResNet model.
Looking at the code in the colabs, I have managed to import pre-trained SimCLR models:
model_path = 'gs://simclr-checkpoints-tf2/simclrv2/pretrained/r50_1x_sk0/saved_model'
saved_model = tf.saved_model.load(model_path)
However, I do not know what to do to get the outputs of the ResNet. In all the colabs, they only get the outputs of the projection head which I am uninterested in.
for x in ds.take(1):
image = x['image']
labels = x['label']
logits = saved_model(image, trainable=False)['logits_sup']
pred = tf.argmax(logits, -1)
Moreover, the way the model is imported makes it difficult to get the variables and layers. For instance if I try obtain a summary of the model, I have this error:
'_UserObject' object has no attribute 'summary'
I also do not want to convert the weights of Tensorflow into Pytorch and import them into a pytorch ResNet.
What then would be the best way to isolate the ResNet from the overall SimCLR v2 architecture in order to get the outputs of the final layer ?
I recently learned about a NN explainable tool called Shapley Values.
For whoever is wondering, I train a dense neural network (although it can be done with CNNs and LSTMs too) and then use the following lines:
background = X[np.random.choice(X.shape[0], 10000, replace=False)]
e = shap.DeepExplainer(model_trained, background)
shap_values = e.shap_values(background)
shap.summary_plot(shap_values[0], background, feature_names=cols_of_X, plot_type='dot',show=False)
The issue is that this only works for models that do not include batch normalization and leaky relus.
I know that I can just use
tf.keras.Model(inputs = model_trained.layers[0].input,outputs=model_trained.layers[1].output) instead of model_trained but then I'll only see how the first layer was affected by the input features and not how the output was affected by the input features.
Anyway have a way to overcome this or just use something other than Shapley?
Thanks!
I have successfully followed this official tutorial on image classification with transfer learning: https://www.tensorflow.org/tutorials/images/transfer_learning_with_hub
My experimental model is now saved and supposed to recognize when it sees a "good" painting. However, I want to test this with an image that the model has not seen before. So far I have only used notebooks where the dataset is already divided into train and test folders. However, this is not the case here.
I assume I need something like
img = tf.keras.preprocessing.image.load_img("/content/mytestimage.jpeg", target_size=(224,224))
among other things; however, for a beginner it would be useful to see an example of this kind of test prediction. So far I have searched without results - if anyone has any advice I'm super happy to hear!
Here's how to do it with mobilenet transfer learning with keras but most of the code should be the same. A full transfer learning tutorial can be found here. I found it very useful.
from PIL import Image
from tensorflow.keras.models import load_model
model = load_model('path/to/model.h5')
img = Image.open(file)
array = np.asarray(img, dtype=np.float32)
arrayexp = np.expand_dims(array, axis=0)
arrayexp = (arrayexp/127)-1 #This is a normalization factor specifically for Mobilenet but I think it's used for many other networks
result = model.predict(arrayexp)
print(np.argmax(result)) #Prints class with highest confidence
print(result[0][np.argmax(result)]) #Prints confidence for the highest
I am currently working on a program that uses a CNN tensorflow neural network and I want to use my own images to train and test it, please I want some advice because I am new in deep learning
Thanks.
Download the ronnie package from python.[{pythonpath}\scripts\pip3 ronnie]
Construct the training data structure as [label, image pixel] and execute the below code to create the training data.
dataFileLoc = os.path.join(dir,"./data/digitalRec/train.csv")
orgDF = collector.initial(dataFileLoc)
########################################################################### Return Object from buildFormDF are ImgXs and LabelYs in dict Structure
{meta.DATA_INPUTXs: imgXs, meta.DATA_LABELYs: labelYs}
########################################################################## trainData = collector.buildFromDF(orgDF,batchNum=1, batchSize=30000)
I'd like to build a conversational modal that can predict a sentence using the previous sentences using TensorFlow LSTMs . The example provided in TensorFlow tutorial can be used to predict the next word in a sentence .
https://www.tensorflow.org/versions/v0.6.0/tutorials/recurrent/index.html
lstm = rnn_cell.BasicLSTMCell(lstm_size)
# Initial state of the LSTM memory.
state = tf.zeros([batch_size, lstm.state_size])
loss = 0.0
for current_batch_of_words in words_in_dataset:
# The value of state is updated after processing each batch of words.
output, state = lstm(current_batch_of_words, state)
# The LSTM output can be used to make next word predictions
logits = tf.matmul(output, softmax_w) + softmax_b
probabilities = tf.nn.softmax(logits)
loss += loss_function(probabilities, target_words)
Can I use the same technique to predict the next sentence ? Is there any working example on how to do this ?
You want to use the Sequence-to-sequence model. Instead of having it learn to translate sentences from a source language to a target language you have it learn responses to previous utterances in the conversation.
You can adapt the example seq2seq model in tensorflow by using the analogy that the source language 'English' is your set of previous sentences and target language 'French' are your response sentences.
In theory you could use the basic LSTM you were looking at by concatenating your training examples with a special symbol like this:
hello there ! __RESPONSE hi , how can i help ?
Then during testing you run it forward with a sequence up to and including the __RESPONSE symbol and the LSTM can carry it the rest of the way.
However, the seq2seq model above should be much more accurate and powerful because it had a separate encoder / decoder and includes an attention mechanism.
A sentence is composed words, so you can indeed predict the next sentence by predicting words sequentially. There are models, such as the one described in this paper, that build embeddings for entire paragraphs, which can be useful for your purpose. Of course there is Neural Conversational Model work that probably directly fits your need. TensorFlow doesn't ship with working examples of these models, but the recurrent models that come with TensorFlow should give you a good starting point for implementing them.