I am trying to set a number of layers, neurons and activation functions to my Encog Model. Following this post, I tried to implement the MLMethodFactory.
EncogModel model = new EncogModel(data);
MLMethodFactory methodFactory = new MLMethodFactory();
MLMethod method = methodFactory.create(MLMethodFactory.TYPE_FEEDFORWARD, "?:B->SIGMOID->6:B->SIGMOID->5:B->SIGMOID->?" ,5,1);
model.selectMethod(data, method.toString());
The above code however returns me this error message:
org.encog.EncogError: Don't know how to autoconfig method: [BasicNetwork: Layers=4]
at org.encog.ml.model.EncogModel.selectMethod(EncogModel.java:379)
at application
How can I solve this issue?
Related
To set the context, if i go to : https://huggingface.co/ProsusAI/finbert and input the following sentence on their hosted API form
Stocks rallied and the British pound gained.
I get the sentiment as 89.8% positive,6.7% neutral and the rest negative, which is as one would expect.
However if I download the tensorflow version of the model from :https://huggingface.co/ProsusAI/finbert/tree/main along with the respective Json files, and it run it locally I get the output as
array([[0.2945392 , 0.4717328 , 0.23372805]] which corresponds to a ~ 30% positive sentiment.
The code i am using locally is as follows ( modfin is the local folder where i have stored the t5_model.h5 alongwith the other files)
model = TFAutoModelForSequenceClassification.from_pretrained("C:/Users/Downloads/modfin",config="C:/Users/Downloads/modfin/config.json",num_labels=3)
tokenizer = AutoTokenizer.from_pretrained("C:/Users/Downloads/modfin",config="C:/Users/Downloads/modfin/tokenizer_config.json")
inputs = tokenizer(sentences, padding = True, truncation = True, return_tensors='tf')
outputs = model(**inputs)
nn.softmax(outputs[0]).numpy()
for the model I also get a warning as follows
All model checkpoint layers were used when initializing TFBertForSequenceClassification.
Some layers of TFBertForSequenceClassification were not initialized from the model checkpoint at C:/Users/Downloads/modfin and are newly initialized: ['classifier']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
which is strange since I would assume a pre-trained model such as finbert should already be fine-tuned. when i replace TFAutoModelForSequenceClassification with TFAutoModel i see that the transofrmer that is chosen automatically is 'TFBertModel' whose output is 10 X768 tensor which i am not able to interpret into sentiment classes. any help here would be greatly appreciated
I am trying to learn Trax. I have previous exprience with Tensorflow, and I prefer writing neural networks with functional api(https://www.tensorflow.org/guide/keras/functional). I was wondering if it is possible to do the same with Trax, the resources I checked are in Serial(Sequential in Tenforflow/keras).
example code:
input = (placeholder)
layer1 = (placeholder)(input)
output = (placeholder)(layer1)
I was building a neural network in tensorflow keras and ended up with the following code as a step in the model:
enc = tfkl.Reshape((-1, 20,input_shape[1]))(input_layer)
encoder_output = []
for i in range(enc.shape[1]):
o = encoder(enc[:, i, ...])
encoder_output.append(o)
encoder_output = tf.stack(encoder_output, axis=1)
The code WORKS FINE. However it is quite long in the summary and in general not very elegant.
I don't know much about functional API and custom layers. But I would like to know if in this case
it's best to use a custom layer or build a functional API block.
Thank you
I am trying to follow this guide to implement a seq2seq machine tranlsation model: https://www.tensorflow.org/tutorials/text/nmt_with_attention
The tutorial's Encoder has an initialize_hidden_state() function that is used to generate all 0 as initial state for the encoder. However I am a bit confused as to why this is neccessary. As far as I can tell, the only times when encoder is called (in train_step and evaluate), they were initialized with the initialize_hidden_state() function. My questions are 1.) what is the purpose of this initial state? Doesn't Keras layer automatically initialize LSTM states to begin with? And 2.) why not always just initialize the encoder with all 0 hidden states if encoder is always called with initial states generated by initialize_hidden_state()?
you are totally right. The code in the example is a little misleading. The LSTM cells are automatically initialized with zeros. You can just delete the initialize_hidden_state() function.
I make a densenet network from gluon.vision
densenet = vision.densenet121(pretrained=True, ctx=mx.cpu())
I want to get the outputs of each convolutionnal layer (after a prediction), to plot them afterwards (features maps).
I can't do densenet.get_internals() (as I saw on internet and Github), as my network is not a Symbol but a HybridBlock.
I find a solution in mxnet forum :
Gluon, get features maps of a CNN
Actually, you have to transform the gluon model to a symbol using methods export() to save parameters (method from HybridBlock), and mx.sym.load() to load them.
function get_interals()["name_of_the_layer"] get all the layers from begining to this layer, so you can do feat_maps = net(image) to get all the features maps for this layer.
Then you can do a SummaryWriter in mxBoard to export it to Tensorboard.