I have searched the documentation, but I couldn't find the answer. Does SPACY uses ReLu, Softmax or both as activation function?
Thanks
By default, SPACY uses both, as we can see in layers architectures page from SPACY 3.0:
https://spacy.io/usage/layers-architectures
Related
I am trying to replicate YOLO in tensoflow.js. But instead of porting an existing model, because I want to learn how to build models from scratch, I am building it using the layers API.
The problem is the YOLO model uses leaky relu and tensorflow.js does not provide leaky relu as an activation option for conv2d layers. My understanding is that I should use no activation in the conv2d layer and simply add a tf.layers.leakyReLU directly in the model.
I found Pyrhon / Keras awnser How do you use Keras LeakyReLU in Python? But that does not apply to the JS API; especially if I want to run it on my GPU via node.js.
This might sound like the most obvious question, but do I add the leakyReLU before or after the conv2d layer?
Am I missing some bit of API where I can specify an arbitrary activation?
Can tf.contrib.data.make_csv_dataset() be used for Keras models in tensorflow 1.9.0?
Yes, tf.contrib.data.make_csv_dataset() returns a tf.data.Dataset and you can pass tf.data.Dataset to the fit method of Keras models.
See some examples here:
https://www.tensorflow.org/guide/keras#input_tfdata_datasets
I see that there are many similar functions between tensorflow and keras like argmax, boolean_mask...I wonder why people have to use keras as backend along with tensorflow instead of using tensorflow alone.
Keras is not a backend, but it is a high-level API for building and training Neural Networks. Keras is capable of running on top of Tensorflow, Theano and CNTK. Most of the people prefer Keras due to its simplicity compared to other libraries like Tensorflow. I recommend Keras for beginners in Deep Learning.
A Keras tensor is a tensor object from the underlying backend (Theano,
TensorFlow or CNTK), which we augment with certain attributes that
allow us to build a Keras model just by knowing the inputs and outputs
of the model.
Theano vs Tensorflow
Tensorflow is necessary if you wish to use coremltools. Apple has promised support for architectures created using Theano but I haven't seen it yet.
Keras will require unique syntax sugar depending on the backend in use. I like the flexibility of Tensorflow input layers and easy-access to strong Google neural networks.
I know that Keras can be used with CNTK backend.
Question:
In my code can I mix both Keras and CNTK? For example, in my model can I mix up layers (e.g. convolution1d) written in CNTK and Keras?
Nope you can't. Keras implements a wrapper around the backends. You have to wrap your cntk code in the same style as keras.
For objective functions you can write directly in cntk (IIRC)
Can we use spacy with MXnet to build a deep neural network(NLP)
We are building an application using mxnet. How to use spacy with Mxnet
Spacy and MXNet serialize their models differently so they are not directly compatible.
You can leverage the pre-trained models of Spacy as part of a preprocessing step for your text data though, and then feed into an MXNet model. Aim to get your text data into an NDArray format (using mx.nd.array).
Also take a look at MXNet's Model Zoo (https://mxnet.apache.org/model_zoo/index.html) which contains a number of models for NLP tasks; Word2Vec embedding being one example.