How to connect different deep learning architectures? - tensorflow

Based on 5 features extracted from a sample of binary files, the idea is to combine different deep learning models each of them processing one feature sample.
Or simply is there a way to connect a CNN and a RNN, in a way that the output of the CNN would be the input of the RNN ?
Any help or reference would be appreciated


The Keras Functional API can be used to combine different Deeplearing models.
It is much more flexible than the Keras Sequential API, in that it can support multiple input, output pipelines.
You can implement non-linear topology with the Functional API.
For example:

Related

I want to use hidden markov model for data prediction

I am new to machine learning models and data science libraries. I wanted to use the Hidden Markov model for statistical data prediction on the fly which read the data from kafka and builds the model which is used to predict the data during the run-time and do the same for continous stream always.
Currently i can see only Tensorflow hidden markov model implementation in tensorflow python (tensorflow_probability distribution). Is their any other library available which can help me acheive the above scenario
Suggestions can involve the libraries of JAVA and python
Please feel free to add any resource links that can help me to understand the usage of tensorflow for hidden markov model

this might be a nice place to start: https://hmmlearn.readthedocs.io/en/latest/tutorial.html

Other alternatives, I found, are
Java:
Mallet library and it's extention GRMM in particular.
Python:
Pommegranate with it's HMM support.
Having said that, TensorFlow is much better known active and supported library, in my impression. I'd try that first.
I'm searching a library that would support Hierarchical HMMs (HHMM). That would probably require some tweaking into one of the listed ones.

How to use google inception model to classify DNA or protein sequences data sets?

I tried to classify protein using its sequences into their families. Can I use deep convolutional models on this purpose even though they use RGB 3 input metrics of an image? Is there any specific way to convert dataset other than the image in order to classify using these models. I'm new to Artificial neural networks, your suggestions are highly appreciated.

First you need to understand that the models you have in mind are tasked with a very difficult problem: Object Recognition in colored images therefore the models used are very big.
Then you need to know the purpose of using CNNs, is to extract as many features as we can from colored images in order to perform detection.
With the knowledge above considered I think classifying protein using its sequences seems achievable with a much more smaller convolutional model. You may need at max 10 layers of convolution. To conclude you should not need a CNN as complex as google inception model.
About your data: There is no rule about CNNs which say you can only use RGB pictures. These pictures are only arrays. If you have any kind of numeric data which can be used in algorithmic operations ofcourse, you can definitely use CNNs for feature extraction. I recommend you to take a look at this example.
I also recommend you to take a look at the following libraries. SK-LEARN, KERAS and PYTORCH. These libraries are very begginer friendly and they have amazing documentaries.
Best of luck.

Which model (GPT2, BERT, XLNet and etc) would you use for a text classification task? Why?

I'm trying to train a model for a sentence classification task. The input is a sentence (a vector of integers) and the output is a label (0 or 1). I've seen some articles here and there about using Bert and GPT2 for text classification tasks. However, I'm not sure which one should I pick to start with. Which of these recent models in NLP such as original Transformer model, Bert, GPT2, XLNet would you use to start with? And why? I'd rather to implement in Tensorflow, but I'm flexible to go for PyTorch too.
Thanks!

It highly depends on your dataset and is part of the data scientist's job to find which model is more suitable for a particular task in terms of selected performance metric, training cost, model complexity etc.
When you work on the problem you will probably test all of the above models and compare them. Which one of them to choose first? Andrew Ng in "Machine Learning Yearning" suggest starting with simple model so you can quickly iterate and test your idea, data preprocessing pipeline etc.
Don’t start off trying to design and build the perfect system.
Instead, build and train a basic system quickly—perhaps in just a few
days
According to this suggestion, you can start with a simpler model such as ULMFiT as a baseline, verify your ideas and then move on to more complex models and see how they can improve your results.
Note that modern NLP models contain a large number of parameters and it is difficult to train them from scratch without a large dataset. That's why you may want to use transfer learning: you can download pre-trained model and use it as a basis and fine-tune it to your task-specific dataset to achieve better performance and reduce training time.

I agree with Max's answer, but if the constraint is to use a state of the art large pretrained model, there is a really easy way to do this. The library by HuggingFace called pytorch-transformers. Whether you chose BERT, XLNet, or whatever, they're easy to swap out. Here is a detailed tutorial on using that library for text classification.
EDIT: I just came across this repo, pytorch-transformers-classification (Apache 2.0 license), which is a tool for doing exactly what you want.

Well like others mentioned, it depends on the dataset and multiple models should be tried and best one must be chosen.
However, sharing my experience, XLNet beats all other models so far by a good margin. Hence if learning is not the objective, i would simple start with XLNET and then try a few more down the line and conclude. It just saves time in exploring.
Below repo is excellent to do all this quickly. Kudos to them.
https://github.com/microsoft/nlp-recipes
It uses hugging face transformers and makes them dead simple. 😃

I have used XLNet, BERT, and GPT2 for summarization tasks (English only). Based on my experience, GPT2 works the best among all 3 on short paragraph-size notes, while BERT performs better for longer texts (up to 2-3 pages). You can use XLNet as a benchmark.

How to combine multiple models together?

I am trying to "parallelize" Neural Network models to speed up training. One idea I had was to run two models on two computers and combine the results somehow.
Is this possible? If not, what are the options to parallelize model training on two computers?
I am open to use any neural network framework.

I think you mean distributed tensorflow?
See official document: https://www.tensorflow.org/deploy/distributed

Object-Detection-using-Fast-R-CNN python and brain_script model different?

For the toy example A2 part of the Beta 12 Release, it is said that there are two option for training:
A2_RunCntk_py3.py (python API)
A2_RunCntk.py (brain_script)
Are the models trained from these two methods the same? Or in other words, can I load the model from brain_script into python API and then detect other testing images?
Also see Object Detection using Fast R CNN.

Yes it is possible to use Python to load a model you trained with Brainscript. A few gotchas in doing this correctly are described here. We are working on making things work seamlessly without too much Python code for massaging the data.