i'm looking for 4 best algorithms in neural network optimization. i need the names and any description ,learning method and topology is very helping me.
i think multilayer perceptron (MLP) is one of them but i'm not sure.
thank you all.
Your question is really wide. It's hard to answer because different optimizers have their pros and drawbacks. I would say that among the best today's optimizers you will find ADAM, RMSPROP, ADAGRAD and SGD with Nestov momentum. You can read about these methods (as well as others) here.
First of all, I agree that the question is indeed too broad, however, I would like to add some resorces I personaly found to be usefull in helping me answer this question. In doing so, I hope that this resources will help someone else in as much as they helped me. Here they are:
Sebastian Ruder Mod Yu Zeng's blog post.
Anish Singh Walia's discussion on this topic.
Chapter 8 of The deep learning book by an Goodfellow, Yoshua Bengio and Aaron Courville.
This post from the Morning paper by Adrian Colyer.
I hope this helps someone.
Related
I was studying for runtime analysis.
One of the question that I encountered is this.
Which function would grow the slowest?
n^0.5
log(n^0.5)
(log(n))^0.5
log(n) * log(n)
I thought the answer was 2 after I draw the graph.
However, the answer is 3. I am not sure why this is the answer.
Could someone explain why this is the case?
Thank you
2.log(n^0.5)=0.5log(n)=log(n)
3.(log(n))^0.5= sqrt(log(n)) which is smaller in magnitude than 2.
I have explained in very layman terms in the picture below with a little mathematics.
Tell me if you still have confusion.
PS: Both are log so I don't think we worry about time complexity of such functions in this modern age.
I am a beginner in machine learning. I want to build a model for finding trending feeds like Instagram.
Please suggest which model is recommended for the same.
I will suggest you to choose these modeling frameworks like Modeling choices, Data freshness trading, and Novelty effect, Experimentation (A/B) Small Effects, Impact, and Scientific Method, Normalization, Iteration Speed — Offline Analysis, Value Modeling, and Parting Thoughts.
Moreover, since you are a beginner, you can get expert guidance for machine learning related questions at Mayazbridge.com. Mayazbridge is the software training institution in kukatpally giving postgraduate courses with career guidance. Hope my answer helps you.
The very recent paper on "Mask R-CNN" is really interesting. Is there somebody in the CNTK community who might implement it in CNTK and share?
We are not aware of a CNTK implementation. We will add this to our backlog. We always welcome community contributions.
Recently I stared toying with tensor flow, dnns etc. now I'm trying to implement something more serious, information retrieval from short sentences (doctor instructions).
Unfortunately the dataset I have is, as always, quite "dirty". As I'm trying to use word embeddings, I actually need "clean" data. Take one example:
"Take two pilleach day". There is a missing white space between pill and each. I am implementing "tokenizer improver" to look at each sentence and propose new tokenization based on joint probability of each word in sentence given the frequency of terms in whole document (tf) . As I was doing it today, a thought came to my mind: why bother writing suboptimal solution for this problem when I can employ powerful learning algorithms such as Lstm networks to do that for me. However, as of today, I have only a feeling that it's actually possible to do that. As we know, feelings are not best when it comes to architecting such complex problems. I don't know where to begin: what should be my training set and learning goal.
I know this is a broad question, but I know there are many brilliant people with more knowledge about tensorflow and neural nets, so I'm sure that somebody has either already solved similar problem or just knows how to approach this problem.
Any guidance is welcome, I do not except you to solve this for me of course:)
Besos and all the best to all the tensorflow community:)
Having the same issue. I solved it by using a character level net. Basically I rewrote Character-Aware Neural Language Models, kicked out the whole "words"-elements and just stayed with the caracter level.
Training Data: I took the data I had, as dirty as it was, used the dirty data as targets and made it even more dirty to create inputs.
So your "Take two pilleach day" will be learned as in many cases you do have a clean and similar phrase, e.g. "Take one pill each morning" that with the regime mentioned will serve as target and you train the net on destroyed inputs like "Take oe pileach mornin"
I'd be very much interested hearing any tips on how
a simple algorithm can be implemented to overcome
lost packets of a binary stream.
I was considering doing something like
ABCDE,BCDEF,CDEFG,DEFGH,.....
or may be with 3 bytes instead of 5, i.e.
ABC,BCE,CDE,DEF,...
Though I'm not too sure of this.
I understand the basic principles behind
error correcting theory, but it doesn't
help a lot .. any suggestions ?
I think I have found what I was looking for - it's the Convolutional
Coding that need to use in this sort of application. It looks like
the most suitable coding technique for a low-power application.
For those who interested in updates on this, I'll write more on my
project's wiki
There is an article on Wikipedia whcih has a nice diagram.