This question is of a more conceptual type.
I was using the pre-trained word-vectors of spacy (the de_core_news_md model).
The problem is that I have a lot of domain specific words which all get a 0-vector assignet and overall the results are in gerneral not too good.
I was wondering how one should proceed now.
should I try to fine tune the existing vectors? If so, how would one approach that?
Or, should I just not use the pre-trained word vectors of spacy and create my own?
Edit:
I want to fine tune the pre trained vectors. I've read, that I could train the already trained model again but on my data. Now my question is, how to do that. When I use spacy, i just load the model. Should I download the vectors of spacy and train a gensim model with them and afterwards again with my vectors? Or is there a better way?
Thank you in advance for any input!
Related
Am a complete noob and I just want to ask, do we need to preprocess images (like manually) before feeding the image into CNN for training? I've read that CNN already has some filtering techniques to extract features and such. I'm thinking what if all the train images are binary images or even just edges (teaching the model for shapes), is it advisable or I'll just feed grayscale images? Additionally, if the answer is yes, may I know what kind of preparation is done normally or what you would use?
I am aware of the other preprocessing techniques by Keras, such as the VGG16, but I would like something simple and manual.
The preprocessing functions of Keras only preprocess the input according to the state-of-the-art models requirements, e.g., changing the data format etc. May be, what you are talking about is hand-engineered features, which is not simple. And I think, it's not advisable because CNN does a better job. But it may also depend on what you actually want to do.
Is it possible to possible to use a pretrained weight of a large model (for example ResNet151) and cut a part out of it to fit a custom, rather small model?
If it is the case, how can it be managed?
Thanks for any ideas!
I am predicting similarities of documents using the pre trained spacy word embeddings. Because I have a lot of domain specific words, I want to fine tune my vectors on a rather small data set containing my domain specific vocabulary.
My idea was to just train the spacy model again with my data. But since the word vectors in spacy are built-in, I am not sure how to do that. Is there a way to train the spacy model again with my data?
After some research, I found out, that I can train my own vectors using Gensim. There I would have to download a pre trained model for example the Google News dataset model and afterwards I could train it again with my data set. Is this the only way? Or is there a way to proceed with my spacy model?
Any help is greatly appreciated.
update: the right term here was "incremental training" and thats not possible with the pre-trained spacy models.
It is however possible, to perform incremental training on a gensim model. I did that with the help of another pretrained vector set (i went with the fasttext model) and then I trained this gensim model trained with the fasttext vectors again with my own corpus. This worked pretty well
If you pre-trained word embeddings with fasttext in your domain and would like to use them with spaCy you can extend/replace the tokens from an existing spaCy model with your new fasttext vocabulary&vectors using something similar to this:
https://github.com/explosion/spaCy/issues/2538#issuecomment-404888091
or from scratch:
https://spacy.io/usage/vectors-similarity#converting
The advantage of this approach is that (1) you can keep using spacy and (2) if some tokens were present in the pre-trained spaCy but not in your corpus you will still be able to use them
from Tensorflow's documentation, there seems to be a large array of options for "running", serving, testing, and predicting using a Tensorflow model. I've made a model very similar to MNIST, where it outputs a distribution from an image. For a beginner, what would be the easiest way to take one or a few images, and send them through the model, getting an output prediction? It is mostly for experimentation purposes. Sorry if this is too redundant, but all my research has led me to so many different ways of doing this and the documentation doesn't really give any info on the pros and cons of the different methods. Thanks
I guess you are using placeholders for your model input and then using feed_dict to feed values into your model.
If that's the case the simplest way would be after you have a trained model you save it using tf.saver. Then you can have a test script where you restore your model and then sess.run on your output variable with a feed_dict of whatever you want your input to be.
I have a Torch Model which is trained on a large scale dataset (Places Dataset) and it's authors uploaded it on github, i am working on a similar project and i want to make use of it and use it's trained weights instead of use the large dataset to train it and save time and efforts, it is possible ? how can i know the only the trained filters weights? i don't want to copy the code, i only want to make use of it and save time and efforts.
NOTE: I use Tensoflow in my implementation.