I'm building a tabular model with multiple facts which are not related but using the same dimensions and exposing the model to power bi. The problem is after adding the second fact to the model, I'm not getting the correct values from the 1st fact.
Related
I want to predict stock by analyzing stock mouvement.
Stock mouvement
enter image description here
STOCK :
enter image description here
Which step to start analzying data using machine learning.
Which algorithme ML and DL to use.
Thanks a lot
I need :
learn step to start analzying data using machine learning and deep learning.
Type of algorithme ML and DL to use.
Mohamed:
Your question requires a long and broad answer. I'll try to provide some steps and reference for you to explore further.
First of all you need the right data. The prediction will be as good as your data. So you need economic data, market trends, and company-specific events.
Then you need to follow the standard steps:
Collect and Preprocess Data: You'll need to gather stock data such as daily closing prices, trading volumes, and other relevant financial information. Preprocessing the data involves cleaning and transforming the data so that it can be used for modeling.
Feature Engineering: This involves creating new features from the existing data that can be used as inputs for the ML/DL models. For example, you can calculate technical indicators such as moving averages, relative strength index (RSI), and Bollinger Bands.
Split the Data: You'll need to split the data into training, validation, and testing sets. The training set is used to train the ML/DL models, the validation set is used to fine-tune the models, and the testing set is used to evaluate the performance of the models.
Select an Algorithm: There are various ML algorithms that can be used for stock price prediction. Some popular algorithms include:
ML Algorithms: Linear Regression, Random Forest, XGBoost, Support Vector Machines (SVM), etc.
Hope that helps.
Reference Book: https://learning.oreilly.com/api/v1/continue/9781800560796/
Machine Learning Engineering with MLflow - Chapter 2
I would like to train a reinforcement learning model on multiple different time series datasets.
Each dataset is independent of one another and have additional attributes, such as location, weather, temperature, hardware etc.
I was wondering whether somebody could tell me the best way to add this information to my dataset. Do I just make a feature for each attribute and duplicate the data for each tick in the time series or is there a more effective efficient way of doing this?
I believe I will have to train/fit the model in batches, whereby each batch is a new/different dataset, but happy to be corrected on that.
Many thanks in advance.
I am training existing standard model for named entity recognition with custom data.After training the model,It is forgetting old entity which was already there in the standard model.I want to merge the custom ner data with the standard model ner data. For example,It should recognize my custom data(e.g-Product,Family) tag along with the entity of Standard model(e.g- Person,ORG,GPE).Also how should I improve reliability of the model with less dataset.Model is predicting correctly for the trained data but failing for the completely new data. I want to leave entity as blank if test data is not there in the Model,but it is predicting wrongly with some incorrect tag. Suggest me some solution for this.I am using the code that is given on the official site of spacy ner.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
I trained doc2vec model in TensorFlow. So now I have embeded vectors for words in dictionary and vectors for the documents.
In the paper
"Distributed Representations of Sentences and Documents"
Quoc Le, Tomas Mikolov
authors write
“the inference stage” to get paragraph vectors D for new paragraphs
(never seen before) by adding more columns in D and gradient
descending on D while holding W,U,b fixed.
I have pretrained model so we have W, U and b as graph variables. Question is how to implement inference of D(new document) efficiently in Tensorflow?
For most neural networks, the output of the network (class for classification problems, number for regression,...) if the value you are interested in. In those cases, inference means running the frozen network on some new data (forward propagation) to compute the desired output.
For those cases, several strategies can be used to deliver quickly the desired output for multiple new data points : scaling horizontally, reduce the complexity of calculation through quantisation of the weights, optimising the freezed graph computation (see https://devblogs.nvidia.com/tensorrt-3-faster-tensorflow-inference/),...
doc2Vec (and word2vec) are different use case is however different : the neural net is used to compute an output (prediction of the next word), but the meaningful and useful data are the weights used in the neural network after training. The inference stage is therefore different : you do not want to get the output of the neural net to get a vector representation of a new document, you need to train the part of the neural net that provides you the vector representation of your document. Part of the neural net is then frozen (W,U,b).
How can you efficiently compute D (document vector) in Tensorflow :
Make experiments to define the optimal learning rate (a smaller value might be a better fit for shorter document) as it defines how quick your neural network representation of a document.
As the other part of the neural net are frozen, you can scale the inference on multiple processes / machines
Identify the bottle necks : what is currently slow ? model computation ? Text retrieval from disk of from external data source ? Storage of the results ?
Knowing more about your current issues, and the context might help.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 2 years ago.
Improve this question
Thanks to Google for providing a few pre-trained models with tensorflow API.
I would like to know how to retrain a pre-trained model available from the above repository, by adding new classes to the model.
For example, the trained COCO dataset model has 90 classes, I would like to add 1 or 2 classes to the existing one and get one 92 class object detection model as a result.
Running Locally is provided by the repository but it is completely replacing those pre-trained classes with newly trained classes. There, only train and eval are mentioned.
So, is there any other way to retrain the model and get 92 classes as a result?
Question : How do we add a few more classes to my already trained network?
Specifically, we want to keep all the network as-is other than the output of the new classes. This means that for something like ResNet, we want to keep everything other than the last layer frozen, and somehow expand the last layer to have our new classes.
Answer : Combine the existing last layer with a new one you train
Specifically, we will replace the last layer with a fully connected layer that is large enough for your new classes and the old ones. Initialize it with random weights and then train it on your classes and just a few of the others. After training, copy the original weights of the original last fully connected layer into your new trained fully connected layer.
If, for example, the previous last layer was a 1024x90 matrix, and your new last layer is a 1024x92 matrix, copy the 1024x90 into the corresponding space in your new 1024x92. This will destructively replace all your training of the old classes with the pre-trained values but leave your training of your new classes. That is good, because you probably didn't train it with the same number of old classes. Do the same thing with the bias, if any.
Your final network will have only 1024x2 new weight values (plus any bias), corresponding to your new classes.
A word of caution, although this will train fast and provide quick results, it will not perform as well as retraining on a full and comprehensive data set.
That said, it'll still work well ;)
Here is a reference to how to replace the last layer How to remove the last layer from trained model in Tensorflow that someone else answered