I am having problems not on understanding the k means algorithm but on how to apply it on training ,validation and testing data.Is it like this:
Training phase: Apply k-means on the input data and then we get centroid value (in my case three).For each centroid value assign a label say 1,2,3.Suppose in training phase I input sixty such samples .So in total i get 60*3 centroids each with label 1,2,3..
Testing phase:Apply k means on the input signal.We get centroids.Compare this with centroid obtained from training phase centroids.Which ever is closest to it assign the same label for it?
k-means does not have a "training" and a "testing" phase. It is an unsupervised algorithm.
At most, it is only applied to testing data.
Do not approach it like a classificator. It is not a classification algorithm.
The objective of k-means is:
Split my input data set into k convex partitions, such that the sum of all squared deviations in all dimensions from the mean of each partition is smallest.
There are no labels.
Related
If my latent representation of variational autoencoder(VAE) is r, and my dataset is x, does vae's latent representation follows normalization based on r or x?
If r= 10, that means it has 10 means and variance (multi-gussain) and distribution comes from data whole data x?
Or r = 10 constructs one distribution based on r, and every sample try to follow this distribution
I'm confused about which one is correct
VAE constructs a mapping e(x) -> Z (encoder), and d(z) -> X (decoder). This means that every elements of your input space x will be mapped through an encoder e(x) into a single, r-dimensional Gaussian. It is not a "mixture", it is just a single gaussian with diagonal covariance matrix.
I'll add my 2 cents to #lejlot answer.
Your encoder in VAE will map your sample to a distribution, that in your case has 10 dimensions... that distribution is used to say "ok my best estimate of this property of this sample is mu, but I'm not too sure, so consider that it might have variance sigma"
Therefore, you have a distribution for each sample.
However, in order to make sampling easier in VAE, we ask the VAE to keep the distributions as close to a known one, that is the standard normal distribution (we know "where the distributions are located", if you check the latent space in a normal AE you will see that you will have groups far from eachother).
I am no expert in this field but more of a beginner with a bit of experience, so please keep the answer as simple as possible.
I cannot be very specific about this topic but what I am trying to do is predict the velocity of multiple objects(that should have a pattern because they are similar). I am taking the optical flow from every tenth frame and building a histogram of every tenth frame(x and y velocities separate). Then, I convert these histograms to vectors and store them in a CSV file. I am trying to use these vectors in an LSTM for Timeseries forecast. I do not know how to input each x and y velocity vector as a time step to output the next x and y velocities every (let's say) 5 steps.
The tutorials I see are usually about predicting temperature, and the input values(not vectors but single values) of humidity, precipitation, etc. and then output a single value(being the temperature)
please help, I hope I made it relatively clear.
Maybe there is a better approach.
I'm using Tensorflow (2.4) and Keras to build my neural network model. It takes two tensors as inputs and gives a scalar output. The network is already trained and, from now on, it has fixed weights. It is possible, given one of the two inputs, to find the value of the other input that maximise the output value?
Thank you in advance
In theory, yes.
Lets call your network model f. It takes two inputs x and y and outputs f(x, y). Then, assuming x and f are fixed, you can find the value y* that maximize f(x, y) as follows:
calculate the gradient of f with respect to y. Then, there are two possibilities.
there exists stationary points. Just set df/dy = 0 and solve for y. This gives the y* at which there is either a maximum or a minimum. Compute f(x, y*) to check weather y* gives a maximum or a minimum.
there are no stationary points (or there is no maximum). Here, you need to study where f decreases or increases if y varies. To do this, look for df/dy > 0 (increases) and df/dy < 0 (decreases). You will find that, asymptotically, the function increases. Simply take y*=a, where a is the closest value to such asymptote that you can take (given your data type precision).
I've reading recently on Expectation Maximization (EM) and it keeps coming up that Initializing EM using K-Means is a good idea but i'm having difficulties in grasping this notion.
So as far as i know when using kmeans, the result you get is coordinates of the clusters' centroids according to the pre-defined numberof clusters, so how can this be used in order to initialize EM. To make things clearer this is the problem i'm currently trying to solve:
I have a dataset of noisy data points Y who originates from Samples X taken from an 8-ASK set. Now i loaded my dataset and have used a kmeans algorithm in order to identify the centroids but can't seem to know what's the next step. The EM algorithm that i use requires the parameters: the initial start values for the centroids and their probability distribution as well as the initial mean and variance but i do not understand how can get those exactly.
To summarize my question is basically how can i calculate the mean, variance and initial diribution of the centroids generated by kmeans algorithm when i ran him on my data Y ?
I'm trying to use XGBoost to predict the rank for a set of features for a given query. I managed to train a model it but I'm confused around the input data when I ask for a prediction.
I'm trying to understand if I'm doing something wrong or this is not the right approach.
What I'm doing:
I parse the training data (see here a sample) and feed it in a DMatrix such that the first column represents the quality-of-the-match and the following columns are the scores on different properties and also send the docIds as labels
I configure the group sizes
The training seems to work fine, I get not errors, and I use the rank:pairwise objective
For prediction, I use a fake entry with fake scores (1 row, 2 columns see here) and I get back a single float value.
I'm trying to understand:
1. Do I need to feed in a label for the prediction ?
My understanding is that labels are similar to "doc ids" so at prediction time I don't see why I need them
2. Do I need to set the group size when doing predictions ? And if so, what does it represent ?
My understanding is that groups are for training data to assist ranking "per query". How does that correlate with predictions? Do I set a group size anyway?
3. How do I correlate the "group" from the training with the prediction?
How do I figure out the pair (score, group) from the result of the prediction, given I only get back a single float value - what group is that prediction for?