Why the inverse of the inverse of one matrix is not itself in python?
Why the inverse of the inverse of one matrix is not itself in python?
code
code
(Deleted the previous answer, since I made a mistake copying the matrix)
Your matrix is perfectly singular, so the inverse does not actually exist. Due to limits of numerical precision, numpy.linalg.inv gives you a matrix with very large values that is the inverse of another (similar) matrix.
I don't know what Matlab does in this situation, but it's possible that it gives the Moore-Penrose pseudoinverse, which would behave as you describe. Note however that this is not the same thing as the inverse, which does not exist. In numpy, you can get the Moore-Penrose pseudoinverse as np.linalg.pinv.
Related
How do you write a matrix multiplication function? Takes two matrices outputs one.
The documentation on assemblyscript.org is pretty short, Float64Array though is a valid type among these but that's 1D so...
AssemblyScript's stdlib is modeled after JavaScript's stdlib, so there are no matrix operations. However, here is a library that might work for you: https://github.com/JustinParratt/big-mult
My questions are specific to https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.PCA.html#sklearn.decomposition.PCA.
I don't understand why you square eigenvalues
https://github.com/scikit-learn/scikit-learn/blob/55bf5d9/sklearn/decomposition/pca.py#L444
here?
Also, explained_variance is not computed for new transformed data other than original data used to compute eigen-vectors. Is that not normally done?
pca = PCA(n_components=2, svd_solver='full')
pca.fit(X)
pca.transform(Y)
In this case, won't you separately calculate explained variance for data Y as well. For that purpose, I think we would have to use point 3 instead of using eigen-values.
Explained variance can be also computed by taking the variance of each axis in the transformed space and dividing by the total variance. Any reason that is not done here?
Answers to your questions:
1) The square roots of the eigenvalues of the scatter matrix (e.g. XX.T) are the singular values of X (see here: https://math.stackexchange.com/a/3871/536826). So you square them. Important: the initial matrix X should be centered (data has been preprocessed to have zero mean) in order for the above to hold.
2) Yes this is the way to go. explained_variance is computed based on the singular values. See point 1.
3) It's the same but in the case you describe you HAVE to project the data and then do additional computations. No need for that if you just compute it using the eigenvalues / singular values (see point 1 again for the connection between these two).
Finally, keep in mind that not everyone really wants to project the data. Someone can only get the eigenvalues and then immediately estimate the explained variance WITHOUT projecting the data. So that's the best gold standard way to do it.
EDIT 1:
Answer to edited Point 2
No. PCA is an unsupervised method. It only transforms the X data not the Y (labels).
Again, the explained variance can be computed fast, easily, and with half line of code using the eigenvalues/singular values OR as you said using the projected data e.g. estimating the covariance of the projected data, then variances of PCs will be in the diagonal.
I am using sparse matrices in python, namely
scipy.sparse.csr_matrix
I am in principle free to choose the exact sparse implementation, as long as the matrices support matrix-vector multiplication and addition/subtraction of matrices with the same sparsity pattern. Currently, at every time step, I construct a new sparse matrix from scratch and add it to the existing matrix. I believe that my code could be unnecessarily losing time due to
Construction time of sparse matrix
Addition of the sparse matrices, assuming that the underlying algorithm inside CSR matrix implementation has to find matching sparse entries before adding them up.
My guess would be that the sparse matrix is internally stored as a numpy array of values + a few index arrays denoting where those values are located. The question is if it is possible to directly add the underlying value arrays without touching the sparsity structure. Is something like this possible?
new_values = np.linspace(0, num_values)
csr_mat.val += new_values
Given two matrices, A and B, where B is symetric (and positive semi-definite), What is the best (fastest) way to calculate A`*B*A?
Currently, using BLAS, I first compute C=B*A using dsymm (introducing a temporary matrix C) and then A`*C using dgemm.
Is there a better (faster, no temporaries) way to do this using BLAS and mkl?
Thanks.
I'll offer somekind of answer: Compared to the general case A*B*C you know that the end result is symmetric matrix. After computing C=B*A with BLAS subroutine dsymm, you want to compute A'C, but you only need to compute the upper diagonal part of the matrix and the copy the strictly upper diagonal part to the lower diagonal part.
Unfortunately there doesn't seem to be a BLAS routine where you can claim beforehand that given two general matrices, the output matrix will be symmetric. I'm not sure if it would be beneficial to write you own function for this. This probably depends on the size of your matrices and the implementation.
EDIT:
This idea seems to be addressed recently here: A Matrix Multiplication Routine that Updates Only the Upper or Lower Triangular Part of the Result Matrix
I want to detect the best rototraslation matrix between two set of points.
The second set of points is the same of the first, but rotated, traslated and affecteb by noise.
I tried to use least squared method by obviously the solution is usually similar to a rotation matrix, but with incompatible structure (for example, where i should get a value that represents the cosine of an angle i could get a value >1).
I've searched for the Constrained Least Squared method but it seems to me that the constrains of a rototraslation matrix cannot be expressed in this form.
In this PDF i've stated the problem more formally:
http://dl.dropbox.com/u/3185608/minquad_en.pdf
Thank you for the help.
The short answer: What you will need here is "Principal Component Analysis".
Apply this to both sets of points centered at their respective centers of mass. The PCA will effectively give you a rotation matrix for each aligned to the data set principal components. Multiplying the inverse matrix of the original set by the new rotation will give you a matrix that takes the old (centered) set to the new. Inverse translations and translations can similarly be applied to the rotation to create a homogeneous matrix that maps the one set to the other.
The book PRINCE, Simon JD. Computer vision: models, learning, and inference. Cambridge University Press, 2012.
gives, in Appendix "B.4 Reparameterization", some info about how to constrain a matrix to be a rotation matrix.
It seems to me that your problem has also a solution based on SVD: see the Kabsch algorithm also described by Olga Sorkine-Hornung and Michael Rabinovich in
Least-Squares Rigid Motion Using SVD and, more practically, by Nghia Kien Ho in FINDING OPTIMAL ROTATION AND TRANSLATION BETWEEN CORRESPONDING 3D POINTS.