I would like a method that could turn [1,2,3] into [1,1,2,2,3,3].
My thought is something like
val = tf.constant([1.,2.,3.]) #1,2,3
tiled = tf.tile(val, 2) # [1,2,3,1,2,3]
reshaped = tf.reshape(2,3) # [[1,2,3], [1,2,3]]
transposed = tf.transpose(reshaped) # [[1,1], [2,2], [3,3]]
flattened = tf.reshape(transposed, (6,)) # [1,1,2,2,3,3]
I haven't tested the above, but it looks like it should work. But, is there a cleaner way to do it? Mine seems ugly.
The motivation is to make some sort of GMM, where I can get a 20-dim vector that is the concatenation of two 10-dim normal distributions, each multiplied by a different random variable. So if there's a different approach for that, I'm interested as well. Thanks in advance.
One alternative without tf.transpose, basically add a second dimension to your tensor, tile by the second axis and then flatten it:
t = tf.expand_dims(val, 1)
t = tf.tile(t, (1, 2))
t = tf.reshape(t, (-1,))
t.eval()
# array([ 1., 1., 2., 2., 3., 3.], dtype=float32)
Related
From linear algebra we know that the eigenvectors of any symmetric matrix (let's call it A) are orthonormal, meaning if M is the matrix of all eigenvectors, we should obtain |det(M)| = 1. I had hoped to see this in numpy.linalg.eig, but got the following behaviour:
import numpy as np
def get_det(A, func, decimals=12):
eigenvectors = func(A)[1]
return np.round(np.absolute(np.linalg.det(eigenvectors)), decimals=decimals)
n = 100
x = np.meshgrid(* 2 * [np.linspace(0, 2 * np.pi, n)])
A = np.sin(x[0]) + np.sin(x[1])
A += A.T # This step is redundant; just to convince everyone that it's symmetric
print(get_det(A, np.linalg.eigh), get_det(A, np.linalg.eig))
Output
>>> 1.0 0.0
As you can see, numpy.linalg.eigh gives the correct result, while numpy.linalg.eig apparently returns a near non-invertible matrix. I presume that it comes from the fact that it has a degenerate eigenvalue (which is 0 in this case) and the corresponding eigenspace is not orthonormal and therefore the total determinant is not 1. In the following example, where there's (usually) no degenerate eigenvalue, the results are indeed the same:
import numpy as np
n = 100
A = np.random.randn(n, n)
A += A.T
print(get_det(A, np.linalg.eigh), get_det(A, np.linalg.eig))
Output
>>> 1.0 1.0
Now, regardless of whether my guess was correct or not (i.e. the difference between eig and eigh comes from the degeneracy of the eigenvalues), I would like to know if there is a way to get a full dimensional eigenvector matrix (i.e. the one that maximises the determinant) using numpy.linalg.eig, because I'm now working with a near-symmetric matrix, but not entirely symmetric, and it gives me a lot of problems if the eigenvector matrix is not invertible. Thank you for your help in advance!
To find the linearly independent eigenvectors, you could try Matrix.rref() which specifies the reduced row-echelon form of the matrix formed by eigenvectors.
Consider a matrix which is not diagonalizable, i.e., its eigenspace is not full rank
array([[0., 1., 0.],
[0., 0., 1.],
[0., 0., 0.]])
We can find its linearly independent eigenvectors using rref()
A = np.diag(np.ones(2), 1)
v = np.linalg.eig(A)[1]
result = sympy.Matrix(np.round(v, decimals=100)).rref()
v[:, result[1]]
which returns
array([[1.],
[0.],
[0.]])
See also: python built-in function to do matrix reduction
Find a linearly independent set of vectors that spans the same substance of R^3 as that spanned
This question already has answers here:
PyTorch memory model: "torch.from_numpy()" vs "torch.Tensor()"
(5 answers)
Closed 11 months ago.
I'm trying to have an in-depth understanding of how torch.from_numpy() works.
import numpy as np
import torch
arr = np.zeros((3, 3), dtype=np.float32)
t = torch.from_numpy(arr)
print("arr: {0}\nt: {1}\n".format(arr, t))
arr[0,0]=1
print("arr: {0}\nt: {1}\n".format(arr, t))
print("id(arr): {0}\nid(t): {1}".format(id(arr), id(t)))
The output looks like this:
arr: [[0. 0. 0.]
[0. 0. 0.]
[0. 0. 0.]]
t: tensor([[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.]])
arr: [[1. 0. 0.]
[0. 0. 0.]
[0. 0. 0.]]
t: tensor([[1., 0., 0.],
[0., 0., 0.],
[0., 0., 0.]])
id(arr): 2360964353040
id(t): 2360964352984
This is part of the doc from torch.from_numpy():
from_numpy(ndarray) -> Tensor
Creates a :class:Tensor from a :class:numpy.ndarray.
The returned tensor and :attr:ndarray share the same memory. Modifications to
the tensor will be reflected in the :attr:ndarray and vice versa. The returned
tensor is not resizable.
And this is taken from the doc of id():
Return the identity of an object.
This is guaranteed to be unique among simultaneously existing objects.
(CPython uses the object's memory address.)
So here comes the question:
Since the ndarray arr and tensor t share the same memory, why do they have different memory addresses?
Any ideas/suggestions?
Yes, t and arr are different Python objects at different regions of memory (hence different id) but both point to the same memory address which contains the data (contiguous (usually) C array).
numpy operates on this region using C code binded to Python functions, same goes for torch (but using C++). id() doesn't know anything about the memory address of data itself, only of it's "wrappers".
EDIT: When you assign b = a (assuming a is np.array), both are references to the same Python wrapper (np.ndarray). In other words they are the same object with different name.
It's just how Python's assignment works, see documentation. All of the cases below would return True as well:
import torch
import numpy as np
tensor = torch.tensor([1,2,3])
tensor2 = tensor
id(tensor) == id(tensor2)
arr = np.array([1, 2, 3, 4, 5])
arr2 = arr
id(arr) == id(arr2)
some_str = "abba"
other_str = some_str
id(some_str) == id(other_str)
value = 0
value2 = value
id(value) == id(value2)
Now, when you use torch.from_numpy on np.ndarray you have two objects of different classes (torch.Tensor and original np.ndarray). As those are of different types they can't have the same id. One could see this case as analogous to the one below:
value = 3
string_value = str(3)
id(value) == id(string_value)
Here it's intuitive both string_value and value are two different objects at different memory locations.
EDIT 2:
All in all concepts of Python object and underlying C array have to be separated. id() doesn't know about C bindings (how could it?), but it knows about memory addresses of Python structures (torch.Tensor, np.ndarray).
In case of numpy and torch.tensor you can have following situations:
separate on Python level but using same memory region for array (torch.from_numpy)
separate on Python level and underlying memory region (one torch.tensor and another np.array). Could be created by from_numpy followed by clone() or a-like deep copy operation.
same on Python level and underlying memory region (e.g. two torch.tensor objects, one referencing another as provided above)
I have a question about the way scipy builds block diagonal matrices. I was expecting that creating a sparse block diagonal matrix would be much quicker and more efficient than creating a dense one (because of sparsity compressions). But it turns out that it's not the case (or maybe am I using some inefficient method) :
from timeit import default_timer as timer
import numpy as np
from scipy.sparse import block_diag as bd_sp
from scipy.linalg import block_diag as bd_la
m = [np.identity(1)] * 10000
before = timer()
res = bd_sp(m)
timer()-before
#takes 33.79 secs
before = timer()
res = bd_la(*m)
timer()-before
#takes 0.069 secs
What am I missing? Thank's in advance for your replies.
In [625]: [np.identity(1)*i for i in range(1,5)]
Out[625]: [array([[1.]]), array([[2.]]), array([[3.]]), array([[4.]])]
In [626]: sparse.block_diag(_)
Out[626]:
<4x4 sparse matrix of type '<class 'numpy.float64'>'
with 4 stored elements in COOrdinate format>
In [627]: _.A
Out[627]:
array([[1., 0., 0., 0.],
[0., 2., 0., 0.],
[0., 0., 3., 0.],
[0., 0., 0., 4.]])
block_diag uses bmat to join the elements. bmat makes coo matrices from all elements, and combines their attributes with offsets, and makes a new coo matrix. The code is readable Python.
It may be more efficient to construct your own data, row, col arrays. block_diag is a convenience, and fine for combining a few large matrices, but not efficient when combining many small ones.
The linalg function is also Python (and pretty short). If creates an out array of the right shape, and inserts the blocks with sliced indexing. That's an efficient dense array solution. Most of the hard work is done in compiled numpy code.
Sparse matrices can be faster when doing matrix multiplication (and related linalg solvers). For most other operations, including initialization, they are slower than equivalent dense code. They are also valuable when the problem is too big.
I'm trying to store a list of variable length arrays in an HDF file with the following procedure:
phn_mfccs = []
# Import wav files
for waveform in files:
phn_mfcc = mfcc(waveform) # produces a variable length multidim array of the shape (x, 13, 1)
# Add MFCC and label to dataset
# phn_mfccs has dimension (len(files),)
# phn_mfccs[i] has variable dimension ([# of frames in ith segment] (variable), 13, 1)
phn_mfccs.append(phn_mfcc)
dt = h5py.special_dtype(vlen=np.dtype('float64'))
mfccs_out.create_dataset('phn_mfccs', data=phn_mfccs, dtype=dt)
It seems like my datatypes aren't working out though -- instead of each element of the mfccs_out dataset containing a multidimensional array, it contains just a 1D array. e.g. if the first phn_mfcc I append originally has dimension (59,13,1), mfccs_out['phn_mfccs'][0] has dimension (59,).
I suspect it is because I'm just using a float64 datatype, and I need something else for an array of arrays? If I don't specify the dataset or try to use dtype='O', though, it spits out an error like "Object dtype 'O' has no native HDF equivalent."
Ideally, what I'd like is for mfccs_out['phn_mfccs'][i] to contain the ith phn_mfcc that I appended to the list phn_mfccs.
The essence of your code is:
phn_mfccs = []
<loop several layers>
phn_mfcc = <some sort of array expanded by one dimension>
phn_mfccs.append(phn_mfcc)
At the end of loops phn_mfccs is a list of arrays. I can't tell from the code what the dtype and shape is. Or whether it differs for each element of the list.
I'm not entirely sure what create_dataset does when given a list of arrays. It may wrap it in np.array.
mfccs_out.create_dataset('phn_mfccs', data=phn_mfccs, dtype=dt)
What does np.array(phn_mfccs) produce? Shape, dtype? If all the elements are arrays of the same shape and dtype it will produce a higher dimensional array. If they differ in shape, it will produce a 1d array with object dtype. Given the error message, I suspect the latter.
I've answered a few vlen questions but haven't worked with it a lot
http://docs.h5py.org/en/latest/special.html
I vaguely recall that the 'ragged' dimension of a h5 array can only be 1d. So a phn_mfccs object array that contains 1d float arrays of varying dimensions might work.
I might come up with a simple example. And I suggest you construct a simpler problem that we can copy-n-paste and experiement with. We don't need to know how you read the data from your directory. We just need to understand the content of the array (list) that you are trying to write.
A 2015 post on vlen arrays
Inexplicable behavior when using vlen with h5py
H5PY - How to store many 2D arrays of different dimensions
1d ragged arrays example
In [24]: f = h5py.File('vlen.h5','w')
In [25]: dt = h5py.special_dtype(vlen=np.dtype('float64'))
In [26]: dataset = f.create_dataset('vlen',(4,), dtype=dt)
In [27]: dataset.value
Out[27]:
array([array([], dtype=float64), array([], dtype=float64),
array([], dtype=float64), array([], dtype=float64)], dtype=object)
In [28]: for i in range(4):
...: dataset[i]=np.arange(i+3)
In [29]: dataset.value
Out[29]:
array([array([ 0., 1., 2.]), array([ 0., 1., 2., 3.]),
array([ 0., 1., 2., 3., 4.]),
array([ 0., 1., 2., 3., 4., 5.])], dtype=object)
If I try to write 2d arrays to dataset I get an error
OSError: Can't prepare for writing data (Src and dest data spaces have different sizes)
The dataset itself may be multidimensional, but the vlen object has to be a 1d array of floats.
I have around 200 3D points that I need to multiply into a rather complex 2D projection matrix. I am currently using numpy and a for loop, essentially iterating through 3D point cloud, applying the matrix transformations and getting my data.
This seems rather slow. Is there any way I might be able to vectorize this, or use some kind of speed up techniques (maps, pools etc.)
F = matrix([
[735.4809, 0., 388.9476, 0.],
[0., 733.6047, 292.0895, 0.],
[0., 0., 1.0000, 0.]
])
VehicleRPY = self.GetRT(Roll=Roll, Pitch=Pitch, Yaw=0., X=IMUX, Y=IMUY, Z=IMUZ);
SonarToCamera = self.GetRT(Roll=RTRoll, Pitch=RTPitch, Yaw=RTYaw, X=RTX, Y=RTY, Z=RTZ);
SpaceMatrix = matrix([
[(sqrt(R**2 - (Y/(cosd(Roll)*cosd(Pitch)))**2)*sind(Theta))],
[(Y/(cosd(Roll)*cosd(Pitch)))],
[(sqrt(R**2 - (Y/(cosd(Roll)*cosd(Pitch)))**2)*cosd(Theta))],
[1]
])
FinalMatrix = F*VehicleRPY*SonarToCamera*SpaceMatrix;
UVMatrix = matrix([
[FinalMatrix.item(0)/FinalMatrix.item(2)],
[FinalMatrix.item(1)/FinalMatrix.item(2)],
])
Something like the above. I need to repeat this 3*3/4*4 multiplication across 200 points per frame