tf.sparse_to_dense() fucntion in tensorflow only support ((data, (row_ind, col_ind)), [shape=(M, N)]) format. How can I convert standard CSR tensor (((data, indices, indptr), [shape=(M, N)])) to dense representation in tensorflow?
For example given, data, indices and indptr the function will return dense tensor.
e.g., inputs:
indices = [1 3 3 0 1 2 2 3]
indptr = [0 2 3 6 8]
data = [2 4 1 3 2 1 1 5]
expected output:
[[0, 2, 0, 4],
[0, 0, 0, 1],
[3, 2, 1, 0],
[0, 0, 1, 5]]
According to Scipy documentation, we can convert it back by the following:
the column indices for row i are stored in indices[indptr[i]:indptr[i+1]] and their
corresponding values are stored in data[indptr[i]:indptr[i+1]].
If the shape parameter is not supplied, the matrix dimensions are
inferred from the index arrays.
It is relatively easily to convert from the CSR format to the COO by expanding the indptr argument to get the row indices. Here is an example using a subtraction, tf.repeat and tf.range. The shape of the final sparse tensor is inferred from the max indices in the rows/columns respectively (but can also be provided explicitly).
def csr_to_sparse(data, indices, indptr, dense_shape=None):
rep = tf.math.subtract(indptr[1:], indptr[:-1])
row_indices = tf.repeat(tf.range(tf.size(rep)), rep)
sparse_indices = tf.cast(tf.stack((row_indices, indices), axis=-1), tf.int64)
if dense_shape is None:
max_row = tf.math.reduce_max(row_indices)
max_col = tf.math.reduce_max(indices)
dense_shape = (max_row + 1, max_col + 1)
return tf.SparseTensor(indices=sparse_indices, values=data, dense_shape=dense_shape)
With your example:
>>> indices = [1, 3, 3, 0, 1, 2, 2, 3]
>>> indptr = [0, 2, 3, 6, 8,]
>>> data = [2, 4, 1, 3, 2, 1, 1, 5]
>>> tf.sparse.to_dense(csr_to_sparse(data, indices, indptr))
<tf.Tensor: shape=(4, 4), dtype=int32, numpy=
array([[0, 2, 0, 4],
[0, 0, 0, 1],
[3, 2, 1, 0],
[0, 0, 1, 5]], dtype=int32)>
Related
Let's say i have the following array:
import numpy as np
a = np.array([[1, 2, 3], [0, 1, 2], [1, 3, 4], [4, 5, 6]])
a = sp_sparse.csr_matrix(a)
and I want to get a submatrix of the sparse array that consists of the first and last rows.
>>>sub_matrix = a[[0, 3], :]
>>>print(sub_matrix)
(0, 0) 1
(0, 1) 2
(0, 2) 3
(1, 0) 4
(1, 1) 5
(1, 2) 6
But I want to keep the original indexing for the selected rows, so for my example, it would be something like:
(0, 0) 1
(0, 1) 2
(0, 2) 3
(3, 0) 4
(3, 1) 5
(3, 2) 6
I know I could do this by setting all the other rows of the dense array to zero and then computing the sparse array again but I want to know if there is a better way to achieve this.
Any help would be appreciated!
Depending on the indexing, it might be easier to construct the extractor/indexing matrix with the coo style of inputs:
In [129]: from scipy import sparse
In [130]: M = sparse.csr_matrix(np.arange(16).reshape(4,4))
In [131]: M
Out[131]:
<4x4 sparse matrix of type '<class 'numpy.int64'>'
with 15 stored elements in Compressed Sparse Row format>
In [132]: M.A
Out[132]:
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11],
[12, 13, 14, 15]])
A square extractor matrix with the desired "diagonal" values:
In [133]: extractor = sparse.csr_matrix(([1,1],([0,3],[0,3])))
In [134]: extractor
Out[134]:
<4x4 sparse matrix of type '<class 'numpy.int64'>'
with 2 stored elements in Compressed Sparse Row format>
Matrix multiplication in one direction selects columns:
In [135]: M#extractor
Out[135]:
<4x4 sparse matrix of type '<class 'numpy.int64'>'
with 7 stored elements in Compressed Sparse Row format>
In [136]: _.A
Out[136]:
array([[ 0, 0, 0, 3],
[ 4, 0, 0, 7],
[ 8, 0, 0, 11],
[12, 0, 0, 15]])
and in the other, rows:
In [137]: extractor#M
Out[137]:
<4x4 sparse matrix of type '<class 'numpy.int64'>'
with 7 stored elements in Compressed Sparse Row format>
In [138]: _.A
Out[138]:
array([[ 0, 1, 2, 3],
[ 0, 0, 0, 0],
[ 0, 0, 0, 0],
[12, 13, 14, 15]])
In [139]: extractor.A
Out[139]:
array([[1, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 1]])
M[[0,3],:] does the same thing, but with:
In [140]: extractor = sparse.csr_matrix(([1,1],([0,1],[0,3])))
In [142]: (extractor#M).A
Out[142]:
array([[ 0, 1, 2, 3],
[12, 13, 14, 15]])
Row and column sums are also performed with matrix multiplication:
In [149]: M#np.ones(4,int)
Out[149]: array([ 6, 22, 38, 54])
import numpy as np
import scipy.sparse as sp_sparse
a = np.array([[1, 2, 3], [0, 1, 2], [1, 3, 4], [4, 5, 6]])
a = sp_sparse.csr_matrix(a)
It's probably easiest to just use a selection matrix and then multiply.
idx = np.isin(np.arange(a.shape[0]), [0, 3]).astype(int)
b = sp_sparse.diags(idx, format='csr') # a
The disadvantage is that this will result in an array of floats instead of integers, but that's easy enough to fix.
>>> b.astype(int).A
array([[1, 2, 3],
[0, 0, 0],
[0, 0, 0],
[4, 5, 6]])
When I updated to the most recent version of numpy, a lot of my code broke because now every time I call np.dot() on a matrix and an array, it returns a 1xn matrix rather than simply an array.
This causes me an error when I try to multiply the new vector/array by a matrix
example
A = np.matrix( [ [4, 1, 0, 0], [1, 5, 1, 0], [0, 1, 6, 1], [1, 0, 1, 4] ] )
x = np.array([0, 0, 0, 0])
print(x)
x1 = np.dot(A, x)
print(x1)
x2 = np.dot(A, x1)
print(x2)
output:
[0 0 0 0]
[[0 0 0 0]]
Traceback (most recent call last):
File "review.py", line 13, in <module>
x2 = np.dot(A, x1)
ValueError: shapes (4,4) and (1,4) not aligned: 4 (dim 1) != 1 (dim 0)
I would expect that either dot of a matrix and vector would return a vector, or dot of a matrix and 1xn matrix would work as expected.
Using the transpose of x doesn't fix this, nor does using A # x, or A.dot(x) or any variation of np.matmul(A, x)
Your arrays:
In [24]: A = np.matrix( [ [4, 1, 0, 0], [1, 5, 1, 0], [0, 1, 6, 1], [1, 0, 1, 4] ] )
...: x = np.array([0, 0, 0, 0])
In [25]: A.shape
Out[25]: (4, 4)
In [26]: x.shape
Out[26]: (4,)
The dot:
In [27]: np.dot(A,x)
Out[27]: matrix([[0, 0, 0, 0]]) # (1,4) shape
Let's try the same, but with a ndarray version of A:
In [30]: A.A
Out[30]:
array([[4, 1, 0, 0],
[1, 5, 1, 0],
[0, 1, 6, 1],
[1, 0, 1, 4]])
In [31]: np.dot(A.A, x)
Out[31]: array([0, 0, 0, 0])
The result is (4,) shape. That makes sense: (4,4) dot (4,) => (4,)
np.dot(A,x) is doing the same calculation, but returning a np.matrix. That by definition is a 2d array, so the (4,) is expanded to (1,4).
I don't have an older version to test this on, and am not aware of any changes.
If x is a (4,1) matrix, then the result (4,4)dot(4,1)=>(4,1):
In [33]: np.matrix(x)
Out[33]: matrix([[0, 0, 0, 0]])
In [34]: np.dot(A, np.matrix(x).T)
Out[34]:
matrix([[0],
[0],
[0],
[0]])
I have tensor T of shape [batch_size, A] with values and tensor S of shape [batch_size] with shift parameters.
I would like to shift values in T[b] by S[b] positions to the right, the last S[b] elements of T[b] should be dropped and new elements should be set to 0.
So basically want to do something like:
for i in range(batch_size):
T[i] = zeros[:S[i]] + T[i, :A-S[i]]
Example:
For:
T = [[1, 2, 3], [4, 5, 6]]
S = [1, 2]
Return:
T' = [[0, 1, 2], [0, 0, 4]]
Is there some easy way to do it?
You can use tf.concat and tf.stack for that purpose:
T_shift = tf.zeros((batch_size, A), tf.float32)
tmp = []
for i in xrange(batch_size):
tmp.append(tf.concat([T_shift[i, :S[i, 0]],T[i, :17 - S[i,0]]], axis = 0))
T_shift = tf.stack(tmp)
If you are working in Tensorflow 2, you can use the tf.roll for that purpose:
"The elements are shifted positively (towards larger indices) by the
offset of shift along the dimension of axis. Negative shift values
will shift elements in the opposite direction. Elements that roll
passed the last position will wrap around to the first and vice versa.
Multiple shifts along multiple axes may be specified."
tf.roll(
input, shift, axis, name=None
)
# 't' is [0, 1, 2, 3, 4]
roll(t, shift=2, axis=0) ==> [3, 4, 0, 1, 2]
# shifting along multiple dimensions
# 't' is [[0, 1, 2, 3, 4], [5, 6, 7, 8, 9]]
roll(t, shift=[1, -2], axis=[0, 1]) ==> [[7, 8, 9, 5, 6], [2, 3, 4, 0, 1]]
# shifting along the same axis multiple times
# 't' is [[0, 1, 2, 3, 4], [5, 6, 7, 8, 9]]
roll(t, shift=[2, -3], axis=[1, 1]) ==> [[1, 2, 3, 4, 0], [6, 7, 8, 9, 5]]
For example:
array = [[1, 2, 3], [4, 5, 6]]
slice = [[0, 0, 1], [0, 1, 2]]
output = [[1, 1, 2], [4, 5,6]]
I've tried array[slice], but that didn't work. I also couldn't get tf.gather or tf.gather_nd to work, although these initially seemed like the correct functions to use. Note that these are all tensors in-graph.
How can I select these values in my array according to slice?
You need to add a dimension to your slice tensor which you can do with tf.pack and then we can use tf.gather_nd no problem.
import tensorflow as tf
tensor = tf.constant([[1, 2, 3], [4, 5, 6]])
old_slice = tf.constant([[0, 0, 1], [0, 1, 2]])
# We need to add a dimension - we need a tensor of rank 2, 3, 2 instead of 2, 3
dims = tf.constant([[0, 0, 0], [1, 1, 1]])
new_slice = tf.pack([dims, old_slice], 2)
out = tf.gather_nd(tensor, new_slice)
If we run the follow code:
with tf.Session() as sess:
sess.run(tf.initialize_all_variables())
run_tensor, run_slice, run_out = sess.run([tensor, new_slice, out])
print 'Input tensor:'
print run_tensor
print 'Correct param for gather_nd:'
print run_slice
print 'Output:'
print run_out
This should give the correct output:
Input tensor:
[[1 2 3]
[4 5 6]]
Correct param for gather_nd:
[[[0 0]
[0 0]
[0 1]]
[[1 0]
[1 1]
[1 2]]]
Output:
[[1 1 2]
[4 5 6]]
An even easier way to calculate the results, which is also of more general nature, is to directly leverage the batch_dims argument of tf.gather:
>>> array = tf.constant([[1,2,3], [4,5,6]])
>>> slice = tf.constant([[0,0,1], [0,1,2]])
>>> output = tf.constant([[1,1,2], [4,5,6]])
>>> tf.gather(array, slice, batch_dims=1, axis=1)
<tf.Tensor: shape=(2, 3), dtype=int32, numpy=
array([[1, 1, 2],
[4, 5, 6]], dtype=int32)>
Lets say I have a sparse tensor with duplicate indices and where they are duplicate I want to merge values (sum them up)
What is the best way to do this?
example:
indicies = [[1, 1], [1, 2], [1, 2], [1, 3]]
values = [1, 2, 3, 4]
object = tf.SparseTensor(indicies, values, shape=[10, 10])
result = tf.MAGIC(object)
result should be a spare tensor with the following values (or concrete!):
indicies = [[1, 1], [1, 2], [1, 3]]
values = [1, 5, 4]
The only thing I have though of is to string concat the indicies together to create an index hash apply it to a third dimension and then reduce sum on that third dimension.
indicies = [[1, 1, 11], [1, 2, 12], [1, 2, 12], [1, 3, 13]]
sparse_result = tf.sparse_reduce_sum(sparseTensor, reduction_axes=2, keep_dims=true)
But that feels very very ugly
Here is a solution using tf.segment_sum. The idea is to linearize the indices in to a 1-D space, get the unique indices with tf.unique, run tf.segment_sum, and convert the indices back to N-D space.
indices = tf.constant([[1, 1], [1, 2], [1, 2], [1, 3]])
values = tf.constant([1, 2, 3, 4])
# Linearize the indices. If the dimensions of original array are
# [N_{k}, N_{k-1}, ... N_0], then simply matrix multiply the indices
# by [..., N_1 * N_0, N_0, 1]^T. For example, if the sparse tensor
# has dimensions [10, 6, 4, 5], then multiply by [120, 20, 5, 1]^T
# In your case, the dimensions are [10, 10], so multiply by [10, 1]^T
linearized = tf.matmul(indices, [[10], [1]])
# Get the unique indices, and their positions in the array
y, idx = tf.unique(tf.squeeze(linearized))
# Use the positions of the unique values as the segment ids to
# get the unique values
values = tf.segment_sum(values, idx)
# Go back to N-D indices
y = tf.expand_dims(y, 1)
indices = tf.concat([y//10, y%10], axis=1)
tf.InteractiveSession()
print(indices.eval())
print(values.eval())
Maybe you can try:
indicies = [[1, 1], [1, 2], [1, 2], [1, 3]]
values = [1, 2, 3, 4]
object = tf.SparseTensor(indicies, values, shape=[10, 10])
tf.sparse.to_dense(object, validate_indices=False)
Using unsorted_segment_sum could be simpler:
def deduplicate(tensor):
if not isinstance(tensor, tf.IndexedSlices):
return tensor
unique_indices, new_index_positions = tf.unique(tensor.indices)
summed_values = tf.unsorted_segment_sum(tensor.values, new_index_positions, tf.shape(unique_indices)[0])
return tf.IndexedSlices(indices=unique_indices, values=summed_values, dense_shape=tensor.dense_shape)
Another solution is to use tf.scatter_nd which will create a dense tensor and accumulate values on duplicate indices. This behavior is clearly stated in the documentation:
If indices contains duplicates, the duplicate values are accumulated (summed).
Then eventually we can convert it back to a sparse representation.
Here is a code sample for TensorFlow 2.x in eager mode:
import tensorflow as tf
indicies = [[1, 1], [1, 2], [1, 2], [1, 3]]
values = [1, 2, 3, 4]
merged_dense = tf.scatter_nd(indices, values, shape=(10, 10))
merged_sparse = tf.sparse.from_dense(merged_dense)
print(merged_sparse)
Output
SparseTensor(
indices=tf.Tensor(
[[1 1]
[1 2]
[1 3]],
shape=(3, 2),
dtype=int64),
values=tf.Tensor([1 5 4], shape=(3,), dtype=int32),
dense_shape=tf.Tensor([10 10], shape=(2,), dtype=int64))
So. As per the solution mentioned above.
Another example.
For the shape [12, 5]:
Lines to be changed in the code:
linearized = tf.matmul(indices, [[5], [1]])
indices = tf.concat([y//5, y%5], axis=1)