Related
From document of unique, I know I can get return_index and return_inverse. It seems like COO matrix compressing.
I wonder how can I reconstruct the array by numpy. I find I can do it with np.fromfunction, but I must write a method to determine every item of the result array. I want to know if there is a easy/neat way.
Not sure if np.fromfunction is needed. The documentation actually gives you the example to reconstruct the array from return_inverse:
https://numpy.org/doc/stable/reference/generated/numpy.unique.html
a = np.array([1, 2, 6, 4, 2, 3, 2])
u, indices = np.unique(a, return_inverse=True)
u # ==> array([1, 2, 3, 4, 6])
indices # ==> array([0, 1, 4, 3, 1, 2, 1])
u[indices] # ==> array([1, 2, 6, 4, 2, 3, 2])
My tensor shape is 32,4 like
input_boxes = [
[1,2,3,4],
[2,2,6,4],
[[1,5,3,4],[1,3,3,8]],#some row has two
[1,2,3,4],#some has one row
[[1,2,3,4],[1,3,3,4]],
[1,7,3,4],
......
[1,2,3,4]
]
I like to expand to 32,5 at the first column like tf.expand_dims(input_boxes, 0).
Then assign value to the first column with row number like
input_boxes = [
[0,1,2,3,4],
[1,2,2,6,4],
[[2,1,5,3,4],[2,1,3,3,8]],#some row has two
[3,1,2,3,4],#some has one row
[[4,1,2,3,4],[4,1,3,3,4]],
[5,1,7,3,4],
......
[31,1,2,3,4]
]
How can I do in Tensorflow?
Mentioning the Solution here (Answer Section) even though it is present in the Comments Section (thanks to jdehesa) for the benefit of the Community.
For example, we have a Tensor of Shape (7,4) as shown below:
import tensorflow as tf
input_boxes = tf.constant([[1,2,3,4],
[2,2,6,4],
[1,5,3,4],
[1,2,3,4],
[1,2,3,4],
[1,7,3,4],
[1,2,3,4]])
print(input_boxes)
Code to expand to (7,5) at the First Column with the values of First Columns being the respective Row Number is shown below:
input_boxes = tf.concat([tf.dtypes.cast(tf.expand_dims(tf.range(tf.shape(input_boxes)[0]), 1), input_boxes.dtype), input_boxes], axis=1)
print(input_boxes)
Output of the above code is shown below:
<tf.Tensor: shape=(7, 5), dtype=int32, numpy=
array([[0, 1, 2, 3, 4],
[1, 2, 2, 6, 4],
[2, 1, 5, 3, 4],
[3, 1, 2, 3, 4],
[4, 1, 2, 3, 4],
[5, 1, 7, 3, 4],
[6, 1, 2, 3, 4]], dtype=int32)>
Hope this helps. Happy Learning!
Is it possible to look up entries from an nd array without throwing an IndexError?
I'm hoping for something like:
>>> a = np.arange(10) * 2
>>> a[[-4, 2, 8, 12]]
IndexError
>>> wrap(a, default=-1)[[-4, 2, 8, 12]]
[-1, 4, 16, -1]
>>> wrap(a, default=-1)[200]
-1
Or possibly more like get_with_default(a, [-4, 2, 8, 12], default=-1)
Is there some builtin way to do this? Can I ask numpy not to throw the exception and return garbage, which I can then replace with my default value?
np.take with clip mode, sort of does this
In [155]: a
Out[155]: array([ 0, 2, 4, 6, 8, 10, 12, 14, 16, 18])
In [156]: a.take([-4,2,8,12],mode='raise')
...
IndexError: index 12 is out of bounds for size 10
In [157]: a.take([-4,2,8,12],mode='wrap')
Out[157]: array([12, 4, 16, 4])
In [158]: a.take([-4,2,8,12],mode='clip')
Out[158]: array([ 0, 4, 16, 18])
Except you don't have much control over the return value - here indexing on 12 return 18, the last value. And treated the -4 as out of bounds in the other direction, returning 0.
One way of adding the defaults is to pad a first
In [174]: a = np.arange(10) * 2
In [175]: ind=np.array([-4,2,8,12])
In [176]: np.pad(a, [1,1], 'constant', constant_values=-1).take(ind+1, mode='clip')
Out[176]: array([-1, 4, 16, -1])
Not exactly pretty, but a start.
This is my first post on any stack exchange site so forgive me for any stylistic errors (hopefully there are only stylistic errors). I am interested in the same feature but could not find anything from numpy better than np.take mentioned by hpaulj. Still np.take doesn't do exactly what's needed. Alfe's answer works but would need some elaboration in order to handle n-dimensional inputs. The following is another workaround that generalizes to the n-dimensional case. The basic idea is similar the one used by Alfe: create a new index with the out of bounds indices masked out (in my case) or disguised (in Alfe's case) and use it to index the input array without raising an error.
def take(a,indices,default=0):
#initialize mask; will broadcast to length of indices[0] in first iteration
mask = True
for i,ind in enumerate(indices):
#each element of the mask is only True if all indices at that position are in bounds
mask = mask & (0 <= ind) & (ind < a.shape[i])
#create in_bound indices
in_bound = [ind[mask] for ind in indices]
#initialize result with default value
result = default * np.ones(len(mask),dtype=a.dtype)
#set elements indexed by in_bound to their appropriate values in a
result[mask] = a[tuple(in_bound)]
return result
And here is the output from Eric's sample problem:
>>> a = np.arange(10)*2
>>> indices = (np.array([-4,2,8,12]),)
>>> take(a,indices,default=-1)
array([-1, 4, 16, -1])
You can restrict the range of the indexes to the size of your value array you want to index in using np.maximum() and np.minimum().
Example:
I have a heatmap like
h = np.array([[ 2, 3, 1],
[ 3, -1, 5]])
and I have a palette of RGB values I want to use to color the heatmap. The palette only names colors for the values 0..4:
p = np.array([[0, 0, 0], # black
[0, 0, 1], # blue
[1, 0, 1], # purple
[1, 1, 0], # yellow
[1, 1, 1]]) # white
Now I want to color my heatmap using the palette:
p[h]
Currently this leads to an error because of the values -1 and 5 in the heatmap:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
IndexError: index 5 is out of bounds for axis 0 with size 5
But I can limit the range of the heatmap:
p[np.maximum(np.minimum(h, 4), 0)]
This works and gives me the result:
array([[[1, 0, 1],
[1, 1, 0],
[0, 0, 1]],
[[1, 1, 0],
[0, 0, 0],
[1, 1, 1]]])
If you really need to have a special value for the indexes which are out of bound, you could implement your proposed get_with_default() like this:
def get_with_default(values, indexes, default=-1):
return np.concatenate([[default], values, [default]])[
np.maximum(np.minimum(indexes, len(values)), -1) + 1]
a = np.arange(10) * 2
get_with_default(a, [-4, 2, 8, 12], default=-1)
Will return:
array([-1, 4, 16, -1])
as wanted.
What is the fastest way to compare neighboring elements in a 3-dimensional array?
Assume I have a numpy array of (4,4,4). I want to loop in the k-direction and compare elements in pairs. So, compare all neighboring elements and assign the lowest index if they are not equal. Essentially this:
if array([0, 0, 0)] != array[(0, 0, 1)]:
array[(0, 0, 0)] = 111
Thus, the comparisons would be:
(0, 0, 0) and (0, 0, 1)
(0, 0, 1) and (0, 0, 2)
(0, 0, 2) and (0, 0, 3)
(0, 0, 3) and (0, 0, 4)
... for all i and j ...
However, I want to do this for every i and j in the array and writing a standard Python for loop for this on huge arrays with millions of cells is incredibly slow. Is there a more 'standard' numpy way to do this without the explicit for loop?
Maybe there's some trick using the slicing step (i.e array[:,:,::2], array[:,:,1::2])?
Try np.diff.
import numpy as np
a = np.arange(9).reshape(3, 3)
A = np.array([a, a, a + 1]).T
same_with_neighbor_on_last_axis = np.diff(A, axis=-1) == 0
print A
print same_with_neighbor_on_last_axis
A is constructed to have 2 consecutive equal entries along the third axis,
>>>print A
array([[[0, 0, 1],
[3, 3, 4],
[6, 6, 7]],
[[1, 1, 2],
[4, 4, 5],
[7, 7, 8]],
[[2, 2, 3],
[5, 5, 6],
[8, 8, 9]]])
The output vector then yields
>>>print same_with_neighbor_on_last_axis
[[[ True False]
[ True False]
[ True False]]
[[ True False]
[ True False]
[ True False]]
[[ True False]
[ True False]
[ True False]]]
Using the axis keyword, you can choose whichever axis you need to do this operation on. If it is all of them, you can use a loop. np.diff does not much else than the following
np.diff(A, axis=-1) == A[..., 1:] - A[..., :-1]
I am trying to create a dictionary from a relatively large numpy array. I tried using the dictionary constructor like so:
elements =dict((k,v) for (a[:,0] , a[:,-1]) in myarray)
I am assuming I am doing this incorrectly since I get the error: "ValueError: too many values to unpack"
The numPy array looks like this:
[ 2.01206281e+13 -8.42110000e+04 -8.42110000e+04 ..., 0.00000000e+00
3.30000000e+02 -3.90343147e-03]
I want the first column 2.01206281e+13 to be the key and the last column -3.90343147e-03 to be the value for each row in the array
Am I on the right track/is there a better way to go about doing this?
Thanks
Edit: let me be more clear I want the first column to be the key and the last column to be the value. I want to do this for every row in the numpy array
This is kind of a hard question on answer without knowing what exactly myarray is, but this might help you get started.
>>> import numpy as np
>>> a = np.random.randint(0, 10, size=(3, 2))
>>> a
array([[1, 6],
[9, 3],
[2, 8]])
>>> dict(a)
{1: 6, 2: 8, 9: 3}
or
>>> a = np.random.randint(0, 10, size=(3, 5))
>>> a
array([[9, 7, 4, 4, 6],
[8, 9, 1, 6, 5],
[7, 5, 3, 4, 7]])
>>> dict(a[:, [0, -1]])
{7: 7, 8: 5, 9: 6}
elements = dict( zip( * [ iter( myarray ) ] * 2 ) )
What we see here is that we create an iterator based on the myarray list. We put it in a list and double it. Now we've got the same iterator bound to the first and second place in a list which we give as arguments to the zip function which creates a list of pairs for the dict creator.