Related
Given a set of 2d data points with coordinates x and y (left picture), is there an easy way to construct a triangular mesh on top of it (right picture)? i.e. return a list of tuples that indicates which vertices are connected. The solution is not unique, but any reasonable mesh would suffice.
You can use scipy.spatial.Delaunay. Here is an example from the
import numpy as np
points = np.array([[-1,1],[-1.3, .6],[0,0],[.2,.8],[1,.85],[-.1,-.4],[.4,-.15],[.6,-.6],[.9,-.2]])
from scipy.spatial import Delaunay
tri = Delaunay(points)
import matplotlib.pyplot as plt
plt.triplot(points[:,0], points[:,1], tri.simplices)
plt.plot(points[:,0], points[:,1], 'o')
plt.show()
Here is the result on an input similar to yours:
The triangles are stored in the simplices attribute of the Delaunay object which reference the coordinates stored in the points attribute:
>>> tri.points
array([[-1. , 1. ],
[-1.3 , 0.6 ],
[ 0. , 0. ],
[ 0.2 , 0.8 ],
[ 1. , 0.85],
[-0.1 , -0.4 ],
[ 0.4 , -0.15],
[ 0.6 , -0.6 ],
[ 0.9 , -0.2 ]])
>>> tri.simplices
array([[5, 2, 1],
[0, 3, 4],
[2, 0, 1],
[3, 0, 2],
[8, 6, 7],
[6, 5, 7],
[5, 6, 2],
[6, 3, 2],
[3, 6, 4],
[6, 8, 4]], dtype=int32)
If you are looking for which vertices are connected, there is an attribute containing that info also:
>>> tri.vertex_neighbor_vertices
(array([ 0, 4, 7, 12, 16, 20, 24, 30, 33, 36], dtype=int32), array([3, 4, 2, 1, 5, 2, 0, 5, 1, 0, 3, 6, 0, 4, 2, 6, 0, 3, 6, 8, 2, 1,
6, 7, 8, 7, 5, 2, 3, 4, 8, 6, 5, 6, 7, 4], dtype=int32))
You can try scipy.spatial.Delaunay. From that link:
points = np.array([[0, 0], [0, 1.1], [1, 0], [1, 1]])
from scipy.spatial import Delaunay
tri = Delaunay(points)
plt.triplot(points[:,0], points[:,1], tri.simplices)
plt.plot(points[:,0], points[:,1], 'o')
plt.show()
Output:
I think Delanuay gives something closer to a convex hull. In OP's picture A is not connected to C, it is connected to B which is connected to C which gives a different shape.
One solution could be running Delanuay first then removing triangles whose angles exceed a certain degree, eg 90, or 100. A prelim code could look like
from scipy.spatial import Delaunay
points = [[101, 357], [198, 327], [316, 334], [ 58, 299], [162, 258], [217, 240], [310, 236], [153, 207], [257, 163]]
points = np.array(points)
tri = Delaunay(points,furthest_site=False)
newsimp = []
for t in tri.simplices:
A,B,C = points[t[0]],points[t[1]],points[t[2]]
e1 = B-A; e2 = C-A
num = np.dot(e1, e2)
denom = np.linalg.norm(e1) * np.linalg.norm(e2)
d1 = np.rad2deg(np.arccos(num/denom))
e1 = C-B; e2 = A-B
num = np.dot(e1, e2)
denom = np.linalg.norm(e1) * np.linalg.norm(e2)
d2 = np.rad2deg(np.arccos(num/denom))
d3 = 180-d1-d2
degs = np.array([d1,d2,d3])
if np.any(degs > 110): continue
newsimp.append(t)
plt.triplot(points[:,0], points[:,1], newsimp)
which gives the shape seen above. For more complicated shapes removing large sides could be necessary too,
for t in tri.simplices:
...
n1 = np.linalg.norm(e1); n2 = np.linalg.norm(e2)
...
res.append([n1,n2,d1,d2,d3])
res = np.array(res)
m = res[:,[0,1]].mean()*res[:,[0,1]].std()
mask = np.any(res[:,[2,3,4]] > 110) & (res[:,0] < m) & (res[:,1] < m )
plt.triplot(points[:,0], points[:,1], tri.simplices[mask])
I have a correlation matrix, and I want to get the count of number of items below the diagonal. Preferably in numpy.
[[1, 0, 0, 0, 0],
[.35, 1, 0, 0, 0],
[.42, .31, 1, 0, 0],
[.25, .38, .41, 1, 0],
[.21, .36, .46, .31, 1]]
I want it to return 10. Or, to return the mean of all numbers under the diagonal.
Setup
a = np.array([[1. , 0. , 0. , 0. , 0. ],
[0.35, 1. , 0. , 0. , 0. ],
[0.42, 0.31, 1. , 0. , 0. ],
[0.25, 0.38, 0.41, 1. , 0. ],
[0.21, 0.36, 0.46, 0.31, 1. ]])
numpy.tril_indices will give the indices of all elements under the diagonal (if you provide an offset of -1), and from there, it becomes as simple as indexing and calling mean and size
n, m = a.shape
m = np.tril_indices(n=n, k=-1, m=m)
a[m]
# array([0.35, 0.42, 0.31, 0.25, 0.38, 0.41, 0.21, 0.36, 0.46, 0.31])
a[m].mean()
# 0.346
a[m].size
# 10
A more primitive and bulky answer since numpy provides np.tril_indices as user3483203 mentioned, but what you want per row iteration i is the following (in terms of [row,col] indices):
(i=0)
[1,0] (i=1)
[2,0] [2,1] (i=2)
[3,0] [3,1] [3,2] (i=3)
...
This is essentially the zip of list [i,i,i,...] = [i]*i (i repetitions of i) with [0,1,...,i-1] = range(i). So iterating over the rows of the table, you can actually get the indices per iteration and perform the operator of your choice.
Example setup:
test = np.array(
[[1, 0, 0, 0, 0],
[.35, 1, 0, 0, 0],
[.42, .31, 1, 0, 0],
[.25, .38, .41, 1, 0],
[.21, .36, .46, .31, 1]])
Function definition:
def countdiag(myarray):
numvals = 0
totsum = 0
for i in range(myarray.shape[0]): # row iteration
colc = np.array(range(i)) # calculate column indices
rowc = np.array([i]*i) # calculate row indices
if any(rowc):
print(np.sum(myarray[rowc,colc]))
print(len(myarray[rowc,colc]))
numvals += len(myarray[rowc,colc])
totsum += np.sum(myarray[rowc,colc])
print(list(zip([i]*i, np.arange(i))))
mean = totsum / numvals
return mean, numvals
Test:
[165]: countdiag(test)
[]
0.35
1
[(1, 0)]
0.73
2
[(2, 0), (2, 1)]
1.04
3
[(3, 0), (3, 1), (3, 2)]
1.34
4
[(4, 0), (4, 1), (4, 2), (4, 3)]
0.346
Out[165]:
(0.346, 10)
I'm calculating the affine transformation I need from a a few points in a 3D space and using numpy.linalg.lstsq to do so. However, the results I'm getting, while not terribly far off, are not accurate enough, even in trivially simple examples:
m = 100
xy = np.array([[0, 0, 0],
[m, 0, 0],
[m, m, 0],
[0, m, 0],
[0, 0, m],
[m, 0, m],
[m, m, m],
[0, m, m]])
uv = np.array([[0.5, 0, 0],
[m + 0.5, 0, 0],
[m+ 0.5, m, 0],
[0.5, m, 0],
[0.5, 0, m],
[m+ 0.5, 0, m],
[m+ 0.5, m, m],
[0.5, m, m]])
pts_a = np.hstack([uv, np.ones((uv.shape[0], 1))])
pts_b = np.hstack([xy, np.ones((xy.shape[0], 1))])
solution_1 = np.linalg.lstsq(pts_a, pts_b, rcond=None)[0]
The result I'm expecting from the above code is:
[[1, 0, 0, -0.5],
[0, 1, 0, 0],
[0, 0, 1, 0],
[0, 0, 0, 1]])
The result I'm getting:
[[ 1.00000000e+00 3.49047642e-16 3.60109527e-16 -5.00000000e-01]
[ 1.77081442e-16 1.00000000e+00 -3.93150475e-16 1.80460546e-15]
[ 2.21351803e-16 -3.11848610e-16 1.00000000e+00 -6.28251374e-15]
[ 2.76689754e-18 1.06035619e-17 -1.19061095e-17 1.00000000e+00]]
Those small differences make a considerable difference in my results. Any ideas how to solve it?
NOTE: I can ONLY use numpy and math for my project, so using a different library is sadly not possible!
Thanks!
Actually, the difference isn't small but quite big - you have the wrong sign for solution[0,3].
The problem is thath you didn't calculate the desired transformation T but the inverse of this transformation, i.e. T^-1.
Let's do some math:
T*X=U, with X - original vectors
U - transformed vectors
transposing it =>
X^t * T^t = U^t
| | |
\|/ \|/ \|/
A * x = b
In your program A=pts_b and b=pts_a, that means the transformation T is (you have to swap pts_b and pts_b and to transpose the result to get the right matrix):
T = np.linalg.lstsq(pts_b, pts_a)[0].T
and voila:
>>> T
array([[ 1.00000000e+00, -8.15320034e-17, -6.59194921e-17, 5.00000000e-01],
[ -4.97379910e-16, 1.00000000e+00, 7.77156117e-16, -1.02678283e-14],
[ -2.13162819e-16, 4.44089210e-16, 1.00000000e+00, 1.91513472e-15],
[ -4.44089205e-18, -8.84708973e-17, 9.88792381e-17, 1.00000000e+00]])
PS: You have solved the equation:
X^t = U^t * (T^t)^(-1)
| | |
\|/ \|/ \|/
b = A * x
I have 3 states (i.e. state is a vector, vector length is 2 in this example), I have probability distribution (such as 10% state 1, 60% state 2, 30% state 3). I would like to derive a new state that is the sum of probability * state. Of cause, I will need to consider batch too.
1> calculated probability distribution, I have a batch of 4, and 3 possibilities and the distribution is defined below.
dist = tf.constant([[0.1, 0.6, 0.3], [0.2, 0.4, 0.4], [0.3, 0.5, 0.2], [0.3, 0.6, 0.1]])
2> the 3 possible states (and batch of 4). This is a tensor of shape [4, 3, 2] or [batch, 3 possible state, state values]
val = tf.constant([[[10.0, 5.0],[10, 5],[10,5]],[[8, 2],[8, 2],[8, 2]],[[7, 3],[9, 1],[6, 4]],[[1, 2],[3, 4],[5, 6]]])
I would like to get a tensor of [4, 2] or [batch, state values]. In this case, the value should be
[
[10*0.1 + 10*0.6 + 10*0.3, 5*0.1 + 5*0.6 + 5*0.3],
[8*0.2 + 8*0.4 + 8*0.4, 2*0.2 + 2*0.4 + 2*0.4],
[7*0.3 + 9*0.5 + 6*0.2, 3*0.3 + 1*0.5 + 4*0.2],
[1*0.3 + 3*0.6 + 5*0.1, 2*0.3 + 4*0.6 + 6*0.1]
]
or
[
[10, 5],
[8, 2],
[7.8, 2.2],
[2.6, 3.6]
]
How could I do that? Thanks!
Here is what I found. It turns out to be simple element wise mulitiplication (* or tf.multiply)
dist = tf.constant([[0.1, 0.6, 0.3], [0.2, 0.4, 0.4], [0.3, 0.5, 0.2], [0.3, 0.6, 0.1]])
val = tf.constant([[[10.0, 5.0],[10, 5],[10,5]],[[8, 2],[8, 2],[8, 2]],[[7, 3],[9, 1],[6, 4]],[[1, 2],[3, 4],[5, 6]]])
dist.get_shape()
TensorShape([Dimension(4), Dimension(3)])
val.get_shape()
TensorShape([Dimension(4), Dimension(3), Dimension(2)])
val2 = tf.transpose(val, perm=[0, 2, 1])
val2.get_shape()
TensorShape([Dimension(4), Dimension(2), Dimension(3)])
dist2 = tf.expand_dims(dist, 1)
dist2.get_shape()
TensorShape([Dimension(4), Dimension(1), Dimension(3)])
c1 = val2 * dist2
c1.get_shape()
TensorShape([Dimension(4), Dimension(2), Dimension(3)])
c2 = tf.reduce_sum(c1, 2)
print(c2.eval())
[[ 10. 5. ]
[ 8. 2. ]
[ 7.80000019 2.20000005]
[ 2.60000014 3.5999999 ]]
I'm using the tf.unsorted_segment_sum method of TensorFlow and it works fine when the tensor i give as data have only one line. For example:
tf.unsorted_segment_sum(tf.constant([0.2, 0.1, 0.5, 0.7, 0.8]),
tf.constant([0, 0, 1, 2, 2]), 3)
Gives the right result:
array([ 0.3, 0.5 , 1.5 ], dtype=float32)
The question is, if i use a tensor with several lines, how can I get the results for each line? For instance, if I try a tensor with two lines:
tf.unsorted_segment_sum(tf.constant([[0.2, 0.1, 0.5, 0.7, 0.8],
[0.2, 0.2, 0.5, 0.7, 0.8]]),
tf.constant([[0, 0, 1, 2, 2],
[0, 0, 1, 2, 2]]), 3)
The result i would expect is:
array([ [ 0.3, 0.5 , 1.5 ], [ 0.4, 0.5, 1.5 ] ], dtype=float32)
But what I get is:
array([ 0.7, 1. , 3. ], dtype=float32)
I want to know if someone know how to obtain the result for each line without using a for loop?
Thanks in advance
EDIT:
While the solution below may cover some additional strange uses, this problem can be solved much more easily just by transposing the data. It turns out that, even though tf.unsorted_segment_sum does not have an axis parameter, it can work only along one axis, as long as it is the first one. So you can do just as follows:
import tensorflow as tf
with tf.Session() as sess:
data = tf.constant([[0.2, 0.1, 0.5, 0.7, 0.8],
[0.2, 0.2, 0.5, 0.7, 0.8]])
idx = tf.constant([0, 0, 1, 2, 2])
result = tf.transpose(tf.unsorted_segment_sum(tf.transpose(data), idx, 3))
print(sess.run(result))
Output:
[[ 0.30000001 0.5 1.5 ]
[ 0.40000001 0.5 1.5 ]]
ORIGINAL POST:
tf.unsorted_segment_sum does not support working on a single axis. The simplest solution would be to apply the operation to each row and then concatenate them back:
data = tf.constant([[0.2, 0.1, 0.5, 0.7, 0.8],
[0.2, 0.2, 0.5, 0.7, 0.8]])
segment_ids = tf.constant([[0, 0, 1, 2, 2],
[0, 0, 1, 2, 2]])
num_segments = 3
rows = []
for data_i, ids_i in zip(data, segment_ids):
rows.append(tf.unsorted_segment_sum(data_i, ids_i))
result = tf.stack(rows, axis=0)
However, this has drawbacks: 1) it only works for statically-shaped tensors (that is, you need to have a fixed number of rows) and 2) it may not be as efficient. The first one could be circumvented using a tf.while_loop, but, it would be complicated, and also it would require you to concatenate the rows one by one, which is very inefficient. Also, you already stated you want to avoid loops.
A better option is to use different ids for each row. For example, you could add to each value in segment_id something like num_segments * row_index, so you guarantee that each row will have its own set of ids:
num_rows = tf.shape(segment_ids)[0]
rows_idx = tf.range(num_rows)
segment_ids_per_row = segment_ids + num_segments * tf.expand_dims(rows_idx, axis=1)
Then you can apply the operation and the reshape to get the tensor that you want:
seg_sums = tf.unsorted_segment_sum(data, segment_ids_per_row,
num_segments * num_rows)
result = tf.reshape(seg_sums, [-1, num_segments])
Output:
array([[ 0.3, 0.5, 1.5 ],
[ 0.4, 0.5, 1.5 ]], dtype=float32)