Finding the distances from each point to the rest, looping - pandas

Question

I am new to python.
I have a csv file containing 400 pairs of x and y in two columns.
I want to loop over the data such that it starts from a pair (x_i,y_i) and finds the distance between that pair and the rest of the 399 points. I want the process to be repeated for all pairs of (x_i,y_i) and the result is appended to to a list Dist_i
import pandas as pd
x_y_data = pd.read_csv("x_y_points400_labeled_csv.csv")
x = x_y_data.loc[:,'x']
y = x_y_data.loc[:,'y']
i=0
j=0
while (i<len(x)):
Dist=np.sqrt((x[i]-x)**2 + (y[j]-y)**2)
i = 1 + i
j = 1 + j
print(Dist)
output:
0 676.144955
1 675.503342
2 674.642602
..
396 9.897127
397 21.659654
398 15.508062
399 0.000000
Length: 400, dtype: float64
This is how far I went, but it is not what I intend to obtain. My goal is to get something like in the picture attached.
Thanks for your help in advance
enter image description here

Answer 1

You can use broadcasting (arr[:, None]) to do this calculation all at once. This will give you the repetitive calculations you want. Otherwise scipy.spatial.distance.pdist gives you the upper triangle of the calculations.
Sample Data
import pandas as pd
import numpy as np
np.random.seed(123)
N = 6
df = pd.DataFrame(np.random.normal(0, 1, (N, 2)),
columns=['X', 'Y'],
index=[f'point{i}' for i in range(N)])
x = df['X'].to_numpy()
y = df['Y'].to_numpy()
result = pd.DataFrame(np.sqrt((x[:, None] - x)**2 + (y[:, None] - y)**2),
index=df.index,
columns=df.index)
point0 point1 point2 point3 point4 point5
point0 0.000000 2.853297 0.827596 1.957709 3.000780 1.165343
point1 2.853297 0.000000 3.273161 2.915990 1.172704 1.708145
point2 0.827596 3.273161 0.000000 2.782669 3.121463 1.749023
point3 1.957709 2.915990 2.782669 0.000000 3.718481 1.779459
point4 3.000780 1.172704 3.121463 3.718481 0.000000 2.092455
point5 1.165343 1.708145 1.749023 1.779459 2.092455 0.000000
With scipy.
from scipy.spatial.distance import pdist
pdist(df[['X', 'Y']])
array([2.8532972 , 0.82759587, 1.95770875, 3.00078036, 1.16534282,
3.27316125, 2.91598992, 1.17270443, 1.70814458, 2.78266933,
3.1214628 , 1.74902298, 3.7184812 , 1.77945856, 2.09245472])
To turn this into the above DataFrame.
L = len(df)
arr = np.zeros((L, L))
arr[np.triu_indices(L, 1)] = pdist(df[['X', 'Y']])
arr = arr + arr.T # Lower triangle b/c symmetric
pd.DataFrame(arr, index=df.index, columns=df.index)
point0 point1 point2 point3 point4 point5
point0 0.000000 2.853297 0.827596 1.957709 3.000780 1.165343
point1 2.853297 0.000000 3.273161 2.915990 1.172704 1.708145
point2 0.827596 3.273161 0.000000 2.782669 3.121463 1.749023
point3 1.957709 2.915990 2.782669 0.000000 3.718481 1.779459
point4 3.000780 1.172704 3.121463 3.718481 0.000000 2.092455
point5 1.165343 1.708145 1.749023 1.779459 2.092455 0.000000