Plot random points a specified distance apart - numpy

I'm trying to come up with a function that plots n points inside the unit circle, but I need them to be sufficiently spread out.
ie. something that looks like this:
Is it possible to write a function with two parameters, n (number of points) and min_d (minimum distance apart) such that the points are:
a) equidistant
b) no pairwise distance exceeds a given min_d
The problem with sampling from a uniform distribution is that it could happen that two points are almost on top of each other, which I do not want to happen. I need this kind of input for a network diagram representing node clusters.
EDIT: I have found an answer to a) here: Generator of evenly spaced points in a circle in python, but b) still eludes me.

At the time this answer was provided, the question asked for random numbers. This answer thus gives a solution drawing random numbers. It ignores any edits made to the question afterwards.
On may simply draw random points and for each one check if the condition of the minimum distance is fulfilled. If not, the point can be discarded. This can be done until a list is filled with enough points or some break condition is met.
import numpy as np
import matplotlib.pyplot as plt
class Points():
def __init__(self,n=10, r=1, center=(0,0), mindist=0.2, maxtrials=1000 ) :
self.success = False
self.n = n
self.r = r
self.center=np.array(center)
self.d = mindist
self.points = np.ones((self.n,2))*10*r+self.center
self.c = 0
self.trials = 0
self.maxtrials = maxtrials
self.tx = "rad: {}, center: {}, min. dist: {} ".format(self.r, center, self.d)
self.fill()
def dist(self, p, x):
if len(p.shape) >1:
return np.sqrt(np.sum((p-x)**2, axis=1))
else:
return np.sqrt(np.sum((p-x)**2))
def newpoint(self):
x = (np.random.rand(2)-0.5)*2
x = x*self.r-self.center
if self.dist(self.center, x) < self.r:
self.trials += 1
if np.all(self.dist(self.points, x) > self.d):
self.points[self.c,:] = x
self.c += 1
def fill(self):
while self.trials < self.maxtrials and self.c < self.n:
self.newpoint()
self.points = self.points[self.dist(self.points,self.center) < self.r,:]
if len(self.points) == self.n:
self.success = True
self.tx +="\n{} of {} found ({} trials)".format(len(self.points),self.n,self.trials)
def __repr__(self):
return self.tx
center =(0,0)
radius = 1
p = Points(n=40,r=radius, center=center)
fig, ax = plt.subplots()
x,y = p.points[:,0], p.points[:,1]
plt.scatter(x,y)
ax.add_patch(plt.Circle(center, radius, fill=False))
ax.set_title(p)
ax.relim()
ax.autoscale_view()
ax.set_aspect("equal")
plt.show()
If the number of points should be fixed, you may try to run find this number of points for decreasing distances until the desired number of points are found.
In the following case, we are looking for 60 points and start with a minimum distance of 0.6 which we decrease stepwise by 0.05 until there is a solution found. Note that this will not necessarily be the optimum solution, as there is only maxtrials of retries in each step. Increasing maxtrials will of course bring us closer to the optimum but requires more runtime.
center =(0,0)
radius = 1
mindist = 0.6
step = 0.05
success = False
while not success:
mindist -= step
p = Points(n=60,r=radius, center=center, mindist=mindist)
print p
if p.success:
break
fig, ax = plt.subplots()
x,y = p.points[:,0], p.points[:,1]
plt.scatter(x,y)
ax.add_patch(plt.Circle(center, radius, fill=False))
ax.set_title(p)
ax.relim()
ax.autoscale_view()
ax.set_aspect("equal")
plt.show()
Here the solution is found for a minimum distance of 0.15.

Related

Calculating the size of an object using opencv and numpy poly1d

I'm looking to use a small numpy array to generate a curve that I can use to predict the height measurement at non-known points. I have several points that I am using to create a poly1d. I know it's possible, we use software that does it just fine at work, and when I used a different image as a tester, plugging the values into Excel and getting the polynomial, it worked fine, but I'm getting pretty drastic measurements on a different calibratable image, I get drastically different results.
Here is the image that I'm trying to measure.
The stick on the front of the pole contains known measurements. From bottom to top, they are 3'6" (42"), 6'6" (78"), 9' 8" (116"), 13' (156)
The picture has been through opencv undistort with a calibrated camera.
This is the function that actually performs the logic. x and y are gathered by cv2 EVENT_LBUTTONUP, and sent to this function.
Checking the lengths of the array is just to help me figure out why this isn't working, trying to generate a line to show the curve fit.
dist = self.firstClick-y
self.yData.append(dist)
if len(self.yData) > 4:
print(self.poly(dist))
if len(self.yData) == 4:
array = np.array(self.xData)
array = np.expand_dims(array, axis=0)
print(self.xData)
print(self.yData)
array=np.append(array, [self.yData], axis=0)
print(array)
x = array[:,0]
y = array[:,1]
self.poly = np.poly1d(np.polyfit(x, y, 2))
poly1d = np.poly1d(self.poly)
xp = np.linspace(-2, 20, 1)
_ = plt.plot(x, y, '.', xp, self.poly(xp), '-', xp, self.poly(xp), '--')
plt.ylim(0,200)
plt.show()
When I run this code, my values tend to quickly go into the tens of thousands when I'm attempting to collect the measurement at 18' 11", (the lowest wire).
Any help would be appreciated, I've been up all night trying to fit this curve.
Edit:
Sorry, I should have included the code used to display and scale the image.
self.img = cv2.imread(imagePath, cv2.IMREAD_ANYCOLOR)
self.scale_percent = 30
self.width = int(self.img.shape[1] * self.scale_percent/100)
self.height = int(self.img.shape[0] * self.scale_percent/100)
dsize = (self.width, self.height)
self.output = cv2.resize(self.img, dsize)
img = self.output
cv2.imshow('image', img)
cv2.setMouseCallback('image', self.click_event)
cv2.waitKey()
I just called this function to display the image and the below code to calibrate the values.
if self.firstClick == 0:
self.firstClick = y
cv2.putText(self.output, "Pole Base", (x, y), font, 1, (255, 255, 0), 2)
cv2.imshow('image', self.output)
elif self.firstClick != 0 and self.secondClick == 0:
self.secondClick = y
print("The difference in first and second clicks is", self.firstClick - self.secondClick)
first = self.firstClick - self.secondClick
inch = first/42
foot = inch*12
self.foot = foot
print("One foot is currently: ", foot)
self.firstLine = 3.5*12
self.secondLine = 6.5*12
self.thirdLine = 9.67*12
self.fourthLine = 13*12
self.xData = np.array([self.firstLine, self.secondLine, self.thirdLine, self.fourthLine])
self.yData.append(self.firstLine)
print(self.firstLine)
print(self.secondLine)
print(self.thirdLine)
print(self.fourthLine)

networkx position (biggest) nodes in the middle of a graph

I've been creating graphs with the networkx package and everything works fine. I would like to make the graphs even better by placing the bigger nodes in the middle of the graph and the layout functions from networkx does not seem to do the job. The nodes represent the size of degree (the higher connected the node, the bigger).
Is there any way to program these graphs in such a way that the bigger nodes are positioned in the middle? It does not have to be automated, i could also manually choose the nodes and give them the middle position but i can also not find how to do this.
If this is not possible with networkx or something else; is there any way to do it with Gephi or cytoscape? I had trouble with Gephi that it does not import the graph the same way i see it in my jupyter notebook (the colors, the node- and edge-sizes do not import).
To summarize; i want to put bigger nodes in the middle of my graph but i dont mind how i get it done (with networkx, matplotlib or whatever).
Unfortunately i cannot provide my actual graphs but here is an example which can look like one of my graphs; it is a directed weighted graph.
G = nx.gnp_random_graph(15, 0.2, directed=True)
d = dict(G.degree(weight='weight'))
d = {k: v/10 for k, v in d.items()}
edge_size = [(float(i)/sum(weights))*100 for i in weights]
node_size = [(v*1000) for v in d.values()]
nx.draw(G,width=edge_size,node_size=node_size)
There are several options:
import networkx as nx
G = nx.gnp_random_graph(15, 0.2, directed=True)
node_degree = dict(G.degree(weight='weight'))
# A) Precompute node positions, and then manually over-ride some node positions.
node_positions = nx.spring_layout(G)
node_positions[0] = (0.5, 0.5) # by default, networkx plots on a canvas with the origin at (0, 0) and a width and height of 1; (0.5, 0.5) is hence the center
nx.draw(G, pos=node_positions, node_size=[100 * node_degree[node] for node in G])
plt.show()
# B) Use netgraph to draw the graph and then drag the nodes around with the mouse.
from netgraph import InteractiveGraph # pip install netgraph
plot_instance = InteractiveGraph(G, node_size=node_degree)
plt.show()
# C) Modify the Fruchterman-Reingold algorithm to include a gravitational force that pulls nodes with a large "mass" towards the center.
# This is left as an exercise to the interested reader (i.e. very non-trivial).
Edit: option C is non-trivial but also very do-able.
Here is my stab at it.
#!/usr/bin/env python
# coding: utf-8
"""
FR layout but with an additional gravitational pull towards a gravitational center.
The pull is proportional to the mass of the node.
"""
import numpy as np
import matplotlib.pyplot as plt
# pip install netgraph
from netgraph._main import BASE_SCALE
from netgraph._utils import (
_get_unique_nodes,
_edge_list_to_adjacency_matrix,
)
from netgraph._node_layout import (
_is_within_bbox,
_get_temperature_decay,
_get_fr_repulsion,
_get_fr_attraction,
_rescale_to_frame,
_handle_multiple_components,
_reduce_node_overlap,
)
DEBUG = False
#_handle_multiple_components
def get_fruchterman_reingold_newton_layout(edges,
edge_weights = None,
k = None,
g = 1.,
scale = None,
origin = None,
gravitational_center = None,
initial_temperature = 1.,
total_iterations = 50,
node_size = 0,
node_mass = 1,
node_positions = None,
fixed_nodes = None,
*args, **kwargs):
"""Modified Fruchterman-Reingold node layout.
Uses a modified Fruchterman-Reingold algorithm [Fruchterman1991]_ to compute node positions.
This algorithm simulates the graph as a physical system, in which nodes repell each other.
For connected nodes, this repulsion is counteracted by an attractive force exerted by the edges, which are simulated as springs.
Unlike the original algorithm, there is an additional attractive force pulling nodes towards a gravitational center, in proportion to their masses.
Parameters
----------
edges : list
The edges of the graph, with each edge being represented by a (source node ID, target node ID) tuple.
edge_weights : dict
Mapping of edges to edge weights.
k : float or None, default None
Expected mean edge length. If None, initialized to the sqrt(area / total nodes).
g : float or None, default 1.
Gravitational constant that sets the magnitude of the gravitational pull towards the center.
origin : tuple or None, default None
The (float x, float y) coordinates corresponding to the lower left hand corner of the bounding box specifying the extent of the canvas.
If None is given, the origin is placed at (0, 0).
scale : tuple or None, default None
The (float x, float y) dimensions representing the width and height of the bounding box specifying the extent of the canvas.
If None is given, the scale is set to (1, 1).
gravitational_center : tuple or None, default None
The (float x, float y) coordinates towards which nodes experience a gravitational pull.
If None, the gravitational center is placed at the center of the canvas defined by origin and scale.
total_iterations : int, default 50
Number of iterations.
initial_temperature: float, default 1.
Temperature controls the maximum node displacement on each iteration.
Temperature is decreased on each iteration to eventually force the algorithm into a particular solution.
The size of the initial temperature determines how quickly that happens.
Values should be much smaller than the values of `scale`.
node_size : scalar or dict, default 0.
Size (radius) of nodes.
Providing the correct node size minimises the overlap of nodes in the graph,
which can otherwise occur if there are many nodes, or if the nodes differ considerably in size.
node_mass : scalar or dict, default 1.
Mass of nodes.
Nodes with higher mass experience a larger gravitational pull towards the center.
node_positions : dict or None, default None
Mapping of nodes to their (initial) x,y positions. If None are given,
nodes are initially placed randomly within the bounding box defined by `origin` and `scale`.
If the graph has multiple components, explicit initial positions may result in a ValueError,
if the initial positions fall outside of the area allocated to that specific component.
fixed_nodes : list or None, default None
Nodes to keep fixed at their initial positions.
Returns
-------
node_positions : dict
Dictionary mapping each node ID to (float x, float y) tuple, the node position.
References
----------
.. [Fruchterman1991] Fruchterman, TMJ and Reingold, EM (1991) ‘Graph drawing by force‐directed placement’,
Software: Practice and Experience
"""
# This is just a wrapper around `_fruchterman_reingold`, which implements (the loop body of) the algorithm proper.
# This wrapper handles the initialization of variables to their defaults (if not explicitely provided),
# and checks inputs for self-consistency.
assert len(edges) > 0, "The list of edges has to be non-empty."
if origin is None:
if node_positions:
minima = np.min(list(node_positions.values()), axis=0)
origin = np.min(np.stack([minima, np.zeros_like(minima)], axis=0), axis=0)
else:
origin = np.zeros((2))
else:
# ensure that it is an array
origin = np.array(origin)
if scale is None:
if node_positions:
delta = np.array(list(node_positions.values())) - origin[np.newaxis, :]
maxima = np.max(delta, axis=0)
scale = np.max(np.stack([maxima, np.ones_like(maxima)], axis=0), axis=0)
else:
scale = np.ones((2))
else:
# ensure that it is an array
scale = np.array(scale)
assert len(origin) == len(scale), \
"Arguments `origin` (d={}) and `scale` (d={}) need to have the same number of dimensions!".format(len(origin), len(scale))
dimensionality = len(origin)
if gravitational_center is None:
gravitational_center = origin + 0.5 * scale
else:
# ensure that it is an array
gravitational_center = np.array(gravitational_center)
if fixed_nodes is None:
fixed_nodes = []
connected_nodes = _get_unique_nodes(edges)
if node_positions is None: # assign random starting positions to all nodes
node_positions_as_array = np.random.rand(len(connected_nodes), dimensionality) * scale + origin
unique_nodes = connected_nodes
else:
# 1) check input dimensionality
dimensionality_node_positions = np.array(list(node_positions.values())).shape[1]
assert dimensionality_node_positions == dimensionality, \
"The dimensionality of values of `node_positions` (d={}) must match the dimensionality of `origin`/ `scale` (d={})!".format(dimensionality_node_positions, dimensionality)
is_valid = _is_within_bbox(list(node_positions.values()), origin=origin, scale=scale)
if not np.all(is_valid):
error_message = "Some given node positions are not within the data range specified by `origin` and `scale`!"
error_message += "\n\tOrigin : {}, {}".format(*origin)
error_message += "\n\tScale : {}, {}".format(*scale)
error_message += "\nThe following nodes do not fall within this range:"
for ii, (node, position) in enumerate(node_positions.items()):
if not is_valid[ii]:
error_message += "\n\t{} : {}".format(node, position)
error_message += "\nThis error can occur if the graph contains multiple components but some or all node positions are initialised explicitly (i.e. node_positions != None)."
raise ValueError(error_message)
# 2) handle discrepancies in nodes listed in node_positions and nodes extracted from edges
if set(node_positions.keys()) == set(connected_nodes):
# all starting positions are given;
# no superfluous nodes in node_positions;
# nothing left to do
unique_nodes = connected_nodes
else:
# some node positions are provided, but not all
for node in connected_nodes:
if not (node in node_positions):
warnings.warn("Position of node {} not provided. Initializing to random position within frame.".format(node))
node_positions[node] = np.random.rand(2) * scale + origin
unconnected_nodes = []
for node in node_positions:
if not (node in connected_nodes):
unconnected_nodes.append(node)
fixed_nodes.append(node)
# warnings.warn("Node {} appears to be unconnected. The current node position will be kept.".format(node))
unique_nodes = connected_nodes + unconnected_nodes
node_positions_as_array = np.array([node_positions[node] for node in unique_nodes])
total_nodes = len(unique_nodes)
if isinstance(node_size, (int, float)):
node_size = node_size * np.ones((total_nodes))
elif isinstance(node_size, dict):
node_size = np.array([node_size[node] if node in node_size else 0. for node in unique_nodes])
if isinstance(node_mass, (int, float)):
node_mass = node_mass * np.ones((total_nodes))
elif isinstance(node_mass, dict):
node_mass = np.array([node_mass[node] if node in node_mass else 0. for node in unique_nodes])
adjacency = _edge_list_to_adjacency_matrix(
edges, edge_weights=edge_weights, unique_nodes=unique_nodes)
# Forces in FR are symmetric.
# Hence we need to ensure that the adjacency matrix is also symmetric.
adjacency = adjacency + adjacency.transpose()
if fixed_nodes:
is_mobile = np.array([False if node in fixed_nodes else True for node in unique_nodes], dtype=bool)
mobile_positions = node_positions_as_array[is_mobile]
fixed_positions = node_positions_as_array[~is_mobile]
mobile_node_sizes = node_size[is_mobile]
fixed_node_sizes = node_size[~is_mobile]
mobile_node_masses = node_mass[is_mobile]
fixed_node_masses = node_mass[~is_mobile]
# reorder adjacency
total_mobile = np.sum(is_mobile)
reordered = np.zeros((adjacency.shape[0], total_mobile))
reordered[:total_mobile, :total_mobile] = adjacency[is_mobile][:, is_mobile]
reordered[total_mobile:, :total_mobile] = adjacency[~is_mobile][:, is_mobile]
adjacency = reordered
else:
is_mobile = np.ones((total_nodes), dtype=bool)
mobile_positions = node_positions_as_array
fixed_positions = np.zeros((0, 2))
mobile_node_sizes = node_size
fixed_node_sizes = np.array([])
mobile_node_masses = node_mass
fixed_node_masses = np.array([])
if k is None:
area = np.product(scale)
k = np.sqrt(area / float(total_nodes))
temperatures = _get_temperature_decay(initial_temperature, total_iterations)
# --------------------------------------------------------------------------------
# main loop
for ii, temperature in enumerate(temperatures):
candidate_positions = _fruchterman_reingold_newton(mobile_positions, fixed_positions,
mobile_node_sizes, fixed_node_sizes,
adjacency, temperature, k,
mobile_node_masses, fixed_node_masses,
gravitational_center, g)
is_valid = _is_within_bbox(candidate_positions, origin=origin, scale=scale)
mobile_positions[is_valid] = candidate_positions[is_valid]
# --------------------------------------------------------------------------------
# format output
node_positions_as_array[is_mobile] = mobile_positions
if np.all(is_mobile):
node_positions_as_array = _rescale_to_frame(node_positions_as_array, origin, scale)
node_positions = dict(zip(unique_nodes, node_positions_as_array))
return node_positions
def _fruchterman_reingold_newton(mobile_positions, fixed_positions,
mobile_node_radii, fixed_node_radii,
adjacency, temperature, k,
mobile_node_masses, fixed_node_masses,
gravitational_center, g):
"""Inner loop of modified Fruchterman-Reingold layout algorithm."""
combined_positions = np.concatenate([mobile_positions, fixed_positions], axis=0)
combined_node_radii = np.concatenate([mobile_node_radii, fixed_node_radii])
delta = mobile_positions[np.newaxis, :, :] - combined_positions[:, np.newaxis, :]
distance = np.linalg.norm(delta, axis=-1)
# alternatively: (hack adapted from igraph)
if np.sum(distance==0) - np.trace(distance==0) > 0: # i.e. if off-diagonal entries in distance are zero
warnings.warn("Some nodes have the same position; repulsion between the nodes is undefined.")
rand_delta = np.random.rand(*delta.shape) * 1e-9
is_zero = distance <= 0
delta[is_zero] = rand_delta[is_zero]
distance = np.linalg.norm(delta, axis=-1)
# subtract node radii from distances to prevent nodes from overlapping
distance -= mobile_node_radii[np.newaxis, :] + combined_node_radii[:, np.newaxis]
# prevent distances from becoming less than zero due to overlap of nodes
distance[distance <= 0.] = 1e-6 # 1e-13 is numerical accuracy, and we will be taking the square shortly
with np.errstate(divide='ignore', invalid='ignore'):
direction = delta / distance[..., None] # i.e. the unit vector
# calculate forces
repulsion = _get_fr_repulsion(distance, direction, k)
attraction = _get_fr_attraction(distance, direction, adjacency, k)
gravity = _get_gravitational_pull(mobile_positions, mobile_node_masses, gravitational_center, g)
if DEBUG:
r = np.median(np.linalg.norm(repulsion, axis=-1))
a = np.median(np.linalg.norm(attraction, axis=-1))
g = np.median(np.linalg.norm(gravity, axis=-1))
print(r, a, g)
displacement = attraction + repulsion + gravity
# limit maximum displacement using temperature
displacement_length = np.linalg.norm(displacement, axis=-1)
displacement = displacement / displacement_length[:, None] * np.clip(displacement_length, None, temperature)[:, None]
mobile_positions = mobile_positions + displacement
return mobile_positions
def _get_gravitational_pull(mobile_positions, mobile_node_masses, gravitational_center, g):
delta = gravitational_center[np.newaxis, :] - mobile_positions
direction = delta / np.linalg.norm(delta, axis=-1)[:, np.newaxis]
magnitude = mobile_node_masses - np.mean(mobile_node_masses)
return g * magnitude[:, np.newaxis] * direction
if __name__ == '__main__':
import networkx as nx
from netgraph import Graph
G = nx.gnp_random_graph(15, 0.2, directed=True)
node_degree = dict(G.degree(weight='weight'))
node_positions = get_fruchterman_reingold_newton_layout(
list(G.edges()),
node_size={node : BASE_SCALE * degree for node, degree in node_degree.items()},
node_mass=node_degree, g=2
)
Graph(G, node_layout=node_positions, node_size=node_degree)
plt.show()

Matplotlib output too small to read

In adjusting the domain of a function to find certain parameters in a matplotlib plot, I found that when I try to isolate the part I need, the output becomes so small that details are impossible to see. I've tried refreshing the kernel with no change and plt.rcParams['figure.figsize'] hasn't been effective either.
This is my current code, with unused options in the function removed.
import numpy as np
import matplotlib.pyplot as plt
def P_cubic(V,T,Tc,Pc,ParamSet,omega=0):
R = 8.31446261815324 #J mol^-1 K^-1
Tr = T/Tc
if ParamSet == 'vdW':
elif ParamSet == 'RK':
elif ParamSet == 'SRK':
elif ParamSet == 'PR':
alpha = (1+(0.37464+1.54226*omega-0.26992*omega**2)*
(1-Tr**(1/2)))**2
sigma = 1+np.sqrt(2)
epsilon = 1-np.sqrt(2)
Omega = 0.07780
Psi = 0.45724
Zc = 0.30740
a = Psi*alpha*R**2*Tc**2/Pc
b = Omega*T*Tc/Pc #m3 mol-1
P = R*T/(V-b)-a/((V+epsilon*b)*(V+sigma*b))
return P
Tc = 512.5 #K
Pc = 8.0840E6 #Pa
omega = 0.565831
T = 473 #K
b = 0.07780*T*Tc/Pc #m3 mol-1
V = np.arange(0,1,0.001)
Vrange = b*V #m3 mol-1
PPa = np.empty(len(Vrange))
for i in range(len(Vrange)):
PPa[i]=P_cubic(Vrange[i],T,Tc,Pc,'PR',omega) #Pa
Pbar = PPa*1.0E-5 #bar
plt.rcParams['figure.figsize']=(1,0.8)
plt.plot(V,Pbar)
plt.xlabel('V/b')
plt.ylabel('P /bar')
plt.xlim(0,np.max(V))
plt.ylim(np.min(Pbar),np.max(Pbar))
plt.title('Variance of Pressure with Volume of Pure Methanol at 473 K')
plt.text(15,-6,f'b = {b:.2E} m^3/mol');
Below are screenshots with the output at varying figsize parameters to show that plt.rcParams['figure.figsize'] is not helping.
How do I fix this so that I can see the details of the plot?
There are two reasons for this. First, the size unit of the graph is inches, so the specified number itself is small, resulting in a smaller graph. Secondly, the default coordinates of the annotations are based on the data, so the x-value is 15, which is far from the graph, so the figure is automatically smaller. So, I think you need to set the graph size and fix the x-value of the annotations.
fig, ax = plt.subplots()
plt.rcParams['figure.figsize']=(8,4)
ax.plot(V,Pbar)
plt.xlabel('V/b')
plt.ylabel('P /bar')
plt.xlim(0,np.max(V))
plt.ylim(np.min(Pbar),np.max(Pbar))
plt.title('Variance of Pressure with Volume of Pure Methanol at 473 K')
plt.text(1.1,-6,f'b = {b:.2E} m^3/mol')
#plt.text(1.1,-6,f'b = {b:.2E} m^3/mol', transform=ax.transData)
plt.show()

Scipy Optimize minimize returns the initial value

I am building machine learning models for a certain data set. Then, based on the constraints and bounds for the outputs and inputs, I am trying to find the input parameters for the most minimized answer.
The problem which I am facing is that, when the model is a linear regression model or something like lasso, the minimization works perfectly fine.
However, when the model is "Decision Tree", it constantly returns the very initial value that is given to it. So basically, it does not enforce the constraints.
import numpy as np
import pandas as pd
from scipy.optimize import minimize
I am using the very first sample from the input data set for the optimization. As it is only one sample, I need to reshape it to (1,-1) as well.
x = df_in.iloc[0,:]
x = np.array(x)
x = x.reshape(1,-1)
This is my Objective function:
def objective(x):
x = np.array(x)
x = x.reshape(1,-1)
y = 0
for n in range(df_out.shape[1]):
y = Model[n].predict(x)
Y = y[0]
return Y
Here I am defining the bounds of inputs:
range_max = pd.DataFrame(range_max)
range_min = pd.DataFrame(range_min)
B_max=[]
B_min =[]
for i in range(range_max.shape[0]):
b_max = range_max.iloc[i]
b_min = range_min.iloc[i]
B_max.append(b_max)
B_min.append(b_min)
B_max = pd.DataFrame(B_max)
B_min = pd.DataFrame(B_min)
bnds = pd.concat([B_min, B_max], axis=1)
These are my constraints:
con_min = pd.DataFrame(c_min)
con_max = pd.DataFrame(c_max)
Here I am defining the constraint function:
def const(x):
x = np.array(x)
x = x.reshape(1,-1)
Y = []
for n in range(df_out.shape[1]):
y = Model[n].predict(x)[0]
Y.append(y)
Y = pd.DataFrame(Y)
a4 =[]
for k in range(Y.shape[0]):
a1 = Y.iloc[k,0] - con_min.iloc[k,0]
a2 = con_max.iloc[k, 0] - Y.iloc[k,0]
a3 = [a2,a1]
a4 = np.concatenate([a4, a3])
return a4
c = const(x)
con = {'type': 'ineq', 'fun': const}
This is where I try to minimize. I do not pick a method as the automatically picked model has worked so far.
sol = minimize(fun = objective, x0=x,constraints=con, bounds=bnds)
So the actual constraints are:
c_min = [0.20,1000]
c_max = [0.3,1600]
and the max and min range for the boundaries are:
range_max = [285,200,8,85,0.04,1.6,10,3.5,20,-5]
range_min = [215,170,-1,60,0,1,6,2.5,16,-18]
I think you should check the output of 'sol'. At times, the algorithm is not able to perform line search completely. To check for this, you should check message associated with 'sol'. In such a case, the optimizer returns initial parameters itself. There may be various reasons of this behavior. In a nutshell, please check the output of sol and act accordingly.
Arad,
If you have not yet resolved your issue, try using scipy.optimize.differential_evolution instead of scipy.optimize.minimize. I ran into similar issues, particularly with decision trees because of their step-like behavior resulting in infinite gradients.

Preparing data to plot contours in Matplotlib's Basemap

I'm having a hard time with plotting a basemap with Matplotlib and I'm fairly new to it so I was hoping for some help.
I have data of the format:
[ (lat1, lon1, data1),
(lat2, lon2, data2),
(lat3, lon3, data3),
...
(latN, lonN, dataN) ]
And here is some sample data:
(32.0, -128.5, 3.99)
(31.0, -128.0, 3.5027272727272734)
(31.5, -128.0, 3.7383333333333333)
(32.0, -128.0, 3.624)
(32.5, -128.0, 3.913157894736842)
(33.0, -128.0, 4.443333333333334)
Finally, here are some basic statistics about my data that I'm planning to plot:
LAT MIN: 22
LAT MAX: 50
LAT LEN: 1919
LON MIN: -128
LON MAX: -97
LON LEN: 1919
DATA MIN: 0
DATA MAX: 12
DATA LEN: 1919
I need to contour plot on a basemap of the continental United States. I can't, for the life of me, seem to figure out how to setup the data for plotting.
I read that the X-Axis (LATS) needs to be a np.array, and Y-Axis (LONS) needs to be an np.array and that Z (DATA) needs to be a MxN matrix where M = len(LATS) and N = len(LONS). So to me, I see Z as a diagonal matrix where the diagonal contains the data on the diagonal is the values found in DATA corresponding to the index of LATS and LONS.
Here is my code:
def show_map(self, a):
a = sorted(a, key = lambda entry: entry[0]) # sort by latitude
a = sorted(a, key = lambda entry: entry[1]) # then sort by longitude
lats = [ x[0] for x in a ]
lons = [ x[1] for x in a ]
data = [ x[2] for x in a ]
lat_min = min(lats)
lat_max = max(lats)
lon_min = min(lons)
lon_max = max(lons)
data_min = min(data)
data_max = max(data)
x = np.array(lats)
y = np.array(lons)
z = np.diag(data)
m = Basemap(
projection = 'merc',
llcrnrlat=lat_min, urcrnrlat=lat_max,
llcrnrlon=lon_min, urcrnrlon=lon_max,
rsphere=6371200., resolution='l', area_thresh=10000
lat_ts = 20, resolution = 'c'
)
fig = plt.figure()
plt.subplot(211)
ax = plt.gca()
# draw parallels
delat = 10.0
parallels = np.arange(0., 90, delat)
m.drawparallels(parallels, labels=[1,0,0,0], fontsize=10)
# draw meridians
delon = 10.
meridians = np.arange(180.,360.,delon)
m.drawmeridians(meridians,labels=[0,0,0,1],fontsize=10)
# draw map features
m.drawcoastlines(linewidth = 0.50)
m.drawcountries(linewidth = 0.50)
m.drawstates(linewidth = 0.25)
ny = z.shape[0]; nx = z.shape[1] # make grid
lo, la = m.makegrid(nx, ny)
X, Y = m(lo, la)
clevs = [0,1,2.5,5,7.5,10,15,20,30,40,50,70,100,150,200,250,300,400,500,600,750]
cs = m.contour(X, Y, z, clevs)
plt.show()
The plot I get, however, is this: http://imgur.com/li1Wg. I need something to this effect: http://matplotlib.org/basemap/_images/plotprecip.png
Can someone point out what I'm doing wrong and help me plot this? Thank You.
Thanks
I figured out how to do it. This is the code that I finally wrote, and I think this can help other users. If there is a better way of doing this, please state it, since I'm new to Matplotlib.
https://gist.github.com/3789221
Your linked gist is a solution but still wrong in another place.
In your question and in your linked gist you switched x and y coordinates with lon and lat.
x represents lon
y represents lat
Therefore you still get wrong results with your linked gist.
why are you writing:
z = np.diag(data)
From the documentation, numpy.diag(v, k=0) extracts a diagonal or construct a diagonal array.
That should be why you only get a "diagonal area" of values...