I execute a program in cplex python (docplex), it has arrived at a gap 48% with 41 solutions. it's already been 2 days, I ask if I can interrupt it and have a result without restarting the execution with limit gap.
If you run on Windows you could try CTRL C
If that does not work,
what you could do is run again your model with 1 new solution each time and then save the current solution and then each time you abort you have the latest solution
Example with the zoo story:
from docplex.mp.model import Model
from docplex.mp.progress import *
mdl = Model(name='buses')
nbbus40 = mdl.integer_var(name='nbBus40')
nbbus30 = mdl.integer_var(name='nbBus30')
mdl.add_constraint(nbbus40*40 + nbbus30*30 >= 300, 'kids')
mdl.minimize(nbbus40*500 + nbbus30*400)
mdl.parameters.mip.limits.solutions=1
while mdl.solve(log_output=False):
for v in mdl.iter_integer_vars():
print(v," = ",v.solution_value)
print("status : ",mdl.solve_details.status)
if ("optimal solution" in str(mdl.solve_details.status)):
break
that gives
nbBus40 = 8.0
nbBus30 = 0
status : solution limit exceeded
nbBus40 = 7.0
nbBus30 = 1.0
status : solution limit exceeded
nbBus40 = 6.0
nbBus30 = 2.0
status : integer optimal solution
Related
!apt install cplex-utils
!pip install cplex
solver = SolverFactory('cplex')
res_NLP= solver.solve(HN_model)
The error is:
WARNING: Could not locate the 'cplex' executable, which is required
for solver
cplex
--------------------------------------------------------------------------- ApplicationError Traceback (most recent call
last) in ()
1 solver = SolverFactory('cplex')
----> 2 res_NLP= solver.solve(HN_model)
2 frames
/usr/local/lib/python3.7/dist-packages/pyomo/opt/solver/shellcmd.py in
available(self, exception_flag)
123 if exception_flag:
124 msg = "No executable found for solver '%s'"
--> 125 raise ApplicationError(msg % self.name)
126 return False
127 return True
ApplicationError: No executable found for solver 'cplex'
Within IBM Watson Studio, CPLEX comes pre-installed in the Notebooks. But with other Notebook cloud providers, you need to find a way to install it, or else call CPLEX as a service in the IBM Cloud.
You could try to use dowml : https://xavier-nodet.medium.com/submit-decision-optimization-jobs-to-wml-using-dowml-be26e0de6b7f
Or directly wml : https://pypi.org/project/ibm-watson-machine-learning/
With google colab
!pip install cplex
!pip install docplex
from docplex.mp.model import Model
mdl = Model(name='buses')
nbbus40 = mdl.integer_var(name='nbBus40')
nbbus30 = mdl.integer_var(name='nbBus30')
mdl.add_constraint(nbbus40*40 + nbbus30*30 >= 300, 'kids')
mdl.minimize(nbbus40*500 + nbbus30*400)
mdl.export("buses.lp")
!cat buses.lp
works fine and gives
Requirement already satisfied: cplex in /usr/local/lib/python3.7/dist-packages (20.1.0.1)
Requirement already satisfied: docplex in /usr/local/lib/python3.7/dist-packages (2.22.213)
Requirement already satisfied: six in /usr/local/lib/python3.7/dist-packages (from docplex) (1.15.0)
\ This file has been generated by DOcplex
\ ENCODING=ISO-8859-1
\Problem name: buses
Minimize
obj: 500 nbBus40 + 400 nbBus30
Subject To
kids: 40 nbBus40 + 30 nbBus30 >= 300
Bounds
Generals
nbBus40 nbBus30
End
From the error message, SolverFactory seems to be a Pyomo class, and require the CPLEX Interactive executable program to be available locally on the machine where the Pyomo code is executed.
Unless you have a way to install arbitrary executable files on the platform you use, which I very highly doubt if you're not using your own computer, you will have to find another way. Alex's answer proposes two...
When you're working in the colab you need to install cplex from pip. When you install cplex from pip, you need to use the cplex_direct interface in pyomo in order to avoid such errors, since the cplex interface will use the shell approach to solve the problem.
Using Google Colab, this should work
!pip install pyomo -q
!pip install cplex -q
import pyomo.environ as pyo
model = pyo.ConcreteModel()
model.s = pyo.Set(initialize=[1,2,3,4,5])
model.x = pyo.Var(model.s, domain=pyo.NonNegativeReals)
model.c = pyo.Constraint(expr=model.x[model.s.last()]>=5)
model.obj = pyo.Objective(expr=sum(model.x[s] for s in model.s), sense=pyo.minimize)
solver = pyo.SolverFactory('cplex_direct')
solver.solve(model)
model.x.display()
x : Size=5, Index=s
Key : Lower : Value : Upper : Fixed : Stale : Domain
1 : 0 : 0.0 : None : False : False : NonNegativeReals
2 : 0 : 0.0 : None : False : False : NonNegativeReals
3 : 0 : 0.0 : None : False : False : NonNegativeReals
4 : 0 : 0.0 : None : False : False : NonNegativeReals
5 : 0 : 5.0 : None : False : False : NonNegativeReals
I don't use CPLEX a lot, therefore, I'm not fully sure, but I think this free approach should have a limit in the number of var, constraints or others
Let us assume that I have created a mathematical model in python and want to solve it using the below code (the docplex library.)
start = time.perf_counter() # CPU time calculator of the CPLEX solver
# Obj 1
mdl.minimize(obj1)
solution = mdl.solve(log_output=True)
if (solution is not None) and (solution.is_feasible_solution()):
lb[0] = obj1.solution_value
if obj2.solution_value > ub[1]: ub[1] = obj2.solution_value
if obj3.solution_value > ub[2]: ub[2] = obj3.solution_value
sol[0, 0] = obj1.solution_value
sol[0, 1] = obj2.solution_value
sol[0, 2] = obj3.solution_value
sol[0, 3] = round(time.perf_counter() - start, 3)
Given that I have set mdl.time_limit=480, why could the time recorded in sol[0, 3] be greater than 480 seconds?
Thanks!
Within CPLEX docplex you can get solve time:
from docplex.mp.model import Model
mdl = Model(name='buses')
nbbus40 = mdl.integer_var(name='nbBus40')
nbbus30 = mdl.integer_var(name='nbBus30')
mdl.add_constraint(nbbus40*40 + nbbus30*30 >= 300, 'kids')
mdl.minimize(nbbus40*500 + nbbus30*400)
mdl.solve(log_output=True,)
mdl.export("c:\\temp\\buses.lp")
for v in mdl.iter_integer_vars():
print(v," = ",v.solution_value)
print(mdl.solve_details)
print("solve time =",mdl.solve_details.time)
gives
status = integer optimal solution
time = 0.109 s.
problem = MILP
gap = 0%
solve time = 0.10900000005494803
To answer your initial question, The time_limit parameter applies only to the solve call, and is handled internally by CPLEX, counting exclusively solve time.
The Python time module, on the other hand, counts time in a different manner, including Python code that is executed after the call to solve but before the second call to time.time()
I would like to estimate IRT model using PyMC3.
I generated data with the following distribution:
alpha_fix = 4
beta_fix = 100
theta= np.random.normal(100,15,1000)
prob = np.exp(alpha_fix*(theta-beta_fix))/(1+np.exp(alpha_fix*(theta-beta_fix)))
prob_tt = tt._shared(prob)
Then I created a model using PyMC3 to infer the parameter:
irt = pm.Model()
with irt:
# Priors
alpha = pm.Normal('alpha',mu = 4 , tau = 1)
beta = pm.Normal('beta',mu = 100 , tau = 15)
thau = pm.Normal('thau' ,mu = 100 , tau = 15)
# Modelling
p = pm.Deterministic('p',tt.exp(alpha*(thau-beta))/(1+tt.exp(alpha*(thau-beta))))
out = pm.Normal('o',p,observed = prob_tt)
Then I infer through the model:
with irt:
mean_field = pm.fit(10000,method='advi', callbacks=[pm.callbacks.CheckParametersConvergence(diff='absolute')])
Finally, Sample from the model to get compute posterior:
pm.plot_posterior(mean_field.sample(1000), color='LightSeaGreen');
But the results of the "alpha" (mean of 2.2) is relatively far from the expected one (4) even though the prior on alpha was well-calibrated.
Would you have an idea of the origin of this gap and how to fix it?
Thanks a lot,
out = pm.Normal('o',p,observed = prob_tt)
Why you are using Normal instead of Bernoulli ? Also, what is the variance of normal ?
Been trying to solve the newtonian two-body problem using RK45 from scipy however keep running into the TypeError:'Required step size is less than spacing between numbers.' I've tried different values of t_eval than the one below but nothing seems to work.
from scipy import optimize
from numpy import linalg as LA
import matplotlib.pyplot as plt
from scipy.optimize import fsolve
import numpy as np
from scipy.integrate import solve_ivp
AU=1.5e11
a=AU
e=0.5
mss=2E30
ms = 2E30
me = 5.98E24
mv=4.867E24
yr=3.15e7
h=100
mu1=ms*me/(ms+me)
mu2=ms*me/(ms+me)
G=6.67E11
step=24
vi=np.sqrt(G*ms*(2/(a*(1-e))-1/a))
#sun=sphere(pos=vec(0,0,0),radius=0.1*AU,color=color.yellow)
#earth=sphere(pos=vec(1*AU,0,0),radius=0.1*AU)
sunpos=np.array([-903482.12391302, -6896293.6960525, 0. ])
earthpos=np.array([a*(1-e),0,0])
earthv=np.array([0,vi,0])
sunv=np.array([0,0,0])
def accelerations2(t,pos):
norme=sum( (pos[0:3]-pos[3:6])**2 )**0.5
gravit = G*(pos[0:3]-pos[3:6])/norme**3
sunaa = me*gravit
earthaa = -ms*gravit
tota=earthaa+sunaa
return [*earthaa,*sunaa]
def ode45(f,t,y,h):
"""Calculate next step of an initial value problem (IVP) of an ODE with a RHS described
by the RHS function with an order 4 approx. and an order 5 approx.
Parameters:
t: float. Current time.
y: float. Current step (position).
h: float. Step-length.
Returns:
q: float. Order 2 approx.
w: float. Order 3 approx.
"""
s1 = f(t, y[0],y[1])
s2 = f(t + h/4.0, y[0] + h*s1[0]/4.0,y[1] + h*s1[1]/4.0)
s3 = f(t + 3.0*h/8.0, y[0] + 3.0*h*s1[0]/32.0 + 9.0*h*s2[0]/32.0,y[1] + 3.0*h*s1[1]/32.0 + 9.0*h*s2[1]/32.0)
s4 = f(t + 12.0*h/13.0, y[0] + 1932.0*h*s1[0]/2197.0 - 7200.0*h*s2[0]/2197.0 + 7296.0*h*s3[0]/2197.0,y[1] + 1932.0*h*s1[1]/2197.0 - 7200.0*h*s2[1]/2197.0 + 7296.0*h*s3[1]/2197.0)
s5 = f(t + h, y[0] + 439.0*h*s1[0]/216.0 - 8.0*h*s2[0] + 3680.0*h*s3[0]/513.0 - 845.0*h*s4[0]/4104.0,y[1] + 439.0*h*s1[1]/216.0 - 8.0*h*s2[1] + 3680.0*h*s3[1]/513.0 - 845.0*h*s4[1]/4104.0)
s6 = f(t + h/2.0, y[0] - 8.0*h*s1[0]/27.0 + 2*h*s2[0] - 3544.0*h*s3[0]/2565 + 1859.0*h*s4[0]/4104.0 - 11.0*h*s5[0]/40.0,y[1] - 8.0*h*s1[1]/27.0 + 2*h*s2[1] - 3544.0*h*s3[1]/2565 + 1859.0*h*s4[1]/4104.0 - 11.0*h*s5[1]/40.0)
w1 = y[0] + h*(25.0*s1[0]/216.0 + 1408.0*s3[0]/2565.0 + 2197.0*s4[0]/4104.0 - s5[0]/5.0)
w2 = y[1] + h*(25.0*s1[1]/216.0 + 1408.0*s3[1]/2565.0 + 2197.0*s4[1]/4104.0 - s5[1]/5.0)
q1 = y[0] + h*(16.0*s1[0]/135.0 + 6656.0*s3[0]/12825.0 + 28561.0*s4[0]/56430.0 - 9.0*s5[0]/50.0 + 2.0*s6[0]/55.0)
q2 = y[1] + h*(16.0*s1[1]/135.0 + 6656.0*s3[1]/12825.0 + 28561.0*s4[1]/56430.0 - 9.0*s5[1]/50.0 + 2.0*s6[1]/55.0)
return w1,w2, q1,q2
t=0
T=10**5
poss=[-903482.12391302, -6896293.6960525, 0. ,a*(1-e),0,0 ]
sol = solve_ivp(accelerations2, [0, 10**5], poss,t_eval=np.linspace(0,10**5,1))
print(sol)
Not sure what the error even means because I've tried many different t_evl and nothing seems to work.
The default values in solve_ivp are made for a "normal" situation where the scales of the variables are not too different from the range from 0.1 to 100. You could achieve these scales by rescaling the problem so that all lengths and related constants are in AU and all times and related constants are in days.
Or you can try to set the absolute tolerance to something reasonable like 1e-4*AU.
It also helps to use the correct first order system, as I told you recently in another question on this topic. In a mechanical system you get usually a second order ODE x''=a(x). Then the first order system to pass to the ODE solver is [x', v'] = [v, a(x)], which could be implemented as
def firstorder(t,state):
pos, vel = state.reshape(2,-1);
return [*vel, *accelerations2(t,pos)]
Next it is always helpful to apply the acceleration of Earth to Earth and of the sun to the sun. That is, fix an order of the objects. At the moment the initialization has the sun first, while in the acceleration computation you treat the state as Earth first. Switch all to sun first
def accelerations2(t,pos):
pos=pos.reshape(-1,3)
# pos[0] = sun, pos[1] = earth
norme=sum( (pos[1]-pos[0])**2 )**0.5
gravit = G*(pos[1]-pos[0])/norme**3
sunacc = me*gravit
earthacc = -ms*gravit
totacc=earthacc+sunacc
return [*sunacc,*earthacc]
And then it never goes amiss to use the correctly reproduced natural constants like
G = 6.67E-11
Then the solver call and print formatting as
state0=[*sunpos, *earthpos, *sunvel, *earthvel]
sol = solve_ivp(firstorder, [0, T], state0, first_step=1e+5, atol=1e-6*a)
print(sol.message)
for t, pos in zip(sol.t, sol.y[[0,1,3,4]].T):
print("%.6e"%t, ", ".join("%8.4g"%x for x in pos))
gives the short table
The solver successfully reached the end of the integration interval.
t x_sun y_sun x_earth y_earth
0.000000e+00 -9.035e+05, -6.896e+06, 7.5e+10, 0
1.000000e+05 -9.031e+05, -6.896e+06, 7.488e+10, 5.163e+09
that is, for this step the solver only needs one internal step.
I am trying to optimize MACD parameters for a trading strategy but unfortunately I am stuck with paramset.label value. This is the code:
################################# MACD PARAMETERS OPTIMIZATION
.fastMA <- (20:40)
.slowMA <- (30:70)
.nsamples = 10
strat.st <- 'volStrat'
# Paramset
add.distribution(strat.st,
paramset.label = 'EMA',
component.type = 'indicator',
component.label = 'macd.out',
variable = list(n = .fastMA),
label = 'nFast'
)
add.distribution(strat.st,
paramset.label = 'EMA',
component.type = 'indicator',
component.label = 'macd.out',
variable = list(n = .slowMA),
label = 'nSlow'
)
add.distribution.constraint(strat.st,
paramset.label = 'EMA',
distribution.label.1 = 'nFast',
distribution.label.2 = 'nSlow',
operator = '<',
label = 'nFast<nSlow'
)
results <- apply.paramset(strat.st,
paramset.label = 'EMA',
portfolio = portfolio2.st,
account = account.st,
nsamples = .nsamples,
verbose = TRUE)
stats <- results$tradeStats
print(stats)
When I run it, this error occurs for every sample:
evaluation # 1:
$param.combo
nFast nSlow
379 23 51
[1] "Processing param.combo 379"
nFast nSlow
379 23 51
result of evaluating expression:
<simpleError in strategy[[components.type]][[index]]: subscript out of bounds>
got results for task 1
numValues: 1, numResults: 1, stopped: FALSE
returning status FALSE
And then, for the last one, this is the error:
evaluation # 10:
$param.combo
nFast nSlow
585 40 60
[1] "Processing param.combo 585"
nFast nSlow
585 40 60
result of evaluating expression:
<simpleError in strategy[[components.type]][[index]]: subscript out of bounds>
got results for task 10
numValues: 10, numResults: 10, stopped: FALSE
first call to combine function
evaluating call object to combine results:
fun(result.1, result.2, result.3, result.4, result.5, result.6,
result.7, result.8, result.9, result.10)
error calling combine function:
<simpleError in fun(result.1, result.2, result.3, result.4, result.5, result.6, result.7, result.8, result.9, result.10): attempt to select less than one element>
numValues: 10, numResults: 10, stopped: TRUE
I really don't understand how can I fix it.
Can anyone how can I solve this?
Thank you so much
You didn't give the code before OPTIMIZATION part, so here is only my guess direction.
I understand you want to test 20:40 and 30:70, but in your OPTIMIZATION code, you add 2 distribution both pointing to " component.label = 'macd.out' ".
I did similar test, although they both use MA type indicators, they generally should not point to the same MA data(" component.label = 'macd.out' "), these code worked one distribution points to "component.label = 'fast'" and another points to "component.label = 'slow'" as they are pointing different datas so that they can be compared.
You can try to debug in this direction.