I am optimizing a MILP model in CPLEX (Python interface) that takes a long time to solve. Sometimes I need to do concurrent runs if my timelimit runs out. In order to continue optimizing with a solution from a previous, unfinished, run I usually provide the .sol file as a warm start.
Now I have a change in the objective function coefficients. The model's constraints and variables stay the same. Is it possible to provide a solution from the 'old', already optimized model to the model with the revised coefficients? Will CPLEX find the optimal solution of the new model faster than just starting fresh, regardless if it is is in the same 'range' as the old solution? And can I provide the .sol file for this as usual or should I use an .mst file?
On a related note, I am finding that when I use a previous solution as a warm start, CPLEX does use the best integer value found previously but often starts with a higher best bound. So the gap initially is higher than what previously has already been reached. Is there a method to overcome this, possibly speeding up the run?
you can warmstart with a .sol file but also with APIS
warm start through API in https://www.linkedin.com/pulse/making-optimization-simple-python-alex-fleischer/
from docplex.mp.model import Model
mdl = Model(name='buses')
nbbus40 = mdl.integer_var(name='nbBus40')
nbbus30 = mdl.integer_var(name='nbBus30')
mdl.add_constraint(nbbus40*40 + nbbus30*30 >= 300, 'kids')
mdl.minimize(nbbus40*500 + nbbus30*400)
warmstart=mdl.new_solution()
warmstart.add_var_value(nbbus40,8)
warmstart.add_var_value(nbbus30,0)
mdl.add_mip_start(warmstart)
sol=mdl.solve(log_output=True)
for v in mdl.iter_integer_vars():
print(v," = ",v.solution_value)
Sometimes warmstart helps, some times it does not. Fixed start helps too sometimes.(The search space is smaller so the search can be faster)
You say you are using ".sol" file as mip start, but there is a dedicated format for mip starts: MST format in ".mst" files. If you are using DOcplex, from a solution produced by Model.solve(), you can export a mst file using export_as_mst, which takes a WriteLevel argument.
This enumerated value controls what is written in the MST file: by default, only discrete values are used in the mip start, which avoids precision issues.
This is one of the reasons you should prefer MST format for mip starts.
Related
In Julia JuMP, I am wondering whether it is possible to access an array, or similar, of the GAP history, the GAP improvements.
I know that at the end of the optimization, I can access the final gap with relative_gap(m) with m of type Model and I want the history of improvements over the solving process.
It could be for instance a vector of Float64 in percent: gaps=[20.2, 16.7, 13.8, 5.1, 0]. In such case my problem is solved to optimality as the final gap is 0.
At the end, I would like to plot the gap improvement in function of time of solving. So maybe all values of gaps could be a couple of two elements, the gap and the solving time until this new gap improvement?
Maybe this is not possible in JuMP so you could answer for Gurobi ?
Thanks!
The Gurobi callback solution has been answered by Oscar at another post - see Julia JUMP Gurobi MIP - query and store best objective and bound at runtime.
However, since Gurobi is very serious about its TimeLimit and has low times to continue computations there is also a simpler approach (this code assumes mo is your JuMP Gurobi model):
set_optimizer_attribute(mo, "TimeLimit", 2.0)
gap_history = Float64[]
max_steps = 1800
for t in 1:max_steps
optimize!(mo)
gap = MOI.get(mo, MOI.RelativeGap())
status=termination_status(mo)
push!(gap_history, gap)
gap > 0.01 || status == MOI.TIME_LIMIT || break
end
It's not possible using JuMP in general.
You could do this in Gurobi using a solver-specific callback: https://github.com/jump-dev/Gurobi.jl#callbacks
Or you could just parse the Gurobi log file that gets printed.
In general though, this information isn't all that useful as it can be quite noisy. Why do you want it?
I'm working with point clouds taken with a Kinect. My goal is the total registration for 3D mapping of places or crops. I'm using the multiway registration code.
I'm wondering if there is a way to change the number of iterations of this code? I've seen that it only does 30 iterations by default.
What kind of iterations do you mean, the iterations performed by ICP for registration or the iterations performed during global optimization?
You can change the number of iterations for global optimization by adjusting the global optimization convergence criteria.
Instead of typing
o3d.registration.global_optimization(
pose_graph, o3d.registration.GlobalOptimizationLevenbergMarquardt(),
o3d.registration.GlobalOptimizationConvergenceCriteria(), option)
write
o3d.registration.global_optimization(
pose_graph, o3d.registration.GlobalOptimizationLevenbergMarquardt(),
o3d.registration.GlobalOptimizationConvergenceCriteria(max_iteration_lm=number_of_iterations), option)
For ICP, it works in a similar way by adjusting the ICP convergence criteria:
result_icp = o3d.registration.registration_icp(source, target,
max_correspondence_distance_coarse, np.identity(4),
o3d.registration.TransformationEstimationPointToPlane(),
o3d.registration.ICPConvergenceCriteria(max_iteration=number_of_iterations))
Hope this could help!
What's the difference between using
scipy.sparse.linalg.factorized(A)
and
scipy.sparse.linalg.splu(A)
Both of them return objects with .solve(rhs) method and for both it's said in the documentation that they use LU decomposition. I'd like to know the difference in performance for both of them.
More specificly, I'm writing a python/numpy/scipy app that implements dynamic FEM model. I need to solve an equation Au = f on each timestep. A is sparse and rather large, but doesn't depend on timestep, so I'd like to invest some time beforehand to make iterations faster (there may be thousands of them). I tried using scipy.sparse.linalg.inv(A), but it threw memory exceptions when the size of matrix was large. I used scipy.linalg.spsolve on each step until recently, and now am thinking on using some sort of decomposition for better performance. So if you have other suggestions aside from LU, feel free to propose!
They should both work well for your problem, assuming that A does not change with each time step.
scipy.sparse.linalg.inv(A) will return a dense matrix that is the same size as A, so it's no wonder it's throwing memory exceptions.
scipy.linalg.solve is also a dense linear solver, which isn't what you want.
Assuming A is sparse, to solve Au=f and you only want to solve Au=f once, you could use scipy.sparse.linalg.spsolve. For example
u = spsolve(A, f)
If you want to speed things up dramatically for subsequent solves, you would instead use scipy.sparse.linalg.factorized or scipy.sparse.linalg.splu. For example
A_inv = splu(A)
for t in range(iterations):
u_t = A_inv.solve(f_t)
or
A_solve = factorized(A)
for t in range(iterations):
u_t = A_solve(f_t)
They should both be comparable in speed, and much faster than the previous options.
As #sascha said, you will need to dig into the documentation to see the differences between splu and factorize. But, you can use 'umfpack' instead of the default 'superLU' if you have it installed and set up correctly. I think umfpack will be faster in most cases. Keep in mind that if your matrix A is too large or has too many non-zeros, an LU decomposition / direct solver may take too much memory on your system. In this case, you might be stuck with using an iterative solver such as this. Unfortunately, you wont be able to reuse the solve of A at each time step, but you might be able to find a good preconditioner for A (approximation to inv(A)) to feed the solver to speed it up.
I have a problem that is solved successfully with ipopt and fmincon. worhp terminates on local infeasibility. My x0 (init) is feasible.
This may happen with the interior point algorithm, but I expect sqp to always stay in the feasible zone?
Maybe also check the derivatives with WORHP by enabling CheckValuesDF, CheckValuesDG, CheckValuesHM, CheckStructureDF, CheckStructureDG and CheckStructureHM if you provide them. What I am pointing at is that WORHP requires a very special coordinate storage format (in particular for the Hessian). Mistakes here lead to false search directions.
Due to the approximation error of the QP subproblem this is not something you can expect in general. Consider the problem
which will have the QP subproblems
for a current x and Lagrangian multiplier lambda, as can be seen by determining the necessary derivatives. With initial values x_0 = 0 and lambda_0 = 1 we have a feasible initial guess. The first QP to be solved is then
which has the unique solution d = 2. Now, depending on the implemented linesearch, the full step might be taken, i.e. the next iterate is x_1 = x_0 + d. That means x_1 = 2 which is not a feasible point anymore. In fact, WORHP's SQP algorithm will iterate like this if you disable the par.InitialLMest and eventually find the global optimum at x = 1.
Apart from this fundamental property there can also be other effects leading to iterates leaving the feasible set, that will very much be specific to the actual solver implementation. For example numerical inaccuracies, difficulties during the solution of a QP or certain recovery strategies. As to why your problem is not solved successfully using the SQP algorithm of WORHP, I am unable to say much without knowing anything about the problem itself.
I use Gensim Word2Vec to train word sets in my database.
I have about 400,000 phrase(Each phrase is short. Total 700MB) in my PostgreSQL database.
This is how I train these data using Django ORM:
post_vector_list = []
for post in Post.objects.all():
post_vector = my_tokenizer(post.category.name)
post_vector.extend(my_tokenizer(post.title))
post_vector.extend(my_tokenizer(post.contents))
post_vector_list.append(post_vector)
word2vec_model = gensim.models.Word2Vec(post_vector_list, window=10, min_count=2, size=300)
But this job getting a lot of time and feels like not efficient.
Especially, creating post_vector_list part took a lot of time and space..
I want to improve speed of training but have no idea how to do.
Want to get your advices. Thanks.
To optimize such code, you need to collect good information about where the time is spent.
Is most of the time spent preparing post_vector_list?
If so, you will want to make sure my_tokenizer (whose code is not shown) is as efficient as possible. You may want to try to minimize the number of extend()s and append()s that are done on large lists. You might have to even take a look at your DB's configuration or options to speed up the DB-to-Object mapping started inside Post.objects.all().
Is most of the time spent in the call to Word2Vec()?
If so, other steps may help:
ensure you're using gensim's Cython-optimized routines – if not, you should be seeing a logged warning (and training will be up to 100X slower)
consider using a workers=4 or workers=8 optional argument to use more threads, if your machine has at least 4 or 8 CPU cores
consider using a larger min_count, which speeds training somewhat (and since vectors for words where there are only a few examples typically aren't very good anyway, doesn't lose much and can even improve the quality of the surviving words)
consider using a smaller window, since training takes longer for larger windows
consider using a smaller vector_size (previously called size), since training takes longer for larger-size vectors
consider using a more-aggressive (smaller) value for the optional sample argument, which randomly skips more of the most-frequent words. The default is 1e-04, but values of 1e-05 or 1e-06 (especially on larger corpuses) can offer additional speedup, and even often improve the final vectors (by spending relatively less training time on words with an excess of usage examples)
consider using a lower-than-default (5) value for the optional epochs parameter (previously called iter). (I wouldn't recommend this unless the corpus is very large – so it already has many redundant, equally-good examples of the same words throughout.)
you could use a python generator instead of loading all the data into the list. Gensim works with python generators too. The code will look something like this
class Post_Vectors(object):
def __init__(self, Post):
self.Post = Post
def __iter__(self):
for post in Post.objects.all():
post_vector = my_tokenizer(post.category.name)
post_vector.extend(my_tokenizer(post.title))
post_vector.extend(my_tokenizer(post.contents))
yield post_vector
post_vectors = Post_Vectors(Post)
word2vec_model = gensim.models.Word2Vec(post_vectors, window=10, min_count=2, size=300, workers=??)
For the gensim speedup, if you have a multi-core CPU, you could use the workers parameter. (By default it is 3)