Julia - Understanding JuMP Gurobi outputs - optimization

I am often using Gurobi with JuMP and I noticed that there are still parts of its outputs that I don't understand. Unless it is documented somewhere and the link would be most welcome, could you help understand the following? :)
Found heuristic solution: objective 5820.0000000
Presolve removed 33 rows and 11 columns
Presolve time: 0.00s
Presolved: 607 rows, 331 columns, 2445 nonzeros
Variable types: 111 continuous, 220 integer (220 binary)
Root relaxation: objective 1.157500e+03, 64 iterations, 0.00 seconds
Nodes | Current Node | Objective Bounds | Work
Expl Unexpl | Obj Depth IntInf | Incumbent BestBd Gap | It/Node Time
0 0 1157.50000 0 25 5820.00000 1157.50000 80.1% - 0s
H 0 0 2535.0000000 1157.50000 54.3% - 0s
0 0 1236.00000 0 23 2535.00000 1236.00000 51.2% - 0s
0 0 1273.65351 0 41 2535.00000 1273.65351 49.8% - 0s
0 0 1274.59375 0 41 2535.00000 1274.59375 49.7% - 0s
0 0 1274.69841 0 42 2535.00000 1274.69841 49.7% - 0s
0 0 1309.98305 0 42 2535.00000 1309.98305 48.3% - 0s
0 0 1310.26027 0 42 2535.00000 1310.26027 48.3% - 0s
0 0 1340.01176 0 47 2535.00000 1340.01176 47.1% - 0s
0 0 1342.47826 0 49 2535.00000 1342.47826 47.0% - 0s
0 0 1342.60000 0 49 2535.00000 1342.60000 47.0% - 0s
0 0 1362.32468 0 50 2535.00000 1362.32468 46.3% - 0s
0 0 1363.08000 0 49 2535.00000 1363.08000 46.2% - 0s
0 0 1363.13077 0 49 2535.00000 1363.13077 46.2% - 0s
0 0 1370.79545 0 53 2535.00000 1370.79545 45.9% - 0s
0 0 1375.50000 0 52 2535.00000 1375.50000 45.7% - 0s
0 0 1375.50000 0 52 2535.00000 1375.50000 45.7% - 0s
0 0 1376.70025 0 52 2535.00000 1376.70025 45.7% - 0s
0 0 1376.70122 0 53 2535.00000 1376.70122 45.7% - 0s
0 0 1376.70122 0 53 2535.00000 1376.70122 45.7% - 0s
0 2 1376.98418 0 53 2535.00000 1376.98418 45.7% - 0s
* 255 157 14 2457.0000000 1473.00000 40.0% 22.5 0s
H 407 223 2397.0000000 1548.00000 35.4% 20.3 0s
* 1962 758 22 2355.0000000 1772.85714 24.7% 16.8 0s
*14326 2205 27 2343.0000000 2088.50000 10.9% 15.7 3s
From what I think I already know, in order of apparition:
Presolve
Found heuristic solution: objective 5820, Gurobi launched a presolve and found a solution of value 5820
Presolve removed 33 redundant or useless variables as well as 11 constraints?
Presvoled: sizes of the model presolved
Types of variables: some are binary and some are continuous.
Root relaxation: LP relaxation has an objective of 1 157.5
Nodes
Expl (1): * empty or H, I don't know the difference when there is these symbols but I noticed all rows with a given symbol (H, *, or empty) will have same outputs.
Expl and Unexpl number of explored nodes and unexplored nodes in the solving tree.
Current Node
Obj: objective of current node being explored?
Depth: tree's depth where solving is currently
IntInf: I have no idea for this one
Objective Bounds
Incumbent: current value of best valid solution found
BestBd: Best lower bound currently found
Gap: This is probably a gap between current best solution and best bound but I couldn't deduce how it is computed.
Work
It/Node: Here we need to find what is an iteration
Time: Time spent in solving until then.

The details can be found at https://www.gurobi.com/documentation/9.1/refman/mip_logging.html.
Let me just cite those that answer your question:
The Nodes section (the first two columns) provides general quantitative information on the progress of the search. The first column shows the number of branch-and-cut nodes that have been explored to that point, while the second shows the number of leaf nodes in the search tree that remain unexplored. At times, there will be an H or * character at the beginning of the output line. These indicate that a new feasible solution has been found, either by a MIP heuristic (H) or by branching (*).
The Current Node section provides information on the specific node that was explored at that point in the branch-and-cut tree. It shows the objective of the associated relaxation, the depth of that node in the branch-and-cut tree, and the number of integer variables that have non-integral values in the associated relaxation.
The Objective Bounds section provides information on the best known objective value for a feasible solution (i.e., the objective value of the current incumbent), and the current objective bound provided by leaf nodes of the search tree. The optimal objective value is always between these two values. The third column in this section (Gap) shows the relative gap between the two objective bounds. When this gap is smaller than the MIPGap parameter, optimization terminates.
The Work section of the log provides information on how much work has been performed to that point. The first column shows the average number of simplex iterations performed per node in the branch-and-cut tree. The final column shows the elapsed time since the solve began.
Looking at your log you are approaching very quickly to optimality and you get the Gap=0.01% solution in probably half minute or so.

Related

Media and Data Integrity Errors

I was wondering if anyone can tell me what these mean. From most people posting about them, there is no more than double digits. However, I have 1051556645921812989870080 Media and Data Integrity Errors on my SK hynix PC711 on my new HP dev one. Thanks!
Here's my entire smartctl output
`smartctl 7.3 2022-02-28 r5338 [x86_64-linux-6.0.7-arch1-1] (local build)
Copyright (C) 2002-22, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF INFORMATION SECTION ===
Model Number: SK hynix PC711 HFS001TDE9X073N
Serial Number: KDB3N511010503A37
Firmware Version: HPS0
PCI Vendor/Subsystem ID: 0x1c5c
IEEE OUI Identifier: 0xace42e
Total NVM Capacity: 1,024,209,543,168 [1.02 TB]
Unallocated NVM Capacity: 0
Controller ID: 1
NVMe Version: 1.3
Number of Namespaces: 1
Namespace 1 Size/Capacity: 1,024,209,543,168 [1.02 TB]
Namespace 1 Formatted LBA Size: 512
Namespace 1 IEEE EUI-64: ace42e 00254f98f1
Local Time is: Wed Nov 9 13:58:37 2022 EST
Firmware Updates (0x16): 3 Slots, no Reset required
Optional Admin Commands (0x001f): Security Format Frmw_DL NS_Mngmt Self_Test
Optional NVM Commands (0x005f): Comp Wr_Unc DS_Mngmt Wr_Zero Sav/Sel_Feat Timestmp
Log Page Attributes (0x1e): Cmd_Eff_Lg Ext_Get_Lg Telmtry_Lg Pers_Ev_Lg
Maximum Data Transfer Size: 64 Pages
Warning Comp. Temp. Threshold: 84 Celsius
Critical Comp. Temp. Threshold: 85 Celsius
Namespace 1 Features (0x02): NA_Fields
Supported Power States
St Op Max Active Idle RL RT WL WT Ent_Lat Ex_Lat
0 + 6.3000W - - 0 0 0 0 5 5
1 + 2.4000W - - 1 1 1 1 30 30
2 + 1.9000W - - 2 2 2 2 100 100
3 - 0.0500W - - 3 3 3 3 1000 1000
4 - 0.0040W - - 3 3 3 3 1000 9000
Supported LBA Sizes (NSID 0x1)
Id Fmt Data Metadt Rel_Perf
0 + 512 0 0
1 - 4096 0 0
=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
SMART/Health Information (NVMe Log 0x02)
Critical Warning: 0x00
Temperature: 34 Celsius
Available Spare: 100%
Available Spare Threshold: 5%
Percentage Used: 0%
Data Units Read: 13,162,025 [6.73 TB]
Data Units Written: 3,846,954 [1.96 TB]
Host Read Commands: 156,458,059
Host Write Commands: 128,658,566
Controller Busy Time: 116
Power Cycles: 273
Power On Hours: 126
Unsafe Shutdowns: 15
Media and Data Integrity Errors: 1051556645921812989870080
Error Information Log Entries: 0
Warning Comp. Temperature Time: 0
Critical Comp. Temperature Time: 0
Temperature Sensor 1: 34 Celsius
Temperature Sensor 2: 36 Celsius
Error Information (NVMe Log 0x01, 16 of 256 entries)
No Errors Logged`
Encountered a similar SMART reading from the same model.
I'm seeing a reported Media and Data Integrity Errors rate of a value that's over 2 ^ 84.
It could just be an error with its SMART implementation or the utility reading from it.
Converting your reported value of 1051556645921812989870080 to hex, we get 0xdead0000000000000000 big endian and 0x0000000000000000adde little endian.
Similarly, when I convert my value to hex, I get 0xffff0000000000000000 big endian and 0x0000000000000000ffff little endian, where f is just denotes a value other than 0.
I'm going to assume that the Media and Data Integrity Errors value has no actual meaning with regard to real errors. I doubt that both of us would have values that are padded with 16 0's when converted to hex. Something is sending/receiving/parsing bad data.
If you poke around the other reported SMART values in your post, and on my end, some of them don't seem to make much sense, either.

How to deal with the error when using Gurobi with cvxpy :Unable to retrieve attribute 'BarIterCount'

How to deal with the error when using Gurobi with cvxpy :AttributeError: Unable to retrieve attribute 'BarIterCount'.
I have an Integer programming problem, using cvxpy and set gurobi as a solver.
When the number of variables is small, the result is ok. After the number of variables reaches a level of like 43*13*6, then the error occurred. I suppose it may be caused by the scale of the problem, in which the gurobi solver can not estimate the BarIterCount, which is the max Iterations needed.
Thus, I wonder, is there any way to manually set the BarItercount attribute of gurobi through the interface of the CVX? Or whether there exists another way to solve this problem?
Thanks for any suggestions you may provide for me.
The trace log is as follows:
If my model is small, like I set a number which indicates the scale of model as 3, then the program is ok. The trace is :
Using license file D:\software\lib\site-packages\gurobipy\gurobi.lic
Restricted license - for non-production use only - expires 2022-01-13
Parameter OutputFlag unchanged
Value: 1 Min: 0 Max: 1 Default: 1
D:\software\lib\site-packages\cvxpy\reductions\solvers\solving_chain.py:326: DeprecationWarning: Deprecated, use Model.addMConstr() instead
solver_opts, problem._solver_cache)
Changed value of parameter QCPDual to 1
Prev: 0 Min: 0 Max: 1 Default: 0
Gurobi Optimizer version 9.1.0 build v9.1.0rc0 (win64)
Thread count: 16 physical cores, 32 logical processors, using up to 32 threads
Optimize a model with 126 rows, 370 columns and 2689 nonzeros
Model fingerprint: 0x70d49530
Variable types: 0 continuous, 370 integer (369 binary)
Coefficient statistics:
Matrix range [1e+00, 7e+00]
Objective range [1e+00, 1e+00]
Bounds range [1e+00, 1e+00]
RHS range [1e+00, 6e+00]
Found heuristic solution: objective 7.0000000
Presolve removed 4 rows and 90 columns
Presolve time: 0.01s
Presolved: 122 rows, 280 columns, 1882 nonzeros
Variable types: 0 continuous, 280 integer (279 binary)
Root relaxation: objective 4.307692e+00, 216 iterations, 0.00 seconds
Nodes | Current Node | Objective Bounds | Work
Expl Unexpl | Obj Depth IntInf | Incumbent BestBd Gap | It/Node Time
0 0 4.30769 0 49 7.00000 4.30769 38.5% - 0s
H 0 0 6.0000000 4.30769 28.2% - 0s
0 0 5.00000 0 35 6.00000 5.00000 16.7% - 0s
0 0 5.00000 0 37 6.00000 5.00000 16.7% - 0s
0 0 5.00000 0 7 6.00000 5.00000 16.7% - 0s
Cutting planes:
Gomory: 4
Cover: 9
MIR: 4
StrongCG: 1
GUB cover: 9
Zero half: 1
RLT: 1
Explored 1 nodes (849 simplex iterations) in 0.12 seconds
Thread count was 32 (of 32 available processors)
Solution count 2: 6 7
Optimal solution found (tolerance 1.00e-04)
Best objective 6.000000000000e+00, best bound 6.000000000000e+00, gap 0.0000%
If the number is 6, then error occurs:
-------------------------------------------------------
Using license file D:\software\lib\site-packages\gurobipy\gurobi.lic
Restricted license - for non-production use only - expires 2022-01-13
Parameter OutputFlag unchanged
Value: 1 Min: 0 Max: 1 Default: 1
D:\software\lib\site-packages\cvxpy\reductions\solvers\solving_chain.py:326: DeprecationWarning: Deprecated, use Model.addMConstr() instead
solver_opts, problem._solver_cache)
Changed value of parameter QCPDual to 1
Prev: 0 Min: 0 Max: 1 Default: 0
Gurobi Optimizer version 9.1.0 build v9.1.0rc0 (win64)
Thread count: 16 physical cores, 32 logical processors, using up to 32 threads
Traceback (most recent call last):
File "model.py", line 274, in <module>
problem.solve(solver=cp.GUROBI,verbose=True)
File "D:\software\lib\site-packages\cvxpy\problems\problem.py", line 396, in solve
return solve_func(self, *args, **kwargs)
File "D:\software\lib\site-packages\cvxpy\problems\problem.py", line 754, in _solve
self.unpack_results(solution, solving_chain, inverse_data)
File "D:\software\lib\site-packages\cvxpy\problems\problem.py", line 1058, in unpack_results
solution = chain.invert(solution, inverse_data)
File "D:\software\lib\site-packages\cvxpy\reductions\chain.py", line 79, in invert
solution = r.invert(solution, inv)
File "D:\software\lib\site-packages\cvxpy\reductions\solvers\qp_solvers\gurobi_qpif.py", line 59, in invert
s.NUM_ITERS: model.BarIterCount,
File "src\gurobipy\model.pxi", line 343, in gurobipy.gurobipy.Model.__getattr__
File "src\gurobipy\model.pxi", line 1842, in gurobipy.gurobipy.Model.getAttr
File "src\gurobipy\attrutil.pxi", line 100, in gurobipy.gurobipy.__getattr
AttributeError: Unable to retrieve attribute 'BarIterCount'
Hopefully this can provide more hint for solution.
BarIterCount is the number of barrier iterations performed to solve an LP. This is not a limit on the number of iterations and it should only be queried when the current optimization process has been finished. You cannot set this attribute either, of course.
To actually limit the number of iterations the barrier algorithm is allowed to take, you can use the parameter BarIterLimit.
Please inspect your log file for further information about the solver's behavior.

CBC Hangs After Finding Optimal Solution

I am trying to use CBC to solve an mps file. It frequently hangs for long periods of time and it will exit without any message.
It will also hang even after it has found the optimal solution. I don't know why it isn't returning to prompt.
I've reinstalled the latest version 2.10, and I've tried on multiple Windows 10 machines.
Prompt inputs and outputs below.
C:\my_directory>cbc
Welcome to the CBC MILP Solver
Version: Trunk (unstable)
Build Date: May 3 2019
CoinSolver takes input from arguments ( - switches to stdin)
Enter ? for list of commands or help
Coin:verbose 15
verbose was changed from 0 to 15
Coin:import SF_STD.mps
At line 8 NAME swolf_co
At line 9 ROWS
At line 179948 COLUMNS
At line 539250 RHS
At line 539271 BOUNDS
At line 539402 ENDATA
Problem swolf_co has 179937 rows, 191943 columns and 619924 elements
Coin0008I swolf_co read with 0 errors
Coin:stat
Presolve 18316 (-161621) rows, 19174 (-172769) columns and 138258 (-481666) elements
Statistics for presolved model
Original problem has 24 integers (24 of which binary)
Problem has 18316 rows, 19174 columns (2 with objective) and 138258 elements
There are 1 singletons with objective
Column breakdown:
19165 of type 0.0->inf, 6 of type 0.0->up, 0 of type lo->inf,
0 of type lo->up, 1 of type free, 0 of type fixed,
0 of type -inf->0.0, 2 of type -inf->up, 0 of type 0.0->1.0
Row breakdown:
9660 of type E 0.0, 0 of type E 1.0, 0 of type E -1.0,
1 of type E other, 6 of type G 0.0, 0 of type G 1.0,
1 of type G other, 8646 of type L 0.0, 0 of type L 1.0,
2 of type L other, 0 of type Range 0.0->1.0, 0 of type Range other,
0 of type Free
Coin:solv
Continuous objective value is 1.25864e+08 - 1.81 seconds
Cgl0002I 6 variables fixed
Cgl0004I processed model has 18298 rows, 19156 columns (0 integer (0 of which binary)) and 137182 elements
Cbc3007W No integer variables - nothing to do
Cuts at root node changed objective from 1.25864e+08 to -1.79769e+308
Probing was tried 0 times and created 0 cuts of which 0 were active after adding rounds of cuts (0.000 seconds)
Gomory was tried 0 times and created 0 cuts of which 0 were active after adding rounds of cuts (0.000 seconds)
Knapsack was tried 0 times and created 0 cuts of which 0 were active after adding rounds of cuts (0.000 seconds)
Clique was tried 0 times and created 0 cuts of which 0 were active after adding rounds of cuts (0.000 seconds)
MixedIntegerRounding2 was tried 0 times and created 0 cuts of which 0 were active after adding rounds of cuts (0.000 seconds)
FlowCover was tried 0 times and created 0 cuts of which 0 were active after adding rounds of cuts (0.000 seconds)
TwoMirCuts was tried 0 times and created 0 cuts of which 0 were active after adding rounds of cuts (0.000 seconds)
ZeroHalf was tried 0 times and created 0 cuts of which 0 were active after adding rounds of cuts (0.000 seconds)
Result - Optimal solution found
Objective value: 125864301.49648696
Enumerated nodes: 0
Total iterations: 0
Time (CPU seconds): 905.83
Time (Wallclock seconds): 905.85
Everything before Result - Optimal solution found ran quickly, but then it prints Cuts at root node changed objective from 1.25864e+08 to -1.79769e+308, and runs for a very long time. Turning cuts off did not change anything. That specific line no longer appears, but the objective still goes from 1.25864e+08 to -1.06e15. The model then takes about an hour to get back to the actual objective value of 1.25864e+08, which is the integer optimal solution as well as the solution to LP relaxation.
It solves in clp in several seconds. The mps file was written by glpk, and solves in several seconds in CPLEX.
Any guidance you could provide would be greatly appreciated.
I've tried multiple version of cbc and I still get this long hang time for relatively simply problems. I've tried .lp and .mps formats (clp seems to solve .lp faster), but nothing changes.

Rasterizing Paths data from Photoshop file

I was able to read the paths data from a Photoshop file.Photoshop File Format. The curves bezier curves. I want to convert this data into pixel format. How do i do this?.
Read thouroughly the documentation given On the Adobe's Website. I separated the data as 26 byte records.
Let's say one of the record is as follows
0 0 | 0 12 0 0 0 1 0 0 | 0 0 0 0 0 0 0 0 | 0 0 0 0 0 0 0 0
The first two bytes of each record is a selector to indicate what kind of path it is. 0 0 indicates that it is a Closed subpath length record.
The next 8 Bytes tell us the control point for the Bezier segment preceding the knot. Now this can again be split into two components X and Y
the first 4 bytes are the vertical components.
0 12 0 0
I converted 12 0 0 to binary format and add them and them convert to decimal
00001100 + 00000000 + 00000000 = 00001100
and then converted the result back to decimal. Which gave me Y co-ordinate.
where 0 indicates that the position is in the positive range(Signed magnitude form).
The next 8 bytes indicate the the anchor point for the knot, and the last 8 bytes the control point for the Bezier segment leaving the knot. The X and Y Components can be found in a similar manner.
I had this data exported to a svg file and then ran a rasterizer to convert the point data to pixel data.
If someone comes across this post I Hope this helps. :)

optaplanner vrp file with road time and time window

I am trying to create VRP file which defines a problem with time window and distance in seconds. I currently do not need capacity (can I turn it off?)
this is my file :
NAME: almirs-test
COMMENT: Generated for OptaPlanner Examples
TYPE: CVRPTW
DIMENSION: 2
EDGE_WEIGHT_TYPE: EXPLICIT
EDGE_WEIGHT_FORMAT: FULL_MATRIX
EDGE_WEIGHT_UNIT_OF_MEASUREMENT: sec
CAPACITY: 125
NODE_COORD_SECTION
0 0 0 BRUSSEL
55 1 1 ANTHISNES
EDGE_WEIGHT_SECTION
0.0 1
1 0.0
DEMAND_SECTION
0 0 0 100 0
55 1 0 10 1
DEPOT_SECTION
0
-1
EOF
it is corcectly parsed, and I see locations on screen, but when I try to solve it I get message : "Not feasible"
org.optaplanner.examples.vehiclerouting.solver/arrivalAfterDueTime/level0/[ANTHISNES]=-990
any idea what am I doing wrong? any samples where I can see how it is done?
thanks
almir