I am trying to create VRP file which defines a problem with time window and distance in seconds. I currently do not need capacity (can I turn it off?)
this is my file :
NAME: almirs-test
COMMENT: Generated for OptaPlanner Examples
TYPE: CVRPTW
DIMENSION: 2
EDGE_WEIGHT_TYPE: EXPLICIT
EDGE_WEIGHT_FORMAT: FULL_MATRIX
EDGE_WEIGHT_UNIT_OF_MEASUREMENT: sec
CAPACITY: 125
NODE_COORD_SECTION
0 0 0 BRUSSEL
55 1 1 ANTHISNES
EDGE_WEIGHT_SECTION
0.0 1
1 0.0
DEMAND_SECTION
0 0 0 100 0
55 1 0 10 1
DEPOT_SECTION
0
-1
EOF
it is corcectly parsed, and I see locations on screen, but when I try to solve it I get message : "Not feasible"
org.optaplanner.examples.vehiclerouting.solver/arrivalAfterDueTime/level0/[ANTHISNES]=-990
any idea what am I doing wrong? any samples where I can see how it is done?
thanks
almir
Related
I am trying to use CBC to solve an mps file. It frequently hangs for long periods of time and it will exit without any message.
It will also hang even after it has found the optimal solution. I don't know why it isn't returning to prompt.
I've reinstalled the latest version 2.10, and I've tried on multiple Windows 10 machines.
Prompt inputs and outputs below.
C:\my_directory>cbc
Welcome to the CBC MILP Solver
Version: Trunk (unstable)
Build Date: May 3 2019
CoinSolver takes input from arguments ( - switches to stdin)
Enter ? for list of commands or help
Coin:verbose 15
verbose was changed from 0 to 15
Coin:import SF_STD.mps
At line 8 NAME swolf_co
At line 9 ROWS
At line 179948 COLUMNS
At line 539250 RHS
At line 539271 BOUNDS
At line 539402 ENDATA
Problem swolf_co has 179937 rows, 191943 columns and 619924 elements
Coin0008I swolf_co read with 0 errors
Coin:stat
Presolve 18316 (-161621) rows, 19174 (-172769) columns and 138258 (-481666) elements
Statistics for presolved model
Original problem has 24 integers (24 of which binary)
Problem has 18316 rows, 19174 columns (2 with objective) and 138258 elements
There are 1 singletons with objective
Column breakdown:
19165 of type 0.0->inf, 6 of type 0.0->up, 0 of type lo->inf,
0 of type lo->up, 1 of type free, 0 of type fixed,
0 of type -inf->0.0, 2 of type -inf->up, 0 of type 0.0->1.0
Row breakdown:
9660 of type E 0.0, 0 of type E 1.0, 0 of type E -1.0,
1 of type E other, 6 of type G 0.0, 0 of type G 1.0,
1 of type G other, 8646 of type L 0.0, 0 of type L 1.0,
2 of type L other, 0 of type Range 0.0->1.0, 0 of type Range other,
0 of type Free
Coin:solv
Continuous objective value is 1.25864e+08 - 1.81 seconds
Cgl0002I 6 variables fixed
Cgl0004I processed model has 18298 rows, 19156 columns (0 integer (0 of which binary)) and 137182 elements
Cbc3007W No integer variables - nothing to do
Cuts at root node changed objective from 1.25864e+08 to -1.79769e+308
Probing was tried 0 times and created 0 cuts of which 0 were active after adding rounds of cuts (0.000 seconds)
Gomory was tried 0 times and created 0 cuts of which 0 were active after adding rounds of cuts (0.000 seconds)
Knapsack was tried 0 times and created 0 cuts of which 0 were active after adding rounds of cuts (0.000 seconds)
Clique was tried 0 times and created 0 cuts of which 0 were active after adding rounds of cuts (0.000 seconds)
MixedIntegerRounding2 was tried 0 times and created 0 cuts of which 0 were active after adding rounds of cuts (0.000 seconds)
FlowCover was tried 0 times and created 0 cuts of which 0 were active after adding rounds of cuts (0.000 seconds)
TwoMirCuts was tried 0 times and created 0 cuts of which 0 were active after adding rounds of cuts (0.000 seconds)
ZeroHalf was tried 0 times and created 0 cuts of which 0 were active after adding rounds of cuts (0.000 seconds)
Result - Optimal solution found
Objective value: 125864301.49648696
Enumerated nodes: 0
Total iterations: 0
Time (CPU seconds): 905.83
Time (Wallclock seconds): 905.85
Everything before Result - Optimal solution found ran quickly, but then it prints Cuts at root node changed objective from 1.25864e+08 to -1.79769e+308, and runs for a very long time. Turning cuts off did not change anything. That specific line no longer appears, but the objective still goes from 1.25864e+08 to -1.06e15. The model then takes about an hour to get back to the actual objective value of 1.25864e+08, which is the integer optimal solution as well as the solution to LP relaxation.
It solves in clp in several seconds. The mps file was written by glpk, and solves in several seconds in CPLEX.
Any guidance you could provide would be greatly appreciated.
I've tried multiple version of cbc and I still get this long hang time for relatively simply problems. I've tried .lp and .mps formats (clp seems to solve .lp faster), but nothing changes.
We recently migrated from MapReduce to TEZ for executing Hive queries on EMR. We are seeing cases where for the exact hive query launches very different number of mappers. See Map 3 phase below. On the first run it requested for 305 resources and on another run it requested for 4534 mappers. ( Please ignore the KILLED status because I manually killed the query.) Why does this happen ? How can we change it to be based on underlying data size instead ?
Run 1
----------------------------------------------------------------------------------------------
VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
----------------------------------------------------------------------------------------------
Map 1 container KILLED 5 0 0 5 0 0
Map 3 container KILLED 305 0 0 305 0 0
Map 5 container KILLED 16 0 0 16 0 0
Map 6 container KILLED 1 0 0 1 0 0
Reducer 2 container KILLED 333 0 0 333 0 0
Reducer 4 container KILLED 796 0 0 796 0 0
----------------------------------------------------------------------------------------------
VERTICES: 00/06 [>>--------------------------] 0% ELAPSED TIME: 14.16 s
----------------------------------------------------------------------------------------------
Run 2
----------------------------------------------------------------------------------------------
VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
----------------------------------------------------------------------------------------------
Map 1 .......... container SUCCEEDED 5 5 0 0 0 0
Map 3 container KILLED 4534 0 0 4534 0 0
Map 5 .......... container SUCCEEDED 325 325 0 0 0 0
Map 6 .......... container SUCCEEDED 1 1 0 0 0 0
Reducer 2 container KILLED 333 0 0 333 0 0
Reducer 4 container KILLED 796 0 0 796 0 0
----------------------------------------------------------------------------------------------
VERTICES: 03/06 [=>>-------------------------] 5% ELAPSED TIME: 527.16 s
----------------------------------------------------------------------------------------------
This article explains the process in which Tez allocates resources. https://cwiki.apache.org/confluence/display/TEZ/How+initial+task+parallelism+works
If Tez grouping is enabled for the splits, then a generic grouping
logic is run on these splits to group them into larger splits. The
idea is to strike a balance between how parallel the processing is and
how much work is being done in each parallel process.
First, Tez tries to find out the resource availability in the cluster for these tasks. For that, YARN provides a headroom value (and
in future other attributes may be used). Lets say this value is T.
Next, Tez divides T with the resource per task (say M) to find out how many tasks can run in parallel at one (ie in a single wave). W =
T/M.
Next W is multiplied by a wave factor (from configuration - tez.grouping.split-waves) to determine the number of tasks to be used.
Lets say this value is N.
If there are a total of X splits (input shards) and N tasks then this would group X/N splits per task. Tez then estimates the size of
data per task based on the number of splits per task.
If this value is between tez.grouping.max-size & tez.grouping.min-size then N is accepted as the number of tasks. If
not, then N is adjusted to bring the data per task in line with the
max/min depending on which threshold was crossed.
For experimental purposes tez.grouping.split-count can be set in configuration to specify the desired number of groups. If this config
is specified then the above logic is ignored and Tez tries to group
splits into the specified number of groups. This is best effort.
After this the grouping algorithm is executed. It groups splits by node locality, then rack locality, while respecting the group size
limits.
I am new to Spark and Scala. I am working on a Scala project where I will have data access from SQL Server.
There is a table in SQL Server has info about clothes. itemCode is the primary key and several attributes with Boolean value 0/1 - Designer, Exclusive, Handloom and several other columns having attributes of the product etc.
Code Designer Exclusive Handloom
A 1 0 1
B 1 0 0
C 0 0 1
D 0 1 0
E 0 1 0
F 1 0 1
G 0 1 0
H 0 0 0
I 1 1 1
J 1 1 1
K 0 0 1
L 0 1 0
M 0 1 0
N 1 1 0
O 0 1 1
P 1 1 0
and the list continues.
I have to select a collection of 32 items out of 320 items that have ATLEAST:
8 Designer, 8 Exclusive, 8 Handloom, 8 WeddingStyle, 8 PartyStyle,
8 Silk, 8 Georgette
I had solved the problem in MS Excel solver (it uses Gradient Descent algo) by adding an extra column and using sumproduct function between added column and required columns. So, the problem was solved there and it took around 1 minute 30 seconds for the same.
Also, the problem can be solved by writing an SQL query with 32 joins (so many), for example, if i want to select 6 items out of those 16 above with atleast 4 items designer, 4 exclusive, 4 handloom, the query would be like in my post: MYSQL - Select rows fulfilling many count conditions
In production, I have to fetch 32 rows like this way, So my question is how do I proceed further with the project.
I am working on Scala IDE for Eclipse, and have added spark mllib there. I have fetched data via JDBC and stored in a dataframe, and the created a temporary table:
dataFrame.registerTempTable("Data")
There is a class optimizer in mllib optimization that uses gradient descent (like excel solver does) to solve problems. But, that is for machine learning and takes as input training data.
I am not able to understand how do I proceed with my project. Can i use mllib, or use a better simple version of the sql with sparkSQL. I need serious help.
I'd recommend you to use https://spark.apache.org/docs/1.3.0/sql-programming-guide.html#creating-dataframes rather than MLLib.
I solved this problem through linear programming. I have now used lpsolver library for java in my scala project. It is giving almost the same result as in excel solver.
Hi all,
Above you'll see a line-graph plotted with SPSS. I want to improve this line-graph according to its data. Meaning that some elements are not presented correctly:
(1) I deliberately adjusted the scaling on the Y-axis from -1 to 10, in order to notice the breaks (i.e. missing values) in the line graph. Otherwise you'll not notice the breaks, as it will overlap with the bottom-line of the graph. Is it possible to notice the breaks, but with a scaling of 0 to 10 (in SPSS)? > SOLVED
(2) On the X-axis, point 14 and 15 are missing, hence the break. However, the line graph shows an upward trend just after point 13, and a downward trend just before point 16. Is it possible to adjust the line-graph (in SPSS), which would delete these described (interpolation) trends?
GGRAPH
/GRAPHDATASET NAME="graphdataset" VARIABLES=Time_Period_Hours
MEAN(MT)[name="MEAN_MT"] MISSING=VARIABLEWISE REPORTMISSING=NO
/GRAPHSPEC SOURCE=INLINE.
BEGIN GPL
SOURCE: s=userSource(id("graphdataset"))
DATA: Time_Period_Hours=col(source(s), name("Time_Period_Hours"), unit.category())
DATA: MEAN_MT=col(source(s), name("MEAN_MT"))
GUIDE: axis(dim(2), delta(1))
SCALE: linear(dim(2), min(-0.5), max(9))
ELEMENT: line(position(Time_Period_Hours*MEAN_MT))
ELEMENT: point(position(Time_Period_Hours*MEAN_MT), color(color.black),
size(size."3px"))
END GPL.
Here is an example, for the line element you need to specify the option missing.gap() - I thought just deleting missing.wings() from the default code would work but maybe it is an internal default. You may want to consider changing Time_Period_Hours to a scale variable and doing the aggregation outside of GGRAPH. Also making the Y axis scale in your example go all the way up to 9 seems a bit superfluous.
DATA LIST FREE / Time_Period_Hours MT.
BEGIN DATA
1 1
2 0
3 0
4 0
5 1
6 0
7 0
8 0
9 0
10 0
11 .
12 0
13 0
14 .
15 .
16 1
17 0
18 0
19 0
20 .
21 0
END DATA.
FORMATS Time_Period_Hours MT (F2.0).
GGRAPH
/GRAPHDATASET NAME="graphdataset" VARIABLES=Time_Period_Hours
MEAN(MT)[name="MEAN_MT"] MISSING=VARIABLEWISE REPORTMISSING=NO
/GRAPHSPEC SOURCE=INLINE.
BEGIN GPL
SOURCE: s=userSource(id("graphdataset"))
DATA: Time_Period_Hours=col(source(s), name("Time_Period_Hours"), unit.category())
DATA: MEAN_MT=col(source(s), name("MEAN_MT"))
GUIDE: axis(dim(2), delta(1))
SCALE: linear(dim(2), min(-0.5), max(9))
ELEMENT: line(position(Time_Period_Hours*MEAN_MT), missing.gap())
ELEMENT: point(position(Time_Period_Hours*MEAN_MT), color(color.black),
size(size."3px"))
END GPL.
I have a text file with a header of information followed by lines with just numbers, which are the data to be read.
I don't know how many lines are there in the header, and it is a variable number.
Here is an example:
filehandle: 65536
total # scientific data sets: 1
file description:
This file contains a Northern Hemisphere polar stereographic map of snow and ice coverage at 1024x1024 resolution. The map was produced using the NOAA/NESDIS Interactive MultisensorSnow and Ice Mapping System (IMS) developed under the directionof the Interactive Processing Branch (IPB) of the Satellite Services Division (SSD). For more information, contact: Mr. Bruce Ramsay at bramsay#ssd.wwb.noaa.gov.
Data Set # 1
Data Label:
Northern Hemisphere 1024x1024 Snow & Ice Chart
Coordinate System: Polar Stereographic
Data Type: BYTE
Format: I3
Dimensions: 1024 1024
Min/Max Values: 0 165
Units: 8-bit Flag
Dimension # 0
Dim Label: Longitude
Dim Format: Device Coordinates
Dim Units: Pixels
Dimension # 1
Dim Label: Latitude
Dim Format: Device Coordinates
Dim Units: Pixels
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0
2 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0
3 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0
..........................
I open the file using:
open(newunit=U, file = ValFile, STATUS = 'OLD', ACCESS = 'SEQUENTIAL', ACTION = 'READ')
Then, I read the file line by line and test for the type of line: header line or data line:
ios = 0
do while ( .NOT. is_iostat_end(ios) )
read(U, '(A)', iostat = ios, advance = 'NO') line ! Shouldn't advance to next line
if (is_iostat_end(ios)) stop "End of file reached before data section."
tol = getTypeOfLine(line, nValues) ! nValues = 1024, needed to test if line is data.
if ( tol > 0 ) then ! If the line holds data.
exit ! Exits the loop
else
read(U, '(A)', iostat = ios, advance = 'YES') line ! We advance to the next line
end if
end do
But the first read in the loop, always advances to the next line, and this is a problem.
After exiting the above loop, enter a new loop to read the data:
read(U, '(1024I1)', iostat = ios) Values(c,:)
The 1024 set of data can span some lines, but each set is a row in the matrix "Values".
The problem is that this second loop doesn't read the last line read in the testing loop (which is the first line of data).
A possible solution is to read the lines in the testing loop, without advancing to the next line. I used for this, advance='no', but it still advances to the next line, Why?.
A non-advancing read will still set the file position to before start of the next record if the end of the current record is encountered while reading from the file to satisfy the items in the output item list of the read statement - non-advancing doesn't mean "never-advancing". You can use the value assigned to the variable nominated in an iostat specifier for the read statement to see if the end of the current record was reached - use the IS_IOSTAT_EOR intrinsic or test against the equivalent value from ISO_FORTRAN_ENV.
(Implicit in the above is the fact that a non-advancing read still advances over the file positions that correspond to items actually read... hence once that getTypeOfLine procedure decides that it has a line of data at least part of that line has already been read. Unless you reposition the file subsequent "data" read statements will miss that part.)