How do I make construction heuristic phase assigns all entities in overconstrained planning? - optaplanner
As title says. I'm a developer new to optaplanner. Before I switched to overconstrained planning the CH phase worked correctly: here is CH assigning 21 entities as expected and LS was able to find an optimal solution (ignore the medium score level):
2021-10-01 12:26:07,933 [main] INFO Solving started: time spent (3221), best score (-21init/0hard/0medium/0soft), environment mode (REPRODUCIBLE), move thread count (NONE), random (JDK with seed 0).
2021-10-01 12:26:10,664 [main] DEBUG CH step (0), time spent (5955), score (-20init/0hard/0medium/-130soft), selected move count (21120), picked move (optaplanner.domain.Allocation#7c51782d {null -> ResourceTimegrain{resource=optaplanner.domain.Resource#276b68af, index=47, availability=2021-10-08T14:00-2021-10-08T22:00}}).
2021-10-01 12:26:12,924 [main] DEBUG CH step (1), time spent (8215), score (-19init/-1hard/0medium/-3025soft), selected move count (21120), picked move (optaplanner.domain.Allocation#77bc2e16 {null -> ResourceTimegrain{resource=optaplanner.domain.Resource#34d644b5, index=45, availability=2021-10-07T14:00-2021-10-07T22:00}}).
2021-10-01 12:26:14,137 [main] DEBUG CH step (2), time spent (9428), score (-18init/-3hard/0medium/-3025soft), selected move count (21120), picked move (optaplanner.domain.Allocation#48e8c32a {null -> ResourceTimegrain{resource=optaplanner.domain.Resource#50915d5, index=0, availability=2021-10-08T14:00-2021-10-08T22:00}}).
2021-10-01 12:26:15,293 [main] DEBUG CH step (3), time spent (10584), score (-17init/-3hard/0medium/-3164soft), selected move count (21120), picked move (optaplanner.domain.Allocation#20a7953c {null -> ResourceTimegrain{resource=optaplanner.domain.Resource#276b68af, index=46, availability=2021-10-08T14:00-2021-10-08T22:00}}).
2021-10-01 12:26:16,027 [main] DEBUG CH step (4), time spent (11318), score (-16init/-4hard/0medium/-6098soft), selected move count (21120), picked move (optaplanner.domain.Allocation#57c00115 {null -> ResourceTimegrain{resource=optaplanner.domain.Resource#34d644b5, index=42, availability=2021-10-07T14:00-2021-10-07T22:00}}).
2021-10-01 12:26:16,654 [main] DEBUG CH step (5), time spent (11945), score (-15init/-6hard/0medium/-6098soft), selected move count (21120), picked move (optaplanner.domain.Allocation#411a5965 {null -> ResourceTimegrain{resource=optaplanner.domain.Resource#50915d5, index=45, availability=2021-10-07T14:00-2021-10-07T22:00}}).
2021-10-01 12:26:17,267 [main] DEBUG CH step (6), time spent (12558), score (-14init/-6hard/0medium/-6247soft), selected move count (21120), picked move (optaplanner.domain.Allocation#4fe533ff {null -> ResourceTimegrain{resource=optaplanner.domain.Resource#276b68af, index=45, availability=2021-10-08T14:00-2021-10-08T22:00}}).
2021-10-01 12:26:17,850 [main] DEBUG CH step (7), time spent (13141), score (-13init/-7hard/0medium/-9221soft), selected move count (21120), picked move (optaplanner.domain.Allocation#5377414a {null -> ResourceTimegrain{resource=optaplanner.domain.Resource#34d644b5, index=39, availability=2021-10-07T14:00-2021-10-07T22:00}}).
2021-10-01 12:26:18,541 [main] DEBUG CH step (8), time spent (13832), score (-12init/-9hard/0medium/-9221soft), selected move count (21120), picked move (optaplanner.domain.Allocation#4e83a98 {null -> ResourceTimegrain{resource=optaplanner.domain.Resource#50915d5, index=42, availability=2021-10-07T14:00-2021-10-07T22:00}}).
2021-10-01 12:26:19,182 [main] DEBUG CH step (9), time spent (14473), score (-11init/-9hard/0medium/-9380soft), selected move count (21120), picked move (optaplanner.domain.Allocation#17aa8a11 {null -> ResourceTimegrain{resource=optaplanner.domain.Resource#276b68af, index=44, availability=2021-10-08T14:00-2021-10-08T22:00}}).
2021-10-01 12:26:19,758 [main] DEBUG CH step (10), time spent (15049), score (-10init/-10hard/0medium/-9416soft), selected move count (21120), picked move (optaplanner.domain.Allocation#71b639d0 {null -> ResourceTimegrain{resource=optaplanner.domain.Resource#34d644b5, index=45, availability=2021-10-08T14:00-2021-10-08T22:00}}).
2021-10-01 12:26:20,332 [main] DEBUG CH step (11), time spent (15623), score (-9init/-12hard/0medium/-12250soft), selected move count (21120), picked move (optaplanner.domain.Allocation#18a25bbd {null -> ResourceTimegrain{resource=optaplanner.domain.Resource#63661fc7, index=45, availability=2021-10-07T14:00-2021-10-07T22:00}}).
2021-10-01 12:26:20,939 [main] DEBUG CH step (12), time spent (16230), score (-8init/-12hard/0medium/-12419soft), selected move count (21120), picked move (optaplanner.domain.Allocation#5d5b9ecb {null -> ResourceTimegrain{resource=optaplanner.domain.Resource#276b68af, index=43, availability=2021-10-08T14:00-2021-10-08T22:00}}).
2021-10-01 12:26:21,499 [main] DEBUG CH step (13), time spent (16790), score (-7init/-13hard/0medium/-15413soft), selected move count (21120), picked move (optaplanner.domain.Allocation#1ee27d73 {null -> ResourceTimegrain{resource=optaplanner.domain.Resource#34d644b5, index=36, availability=2021-10-07T14:00-2021-10-07T22:00}}).
2021-10-01 12:26:22,085 [main] DEBUG CH step (14), time spent (17376), score (-6init/-14hard/0medium/-15449soft), selected move count (21120), picked move (optaplanner.domain.Allocation#5e5aafc6 {null -> ResourceTimegrain{resource=optaplanner.domain.Resource#50915d5, index=44, availability=2021-10-08T14:00-2021-10-08T22:00}}).
2021-10-01 12:26:22,669 [main] DEBUG CH step (15), time spent (17960), score (-5init/-14hard/0medium/-15628soft), selected move count (21120), picked move (optaplanner.domain.Allocation#542f6803 {null -> ResourceTimegrain{resource=optaplanner.domain.Resource#276b68af, index=42, availability=2021-10-08T14:00-2021-10-08T22:00}}).
2021-10-01 12:26:23,253 [main] DEBUG CH step (16), time spent (18544), score (-4init/-15hard/0medium/-18662soft), selected move count (21120), picked move (optaplanner.domain.Allocation#5583098b {null -> ResourceTimegrain{resource=optaplanner.domain.Resource#34d644b5, index=33, availability=2021-10-07T14:00-2021-10-07T22:00}}).
2021-10-01 12:26:23,833 [main] DEBUG CH step (17), time spent (19124), score (-3init/-16hard/0medium/-18698soft), selected move count (21120), picked move (optaplanner.domain.Allocation#5807efad {null -> ResourceTimegrain{resource=optaplanner.domain.Resource#63661fc7, index=43, availability=2021-10-08T14:00-2021-10-08T22:00}}).
2021-10-01 12:26:24,427 [main] DEBUG CH step (18), time spent (19718), score (-2init/-16hard/0medium/-18887soft), selected move count (21120), picked move (optaplanner.domain.Allocation#53a84ff4 {null -> ResourceTimegrain{resource=optaplanner.domain.Resource#276b68af, index=41, availability=2021-10-08T14:00-2021-10-08T22:00}}).
2021-10-01 12:26:25,010 [main] DEBUG CH step (19), time spent (20301), score (-1init/-17hard/0medium/-18923soft), selected move count (21120), picked move (optaplanner.domain.Allocation#7ce85af2 {null -> ResourceTimegrain{resource=optaplanner.domain.Resource#34d644b5, index=42, availability=2021-10-08T14:00-2021-10-08T22:00}}).
2021-10-01 12:26:25,660 [main] DEBUG CH step (20), time spent (20951), score (-19hard/0medium/-18953soft), selected move count (21120), picked move (optaplanner.domain.Allocation#316acbb5 {null -> ResourceTimegrain{resource=optaplanner.domain.Resource#65130cf2, index=45, availability=2021-10-08T14:00-2021-10-08T22:00}}).
2021-10-01 12:26:25,662 [main] INFO Construction Heuristic phase (0) ended: time spent (20953), best score (-19hard/0medium/-18953soft), score calculation speed (25027/sec), step total (21).
2021-10-01 12:26:25,961 [main] DEBUG LS step (0), time spent (21252), score (-17hard/0medium/-24372soft), new best score (-17hard/0medium/-24372soft), accepted/selected move count (1000/1000), picked move (optaplanner.domain.Allocation#5296ab0c {ResourceTimegrain{resource=optaplanner.domain.Resource#276b68af, index=47, availability=2021-10-08T14:00-2021-10-08T22:00} -> ResourceTimegrain{resource=optaplanner.domain.Resource#34d644b5, index=41, availability=2021-10-05T14:00-2021-10-05T22:00}}).
2021-10-01 12:26:26,114 [main] DEBUG LS step (1), time spent (21405), score (-15hard/0medium/-40094soft), new best score (-15hard/0medium/-40094soft), accepted/selected move count (1000/1067), picked move (optaplanner.domain.Allocation#730794bb {ResourceTimegrain{resource=optaplanner.domain.Resource#276b68af, index=45, availability=2021-10-08T14:00-2021-10-08T22:00} -> ResourceTimegrain{resource=optaplanner.domain.Resource#276b68af, index=46, availability=2021-10-01T14:00-2021-10-01T22:00}}).
2021-10-01 12:26:26,309 [main] DEBUG LS step (2), time spent (21600), score (-13hard/0medium/-47776soft), new best score (-13hard/0medium/-47776soft), accepted/selected move count (1000/1160), picked move (optaplanner.domain.Allocation#73ed094c {ResourceTimegrain{resource=optaplanner.domain.Resource#276b68af, index=46, availability=2021-10-08T14:00-2021-10-08T22:00} -> ResourceTimegrain{resource=optaplanner.domain.Resource#276b68af, index=20, availability=2021-10-04T14:00-2021-10-04T22:00}}).
2021-10-01 12:26:26,471 [main] DEBUG LS step (3), time spent (21762), score (-11hard/0medium/-65742soft), new best score (-11hard/0medium/-65742soft), accepted/selected move count (1000/1145), picked move (optaplanner.domain.Allocation#3c37489b {ResourceTimegrain{resource=optaplanner.domain.Resource#276b68af, index=42, availability=2021-10-08T14:00-2021-10-08T22:00} -> ResourceTimegrain{resource=optaplanner.domain.Resource#276b68af, index=0, availability=2021-10-01T14:00-2021-10-01T22:00}}).
2021-10-01 12:26:26,649 [main] DEBUG LS step (4), time spent (21940), score (-9hard/0medium/-75202soft), new best score (-9hard/0medium/-75202soft), accepted/selected move count (1000/1148), picked move (optaplanner.domain.Allocation#a386ccf {ResourceTimegrain{resource=optaplanner.domain.Resource#276b68af, index=41, availability=2021-10-08T14:00-2021-10-08T22:00} -> ResourceTimegrain{resource=optaplanner.domain.Resource#276b68af, index=0, availability=2021-10-05T14:00-2021-10-05T22:00}}).
2021-10-01 12:26:26,832 [main] DEBUG LS step (5), time spent (22123), score (-8hard/0medium/-75202soft), new best score (-8hard/0medium/-75202soft), accepted/selected move count (1000/1172), picked move (optaplanner.domain.Allocation#144402f6 {ResourceTimegrain{resource=optaplanner.domain.Resource#34d644b5, index=42, availability=2021-10-07T14:00-2021-10-07T22:00} -> ResourceTimegrain{resource=optaplanner.domain.Resource#34d644b5, index=11, availability=2021-10-05T14:00-2021-10-05T22:00}}).
--------------------(omitting other steps)--------------------
2021-10-01 12:27:04,541 [main] DEBUG LS step (347), time spent (59832), score (0hard/0medium/-42754soft), best score (0hard/0medium/-42754soft), accepted/selected move count (1000/1359), picked move (optaplanner.domain.Allocation#64047c70 {ResourceTimegrain{resource=optaplanner.domain.Resource#276b68af, index=46, availability=2021-10-06T14:00-2021-10-06T22:00}} <-> optaplanner.domain.Allocation#3c37489b {ResourceTimegrain{resource=optaplanner.domain.Resource#276b68af, index=44, availability=2021-10-06T14:00-2021-10-06T22:00}}).
2021-10-01 12:27:04,635 [main] DEBUG LS step (348), time spent (59926), score (0hard/0medium/-42754soft), best score (0hard/0medium/-42754soft), accepted/selected move count (1000/1365), picked move (optaplanner.domain.Allocation#68bb44fe {ResourceTimegrain{resource=optaplanner.domain.Resource#34d644b5, index=6, availability=2021-10-07T14:00-2021-10-07T22:00}} <-> optaplanner.domain.Allocation#62f89e6a {ResourceTimegrain{resource=optaplanner.domain.Resource#34d644b5, index=33, availability=2021-10-07T14:00-2021-10-07T22:00}}).
2021-10-01 12:27:04,709 [main] DEBUG LS step (349), time spent (60000), score (0hard/0medium/-42754soft), best score (0hard/0medium/-42754soft), accepted/selected move count (802/1086), picked move (optaplanner.domain.Allocation#7f77b1b0 {ResourceTimegrain{resource=optaplanner.domain.Resource#34d644b5, index=13, availability=2021-10-07T14:00-2021-10-07T22:00}} <-> optaplanner.domain.Allocation#6f8a9b12 {ResourceTimegrain{resource=optaplanner.domain.Resource#34d644b5, index=39, availability=2021-10-07T14:00-2021-10-07T22:00}}).
2021-10-01 12:27:04,709 [main] INFO Local Search phase (1) ended: time spent (60000), best score (0hard/0medium/-42754soft), score calculation speed (11876/sec), step total (350).
2021-10-01 12:27:04,712 [main] INFO Solving ended: time spent (60000), best score (0hard/0medium/-42754soft), score calculation speed (15115/sec), phase total (2), environment mode (REPRODUCIBLE), move thread count (NONE).
After switching to nullable planning variable, medium score level has a constraint that penalizes for every unassigned entity. This is log from solving the same problem instance to which optplanner couldn't find an optimal solution:
2021-10-01 12:39:29,144 [main] INFO Solving started: time spent (6235), best score (0hard/-21medium/0soft), environment mode (REPRODUCIBLE), move thread count (NONE), random (JDK with seed 0).
2021-10-01 12:39:33,797 [main] DEBUG CH step (0), time spent (10892), score (0hard/-20medium/-130soft), selected move count (21121), picked move (optaplanner.domain.Allocation#7c51782d {null -> ResourceTimegrain{resource=optaplanner.domain.Resource#668ea404, index=47, availability=2021-10-08T14:00-2021-10-08T22:00}}).
2021-10-01 12:39:36,698 [main] DEBUG CH step (1), time spent (13793), score (0hard/-20medium/-130soft), selected move count (21121), picked move (optaplanner.domain.Allocation#77bc2e16 {null -> null}).
2021-10-01 12:39:39,032 [main] DEBUG CH step (2), time spent (16127), score (0hard/-20medium/-130soft), selected move count (21121), picked move (optaplanner.domain.Allocation#48e8c32a {null -> null}).
2021-10-01 12:39:40,857 [main] DEBUG CH step (3), time spent (17952), score (0hard/-19medium/-269soft), selected move count (21121), picked move (optaplanner.domain.Allocation#20a7953c {null -> ResourceTimegrain{resource=optaplanner.domain.Resource#668ea404, index=46, availability=2021-10-08T14:00-2021-10-08T22:00}}).
2021-10-01 12:39:41,878 [main] DEBUG CH step (4), time spent (18973), score (0hard/-19medium/-269soft), selected move count (21121), picked move (optaplanner.domain.Allocation#57c00115 {null -> null}).
2021-10-01 12:39:42,586 [main] DEBUG CH step (5), time spent (19681), score (0hard/-19medium/-269soft), selected move count (21121), picked move (optaplanner.domain.Allocation#411a5965 {null -> null}).
2021-10-01 12:39:43,354 [main] DEBUG CH step (6), time spent (20449), score (0hard/-18medium/-418soft), selected move count (21121), picked move (optaplanner.domain.Allocation#4fe533ff {null -> ResourceTimegrain{resource=optaplanner.domain.Resource#668ea404, index=45, availability=2021-10-08T14:00-2021-10-08T22:00}}).
2021-10-01 12:39:44,007 [main] DEBUG CH step (7), time spent (21102), score (0hard/-18medium/-418soft), selected move count (21121), picked move (optaplanner.domain.Allocation#5377414a {null -> null}).
2021-10-01 12:39:44,633 [main] DEBUG CH step (8), time spent (21728), score (0hard/-18medium/-418soft), selected move count (21121), picked move (optaplanner.domain.Allocation#4e83a98 {null -> null}).
2021-10-01 12:39:45,251 [main] DEBUG CH step (9), time spent (22346), score (0hard/-17medium/-577soft), selected move count (21121), picked move (optaplanner.domain.Allocation#17aa8a11 {null -> ResourceTimegrain{resource=optaplanner.domain.Resource#668ea404, index=44, availability=2021-10-08T14:00-2021-10-08T22:00}}).
2021-10-01 12:39:45,898 [main] DEBUG CH step (10), time spent (22993), score (0hard/-17medium/-577soft), selected move count (21121), picked move (optaplanner.domain.Allocation#71b639d0 {null -> null}).
2021-10-01 12:39:46,485 [main] DEBUG CH step (11), time spent (23580), score (0hard/-17medium/-577soft), selected move count (21121), picked move (optaplanner.domain.Allocation#18a25bbd {null -> null}).
2021-10-01 12:39:47,151 [main] DEBUG CH step (12), time spent (24246), score (0hard/-16medium/-746soft), selected move count (21121), picked move (optaplanner.domain.Allocation#5d5b9ecb {null -> ResourceTimegrain{resource=optaplanner.domain.Resource#668ea404, index=43, availability=2021-10-08T14:00-2021-10-08T22:00}}).
2021-10-01 12:39:47,862 [main] DEBUG CH step (13), time spent (24957), score (0hard/-16medium/-746soft), selected move count (21121), picked move (optaplanner.domain.Allocation#1ee27d73 {null -> null}).
2021-10-01 12:39:48,556 [main] DEBUG CH step (14), time spent (25651), score (0hard/-16medium/-746soft), selected move count (21121), picked move (optaplanner.domain.Allocation#5e5aafc6 {null -> null}).
2021-10-01 12:39:49,193 [main] DEBUG CH step (15), time spent (26288), score (0hard/-15medium/-925soft), selected move count (21121), picked move (optaplanner.domain.Allocation#542f6803 {null -> ResourceTimegrain{resource=optaplanner.domain.Resource#668ea404, index=42, availability=2021-10-08T14:00-2021-10-08T22:00}}).
2021-10-01 12:39:49,781 [main] DEBUG CH step (16), time spent (26876), score (0hard/-15medium/-925soft), selected move count (21121), picked move (optaplanner.domain.Allocation#5583098b {null -> null}).
2021-10-01 12:39:50,407 [main] DEBUG CH step (17), time spent (27502), score (0hard/-15medium/-925soft), selected move count (21121), picked move (optaplanner.domain.Allocation#5807efad {null -> null}).
2021-10-01 12:39:51,168 [main] DEBUG CH step (18), time spent (28262), score (0hard/-14medium/-1114soft), selected move count (21121), picked move (optaplanner.domain.Allocation#53a84ff4 {null -> ResourceTimegrain{resource=optaplanner.domain.Resource#668ea404, index=41, availability=2021-10-08T14:00-2021-10-08T22:00}}).
2021-10-01 12:39:51,826 [main] DEBUG CH step (19), time spent (28921), score (0hard/-14medium/-1114soft), selected move count (21121), picked move (optaplanner.domain.Allocation#7ce85af2 {null -> null}).
2021-10-01 12:39:52,372 [main] DEBUG CH step (20), time spent (29467), score (0hard/-14medium/-1114soft), selected move count (21121), picked move (optaplanner.domain.Allocation#316acbb5 {null -> null}).
2021-10-01 12:39:52,374 [main] INFO Construction Heuristic phase (0) ended: time spent (29469), best score (0hard/-14medium/-1114soft), score calculation speed (19105/sec), step total (21).
2021-10-01 12:39:52,692 [main] DEBUG LS step (0), time spent (29787), score (0hard/-14medium/-1114soft), best score (0hard/-14medium/-1114soft), accepted/selected move count (1000/1000), picked move (optaplanner.domain.Allocation#41418e53 {ResourceTimegrain{resource=optaplanner.domain.Resource#668ea404, index=43, availability=2021-10-08T14:00-2021-10-08T22:00}} <-> optaplanner.domain.Allocation#7f77b1b0 {ResourceTimegrain{resource=optaplanner.domain.Resource#668ea404, index=46, availability=2021-10-08T14:00-2021-10-08T22:00}}).
2021-10-01 12:39:52,873 [main] DEBUG LS step (1), time spent (29968), score (0hard/-14medium/-1114soft), best score (0hard/-14medium/-1114soft), accepted/selected move count (1000/1213), picked move (optaplanner.domain.Allocation#c4d9e83 {ResourceTimegrain{resource=optaplanner.domain.Resource#668ea404, index=44, availability=2021-10-08T14:00-2021-10-08T22:00}} <-> optaplanner.domain.Allocation#6f8a9b12 {ResourceTimegrain{resource=optaplanner.domain.Resource#668ea404, index=47, availability=2021-10-08T14:00-2021-10-08T22:00}}).
2021-10-01 12:39:53,061 [main] DEBUG LS step (2), time spent (30156), score (0hard/-14medium/-1114soft), best score (0hard/-14medium/-1114soft), accepted/selected move count (1000/1503), picked move (optaplanner.domain.Allocation#7b174491 {ResourceTimegrain{resource=optaplanner.domain.Resource#668ea404, index=45, availability=2021-10-08T14:00-2021-10-08T22:00}} <-> optaplanner.domain.Allocation#d464e23 {ResourceTimegrain{resource=optaplanner.domain.Resource#668ea404, index=41, availability=2021-10-08T14:00-2021-10-08T22:00}}).
2021-10-01 12:39:53,270 [main] DEBUG LS step (3), time spent (30365), score (0hard/-14medium/-1114soft), best score (0hard/-14medium/-1114soft), accepted/selected move count (1000/1535), picked move (optaplanner.domain.Allocation#7f77b1b0 {ResourceTimegrain{resource=optaplanner.domain.Resource#668ea404, index=43, availability=2021-10-08T14:00-2021-10-08T22:00}} <-> optaplanner.domain.Allocation#41418e53 {ResourceTimegrain{resource=optaplanner.domain.Resource#668ea404, index=46, availability=2021-10-08T14:00-2021-10-08T22:00}}).
2021-10-01 12:39:53,457 [main] DEBUG LS step (4), time spent (30552), score (0hard/-14medium/-1114soft), best score (0hard/-14medium/-1114soft), accepted/selected move count (1000/1532), picked move (optaplanner.domain.Allocation#c4d9e83 {ResourceTimegrain{resource=optaplanner.domain.Resource#668ea404, index=47, availability=2021-10-08T14:00-2021-10-08T22:00}} <-> optaplanner.domain.Allocation#6f8a9b12 {ResourceTimegrain{resource=optaplanner.domain.Resource#668ea404, index=44, availability=2021-10-08T14:00-2021-10-08T22:00}}).
2021-10-01 12:39:53,605 [main] DEBUG LS step (5), time spent (30700), score (0hard/-14medium/-1114soft), best score (0hard/-14medium/-1114soft), accepted/selected move count (1000/1548), picked move (optaplanner.domain.Allocation#2957f567 {ResourceTimegrain{resource=optaplanner.domain.Resource#668ea404, index=42, availability=2021-10-08T14:00-2021-10-08T22:00}} <-> optaplanner.domain.Allocation#7b174491 {ResourceTimegrain{resource=optaplanner.domain.Resource#668ea404, index=41, availability=2021-10-08T14:00-2021-10-08T22:00}}).
--------------------(omitting other steps)--------------------
2021-10-01 12:40:22,649 [main] DEBUG LS step (367), time spent (59744), score (0hard/-14medium/-1114soft), best score (0hard/-14medium/-1114soft), accepted/selected move count (1000/1455), picked move (optaplanner.domain.Allocation#d464e23 {ResourceTimegrain{resource=optaplanner.domain.Resource#668ea404, index=43, availability=2021-10-08T14:00-2021-10-08T22:00}} <-> optaplanner.domain.Allocation#41418e53 {ResourceTimegrain{resource=optaplanner.domain.Resource#668ea404, index=41, availability=2021-10-08T14:00-2021-10-08T22:00}}).
2021-10-01 12:40:22,731 [main] DEBUG LS step (368), time spent (59826), score (0hard/-14medium/-1114soft), best score (0hard/-14medium/-1114soft), accepted/selected move count (1000/1545), picked move (optaplanner.domain.Allocation#2957f567 {ResourceTimegrain{resource=optaplanner.domain.Resource#668ea404, index=42, availability=2021-10-08T14:00-2021-10-08T22:00}} <-> optaplanner.domain.Allocation#c4d9e83 {ResourceTimegrain{resource=optaplanner.domain.Resource#668ea404, index=44, availability=2021-10-08T14:00-2021-10-08T22:00}}).
2021-10-01 12:40:22,812 [main] DEBUG LS step (369), time spent (59907), score (0hard/-14medium/-1114soft), best score (0hard/-14medium/-1114soft), accepted/selected move count (1000/1535), picked move (optaplanner.domain.Allocation#7b174491 {ResourceTimegrain{resource=optaplanner.domain.Resource#668ea404, index=45, availability=2021-10-08T14:00-2021-10-08T22:00}} <-> optaplanner.domain.Allocation#7f77b1b0 {ResourceTimegrain{resource=optaplanner.domain.Resource#668ea404, index=46, availability=2021-10-08T14:00-2021-10-08T22:00}}).
2021-10-01 12:40:22,890 [main] DEBUG LS step (370), time spent (59985), score (0hard/-14medium/-1114soft), best score (0hard/-14medium/-1114soft), accepted/selected move count (1000/1525), picked move (optaplanner.domain.Allocation#6f8a9b12 {ResourceTimegrain{resource=optaplanner.domain.Resource#668ea404, index=47, availability=2021-10-08T14:00-2021-10-08T22:00}} <-> optaplanner.domain.Allocation#41418e53 {ResourceTimegrain{resource=optaplanner.domain.Resource#668ea404, index=43, availability=2021-10-08T14:00-2021-10-08T22:00}}).
2021-10-01 12:40:22,905 [main] DEBUG LS step (371), time spent (60000), score (0hard/-14medium/-1114soft), best score (0hard/-14medium/-1114soft), accepted/selected move count (189/293), picked move (optaplanner.domain.Allocation#c4d9e83 {ResourceTimegrain{resource=optaplanner.domain.Resource#668ea404, index=42, availability=2021-10-08T14:00-2021-10-08T22:00}} <-> optaplanner.domain.Allocation#d464e23 {ResourceTimegrain{resource=optaplanner.domain.Resource#668ea404, index=41, availability=2021-10-08T14:00-2021-10-08T22:00}}).
2021-10-01 12:40:22,905 [main] INFO Local Search phase (1) ended: time spent (60000), best score (0hard/-14medium/-1114soft), score calculation speed (18451/sec), step total (372).
2021-10-01 12:40:22,906 [main] INFO Solving ended: time spent (60000), best score (0hard/-14medium/-1114soft), score calculation speed (16769/sec), phase total (2), environment mode (REPRODUCIBLE), move thread count (NONE).
Observation:
CH didn't assign all entities which also affects LS (I guess since it is optional).
CH had pointless steps (assign null -> null)
How do I make sure that all entities are assigned?
The step ("assign null -> null") is not pointless. It shows OptaPlanner making a decision to keep something null. And it made that decision, because it was the decision that, of all the possible decisions, resulted in the best possible score. (Otherwise that decision wouldn't have been made.)
Most likely, assigning to non-null values would have broken a hard constraint - and OptaPlanner avoided that by assigning null. OptaPlanner behaves as expected here - you say you have more important concerns than nullity, and OptaPlanner respects your choice. (Nullity is only a medium constraint, while some other constraints are hard and therefore more important.)
If you wish to have all variables assigned, do not make them nullable.
Alternatively, design heavily weighted hard constraints for situations where unassigned variables are a problem. (But how would that differ from just not having them nullable in the first place?)
Related
Optaplanner 7.9.0 Multithreading - Solution not reproducable
Having updated to 7.9.0 and after initial problems: https://stackoverflow.com/questions/51597744/optaplanner-7-9-0-and-adding-multithreading-same-planningid-exception I now have been trying to test and compare to my 7.7.0 version. However I cannot get it to reproduce the same solution everytime (obviously only with the same problem data) as in my older version, even when explicitly set in the config xml. Is there additional set up for this version required? Edit: Did some testing and found switching to TABU (haven't been through them all) gave me the expected consistency: Extra Cores:1 - DEFAULT: Score (Hrd:Med:Sft) Time Taken (Minutes:Seconds) Test1 0:0:-7609 0:08 Test2 0:-1:-7758 0:13 Test3 0:-1:-7705 0:14 Extra Cores:1 - TABU: Score (Hrd:Med:Sft) Time Taken (Minutes:Seconds) Test1 0:0:-7763 1:29 Test2 0:0:-7763 1:29 Test3 0:0:-7763 1:28 Between two runs of the former the solution diverges at LS step 28: LS step (25), time spent (1869), score (0hard/-3medium/-8155soft), LS step (26), time spent (1890), score (0hard/-3medium/-8339soft), LS step (27), time spent (1895), score (0hard/-3medium/-8126soft), **LS step (28), time spent (1909), score (0hard/-3medium/-8256soft), LS step (29), time spent (1915), score (0hard/-3medium/-8438soft), LS step (30), time spent (1924), score (0hard/-3medium/-8620soft), LS step (31), time spent (1952), score (0hard/-3medium/-8639soft),** ...and... LS step (25), time spent (1385), score (0hard/-3medium/-8155soft), LS step (26), time spent (1407), score (0hard/-3medium/-8339soft), LS step (27), time spent (1412), score (0hard/-3medium/-8126soft), **LS step (28), time spent (1422), score (0hard/-3medium/-8217soft), LS step (29), time spent (1436), score (0hard/-3medium/-8336soft), LS step (30), time spent (1442), score (0hard/-3medium/-8517soft), LS step (31), time spent (1448), score (0hard/-3medium/-8571soft),** Don't know if that all makes it more or less likely to be problem/solution set up or something else.
First, let's define reproducible: getting the same result at the same step iteration. If the CPU time is completely the same (which it never is), this also means getting the same result after the same amount of time. So you might need to run it a bit longer to get the same amount of steps. Multithreaded solving is reproducible if and only if the same moveThreadCount is used. A run with <moveThreadCount>4</> is NOT reproducible by a <moveThreadCount>2</>. Even a run with <moveThreadCount>NONE</> is not reproducible with a run with <moveThreadCount>1</> (the latter being useless except for QA btw). Look at your score calculation speed. If it's lower, you probably forgot to increase the memory size and GC churns are hurting you. If that's higher, you'd normally get a better result. If you see a worse result anyway, the config might have been overfitted for the no move threads. Use additional datasets in optaplanner-benchmark to prove or disprove that. Furthermore, if you use <moveThreadCount>AUTO</>, it's only reproducible on the same machine, because AUTO changes the moveThreadCount depending on the number of CPU cores in the machine.
Apache Ignite poor performance compared to Redis
I did a simplistic benchmark of apache ignite recently and found that when I made use of the ignite client across 50 threads, the performance degraded tremendously. The benchmark pits Redis against Ignite and Jedis, the Redis java client, seems to do much better in the threaded scenario since it keeps multiple clients in a pool rather than using one client among multiple threads. My question is, am I seeing the degraded performance as a result of what i've just articulated, or is this actually expected and the client isn't the bottleneck? Here were my results. The local host is a mac pro with 12 2.7GHz cores and 64gb of ram. The remote host was an AWS m4.4xlarge. Benchmark Results *50 - 118kb Audio File Streamed and Read Simultaneously from One Node in 4096 byte chunks* Redis *Notes:* I used a 50ms polling interval to check for updates to the cache. *Results:* Local: - Total Time to stream and read 50 audio files: 226ms. - average complete read and write: 125ms - average time to first byte read: 26ms - average read time per runner: 103ms - average write time per runner: 71ms - p99 time to first byte: 59ms - p90 time to first byte: 57ms - p50 time to first byte: 6ms Remote (Over SSH | Seattle → IAD): - Total Time to stream and read 50 audio files: 1405ms. - average complete read and write: 1298ms - average time to first byte read: 81ms - average read time per runner: 1277ms - average write time per runner: 1238ms - p99 time to first byte: 148ms - p90 time to first byte: 126ms - p50 time to first byte: 84ms Remote (Through VIP | Seattle → IAD): - Total Time to stream and read 50 audio files: 2035ms. - average complete read and write: 1245ms - average time to first byte read: 67ms - average read time per runner: 1226ms - average write time per runner: 1034ms - p99 time to first byte: 161ms - p90 time to first byte: 87ms - p50 time to first byte: 74ms Ignite *Notes:* I have a feeling these numbers are artificially inflated. I think the client is not well built for extreme parallelism. I believe it's doing quite a bit of locking. I think if you were to have many nodes doing the same amount of work, the numbers might be better. This would require more in depth benchmarking. This is 50 caches, one cache group. *Results:* Local: - Total Time to stream and read 50 audio files: 327ms. - average complete read and write: 321ms - average time to first byte read: 184ms - average read time per runner: 225ms - average write time per runner: 35ms - p99 time to first byte: 212ms - p90 time to first byte: 197ms - p50 time to first byte: 191ms Remote (Over SSH | Seattle → IAD): - Total Time to stream and read 50 audio files: 5148ms. - average complete read and write: 4483ms - average time to first byte read: 947ms - average read time per runner: 3224ms - average write time per runner: 2779ms - p99 time to first byte: 4936ms - p90 time to first byte: 926ms - p50 time to first byte: 577ms Remote (Through VIP | Seattle → IAD): - Total Time to stream and read 50 audio files: 4840ms. - average complete read and write: 4287ms - average time to first byte read: 780ms - average read time per runner: 3035ms - average write time per runner: 2562ms - p99 time to first byte: 4458ms - p90 time to first byte: 857ms - p50 time to first byte: 566ms *1 - 118kb Audio File Streamed and Read Simultaneously from One Node in 4096 byte chunks* Redis *Notes:* I used a 50ms polling interval to check for updates to the cache. *Results:* Local: - Total Time to stream and read 1 audio files: 62ms. - average complete read and write: 62ms - average time to first byte read: 55ms - average read time per runner: 61ms - average write time per runner: 3ms - p99 time to first byte: 55ms - p90 time to first byte: 55ms - p50 time to first byte: 55ms Remote (Over SSH | Seattle → IAD): - Total Time to stream and read 1 audio files: 394ms. - average complete read and write: 394ms - average time to first byte read: 57ms - average read time per runner: 394ms - average write time per runner: 342ms - p99 time to first byte: 57ms - p90 time to first byte: 57ms - p50 time to first byte: 57ms Remote (Through VIP | Seattle → IAD): - Total Time to stream and read 1 audio files: 388ms. - average complete read and write: 388ms - average time to first byte read: 61ms - average read time per runner: 388ms - average write time per runner: 343ms - p99 time to first byte: 61ms - p90 time to first byte: 61ms - p50 time to first byte: 61ms Ignite *Notes:* None *Results:* Local: - Total Time to stream and read 1 audio files: 32ms. - average complete read and write: 32ms - average time to first byte read: 2ms - average read time per runner: 23ms - average write time per runner: 11ms - p99 time to first byte: 2ms - p90 time to first byte: 2ms - p50 time to first byte: 2ms Remote (Over SSH | Seattle → IAD): - Total Time to stream and read 1 audio files: 259ms. - average complete read and write: 258ms - average time to first byte read: 19ms - average read time per runner: 232ms - average write time per runner: 169ms - p99 time to first byte: 19ms - p90 time to first byte: 19ms - p50 time to first byte: 19ms Remote (Through VIP | Seattle → IAD): - Total Time to stream and read 1 audio files: 203ms. - average complete read and write: 203ms - average time to first byte read: 20ms - average read time per runner: 174ms - average write time per runner: 93ms - p99 time to first byte: 20ms - p90 time to first byte: 20ms - p50 time to first byte: 20ms UPDATE: To make more apparent what I'm trying to do: I'm going to have 50+ million devices streaming audio. The streams could be 100kb on average and 200k streams/minute at peak traffic. I'm looking for a storage solution to accommodate that need. I've been examining Bookkeeper, Kafka, Ignite, Cassandra, and Redis. So far i've only benchmarked redis and ignite, but i'm surprised ignite is so slow.
I reviewed your benchmark and made a couple runs locally. I was able to make it much faster: 30 iteration isn't enough for JVM to warm up, on my laptop, it requires ~150 iterations. So I increased it to 300 iterations. I moved cache creation out of the benchmark, I added it right after cache destroying. Also, I moved out of benchmark ignite client creation, it's extremely expensive operation and in real life, you should reuse it. Please take a look at my changes, I created a pull request: https://github.com/Sahasrara/AudioStreamStoreDemo/pull/1/files
I don't think you should be creating a cache for every operation. This is a heavy operation with Ignite. What are your requirements for this? I can see how performance greatly improves for subsequent runs. Ignite is based on Java, which needs some time to warm-up.
You should definitely avoid creating a lot of caches. Use a cache group in order to share infrastructure between your caches. Keep a barrel of N caches and re-purpose them for a new files as time goes by, freeing them once file is no longer used. This will need keeping some accounting. Better yet, find a way to just use one cache, keep file identifier in composite cache key, keep track of what you have in cache.
optaplanner incorrect cloud balance output
I am trying a simple Cloud balance example in the opta planner tutorial.
I have 2 computers and 4 process ( the same example which is explained in the opta planner documentation ).Each process needs certain amount of RAM and CPU .
The final solved output is incorrect which you can see from the logs below. Hard rules are violated.
Ideally Process 1 and 4 should fit in computer 1 .
Process 2 and 3 should fit in computer 2.
From the logs looks like HeuristicPhase happens but local search is not showing up in the logs.
Please guide me what I am dong wrong.
After building unsolvedCloudBalance Computers [ Computer [C1][7 CPU][6 RAM], Computer [C2][6 CPU][6 RAM]] Processes [ Process [P1][5 CPU][5 RAM], Process [P2][4 CPU][3 RAM], Process [P3][2 CPU][3 RAM], Process [P4][2 CPU][1 RAM]] Hard Soft Score null
01:35:57.099 [main] INFO o.o.core.impl.solver.DefaultSolver - Solving started: time spent (3), best score (4uninitialized/0hard/0soft), environment mode (REPRODUCIBLE), random (JDK with seed 0).
01:35:57.099 [main] INFO o.o.core.impl.solver.DefaultSolver - Solving started: time spent (3), best score (4uninitialized/0hard/0soft), environment mode (REPRODUCIBLE), random (JDK with seed 0).
01:35:57.104 [main] DEBUG o.o.c.i.c.DefaultConstructionHeuristicPhase - CH step (0), time spent (9), score (0hard/-800soft), selected move count (2), picked move ( Process [P1][5 CPU][5 RAM] {null -> Computer [C2][6 CPU][6 RAM]}).
01:35:57.104 [main] DEBUG o.o.c.i.c.DefaultConstructionHeuristicPhase - CH step (0), time spent (9), score (0hard/-800soft), selected move count (2), picked move ( Process [P1][5 CPU][5 RAM] {null -> Computer [C2][6 CPU][6 RAM]}).
01:35:57.104 [main] DEBUG o.o.c.i.c.DefaultConstructionHeuristicPhase - CH step (1), time spent (9), score (0hard/-1800soft), selected move count (2), picked move ( Process [P2][4 CPU][3 RAM] {null -> Computer [C1][7 CPU][6 RAM]}).
01:35:57.104 [main] DEBUG o.o.c.i.c.DefaultConstructionHeuristicPhase - CH step (1), time spent (9), score (0hard/-1800soft), selected move count (2), picked move ( Process [P2][4 CPU][3 RAM] {null -> Computer [C1][7 CPU][6 RAM]}).
01:35:57.104 [main] DEBUG o.o.c.i.c.DefaultConstructionHeuristicPhase - CH step (2), time spent (9), score (0hard/-1800soft), selected move count (1), picked move ( Process [P3][2 CPU][3 RAM] {null -> Computer [C1][7 CPU][6 RAM]}).
01:35:57.104 [main] DEBUG o.o.c.i.c.DefaultConstructionHeuristicPhase - CH step (2), time spent (9), score (0hard/-1800soft), selected move count (1), picked move ( Process [P3][2 CPU][3 RAM] {null -> Computer [C1][7 CPU][6 RAM]}).
01:35:57.105 [main] DEBUG o.o.c.i.c.DefaultConstructionHeuristicPhase - CH step (3), time spent (10), score (-1hard/-1800soft), selected move count (2), picked move ( Process [P4][2 CPU][1 RAM] {null -> Computer [C2][6 CPU][6 RAM]}).
01:35:57.105 [main] DEBUG o.o.c.i.c.DefaultConstructionHeuristicPhase - CH step (3), time spent (10), score (-1hard/-1800soft), selected move count (2), picked move ( Process [P4][2 CPU][1 RAM] {null -> Computer [C2][6 CPU][6 RAM]}).
01:35:57.106 [main] INFO o.o.c.i.c.DefaultConstructionHeuristicPhase - Construction Heuristic phase (0) ended: step total (4), time spent (11), best score (-1hard/-1800soft).
01:35:57.106 [main] INFO o.o.c.i.c.DefaultConstructionHeuristicPhase - Construction Heuristic phase (0) ended: step total (4), time spent (11), best score (-1hard/-1800soft).
01:36:07.095 [main] DEBUG o.o.c.i.l.DefaultLocalSearchPhase - LS step (0), time spent (10000), score (-2hard/-1800soft), best score (-1hard/-1800soft), accepted/selected move count (0/9152031), picked move ( Process [P4][2 CPU][1 RAM] { Computer [C2][6 CPU][6 RAM] -> Computer [C1][7 CPU][6 RAM]}).
01:36:07.095 [main] DEBUG o.o.c.i.l.DefaultLocalSearchPhase - LS step (0), time spent (10000), score (-2hard/-1800soft), best score (-1hard/-1800soft), accepted/selected move count (0/9152031), picked move ( Process [P4][2 CPU][1 RAM] { Computer [C2][6 CPU][6 RAM] -> Computer [C1][7 CPU][6 RAM]}).
01:36:07.096 [main] INFO o.o.c.i.l.DefaultLocalSearchPhase - Local Search phase (1) ended: step total (1), time spent (10001), best score (-1hard/-1800soft).
01:36:07.096 [main] INFO o.o.c.i.l.DefaultLocalSearchPhase - Local Search phase (1) ended: step total (1), time spent (10001), best score (-1hard/-1800soft).
01:36:07.096 [main] INFO o.o.core.impl.solver.DefaultSolver - Solving ended: time spent (10001), best score (-1hard/-1800soft), average calculate count per second (915112), environment mode (REPRODUCIBLE).
01:36:07.096 [main] INFO o.o.core.impl.solver.DefaultSolver - Solving ended: time spent (10001), best score (-1hard/-1800soft), average calculate count per second (915112), environment mode (REPRODUCIBLE).
time taken 10 sec
It worked as expected after I added difficultyComparatorClass to the #PlanningEntity annotation.
How can i get OptaPlanner Construction Heuristics to use different values?
When i use the Construction Heuristics i always get the same values assigned to my PlanningVariables which leads to a bad initial solution.
I am quite sure i make an error here but cannot see where i fail.
My Problem Description:
I try to assign resources to WorkOrders. One WorkOrder can have several ResourceAssignments. This ResourceAssignment is my main PlanningEntity.
The resource is the PlanningVariable.
WorkOrders are also PlanningEntities but that information is only necessary here to show i have more than one PlanningEntity and therefore cannot go with the more simple ConstructionHeuristic configurations.
I use value range provider from entity so i can give back different sets of Resources for different types of ResourceAssignments. (e.g. Human, Forklift, Handscanner ...)
My problem is that the Construction Heuristic seems to assign always the same Resource to all ResourceAssignment.
Here is the output from the CH - as you can see the same Human Resource (id 100) is assigned to both ResourceAssignments. This is the same for all following PlanningEntities.
2015-07-23 10:57:12,291 [main] INFO Solving started: time spent (185), best score (uninitialized/-9900hard/0soft), environment mode (FAST_ASSERT), random (JDK with seed 0).
2015-07-23 10:57:12,429 [main] DEBUG CH step (0), time spent (325), score (-9900hard/0soft), selected move count (200), picked move (ResourceAssignment{id=0, workorder=WorkOrder{id=0, name=WO0, start=Thu Jul 23 12:57:11 CEST 2015, stop=Thu Jul 23 14:57:11 CEST 2015, offset=0}, resource=null, type=human} => Resource{type=human, id=100}).
2015-07-23 10:57:12,523 [main] DEBUG CH step (1), time spent (419), score (-9900hard/0soft), selected move count (200), picked move (ResourceAssignment{id=1, workorder=WorkOrder{id=0, name=WO0, start=Thu Jul 23 12:57:11 CEST 2015, stop=Thu Jul 23 14:57:11 CEST 2015, offset=0}, resource=null, type=human} => Resource{type=human, id=100}).
2015-07-23 10:57:12,576 [main] DEBUG CH step (2), time spent (472), score (-9900hard/0soft), selected move count (99), picked move (ResourceAssignment{id=2, workorder=WorkOrder{id=0, name=WO0, start=Thu Jul 23 12:57:11 CEST 2015, stop=Thu Jul 23 14:57:11 CEST 2015, offset=0}, resource=null, type=forklift} => Resource{type=forklift, id=0}).
2015-07-23 10:57:12,667 [main] DEBUG CH step (3), time spent (563), score (-9900hard/0soft), selected move count (200), picked move (ResourceAssignment{id=3, workorder=WorkOrder{id=1, name=WO1, start=Thu Jul 23 12:57:11 CEST 2015, stop=Thu Jul 23 14:57:11 CEST 2015, offset=0}, resource=null, type=human} => Resource{type=human, id=100}).
2015-07-23 10:57:12,757 [main] DEBUG CH step (4), time spent (653), score (-9900hard/0soft), selected move count (200), picked move (ResourceAssignment{id=4, workorder=WorkOrder{id=1, name=WO1, start=Thu Jul 23 12:57:11 CEST 2015, stop=Thu Jul 23 14:57:11 CEST 2015, offset=0}, resource=null, type=human} => Resource{type=human, id=100}).
2015-07-23 10:57:12,795 [main] DEBUG CH step (5), time spent (691), score (-9900hard/0soft), selected move count (99), picked move (ResourceAssignment{id=5, workorder=WorkOrder{id=1, name=WO1, start=Thu Jul 23 12:57:11 CEST 2015, stop=Thu Jul 23 14:57:11 CEST 2015, offset=0}, resource=null, type=forklift} => Resource{type=forklift, id=0}).
2015-07-23 10:57:12,867 [main] DEBUG CH step (6), time spent (763), score (-9900hard/0soft), selected move count (200), picked move (ResourceAssignment{id=6, workorder=WorkOrder{id=2, name=WO2, start=Thu Jul 23 12:57:11 CEST 2015, stop=Thu Jul 23 14:57:11 CEST 2015, offset=0}, resource=null, type=human} => Resource{type=human, id=100}).
2015-07-23 10:57:12,917 [main] DEBUG CH step (7), time spent (813), score (-9900hard/0soft), selected move count (200), picked move (ResourceAssignment{id=7, workorder=WorkOrder{id=2, name=WO2, start=Thu Jul 23 12:57:11 CEST 2015, stop=Thu Jul 23 14:57:11 CEST 2015, offset=0}, resource=null, type=human} => Resource{type=human, id=100}).
It seems like it ignores all other moves - and of course i have a rule that punishes using the same resource at the same time with a hard score.
Here is my solution configuration.
<?xml version="1.0" encoding="UTF-8"?>
<solver>
<environmentMode>FAST_ASSERT</environmentMode>
<termination>
<unimprovedSecondsSpentLimit>10000</unimprovedSecondsSpentLimit>
<secondsSpentLimit>220000</secondsSpentLimit>
</termination>
<!-- Domain model configuration -->
<solutionClass>com.opal.solver.resource.ResourceAssignmentSolution</solutionClass>
<entityClass>com.opal.solver.resources.entity.ResourceAssignment</entityClass>
<entityClass>com.opal.solver.resources.entity.WorkOrder</entityClass>
<!-- Score configuration -->
<scoreDirectorFactory>
<scoreDefinitionType>HARD_SOFT</scoreDefinitionType>
<scoreDrl>resourceAsssignmentRules.drl</scoreDrl>
</scoreDirectorFactory>
<constructionHeuristic>
<queuedEntityPlacer>
<entitySelector id="placerEntitySelector">
<cacheType>PHASE</cacheType>
<entityClass>com.opal.solver.resources.entity.ResourceAssignment</entityClass>
</entitySelector>
<changeMoveSelector>
<entitySelector mimicSelectorRef="placerEntitySelector"/>
<valueSelector>
<selectionOrder>ORIGINAL</selectionOrder>
<variableName>resource</variableName>
</valueSelector>
</changeMoveSelector>
</queuedEntityPlacer>
</constructionHeuristic>
</solver>
I am sure I just miss some core concept of optaplanner here. I would be grateful for any hint into the right direction.
Looks like you have a verbose configuration of FIRST_FIT right now. Use FIRST_FIT_DECREASING instead, by declaring difficulty comparison and using that construction heuristic type. <constructionHeuristic> <constructionHeuristicType>FIRST_FIT_DECREASING</> </> See docs chapter on First Fit Decreasing. Normally, FFD is clearly better than FF (for example 4% on CloudBalancing on average), but use Local Search to improve upon that solution. If you're really looking for random different CH results (for example for GeneticAlgorithm population) use selectionOrder RANDOM instead of ORGINAL.
Unusually long Pig job start time
A pig script (not particularly more complex than any others I have built) before the job starts it seems to loop on this for a long time: 2013-10-08 10:46:07,655 [main] INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 10 2013-10-08 10:46:07,659 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths to process : 10 2013-10-08 10:46:09,168 [main] INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 10 2013-10-08 10:46:09,168 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths to process : 10 2013-10-08 10:46:11,381 [main] INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 10 2013-10-08 10:46:11,381 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths to process : 10 2013-10-08 10:46:13,875 [main] INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 10 2013-10-08 10:46:13,875 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths to process : 10 2013-10-08 10:46:16,303 [main] INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 10 It repeats the above for around 4 minutes when usually this step is completed in seconds. I have not been able to identify the cause - other than removing parts of the script but the issue does not seem to be caused by any particular part of the script. I have other scripts as complex as this one and I have not had this problem. What could be causing the issue?
I can't say for certain without more information, but it appears that pig is waiting for your cluster's JobTracker to start running the underlying Map/Reduce jobs generated by your script. There are numerious reasons why this could be happening such as running on a shared cluster which has run out of resources. You'll most likely have to look at your cluster's JobTracker and/or TaskTrackers to know the exact reason.