Error in OptaPlanner when multithreading (External object cannot be looked up) - optaplanner

I have developped an optimization module using OptaPlanner but have encountered an error when multithreading. Note that the module works when multithreading is disabled. I tried several OptaPlanner versions (8.4, 8.1, 7.51) and different JDKs (16 and 14).
I used a simple solver configuration file
**
My planning entity is also straghtforward :
**
#PlanningSolution public class SuccessPromotion {
#ProblemFactCollectionProperty
public ArrayList<RCT> rcts;
#PlanningEntityCollectionProperty
public ArrayList<RCT_Planning> rcts_promos;
#ValueRangeProvider(id = "availablePromotion")
#ProblemFactCollectionProperty
public List<int[]> promotions;
#PlanningScore
public HardSoftScore score;
#PlanningId
public String name; //Constructors //Getter & Setter }
**
The error I get is as follows :
**
Exception in thread "main" java.lang.IllegalStateException: The move
thread with moveThreadIndex (3) has thrown an exception. Relayed here
in the parent thread. at
org.optaplanner.core.impl.heuristic.thread.OrderByMoveIndexBlockingQueue.take(OrderByMoveIndexBlockingQueue.java:147)
at
org.optaplanner.core.impl.localsearch.decider.MultiThreadedLocalSearchDecider.forageResult(MultiThreadedLocalSearchDecider.java:188)
at
org.optaplanner.core.impl.localsearch.decider.MultiThreadedLocalSearchDecider.decideNextStep(MultiThreadedLocalSearchDecider.java:159)
at
org.optaplanner.core.impl.localsearch.DefaultLocalSearchPhase.solve(DefaultLocalSearchPhase.java:71)
at
org.optaplanner.core.impl.solver.AbstractSolver.runPhases(AbstractSolver.java:99)
at
org.optaplanner.core.impl.solver.DefaultSolver.solve(DefaultSolver.java:189)
at test.RealSolver.main(RealSolver.java:140) Caused by:
java.lang.IllegalArgumentException: The externalObject ([I#5ce1e52a)
cannot be looked up. Some functionality, such as multithreaded
solving, requires this ability. Maybe add an #PlanningId annotation on
an identifier property of the class (class [I). Or otherwise, maybe
change the #PlanningSolution annotation's LookUpStrategyType (not
recommended). at
org.optaplanner.core.impl.domain.lookup.NoneLookUpStrategy.lookUpWorkingObject(NoneLookUpStrategy.java:44)
at
org.optaplanner.core.impl.domain.lookup.LookUpManager.lookUpWorkingObject(LookUpManager.java:76)
at
org.optaplanner.core.impl.score.director.AbstractScoreDirector.lookUpWorkingObject(AbstractScoreDirector.java:512)
at
org.optaplanner.core.impl.heuristic.selector.move.generic.ChangeMove.rebase(ChangeMove.java:83)
at
org.optaplanner.core.impl.heuristic.selector.move.generic.ChangeMove.rebase(ChangeMove.java:31)
at
org.optaplanner.core.impl.heuristic.thread.MoveThreadRunner.run(MoveThreadRunner.java:140)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
**

Your error message is
The externalObject ([I#5ce1e52a) cannot be looked up. Some functionality, such as multithreaded solving, requires this ability. Maybe add an #PlanningId annotation on an identifier property of the class (class [I).
That class[I talks about your planning value class (int[]), not your planning entity class (ArrayList<RCT_Planning>).
The planning value class int[] is neither immutable, nor has a #PlanningId, so it can't rebase it easily to another thread (although in theory it could copy it, it doesn't do so at the moment). The error message could be better.
Workaround: wrap int[] in a planning value class.

Related

When I subclass a class using ByteBuddy in certain situations I get IllegalAccessErrors. Why?

(I am a new ByteBuddy user. I'm using ByteBuddy version 1.10.8 and JDK 11 without the module path or any other part of the module system.)
I have a nested class declared like this:
public static class Frob {
protected Frob() {
super();
}
public String sayHello() {
return "Hello!";
}
}
(Its containing class is foo.bar.TestExplorations.)
When I create a dynamic subclass of Frob named foo.bar.Crap like the following, everything works OK as I would expect:
final String className = "foo.bar.Crap";
final DynamicType.Unloaded<?> dynamicTypeUnloaded = new ByteBuddy()
.subclass(Frob.class)
.name(className)
.make();
final Class<?> mySubclass = dynamicTypeUnloaded
.load(this.getClass().getClassLoader(), ClassLoadingStrategy.Default.WRAPPER)
.getLoaded();
assertNotNull(mySubclass);
assertEquals(className, mySubclass.getName());
final Object frobSubclass = mySubclass.newInstance();
assertTrue(frobSubclass instanceof Frob);
But if I change Frob's constructor so that it is package private, I get the following error from the final assertion:
java.lang.IllegalAccessError: class foo.bar.Crap tried to access method 'void foo.bar.TestExplorations$Frob.<init>()' (foo.bar.Crap is in unnamed module of loader net.bytebuddy.dynamic.loading.ByteArrayClassLoader #5e3d57c7; foo.bar.TestExplorations$Frob is in unnamed module of loader 'app')
For some reason, Crap's constructor cannot call super(), even though Crap and Frob are in the same package, and Frob() is defined as package-private.
I have a sense the JDK module system is to blame here, even though I am deliberately (very, very deliberately) not using it. I know the module system does not like split packages, which is what it looks like to me is going on here. Is there a constructor strategy or other mechanism to work around this problem?
In Java, a package is only equal to another package if it has the same name and is loaded by the same class loader (the same as it is with classes). If you are using the WRAPPER strategy, you cannot access package-private members of any super class. Byte Buddy does not forbid the generation as it would be legal to do in javac but you would need to use the INJECTION strategy to do what you want to make sure that classes are loaded by the same class loader. Mind that it uses internal API, therefore, from Java 9, you'd rather use a ForLookup class loading strategy.

Optaplanner 7.9.0 and Adding Multithreading: same planningId exception

I've had an optaplanner project working for a while now that is allocating jobs to a list of workers using various scoring properties all written in java. Happy with the way it was working I decided to update from 7.7.0 to 7.9.0 and turn on the new multithreading option and see if it works any faster. However, I now get the following error:
Exception in thread "Thread-8" java.lang.IllegalStateException: The move thread with moveThreadIndex (0) has thrown an exception. Relayed here in the parent thread.
Caused by: java.lang.IllegalArgumentException: The externalObject (...Worker#72d2327a) cannot be looked up.
Maybe give the class (class Worker) a PlanningId annotation or change the PlanningSolution annotation's LookUpStrategyType or don't rely on functionality that depends on ScoreDirector.lookUpWorkingObject().
So I tried adding #PlanningId to the unique identifier of the worker class and I get:
Exception in thread "Thread-7" java.lang.IllegalStateException: The workingObjects (Worker#4438b624, Worker#4438b624) have the same planningId ((class Worker,50)). Working objects must be unique.
...but it looks like it's comparing a Worker object with itself, so of course it's the same id????
Perhaps this is down to my java code in the assign/unassign methods but I don't understand where this issue is coming from, anyone have any ideas?
Edit:
So after a lot of help from #n1ck I got as far as changing the method in the #PlanningSolution class that returns the list of Workers to create new instances each time but with the same values. This the changes the error to be : '...The workingObjects (Worker#1eba2b4d, Worker#67a7ef08) have the same planningId...' So it's no longer comparing identical instances but does seem to be comparing it to an instance from a previous call. This does however at least confirm there is no duplication per List returned. So I then changed the planning id to be a new (Long) field whose value is always incremented from the last instance created i.e. first call Worker ids are 1-10, next call 11-20 and so on. This bypasses the error altogether but now nothing gets allocated to the Workers. It does the initial set up and null assigns and then stops until the time limit on the solver is reached. I still think this is something I haven't set right but the error is possibly a bit of a red herring, however as requested the full stack trace as I can see it is:
Exception in thread "Thread-10" java.lang.IllegalStateException: The workingObjects (Worker#42094cc1, Worker#647a1c5c) have the same planningId ((class Worker,50)). Working objects must be unique.
at org.optaplanner.core.impl.domain.lookup.PlanningIdLookUpStrategy.addWorkingObject(PlanningIdLookUpStrategy.java:40)
at org.optaplanner.core.impl.domain.lookup.LookUpManager.addWorkingObject(LookUpManager.java:49)
at org.optaplanner.core.impl.domain.lookup.LookUpManager.resetWorkingObjects(LookUpManager.java:43)
at org.optaplanner.core.impl.score.director.AbstractScoreDirector.setWorkingSolution(AbstractScoreDirector.java:167)
at org.optaplanner.core.impl.score.director.incremental.IncrementalScoreDirector.setWorkingSolution(IncrementalScoreDirector.java:62)
at org.optaplanner.core.impl.solver.scope.DefaultSolverScope.setWorkingSolutionFromBestSolution(DefaultSolverScope.java:230)
at org.optaplanner.core.impl.solver.AbstractSolver.solvingStarted(AbstractSolver.java:75)
at org.optaplanner.core.impl.solver.DefaultSolver.solvingStarted(DefaultSolver.java:210)
at org.optaplanner.core.impl.solver.DefaultSolver.solve(DefaultSolver.java:190)
at SolverThread.run(SolverThread.java:86)
Edit: #PlanningSolution class as requested:
package code.test.scheduler;
import org.optaplanner.core.api.domain.solution.PlanningEntityCollectionProperty;
import org.optaplanner.core.api.domain.solution.PlanningScore;
import org.optaplanner.core.api.domain.solution.PlanningSolution;
import org.optaplanner.core.api.domain.solution.drools.ProblemFactCollectionProperty;
import org.optaplanner.core.api.domain.valuerange.ValueRangeProvider;
import org.optaplanner.core.api.score.buildin.hardmediumsoft.HardMediumSoftScore;
import org.optaplanner.persistence.xstream.api.score.buildin.hardmediumsoft.HardMediumSoftScoreXStreamConverter;
import com.thoughtworks.xstream.annotations.XStreamConverter;
#PlanningSolution
public class Planner
{
#ProblemFactCollectionProperty
private java.util.List<Worker> _workerList;
#ValueRangeProvider(id="workerRange")
#ProblemFactCollectionProperty
public java.util.List<Worker> getWorkerList() {
return _workerList;
}
public void setWorkerList(java.util.List<Worker> newWorkerList) {
_workerList = newWorkerList;
}
public int getWorkerListSize() {
return _workerList.size();
}
#ProblemFactCollectionProperty
private int[][] _travelTimes;
#ValueRangeProvider(id="travelTimes")
#ProblemFactCollectionProperty
public int[][] getTravelTimes() {
return _travelTimes;
}
public void setTravelTimes(int[][] newTravelTimes) {
_travelTimes = newTravelTimes;
}
#PlanningEntityCollectionProperty
private java.util.List<Job> _jobList;
public java.util.List<Job> getJobList() {
return _jobList;
}
public void setJobList(java.util.List<Job> newJobList) {
_jobList = newJobList;
}
#XStreamConverter(HardMediumSoftScoreXStreamConverter.class)
private HardMediumSoftScore _score;
#PlanningScore
public HardMediumSoftScore getScore() {
return _score;
}
public void setScore(HardMediumSoftScore score) {
_score = score;
}
}
I think you have some duplicate entities in your PlanningEntityCollection. Are you sure all Worker objects in this list are unique (as in, not the same instance)?
I duplicated an entity in my entity collection on purpose, and got the same error.
Edit: it turned out the issue was declaring the #ProblemFactCollectionProperty annotation above both the field and getter. It should only be declared above one or the other. Declaring it above both the field and getter, in conjunction with using the #PlanningId annotation, will result in the 'Working objects must be unique' error.

Mule DataMapper IOException

I have a class hierarchy with a base class called EntityModel, and two classes InvestorModel and AgentModel that each inherit directly from it and add a few properties. I am then creating Mule Data Maps to map JSON to each child class individually.
The InvestorModel map works fine, but the AgentModel map fails (in the IDE preview) with an IOException stating that it can't instantiate EntityModel. This seems strange as it can instantiate it in the InvestorModel map. I'm posting the error, but I don't really have any source to post as these are just mapping files. I just don't know where to start looking.
Mule Studio is up to date and v3.5.0
java.io.IOException: org.jetel.exception.JetelException: za.co.sci.core.shared.EntityModel can not be instantiated.
at org.jetel.component.tree.writer.TreeFormatter.write(TreeFormatter.java:72)
at org.jetel.util.MultiFileWriter.writeRecord2CurrentTarget(MultiFileWriter.java:420)
at org.jetel.util.MultiFileWriter.write(MultiFileWriter.java:297)
at org.jetel.component.TreeWriter.execute(TreeWriter.java:464)
at org.jetel.graph.Node.run(Node.java:465)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: org.jetel.exception.JetelException: za.co.sci.core.shared.EntityModel can not be instantiated.
at com.opensys.cloveretl.component.tree.writer.bean.BeanWriter.a(Unknown Source)
at com.opensys.cloveretl.component.tree.writer.bean.BeanWriter.a(Unknown Source)
at com.opensys.cloveretl.component.tree.writer.bean.BeanWriter.a(Unknown Source)
at com.opensys.cloveretl.component.tree.writer.bean.BeanWriter.writeStartNode(Unknown Source)
at org.jetel.component.tree.writer.model.runtime.WritableObject.writeContent(WritableObject.java:67)
at org.jetel.component.tree.writer.model.runtime.WritableContainer.write(WritableContainer.java:67)
at org.jetel.component.tree.writer.model.runtime.WritableObject.writeContent(WritableObject.java:77)
at org.jetel.component.tree.writer.model.runtime.WritableContainer.write(WritableContainer.java:67)
at org.jetel.component.tree.writer.model.runtime.WritableObject.writeContent(WritableObject.java:77)
at org.jetel.component.tree.writer.model.runtime.WritableContainer.write(WritableContainer.java:67)
at org.jetel.component.tree.writer.model.runtime.WritableObject.writeContent(WritableObject.java:77)
at org.jetel.component.tree.writer.model.runtime.WritableContainer.write(WritableContainer.java:67)
at org.jetel.component.tree.writer.model.runtime.WritableObject.writeContent(WritableObject.java:77)
at org.jetel.component.tree.writer.TreeFormatter.write(TreeFormatter.java:69)
... 7 more
Class snippets:
public abstract class EntityModel implements Serializable {
protected Long id;
private long entityNumber;
private EntityStatus status;
private String entityName;
...
public class AgentModel extends EntityModel implements Serializable{
private int agentCode;
private AgentType agentType;
private AgentClass agentClass;
...
public class InvestorModel extends EntityModel implements Serializable {
private boolean blockedRand;
private String utAUTType;
...
Turns out the error was due to the base class being abstract. Kind of obvious really.
The reason the one map worked, but not the other was because of the order of the fields. The first field to be mapped on the InvestorModel was a field defined in InvestorModel so the mapper knew which class to instantiate. On the AgentModel map the first field was defined on the abstract class EntityModel so the mapper tried to instantiate that class, but failed, it didn't matter that I have chosen AgentModel as the destination.

GemFire: serialize objects in Java and then deserialize them in c#

To cross the language boundary in Java side the class to be serialized needs to implement the DataSerializable interface; and in order to let the deserializer in c# know what class it is , we need to register a classID. Following the example, I write my class in Java like this:
public class Stuff implements DataSerializable{
static { // note that classID (7) must match C#
Instantiator.register(new Instantiator(Stuff.class,(byte)0x07) {
#Override
public DataSerializable newInstance() {
return new Stuff();
}
});
}
private Stuff(){}
public boolean equals(Object obj) {...}
public int hashCode() {...}
public void toData(DataOutput dataOutput) throws IOException {...}
public void fromData(DataInput dataInput) throws IOException, ClassNotFoundException { ...}
}
It looks OK but when I run it I get this exception:
[warning 2012/03/30 15:06:00.239 JST tid=0x1] Error registering
instantiator on pool:
com.gemstone.gemfire.cache.client.ServerOperationException: : While
performing a remote registerInstantiators at
com.gemstone.gemfire.cache.client.internal.AbstractOp.processAck(AbstractOp.java:247)
at
com.gemstone.gemfire.cache.client.internal.RegisterInstantiatorsOp$RegisterInstantiatorsOpImpl.processResponse(RegisterInstantiatorsOp.java:76)
at
com.gemstone.gemfire.cache.client.internal.AbstractOp.attemptReadResponse(AbstractOp.java:163)
at
com.gemstone.gemfire.cache.client.internal.AbstractOp.attempt(AbstractOp.java:363)
at
com.gemstone.gemfire.cache.client.internal.ConnectionImpl.execute(ConnectionImpl.java:229)
at
com.gemstone.gemfire.cache.client.internal.pooling.PooledConnection.execute(PooledConnection.java:321)
at
com.gemstone.gemfire.cache.client.internal.OpExecutorImpl.executeWithPossibleReAuthentication(OpExecutorImpl.java:646)
at
com.gemstone.gemfire.cache.client.internal.OpExecutorImpl.execute(OpExecutorImpl.java:108)
at
com.gemstone.gemfire.cache.client.internal.PoolImpl.execute(PoolImpl.java:624)
at
com.gemstone.gemfire.cache.client.internal.RegisterInstantiatorsOp.execute(RegisterInstantiatorsOp.java:39)
at
com.gemstone.gemfire.internal.cache.PoolManagerImpl.allPoolsRegisterInstantiator(PoolManagerImpl.java:216)
at
com.gemstone.gemfire.internal.InternalInstantiator.sendRegistrationMessageToServers(InternalInstantiator.java:188)
at
com.gemstone.gemfire.internal.InternalInstantiator._register(InternalInstantiator.java:143)
at
com.gemstone.gemfire.internal.InternalInstantiator.register(InternalInstantiator.java:71)
at com.gemstone.gemfire.Instantiator.register(Instantiator.java:168)
at Stuff.(Stuff.java)
Caused by: java.lang.ClassNotFoundException: Stuff$1
I could not figure out why, is there anyone who has experience can help? Thanks in advance!
In most configurations GemFire servers need to deserialize objects in order to index them, run queries and call listeners. So when you register instantiator the class will be registered on all machines in the Distributed System. Hence, the class itself must be available for loading everywhere in the cluster.
As exception stack trace says the error happens on a remote node.
Check if you have the class Stuff on all machines participating in the cluster. At least on cache servers.

Suppress obsolete warning on factory method in vb.net

Is it possible to prevent compiler warning caused by a static factory method returning an instance of an obsolete class?
For example if an instance of an obsolete class (Foo) is created through a static method (Create) a compiler warning is caused by the factory method.
<Obsolete()> _
Public Class Foo
Public Shared Function Create() As Foo
' Instantiate and return
End Function
End Class
In C# the warning could be suppressed using "#pragma warning..." but I don't think that this exists within VB.Net. Migrating to C# is not an option due to business requirements.
I think you have to open the .vbproj-file with notepad, search the NoWarn-Tag and add the error-id there(618 should be obsolete-warning):
<NoWarn>618,42016,41999,42017,42018,42019,42032,42036,42020,42021,42022</NoWarn>