I am writing a modified version of the Task Assignment example with my own domain model.
In my model each Task can have a NextTask and a PreviousTask and an Assignee. All 3 are configured as PlanningVariables:
...
/** PreviousTask is a calculated task that the Resource will complete before this one. */
#PlanningVariable(valueRangeProviderRefs = { "tasksRange" }, graphType = PlanningVariableGraphType.CHAINED)
public Task PreviousTask;
#InverseRelationShadowVariable(sourceVariableName = "PreviousTask")
public Task NextTask;
#AnchorShadowVariable(sourceVariableName = "PreviousTask")
public Resource Assignee;
I have been stuck on the Local Search Phase step for some time now as it appears to require an initialized state of my planning variables (Task.PreviousTask in this case).
Error log:
2020-07-16 15:00:15.341 INFO 4616 --- [pool-1-thread-1] o.o.core.impl.solver.DefaultSolver : Solving started: time spent (65), best score (-27init/[0]hard/[0/0/0/0]soft), environment mode (REPRODUCIBLE), random (JDK with seed 0).
2020-07-16 15:00:15.356 INFO 4616 --- [pool-1-thread-1] .c.i.c.DefaultConstructionHeuristicPhase : Construction Heuristic phase (0) ended: time spent (81), best score (-27init/[0]hard/[0/0/0/0]soft), score calculation speed (90/sec), step total (0).
2020-07-16 15:00:15.376 ERROR 4616 --- [pool-1-thread-1] o.o.c.impl.solver.DefaultSolverManager : Solving failed for problemId (5e433f57-8c75-4756-8a9c-4c4ca4a83d6d).
java.lang.IllegalStateException: Local Search phase (1) needs to start from an initialized solution, but the planning variable (Task.PreviousTask) is uninitialized for the entity (com.redhat.optaplannersbs.domain.Task#697ea710).
Maybe there is no Construction Heuristic configured before this phase to initialize the solution.
I've been pouring over the documentation and trying to figure out what I've missed or broken between it and the source example but I cannot work it out. I have no Construction Heuristic configured (Same as the example), and I could not see where in the example does it ever set the previousTaskOrEmployee variable before solving.
I could supply a random PreviousTask in the initial solution model, but surely at least 1 of the tasks would have no previous task?
May I ask if you have <localSearch/> in your solverConfig.xml? If so, the <constructionHeuristic/> has to be there as well.
If none of these phases (CH nor LS) is configured, both Construction Heuristic and Local Search is added with default parameters. But once the appears in the solverConfig.xml, it's considered an override of the defaults, and a user is supposed to take care of initializing the solution (most often by providing the Construction Heuristic configuration).
Related
I´ve got a Job that runs a TaskletStep, then a chunk-based step and then another TaskletStep.
In each of these steps, errors (in the form of Exceptions) can occur.
The chunk-based step looks like this:
stepBuilderFactory
.get("step2")
.chunk<SomeItem, SomeItem>(1)
.reader(flatFileItemReader)
.processor(itemProcessor)
.writer {}
.faultTolerant()
.skipPolicy { _ , _ -> true } // skip all Exceptions and continue
.taskExecutor(taskExecutor)
.throttleLimit(taskExecutor.corePoolSize)
.build()
The whole job definition:
jobBuilderFactory.get("job1")
.validator(validator())
.preventRestart()
.start(taskletStep1)
.next(step2)
.next(taskletStep2)
.build()
I expected that Spring Batch somehow picks up the Exceptions that occur along the way, so I can then create a Report including them after the Job has finished processing. Looking at the different contexts, there´s also fields that should contain failureExceptions. However, it seems there´s no such information (especially for the chunked step).
What would be a good approach if I need information about:
what Exceptions did occur in which Job execution
which Item was the one that triggered it
The JobExecution provides a method to get all failure exceptions that happened during the job. You can use that in a JobExecutionListener#afterJob(JobExecution jobExecution) to generate your report.
In regards to which items caused the issue, this will depend on where the exception happens (during the read, process or write operation). For this requirement, you can use one of the ItemReadListener, ItemProcessListener or ItemWriteListener to keep record of the those items (For example, by adding them to the job execution context to be able to get access to them in the JobExecutionListener#afterJob method for your report).
I am trying implement a job shop scheduling application using Fisher & Thompson mt10 dataset. Basically it contains of
10 jobs, each having 10 dependent steps
10 machines
each step of a job is assigned to a specific machine
I have implemented an Optaplanner use case based on the "Taskassigning" example. I removed the speed and priority concepts but kept the skill concept to make jobs run only on machines where they are able to run. I introduced a "predecessor" concept to build the dependencies between jobs/steps.
As there will be gaps in the schedule (which is different from the Taskassigning example), removed starttime and endtime and introduced a starttime PlanningVariable, fed by a list of possible start times.
However, I only get two different start times in the schedule - Optaplanner does not seem to utilize my value range provider. Therefore, hard constraints are violated because the sequence of dependent steps is not kept.
Job:
private JobType jobType;
private Job predecessor;
private Job successor;
private int indexInJobType;
// Planning variables: changes during planning, between score calculations.
#PlanningVariable(valueRangeProviderRefs = {"machineRange", "jobRange"},
graphType = PlanningVariableGraphType.CHAINED)
private JobOrMachine previousJobOrMachine;
#AnchorShadowVariable(sourceVariableName = "previousJobOrMachine")
private Machine machine;
#PlanningVariable(valueRangeProviderRefs = {"startTimeRange"})
private StartTime startTime=new StartTime(0); // In minutes
My PlanningSolution has a range provider:
#ValueRangeProvider(id = "startTimeRange")
#ProblemFactCollectionProperty
public List<StartTime> getStartTimeList() {
return startTimeList;
}
I am relatively new to Optaplanner and might be missing something very basic. I am struggling to identify what I am doing wrong, even after extensive reading of the docs and examples.
Any idea?
I found a problem with a hard constraint rule related to the planning variable. This question is no longer valid. Thanks.
I'm having an issue with a particular team project collection on my TFS2015.4. This collection caused issues when I wanted to upgrade TFS sometime ago as well. I was able to detach it in TFS2013.3 and then upgraded. Now I want to upgrade to TFS2017 and I don't know how to resolve the issue with this collection.
TF400868: Job definition not found for JobId d891ac97-ddf1-42df-8242-3cd4bd607790
Here is the current status:
Won't detach
Won't start
Status -> ApplyPatch -> won't execute
Stays Offline
One project inside and stays at 'Deleting' state
If I try to start, I get this error:
TF400783: The host 'MyDAS' cannot be started. The host is in the process of being serviced. The servicing may have failed and needs to be restarted and completed before the host can be started.
I did a pre-production upgrade to TFS2017 and there was a validation error with this collection's state that prevented me from finishing the upgrade.
The detailed log for ApplyPatch Rerun has just one failure point:
[12:29:58.457] +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
[12:29:58.457] Executing step: Populate commit changes
[12:29:58.457] Executing step: 'Populate commit changes' Git.M83PopulateCommitChanges (1017 of 1201)
[12:29:58.477] [Error] TF400868: Job definition not found for JobId d891ac97-ddf1-42df-8242-3cd4bd607790.
[12:29:58.480] Microsoft.TeamFoundation.Framework.Server.JobDefinitionNotFoundException: TF400868: Job definition not found for JobId d891ac97-ddf1-42df-8242-3cd4bd607790.
[12:29:58.480] at Microsoft.TeamFoundation.Framework.Server.TeamFoundationJobService.ResolveJobPriorityClasses(IVssRequestContext requestContext, IEnumerable`1 jobReferences, ITFLogger logger)
[12:29:58.480] at Microsoft.TeamFoundation.Framework.Server.TeamFoundationJobService.QueueJobsRaw(IVssRequestContext requestContext, IEnumerable`1 jobReferences, JobPriorityLevel priorityLevel, Int32 maxDelaySeconds, ITFLogger logger, Boolean queueAsDormant)
[12:29:58.480] at Microsoft.TeamFoundation.Server.Deploy.TFCollection.GitStepPerformer.M83PopulateCommitChanges(IVssRequestContext requestContext, ServicingContext servicingContext)
[12:29:58.480] at Microsoft.TeamFoundation.Framework.Server.TeamFoundationStepPerformerBase.PerformHostStep(String servicingOperation, ServicingOperationTarget target, IServicingStep servicingStep, String stepData, ServicingContext servicingContext)
[12:29:58.480] at Microsoft.TeamFoundation.Framework.Server.TeamFoundationStepPerformerBase.PerformStep(String servicingOperation, ServicingOperationTarget target, String stepType, String stepData, ServicingContext servicingContext)
[12:29:58.480] at Microsoft.TeamFoundation.Framework.Server.ServicingStepDriver.PerformServicingStep(ServicingStep step, ServicingContext servicingContext, ServicingStepGroup group, ServicingOperation servicingOperation, Int32 stepNumber, Int32 totalSteps)
[12:29:58.480] Step failed: Populate commit changes. Execution time: 23 milliseconds.
[12:29:58.480] [StepDuration] 0.0236576
[12:29:58.480] [GroupDuration] 0.2517195
[12:29:58.480] [OperationDuration] 0.2517302
[12:29:58.587] Clearing dictionary, removing all items.
======================================================================================================
Step execution times in descending order
======================================================================================================
Updates all rows in tbl_GitCommit and sets the Status to ... (GitToDev14M83Collection, ToDev14M83Collection) - 227 milliseconds
Populate commit changes (GitToDev14M83Collection, ToDev14M83Collection) - 23 milliseconds
Write service level to stamp (StartInstallUpdates, StartInstallUpdates) - 20 milliseconds
Configure framework servicing tokens (VsspToDev14M71Collection, VsspToDev14M71Collection) - 20 milliseconds
Setup integration environment (TestManagementToDev12M65FinalConfiguration, ToDev12M65FinalConfiguration) - 3 milliseconds
Setup Git environment (GitToDev14M74Collection, ToDev14M74Collection) - 1 millisecond
Setup Git environment (GitToDev14M83Collection, ToDev14M83Collection) - 1 millisecond
Set the collection partition id tokens in servicing context (GitToDev14M83Collection, ToDev14M83Collection) - 1 millisecond
======================================================================================================
Execution times by group in descending order
======================================================================================================
GitToDev14M83Collection (ToDev14M83Collection) - 250 milliseconds
StartInstallUpdates (StartInstallUpdates) - 20 milliseconds
VsspToDev14M71Collection (VsspToDev14M71Collection) - 20 milliseconds
TestManagementToDev12M65FinalConfiguration (ToDev12M65FinalConfiguration) - 3 milliseconds
GitToDev14M74Collection (ToDev14M74Collection) - 1 millisecond
I was pondering the fact that this is a rouge job that needs to be deleted from the databse manually but I mmight be wrong. Any pointer will be greatfully +1-ed.
The main reason for TFS being unable to upgrade team project collections usually is that their data was already either corrupted, incomplete, or stuck between schema versions.
You will need to contact customer support for assistance troubleshooting these data issues and getting your data back to a workable state.
Since there is only one project inside. If you don't care the source control history and have the back up of project. A crude (not recommend) way should be deleting the special project collection, delete the database from SQL Server and create a new collection, restore the project inside.
Also take a look at this similar question with the same error as you: Starting a team project collection after detaching and attaching again
Ultimately, there was no way to recover the collection. So I used the admin command to permanently remove the collection. In my case, n one came back to me asking where the collection go so it's solved for me. I'm running the latest TFS 2017.2 and the rest of the dev are happy.
I am developing a solver using Optaplanner 6.1.0, similar to the Vehicle Routing Problem. When I run my solver on 700 installers and 200 bookings, it will successfully solve the planning problem. But, when I used against a larger dataset (700 installers and 1220 bookings), I get
Caused by: java.lang.IllegalStateException: Local Search phase started with an uninitialized Solution. First initialize the Solution. For example, run a Construction Heuristic phase first.
but right before the exception,
16:10:40,378 INFO [DefaultConstructionHeuristicPhase] [http-listener-1(4)] Construction Heuristic phase (0) ended: step total (194), time spent (30693), best score (-1hard/-688803soft).
I am using <constructionHeuristicType>FIRST_FIT_DECREASING</constructionHeuristicType>
in my config.
Am I using it wrong?
Maybe the value range for a planning variable is empty. Especially with value range provider from entity, this is more likely. Feel free to file a jira that the error message should improve in such a case.
Diagnostic todo: Comment out the local solver phase, run the solver (so it only does the construction heuristic) and then iterate through the planning entities and print out the value for each planning value. Check if there are any nulls in there.
The fact that you have 194 steps, instead 200 steps in your CH indicates this. (If those other 6 planning entities are immovable, this won't trigger this exception (more info), so that's not the problem.)
I am running a Spark job on a two node standalone cluster (v 1.0.1).
Spark execution often gets stuck at the task mapPartitions at Exchange.scala:44.
This happens at the final stage of my job in a call to saveAsTextFile (as I expect from Spark's lazy execution).
It is hard to diagnose the problem because I never experience it in local mode with local IO paths, and occasionally the job on the cluster does complete as expected with the correct output (same output as with local mode).
This seems possibly related to reading from s3 (of a ~170MB file) immediately prior, as I see the following logging in the console:
DEBUG NativeS3FileSystem - getFileStatus returning 'file' for key '[PATH_REMOVED].avro'
INFO FileInputFormat - Total input paths to process : 1
DEBUG FileInputFormat - Total # of splits: 3
...
INFO DAGScheduler - Submitting 3 missing tasks from Stage 32 (MapPartitionsRDD[96] at mapPartitions at Exchange.scala:44)
DEBUG DAGScheduler - New pending tasks: Set(ShuffleMapTask(32, 0), ShuffleMapTask(32, 1), ShuffleMapTask(32, 2))
The last logging I see before the task apparently hangs/gets stuck is:
INFO NativeS3FileSystem: INFO NativeS3FileSystem: Opening key '[PATH_REMOVED].avro' for reading at position '67108864'
Has anyone else experience non-deterministic problems related to reading from s3 in Spark?