Optaplanner 7.9.0 Multithreading - Solution not reproducable - optaplanner

Question

Having updated to 7.9.0 and after initial problems: https://stackoverflow.com/questions/51597744/optaplanner-7-9-0-and-adding-multithreading-same-planningid-exception I now have been trying to test and compare to my 7.7.0 version. However I cannot get it to reproduce the same solution everytime (obviously only with the same problem data) as in my older version, even when explicitly set in the config xml. Is there additional set up for this version required?
Edit: Did some testing and found switching to TABU (haven't been through them all) gave me the expected consistency:
Extra Cores:1 - DEFAULT:
Score (Hrd:Med:Sft) Time Taken (Minutes:Seconds)
Test1 0:0:-7609 0:08
Test2 0:-1:-7758 0:13
Test3 0:-1:-7705 0:14
Extra Cores:1 - TABU:
Score (Hrd:Med:Sft) Time Taken (Minutes:Seconds)
Test1 0:0:-7763 1:29
Test2 0:0:-7763 1:29
Test3 0:0:-7763 1:28
Between two runs of the former the solution diverges at LS step 28:
LS step (25), time spent (1869), score (0hard/-3medium/-8155soft),
LS step (26), time spent (1890), score (0hard/-3medium/-8339soft),
LS step (27), time spent (1895), score (0hard/-3medium/-8126soft),
**LS step (28), time spent (1909), score (0hard/-3medium/-8256soft),
LS step (29), time spent (1915), score (0hard/-3medium/-8438soft),
LS step (30), time spent (1924), score (0hard/-3medium/-8620soft),
LS step (31), time spent (1952), score (0hard/-3medium/-8639soft),**
...and...
LS step (25), time spent (1385), score (0hard/-3medium/-8155soft),
LS step (26), time spent (1407), score (0hard/-3medium/-8339soft),
LS step (27), time spent (1412), score (0hard/-3medium/-8126soft),
**LS step (28), time spent (1422), score (0hard/-3medium/-8217soft),
LS step (29), time spent (1436), score (0hard/-3medium/-8336soft),
LS step (30), time spent (1442), score (0hard/-3medium/-8517soft),
LS step (31), time spent (1448), score (0hard/-3medium/-8571soft),**
Don't know if that all makes it more or less likely to be problem/solution set up or something else.

Answer 1

First, let's define reproducible: getting the same result at the same step iteration. If the CPU time is completely the same (which it never is), this also means getting the same result after the same amount of time. So you might need to run it a bit longer to get the same amount of steps.
Multithreaded solving is reproducible if and only if the same moveThreadCount is used. A run with <moveThreadCount>4</> is NOT reproducible by a <moveThreadCount>2</>. Even a run with <moveThreadCount>NONE</> is not reproducible with a run with <moveThreadCount>1</> (the latter being useless except for QA btw).
Look at your score calculation speed.
If it's lower, you probably forgot to increase the memory size and GC churns are hurting you.
If that's higher, you'd normally get a better result. If you see a worse result anyway, the config might have been overfitted for the no move threads. Use additional datasets in optaplanner-benchmark to prove or disprove that.
Furthermore, if you use <moveThreadCount>AUTO</>, it's only reproducible on the same machine, because AUTO changes the moveThreadCount depending on the number of CPU cores in the machine.