activeFocus very unreliable in TestCase - qml

Question

How can I make QML TestCase behave more predictably when doing keyboard navigation tests, where I check various steps using compare() and the activeFocus of an element?
Right now, the tests are randomly passing and failing with no repeatability. So badly, in fact, that even THIS fails randomly:
import QtQuick 2.5
import QtTest 1.1
TestWindow {
TestCase {
when: windowShown
name: 'Example test'
property string myView: 'mainView.homeWrapper.mainView.myView'
// Some other tests, but should not matter for the below
function test_5_keyboard () {
var testView = getElement(myView)
var uiUpButton = findChild(testView, 'catalogUpButton')
uiUpButton.forceActiveFocus()
wait(500)
compare(uiUpButton.activeFocus, true)
}
}
}
Results randomly (but quite often, about 50 % of runs) in:
test-ui_1 | FAIL! : qmltestrunner::Example test::test_5_keyboard() Compared values are not the same
test-ui_1 | Actual (): false
test-ui_1 | Expected (): true
test-ui_1 | Loc: [/app/test/tst_MyTest.qml(100)]
The same happens in "real" tests too, where I don't use forceActiveFocus() but simply check for activeFocus of an element after moving the keyboard using for example keyClick(Qt.Key_Down). They often work, but again, fail randomly but too often to be usable.
The tests are run inside a container running Ubuntu 16.04, using these libraries:
RUN apt-get install -y qtcreator
# Install modules required by qml
RUN apt-get install -y \
qml-module-qtquick-localstorage \
qml-module-qt-websockets \
libqt5qml-graphicaleffects \
qt5-default \
qtdeclarative5-test-plugin
# Install XVFB virtual display
RUN apt-get install -y xvfb