In our Yii project we're using Jenkins CI and Codeception for different types of tests. The problem is, that codeception report is empty, which causes whole Jenkins build failure.
All tests are running without errors. Jenkins execute shell for codeception:
php codecept.phar run --xml --html
Console output error line which causes failure:
[xUnit] [ERROR] - The result file '/var/lib/jenkins/jobs/project/workspace/code/protected/tests/_output/report.xml' for the metric 'PHPUnit' is empty. The result file has been skipped.
I understand simple logic, if report is empty -> build failed. But why is report empty? Is that a bug or can I do something about this?
The problem was, that in one of our tearDowns was the following line:
Yii::app()->end();
which makes Yii-Application die. For some reasons this caused that codeception has not generated the report.
Related
I'm trying to run some TestCafe tests from our build server, but getting the following error...
"Could not find test files at the following location: "C:\Testing\TestCafe".
Check patterns for errors:
tests/my-test.ts
or launch TestCafe from a different directory."
I did have them running or able to be found on this machine previously, but others have taken over the test coding and changed the structure a bit when moving it to a Git repository. Now when I grab the tests from Git and try to run, the problem presents itself. I'm not sure if there is something in a config file that needs adjustment but don't know where to start looking.
The intention is to have it part of our CI process, but the problem is also seen when I attempt to run the tests from the command line. The build process does install TestCafe, but there is something strange around this as well.
When the build failes with the can't find tests error, if I try to run the following command in the proper location...
tescafe chrome tests/my-test.ts
... I get, 'testcafe' is not recognized as an internal or external command,
operable program or batch file.
Just can't understand why I can't get these tests running. TestCafe setup was pretty much easy previously.
ADDENDUM: I've added a screenshot of the working directory where I cd to and run the testcafe command as well as the tests subdirectory containing the test I'm trying to run.
Any help is appreciated!!
testcafe chrome tests/my-test.ts is just a template; it isn't a real path to your tests. This error means that the path that you set in CLI is wrong, and there aren't any tests. You need to:
Find out where you start CLI. Please attach a screenshot to your question.
Define an absolute path to tests or a path relative to the place where CLI was started. Please share a screenshot of your project tree where the directory with tests is open.
Also, you missed t in the tescafe chrome tests/my-test.ts command. It should be tesTcafe chrome tests/my-test.ts. That is why you get the "'tescafe' is not recognized as an internal or external command" error.
I was able to get things working by starting from scratch. I uninstalled TestCafe and cleaned the working folder. During next build it was fine. I'm sure I've tried this several times, but it just started working.
One positive that came out of it was that I discovered a typo in a test file name, which was also causing issues finding the test I was using to check testing setup.
Thanks for helping!!
I have a .side file generated by the Selenium IDE, which I need to run on CI using Jenkins.
I am running it as a build step with the following shell command:
selenium-side-runner /path/to/file.ide
The problem arises due to the fact that no matter if the selenium test fails, Jenkins always shows is as success.
In this thread it's suggested to upload the file as generic, but still, the commands to execute it are missing
How to upload a generic file into a Jenkins job?
I've found a possible solution to it on this posts, but I would appreciate having a cleaner way to solve this instead of parsing the results checking for errors.
How to mark a build unstable in Jenkins when running shell scripts
Is there a plugin able to run selenium .side files on Jenkins and this one showing the success/failures of the test?
You can generate a Junit test report file and then use the Jenkins Junit plugin after your tests execution.
selenium-side-runner --output-directory=results --output-format=junit
# Outputs results in `junit` frormat in `./results/projectName.xml'
Check the official documentation for more details.
I’m executing automatic tests using Katalon in console mode.
Sometimes job executed by Jenkins ends unexpectedly without any error.
Katalon Studio version 5.10.1,
Jenkins version 2.121.3
I've tried to find some Jenkins logs other than console logs, which logged why that job is aborted
Jenkins console logs:
/var/lib/jenkins/.katalon/5.10.1/Katalon_Studio_Linux_64-5.10.1/katalon -noSplash -runMode=console -projectPath=/var/lib/jenkins/workspace/(...) -browserType=Firefox -testSuitePath=Test Suites/MainTestingSuite
Delete folder: Libs
Opening project file: /var/lib/jenkins/workspace/(...).prj
Request sent successfully.
[EL Warning]: 2019-05-10 13:54:15.046–Ignoring attribute [lastRun] on class [TestSuiteEntity] as no Property was generated for it.
**Build step ‘Execute shell’ changed build result to UNSTABLE**
Got SIGTERM, exiting
In exit
Terminating xvnc.
$ vncserver -kill :80
Killing Xvnc4 process ID 29224
Archiving artifacts
‘Reports/MainTestingSuite/’ doesn’t match anything, but ‘’ does. Perhaps that’s what you mean?
No artifacts found that match the file pattern “Reports/MainTestingSuite/”. Configuration error?
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Finished: FAILURE
Someone already had that problem? How to track that case?
I’m waiting for your answer.
I’ve finally found cause of that problem.
Katalon job failed like that, when you try executed two Katalon instances at the moment. I’ve changed number of executors at jenkins to 1, and problem is not occuring anymore.
Best Regards
As I am implementing an automated way to GUI test our webapplication with selenium I ran into some issues.
I am using selenese-runner to execute our Selenium test suites, created with Selenium IDE as a post build action in Jenkins.
This works perfeclty fine, as the build fails when something is wrong, and the build succeeds if all tests are passed. And the results are stored on a per build basis as HTML files, generated be selenese-runner.
My problem is however, that I seem to be unable to find a way, how to display these results in the respective jenkins build.
Does anyone have an idea how to solve this issue. Or maybe I am on the wrong path at all?
Your help is highly appreciated!
I believe the JUnit plugin should do what you want, but it doesn't work for me.
My config uses this shell script to run the tests (you can see the names of all my test suites):
/usr/bin/Xvfb &
export DISPLAY=localhost:0.0
cd ${WORKSPACE}
java -jar ./test/selenium/bin/selenese-runner.jar --baseurl http://${testenvironment} --screenshot-on-fail ./seleniumResults/ --html-result ./seleniumResults/ ./test/selenium/Search_TestSuite.html ./test/selenium/Admin_RegisteredUser_Suite.html ./test/selenium/Admin_InternalUser_Suite.html ./test/selenium/PortfolioAgency_Suite.html ./test/selenium/FOAdmin_Suite.html ./test/selenium/PublicWebsite_Suite.html ./test/selenium/SystemAdmin_Content_Suite.html ./test/selenium/SystemAdmin_MetaData_Suite.html
killall Xvfb
And I can see the result of the most recent test (you can see the name of my jenkins task folder)
http://<JENKINS.MY.COMPANY>/job/seleniumRegressionTest/ws/seleniumResults/index.html
Earlier tests are all saved on the Jenkins server, so I can view them if I need to.
I've setup a TC configuration ready to run Selenium tests once we get a build agent able to run them.
This is the first TC config I've created but it was running until I got TC to run the Selenium test runner. Now it fails when it tries to cleanup the Selenium-server.jar.
Can you exclude file types from the cleanup or is there another solution that I'm missing here?
TC build errors;
> Problem reported from build script. New build status text is: : {build.status.text}; Swabra cleanup failed
> Error while applying patch: Error while applying patch: Failed to delete: C:\BuildAgent\work\f43641868cf93216\src\django_selenium\selenium-server.jar
You can configure Swabra cleaner to ignore your selenium-server.jar using path rule:
-:src\django_selenium\selenium-server.jar
but it looks like there is a problem in selenium tests setup. The error message you posted may be caused by some process (most probably, selenium server) still running after tests are finished. It has locked the library and thats why swabra can't delete it.