I can run all of my test cases in a solution file ( in DEBUG mode) by using the following command:
nunit3-console.exe "%SlnFile%" /config:Debug --skipnontestassemblies --result="%TestResult.xml%"
But this command will run for all tests under different platforms, x86, x64 and AnyCPU.
Is there anyway to restrict it, so that nunit3-console only runs for DEBUG|AnyCPU?
Related
I have more than 50 test cases in my UI-TEST scheme and I am running it via the below command
xcodebuild test -workspace tribo.xcworkspace -scheme triboUITests -sdk iphonesimulator -destination 'platform=iOS Simulator,name=iPhone X,OS=12.4'
what happens is the RAM of my Mac machine shoots up and the system hangs up , I have attached the activity monitor report tooenter image description here
can anyone help me in running all XCUITEST cases present in scheme from terminal
The console command itself seems legit.
The RAM problem also could be caused by inaccurate resource management in test code.
Consider inspecting your code for possible memory leakings and other bugs.
You can run a specific test or tests as part of the xcodebuild invocation.
From the xcodebuild help command line:
-only-testing:TEST-IDENTIFIER constrains testing by specifying tests to include, and excluding other tests
-skip-testing constrains testing by specifying tests to exclude, but including other tests
-skip-testing:TEST-IDENTIFIER constrains testing by specifying tests to exclude, but including other tests
Using these commands may help you understand which tests are causing the issue.
You could also use the Xcode Memory Graph Debugger (see this link:
https://medium.com/zendesk-engineering/ios-identifying-memory-leaks-using-the-xcode-memory-graph-debugger-e84f097b9d15
Finally, you if you are trying to run all your tests in parallel, you may have too many simulators running on the machine.
I have a .side file generated by the Selenium IDE, which I need to run on CI using Jenkins.
I am running it as a build step with the following shell command:
selenium-side-runner /path/to/file.ide
The problem arises due to the fact that no matter if the selenium test fails, Jenkins always shows is as success.
In this thread it's suggested to upload the file as generic, but still, the commands to execute it are missing
How to upload a generic file into a Jenkins job?
I've found a possible solution to it on this posts, but I would appreciate having a cleaner way to solve this instead of parsing the results checking for errors.
How to mark a build unstable in Jenkins when running shell scripts
Is there a plugin able to run selenium .side files on Jenkins and this one showing the success/failures of the test?
You can generate a Junit test report file and then use the Jenkins Junit plugin after your tests execution.
selenium-side-runner --output-directory=results --output-format=junit
# Outputs results in `junit` frormat in `./results/projectName.xml'
Check the official documentation for more details.
I need to run python units test cases as part of bamboo build step and the build needs to fail if unit tests fail.
For this, I have a Script step in bamboo build and i am trying to run the following in it:
python -m unittest discover /test
Here, /test folder has all the unit tests.
The output of the above script it
Ran (0) tests
So the problem is that bamboo isn't able to discover these tests. Bamboo agent is linux.
Wondering if anyone has done such a thing before and has any suggestions.
The following worked. Used -p (pattern) attribute to discover/run the unit tests in bamboo (unix build agent)
python -m unittest discover -s test -p "T*.py"
Note: 1. all my test cases start with "T" e.g. Test_check.py
2. "test" is the package where all my test cases are.
If you haven't figured it out, likely because in windows filenames aren't case sensitive but in Linux they are...
And you're test file named Test_xxxx.py isn't the same as test_xxxx.py which is the pattern that discovery is trying to use...
As I am implementing an automated way to GUI test our webapplication with selenium I ran into some issues.
I am using selenese-runner to execute our Selenium test suites, created with Selenium IDE as a post build action in Jenkins.
This works perfeclty fine, as the build fails when something is wrong, and the build succeeds if all tests are passed. And the results are stored on a per build basis as HTML files, generated be selenese-runner.
My problem is however, that I seem to be unable to find a way, how to display these results in the respective jenkins build.
Does anyone have an idea how to solve this issue. Or maybe I am on the wrong path at all?
Your help is highly appreciated!
I believe the JUnit plugin should do what you want, but it doesn't work for me.
My config uses this shell script to run the tests (you can see the names of all my test suites):
/usr/bin/Xvfb &
export DISPLAY=localhost:0.0
cd ${WORKSPACE}
java -jar ./test/selenium/bin/selenese-runner.jar --baseurl http://${testenvironment} --screenshot-on-fail ./seleniumResults/ --html-result ./seleniumResults/ ./test/selenium/Search_TestSuite.html ./test/selenium/Admin_RegisteredUser_Suite.html ./test/selenium/Admin_InternalUser_Suite.html ./test/selenium/PortfolioAgency_Suite.html ./test/selenium/FOAdmin_Suite.html ./test/selenium/PublicWebsite_Suite.html ./test/selenium/SystemAdmin_Content_Suite.html ./test/selenium/SystemAdmin_MetaData_Suite.html
killall Xvfb
And I can see the result of the most recent test (you can see the name of my jenkins task folder)
http://<JENKINS.MY.COMPANY>/job/seleniumRegressionTest/ws/seleniumResults/index.html
Earlier tests are all saved on the Jenkins server, so I can view them if I need to.
I would like to add a command line start for Selenium IDE test scripts to our build process so some of the tests are run immediately after the latest code is built for our products.
I have been looking and everything I find seems to relate to older versions. I can't find anything that looks up to date.
Can I run tests that I have recorded in Selenium IDE 2.9.1 from the command line - and by extension, a batch file to run multiple tests.
Is there documentation that I did not find? Can someone direct me to it?
Thanks.
You didnt specify an operating system but i think you mean windows, and yes, you can run selenium IDE test suites from powershell or CMD on windows.
You will need Java installed on your machine, and also the Selenium Server.jar file which should have come with your installation of Selenium, if not you can download from the Selenium Site.
Then, create a test suite in IDE and save it somewhere on your machine. (you can only run test suites, not testcases - i wont explain the difference between the two here)
Then in powershell or CMD you run this command, you will need to change it depending on your installation directories.
C:\Java\JDK1.8.0_66-X64\bin\java -jar C:\Selenium\Selenium-Server\selenium-server.jar -htmlSuite *firefox "baseurl" "C:\temp\testsuitename" "C:\TEMP\logs" -timeout 0060 -trustAllSSLCertificates
I will break this down for you -
C:\JDK1.8.0_66-X64\bin\java -jar - this is your Java installation directory (mine is Java JDK 1.8)
C:\Selenium\Selenium-Server\selenium-server.jar - This is the location of your Selenium .jar file.
-htmlSuite *firefox These are .jar file options to tell the jar file you are running it from a command line and to use firefox
"baseurl" In here, you should enter your base url which would normally be at the top of the IDE gui. eg, www.google.com
"C:\temp\testsuitename" The full path where you have saved your test suite
"C:\TEMP\logs" Running from command line produces a html log, advise where you want this placed
-timeout 0060 -trustAllSSLCertificates There are additional options, of which there are many. In this case the total time the command will run before it times out is 60 seconds, and it will ignore all those untrusted certificate errors that firefox often gets (where you need to keep adding exceptions etc).
Hope i could help!