I want to run 5 tests from a suite of 200 tests via commandline. The framework uses Selenium with nunit and is built on c#.
After looking for solution online, /run was what I thought could fix my problem but it says invalid argument. Can someone please guide me with what is the parameter to run only few tests from suite via command line for nunit selenium?
You are using obsolete documentation or online answers relating to an older version of NUnit. For NUnit 3, you want the /test option.
BTW, the docs link posted by #vitalis is to legacy docs, as it says on the top of that page. Current docs for the nunit3-console command-line are at https://docs.nunit.org/articles/nunit/running-tests/Console-Command-Line.html
(You're dealing with a program that has been around in various versions for more than 20 years, so it's extra important to make sure you are looking at current information.)
Related
I am using the Karate framework to do the API testing. As part of CI efforts, we send an email at the end of test execution listing the summary of test results. There is a need to include the screeshot of the test execution counts from 'overview-feature.html' file.
I did so through the TestRunner.java file - launched Chrome using Chrome.start() and then using it to take screenshot. It all works well locally on Windows.
However when executing on CI server which is a Unix box, the chrome executable is not present in the default location (usr/bin/google-chrome) and hence the connection for the localhost fails.
Is there a way we can change the default location of the chrome executable?
PS: Apologies if this was too trivial to be asked.
Yes Chrome on CI is hard to get right, refer: https://stackoverflow.com/a/62325328/143475 - note that CI boxes typically are "headless" a browser may not be even installed.
I think the best thing for you is to ZIP the HTML and send it. But I really think you need to work with some CI experts, because the report generation and e-mailing business is normally done by things like Jenkins. What you are doing is certainly not normal or best-practice.
If you really want, there is a Karate Docker container that can give you a proper Chrome instance (see docs) but that is overkill for what you need.
EDIT: The Chrome Java API allows for customization of the executable path and this is in the docs: https://github.com/intuit/karate/tree/master/karate-core#chrome-java-api
It should be something like this:
Chrome.start("/opt/blah/chrome");
I know there is an existing post asking if it's possible to generate an HTML report of code coverage analysis for tests written and run with Intern, and it's been answered:
Generate HTML code coverage reports with intern
However, the post doesn't mention what type of environment that the OP runs in; i.e., are the tests running in a Node.js client? I ask because I am running my unittests using the Intern framework in a browser [edit: invoking tests comparably to http://path/to/intern-tutorial/node_modules/intern/client.html?config=tests/intern]. The article here:
https://github.com/theintern/intern/wiki/Using-and-Writing-Reporters#custom-reporters
outlines that HTML is the only reporter available for the browser platform; LCOV and LCOVHTML are not. But has that changed at all? This limited array of reporters
for browsers isn't very convenient, and I was hoping to take advantage of Istanbul built into Intern, rather than try to plug in another code coverage analysis tool (or hack my own thing :( ).
Code coverage information will be correctly retrieved from code running in browsers if you run your tests with intern-runner. The actual collation and output of the coverage results occurs on the server (Node.js) side.
I'm running tests using Siesta from command line.
Is that possible to generate screenshots on each test fail while running Siesta test runner?
Edit:
Need to wait for the feature
http://www.bryntum.com/products/siesta/changelog/
It's not possible in current version 2.0.8, but they are planning to add this in future.
More info can be found on their forum http://www.bryntum.com/forum/
Also Siesta can be runned in cloud on http://www.browserstack.com/ or https://saucelabs.com/. These services offers evene whole screencasts of the tests.
I'm trying to do set up some Selenium WebTests using PHPUnit, but php is no longer supported by Selenium IDE, does this mean I have to re-write all my tests into php in order to use this method?
I'm trying to set up a continuous integration system, but have never even used one before, I tried using jenkins-php.org but it wasn't very helpful.
The newer releasee Selenium IDE "claims" it does not support PHP,
But you can still convert your IDE to PHP or PHPUnit.
(In Selenium IDE) Option > Options > Enable Experimental Functions.
With this option checked you can switch your format under Options again.
You may still want to review your code.
Hope this helps and it is what you meant by "Selenium IDE does not support PHP".
I was able to find: https://addons.mozilla.org/en-US/firefox/addon/selenium-ide-php-formatters/
It allows output from Selenium to PHP, either PHPUnit format or Selenium Testing.
However after all this playing around it became apparent that the best way was to use htmlSuite, it has the most support when it comes to automation and seems to work out of the box, whereas PHPUnit testing requires lots of tinkering.
After a lot of hacking around it became apparent that the best solution is this tool:
http://www.enjoyxstudy.com/selenium/autoexec/index.en.html
It works out of the box, supports htmlSuite, works with windows and linux, runs most browsers, sends email reports, can get new tests from SVN, integrates with Jenkins well, and works with Selenium tests, its not PHP based tests but it works wonders compared to the other tools I tried out.
Really, so much easier, very user friendly, free and works out the box.
We are looking for a software to run our test cases automatically.
We want a software which will run on our server (or a commercial), which automatically gets the newest commit on github. Then compiles the commit of the project with CMake and run Ctest on our test cases. The results should then be visualized on a nice website.
I had a look at CDash, but as the documentation is so bad I did not even get it to get the latest commit from github.
So my questions are:
Is there a good tutorial to CDash? Except the bad wiki page.
What software is available for running tests on new commits to github, what are their advantages and drawbacks?
In answer to your second question, Jenkins is a robost and extensible continuous integration tool that can be integrated tightly with GitHub using a plug-in (or loosely using standard Git support). It also supports CMake via a plug-in. Whether it has disadvantages that will make it less useful for you depends on your organization and build process, but I've found it to be highly customizable to a wide variety of processes. I recommend taking a look at it.
There's also a third-party Ctest plugin available for Jenkins.
CDash works in pair with CTest. If you are already using CMake then it should be fairly easy to submit your testing results to CDash. I'd recommend reading the CTest documentation:
http://www.vtk.org/Wiki/CMake_Testing_With_CTest
You can either install your own CDash server or use Kitware's hosted server at my.cdash.org. You can test your server with a sample project available at:
http://www.cdash.org/cdash/resources/software.html