I have a project where I have configured cucumber to run in parallel on browserstack. Each test will run and complete but the auto generated serenity report includes results for only one of the tests. Is there a way the report can include all results from each individual test that run in parallel?
Related
I'm trying to improve the testing process where I work, but without adjusting the structure.
What we have: VSTS, Selenium IDE, Testers who write test cases, but not code.
What I'd like to do is manage a way to marry our TFS continuous integration with the Selenium tests we write. These are NOT the code-driven selenium tests, but rather the IDE version where users click through, and set assertions using the IDE (All are just UI tests). I know we can export those tests plans as a .SIDE file, but what I can't figure out, is how to have our TFS server execute those as part of a deployment or build pipeline.
Ideally, developers/devops would setup projects in TFS from the onset with whatever solution makes sense to execute these Selenium .SIDE files, but afterwards, the testers would manage adding/modifying those tests cases elsewhere.
The real goal here is to not have testers writing code, or checking in code. Only writing these UI Selenium tests, but having TFS execute those as part of CI.
Researching this on the internet drives me basically always to something that requires testers to write code.
I don't think it can automate testing without code, at lease, you need a test project containing your automated tests.
Generally, in Azure DevOps, we use Visual Studio Test task to run tests. This task supports using the following tests:
Test assembly: Use this option to specify one or more test assemblies that contain your tests. You can optionally specify a
filter criteria to select only specific tests.
Test plan: Use this option to run tests from your test plan that have an automated test method associated with it. To learn more about
how to associate tests with a test case work item, see Associate
automated tests with test cases.
Test run: Use this option when you are setting up an environment to run tests from test plans. This option should not be used when
running tests in a continuous integration/continuous deployment
(CI/CD) pipeline.
This was a question that I had as well, and I think I found an imperfect but better solution.
I wasn't able to get my Selenium IDE tests running with Jenkins, but I was able to get them to run with TeamCity, another CI.
I created a build step like the following :
Runner type: Command Line
Working Directory: where the selenium IDE .side file is located
Run: Custom Script
With the build script content that I usually use to run my Selenium IDE Tests, such as selenium-side-runner sidefile.side
I also added the following so I could output the results in Junitor another form: --output-directory=results --output-format=junit
You can also add the following so the tests are run headlessly, this only works in Chrome : -c "goog:chromeOptions.args=[--headless,--nogpu] browserName=chrome"
Finally, I also use --filter to run one test suite at a time, but that is optional too.
I then used another build step to export the results to our test manger, xray, but I think that is beyond the scope of this question.
The problem with this solution is that it runs directly from a users individual machine still, but this can be work around.
I am new to Cypress, I have tests ready running on cypress gui, but now want to automate these tests running on multiple browsers. Can I get help running my tests in multiple browsers.
I have my tests written in feature files and step definitions
Have you checked docs? You can prepare cli commands with desired browsers and run them all. Or as separate processes.
https://docs.cypress.io/guides/guides/launching-browsers.html#Browsers
I have list of Selenium Webdriver testcase written in Junit Test cases which runs, on demand. currently i am running that and manually sending the report for Pass/Fail count on weekend.
I need that it should run on each weekend and send out the report to me for pass/fail count. There are two tools i come across. Jenkins and maven. I am new to both.. which is best for requirement?
Thanks,
Chandra
Jenkins is the correct option here
Maven is a build tool, where we know the build procedure and its dependencies
It can be used in your project to create the class file or jar files required
Jenkins is a Continuous Integration tool, which can be used to schedule and run tasks where you can execute your test cases , report the test result via email.. you can view it as server which executes your tests and provide you the results
We have seperate integration test project which fires the integration test cases on different modules . At present we do not have unit test cases within each module. We would like to ensure that the integration test covers most of the domain functionality.
Since we have the integration tests in a different project , Sonar always reports the test coverage as zero for the modules under test.
Is there any way to have the test coverage reported on a project , when the actual test is run from a different project
Thanks
You should be able to achieve what you want by reading the Code Coverage documentation page on the wiki. Most notably, you'll be able to use the following sample project to see how it works:
IT JaCoCo Sonar Runner sample project
Basically, you have to run your integration tests first using JaCoCo to generate the coverage report (jacoco.exec) and then you reuse this report during the SonarQube analysis.
Is it possible to run unit tests when a non-maven project is analyzed with Sonar, in Sonar light mode?
Sonar doesn't run unit tests. But it should be able to analyze existing unit tests reports. From Reuse in Sonar unit test reports generated by other systems:
2. Using Sonar in its full capability in an ANT environment
If you are using ANT to build your
applications, the main weakness so far
in Sonar was that it did not allow to
display Unit tests results nor Code
coverage. I am sure that now you have
read the first use case, you know that
by using the
“-Dsonar.dynamicAnalysis=reuseReports”
parameter, this limitation does not
exist anymore. You simply need to
specify where those reports to reuse
are going to be found, by using the
following properties :
sonar.cobertura.reportPath,
sonar.clover.reportPath,
sonar.surefire.reportsPath...