I have a C# project that I have used OpenCover and Gallio to run the MbUnit UnitTests from Nant script. I'm trying to set this up in Bamboo. I see Bamboo has NUnit runner, but doesn't seem to have anything for MbUnit.
Any idea how one would run the MbUnit UnitTests from Bamboo ?
Most CI systems have the ability to execute generic tasks (for this type of scenario)
A quick peruse of the site shows that the Console (Builder) Task would be the one; or you could also get it working in a nant script and then use that.
You would then use OpenCover to run NUnit or MbUnit runners as required.
Related
I'm trying to improve the testing process where I work, but without adjusting the structure.
What we have: VSTS, Selenium IDE, Testers who write test cases, but not code.
What I'd like to do is manage a way to marry our TFS continuous integration with the Selenium tests we write. These are NOT the code-driven selenium tests, but rather the IDE version where users click through, and set assertions using the IDE (All are just UI tests). I know we can export those tests plans as a .SIDE file, but what I can't figure out, is how to have our TFS server execute those as part of a deployment or build pipeline.
Ideally, developers/devops would setup projects in TFS from the onset with whatever solution makes sense to execute these Selenium .SIDE files, but afterwards, the testers would manage adding/modifying those tests cases elsewhere.
The real goal here is to not have testers writing code, or checking in code. Only writing these UI Selenium tests, but having TFS execute those as part of CI.
Researching this on the internet drives me basically always to something that requires testers to write code.
I don't think it can automate testing without code, at lease, you need a test project containing your automated tests.
Generally, in Azure DevOps, we use Visual Studio Test task to run tests. This task supports using the following tests:
Test assembly: Use this option to specify one or more test assemblies that contain your tests. You can optionally specify a
filter criteria to select only specific tests.
Test plan: Use this option to run tests from your test plan that have an automated test method associated with it. To learn more about
how to associate tests with a test case work item, see Associate
automated tests with test cases.
Test run: Use this option when you are setting up an environment to run tests from test plans. This option should not be used when
running tests in a continuous integration/continuous deployment
(CI/CD) pipeline.
This was a question that I had as well, and I think I found an imperfect but better solution.
I wasn't able to get my Selenium IDE tests running with Jenkins, but I was able to get them to run with TeamCity, another CI.
I created a build step like the following :
Runner type: Command Line
Working Directory: where the selenium IDE .side file is located
Run: Custom Script
With the build script content that I usually use to run my Selenium IDE Tests, such as selenium-side-runner sidefile.side
I also added the following so I could output the results in Junitor another form: --output-directory=results --output-format=junit
You can also add the following so the tests are run headlessly, this only works in Chrome : -c "goog:chromeOptions.args=[--headless,--nogpu] browserName=chrome"
Finally, I also use --filter to run one test suite at a time, but that is optional too.
I then used another build step to export the results to our test manger, xray, but I think that is beyond the scope of this question.
The problem with this solution is that it runs directly from a users individual machine still, but this can be work around.
I am working using TFS to create test case. I have a automated test written in Robotframework with Selenium2Library.
I'd like to be able to associate these tests with test cas in test. Is it possible to link the automated test cases with the test case in TFS, and if so, how is it done?
The web Test page does not support to run automatic tests associated with test cases in TFS.
You can achieve the same goals using the Build & Release features in Team Foundation Server (TFS). (https://www.visualstudio.com/en-us/docs/test/lab-management/use-build-or-rm-instead-of-lab-management). Simply integrate Selenium testing into a continuous integration/continuous delivery pipeline.
Instead of build, suggest you use a step in a release pipeline. Take a look at this question:How can i run automated selenium tests using TFS build 2015?
More detail steps about how to integrate with TFS please go through the official tutorials from MSDN: Get started with Selenium testing in a CD pipeline
Presently we built a Automation framework which uses Selenium Webdriver+ specflow + Nunit, and we suing bamboo as our CI to run our Job against our every build.
we written a build.xml to handle our targets (like clean, init, install latest build, run Selenium scripts, uninstall build. etc)
ant command will read the tag name from the build.xml and runs the respective feature/scenarios based on Tags (like #smoke, #Regression)with Nunit in CI machine.
Now our requirement is to use Selenium Grid to divide scripts into different machine and execute with above set-up. Grid has to divide the scripts based on feature file or based on Tags.How to achieve this.
Is there any thing need to done under [BeforeFeature] and [BeforeScenario] ?
If you provide in details steps or any link which explains detail steps that would be a great help.
Please any one can help in this regards.
Thanks,
Ashok
You have misunderstood the role Grid plays in distributed parallel testing. It does not "divide the scripts", but simply provides a single hub resource through which multiple tests can open concurrent sessions.
It is the role of the test runner (in your case Specflow) to divide tests and start multiple threads.
I believe that you require SpecFlow+ (http://www.specflow.org/plus/), but this does have a license cost.
It should be possible to create your own multithread test runner for Specflow but will require programming and technical knowledge.
If you want a free open source approach to parallel test execution in DotNet, then there is MbUnit (http://code.google.com/p/mb-unit) but this would require you to rewrite your tests
I was wondering if it is possible for Jenkins to be notified after TFS finishes a build.
I'm trying to make a system where, after the TFS build, Jenkins is used to run automated selenium tests.
Thanks in advance!
You have a few options:
You could use a GenericTest to trigger Jenkins and include as part of the Build Definition. This also allows your script to communicate pass/fail information back to the build.
You could just include an InvokeProcess workflow activity to trigger Jenkins. The process would be similar to this or this.
Or you could use a tool that would trigger after a Team Build has finished. Normally people would use tools that then deploy stuff. Most of these tools trigger a Powershell script at some point, which would allow you to trigger the test run:
Octopus Deploy
TFS Deployer
I have setup Bamboo to run JBehave tests on a remote agent (with JBehave-web plugin launching test using webdriver), and everything runs fine. Only problem is after the execution is finished Bamboo shows no test executed. I can see the option in Bamboo to select the output of the test results, but it has to be a JUnit xml, and Jbehave reports are only generated in plain text or html.
Any idea how to solve this?
Thanks
I ran in the same situation about a year ago. JBehave "doesn't" integrate with Bamboo out of the box. Although, they have a plugin for Hudson CI.
In my case, such as yours, I resorted in running the tests through the Surefire plugin; the outputs are considered as JUnit tests results and Bamboo can recognize them.
Hope it helps.
There is a really simple way to do this. And I'm currently doing this for our build system.
Write a simple parse script that transforms your plain of html report into JUnit compatible results. And add that script as a task in your Bamboo task, then use Junit parser to parse the results. Boo! You're done! Plus, you've got the capability to quarantine!
This is way much faster than writing a plugin for Bamboo, which involves considerable more time to learn/write.
Setup JBehave with Maven.In Bamboo build plan use Maven task to run it. For getting results in Bamboo use JBehave Task for Bamboo. It will convert JBehave scenarios in tests in Bamboo. If scenario names contains JIRA issue ids, it will link them to JIRA issues.
https://marketplace.atlassian.com/plugins/com.mdb.plugins.jebehaveforbamboo/server/overview
Sample JBehave as Maven Project
https://bitbucket.org/vikasborse/jbehavesampleproject/overview
Download or clone this repository on your local machine.
To run navigate to this project in command line and use command:
"mvn integration-test"