I want to run a codedUI test case using MTM 2013
Here is what I already have so far:
I have codedUI test cases to test my web application.
The solution of these test cases has got nothing to do with my web application.
Now I have few test cases written in MTM 2013 (initially for manual execution). Later I had plan to automate these Test Cases Using codedUI and associate these automated Test cases to MTM test cases so that I don't have to open VSTS to run my automated test cases. Also this may serve the purpose that any other tester can also execute them using MTM.
Here is all I have done so far:
Installed and configured "Test Controller" in my TFS server.
Installed "Test Agent" on the machine I want to execute the automated test cases.
Created Physical Lab Environment successfully in MTM.
Associated the codedUI automation with test cases in MTM.
Now when I go to play test case in MTM, it gives me an error "Cannot create automated test run. A build associated with the active test plan is required to submit an automated test run."
Can you please help me with this. This is the first time I am using MTM test cases to associate codedUI automated test cases.
You need to assign a build to your test plan that contains your Coded UI tests. Otherwise, how does it know which version of your tests to run, and which version of your coded is being tested?
MSDN: http://msdn.microsoft.com/en-us/library/dd490760(v=vs.100).aspx
Check the answers to following SO questions, they may help you to understand how it is working:
Couldn't run my test using Microsoft Test Manager
Attach Current Build to Test
Related
I'm trying to improve the testing process where I work, but without adjusting the structure.
What we have: VSTS, Selenium IDE, Testers who write test cases, but not code.
What I'd like to do is manage a way to marry our TFS continuous integration with the Selenium tests we write. These are NOT the code-driven selenium tests, but rather the IDE version where users click through, and set assertions using the IDE (All are just UI tests). I know we can export those tests plans as a .SIDE file, but what I can't figure out, is how to have our TFS server execute those as part of a deployment or build pipeline.
Ideally, developers/devops would setup projects in TFS from the onset with whatever solution makes sense to execute these Selenium .SIDE files, but afterwards, the testers would manage adding/modifying those tests cases elsewhere.
The real goal here is to not have testers writing code, or checking in code. Only writing these UI Selenium tests, but having TFS execute those as part of CI.
Researching this on the internet drives me basically always to something that requires testers to write code.
I don't think it can automate testing without code, at lease, you need a test project containing your automated tests.
Generally, in Azure DevOps, we use Visual Studio Test task to run tests. This task supports using the following tests:
Test assembly: Use this option to specify one or more test assemblies that contain your tests. You can optionally specify a
filter criteria to select only specific tests.
Test plan: Use this option to run tests from your test plan that have an automated test method associated with it. To learn more about
how to associate tests with a test case work item, see Associate
automated tests with test cases.
Test run: Use this option when you are setting up an environment to run tests from test plans. This option should not be used when
running tests in a continuous integration/continuous deployment
(CI/CD) pipeline.
This was a question that I had as well, and I think I found an imperfect but better solution.
I wasn't able to get my Selenium IDE tests running with Jenkins, but I was able to get them to run with TeamCity, another CI.
I created a build step like the following :
Runner type: Command Line
Working Directory: where the selenium IDE .side file is located
Run: Custom Script
With the build script content that I usually use to run my Selenium IDE Tests, such as selenium-side-runner sidefile.side
I also added the following so I could output the results in Junitor another form: --output-directory=results --output-format=junit
You can also add the following so the tests are run headlessly, this only works in Chrome : -c "goog:chromeOptions.args=[--headless,--nogpu] browserName=chrome"
Finally, I also use --filter to run one test suite at a time, but that is optional too.
I then used another build step to export the results to our test manger, xray, but I think that is beyond the scope of this question.
The problem with this solution is that it runs directly from a users individual machine still, but this can be work around.
I am working using TFS to create test case. I have a automated test written in Robotframework with Selenium2Library.
I'd like to be able to associate these tests with test cas in test. Is it possible to link the automated test cases with the test case in TFS, and if so, how is it done?
The web Test page does not support to run automatic tests associated with test cases in TFS.
You can achieve the same goals using the Build & Release features in Team Foundation Server (TFS). (https://www.visualstudio.com/en-us/docs/test/lab-management/use-build-or-rm-instead-of-lab-management). Simply integrate Selenium testing into a continuous integration/continuous delivery pipeline.
Instead of build, suggest you use a step in a release pipeline. Take a look at this question:How can i run automated selenium tests using TFS build 2015?
More detail steps about how to integrate with TFS please go through the official tutorials from MSDN: Get started with Selenium testing in a CD pipeline
I have several test cases in TFS. I would like to run those test cases using MTM(Microsoft test manager) automatically and update the test results. How to connect or configure MTM with TFS so that I can automate all the test cases.
Connecting TFS from MTM is easy, just enter the TFS name or URL and choose your team project:
Details of Automate a test case in Microsoft Test Manager, check: https://msdn.microsoft.com/en-us/library/dd380741(v=vs.120).aspx
If you use TFS 2015.2 and later versions, then it's not needed to use MTM, you can use new build system and Run Functional Tests task to executing automated tests, check: https://blogs.msdn.microsoft.com/visualstudioalm/2016/03/31/executing-automated-tests-in-build-vnext-using-test-plan-test-suites/
I am trying to execute automated test cases (codedui) from Microsoft Test Manager 2015. I have executed by triggering tfs bui. But my requirement is to run through MTM 2015.
So far I have done the following:
Created an environment in lab - MTM
Created XAML Build definition
If you just want to run code ui test through MTM, seems not related to XAML build.
What you need to do is importing the test methods to TFS as test case and then which can be added to MTM through tcm testcase /import
Such as creating test suites which are tied to the CodedUI tests, and use these sub-selections of tests to complete a test run that validates a specific change made to the system.
More detail steps please refer this tutorial: How to manage and run CodedUI tests using Microsoft Test Manager
I have written a bunch of automated test scripts using visual studio 2013, and have categorized those in the hierarchy of Test Plan & Test suite using Microsoft Test Manager.
When I am executing a test suite, I am getting proper result whether a test case has failed or passed.
I can select a failed test case and create a bug in TFS project collection.
Is there any way which will automatically create the bugs for each failed test case instead of the tester creating the bug manually?
I'm afraid you need to create bug manually. In TFS, there is no event associated with test run failed, so you can't determine when test run failed.