The background:
We have a project starting a service that gets controlled from the web interface GUI.
We're not using a specific (from a commercial point of view) web server, but an in-house created wrapper around the windows service that manages all the web interface interactions.
What we have:
Now we've started using Selenium & MSTest for testing the web interface and we're trying to get a coverage for these kind of tests, and OpenCover seemed to do the deal. The problem is that is not (or we're doing something different or wrong).
The only code coverage that I'm not getting is the one for the method used to start the windows service and all others that get called in the process (since I have all access to all the PDBs too), but afterwards nothing is covered, based on the action that take place from the Selenium's interaction with the browser.
Any hints/ideas or maybe other tools that are able to do the job (if even possible) are appreciated.
If you're running an ASP.net app, you're going to need to attach OpenCover to IIS or IISExpress to get accurate code coverage with selenium. That makes it a little hard to use MSTest with. You may want to consider moving as much logic into your services, and write unit tests against those.
Here's a quick example hot to attach open cover to IIS
OpenCover.Console.exe -target:C:\Windows\System32\inetsrv\w3wp.exe -targetargs:-debug
-targetdir:C:\Inetpub\wwwwoot\MyWebApp\bin\ -filter:+[] -register:user
Related
Whe are making a proposal to a client for a test automation framework and we are between MTM and Selenium. For what they need (CI process, and manual launch of testing bateryes) is better Selenium.
But in our last meeting the client's head of develop told us that they like the MTM's interface Test Center, because they could unify the automated testing and the manual testing in one interface. So I'm looking for some framework that would provide a similar interface, work with Selenium automated tests and, If possible, interact with their TFS.
Any suggestions?
Thank you a lot.
This is my first time playing with cucumber and also creating a suite which tests and API. My questions is when testing the API does it need to be running?
For example I've got this in my head,
Start express server as background task
Then when that has booted up (How would I know if that happened?) then run the cucumber tests?
I don't really know the best practises for this. Which I think is the main problem here sorry.
It would be helpful to see a .travis.yml file or a bash script.
I can't offer you a working example. But I can outline how I would approach the problem.
Your goal is to automate the verification of a rest api or similar. That is, making sure that a web application responds in the expected way given a specific question.
For some reason you want to use Cucumber.
The first thing I would like to mention is that Behaviour-Driven Development, BDD, and Cucumber are not testing tools. The purpose with BDD and Cucumber is to act as a communication tool between those who know what the system should do, those who write code to make it happen, and those who verify the behaviour. That’s why the examples are written in, almost, a natural language.
How would I approach the problem then?
I would verify the vast majority of the behaviour by calling the methods that make up the API from a unit test or a Cucumber scenario. That is, verify that they work properly without a running server. And without a database. This is fast and speed is important. I would probably verify more than 90% of the logic this way.
I would verify the wiring by firing up a server and verify that it is possible to reach the methods verified in the previous step. This is slow so I would do as little as possible here. I would, if possible, fire up the server from the code used to implement the verification. I would start the server as a part of the test setup.
This didn’t involve any external tools. It only involved your programming language and some libraries. The reason for doing it this way is that I want to to be as portable as possible. The fewer tools you use, the easier it gets to work with something.
It has happened that I have done some of the setup in my build tool and had it start a server before running the integration tests. This is usually more heavy weight and something I avoid if possible.
So, verify the behaviour without a server. Verify the wiring with a server. It is important to only verify the wiring in this step. The logic has been verified earlier, there is no need to repeat it.
Speed, as in a fast feedback loop, is very important. Building and testing the entire system should, in a good world, take seconds rather than minutes.
I have a working example if you're interested (running on travis).
I use docker-compose to launch the API & required components such as database, then I run cucumber-js tests against the running stack.
docker-compose is also used for local development & testing.
I've also released a library to help writing cucumber for APIs, https://github.com/ekino/veggies.
Guys I need to decide feasibility of automation testing in web application. Application is developed in c#.net. Can any one guide me about factor which I need to consider for automation testing. I have some basic knowledge of Selenium WebDriver using Java, can I test web application using it? This application is leasing application it contains many calculations to calculate plan. Application also contains many reports as well as graphs to analyse enquiries as well as results.
I suggest some basic links to start with
http://www.softwaretestinghelp.com/choosing-automation-tool-for-your-organization/
http://searchsoftwarequality.techtarget.com/answer/How-to-choose-the-best-software-test-automation-tool-for-your-team
http://www.xoriant.com/blog/software-testing-and-qa/selecting-right-test-automation-tool.html
Your approach is definitely good. You can develop a ASP.NET MVC web application in C#, setup some core business logic by classic unit test and make integration test using selenium driver.
I suggest following that you first create a lightweight skeleton of the project containing:
git repository
ASP.NET web application
continual integration with build machine (use TeamCity or similar software)
unit test testing your core business logic
integration test written in selenium driver using C# running for all common browsers with good selenium support.
Check that with every commit following happen:
build is triggered
deployment package is created
unit test are triggers
integration test using selenium driver are triggers
all reports are collected
test results are stable (same results for same build number for repeated testrun)
I downloaded nUnit and TestDriven.net. I have a legacy Web Site application and I would like to implement some unit testing. I created a class in the app_code folder and added Imports NUnit.Framework etc... After writing a basic test, I get the "Can't execute tests in 'Web Site' application." error. I guess the Web Site project is not supported. Converting to a Web Application is not an option at this time. I have Visual Studio Test Edition, so I tried that route. I created a test project, wrote a couple tests in my test class etc... only to find out I cannot reference the classes in the app_code folder of the Web Site project. Can anyone out there give me some pointers??? Am I doing something wrong. I am pretty new to TDD. I just want to properly implement a bit of testing on this application.
Thanks In Advance,
~ck in San Diego
Create a separate project for unit tests, especially if you are using a web site.
You should probably not be unit testing the classes in your web site. Rather, any classes you'd want to unit test, should probably not be in the web site. Rather, they should be in a class library project that is referenced by the web site.
This is yet another reason to not use web sites: they do not compile until they are used, so there is no compiled version of the code in app_code for your test project to test.
Web sites should be used for nothing other than simple web sites. Anything else is more sophisticated than they were intended to handle. Use a Web Application Project instead (but still keep must interesting code out of the project).
I think John Saunders has sound advice. You want to try the web site a presentation layer and isolate the classes that comprise the business logic into a separate library for testing. This allows you to focus solely on the processes that the classes are intended to implement.
So you know I started out TDD with nUnit then used the MS Test suite for a major project. I would chose nUnit over MS as it was much faster.
In my current project we are testing our ASP.NET GUI using WatiN and Mbunit.
When I was writing the tests I realized that it would be great if we also could use all of these for stresstesting. Currently we are using Grinder to stresstest but then we have to script our cases all over again which for many reasons isent that good.
I have been trying to find a tool that can use my existing tests to create load on the site and record stats, but so far i have found noting. Is there such a tool or is there an easy way to create one?
We have issues on our build server when running WatiN tests as it often throws timeouts trying to access the Internet Explorer COM component. It seems to hang randomly while waiting for the total page to load.
Given this, I would not recommend it for stress testing as the results will be inaccurate and the tests are likely to be slow.
I would recommend JMeter for making threaded calls to the HTTP requests that your GUI is making
For load testing there is a tool which looks promising - LoadStorm. Free for 25 users. It has zero deployment needs as this is a cloud based service.
You could build a load controller for your stress testing. It could take your watin tests and run them in a multithreaded/multiprocessed way.
If you are comfortable using Selenium instead of WatiN, check out BrowserMob for browser-based load testing. I'm one of the Selenium RC authors and started BrowserMob to provide a new way to load test. By using real browsers, rather than simulated traffic, tests end up being much easier to script and maintain.