Greetings, we have a project with loads of beans, JSP and etc. There is a desperate need for performing automated tests in our environment (we use Maven). Now, we can easily write tests for database project layer, for various security utilities we implemented. But the JSP pages remain untested.
I searched for utilities for server-side testing and Cactus seems the best option. However, according to their changelist, their last release is 1.8 and it was released more than two years ago!
So the question is - what happened to Cactus, is it still developing or what? And what are the recent alternates for Jakarta Cactus (if any exists)?
I've used a combination of Spring, JUnit and HttpClient with some success in recent projects.
Apache HttpClient provides a powerful and flexible API for constructing and sending http requests into your application. It cannot replicate a web browser, say by running client side scripts, however if there is sufficient content within the resulting http responses (headers, URI, body), then you can use this information to traverse pages within the application and validate the behavior. You can post forms, follow re-directs, process cookies and supply the inputs into your application.
JUnit (junit.org) drives the tests, invoking a series of pages with HttpClient and can be deployed alongside the application, run standalone with ant/maven, or run separately inside your IDE.
Spring (springsource.org) is, of course, optional as you may not be using it for your project. I've found it useful to stub/mock out parts of the application, such that I can isolate specific areas, such as front-end controllers, through to the business logic, by substituting the DAOs to return specific data values. It provides an excellent Test Context Framework and specialized TestRunners that hook in well to testing frameworks like JUnit (or TestNG if you prefer).
Cactus served as a good server-side testing framework in the ejb2 ages and but it's not supported anymore.
You can use combination of both Mock testing (fine-grained) and In-Container testing (coarse-grained) strategy to test your application completely.
Mock Testing Frameworks : Mockito, Jmockit, EasyMock etc..
Integration Testing Frameworks (Java EE) : Arquillian, Embeddable API, etc..
I prefer Mockito and Arquillian for server-side testing.
How about Arquillian? I haven't used it and it doesn't even have a stable version yet, but at least it's in active development.
You might want to try selenium. That with jBehave is a good combination I'm finding. And the more support for both those projects, the more they will not go defunct (like cactus).
Related
Where does the functionality of the Selenium lie?
I've read somewhere that the language binding or API provides glue code to use Selenium in java.
Glue code or Binding code: The code that connects incompatible software components.
Jason Huggings wrote JavaScriptTestRunner to control the browser. The functionality to control the browser lie in the JavaScriptTestRunner.
It suffers from Same-origin policy. Paul Hammant created HTTP Proxy to bypass the Same-origin policy. This opened the doors to write tests in multiple languages. We are provided with an API to write tests.
Then WebDriver is created. WebDriver and RC are merged to form Selenium 2.
Where does the functionality to control the browser gone to?
Within the API?
Within the browser driver?
Why does the API is called as glue code or binding code? What does it bind?
Selenium in the general usage of the term is a library - a collection of code organized in modules and packages. In this form it's a WebDriver client - it can communicate with that kind of server, following the WebDriver protocol; thus enabling a programmer to control a browser.
Where does the functionality to control the browser gone to?
In the WebDriver protocol, and the server that translates the WebDriver commands to browser instructions, and back.
Selenium being a client gives you the ability to use this functionality, in your chosen language - conveniently, without you having to learn or directly run WebDriver requests and parse their response.
Why does the API is called as glue code or binding code? What does it bind?
In programming binding is the uniform / or well-defined API of a library, that connects lower-level code (other programs, or the OS) or protocols - WebDriver in this case, to higher-level concepts - your code. Wikipedia to the rescue, with more details.
Glue Code
Glue Code is the executable code most often the source code that serves the purpose of adapting different parts of code that would otherwise be incompatible. Glue code does not contribute any functionality towards meeting any program requirements as such. Instead, it often appears in code that lets the existing libraries or programs to interoperate among themselves, as in language bindings or foreign function interfaces. Glue code may be written in the same language as the code it is gluing together, or in a separate glue language. Glue code is very efficient in rapid prototyping environments where several components are quickly put together into a single language or framework.
JavaScriptTestRunner
Jason Huggins while testing an internal application at ThoughtWorks reduced the time which was required to manually step through the same tests with every change he made developing a Javascript library that could drive the interactions with the webpage which allowed him to automatically rerun tests against multiple browsers. He named this program as the JavaScriptTestRunner. Later, he made JavaScriptTestRunner open source. This library eventually became Selenium Core which underlies all the functionality of Selenium Remote Control (RC) and Selenium IDE.
Unfortunately, to work within the Same Origin Policy, Selenium Core must be placed in the same origin as the Application Under Test (AUT). So another ThoughtWork engineer, Paul Hammant, created a server that would act as an HTTP proxy masking the AUT under a fictional URL, embedding Selenium Core and the set of tests and delivering them as if they were coming from the same origin. This system became known as the Selenium Remote Control (Selenium RC), or Selenium 1.
Why to use HTTP proxy
Again, when web frameworks were becoming more complex and powerful, the restrictions of Web Browsers’ sandboxed Javascript environment were increasingly limiting the effectiveness of Selenium Core. Simon wanted a testing tool that spoke directly to the browser using the "native" method for the browser and operating system, thus avoiding the restrictions of a sandboxed Javascript environment. That was when WebDriver and Selenium RC were merged to form Selenium 2. All implementations of WebDriver that communicate with web browsers started using a common wire protocol. This wire protocol defined the RESTful web service using JSON over HTTP.
Conclusion
In short, the functionality to control the browser was always within the Javascript library that driven the interactions with the webpage which was part of the JavaScriptTestRunner and later Selenium Core.
As per the diagram below with respect to the different classes/interfaces the functionality to control the browser is with the selenium-api
Guys I need to decide feasibility of automation testing in web application. Application is developed in c#.net. Can any one guide me about factor which I need to consider for automation testing. I have some basic knowledge of Selenium WebDriver using Java, can I test web application using it? This application is leasing application it contains many calculations to calculate plan. Application also contains many reports as well as graphs to analyse enquiries as well as results.
I suggest some basic links to start with
http://www.softwaretestinghelp.com/choosing-automation-tool-for-your-organization/
http://searchsoftwarequality.techtarget.com/answer/How-to-choose-the-best-software-test-automation-tool-for-your-team
http://www.xoriant.com/blog/software-testing-and-qa/selecting-right-test-automation-tool.html
Your approach is definitely good. You can develop a ASP.NET MVC web application in C#, setup some core business logic by classic unit test and make integration test using selenium driver.
I suggest following that you first create a lightweight skeleton of the project containing:
git repository
ASP.NET web application
continual integration with build machine (use TeamCity or similar software)
unit test testing your core business logic
integration test written in selenium driver using C# running for all common browsers with good selenium support.
Check that with every commit following happen:
build is triggered
deployment package is created
unit test are triggers
integration test using selenium driver are triggers
all reports are collected
test results are stable (same results for same build number for repeated testrun)
The background:
We have a project starting a service that gets controlled from the web interface GUI.
We're not using a specific (from a commercial point of view) web server, but an in-house created wrapper around the windows service that manages all the web interface interactions.
What we have:
Now we've started using Selenium & MSTest for testing the web interface and we're trying to get a coverage for these kind of tests, and OpenCover seemed to do the deal. The problem is that is not (or we're doing something different or wrong).
The only code coverage that I'm not getting is the one for the method used to start the windows service and all others that get called in the process (since I have all access to all the PDBs too), but afterwards nothing is covered, based on the action that take place from the Selenium's interaction with the browser.
Any hints/ideas or maybe other tools that are able to do the job (if even possible) are appreciated.
If you're running an ASP.net app, you're going to need to attach OpenCover to IIS or IISExpress to get accurate code coverage with selenium. That makes it a little hard to use MSTest with. You may want to consider moving as much logic into your services, and write unit tests against those.
Here's a quick example hot to attach open cover to IIS
OpenCover.Console.exe -target:C:\Windows\System32\inetsrv\w3wp.exe -targetargs:-debug
-targetdir:C:\Inetpub\wwwwoot\MyWebApp\bin\ -filter:+[] -register:user
I've done load tests for users building scripts in LoadRunner or The Grinder, and now I'm trying out JMeter and it all feels incredibly clunky. Where are the scripts? Does everything have to be done through the UI? Is JMeter able to do complex scripting?
JMeter has good user friendly GUI. We create scripts in JMeter using the UI. JMeter saves the script in XML format with .JMX extension. Script creation is NOT very difficult as you say.
Check this site to get an idea.
Complex scripting can be done in JMeter using logic controllers.
Also JMeter
Is Free & Open source
Is Light weight & easy to install
supports any platform
Supports many protocols - HTTP/HTTPS, FTP, SOAP, LDAP, JDBC, JMS, SMTP, POP etc
Supports external plugins
Can be extended with Beanshell scripts, Groovy, Javascript, Java.
I have tried out Jmeter briefly but I think to it's for you to try it out yourself, there are a number of Jmeter tutorials on the Jmeter site itself as well as on youtube which should prove useful.
Below are some links:
http://jmeter.apache.org/usermanual/intro.html
https://www.youtube.com/watch?v=cv7KqxaLZd8
Hope these links help you in getting a better understanding of Jmeter.
What do you call "scripts"?
JMeter is designed so anyone could create tests using UI only. JMeter's Logic Controllers, Pre/Post-Processors, Assertions, etc. are quite enough to build a load test of any complexity.
If you feel yourself too creative and you're limited by JMeter Test Elements you're welcome to extend it using
Code-enabled test elements like BSF, Beanshell, JSR-223 Pre/PostProcessors, Samplers, Timers and Assertions
Developing custom functions
Developing custom implementation of Java Request Sampler
Developing custom implementation of your own Sampler or Function
Finally it is possible to either run existing or create new tests using JMeter API
Is there an Objective-C version of Artifice?
If not, how would I design/develop/create it?
Related Questions
Mock HTTP response via Objective-C
Mock NSURLConnection
I think I might be able to help you here.
I have a Ruby library that is somewhat similar to artifice, albeit more self-contained and built on top of Sinatra, called Mimic. I'm pretty happy with it and one of my favourite features is that as well as being configured using it's Ruby DSL (or using the Sinatra API directly), it can be configured remotely or from any process that speaks HTTP. This means you can use it in your Objective-C tests and configure it from the tests too (rather than having say, a set of external fixtures in a Ruby file).
In the name of eating my own dog food, I recently converted the acceptance tests for my Objective C RestClient port, Resty to use Mimic. The Mimic daemon is started up as part of the build process and my stubs are configured directly in the tests, using a thin Objective-C wrapper around the Mimic REST API.
As you can see, I strive very hard for test clarity!
Those tests use OCUnit but you can use this with Kiwi. In fact, the assertEventually macro in the above tests was the basis of the asynchronous testing support that I ported to Kiwi.
I've since extracted the Objective-C wrapper for Mimic from LRResty and moved it into the Mimic repository. You may want to check out the Resty project to see how my project and the tests are configured. If you have any questions, please ask.
One caveat: I haven't found a way of getting these tests to run successfully in Xcode 4, using the "Test" option, due to the way that it runs. In Xcode 3, I rely on Run Script build phases to start and stop the Mimic daemon, but because Xcode 4 doesn't run the tests as part of the build process this doesn't work. I've tried to accomplish something similar using pre/post test actions but unfortunately these are woefully inadequate due to various bugs.
Bonus tip: I find Charles Debugging Proxy as massive help when working with web services and you can use it with Mimic too; the Objective-C wrapper can be proxied through Charles so you can see exactly what is happening, both in terms of stub configuration and actual HTTP requests (Mimic can even be configured to return some helpful debugging data in the response headers).
Do let me know if you have any questions.