Currently we use soap-ui/manual posting of xmls etc for load testing our spring boot webapplications.
Looking for any free load testing tools that others have been using and want to recommend?
Thanks for your help!
The specific load testing or in general testing tool is the one you will "like" more by lots of personal/company needs. There are plenty of them, here is the short list of them I used:
Blazemeter
Gatling
JMeter
I presonally spent most of load testing time in Gatling, first of all it is using scala development language and quite easy to include in your Java project via maven/gradle, secondaly apart from other benefits it have direct JDBC connection possibilty which let's you to have your test data directly in database. Lots of other pros. But one more time it is strongly my opinion and my preferences. E.g. if you are big fan of XML you will most probably like JMetter, in general Blazemter is kind of a next level of JMeter.
I usually code the test using my current favorite language, Python in recent times. For example in python this would be some code around the requests library, and possibly some multi-process code- nothing to heavy or complex.
I find it more flexible to code myself, on average having more control over the load (there are pros and cons to using a prebuilt tool in this sense) and it usually integrates better with other code in my automation suite.
But the answer is somewhat context dependent, is there someone with the knowledge and resource to develop a tool ? do you have to document the results or make them comparable with other systems?
Using Selenium RC I am planing for creating an automation testing framework for our Java web-application.... please share some way like how to create framework ...which things is important in framework.....
Thanks in advance
-Ravi
There are a few key factors that would greatly increase the maintainability, speed, and general usability of your tests but none are more important than abstracting your element locators out of your tests and into a central repository. I've heard of many ways of doing that but the best option I've seen is using the Page Object pattern. Essentially, each page becomes a java class, with properties representing the elements you want to use. Because you only define this page once in your PageObject and not in your many tests, if you ever need to change an element locator, there's only one place to do it.
Selenium 2 has a wonderful PageObject Factory built in but you can't use that because you want to use Selenium RC. Fortunately The Automated Tester, David Burns, has an excellent article on using Page Objects in C#, it should be close enough to get you started. http://www.theautomatedtester.co.uk/tutorials/selenium/page-object-pattern.htm For more on the Selenium 2 / WebDriver page object implementation see teh Selenium Google Code Wiki http://code.google.com/p/selenium/wiki/PageObjects
Some other really important factors to consider
Use an external framework for Data Driven tests one of the most common questions on the Selenium Google Group is how to test run data driven tests. The answer is almost always to use the data driven features that came with your unit testing tool. TestNG has become really popular with Selemium users because of it's excellent support for Data Driven tests. There's even guidance on the TestNG site for Selenium
Parallelize your tests Because most tests drive a live browser (unless you're using HTMLUnit) tests will take some time. The more you can parallelize your tests across multiple machines the faster they will run. Sauce Labs has an excellent blog series on this
Screenshot on errors less common but also a good idea. Sometimes you need to look at your webpage when there's a problem
I've got analysis paralysis looking at all the different functional testing options for a new grails (v1.3.4) application. I've looked at
WebDriver/Selenium (which I've used before)
WebTest/Canoo
Geb
Tellurium
Grails Functional Test
and there must be others. I think some of the criteria that I would use to make a decision include (in no particular order):
Likely longevity, active development
Can do ajax/javascript
Support for PageObject or similar patterns
Maturity
Headless (eg htmlunit) is ok, especially if it makes things go faster
Good reporting
Support for NTLM credential provider or similar
Compact, robust test scripts
Takes advantage of groovy language
I would be particularly interested to hear from people who have tried more than one framework. Thanks!
I maintain the plugin for WebTest but I'd recommend giving Geb a try. I haven't used it personally on a project yet but I think it will tick off most of your criteria.
It is the most actively developed (IMO) but is quite new. It is built on WebDriver/Selenium so should also be a good fit with your past experience.
WebTest doesn't see a lot of development these days and does not have built in support for page objects. But it does give you great reports. It's downside is it's historical foundation of Ant. It makes it hard to test in a dynamic fashion as your test steps are built up when the test case is executed and then the actual test is run as a second pass.
I have used GFunc as well but the lack of reporting is a real pain and it does not get as much development as Geb. It is far more "groovy" than WebTest though and is a thin wrapper over HtmlUnit allowing you to "roll your own" testing functionality quite easily.
cheers
Lee
Grails Functional Test is HtmlUnit only. I wouldn't suggest coding directly to HtmlUnit these days with the other frameworks that are available that give you page object and better abstraction out of the box.
I've used the webdriver plugin which seems to work pretty well and has page object built in - but the page object support is coupled to the JUnit hierarchy, which makes it hard for me to use because I want to also use Cucumber.
I'm very excited about Geb and am hearing a lot of success stories with it but haven't gotten to finish setting it up myself.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
I have been working as a manual tester and documentation (Test plans etc.) personnel in a company's software/IT division. I do not have a big programming background but I want learn programming language(s) and tool(s). Hence, I am looking for automated testing tools and programming language(s) to learn which also give me industrial advantage.
Some points to note are:
I need to know which tools are the most common in the industry and languages associated with those tools.
I do not have a lot of time to learn loads of stuff. Hence, I am looking for tools which use languages that are beneficial to learn in case I want to do more programming.
I'd prefer Web based app testing but that is not a constraint.
You can give 2-3 languages as I understand there cannot be one winner.
I have done some QTP which uses VBScript but VBScript is not broadly used. I want to learn a language that had broader base.
Please still give your inputs and ideas even if this question seems really difficult to answer.
Thanks in advance.
EDIT:
I think so far I have decided that I will go for:
QTP
Selenium
Test Driven Development Methodologies
I just need to figure out a good programming language that gives me a programmer's edge and is suitable for multiple testing tools (including Selenium). Maybe Python, Ruby or Java?
Wow this is quite a widesweeping question. I'd say you were in a good position as the industry moves towards a Test Driven Development (or write your tests before your code) model.
First of all you'll want to know about Unit Testing, Continuous Integration and Web Automation.
I'm going to focus on the areas I know (.NET, Java, Javascript, Build automation, Selenium)
In .NET NUnit is probably the most widely used unit testing framework. It is a port (copy of) JUnit in the java world. Most unit testing frameworks are very similar to these in terms of the concepts. So learn one and it won't take you long to pick up the others.
I think reading the above links and having an idea about these concepts will get you on your way. It's probably worth experimenting with Python or Ruby as these have low barriers to entry to mess around with some simple tests.
For web based testing the most well known are probably Selenium and Watin. These allow you to script browsers to perform actions automatically. However, I have come across very few good usage of these and they are very fiddly (you're talking days/weeks of effort) to get set up and useful. Again the concepts behind these are similar whatever framework you use.
Think I answered 4,5,6 in 1,2,3 :-)
HTH
As for 'industrial advantage' it depends what you mean by that.
For big companies QTP is widely used. Thing to learn with QTP is not really the language, but the tool itself. For me it was more troublesome to overcome tool specifics than learning VBscript. If you would think about performance testing, than PerformanceCenter would be probably the choice. Here you have C as language but again, there is more struggle with a tool than with a language.
In some companies there is stack from IBM, so Rational software comes to play (FunctionalTester, Robot, PerformanceTester). There is also a bit of tool complexity to get through (IBM Redbobks can help).
In smaller companies Java/C# could be beneficial. Here you can get familiar with junit/nunit/TestNG/MSTest. For webapps things like Selenium/Watij/Watin/Fit/Fitness/Concordian are worth to look at. If you have a moment look at AutoIT,PowerShell,Python,Fiddler,Jmeter, Abbot, Watir/Cucumber,Sikuli.
There is also lots of other tools like WebAii, TestComplete, Twist.. some others commercial tools. Usually you can get trials for them to play with it for week or two.
Everything depends on what kinds of project you will be exposed to, and what kind of company it will be.
Regardless, in my opinion it is enough to learn Java/C# to some degree. You don't need to be C#/Java ninja, know every library, and study really hard given language. You see Developing software is more than knowing a programming language, and test automation is more than knowing a tool/language (M. Fewster & D. Graham), so don't get obsessive about it. If you can program script in QTP (not record but program, using Descriptive Programming, decomposing logic to libraries, having full control of TestResults via proper methods etc.), than other tools won't be an issue. You may need like week or two to learn basics of given language, but that is all for test automation (for start at least). You need more to have some experience, to know tools, to know concepts, to know pros and cons of different approaches, to have bigger picture in head, than just 'how do I write this in that language' (you have SO for this :D ).
The answer to this question really depends on a couple of things. First, what kind of testing are you doing? Testing web-based applications? Testing libraries/components of a larger program? GUI testing? Testing command-line apps?
I have found the following useful when creating scripted/automated test facilities:
TCL - TCL can interface with the command line as easily as a batch file, but has a powerful language that blows batch out of the water (including GUI capabilities). If you are scripting Windows-based console apps (either to test them or to use them to test something else), I recommend taking the time to lean TCL.
AutoIt - This scripting language lets you automate UI interaction with Windows apps quickly and easily. It is also fairly easy to learn. If you want to take the time and trouble to develop a screen-scraping utility, you can combine it with AutoIt to create some pretty sophisticated test systems.
C - When testing compiled libraries, I have had a surprising degree of success using good 'ol C. I'll write the basic skeleton for a test app and create a scripted utility that will dynamically generate the rest of the code based on what tests need to be run (I give it a library of sample C snippets that it can use for more complicated tasks).
Ruby - When you are doing automated testing, a robust scripting language can go very far. Whether you need to parse test logs, generate reports, or auto-generate test scripts, a scripting language that you know well can save you a lot of time and headache. My language of choice is Ruby, but others at my company prefer other languages such as Perl or Python. Language choice isn't too important, pick one and learn it well.
Depending on what it is that you are testing, you may also have some sort of testing tools built into the code that you are testing. Java code is sometimes written with JUnit tests, Ruby code can use the Test::Unit framework, etc. Here is a list of unit testing frameworks for a wide variety of languages. If the code you are testing was built with unit testing in mind, you will want to learn the language the code was written in so that you can take advantage of these unit tests.
For web based testing you might want to check out fitnesse. Also, an automated test framework built on python is the robot framework.
Selenium is very popular for automated testing at the QA level.
No matter what you do, there will be some learning curve associated with your solution
The point of Selenium is to provide a testing framework for webapps -- you use Selenium to manipulate DOM elements in the browser, no a running version of your application.
Selenium is a client/server model (you run the server somewhere that has access to your QA app instance; you code your tests using the client libraries) that provides clients in a number of languages (java, ruby, etc)
This type of testing is a good idea if you have have dedicated QA resources that can write code. It provides some high level regression testing. However it is not without its drawbacks: (1) tests can take some time to run, (2) you might need to make some changes to your web application dom elements to use the framework easily, (3) changing the web app will require changing your testing code, which can be non trivial.
Your other options is to write unit tests for the application code. Usually the developers do this as part of a test-driven process. So there is TDD for developers and then automated QA testing for QA.
I will also add a small comment
There is a new testing automation platform XML2Selenium
This platform is built on top of classis Selenium framework and provides many useful features
But it is quite easy to use
You don't need to learn any java or ruby - you can write in XML
SMth like that:
<!-- here we have some imports Frame - is like a class, you can extend it with extends -->
<import resource="Demo.a-server/03Registration/framesRegist.xml" name="frames"/>
<import resource="Demo.a-server/01Action/FramesAction.xml" name="framesAct"/>
<import resource="Demo.a-server/03Registration/FramesLogin.xml" name="framesLog"/>
<property resource="Demo.a-server/LogValue.properties" name="logValue"/>
<test name="creatAction" description="create and check action">
<frame extends="frames:logAdmin"/>
<frame extends="framesAct:createAct"/>
// here we upload the image
<upload id="img_id0" resource="image.jpg"/>
// click save button with id=save
<button id="save" />
// make a screen shot
<screenshot/>
// sleep for 3 seconds
<pause time="3000" />
// make a screenshot again
<screenshot/>
// navigate to the given URL:
<navigate to="${logValue.baseUrl}/admin"/>
<frame extends="framesLog:logOutAdmin"/>
<pause time="1000"/>
</test>
Here you can see report with errors:
http://xml2selenium.com/demored/xml2selenium/
And here you can see report with successful state:
http://xml2selenium.com/demogreen/xml2selenium/
PS: It supports testing in All browsers, not only in Firefox as Selenium IDE
Yes you can learn selenium IDE, Its just as an add-on on the firefox browser, using which you can record your tests and play them when required. The limitation of IDE is it can only be used on firefox.
If you are looking to test on multiple browsers, you need Selenium Grid or WebDriver, which requires a knowledge of java. And again you are not needed to be a Java pro to do this, a simple learning will help.
You can search the youtube.com for Selenium Tutorials.
Advantages of Selenium :
Its Open sourse!!
Its easy to learn and impliment..
Supports multiple browsers.
Supports multiple scripting/programming languages.
Flexible to your choice, simple record and playback (IDE), Comple Framework design to your choice by using your favorite programming language.
Supports parellel execution.
Hope this will help you.
Can anybody suggest a good tool to test GWT UI. What is the opinion about using Selenium & Webdriver ?
Any pointers and suggestions are welcome.
Thanks,
SD
Google themselves have an excellent article describing when and how to apply different testing strategies to GWT apps. The basic idea is to use unit-testing whenever you are testing business logic, but to use Selenium for specific end-to-end tests where you want to simulate user interaction.
The most important thing is limit how much GWT-specific testing you need to do since GWTTestCase and Selenium are so slow. The MVP pattern referenced in the Testing Methodologies helps enormously with this. Always ask yourself "Can I make this view any dumber?" Push logic for dynamically changing the UI into presenters that manipulate views so that you can test the logic in JUnit.
Then I'd suggest doing most of the UI testing in Selenium/Webdriver. Those tests should be fairly simple though: "Does X show up when I click Y?" kinds of things. GWTTestCase ends up being the most useful for me for code that needs to run under Java and GWT.
Or, as an alternative to the MVP pattern, you may use the gwt-test-utils framework to write simple tests in java, which runs faster then GWTTestCase tests :-)
You can do bog standard unit testing for your GWT components, if you want to do some layout checking things which can't be done through unit tests (can't give you examples since I don't know GWT nearly at all) then you may use Selenium, however Selenium does have some issues in specific cases, namely selecting elements and very rich content which may prove to be hard to handle since GWT is all about rich content.