For web application automation Selenium can be used with Robot framework. But both are frameworks.What is the relation between these two?
You have selenium to automate all the web related work e.g login,click,button and many more thing. But then you have to use it with some languages , e.g. Java ,Ruby, Python.
Suppose you get a project to automate a webbrowser by using any other languages , where your task would be
1)login to browser
2)Fill user details
3)Click on submit
Now to have a good framework , you need to break down these tasks into smaller component
1) you need to define test cases
2) You need to have a separate file to store variables
3) You need to have a good reporting tool , which will show how many test cases passed or failed and further drill down.
i am a python user,so lets talk the problem with python,selenium
1) you can write the test cases with unittest module - But then generating a good test reports would be hedache , you have to spend a lot of time to create good test reports
this is one of the major disadvantage with python
Now coming to RobotFramework
if you integrate selenium library in robotframe work, you would be able to do almost similar thing which can be done by any other languages with lot of ease and control.
Taking example of your assignment in hand
1) You can define Test cases
2) You can Create a seprate variable file and then pass it with main file during run time (check pybot -V)
3)You dont need to be worry about reporting part , all the reports would be generated and with betterdrill down options
additional advantage
1) There are lots of inbuilt libraries, which will help you to do your task easily
2) You can create your own custom library and import in Robotframework
3) You would be able to drill down till last variable where the problem is with help of RobotFramework Reports, which will save lots of time
In Nutshell i can say Robotframework provide a building block to your framework where you just need to worry about the functional aspect of your program
Your original assertion is incorrect. They are not both frameworks. Or at least not the same type of framework.
robot framework is a set of programs and libraries for creating test cases. With it you can create test suites built upon reusable keywords written either from other keywords or written in other programming languages. The framework provides a test runner, and generates test reports.
selenium is a library interface to a driver that controls a browser. You cannot write tests using only selenium -- you need something else such as a programming language (python, ruby, etc) or testing framework (robot, cucumber, etc). Selenium itself provides no way to run tests, and no way to generate reports.
Related
What to answer if someone asks both TestNG and Hybrid(Data Driven and Keyword driven etc) are frameworks how will you differentiate both of them?
Keyword Driven
Keyword Driven Framework is a type of Functional Automation Testing Framework which is also known as Table-Driven testing or Action Word based testing. The basic working of the Keyword Driven Framework is to divide the Test Case in to four different parts. First is called as Test Step, second is Object of Test Step, third is Action on Test Object and fourth is Data for Test Object.
In other words, Keyword driven framework in an approach which you can write in any language. The main goal is to bind an keyword with an even like suppose you need to send your text to an textbox. Now you can select an keyword like sendText, now internally you bind this keyword with sendKeys. In excel sheet you will mention sendText and keyword framework will identify that you want use sendkeys events.
TestNg:-
TestNG is a testing framework inspired from JUnit and NUnit but introducing some new functionalities that make it more powerful and easier to use, such as:
Annotations.
Run your tests in arbitrarily big thread pools with various policies available (all methods in their own thread, one thread per test class, etc...).
Test that your code is multithread safe.
Flexible test configuration.
Support for data-driven testing (with #DataProvider).
Support for parameters.
Powerful execution model (no more TestSuite).
Supported by a variety of tools and plug-ins (Eclipse, IDEA, Maven, etc...).
Embeds BeanShell for further flexibility.
Default JDK functions for runtime and logging (no dependencies).
Dependent methods for application server testing.
In other words, TesNG is a plug-in for eclipse. It's has some annotation by using that you can run your scripts. It's java eclipse specific. It is not only design for selenium
Hope it will help you :)
I am new to testing. The commonly used terms like Framework and tool confuses me a lot. Can anyone please explain me the difference between a Framework like STAF[software testing automation framework]and Tool like selenium.
Also how to select a tool for a particular framework. What are the criterias used for selection?
Brief explanations are welcomed!!
Tool:
Simply put, a tool is a software. In case of test automation, tools are software that let you automate your tests on an application. There are many test automation tools that you can choose from depending on your requirements. Some examples are Selenium, UFT, Visual Studio CUIT, Jamo Solutions Meux Test, T-Plan Robot, Telerik Test Studio etc.
Often, you'll have to write tests in the tools using a supported programming language. For instance, testers using UFT need to code in VB while those using Visual Studio can code in both VB and C#. However, some testing tools (like Telerik Test Studio) let you write script-less tests where your tests will consist of a bunch of easily understandable keywords, not code.
Framework:
The most popular test automation tools like Selenium and Visual Studio provide all the basic features you require to build your own tests. However, they do not provide ready-made features (like Reporting and Exception Handling) for testing. This requires the creation of a 'Framework' which is nothing but a collection of code written using a tool of one's choice that makes testing an application easy. Simply put, a framework is what you create with a tool (or a collection of tools) to test your application.
A typical framework consists of two parts: test scripts and function libraries. Test Scripts are the pieces of code that need to be executed to perform actions on the application under test (AUT). Function Libraries are classes consisting of important functions that are called by your test scripts. These can include timing functions, reporting/logging functions, exception handling functions, data communication functions etc.
You can also use an external database to pass test data to your test scripts during run-time instead of hard-coding it in your test scripts. Frameworks that employ external databases are called data-driven frameworks. The external database can be of your choice, be it a SQL Server, an XML file or a simple Excel spreadsheet. Data-driven frameworks make use of APIs or include custom-made classes that let you communicate with the database to transfer data.
Another type of framework is the keyword-driven framework. These frameworks are used in long-term test automation projects that require scripting of thousands of test cases. The main objective of these frameworks is to reduce the time taken to script a test case by reusing code that has already been written. They often include very strong function libraries which enable scripting of test cases using just predefined keywords. For example, common actions on an application like login and logout are performed by single line codes like:
Actions.Login();
and
Actions.Logout();
where Actions is a Function Library that consists of the Login() and Logout() functions. This massively reduces the script size and the long-term maintenance requirements of the test script, among other benefits.
Of course, you can either build a test automation framework and use it for your own application or create a generic test automation framework and make it available to the testing community for everyone to use, which is what STAF is.
Selection of testing tools:
To address your second question, there is no straight-forward answer to it. There are a number of criteria that can affect your decision. But in the end, it is all about your requirements and the requirements of your AUT.
If it is a Windows desktop app, you have to use Coded UI Tests in
Visual Studio.
If it is a Web application, you can use Selenium, UFT, Visual Studio or Test Studio.
If it is a mobile app, you can use Appium, Jamo Solutions Meux Test or T-Plan Robot.
If you want to test your mobile app over a large number of devices
and platforms, you can use cloud-based tools like Sauce Labs,
Perfecto Mobile or Device Anywhere.
If you are short on budget, you'll be better off using open source tools over
commercial tools, and so on.
Application Testing is a huge industry now and there is no dearth of testing tools available in the market. You will find the perfect tool for you if you know what you want and do some research on Google.
I will try to answer what I believe people normally use these terms to mean, lets start with the simpler term: A tool.
A tool like selenium is what actually does the automation, it has an API that will work for pretty much anything it covers (in this case websites) but it knows nothing about how the website you want to test works, this means it deals with low level constructs such as elements on a page and clicks.
A framework is normally just wrapping a tool to make it easier to make a test by imparting knowledge of your application, a standard example is login.
Say you want a test that checks when you enter a correct username and password in you get access to application. Using just selenium it would something like:
driver.findElement(By.id("username").sendKeys("MyUsername");
driver.findElement(By.id("password").sendKeys("password123");
driver.findElement(By.id("login").click();
Thats pretty simple, but as you can guess login is going to be used a lot across your tests and so it makes sense to move this into a place that makes it easier to reuse (both from a less code stand point and maintainability). This is where a framework comes into play, normally with selenium it will be page objects (see here)
Base on my understanding:
TOOLS
We "USE" tools to meet our objective (can be own self or your small groups of team).
Example: We use Selenium IDE as a tools helps us to automate some repetitive steps to do certain verification during our smoke test.
FRAMEWORK
We "DESIGN" a framework to meet the organization mission.
Things to consider when we design the framework including:
Maintainability
Reusability
Data Driven
Reporting
Schedule running through CI tools like Jenkins
Example: We design a Test Automation Framework by using WebDriver + Java + TestNG + ANT, to meet the objective to identify our current code base stability, and the test will be trigger and run by jenkins in a daily basis, SSRS report will captured in a daily basis as well each time the test finished. Stakeholders can review the daily code stability report anytime he/she needed.
Hope that can help you :D
is there any tool out there that i can used to set-up run automatically and i was goggling and i found selenium test runner? there are so many tools out there its hard to figured out which is best
I'm using C# and using MSTest as a test framework and I'm looking forward to see if I can get a way from testing in MSTEST
any help?
This is very subjective question. Every requirement will have its own correct answer. Anyhow I will try to address few requirements and will be updating as I learn more.
If you are automating web app browser tests (sans flash player and silverlight) I would say that selenium is the way to go. There are ways to automate flash and silverlight too, but that is answer for another question.
Selenium is anyways an automation too and your choice will rather is of which test framework to select. So here are few options:
1. Integrating with CI tools:
If you want to organize your tests as segregated atomic units and want them to be integrated to some CI server (e.g. TeamCity). I will recommend using NUnit to run your selenium tests.
2. Behavioral Tests
It is a new trend in the software development and how we test our products. Using behavioral (i.e. business specification) like language. In my experience it is also a very good format to write up acceptance tests. You can use selenium with something like Nbehave or SpecFlow
3. Centralize Test management and Execution
Now this might not fit for everyone but I have found FitNesse (and its c# binding) to be very useful in maintaining and executing selenium test cases.
Please note this answer may not be right and is certainly not complete given the scope of the question. I have nevertheless tried provide few pointers.
What open source software should I use to write scripts to test for no errors on a site?
Could I / we write something better ourselves if there a limited number of goals outlined?
- yet flexible enough to take on new rules etc.
The only consistent response we want is no errors, period.
I know Java, ASP and scripting languages if that helps.
Thanks!
Selenium is one good Website automated testing tool. It allows macros as well as hand-written scripts. Also has support for Firefox browser.
Understanding of Java should suffice.
You can check it out at http://seleniumhq.org/
Another good open source alternative is HTMLUnit http://htmlunit.sourceforge.net/
Again this requires knowledge of Java
You might want to consider the robotframework, combined with the selenium2library keyword library. It lets you write very human-readable tests and gives very nice reports. It integrates nicely with jenkins. Robotframework is written in python, and can be extended with python. It allows you to create data driven tests, BDD-style tests, or more traditional procedural tests.
Is anyone aware of any ongoing open source project that integrates robotframework with a load testing tool such as grinder, jmeter, funkload etc?
Thanks
Yes. There is a Python library for integration of Robot Framework and JMeter: Robot Framework JMeter Library . It can be used for running JMeter and parsing and converting results. I am author of this library so I might not be objective.
No, and that's likely not to happen. Robot Framework is for functional not load testing. How would you deem a load test as pass/fail and how long does it run?
Robot Framework and functional tests have a finite set execution time (takes as long as it needs to complete testing the particular feature or times out before doing so in case it hung, etc.), and has strict criteria as to what is pass/fail when test runs.
With load testing, you at least during exploratory runs and design of test, you don't run for fixed time, or even if fixed, it's usually not short (except trial runs and scalable burst increases). And criteria for pass/fail is usually within ranges rather than yes/no.
So it's harder to integrate and design a test library that can offer pass/fail and run within some set time for load testing. Unless someone can define a good architectural design of a test and test library for how to do so with Robot Framework.
I think the idea would be that a test case is created only once and can be used in both functional tests as in load tests and even in end user monitoring. In this (utopic) way a test case can be used during the whole lifecycle of an application. With a tag (for instance) a test case can be promoted to be also a loadtesting test case with another type of response validation. Would be nice to run Robot framework and create a Loadrunner-TrueClient (or another browser-driven loadtesttool) script. Main purpose of the integration would be to automate the scripting.