Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 7 years ago.
Improve this question
What do you recommend for testing purposes in Java environment (Selenium or Rational Functional Tester)? Could you write some pros and cons? I heard about MicroFocus TestPartner, do you have experience with it?
Both automated test tools offer some of the basic capabilities of automation. However there is a massive difference between Rational Functional Tester (RFT) and Selenium. IF your aim is basic testing, where the outcome is short term -1-3 months and you don't mind that the scripts will need a high level of maintenance, then Selenium is fine. RFT provides a number of critical differences from Selenium that impact your ability to have a long term automation solution. These are:
RFT uses an object map between the script and the application under test. This means that as the application changes you don’t need to find and replace object properties in the scripts. This will save you a lot of time
RFT uses datapools for driving large datasets into a test. this allows permutations and combinations to be created.
RFT has very powerful connectors to multiple application styles like SAP, Siebel, Oracle, VB, PowerBuilder, various web and terminal server functions. This means one tool for every situation where selenium provides only one solution.
RFT works with RQM for full end to end test management.
The net result of all this is time and longevity of the automation. I would not choose Selenium for a long term solution because RFT will cost substantially less in maintenance than RFT ever will - even for a novice.
Please clarify 'Java environment'.
If Java environment == a web based application delivered through a browser, then you have tons of choices at your disposal, not just Selenium, Rational Functional Tester and TestPartner. There are a number of open source and commercial tools for testing web apps.
If, however, Java environment == standalone Java application, that runs on your desktop, then your choices are more limited. Selenium will not work, as Selenium only supports browser-based web apps. Functional Tester and TestPartner are both good tools, I have worked extensively with both, and both support the Java platform. There are other tools, such as HP QuickTest Pro, and there are probably some Java-specific open source tools.
I'd recommend mixing the two: When there are things that you cannot do in RFT, you can get help from Selenium, which is simply a Java API which can be used in RFT. Writing cross-browser script can be a good point to use this mixed approach.
One more point to consider is: Selenium is a free and open source tool.
Another point is: Selenium has a much more activity in its discussion forums. Its is more likely to get a faster answer from other users of Selenium.
Since you mentioned TestPartner which is a commercial tool.
QTP and TestPartner have a comparable pricing level. So you should consider other important factors: integrity of the tool, learning curve, level and quality of commercial support, level of community support.
You may also want to look at TestComplete, which is also very powerful, and incorporates support for Load Testing, and Unit Testing, but has significantly lower price.
Generally, I would say, automation success is much more dependant on automation skills of a person rather than on capacities of a specific tool.
Thank you,
Albert Gareev
http://automation-beyond.com/
Take a look at TestMaker by PushToTest.com . It's written in Java and runs Java and Selenium scripts as well as soapUI, Groovy, Ruby, Python, PHP, VB and .Net. TestMaker allows you to take the Selenium scripts and run them as automated regression tests, load or performance tests and production SLA monitoring scripts without having to write the tests in multiple languages or for multiple tools. You can also run them in clouds such as EC2, GoGrid, Rackspace and CollabNet.
There is a free community version and a commercial Enterprise Version. When you purchase the Enterprise version, you get support for the tools including support on Selenium. You can download TestMaker from the website.
If you're not sure, there are free webinars several times a month on how to get the most out of the latest proven Open Source Testing tools. Definitely worth checking out.
If you are targeting only web based application with UI having HTML , we can use selenium.
If you want to use commerical tool you can go ahead with RFT and it supports different UI interfaces when compared to selenium
Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed last year.
Improve this question
I am job searching and am seeing a lot of Selenium requirements in job descriptions lately. I know that Selenium is an automation tool for web testing but what I don't know is how it compares to using AutoIt and Sikuli. I am very familiar with AutoIt and Sikuli tools but not familiar with Selenium as far as enterprise use. I could simply download Selenium and fool around with it but as far as professional experience, I have none.
So my question to automation professionals out there is if I have a good deal of experience in AutoIt and Sikuli, would I be a good fit for a position that deals solely with Selenium?
Have experience only with AutoIt and Selenium .
What I can say about AutoIt -> script written on it are suitable for WIN platform only.
Imagine the situation when you need to run you test (that covers a piece of functionality on a web page) on 2 notes: win note and macBook.
For Win your script be working OK , for Mac it will fail.
OR, a little bit modified: if you need run your script simultaneously on several machines and/or in several browsers. Selenium will fit OK for this.
But for another side: such case like attachment a file on a webPage or any other kinda of attachment - will not work with pure Selenium. AutoIt will handle it nice.
So it was 2 little casees in particular.
In overall, selenium is used for functional UI automation of web application(-s) . And if you have experience of scripting you will be able to adjust your skills.
My recommendation to you - start recording script with SElenium IDE , then export these recorded scripts in testNg , and then try to understand their structure.
Good luck! :)
As someone who has fairly extensive experience with Sikuli and Selenium (but none at all with Autolt), I would say that Sikuli skills are not very transferable to Selenium. The theory side--the logic and algorithmic thinking you'd use to problem solve would be much the same, but the technique of execution are so very different between the two, that if I were doing the hiring, I wouldn't hire a person with no experience in Selenium but vast experience in Sikuli, if there were any reasonable Selemium-experienced people to choose from.
I imagine, though, that if you could demonstrate proficiency, or create and show your own projects that use an array of Selenium skills, that could go a long way to smoothing reservations a hiring company might have...
Take that for what it's worth, esp since I have no experience with Autolt.
Selenium is used for Web-Automation or Web Applications or website related automation.
Autoit can only be used for Windows GUI Automation or Windows based applications. No Image Recognition purely on elements based usage.
Sikuli on the other hand, can be used for automating both i.e. Desktop Apps and Web-Apps. It basically finds the match of the given Image in the screen by using Image Recognition and then you can perform tasks on that like click, double click, input text etc.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 8 years ago.
Improve this question
Guys I am looking for Web Testing Generic Automation Framework which can be used to do automation testing of various web based applications .Looking for C# based framework as that is the language I am more familiar with. But any other language framework will also do and it should not use any proprietary/licensed language.
Framework should have some open source and free of cost license model.
I searched for selenium based framework on Google and SO. But could not come with any which have source code available. It will be good if the framework encapsulates all the functionality provided by Selenium WebDriver and/or Selenium RC and empower the functional tester to create and maintain test in human readable scripts.
Requirements of the framework:
The framework code should avoid hard coding of test steps. My idea is to maintain the test scripts outside the automation framework code , so that they can be easily be modified if needed. The framework should read through the step tables and the data tables and run the test accordingly.
If there is no such framework available now right then we can collectively build such a framework in a open source community model.
P.S.
I have read a little about Hermes Framework and Robot Framework, but not yet tried them, any help is welcome.
The good side of this problem: there are a lot of flexible tools and approaches, you can get together and build a flexible, reliable and robust test automation framework.
The hard part is: yes, there is no “out of box” solution, and you’ll need to find and put together lots of tools in order to solve this test automation puzzle.
What I would recommend:
First you need to choose a unit-test test framework. This is a tool which helps to identify separate methods in code as tests, so you can run them together or separately and get the run results, such as pass or fail.
My personal opinion, is that the testing tool – MS-Test – which ships with Visual Studio 2013 (and also Express Edition) is good enough. Another alternatives are: NUnit or Gallio Icarus
All unit-testing frameworks includes a mechanism for doing assertions inside the test. The capability of assertions class depends on given unit-testing framework. Here, I would like to recommend a popular library which works great for the entire unit testing framework.
This is Fluent Assertions (also available from NuGet repository).
That’s a hard moment. You need to decide: are you going to use the PageObject approach in order to build your test automation framework, or you are going to choose simpler approach, without heavy utilization of the Object Oriented Programming.
Properly designed Page Objects makes your test automation code much maintainable. Utilizing the OOP – you can do a magic in your code: write less to do more. Although, such approach requires more skill.
Here are a good articles on this topic:
Maintainable Automated UI Tests
And this one:
Tips to Avoid Brittle UI Tests
The alternative to the PageObject is a scripted approach. This approach can be also successful and requires less time to start.
Coypu is a good and usable example of such framework for Selenium Web Driver.
All the popular unit-testing frameworks support data-driven tests. The best support is in NUnit – you can run/re-run and see the tests generated for individual data row in the tests tree.
MS-Test supports reading data from different data-sources: text files, excel, mssql etc., but it is not possible to re-run the test for individual data row. Although, there is a hack for this – Ms-Test Rows.
For my data-driven tests, I am using a great library – Linq to Excel
I have a lot more to say. There are so many approaches to build test automation framework – and there is no ready solution yet.
I am trying to build one according to my testing methodology – SWD.Starter .
This project is still on its early development stages. But, at least, probably you’ll find a few tips how to build and organize the test automation code.
I've implemented https://github.com/leblancmeneses/RobustHaven.IntegrationTests based on my prior experience on large projects "trying" to implement full end to end testing.
I've been using this and and have a lot of useful extensions for general selenium, angularjs, and kendo ui work. Since this framework is not obtrusive you could just use these extensions without using anything else.
I'm using this on my latest project and everyone is loving it.
There are a lot of bdd/spec frameworks (specflow, mspec, nspec, storyq) to help wire the behavior of your system to tests.
What I've learned:
make it frictionless for any .net developer/tester to begin writing/running tests.
Most fail here because it requires installing additional pluggins into visual studio.
mine uses the standard nunit
Logically you would think that a feature is a class file and scenarios are [Test] methods - to support some of these frameworks they make each scenario a class file.
use the original spec to create stubs of your tests - hopefully readable code
I used spec flow back in 2010 - so things might have changed. I generated my tests from my bdd document. A year later when I went to add more tests and update existing tests, I felt I wasted a lot of time with ceremony than writing code I really wanted - I stopped using it.
My approach uses t4 to generate stubs - developer has a choice to generate from feature file, for a specific scenario or don't use generated code at all.
how is state shared across steps / nested steps
most use dictionary<string,object> to help you separate data from being hardcoded in your tests accessed from a context object.
mine uses viewmodels and pointers to those viewmodels - if your using something like angularjs you are using viewmodels in your server side display/editor templates and in angularjs controller so why not reuse these in your tests!
start early with CI - make development transparent
My project has ResultDiff that given the nunit testresult.xml file, folder location to your gherkin feature files, and output json file; Read description on why this is important on the screenshot: https://github.com/leblancmeneses/RobustHaven.IntegrationTests#step-5-ci-setup-resultdiff
Example:
Modified means business and developers have a mismatch of Gherkin statements - did something change that we need to talk about?
What is missing? a dashboard to render the .json file created by ResultDiff. It's on my backlog.....
With a centralized dashboard that supports multiple environments(branches of your code) this dashboard will serve all stakeholders (business, developers) what is the status of features being developed.
There is a framework named "omelet" which is built in java on top of testng for selenium,
For cross browser multi-parallel testing , it easily blends with your CI tools and have some cool reporting features with step level reports
Running your test cases on BrowserStack and Grid was never so easy as with omelet with few config changes.
if you want to give it a try then do follow the 5 min tutorial available on the website, there is archetype available on maven central + there are many more features available
Stable version is 1.0.4 and we are currently looking for people to contribute to project.
Documentation over here
Github link
is there any tool out there that i can used to set-up run automatically and i was goggling and i found selenium test runner? there are so many tools out there its hard to figured out which is best
I'm using C# and using MSTest as a test framework and I'm looking forward to see if I can get a way from testing in MSTEST
any help?
This is very subjective question. Every requirement will have its own correct answer. Anyhow I will try to address few requirements and will be updating as I learn more.
If you are automating web app browser tests (sans flash player and silverlight) I would say that selenium is the way to go. There are ways to automate flash and silverlight too, but that is answer for another question.
Selenium is anyways an automation too and your choice will rather is of which test framework to select. So here are few options:
1. Integrating with CI tools:
If you want to organize your tests as segregated atomic units and want them to be integrated to some CI server (e.g. TeamCity). I will recommend using NUnit to run your selenium tests.
2. Behavioral Tests
It is a new trend in the software development and how we test our products. Using behavioral (i.e. business specification) like language. In my experience it is also a very good format to write up acceptance tests. You can use selenium with something like Nbehave or SpecFlow
3. Centralize Test management and Execution
Now this might not fit for everyone but I have found FitNesse (and its c# binding) to be very useful in maintaining and executing selenium test cases.
Please note this answer may not be right and is certainly not complete given the scope of the question. I have nevertheless tried provide few pointers.
Hi I have a Swing application to test and I found Sikuli a nice tool to do it, but I am a little worried about the size of the community and if it's being continually developed and it's being used by other companies.
Do you use it?
For what?
Is it stable?
Is it the best tool for the job you needed?
I use it in my company, too.
It can be used quite easily for not too complex gui-tests.
Sikuli was not developed for the last year but development is now increasing again.
Questions in the Sikuli-FAQ section on launchpad are answered fast although the community is not that big.
In my company, Sikuli is used for gui testing which was previously done by human testers.
It saves some time but not everything is automatable with Sikuli, e.g. the OCR functionality is not dependable (but will be updated from tesseract 2.04 to 3 in the near future).
For my job it was the best tool because it is the only open source (=free) tool I found that provides screenshot based automation that can be integrated with other systems like CI-systems and is programmable with Java and Python which makes easy unit testing possible with JUnit or PyUnit.
Hope I could help.
Yes we use it in-house for testing. It is actively supported. I have reported bugs in Sikuli and have had tickets and workarounds suggested within days with the bugs fixed in the next revision.
It is quite stable. The problems I have encountered typically come from not specifying images correctly and the program selecting an incorrect area of the screen.
One of our more unique uses was creating a set of automated bench tests for a legacy embedded system. The system was written in assembly and had no unit testing capabilities. It communicated with a custom legacy PC application. Rather than try to locate the PC source code, reverse engineer the design, and then write some meaningful bench tests, we created a number of Sikuli scripts to interface with the PC app. It saved weeks of development.
Yes, we use it for automating GUI tests. It's used mostly for old systems that were developed with no test driven back end. (ie: no testing api)
We tests some very complex tools including a debugger using Sikuli.
We tend not to use the Sikuli IDE though.
I have been working on Test Automation from last few months and have been using the tool named "Testcomplete". But I have noticed that the tools do not matter a lot in the field of automation. Only thing you expect from an automation tool is the ability of the tool to spit out the recognition strings for the different controls used in the test application.
Apart from this, you will always have to build a automation framework which will serve your needs writing code.
So my question is, Is my thinking that automation tools do not matter a lot in the field of automation correct? In the sense, you can use any tools to get your automation running. Or Do the tools really matter? (Please ignore the costs factor of the tools). Also if I need to learn a new automation tool, then what do I concentrate on? Or how do I go about learing the tool? In short, what exactly does "learning a tool" mean?
My 3 best reasons for choosing which tool to use:
it works. This is important, not all tools work in all scenarios i.e. flash, silver light, adobe air, legacy apps with no automation support, etc.
whole team skills. This includes not only testers, but also developers. Test automation shouldn't be an isolated effort, developers should also collaborate on it. This is far easier when dev and test are using the same language/platform.
price. Doesn't have to be free (but it could), but of course its an important factor.
Personally we use the same test runner as the one for the unit tests. That along with extra third party automation pieces that do the plumbing for you.
Some additional thoughts on why the tool is important:
Community - What's the user community like? Are there a lot of user-generated resources out there to help?
Support - (if vendor) What's customer support like? Do they fix problems quickly? Is it easy to find solutions to common problems?
Extensibility - Often in test automation, you'll need to roll your own or code work-arounds, if the tool does not support a particular type of object in your application. How easy is it to extend the product? What programming language does the tool use? What kind of support do you get from the IDE?
An other piece of advice: sometimes you'll need wrapper classes around certain frameworks. We were using WatiN, which was really good at its time, but it lacked Chrome (it had a small percentage that time) support. The thing that killed WatiN for us was the lack of coping with new FireFox releases: FireFox 8 was out, and we had to run on our tests on FireFox 3.6...
Selenium was the solution, but it had a totally different logic and we already had more than a 1000 tests.
So we had to create a wrapper class around Selenium to "fake" it was WatiN. We had some issues, but we had to rewrite only some special cases... And not all tests.
The point is, sometimes, support for frameworks just cease to exist. But with an own framework focusing on what the test actually does instead of how it works would save you in this situation.
Variety of Test Automation Frameworks and Test Automation Tools are already available in the market. Thus, I would not recommend the built your own Test Automation Framework at all.
As far as selection of automation tools is a concern, I would say it does matter on the following basis:
Support: How much level of support you have when you are choosing an automation tool for your project.
Community: How big community is using that tool and how responsive that community is about sharing knowledge?
Pricing: (Proprietary or Open Source) Last but not the least is the pricing of the automation tool that you are planning to introduce in your project.
QA teams' expertise also matters sometimes. For example, in case your QA team does not have a developer or semi-developer skills vs Not-Technical QA Team, etc.
Regarding the Automation framework, there are many automation frameworks also available in the market already, therefore no need to reinvent the wheel. and selection of automation framework mostly depends on your selection of scripting language.
For example, if you choose python as your scripting language, then you have option to choose UnitTest, PyTest etc. as an automation framework.
In case of Java, you have option of JUnit and TestNG as an automation framework.
and so on, the base on your selection of scripting language.
Finally, when it comes to structuring your automation framework, it solely depends on many things as the following:
Your nature of the project
Single product vs multiple products
and many more...
Check an example of multiple product automation project directory structure. https://github.com/pancht/python-selenium-framework
I hope, in some way, I would have helped you out in giving an answer to your question.
Thanks,
Panchdev Singh Chauhan