Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 8 years ago.
Improve this question
I have so far been a partly QA Automation/partly Web developer person, mostly been working on QTP / VBScript for Automation.
Soon I will be moving on to a new project as Automation Architect.
While I have the overall approach and idea for Automation Framework, that I want to develop, ready in my mind, I need some help on deciding the Automation tool. I would love to use QTP as I would be able to use the full library for low level functions as is from my current project.
However one thing where I feel restrained with QTP is ‘cross browser’ support, I feel only IE can be targeted best with QTP, Firefox/Chrome support is still bit limited. Also, QTP doesn’t allow to test on multiple browser windows at a time. (I maybe wrong here, the latest QTP version maybe supporting this, and if so, I will appreciate if you can share your experience) Our target apps would mostly be Web Application and target Web platforms would be IE/Chrome/Firefox/Safari/Opera.
So I am also thinking to give Selenium or Sahi a try.
Being a Web developer, I can say I am good at JavaScript, so adapting to Selenium should not be a big shift for me.
However before I go ahead with getting my hands dirty with Selenium, I want to hear the thoughts from the expert community here.
What do you feel, how good has been yours ‘cross browser’ automation experience with Selenium /Sahi, or your grievances w.r.to Selenium. Also how much Selenium would be helpful to when it comes to something like the flexibility we get from QTP in terms of opening the door for Automation Developers to VBscripting world – we can extend the scripts using lot of supporting VBScripts , system level scripting, or networking related scripting or IE settings programming , runtime DLL registrations or other batch files, database linking (ADODB), using 3rd party APIs / Web APIs etc, XML programming, using .net libraries etc.
thanks in advance, much appreciate sharing your valuable experience.
This is an ongoing debate. However, I, from my experience, would say SELENIUM. There are of course some advantages of using QTP. However, with the ongoing technology in development world selenium is much better tool. Having the capability of implementing the OOP and taking it's advantage is a big factor here where qtp easily fails. Thinking abount angulars, other ajax technology where selenium is improving day by day and qtp is not. And, think about working in agile team side by side with developers who are building your app with java/c#. You can pretty much share there knowledge and techniques to equally support your project where using qtp your project set up will be whole lot different. And, at any case if you need help from google selenium wins without any doubt.
Related
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 5 years ago.
Improve this question
I have custom API included in a website which creates new UID on new unique user visit like Google Analytics and sends the UID data to the backend server(NodeJs) for computation.
I need to check concurrent users and max the limit of users can be created/handled per current cloud config.
Also, need to check, is there any limit on API creating and sending users data. The API is on CDN(fastly)
Please suggest some testing tools, to check for above scenario.
SoapUI is a kind of standard for web services functional testing, it has also certain load testing capabilities
Web Services are basically JSON or SOAP over HTTP so any tool which supports HTTP protocol will suit. Here you can find the list of free and open source load testing tools. Narrowed down to the most powerful ones it will look like:
Grinder
Gatling
Apache JMeter
Tsung
Check out Open Source Load Testing Tools: Which One Should You Use? article for the main features comparison, sample scripts and reports.
I agree with Dmitry that those four (Grinder/Gatling/Tsung/Jmeter) are good tools, with a lot of functionality, but they are also fairly complex, require dependencies and can be somewhat painful to get started with. It all depends on your requirements which tool is best for you.
It sounds to me like you want to test one or two REST API end points powered by NodeJS. If you want a simple-to-get-started with tool that can be scripted, there are some good command-line tools available:
Wrk - very fast, scriptable in Lua
Artillery - NodeJS-based, scriptable in JS
k6 - our own newly released tool, currently the fastest tool scriptable in JS
There is also Locust which is scriptable in Python, but very low-performing.
I like these tools because they offer simple command-line usage and can be scripted in a real language, as opposed to Jmeter and Tsung, where you'll have to resort to XML if you want to do something slightly out of the ordinary. Gatling is a bit better, offering a DSL based on Scala classes where you can do most things but it is still not "real" Scala. The Grinder is the only one of those other tools that offers true scripting (in Jython), but again, it is not a simple one-line command to get started.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 3 years ago.
Improve this question
We are looking for automation processes and we read about automation anywhere . We do have regular automated functional testing tool. Why cant we use them in production versus buying huge license product such as automation anywhere ? Writing automation in functional testing tool involves development effort but it heavily reduces license cost versus automation anywhere RPA.
Can someone expertise in automation anywhere please explain how RPA is different in these perspective ?
The biggest hype over the RPA technologies is that it is relatively easier to create automation with those tools (they are almost like scratch programming, using diagram flow to create common tasks and processes) so even business people can own the robot and edit when necessary. Of course this depends heavily on the complexity of the process.
Companies nowadays are creating RPA team's to create a repository of objects (think of libraries of code) or API's and then the business units with lower level of programming skills can just develop their specific process using those objects pre-made by the real developers ;)
Automation Anywhere comes with lot of features to handle complex business scenarios such as validating data and entering data into a database test automation tools focus on screen automation to pass test cases while tools like AA user to do automation + implementing desicion making steps
Choose RPA tool for functional testing only when :-
If you need to communicate with multiple types Application Desktop App , services,
SAP,PDF etc.
if you need proper analysis of ROI or business and other Operational Analytics based
on some input like price.
if you need Surface Automation using RDP or citrix.
if you have some repetitive task for large sets of data then you can save your man
effort.
The key difference between automated testing tools and a full service RPA, like Automation Anywhere, is the intention of making this a production level solution. RPA is intended to be scaled to touch various levels in your organization. It is not designed to only be ran attended on one local machine. AA treats created bots just like a well-tuned development department, with built-in version control, scheduling, and even ROI analytics on the use of those bots. If you only have a few processes that you want to automate, then I would suggest investigating the use of automated testing tools that could run on a dedicated vm. But if your organization is wanting to setup and allow for multiple bot creators, scheduling and prioritizing unattended bot runners, all while maintaining appropriate change control oversight then going with a full service RPA solution is probably more appropriate.
As a side note, I am an IT auditor for a public company that has just started using Automation Anywhere. Their product is mature enough and has enough appropriate production level controls that we are comfortable with allowing it to handle financial data in an unattended state. RPA tools like this one are certainly intended for your more intermediate users. The interface is easy for a non-programmer, but advanced users also have available features that make it more powerful. I doubt most automated testing tools are as portable or user friendly.
You can also try out their community edition or watch some of their videos for free.
Automation anywhere and automation testing are two different concept.
Main focus of the automation anywhere is automate the Repetitive tasks.
e.g Download the email attachment and upload to OneDrive.
If you are looking for automate the testing and create test cases for the application with fail or pass results then go for Selenium.
Automation anywhere has cognitive automation, you do an action couple of times like copying the customer name from pdf file to the 3rd column of excel sheet and your bots will learn it to do it automatically for thousands of records. You can't expect it to happen via traditional web or API automation tools. This is just one example, there are many complex business scenarios which can't be done via test automation tools.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
I have been working as a manual tester and documentation (Test plans etc.) personnel in a company's software/IT division. I do not have a big programming background but I want learn programming language(s) and tool(s). Hence, I am looking for automated testing tools and programming language(s) to learn which also give me industrial advantage.
Some points to note are:
I need to know which tools are the most common in the industry and languages associated with those tools.
I do not have a lot of time to learn loads of stuff. Hence, I am looking for tools which use languages that are beneficial to learn in case I want to do more programming.
I'd prefer Web based app testing but that is not a constraint.
You can give 2-3 languages as I understand there cannot be one winner.
I have done some QTP which uses VBScript but VBScript is not broadly used. I want to learn a language that had broader base.
Please still give your inputs and ideas even if this question seems really difficult to answer.
Thanks in advance.
EDIT:
I think so far I have decided that I will go for:
QTP
Selenium
Test Driven Development Methodologies
I just need to figure out a good programming language that gives me a programmer's edge and is suitable for multiple testing tools (including Selenium). Maybe Python, Ruby or Java?
Wow this is quite a widesweeping question. I'd say you were in a good position as the industry moves towards a Test Driven Development (or write your tests before your code) model.
First of all you'll want to know about Unit Testing, Continuous Integration and Web Automation.
I'm going to focus on the areas I know (.NET, Java, Javascript, Build automation, Selenium)
In .NET NUnit is probably the most widely used unit testing framework. It is a port (copy of) JUnit in the java world. Most unit testing frameworks are very similar to these in terms of the concepts. So learn one and it won't take you long to pick up the others.
I think reading the above links and having an idea about these concepts will get you on your way. It's probably worth experimenting with Python or Ruby as these have low barriers to entry to mess around with some simple tests.
For web based testing the most well known are probably Selenium and Watin. These allow you to script browsers to perform actions automatically. However, I have come across very few good usage of these and they are very fiddly (you're talking days/weeks of effort) to get set up and useful. Again the concepts behind these are similar whatever framework you use.
Think I answered 4,5,6 in 1,2,3 :-)
HTH
As for 'industrial advantage' it depends what you mean by that.
For big companies QTP is widely used. Thing to learn with QTP is not really the language, but the tool itself. For me it was more troublesome to overcome tool specifics than learning VBscript. If you would think about performance testing, than PerformanceCenter would be probably the choice. Here you have C as language but again, there is more struggle with a tool than with a language.
In some companies there is stack from IBM, so Rational software comes to play (FunctionalTester, Robot, PerformanceTester). There is also a bit of tool complexity to get through (IBM Redbobks can help).
In smaller companies Java/C# could be beneficial. Here you can get familiar with junit/nunit/TestNG/MSTest. For webapps things like Selenium/Watij/Watin/Fit/Fitness/Concordian are worth to look at. If you have a moment look at AutoIT,PowerShell,Python,Fiddler,Jmeter, Abbot, Watir/Cucumber,Sikuli.
There is also lots of other tools like WebAii, TestComplete, Twist.. some others commercial tools. Usually you can get trials for them to play with it for week or two.
Everything depends on what kinds of project you will be exposed to, and what kind of company it will be.
Regardless, in my opinion it is enough to learn Java/C# to some degree. You don't need to be C#/Java ninja, know every library, and study really hard given language. You see Developing software is more than knowing a programming language, and test automation is more than knowing a tool/language (M. Fewster & D. Graham), so don't get obsessive about it. If you can program script in QTP (not record but program, using Descriptive Programming, decomposing logic to libraries, having full control of TestResults via proper methods etc.), than other tools won't be an issue. You may need like week or two to learn basics of given language, but that is all for test automation (for start at least). You need more to have some experience, to know tools, to know concepts, to know pros and cons of different approaches, to have bigger picture in head, than just 'how do I write this in that language' (you have SO for this :D ).
The answer to this question really depends on a couple of things. First, what kind of testing are you doing? Testing web-based applications? Testing libraries/components of a larger program? GUI testing? Testing command-line apps?
I have found the following useful when creating scripted/automated test facilities:
TCL - TCL can interface with the command line as easily as a batch file, but has a powerful language that blows batch out of the water (including GUI capabilities). If you are scripting Windows-based console apps (either to test them or to use them to test something else), I recommend taking the time to lean TCL.
AutoIt - This scripting language lets you automate UI interaction with Windows apps quickly and easily. It is also fairly easy to learn. If you want to take the time and trouble to develop a screen-scraping utility, you can combine it with AutoIt to create some pretty sophisticated test systems.
C - When testing compiled libraries, I have had a surprising degree of success using good 'ol C. I'll write the basic skeleton for a test app and create a scripted utility that will dynamically generate the rest of the code based on what tests need to be run (I give it a library of sample C snippets that it can use for more complicated tasks).
Ruby - When you are doing automated testing, a robust scripting language can go very far. Whether you need to parse test logs, generate reports, or auto-generate test scripts, a scripting language that you know well can save you a lot of time and headache. My language of choice is Ruby, but others at my company prefer other languages such as Perl or Python. Language choice isn't too important, pick one and learn it well.
Depending on what it is that you are testing, you may also have some sort of testing tools built into the code that you are testing. Java code is sometimes written with JUnit tests, Ruby code can use the Test::Unit framework, etc. Here is a list of unit testing frameworks for a wide variety of languages. If the code you are testing was built with unit testing in mind, you will want to learn the language the code was written in so that you can take advantage of these unit tests.
For web based testing you might want to check out fitnesse. Also, an automated test framework built on python is the robot framework.
Selenium is very popular for automated testing at the QA level.
No matter what you do, there will be some learning curve associated with your solution
The point of Selenium is to provide a testing framework for webapps -- you use Selenium to manipulate DOM elements in the browser, no a running version of your application.
Selenium is a client/server model (you run the server somewhere that has access to your QA app instance; you code your tests using the client libraries) that provides clients in a number of languages (java, ruby, etc)
This type of testing is a good idea if you have have dedicated QA resources that can write code. It provides some high level regression testing. However it is not without its drawbacks: (1) tests can take some time to run, (2) you might need to make some changes to your web application dom elements to use the framework easily, (3) changing the web app will require changing your testing code, which can be non trivial.
Your other options is to write unit tests for the application code. Usually the developers do this as part of a test-driven process. So there is TDD for developers and then automated QA testing for QA.
I will also add a small comment
There is a new testing automation platform XML2Selenium
This platform is built on top of classis Selenium framework and provides many useful features
But it is quite easy to use
You don't need to learn any java or ruby - you can write in XML
SMth like that:
<!-- here we have some imports Frame - is like a class, you can extend it with extends -->
<import resource="Demo.a-server/03Registration/framesRegist.xml" name="frames"/>
<import resource="Demo.a-server/01Action/FramesAction.xml" name="framesAct"/>
<import resource="Demo.a-server/03Registration/FramesLogin.xml" name="framesLog"/>
<property resource="Demo.a-server/LogValue.properties" name="logValue"/>
<test name="creatAction" description="create and check action">
<frame extends="frames:logAdmin"/>
<frame extends="framesAct:createAct"/>
// here we upload the image
<upload id="img_id0" resource="image.jpg"/>
// click save button with id=save
<button id="save" />
// make a screen shot
<screenshot/>
// sleep for 3 seconds
<pause time="3000" />
// make a screenshot again
<screenshot/>
// navigate to the given URL:
<navigate to="${logValue.baseUrl}/admin"/>
<frame extends="framesLog:logOutAdmin"/>
<pause time="1000"/>
</test>
Here you can see report with errors:
http://xml2selenium.com/demored/xml2selenium/
And here you can see report with successful state:
http://xml2selenium.com/demogreen/xml2selenium/
PS: It supports testing in All browsers, not only in Firefox as Selenium IDE
Yes you can learn selenium IDE, Its just as an add-on on the firefox browser, using which you can record your tests and play them when required. The limitation of IDE is it can only be used on firefox.
If you are looking to test on multiple browsers, you need Selenium Grid or WebDriver, which requires a knowledge of java. And again you are not needed to be a Java pro to do this, a simple learning will help.
You can search the youtube.com for Selenium Tutorials.
Advantages of Selenium :
Its Open sourse!!
Its easy to learn and impliment..
Supports multiple browsers.
Supports multiple scripting/programming languages.
Flexible to your choice, simple record and playback (IDE), Comple Framework design to your choice by using your favorite programming language.
Supports parellel execution.
Hope this will help you.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 3 years ago.
Improve this question
I believe there has been some up-take of model-driven development / engineering (aka OMGs model driven architecture) in the real-time and embedded software development sectors. What tools and tool vendors have people had experience with?
Google gives me lots of academic papers and a vendor or two (IBMs Rational Rose Technical Developer and VisSim).
Additionally, any information on model format (UML?), target languages/environments for the platform specific models (C? RTOSs?), and testing (logic-based?) would be greatly appreciated.
We have used Enterprise Architect and IBM Software - Rational Rhapsody. We have used the build in code generation engine and our own code generation engine that generate code suitable to Do178B qualification. With Rational Rhapsody we targeted VX Works as well as our own OS. These tools uses UML models. Since it generates code you can do unit testing with what ever you used to tools that provide integration into these tools.
Scade is also a option if you write safety critical software. Some of the other divisions in our company has used this successfully. It is very logic orientated so it is not able to do everything but it can generate up to 70% of the code for some projects. Using a qualified tool eliminates most of the testing. It has an model verification tool and if the model is correct then code is correct. It integrates in requirement and configuration management tools.
For non safety critical development by experienced developers it is difficult to say if using model driven development will provide you with any saving. It is worth trying, as technology matures and more developers get used to model based development, we will see a lot more of this in the embedded environment.
I have used MS Visio for drawings only; no code generation. Just starting to look at Enterprise Architect, and this is looking promising.
Others in our company have used Simulink/Stateflow for design modelling in an automotive environment. Not for auto code generation I think, but for running the model on the PC.
NI LabVIEW is another possibility. We've only used it in a PC-based automated testing system, but it can also be used for model-based design.
Both these systems can generate code, but we don't have much experience with that so far. Even without using code generation, model-based design has several advantages to help the high-level and mid-level design process and design documentation. Code generation is something we could consider in future.
If you want to model a state machine you could do worse than try visualState from IAR Systems (the embedded compiler company).
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
About three years ago I switched from using commercial testing tools to using an open source testing framework (WatiN), and a UI automation framework I developed myself.
Since doing this I think life is much better using these libraries and Visual Studio, than expensive dedicated testing tools with either their own languages, or VBA.
Are open source automated testing tools and frameworks better than commercial products
Or do expensive commercial automated testing tools such as WinRunner, QuickTest Pro, Testpartner etc ... still have a future?
It's hard to make that sort of determination for a generic category like "testing tools". Usually, it's best to evaluate both the commercial solution and the open source solution on a case-by-case basis. From the wording of your question, it sounds like you've found the open source solution a better fit for your needs.
However, there are some points you can use in your decision making process:
Commercial Tool Benefits:
Support - usually, the company is paying people to support the product. In addition, many commercial companies offer support contracts for various levels of support. If you need support in a crunch, commercial support is the way to go.
Open Source Tool Benefits
Price - pretty obvious...it's hard to compete against free
Openness - open source projects tend to adapt open standards more readily than commercial products (a lot of commercial products also adapt open standards, but open source software tends to do so more frequently).
Self-support - If your company allows it, you can fix the bugs you find in the tools yourself. No need to wait for a third party to get around to fixing them.
It's also worth mentioning that a lot of the commercial testing tools are built on their open source counterparts. If that is the case, then you might be better off going with the open source versions, unless you need the added support.
I think you'll find that commercial products and Open Source products tend to have similar feature sets. In other cases, they may solve the problem with completely different approaches. Again, you'll probably want to make the evaluation for every case.
PyWinAuto is a great open source tool to automate thick client or windows based standalone apps.
I got the experience, that tools for developers are the first and best class of software adopted in open-source. So you can see a big amount of great testing-tools in OS. So I think in most common environments you have test-frameworks in OS that work very well. But I do not code in every environment, so maybe you want to say which language/technology you use, and we can mention some good tools (OS or commercial).
All commercial tools have some +s's and -s's.
Not all of them can be considered as an exact fit in an automation environment which an AUT requires.
I have worked with all the big names it all depends on the automation Engineer how well He/She understands the concept and leverages the tool to the fullest.
Last 5 years I came to the conclusion it is high time we divorce these Commercial tools and go after Open Source Why?
I can customize the Open source to my needs.
I know where and what went wrong and at the same time know how to fix it.
I can customize things to the extent required write from planning to the execution to Reporting is in our control as automation engineers so creativity plays a role to implement the needs of the organization and the individuals who are affected by the automation testing.