In my company we use Ruby to create a framework for API automation and I have heard we can automate using Postman or SoapUI. So why do we have to create a automation framework when we already have tools for it?
It is like a buying a suit at the clothes shop versus going to a tailor to get a suit that is measured especially to your needs.
Using an existing tool will require less initial setup, you will have access to a lot of commonly needed features, without reinventing the wheel. For instance, in Postman there are available test snippets that you can use with little or no programming knowledge. Tools such as ReadyAPI, Katalon Studio, Robot Framework, SoapUI, etc. usually don't have a too steep of a learning curve, compared to developing a customer automation framework from scratch.
Using tools is fine, especially if you understand how they work in the background and have analysed the testing needs for your particular project. For example, a tool like REST Assured makes writing tests for RESTful webservices very easy, but it's actually very complex in the background.
You would build an inhouse automation framework if after researching the existing solutions, you realize they don't fully provide all that you need. A well designed/architectured framework will be far more customizable than any other tool, although it will require more initial work and maintenance as well.
In terms of using a custom test automation framework your testers will generally have to be more technical, more like SDETs rather than typical testers, but does not always has to be the case - I have seen automation frameworks build by developers and the testers would only write tests inside it by re-using the methods in the framework.
Lastly I would advise you to do some experimentation, try one of the commercial or Open Sources tools for API testing and after doing some testing with it try doing the same with a more hands-on approach, like using Python's Request client Apache Http client for Java, but every language has it's equivalent.
Related
We are in the process of transitioning towards SOA.
Our current goal is to try and ensure that more of the application is developed as "Services" (mainly to improve visibility of capability, re-use and de-risk change). Some of those services will be exposed as web services, but many (and probably the majority) will not, and be used for "internal" use only to help reap some of the benefits of SOA.
For those "internal" services we are currently intending on implementing them as OSGi bundles; however we are struggling to understand how best to test them. Our goal is to enable the current System Test team to test all types of services and we have been investigating tools like SoapUI and SOA Test; however it's becoming clearer that we may face some challenges in testing our services implemented as OSGi bundles using tools like these; and indeed asking the test team to do so.
So we're looking for some advice on how best to test aspects of our capability designed to act as a "service", but implemented as an OSGi bundle instead of a web service.
What tools would people recommend, and is this a type of testing that's traditionally done by a developer during unit test, or can it be done by a less technical tester, undertaking the same basic principles of testing interfaces (i.e. inputs, processing, outputs)?
You could theoretically use a Remote Service Admin implementation (like Aries RSA or Eclipse ECF) to expose your internal services to the outside during testing to access them using an external system test tool.
I would not recommend to let an external team test your OSGi services though. It is much better to test the services in your own build using an integration testing tool like pax exam. It allows to define which bundles and other config to install. Then it boots up an OSGi framework with your setup and runs modified junit tests against it. The advantage is that such tests are quite realistic and still quite simple.
See here for some pax exam tests in aries rsa or apache karaf.
The first example uses the pax exam forked container for a very fast test (<1s per Test) while the second example uses the apache karaf container (~10s per Test) for tests that are very near a production system.
So you get much faster feedback than with an external system test team that will always lag a bit behind your current development. It also allows you to establish the policy that each team member runs the tests locally before committing.
If I am that stupid to ask this question, please excuse me.
Anyway, I have searched and found out some notes about the tool which I have given below.
SoapUI is a complete and automated testing solution. In a single test
environment, it provides industry-leading technologies and standards
support, from SOAP- and REST-based Web services, to JMS enterprise
messaging layers, databases, Rich Internet Applications, and much
more.
From the above content, whether we can assume that it will be well-suited for web-based automation testing perfectly? It would be good if you could share some relevant info on the same.
Thanks :)-
I would question as to why you want to use soapUI for web UI testing? For web UI testing i would recommend using some other tool like selenium.
As clarified in my comment below, it would seem that you can make soapUI and Selenium work together. See http://learnsoapui.wordpress.com/2012/09/27/selenium-soapui-can-they-work-together/ for details.
We have written clients in different programming languages (Java, .NET/Silverlight, Flash, Javascript) that communicate with a server, as our target is to support various technologies on client side. The functionality they are supposed to perform is the same.
One of the main challenges we are having now is finding a simple and effective approach for testing this variety of client technologies against the server. Currently we use maven, hooked with many maven plugins such as JSTestDriver, Flexmojo, NPanday and others which we have developed by our own to do this. Is there any better approach?
Any help would be appreciated, whether it is recommendation for available frameworks/tools or innovative ideas to do this.
Thanks
What you need is a clean design, otherwise everything is a mess and you have to test everything together.
Your server should have an interface with other systems (Browsers, desktop applications, mobile apps) and then test thoroughly this API. You can do that by using the appropriate framework, depending on technology used for the server. This should be your main test effort and then try to keep API stable, so that for every new version of the server you just run a regression test.
Meanwhile you can test the client applications alone by creating a mock server that uses the same API.
Last one would be your integration test where you run a live version of your server and your client application and you run integration tests.
expect is a good framework for testing program-external text interfaces such as client-server interaction. It operates with tests formulated in Tcl on a purely black-box logic level.
Recently I was asked by someone if there is such a magic framework that will allow one let's say to design and build once a single library of controls and then use them separately to build web and desktop applications.
Does Google, Microsoft or other company have such a RAD framework and tools?
Thx
Depends on how you define a Web App. If you consider RIAs to be Web Apps, then yes...
Microsoft has Silverlight and Adobe has Flex/AIR.
Well, you can always go and look for adobe flex, adobe flash or even microsoft silverlight, which in the next version will support desktop applications as well.
Other types of frameworks currently do not build so well under desktop/web environments.. Take .NET for example, which can be used in so many set of environments, but there are limitations if you want to use it for mobile, web, desktop or XNA, everyone has his own set of tools.
That is mostly resumed in the capability of the desired environment, since you can browse a web page in an iphone, which does not have the same capabilities of a desktop or even laptop PC.
Seva is telling you how this is normally dealt with. MVC is a good start.
The Eclipse foundation offers Rich Client Platform vs Rich Ajax Platform - one allows you to build desktop apps, the other web-based apps, all using SWT concepts. I wouldn't describe it as magical though - the reality is it is much more difficult and I would not recommend RAP.
Adobe has AIR, and Google wants everybody write HTML5+JavaScript. both Chrome and FireFox can run those as standalone apps.
Were such thing to exist, it would produce lousy desktop apps and lousy Web apps. The underlying platforms and ideologies are too different. You'll do better by isolating as much of business logic away as possible, then building two separate clients.
EDIT: assuming that by a "Web app" you mean an HTML-based app, as opposed to something that happens to execute within a browser. For the latter option, you can have Flash, Silverlight, Java Applets, ActiveX controls... And for the vice versa, you can have a desktop app that opens a Web browser control and runs a Web app in it. :)
Microsoft has a Smart Client Factory that has a lot of built-in guidance packages that make development fairly quick and standardized.
http://msdn.microsoft.com/en-us/library/aa480482.aspx
It is entirely possible that I misunderstand this question, but if all you need, is the ability to seperate models an business logic into some kind of reusable library, just about any programming platform/language will do.
Java: keep models and business logic in seperate project, compiled to jar
.NET: keep models and business logic in seperate project, compiled to DLL
Python, Ruby, Perl: keep models and business logic in separate directory, and include as module
What are the requirements for the desktop application? On which platforms should it run? If Windows only, .NET seems like an obvious choice, otherwise Python and Qt or Java if you like to Swing.
I also would go for a Webservice and implement as much Business-Logic as possible in the Service. You can then build a very small Clinet in Html/JS, Java, .Net or Whatever.
You should choose that Framework you feel most familiar with.
If you are a .Net-Developer you can develop a WCF-Service (is also great with WP7).
For Java, i would choose Google-App-Engine or the Play-Framework.
And of course you can make a PHP-Webservice, for examble with the Flow-Framework
Check CrossUI RAD tool. It enables developers to rapidly develop and package the exactly same code and UI into Web Apps, Native Desktop Apps(Windows, OS X, Linux and UNIX) as well as Mobile Apps.
I am looking form a test automation frameowrk which can work in Clinet/Server architecture (distributed env) and support a mix of operating systems like Windows, unix/linux and Mac
I am currently evaluating and looking for only OpenSource solutions, I found just one framework
STAF , any help on pointing out other similar framework would be of great help
This is not for a web based application , but for a application using CLI & GUI interface & no proper API layer :(
STAF is probably the best option for this sort of thing. It has really good cross-platform support, thorough documentation, and has been pretty stable for years. It's also extensible if you need to customize the services for your own needs.
STAF does seem to be the standard. However if you have something which will handle the distribution for you then it is easy to hook up results gathering. I might suggest starting with hudson which is actually a distributed build tool with some support for testing through the plugins.
The Robot Framework has a remote testing library that uses XML-RPC to send commands from the testing framework to remote servers. It has sample remote servers in python and ruby.