I need to test web API functions in REST format. Currently using Selenium RC for functionally testing the website. And using XmlUnit and HttpClient to functionally test our REST API. But it seems life would be easier if we could really separate our functional testing code into all selenium. Particularly with Selenium 2.0's WebDriver, it looks easier than ever to test XML responses using XPath.
My thoughts in the Pro XmlUnit+HttpClient camp:
allows easily testing with direct database calls or Spring beans
more easily allows testing JSON responses when/if we support that in the future
selenium was meant for web UI, not REST API testing
My thoughts in the Pro Selenium WebDriver camp:
separates out the functional testing all into standard selenium tests
easily test results by using our other selenium UI tests, further testing the UI
allows others than core developers to review functional API tests
maybe easier to use services like BrowserMob, which uses selenium tests, to load test API?
hopefully quicker functional tests with selenium framework
We're using Spring 3 and hibernate. What's best for functionally testing our API?
Another option might be to use REST Assured, a Java DSL for testing REST services. It allows you to write unit style tests with little boilerplate code for both XML and JSON. Additionally, it provides more advanced features like authentication, XSD / DTD validation, response codes, cookies, etc. More information is available at their usage page.
Disclaimer: REST Assured is an open source project initiated by a colleague of mine.
If you want to do real functional testing on rest API's, then I can recommend Fitnesse with this fixture: https://github.com/smartrics/RestFixture
Selenium is just a poor choice for this kind of thing.
As for performance testing you can knock up rest API tests in Jmeter in a matter of minutes, no need to pay a 3rd party.
Testing an API does not sound like functional testing but more like unit-testing. So I'd go for XmlUnit+HttpClient
Related
Trying to select between the Karate framework and Rest Assured for API automation which includes Rest APIs and GraphQL. Also we can enhance the framework to execute the UI automation
I have gone through the general process of finding:
Karate is designed for GraphQL: https://stackoverflow.com/a/47469363/143475
For a comparison of Karate and REST-assured: https://github.com/intuit/karate#comparison-with-rest-assured
UI automation: https://github.com/intuit/karate/tree/master/karate-core
For UI automation, can it be done in the hybrid framework with Rest Assured.
Need to understand what will be the result in long term handling of both the frameworks
Moved the discussion to QA group unless it is ok to discuss here.
https://sqa.stackexchange.com/questions/44538/selecting-between-karate-and-rest-assured
Karate is designed for GraphQL: https://stackoverflow.com/a/47469363/143475
For a comparison of Karate and REST-assured: https://github.com/intuit/karate#comparison-with-rest-assured
And it already supports UI automation: https://github.com/intuit/karate/tree/master/karate-core
I have been going through many tool, but not able to conclude.
I have been using HP UFT for sometime and it's quite OK for my project but for some reason we need to find alternative to HPUFT with some advantage.
I have been exploring tool like selenium, IBM tools.soap ui
Could anyone suggest me test automation tool with advantages , so that I can explore.
My scenario,
More REST API based validation.
Need to make REST call and parse the response for validation.
From my experience, it all depends on if you are comfortable coding then you can use REST ASSURED if you are not too comfortable with coding then there are tools like POSTMAN and Ready API (SOAPUI) Which you can automate with very well which comes with loads of assertions.
Since you are testing REST API and validating result, I would not any testing related to UI here. So Selenium ruled out if there is no web UI involved.
For rest API I would recommend you can use Java with Junit/TestNG. Where you write keywords in Java and use the same ones to write Junit/TestNG tests.
Another alternative could be RobotFramework, for which u can have keywords in python.
is there any tool out there that i can used to set-up run automatically and i was goggling and i found selenium test runner? there are so many tools out there its hard to figured out which is best
I'm using C# and using MSTest as a test framework and I'm looking forward to see if I can get a way from testing in MSTEST
any help?
This is very subjective question. Every requirement will have its own correct answer. Anyhow I will try to address few requirements and will be updating as I learn more.
If you are automating web app browser tests (sans flash player and silverlight) I would say that selenium is the way to go. There are ways to automate flash and silverlight too, but that is answer for another question.
Selenium is anyways an automation too and your choice will rather is of which test framework to select. So here are few options:
1. Integrating with CI tools:
If you want to organize your tests as segregated atomic units and want them to be integrated to some CI server (e.g. TeamCity). I will recommend using NUnit to run your selenium tests.
2. Behavioral Tests
It is a new trend in the software development and how we test our products. Using behavioral (i.e. business specification) like language. In my experience it is also a very good format to write up acceptance tests. You can use selenium with something like Nbehave or SpecFlow
3. Centralize Test management and Execution
Now this might not fit for everyone but I have found FitNesse (and its c# binding) to be very useful in maintaining and executing selenium test cases.
Please note this answer may not be right and is certainly not complete given the scope of the question. I have nevertheless tried provide few pointers.
I will start using JBehave Web to create tests for a web applications, I've been checking and I saw that Selenium now integrates WebDriver api besides the original one from Selenium.
My question is, which one should I use? The application needs to be tested on Firefox and Internet Explorer and has some Ajax functionalities (although nothing too complex)
Also any reference to some good tutorial or book would be greatly appreciated!
The old Selenium (1.0) API is not really being developed any more, it's all about WebDriver now in Selenium 2.0. I recommend it for future-proofing your code.
Use whichever you feel more comfortable with. The WebDriver API has only recently settled down in Selenium 2.0RC3, while the Selenium RC API hasn't been changed at all in at least a year. artbristol is right that the RC API (what he calls 1.0) isn't being developed further, but it's not going away any time soon either.
As to books, there are two very good ones. Both are great beginner’s books for Selenium and a great deal . Both are available in paperback and as e-books. And if you want to peek at them, both authors or their publishers offer partial downloads for free. The e-book prices are very reasonable, and the paperback prices are competitive.
David Burns’ Selenium 1.0 Testing Tools: Beginner’s Guide, free sample at https://www.packtpub.com/sites/default/files/0264OS-Chapter-6-First-Steps-with-Selenium-RC.pdf?utm_source=packtpub&utm_medium=free&utm_campaign=pdf
Alan Richardson’s Selenium Simplified - Automated Web Testing with Java and Selenium RC, free sample at https://docs.google.com/leaf?id=0B2l9DD6OObL9MGQ3NGU4ODYtZmJlYi00MGY1LWIwNDUtZmM2N2UxODU4OWNm&sort=name&layout=list&num=50).
As far as I know, since the two projects (selenium and webdriver) are merged, the underlying infrastructure is the same. So you shouldn't gain speed by using one API instead of another. You should use the API you like most.
My opinion is that the webdriver API is more convenient. It's much more convenient to use objects and not write procedural code. It shortens your xpaths and makes code easier and much more elegant. Plus, it's easier to write page objects when your working constantly with objects.
All the documents I have read so far on various testing frameworks (Selenium, Tellurium, Canoo etc) encourage (or at least concentrate) on functional testing an application through UI. I have seen very less on the below which I believe are most important ones
localization testing
usability testing
cross browser compatibility testing
testing against multiple versions of the same browser
xss/sql injection
validation
Is the UI/presentation layer calling the correct service layer API (service layer being tested thoroughly through component based testing)
So here are my questions
1) What are you specifically automating in UI test automation?
2) Which UI testing framework can meet all the above items?
I have used Selenium in the past and it works great for all of the 1-7 except maybe localization since I did not have to test that. As 2, the applications I have tested were mostly just "write something in textbox and see if it works", so it was quite easy. And, for 4, I had multiple virtual machines with only specific browsers installed. It was hackjob, but, worked, sort of.
Automation projects till now is not that mature to incorporate all the points you mentioned. Majority cases automation is used to test functionality through UI. Also backed up with unit testing to have more coverage on Functional part.
Framework, there will not be any ready to use framework available. But there are tools which has these capabilities, like eggPlant (TestPlant). Generally industry follows is hybrid kind of automation. Like use jmeter to fire number of REST calls and check response. Which will cover client-server calls. And use selenium kind of tool, to check Client UI.
For Cross browser, cross platforms Selenium Grid with junit framework can be formed easily.