Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 5 years ago.
Improve this question
I have custom API included in a website which creates new UID on new unique user visit like Google Analytics and sends the UID data to the backend server(NodeJs) for computation.
I need to check concurrent users and max the limit of users can be created/handled per current cloud config.
Also, need to check, is there any limit on API creating and sending users data. The API is on CDN(fastly)
Please suggest some testing tools, to check for above scenario.
SoapUI is a kind of standard for web services functional testing, it has also certain load testing capabilities
Web Services are basically JSON or SOAP over HTTP so any tool which supports HTTP protocol will suit. Here you can find the list of free and open source load testing tools. Narrowed down to the most powerful ones it will look like:
Grinder
Gatling
Apache JMeter
Tsung
Check out Open Source Load Testing Tools: Which One Should You Use? article for the main features comparison, sample scripts and reports.
I agree with Dmitry that those four (Grinder/Gatling/Tsung/Jmeter) are good tools, with a lot of functionality, but they are also fairly complex, require dependencies and can be somewhat painful to get started with. It all depends on your requirements which tool is best for you.
It sounds to me like you want to test one or two REST API end points powered by NodeJS. If you want a simple-to-get-started with tool that can be scripted, there are some good command-line tools available:
Wrk - very fast, scriptable in Lua
Artillery - NodeJS-based, scriptable in JS
k6 - our own newly released tool, currently the fastest tool scriptable in JS
There is also Locust which is scriptable in Python, but very low-performing.
I like these tools because they offer simple command-line usage and can be scripted in a real language, as opposed to Jmeter and Tsung, where you'll have to resort to XML if you want to do something slightly out of the ordinary. Gatling is a bit better, offering a DSL based on Scala classes where you can do most things but it is still not "real" Scala. The Grinder is the only one of those other tools that offers true scripting (in Jython), but again, it is not a simple one-line command to get started.
Related
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 3 years ago.
Improve this question
I want to test a GraphQL API.
For now, I'm using GraphiQL, but I'm looking for any automated tool...
It seems that SOAPUI does not support GraphQL testing.
Any ideas?
Thanks.
You basically have a few options that I've seen:
Apollo's GraphQL Platform. It gives you full-blown telemetry on your individual resolvers, and can integrate with VS Code to let your developers know how expensive their query is in real time. You'll pay for it though.
An observation tool such as HoneyComb or DataDog, also paid.
Write your own. For a simple enough use-case it may make sense, but if you're looking for a rich feature set it probably makes more sense to buy rather than to build.
I am using SoapUI 5.4.0 (community edition) and have no trouble testing GraphQL requests.
Treat them as a Rest request and add a header, e.g. Content-Type: application/graphql
see image for all details.
What, specifically, do you want to test?
We have a number of automated sanity-check tests that we run on every build:
Is the schema valid (according to graphql-js)? This can be surprisingly easy to mess up if your implementation allows for e.g. multiple definitions of the same type name, or any other number of subtle bugs.
Is this a breaking schema change? If so, break the build unless there's a specific git commit message acknowledging and accepting it. With graphql-js this is fairly easy - run the introspection query against current production, run it against the current build, and use the built in findBreakingChanges function.
Note that the graphql-js tests don't mean your server has to be written in JS - ours is written in ReasonML using ocaml-graphql-server, and then on build we use a node test suite to hit it as any other client would.
Finally, beyond that, we have some tests that run queries/mutations for an end-to-end API server test. Overall, this has been quite robust against regressions so far.
And keep in mind that you can simply hit your GraphQL server with any http client, there doesn't have to be GraphQL-awareness in your test suite. I'd recommend this route on top of the sanity checks I mentioned above.
For automated testing there is https://github.com/ohler55/graphql-test-tool/gtt. It's written in go but as a standalone application it can be used with any GraphQL server. We use it for unit testing and CI.
Karate is the only open-source tool to combine API test-automation, mocks and performance-testing into a single, unified framework.
https://github.com/intuit/karate
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 8 years ago.
Improve this question
I have so far been a partly QA Automation/partly Web developer person, mostly been working on QTP / VBScript for Automation.
Soon I will be moving on to a new project as Automation Architect.
While I have the overall approach and idea for Automation Framework, that I want to develop, ready in my mind, I need some help on deciding the Automation tool. I would love to use QTP as I would be able to use the full library for low level functions as is from my current project.
However one thing where I feel restrained with QTP is ‘cross browser’ support, I feel only IE can be targeted best with QTP, Firefox/Chrome support is still bit limited. Also, QTP doesn’t allow to test on multiple browser windows at a time. (I maybe wrong here, the latest QTP version maybe supporting this, and if so, I will appreciate if you can share your experience) Our target apps would mostly be Web Application and target Web platforms would be IE/Chrome/Firefox/Safari/Opera.
So I am also thinking to give Selenium or Sahi a try.
Being a Web developer, I can say I am good at JavaScript, so adapting to Selenium should not be a big shift for me.
However before I go ahead with getting my hands dirty with Selenium, I want to hear the thoughts from the expert community here.
What do you feel, how good has been yours ‘cross browser’ automation experience with Selenium /Sahi, or your grievances w.r.to Selenium. Also how much Selenium would be helpful to when it comes to something like the flexibility we get from QTP in terms of opening the door for Automation Developers to VBscripting world – we can extend the scripts using lot of supporting VBScripts , system level scripting, or networking related scripting or IE settings programming , runtime DLL registrations or other batch files, database linking (ADODB), using 3rd party APIs / Web APIs etc, XML programming, using .net libraries etc.
thanks in advance, much appreciate sharing your valuable experience.
This is an ongoing debate. However, I, from my experience, would say SELENIUM. There are of course some advantages of using QTP. However, with the ongoing technology in development world selenium is much better tool. Having the capability of implementing the OOP and taking it's advantage is a big factor here where qtp easily fails. Thinking abount angulars, other ajax technology where selenium is improving day by day and qtp is not. And, think about working in agile team side by side with developers who are building your app with java/c#. You can pretty much share there knowledge and techniques to equally support your project where using qtp your project set up will be whole lot different. And, at any case if you need help from google selenium wins without any doubt.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a book, tool, software library, tutorial or other off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 8 years ago.
Improve this question
I have to test windows 8.1 app for tablet. App has to be tested for nearly 1500 users. Technical description: Image file between 2-20 mb will be uploaded and downloaded. With increasing number of user upto 1500 I also have to capture the time required when the user load is say 400, 800, 1200 and finally1500.
Please share your valuable suggestion how can I achieve this with minimum effort as my team is running out of time. I have thought about jmeter, soap ui, load runner. If you know any other tool please share your thoughts.
Short answer: go with JMeter with JMeter Plugins.
Soap UI is not a load testing tool, but Load UI is. It has a pretty visual interface which is good for beginners, but not so convinient when you have a lot of tests to run. Last time I tryed it there also were some performance issues in the tool itself.
Load runner is good choice if you have some expertise in it (there are a lot of checkboxes and radio buttons). It has a lot of protocols supported and you could "record" the user session and then replay it like it was more users. Sometimes it works out-of-the-box, sometimes you will then need to tune your test.
If you are familiar with your app, the better way is to define the requests which you will be sending to your server manually because that gives you more control. JMeter has a GUI for that. If you go with JMeter you should consider downloading JMeter Plugins that make load tester's work with the tool much more convinient.
If you want more tools, have a look at:
Gatling tool
Twitter's Iago
Yandex Tank (that's the one I use at work for testing web services under heavy load)
I have to mention also that with all these tools you'll be able to measure only the server's performance and not the frontend. Testing the frontend is the different and distinct task. For example you may apply heavy load with JMeter and try to use your app during that in order to see what kind of user experience would you have when there are a lot of users.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 7 years ago.
Improve this question
Is keyword driven testing something that could be implemented using Selenium? If so, how exactly and where can I learn more about it? A simple example might help me get started :)
Thanks!
The Page Object Model is a way to represent your pages using Selenium 1 or 2/webdriver that might be of interest to you. With proper setup your tests become human readable and within an IDE that supports code completion, simple to write.
I know this isn't quite what you are asking for, but it provides excellent abstraction and makes tests readable and powerful. You can mock your test flow with somewhat plain language and then fill it in later.
U can look into "robotframework". The documentation also available in the wikipage.
It promises to replace the code to keyword.
http://code.google.com/p/robotframework/
Yes. But keyword driven testing is not something particular to Selenium. Selenium is just the tool/framework for interacting with the browser UI elements in an automated fashion. Keyword driven testing frameworks are typically independent from the automation tool. Try googling keyword driven test automation frameworks to get started.
Using TestPlan with Selenium as a backend is a good option for such testing. I have written several scripts which load CSV files, have hand-coded tables, or use automatic generators to drive the testing.
The language in TestPlan is however clear enough that a typical non-programmer can pick it up and work with it. This further alleviates the trouble. For example, the below is a simple script to submit a form.
GotoURL http://mydomain.com/
SubmitForm with
%Params% with
%name% Tom
%age% 45
end
end
Check //p[#class='success']
That goes to a page, submits the form, and ensures that the result has a specific element (XPath is used, but there are other predicates to locate things).
Open2Test is like add-on style keyword driven framework.
It aims to replace from writing test code to keyword.
But... I'm not sure anybody really using it. There are too little info on the web.
http://www.open2test.org/index.html
Take a look at Test Automation Framework which comes with plug-n-play Keyword driven model.
http://menonvarun.github.io/taf/index.html
http://menonvarun.github.io/taf/pages/keyword_model_in_taf.html
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 7 years ago.
Improve this question
Basically, what it says on the tin; I need a JTA implementation useable in a Java SE application, ideally one that doesn't carry too much of a framework burden with it.
I recommend Bitronix. Before using any other transaction manager, I recommend thorough testing. Tests like killing power to various machines during each phase of the transaction. You want transactionality to protect you when failures occur. It is surprising how many transaction managers have failed to correctly implement recovery.
Bitronix does need JNDI, which is typically provided for you in a Java EE container, but they include a simple embedded implementation that will support Bitronix in a Java SE application.
From its webpage:
"With over 20 years of expertise in the area of transaction processing,
JBoss Transactions (JBossTS) is the premier open source transaction manager.
It can be deployed within a range of application servers, containers or run stand-alone. Over the past 20 years it has been used extensively within industry and to drive standards including the OMG and Web Services."
Try Atomikos TransactionsEssentials.
Atomikos is the inventor of "Java EE without Application Server", so if any open source JTA/XA does well in Java SE then it must be TransactionsEssentials: it was designed for Java SE right from the start, and includes JDBC and JMS connectors/pools so you don't need any application server functionality.
Because of its Java SE focus, TransactionsEssentials also has very few dependencies on third-party jars or platforms.
Best
Guy
Disclaimer: I work for Atomikos, but still...
Quote from http://jotm.ow2.org/
JOTM is an open source Transaction Manager implemented in Java.
It supports several transaction models and specifications providing transaction support for clients using a wide range of middleware platforms (J2EE, CORBA, Web Services, OSGi).
JOTM is hosted by the OW2 consortium. The license used by JOTM is BSD-style License.
This implementation is fully functional and mature since it has been used for several years in the JOnAS application server project.
You can try SimpleJTA: -- http://simplejta.sourceforge.net/