I need to create e2e test suite on react native project and after some reseach and tests with different solutions, I have choose Appium with Jest to made them.
Now I have to chose betwean Webdriver I/O & WD.js. I search a comparison betwean them online but I don't find anything.
Can you share yours exprerience with them and tell me what is the best and why?
There is a Slant comparison here https://www.slant.co/versus/9647/9649/~webdriverio_vs_wd-js
Which in summary favours WebdriverIO significantly due to simpler syntax, (apparently) broader support for build and testing tools, better documentation, and is used by more/bigger companies.
WD's main pro, is just it's a bit better/simpler than webdriverjs.
The biggest con for WebdriverIO on that review that worried me was: "Must run with WDIO to debug. Tasks written in this beautiful Selenium API can only be debugged using the provided WDIO task runner. You can't set breakpoints within tasks, but you can have WDIO pause the run between Selenium commands."
However, this doesn't seem to be true - I set up Mocha in my IDE as usual and could run and debug single test cases as normal.
Personally I have found more support, documentation and tutorials which use WebdriverIO e.g. in tools like Appium which I am using for React Native. Also "wd" is just a really difficult term to search for!
I failed to use "wd" in a Typescript project because there are no type definitions. I switched to JS, but it's rare to have to do that these days.
If you can get either of them up and running in your project (i.e. prove the tools integration), then the main deciding aspect would be the syntax. In this case I went for WebdriverIO beause I've personally always found the Selenium (Java-esque?) syntax to be weirdly verbose, clunky or even backwards.
Related
I am new to test and during my intership, I had to look for some good tools to automate functional tests.
So I made a lot of searches and decided to use Cucumber, linked with Selenium and SoapUI.
But the fact is that another search had been made before to automate load tests and Gatling (used with Jenkins) had been chosen for that.
Then I am asking to find a solution to gather the 2 solutions but it is quite hard to find any solution about that.
The only information I catch was using Taurus to have a single configuration file.
If you have any information which you think coul help me, I would be very thankful.
The purpose and techniques used by cucumber and gatling are very different so you will probably not want to push interoperation too far.
For example, you plan to use Selenium in your cucumber steps. That would not work well with the performance testing.
If you use jvm version of cucumber, you'll be able to reuse some of the utilities, for example test builders.
is there any tool out there that i can used to set-up run automatically and i was goggling and i found selenium test runner? there are so many tools out there its hard to figured out which is best
I'm using C# and using MSTest as a test framework and I'm looking forward to see if I can get a way from testing in MSTEST
any help?
This is very subjective question. Every requirement will have its own correct answer. Anyhow I will try to address few requirements and will be updating as I learn more.
If you are automating web app browser tests (sans flash player and silverlight) I would say that selenium is the way to go. There are ways to automate flash and silverlight too, but that is answer for another question.
Selenium is anyways an automation too and your choice will rather is of which test framework to select. So here are few options:
1. Integrating with CI tools:
If you want to organize your tests as segregated atomic units and want them to be integrated to some CI server (e.g. TeamCity). I will recommend using NUnit to run your selenium tests.
2. Behavioral Tests
It is a new trend in the software development and how we test our products. Using behavioral (i.e. business specification) like language. In my experience it is also a very good format to write up acceptance tests. You can use selenium with something like Nbehave or SpecFlow
3. Centralize Test management and Execution
Now this might not fit for everyone but I have found FitNesse (and its c# binding) to be very useful in maintaining and executing selenium test cases.
Please note this answer may not be right and is certainly not complete given the scope of the question. I have nevertheless tried provide few pointers.
Does anyone know a good way to mix and merge multiple testing frameworks together so that they can all be run in batch and return an solid overall total of which tests failed for which frameworks and suites/specs.
So lets say my testing setup for a particular project looks like so:
I'm using Rails (Ruby) and using RSpec to test that.
I'm also using Cucumber with my Rails application.
I'm using MochaJS with the Testacular runner for JavaScript testing.
I'm using Jasmine to test for some NodeJS applications that I'm using as well.
Now to test each of these test groups I would have to launch each of their respective frameworks/instances, start and run the "run tests" operation for each, and then tally up the results and figure out which tests failed and which ones didn't.
Does anyone know of a tool that is designed to do this?
You probably need a build automation software to perform all these task together.
Whenever one of your test process fails you'll get a detailed feedback.
As you're developing a ruby application maybe Buildr is the best choice, but you could as well use Ant or Rant...
you can find a complete list of tools is here:
http://en.wikipedia.org/wiki/List_of_build_automation_software
I have been writing tests for my Ruby code for a while, but as a frontend developer I am obviously interested in bring this into the code I write for my frontend code. There is quite a few different options which I have been playing around with:
CasperJS
Capybara & Rspec
Jasmine
Cucumber or just Rspec
What are people using for testing? And further than that what do people test? Just JavaScript? Links? Forms? Hardcoded content?
Any thoughts would be greatly appreciated.
I had the same questions a few months ago and, after talking to many developers and doing a lot of research, this is what I found out. You should unit test your JavaScript, write a small set of UI integration tests and avoid record and playback testing tools. Let me explain that in more detail.
First, consider the test pyramid. This is a interesting analogy created by Mike Cohn that will help you decide which kind of testing you should be doing. At the bottom of the pyramid are the unit tests, which are solid and provide fast feedback. These should be the foundation of your test strategy and thus occupy the largest part of the pyramid. At the top, you have the UI tests. Those are the tests that interact with your UI directly, like Selenium does for example. Although these tests might help you find bugs, they are more expensive and provide very slow feedback. Also, depending on the tool you use, they become very brittle and you will end up spending more time maintaining these tests than writing actual production code. The service layer, in the middle, includes integration tests that do not require an UI. In Rails, for instance, you would test your REST interface directly instead of interacting with the DOM elements.
Now, back to your question. I found out that I could greatly reduce the number of bugs in my project, which is a web application written in Spring Roo (Java) with tons of JavaScript, simply by writing enough unit tests for JS. In my application, there is a lot of logic written in JS and that is the kind of thing that I am testing here. I am not concerned about how the page will actually look or if the animations plays as they should. I test if the modules I write in JS will execute the expected logic, if element classes are correctly assigned and if error conditions are well handled. For these tests, I've been using Jasmine. This is a great tool. It is very easy to learn and has nice mocking capabilities, which are called spies. Jasmine-jQuery adds more great functionality if you are using jQuery. In particular, it allows you to specify fixtures, which are snippets of the HTML code, so you don't have to manually mock the DOM. I have integrated this tool with maven and these tests are part of my CI strategy.
You have to be careful with UI tests, specially if you rely on record/playback tools like Selenium. Since the UI changes often, these tests keep breaking and you will spend a lot of time finding out if the tests really failed or if they are just outdated. Also, they don't add as much value as unit tests. Since they need an integrated environment to run, you will mostly like run them only after you finished developing, when the cost of fixing things is higher.
For smoke/regression tests, however, UI tests are very useful. If you need to automate these, then you should watch out for some dangers. Write your tests, don't record them. Recorded tests usually rely on automatically generated xpaths that break for every little change you do on your code. I believe Cucumber is a good framework for writing these tests and you can use it along with WebDriver to automate the browser interaction. Code thinking about tests. In UI tests, you will have to make elements easier to find so you don't have to rely on complex xpaths. Adding class and id elements where you usually wouldn't will be frequent. Don't write tests for every small corner case. These tests are expensive to write and take too long to run. You should focus on the cases that explore most of your functionality. If you write too many tests at this level you will probably test the same functionality that you have previously tested on your unit tests (supposing you have written them).
In my current project I am using Spock and Geb to write the UI tests. I find these tools amazing. They are written in Groovy, which suits better my Java project.
There are lots of options and tools for that. But their choice depends on whether you have a web UI or it's a desktop app?
Supposing from the tools you've mentioned it's Web UI. I would suggest Selenium (aka WebDriver): http://seleniumhq.org/docs/
There is a variety of languages it supports (Ruby is in the list). It can be run against a variety of browsers, ad it's quite easy to use with lots of tutorials and tips available.
Oh, and it's free, of course :)
I though as this post gets a lot of likes, I would post my answer to my question as I do write lots of tests now and how you test front end has moved on a lot now.
So in terms of FE testing I spent lot of time using karma with Jasmine, although karma will work nicely with other test suites like mocha & qunit. While these are great and karma allows you to interface directly with browsers to run your tests. The downside is as your test suite gets large it can become quite slow.
So recently I have moved to Jest which is much faster and if your writing react app, using enzyme with snap shot testing give you really good coverage. Talking of coverage Jest has Istanbul coverage built in and set up and mocking is really easy simple to use. The downside it doesn't test in browser and it using something called jsdom which is fast, but does have a few nuisances. Personally I don't find this a big deal particularly when I compile my code through webpack/babel which means the cross browser bugs are fairly few and far between, so generally isn't an issue if you manually test anyway (and imo you should).
In terms of working within the rails stack, this much easy now that the webpacker gem is now available and using npm and node is generally much more excepted. I would recommend using nvm to manage your node versions
While this isn't strictly testing, I would also recommend using linting as this also picks up a lot of issues in your code. For JS I use eslint with prettier and scss/css I use stylelint
In terms on what to test, I think as Carlos talks about the test pyramid is still relevant, after all the theory doesn't change, just the tools. I would also add to be practical about tests, I would always test, but to what level and coverage will depend on the project. It is important to manage your time and spending hours/days testing a short lifecycle project. Larger/longer term projects the benefits of a larger test suite is obviously greater.
Anyway I hope that helps people that look at the question.
My project is compatible only with Internet Explorer. I want the test scripts to get generated automatically as it is done in Selenium IDE.
Can i use Selenium RC to test my application? I could not use Selenium IDE as it can be used only with Mozilla Firefox.
seleniumrc works with IE. You can specify the browser and the path to it within the config file.
It can be easily integrated into night builds via ant.
stick to writing the testcase in java.
Selenium RC and Selenium Grid are both really good at running tests against IE. You can see all the browsers that are supported by Selenium here and Selenium is Designed to write for one browser and work in the rest. THere are a few little quirks that wont work in every browser but 99% of the time it will.
Selenium RC works with IE, but is very buggy with IE 6 (to the point of being unusable). Generating the scripts is not trivial and there are many methods of doing it. We have created a Firefox extension that examines objects via introspection to make click recording easy. There are many options out there but your best bet is to write your tests with Firefox/Firebug (or Chrome). They will make object location much simpler and if you are careful the locator strings should still work in IE.
There could be two answer to you question:
Besides Selenium, though it has ample of advantages, I am reading about another tool which uses same API which Selenium use. The only changes in API I have seen so far is it reduces the complexity of functions thus making it more easier and simpler for user who is learning.
The tool is called 'Helium' and it has 50% (and more) less complex functions and code as Selenium has.
The only problem with this tool is it is paid tool for learning purpose and for implementing not-so-big scale project you can use it. But yeah after some time its gonna cost you.
I have implemented some code on Helium. Please let me know , if you face any issue initially or you are thinking to implement it.
Other being, you can use Selenium Builder(http://khyatisehgal.wordpress.com/2014/05/26/selenium-builder-exporting-and-execution/) which is an advanced form of Selenium IDE. It imports your command in different languages and does work more effectively and efficiently as Selenium IDE does(http://khyatisehgal.wordpress.com/2014/05/25/selenium-builder/)
Please let me know , if you have any doubt in any of the tool.
I know Watin is compatible with IE and Firefox. If you want to generate the test code you can use the Watin Test Recorder
This of course is implying that you are using .Net
... Or you could just use the .net bindings that comes along with the latest couple of versions, then you can just run 'em through nUnit.
For ex. Selenium IDE users Katalon Recorder might be a good match. Supports different browsers.