My team is currently starting to develop a web application with vue.js. We are currently discussing the test framework and a team member prefers cypress. The only issue is, that cypress is not cross platform testing framework. We have requirements to support Chrome, Safari, Edge, tablet and mobile devices.
Most of the modern frameworks such as vuejs promise cross platform functionality. And the raised discussion that we had in the team is the need of cross platform testing. Is it still so important in 2018(almost 2019) to do cross browser and cross OS Platform tests? What do you use for testing and how do you test your web application?
Thank you in advance for your answers.
Of cource the alternative is Selenium using selenium wrappers like Protractor or Webdriver.io or Nightwatch.js.
For same reason, we are not using Cypress. We are using Nightwatch.js. Coz in the above list of wrappers it only has everything inbuilt like cypress. For others in the list we have to find the respective npm for test runner, assertion reporting, parallelisation, etc.
Mainly we have choosed it for internal test runner and junit xml reporting for ci which it gives along with selenium wrapping functionality.
Also one more good thing cypress is working on cross browser support. Recently they have progress in firefox browser.
Refer this issue tracker
We have this same dilemma at my organization. I find Cypress so easy to use that I am ok using it for 99% of my test (that does not include my unit or api test). If I need something special for another browser, which is probably going to be minimal, I would then use a selenium wrapper, for me that would be protractor since we mainly do Angular. I hear a lot of argument not to choose cypress due to the issue that it currently only runs in Chrome, I think sometimes it is an excuse for someone to hate it because they don't want to learn something new. I would bet 90% or more of your test cases Cypress will be just fine, where I need something else I do something else. When I look at the testing triangle, I know most of my test should be UNIT test so I don't have enough UI automation test to worry about it. I will say I have ran into this same argument at my job from developers who say why don't use protractor, and they have the freedom to use protractor, but I notice they never get around to using it. They like to argue about what someone else wants to do, but then they don't even use what they say they prefer. I would ask myself what is going to be the cheapest to implement and be the most effective for me. For me that is mainly cypress and then protractor if I have some special case..
Related
I'm running Jest integration tests on Jenkins and I want to integrate them with TestRail in order to automatically put test results to TestRail. In this way I will know easily how many tests are passed/failed?
Does anyone tried that?
I guess you are looking for something like this one https://github.com/zeljkosimic95/Jest-2-Testrail . Although your question is too old but it might help someone else. This is not official plugin but it may help you.
Adding a suggestion here, we should choose the tool after analyzing all the requirements in your software testing services and product. Because there is no official plugin for this except this https://gitlab.com/craydent/jest-testrail (haven't tried). But you still can do this without plugin with this library/code https://github.com/zeljkosimic95/Jest-2-Testrail
You can probably have a look at Agiletestware Pangolin solution which allows you to export results of your tests into TestRail automatically from popular CI systems.
In order to be able to upload test results, Pangolin requires you to create report in JUnit format which can be done by using https://github.com/jest-community/jest-junit
Disclaimer: I'm a developer of Agiletestware Pangolin
Is anyone successfully running automated functional tests against a static Dart web app? If so, what automated tool(s) are you using?
I tried to run automated functional tests using HtmlUnit (a headless browser written in Java). It uses the Mozilla Rhino Javascript engine. But it failed while running tests. Here are two errors I got:
1) java.lang.RuntimeException: com.gargoylesoftware.htmlunit.ScriptException: TypeError: Cannot read property "RQ" from null (http://127.0.0.1/my/hab/settings.dart.js#6369)
2) java.lang.RuntimeException: com.gargoylesoftware.htmlunit.ScriptException: TypeError: Cannot find function postMessage in object [object Window]. (http://127.0.0.1:3030/my/_dev/hab/dart-web/web/hab_list.dart.js#16070)
The authors of HtmlUnit say their Javascript support is good, but not great. Maybe the Javascript generated by dart2js is more than Rhino can handle at this time.
So I am asking if anyone is successfully using any other tools for automated functional tests.
I've heard of phantomjs (a headless Chrome browser I think). Rather than waste a lot of time investigating testing tools that may not work with dart2js generated code, I'd like to find tools that are known to work.
Thank you
I've had good experiences with PhantomJS, though not tested any dart2js code in it.
If you're worried about compatibility, I would test it directly through a browser (eg. something like Selenium). If it's running through a browser, you're in control of what the browser supports (your error suggests their browser does not support web workers), though it is a bit fiddlier to test up.
I'm writing an API using a non-rails framework. I normally use Cucumber for BDD and testing of API's when I'm writing them in RAILS.
Is there a strong reason not to use Cucumber when I'm testing an API written on top of node.js (or any other framework, really)?
Strong technical reasons to not do this might include:
Dependency of Cucumber on Rails
The existence of a better framework for node.js
Some other technical reason I don't know about.
First of all, Cucumber doesn't depend on Rails.
I have been trying out some different javascript testing frameworks to try to work out what I want to use. So far I like mocha with expect.js for unit testing but I haven't found anything for integration testing a node app that I'm happy with. It's possible to do with mocha and supertest, and I've also tried cucumber.js which is getting there but it's not as mature as the Ruby version - so that's what I intend to use.
I can't think of a good reason not to do this, apart from it may be harder to do things like set up fixtures - but that may well be a good thing as it would force your tests to deal only with the public interface of your API - as it was intended to be used.
As said cucumber has nothing to do with rails.
You could perfectly use it to test API's written in any languages.
In case you want to use node.js full stack cucumber.js is a good option.
I'm trying to do set up some Selenium WebTests using PHPUnit, but php is no longer supported by Selenium IDE, does this mean I have to re-write all my tests into php in order to use this method?
I'm trying to set up a continuous integration system, but have never even used one before, I tried using jenkins-php.org but it wasn't very helpful.
The newer releasee Selenium IDE "claims" it does not support PHP,
But you can still convert your IDE to PHP or PHPUnit.
(In Selenium IDE) Option > Options > Enable Experimental Functions.
With this option checked you can switch your format under Options again.
You may still want to review your code.
Hope this helps and it is what you meant by "Selenium IDE does not support PHP".
I was able to find: https://addons.mozilla.org/en-US/firefox/addon/selenium-ide-php-formatters/
It allows output from Selenium to PHP, either PHPUnit format or Selenium Testing.
However after all this playing around it became apparent that the best way was to use htmlSuite, it has the most support when it comes to automation and seems to work out of the box, whereas PHPUnit testing requires lots of tinkering.
After a lot of hacking around it became apparent that the best solution is this tool:
http://www.enjoyxstudy.com/selenium/autoexec/index.en.html
It works out of the box, supports htmlSuite, works with windows and linux, runs most browsers, sends email reports, can get new tests from SVN, integrates with Jenkins well, and works with Selenium tests, its not PHP based tests but it works wonders compared to the other tools I tried out.
Really, so much easier, very user friendly, free and works out the box.
I am wondering what tool(s) do you use for front-end testing...
Currently I am using Selenium RC as tool to test the front-end. I am quite happy with the result as I managed to integrate it with the ms build process etc. The problem with Selenium tests is that they are not always reliable especially if you browse with something else than Firefox.
I am looking for open source alternatives (tools for front-end testing)?
I'd recommend TestPlan which can use Selenium as a backend, or HTMLUnit. It also allows you to do a myriad of other testing. It also works around several of the problems Selenium has, making it a bit easier to user than Selenium directly.
My experience shows, Selenium works the best from everything I tried. Even now I work with Firefox 6 and Selenium IDE works perfectly with it.
Together with selenium, we're also using twill. However, it is because of its speed and it is used only for "quick & fast" tests, I'm afraid not a lot of things are better then selenium out there.
On the other hand, I find Selenium quite reliable, even in another browsers - it's just hard to build tests in such way, to think about race conditions etc.
Have you heard of Watin ?
There is a great tool named cypress.
With cypress it is possible to write
End-to-end tests, Integration tests & Unit tests.
It is open source.
Learning curve is very low.
https://docs.cypress.io/guides/overview/why-cypress