How to generate a cypress report which can be understood by a person who has no knowledge on software.
Cypress allows you to generate a report using any supported Mocha reporter, or even specify a custom one. I would suggest that you look at the Cypress reporters documentation. I'd also throw in Mochawesome as a pretty nice reporter that requires little understanding of the software to grasp.
Related
I need to create e2e test suite on react native project and after some reseach and tests with different solutions, I have choose Appium with Jest to made them.
Now I have to chose betwean Webdriver I/O & WD.js. I search a comparison betwean them online but I don't find anything.
Can you share yours exprerience with them and tell me what is the best and why?
There is a Slant comparison here https://www.slant.co/versus/9647/9649/~webdriverio_vs_wd-js
Which in summary favours WebdriverIO significantly due to simpler syntax, (apparently) broader support for build and testing tools, better documentation, and is used by more/bigger companies.
WD's main pro, is just it's a bit better/simpler than webdriverjs.
The biggest con for WebdriverIO on that review that worried me was: "Must run with WDIO to debug. Tasks written in this beautiful Selenium API can only be debugged using the provided WDIO task runner. You can't set breakpoints within tasks, but you can have WDIO pause the run between Selenium commands."
However, this doesn't seem to be true - I set up Mocha in my IDE as usual and could run and debug single test cases as normal.
Personally I have found more support, documentation and tutorials which use WebdriverIO e.g. in tools like Appium which I am using for React Native. Also "wd" is just a really difficult term to search for!
I failed to use "wd" in a Typescript project because there are no type definitions. I switched to JS, but it's rare to have to do that these days.
If you can get either of them up and running in your project (i.e. prove the tools integration), then the main deciding aspect would be the syntax. In this case I went for WebdriverIO beause I've personally always found the Selenium (Java-esque?) syntax to be weirdly verbose, clunky or even backwards.
I have been using Karate for our Java application & its working out pretty good. Some teams in our company have been using python in their Development & QE process. They are interested in karate, but is there a possibility to run python scripts within karate tests?
Atleast 3 different teams have come to us asking for this. Please let me know if anyone is already doing this (or) if there is a possibility to do this. Thanks.
I think you have some pretty good options here.
First, look at this answer. It will be very easy to call any command line program from a Karate test: https://stackoverflow.com/a/51150286/143475
A new development in Karate is that there is a stand-alone JAR (binary) which contains all the capabilities, including parallel running and HTML reports. This is ideal for teams that don't want to set up a Java IDE - only a JRE is sufficient. So you can even invoke Karate from the command-line. You can even edit / debug tests using the Karate UI - but this still needs work and we are looking for contributions.
Please refer the documentation here, and you can try this within a few minutes: https://github.com/intuit/karate/tree/develop/karate-netty#standalone-jar
When creating automated tests with selenium, I thought one would use easier cucumber with selenium or testng with selenium or just junit with selenium although using only junit is not very popular. I have recently found out that you could use cucumber with testng but I don't see what is the gain of doing this. If someone is using both of them together can you tell me why ?
EDIT:
Using Testng over junit has many advantages. My question is if i use cucumber does it still make a difference or not anymore.
P.S I am not trying to start this tool vs this tool war
The answer that you seem to be looking for, is one of interest in what Cucumber, as a tool, adds to existing test frameworks.
The answer:
Cucumber adds an extra level of communication between you (the development team) and the management team. You are able to link test cases to scenarios that are now understandable by the business, which means that everybody is on the same page. You can even use the BDD tool to start talking about behaviours of the feature:
What things should be included?
Do we need more information?
Lets add that to the file, so that we can test that use case later.
Any new functionality added to the feature later?
Need to understand which section has gone wrong quickly, without having to decipher code written by the intern that was in for 2 months in the summer?
Cucumber helps with all of this, and that's just scraping the surface.
TestNG, JUnit, Selenium? You imagine it, you can do it. With Cucumber as your helpful neighbourhood BDD tool, you can pull together your test suite and bolt an abstraction layer on top. The business will now be able to look at the test results. Where tests have failed, they will be able to describe exactly what section has gone wrong to other members of management, without having to go too far into technical details.
If you're wondering whether to use JUnit or TestNG for this, it is most likely to be your choice. Using whatever is the current test tool to bolt cucumber on top of is the best option if you have an existing suite.
Also, make sure you are using the right language for your team. For instance:
Are you introducing a team of manual testers to developing test automation?
Maybe you should use Ruby or JavaScript, as they are easier languages to pick up as a first language
Are you a development team, using cucumber to add an abstraction layer to your unit tests?
Use the language that you are using for development, with the unit test tool that you are using.
Are you developers in test, using cucumber for automating tests for your website?
Use the language that you and your team are most comfortable with, taking the language being used for development over any others that tie with this (based on a team vote).
I think it depends on what are your other tests (unit ones for example) and how you run them.
If your current tests are already using TestNG, then it will be easier to run Cucumber tests with TestNG engine.
At the opposite, if you already have JUnit tests, it could be easier to use JUnit for Cucumber run (but TestNG is able to run JUnit tests, so you can use TestNG in that case too).
And if you have no other tests, so the choice of the test runner will depend on your own taste.
Yes.. I understand your question. Even I had the same doubt as below:
We use selenium for automation testing. Since they don't provide proper reports, we add TestNG to it (and also for other features). But now, we have cucumber, which gives proper reports. So why do we need TestNG?
I realized, though we get proper results with cucumber, TestNG provides us with many other features which cucumber cannot; like setting priority, setting method dependency, timeouts, grouping , etc.
Though cucumber provides a tag feature, it does not provide all the features provided by TestNG. Maybe when cucumber incorporates all those features, we can eliminate TestNG.
I am new to test and during my intership, I had to look for some good tools to automate functional tests.
So I made a lot of searches and decided to use Cucumber, linked with Selenium and SoapUI.
But the fact is that another search had been made before to automate load tests and Gatling (used with Jenkins) had been chosen for that.
Then I am asking to find a solution to gather the 2 solutions but it is quite hard to find any solution about that.
The only information I catch was using Taurus to have a single configuration file.
If you have any information which you think coul help me, I would be very thankful.
The purpose and techniques used by cucumber and gatling are very different so you will probably not want to push interoperation too far.
For example, you plan to use Selenium in your cucumber steps. That would not work well with the performance testing.
If you use jvm version of cucumber, you'll be able to reuse some of the utilities, for example test builders.
My goal is:
Our customers could generate new web-tests.
Our continuous integration server makes a test-environment deployment; it should execute the tests against it
The test could also be run against some other environment.
(Final acceptance tests should be made by the customer, to test fonts etc, but this would be a great pre-acceptance check for our test-environment. Customers could focus on other things than now.)
Usually some property (like text field id) has changed or something and the tests will break in a few weeks. It seems that recorded tests broke often, so it's better to easily record a new one than trying to maintenance and modify an old test.
Now, I found a whole new approach. Maybe recording is not the right way.
How about, if our customers could make use cases in a human readable own language which the machine would understand and compile to web-recording (with Domain Specific Language, DSL).
This is not sci-fi, it has been already made, so read on. :-)
I have tried to use these automatic web testing frameworks:
Visual Studio web test (Customers can't execute)
Selenium (Works only with Firefox, our customers have IE)
WatiN (.NET version of Watir, recorder seems to be a bit buggy)
HP Quick Test Pro (Not easy enough to make new tests)
None of these have provided actually what I need... but Selenium is the closest one.
Our customers speak Finnish, so in the beginning of a software project, in specification phase, user writes a use case like:
Avaa "OmaLomake"
Syötä "Tuomas" kohtaan "nimi"
Paina "Seuraava"
Translation:
Open "MyForm"
Insert "Tuomas" into field "name"
Press "Next"
Now... This is a human-readable use case, but also it can be compiled to automatic web acceptance test. Open, Insert, into field and Press are keywords, others are values.
What kind of DSL tool would be good for this?
Microsoft is making a new DSL-making-tool in their Oslo-project called MGrammar. It means that you can make a custom language to make it easy for non-technical people to work with machines. (The same basic idea that was (and failed) with Cobol and Visual Basic.)
I found that someone has already made this kind of DSL with MGrammar, but it is for Watin, not Selenium:
http://www.codinginstinct.com/2008/11/creating-watin-dsl-using-mgrammar.html
So the continuous integration server process will be:
Fetch a new version from source control (as usual).
Build, run unit tests and analyze the code (as usual).
Make an installation package and tag version in version control (as usual).
Compile use cases to web tests
Run web tests
Accept/Reject the software :-)
Running a web-test in the continuous integration server usually means a lot of configuration work. So, before I try this, I'm curious, what do you think?
Have you used same kind of setup, and what are your experiences? (What exact environment?)
How about DSL, will it have enough power for use cases or will it be another endless development task? Will the customers ever generate the tests?
First of all, Selenium does work with IE and other browsers as well as Firefox; cross browser support is one of its strengths. Here's the list of supported browsers.
However, if you want a human language-based DSL for writing your tests, take a look at Cucumber - the syntax is almost exactly like your example above. Cucumber already has Finnish language support - see the examples at this link.
Fitnesse and Selenium Integration tools such as Selenesse(http://github.com/marisaseal/selenesse) or Fitnium(http://www.magneticreason.com/tools/fitnium/fitnium.html) can also serve your purpose. However, you need to find answer for who will put the element locators in the test cases written by customers. If customers put the locators using the recorders, it may not be possible to maintain. If customers write the steps and a automation tester/developer can put those locators using regex, custom location strategy, this approach may work.
The TestPlan software uses a specialize language for writing tests. It is highly domain specific and works very well in web environments. It supports the Selenium backend so you gain that compatibility, plus it can run without a browser, for even faster tests. I have used it on some fairly large web projects in the type of setup you are looking for.
Your example script might look like this:
GotoURL /SomePage
Click MyForm
SubmitForm with
%Params:name% Tuomos
%Submit% value:Next
end
That's it. It nicely describes what the user wants to do and is a functioning test. You can combine scripts into units and have custom function as well. So if you really wanted you could write the Finish equivalents to the names.