How do I detect a test has failed in Selenium html test? - selenium

We have a bank of tests that all start by logging in.
They're recorded by QA so are html tests.
However occasionally something goes wrong and the tests fails. When
that happens the logout at the end of the test doesn't get called, so
the next test tries to login again - using open ./Login
If you're logged out that works fine.
However if you didn't log out because the test fails, that command puts you in a different path and then the rest of the tests in that suite all fail.
How do I tell Selenium to log out if the test fails?
Or how do I tell Selenium if LogOut link is available logout else
continue?

From My point of view i would prefer Following steps
create lib with all test cases. create Suite which will call the required function from libraries. In Suite use following flow
Call login
if login function returns zero call required function to execute.
If the called function returns zero call logout.
::::::::::::::::::::::::::::::::::::::::::::::::
If one of the function returns non zero store it in some variable or array with function name and error.
If want more details let me know.
e.g. if function gives error returns non zero value call errorLogout

You can used below approach:
Approach 1: TestNg annotation.
Approach 2. Use try catch block in catch block call logout function and then throw exception
Let me know if you need more explanation.

Related

Tests retrieved from collection variable - test failures stop subsequent tests from running

I have tests that I want to use in multiple API calls.
Using JavaScript from external files has been an open issue for 6 years now but isn't officially supported (yet). I'm storing tests in collection variables so they can be retrieved in each APIs Tests.
The issue is tests that fail stop execution like a general JS failure.
Tests being stored in collection variables via an API’s Pre-req
In a setup API call I store the shared library of tests via a Pre-request Script. This is working fine.
A "normal" test failure
When a test is coded in an API's Test area, failures don't stop subsequent tests from running.
A failure for a test pulled in from a collection variable
I can pull tests from the collection variable and run them just fine. However when a Chai expectation fails it seems to be treated like a JavaScript failure instead of an test/expectation failure.
The test run fails, subsequent tests for this API don't run nor do other APIs in the collection run.
How can I have tests retrieved from a collection variable run/fail like hard coded tests?
Problem I guess is how you call function from utils
Just utils.payloadIs204, not utils.payloadIs204()
And it works for me
Update reuse with passing parametes.
Tab pre-request
pm.environment.set("abc", function print(text1, text2){
console.log(text1)
console.log(text2)
} + "");
Tab Test
let script = pm.environment.get("abc");
eval((script + "print('name', 'age')"))
The problem was in how I was calling the library function. It works as desired if the library function (with the expectations) is invoked in the anonymous function passed to pm.test(), not be the passed function.

Is there a way to abort a test suite in TestCafe if a test in the middle fails?

I have a series of tests that are dependent on a step in the middle (such as creating an account). The API that I'm using for this is a kind of brittle (which is a separate problem), and sometimes fails. I'd like to be able to just quit the tests in the middle there when that fails, instead of wait for TestCafe to fail the initial assertions for the next few tests that follow. Is there a way to get the test controller to stop, or signify to the fixture that the tests should stop? I immediately thought of Spock's #Stepwise annotation, but I can't find anything like that in the TestCafe docs.
The Stop on First Fail option stops the entire run once a failed test occurred. If I understand your scenario correctly, you could add an assertion for a successful account creation and if it fails, exit the entire run with this option.
CLI Documentation
API Documentation (under Parameters)

SoapUI with Groovy Script calling multiple APIs

I am using SoapUI with Groovy script and running into an issue when calling multiple APIs. In the system I am testing one WSDL/API handles the account registration, and returns an authenticator. I then use that returned authenticator to call a different WSDL/API and verify some information. I am able to call each of these WSDLs/APIs separate but when I put them together in a Groovy Script it doesn't work.
testRunner.runTestStepByName("RegisterUser");
testRunner.runTestStepByName("Property Transfer");
if(props.getPropertyValue("userCreated") == "success"){
testRunner.runTestStepByName("AuthenticateStoreUser");
To explain the first line will run the TestStep "RegisterUser". I then do a "Property Transfer" step which takes a few response values from "RegisterUser" - the first is "Status" to see if it succeeded or failed, second is the "Authenticator". I then do an if statement to check if "RegisterUser" succeeded then attempt to call "AuthenticateStoreUser". At this point everything looks fine. Though when it calls "AuthenticateStoreUser" it shows the thinking bar then fails like a timeout, and if I check the "raw" tab for the request it says
<missing xml data>.
Note, that if I try the "AuthenticateStoreUser" by itself the call works fine. It is only after calling "RegisterUser" in the Groovy Script that it behaves strange. I have tried this with a few different calls and believe it is an issue calling two different APIs.
Has anyone dealt with this scenario, or can provide further direction to what may be happening?
(I would have preferred to simply comment on the question, but I don't have enough rep yet)
Have you checked the Error log tab at the bottom when this occurs? If so, what does it say and is there a stacktrace you could share?

Create screenshot after failed selenium command

The PHPUnit Selenium base class has an option to make a screenshot on failure, which is a huge help in finding out why the test failed. The selenium server, however, returns an error instead of a failure on any error condition other than explicit assert* calls (such us trying to do something with a non-existent element). If I try to make a screenshot after the server reports the error, I get another error saying that the server already discarded the session. Is there any way to change that behavior?
Update: this is because PHPUnit breaks the connection when it receives an error. I was able to change it by some (rather ugly) manipulation of the PHPUnit code.
Make those interactions as test cases.
For example in perl,
If it is written as below and fails due to a non-existent element. the script will error out
$sel->type("email-id","trial\#trial.com");
While if the above step is made as a test case by writing it as follows
$sel->type_ok("email-id","trial\#trial.com");
If there is a non-existent element, the test case will only fail, and the script will continue.
So using TAP (test any protocol) by using the module use Test::More; , if _ok is added after a function, the function return will be used to determine the fate of the test case.
ie. - A return of 'O' means the test Failed
and A return of '1' means the test Passed
It is not the Selenium server but the SeleniumTestCase class for PHPUnit 3.4 which automatically sends a stop command when it detects an error (Driver.php line 921). PHPUnit 3.6 seems handle errors better.
I think you can overwrite method 'travelbox' and make something like this:
public function onNotSuccessfulTest(Exception $e){
file_put_content('/xxx/xxx.jpg', $this->currentScreenshot());
}

Try catch in assert statement

I am doing regression testing using NUnit, WatiN and VB.net. What i am doing is opening an IE page, seleting some data, making a registration and then on view registration page testing the registration by assertion.
I want to ask is it a good way to use try and catch on every assert. i am using it because if some assert fails it will stop executing the rest of the statement and quits without running rest of the tests. Now I have put try and catch on every assert and writting the fail message in log file. Kindly let is it ok to go with this approach or suggest any better one.
Hello Ray
For instance If I am checking for some airline resevation booking. After creating a booking, On view Booking Summary Page I am testing weather it is diplaying cancel booking button or not. For this I am ussing the following code Try Assert.IsTrue(_internetExplorer.Button(Find.ById(New Regex("CBooking"))).Exists) Catch ex As Exception d_logger.LogResultTextFile("Cancel Button doesnot Exist", True, False) End Try I am checking this by running this in a loop for no of bookings created. I want to keep running the test even if in one booking it wont finds the button but keep checking for other bookings. Thats why I am using it. What I want is is iit a good approach to do so or not
This should be the case. If one Assert fails in your test, no other asserts should happen either. The best way is to run your tests, fix the assert that failed and run again.