I am using Fitnesse 20130530 to execute a test suite that contains multiple tests. Most of my tests use script tables with SLIM to drive Selenium. I use a Stop Test Exception to stop the execution of a test when one of the method calls raises an exception. Unfortunately, this also stops the execution of the whole suite. Is there a way to just stop the current test and then continue execution with the next test in the suite?
Not in FitNesse itself, but you can build it into your fixtures.
When I had a similar problem I was able to solve it using what we called "fail fast" mode. This was a static variable that could be set to true under certain conditions (typically by an element not found exception or similar).
Our main driver was structured such that we could pass through one spot that could check for that value before calling the browserDriver. This would then skip the broswerDriver calls until the test ended.
The next text would clear the flag and start up again.
You would need to manage the whole process, but it can work.
Related
I have a project where I am running 100 scenarios every day. After the run has complete, through listeners I am updating the pass/fail in an Excel sheet. I want to hear about a solution where, if I am running the test suite again, the passed test cases should be skipped and only failed test cases should run. I dont want to use retry.I tried to use skipException in beforeInvocation listener method but the test case is still executing the passed test case. How can I skip the passed test cases and execute only the failed one through listeners ?
Every time before the start of the scenario, it should go to the listener and check the excel sheet whether the scenario is passed or fail. If passed then the scenario should be skipped.
Any help will be greatly appreciated.
Update: I am able to do it through listeners, with skipException, but in my report it is showing test as failed and not as skipped
When you run bdd tests, qaf generates configuration file with name testng-failed-qas.xml under reports dir. You should use that config file to run only failed scenarios.
I am writing a step definition for a behat in which I need to check if array is empty and if array is not empty print the array and fail the step. To do this I have written following code in step definition.
if (!empty($issues)) {
print_r($issues);
throw new \Exception("Above issues were found for");
}
Currently when exception is shown it stop the execution and does not execute future scenarios.
You cannot fail only a step from the entire scenario.
If any of the steps of a scenario fails then the scenario fails and the rest of the steps from the scenario will not be executed since the rest of the steps should continue the actions from previous ones that failed.
Please recheck the logic of the scenario and review the behat/bdd documentation.
Jeevan, Behat does not stop script execution if one or multiple scenarios fail in a feature file. For example, if one feature file "test.feature" has 10 scenarios, if you have run the command to run the entire feature file as behat features/test.feature, then all the scenarios will run continuously even if Scenario 2 fails.
In the end, you would see the summary as per the screenshot attached.
Behat output formatter to show progress as TAP and fails inline.
https://github.com/drevops/behat-format-progress-fail
I have a small collection of integration tests that utilize selenium in a class. The idea is that these tests run every time there is a merge to the codebase, with the merge proceeding through the pipeline and having a series of tests running against the new code.
The thing is, these selenium tests have to run one at a time. They're using the browser to log into a website, and the account will just log out if more than one person tries to log into the account at once, it'll just log out, and the test will obviously fail, so I need these tests to run one at a time. I've tried using the #NotThreadSafe annotation, doesn't seem to have changed anything, and I've searched through for some sort of switch or parameter that defines how many tests run at once with no luck. These tests are using junit 4.12.
I have a large suite of SpecFlow tests executing against a selenium grid running locally. The grid has a single host configured for max 10 firefox instances. The tests are run from NUnit serially, so I would only expect to require a single session at a time.
However, when approximately half of the test cases have been run, the console window reporting output from the hub starts reporting
INFO: Node host [url] has no free slots
Why?
All the test cases are associated with a TearDown method that closes and disposes the WebDriver, although I haven't verified that absolutely every test gets to this method without failing. I would expect a maximum of one session to be active at once. How can I find out what is stopping the host from recycling those sessions?
edit #1:
I think I've narrowed down the cause of the issue - it is indeed to do with not closing the WebDriver. There are [AfterScenario] attributes on the teardown methods that are meant to do this, but they only match a subset of scenarios as they have parameters on them. Removing the parameter so that the teardown associates with every scenario fixes the session exhaustion (or seems to) but there are some tests that expect to reacquire an existing session, so I'll have to fix them separately.
A bit of background: This test suite was inherited as part of a 'complete' solution and it's been left untouched and never run since delivery. I'm putting it back into service and have had to discover its quirks as I go - I didn't write any of this. I've had brief encounters with both Selenium and SpecFlow but never used the two together.
The issue turned out to be a facepalm-level fail - mostly in the sense that I didn't spot it. Some logging code was trying to write to a file that wasn't there, the thrown exception bypassed the call to Dispose() on the WebDriver, and was then swallowed with no error reporting. Therefore the sessions were hanging around. Removing the logging code fixed the session exhaustion.
Look on the node (remote desktop) and see what is happening on the box. It does sound like your test isn't closing out it's session properly.
I'm using Selenium IDE 2.3.0 to record actions in my web application and create tests.
Before every test I have to clear all cookies, load the main page, log in with a specific user and submit the login form. These ~10 commands are fix and every test case needs them, but I don't want to record or copy them from other tests every time.
Is there a way to configure how "empty" test cases are created?
I know I could create a prepare.html file or something and prepend it to a test suite. But I need to be able to run either a single test or all tests at once, so every test case must include the commands.
Ok I finally came up with a solution that suits me. I wrote custom commands setUpTest and tearDownTest, so I only have to add those two manually to each test.
I used this post to get started:
Adding custom commands to Selenium IDE
Selenium supports object-oriented design. You should create a class that takes those commands that you are referring to and always executes those, in each of the tests that you are executing you could then make a call to that class and the supporting method and then execute it.
A great resource for doing this is here.