How to handle a failed test in JBehave / Thucydides / Selenium? - selenium

I'm pretty new to Thucydides / JBehave, but haven't found any solutions posted to this one. This is the first time I've used Thucydides / JBehave, but have used Selenium before.
I have my .story file. The .story file lists 5 scenario's.
Each scenario is implemented in it's own POJO java class.
e.g
public class ManagerBypassLoginSteps
{
#Steps
ManagerSteps managerSteps;
#Given("the manager is not logged in")
public void mangerLogsIn()
{
managerSteps.start();
}
#When("the manager goes to a different page")
public void managerToDashboardPage()
{
managerSteps.goesToDashboardPage();
}
#Then("they should see the login page")
public void managerShouldSeeLoginPage()
{
managerSteps.verifyOnLoginPage();
managerSteps.close();
}
}
The ManagerSteps class extends net.thucydides.core.steps.ScenarioSteps, but is otherwise just a POJO. The ManagerSteps calls the page objects - all very normal stuff and as per the examples.
In the scenario above, the test fails as the code displays an error message instead of returning the user to the log in page - that's fine, the security code doesn't meet the specified requirements (yet) But, after the assert inside the ManagerSteps class fails, the test appears to stop. This means I have a browser instance just sitting there until I close it. When run as a series of tests, this means a broswer in the Selenium grid is now tied up.
I need a way to detect a test failure, to call to the page object to close / quit.
I can't find the equivalent of a #Before or #After that will always run, I could find it I could use the #After to close the page object's webDriver.
I am not using Thucydides to manage the WebDriver lifecycle as I couldn't find any examples of this that did not use the ThucydidesRunner.class.
Do I have to use the ThucydidesRunner.class ? My impression from the Thucydides manual for integrating with JBehave (http://thucydides.info/docs/thucydides/_writing_acceptance_tests_with_jbehave.html) didn't suggest it was required.
How does everyone else handle test failures in this situation ?

The real problem I had was closing the browser, after a test failure.
The best way to manage the WebDriver appear to be to use the Thucydides #ManagedDriver annotation to handle the lifecyle.
I ended up using a very simple JUnit runner like so
public class UserLogin extends ThucydidesJUnitStory
{
#Managed
public WebDriver webdriver;
#ManagedPages(defaultUrl = "http://localhost:8080/dc-bus-main-web")
public Pages pages;
}
When this is started, it maps to the user_login.story file, through JBehave's naming convention, then starts normally.

Related

Serenity/jBehave - how to open current Serenity report when test is finished

Using jbehave and serenity for reporting: How to pass code after the report is generated. Annontation #AfterStories is not helping, current one is generated after that. (I am not asking how to open any report, but how to do it to get current report, now I am opening previous one which is the latest in the target folder)
#Managed
private WebDriver driver;
#BeforeStories
.....
#AfterStories
public void openLatestSerenityReport {
myMethod.openLatestSerenityReport();
}
This would be non-trivial, as there is no built-in mechanism to do this. You might be able to use a JVM hook or write your own JUnit runner class to run the Serenity class itself.

Selenium:how to create test evidence report with NUnit

I was wondering how one would create proper test evidence of Selenium UI tests.
I was thinking about screenshots but they do not actually cover everything you have done as it is difficult to determine when to screenshot. (Every click or every wait or every pageload).
Another option I thought about was screen recording, but this makes it rather difficult to work with parallelism as you record the whole screen and not a specific chrome window.
However, you could possibly also take a screenshot every second via the webdriver and turn this into a video. Then one would have to work with a separate thread which can be quite challenging considering the condition you have to provide to stop the thread from taking screenshots. Else the test will run forever.
As I was not able to draw a convincing conclusion based on my own thoughts about creating a test evidence report for UI tests I was hoping someone could explain to me how to properly do this.
I had similar issue and I've introduced in my automation framework ExtentReports + klov server with Testrail as tool for test-management.
I think that nobody would ask of You to show testcases via video or screenshot, but if that is necessary, You can check out several libraries for taking 'video', because this is not actual video, than rather bunch of screenshots that are mashed into one video.
What has actually proven really good investment of time is to take fallen tests cases screenshot and attach it in testcase result (Testrail, Bugzila, Extentreports whatever).
Actualy if using selenium/appium You can check this repo
[https://github.com/groupon/Selenium-Grid-Extras] they make 'video' like mentioned and stored on local hub/node.
But best menthod that has been real good method was report with detail steps of each testcase:
Screenshot testcase with detailed steps and action:
The way I handled the reporting in my application goes in the line of what Kovacic said.
I also used ExtentReports as a way to generate metrics and having a step by step record of what happened.
I created a method reponsible for recording a step ( clicked that, navigated there, asserting that... ) with the option of taking a screenshot if needed, and another one for starting a new test.
Then , it's a matter of calling those methods in the PageObject style testing framework and pretty much having those method called in every action made by your framework.
To better illustrate here are some implementation examples (c#) :
Log a step method
public void LogStep(Status status,string MessageToLog, bool hasScreenshot)
{
//we leave the possibility of taking the screenshot with the step or not
if (hasScreenshot)
{
Test.Log(logstatus, messageToLog)
.AddScreenCaptureFromPath(GetScreenshot());
}
else
Test.Log(logstatus, messageToLog);
}
Screenshot capture method
public static string GetScreenshot()
{
ITakesScreenshot ts;
//Browser.Driver here is the instance of the Driver you want to take screenshots with
ts = (ITakesScreenshot)Browser.Driver;
var screenshot = ts.GetScreenshot();
// Here just input the name you want your screenshot to have, with path
var screenshotPath = ScreenShotFolder + #"\" + _screenshotcount + ".bmp";
screenshot.SaveAsFile(screenshotPath);
// I've introduced a variable to keep track of the screenshot count (optional)
return (ScreenShotFolder.Substring(_reportRoot.Length) +"/"+ _screenshotcount + ".bmp");
}
Example of call in the framework
public void BlockAccount()
{
try
{
_blockAccBtn.Click();
_confirmBtn.Click();
ExtentReportGenerator.LogStep(Status.Info, "Blocking Account");
}
catch (NoSuchElementException)
{
ExtentReportGenerator.LogStep(Status.Fail, "Could not find block button", true);
}
}
A NunitTest using the whole system
[TestCase, Order(1)]
public void CanBlockCard()
{
//Creates a new test in the report
ExtentReportGenerator.Test = ExtentReportGenerator.Extent.CreateTest(GetCurrentMethod());
//Each one of these calls to the framework has logged steps
CashlessPages.CashlessAffiliationsPage.AccessAccount(1, 1);
CashlessPages.CashlessAccountsPage.BlockAccount();
Assert.IsTrue(CashlessPages.CashlessAccountsPage.IsAccBlocked());
}
Example of generated Report
Hope this helps

How do I "recycle" a static class in selenium?

I have been trying to write a Selenium project to test against a Salesforce app. After much trial and numerous error messages popping up intermittently in various places, I think I narrowed it down to something on Saleforces' end. It appears to get hung up after the 10th record. When I tested the next 10 it got hung up at the same spot. I am thinking this may be some sort of DoS defense used by Salesforce. I was also thinking that after every 9 records I would "recycle" the web driver. Is this possible, as it is a static object (following the code by a guru on Pluralsite).
static OpenQA.Selenium.Chrome.ChromeOptions options = new OpenQA.Selenium.Chrome.ChromeOptions();
public static IWebDriver driver = new ChromeDriver(#"C:\mydocs\", options);
But if I close the browser using
driver.Close();
or
driver.Quite();
the entire static class gets hosed. I do not use static classes much. Is there a way to "clear" the class?
Why not just write a method that resets it to a new instance?
public static void resetDriver() {
driver.quit();
driver = new ChromeDriver(#"C:\mydocs\", options);
}

CWebDriverTestCase by SeleniumTestCasr

I am trying Yii Selenium unit testing. How can I manage confirm box in that?
Which method is used to get alert message text?
PHPUnit_Extensions_SeleniumTestCase provides
void assertAlertPresent() Reports an error if no alert is present.
void assertNoAlertPresent() Reports an error if an alert is present.
For managing alerts, you can use methods from Selenium API
http://release.seleniumhq.org/selenium-core/1.0/reference.html
storeAlert()
chooseCancelOnNextConfirmation()
chooseOkOnNextConfirmation()
P.S. Selenium tests are functional, not unit

How to run Selenium system tests without requiring to open the browser?

I have a test method created using Selenium, something similar to this:
[TestFixture]
public class Test_Google
{
IWebDriver driver;
[SetUp]
public void Setup()
{
driver = new InternetExplorerDriver();
}
[TearDown]
public void Teardown()
{
driver.Quit();
}
[Test]
public void TestSearchGoogleForTheAutomatedTester()
{
//Navigate to the site
driver.Navigate().GoToUrl("http://www.google.co.uk");
//Find the Element and create an object so we can use it
IWebElement queryBox = driver.FindElement(By.Name("q"));
//Work with the Element that's on the page
queryBox.SendKeys("The Automated Tester");
queryBox.SendKeys(Keys.ArrowDown);
queryBox.Submit();
//Check that the Title is what we are expecting
Assert.True(driver.Title.IndexOf("The Automated Tester") > -1);
}
}
When the test runs, it opens an IE and carries out its test.
Imagine there are 200 test methods like this spread across multiple test fixtures, which means IE has to be opened and closed many times (as many as test fixtures since 1 browser will be opened per test fixture).
How to run Selenium system tests without requiring to open the browser?
I mean for example I was thinking it might be possible to develop a windows service to run the Selenium tests in the WinForms Web Browser Control, in which case the browser doesn't have to be opened each time and the tests can run automatically and seemlessly. Not sure how to implement this though?
Or is there any other better known way?
Thanks,
No. Selenium is written in JavaScript; it's designed to run in a real browser to test compatibility with a real browser. There are a variety of other tools out there designed to run tests that simulate a browser; you can look into HtmlUnit or Canoo WebTest.
Hope this will help you.
Have you tried XLT?
It doesnt require an open browser at all
http://www.xceptance.com/products/xlt/what-is-xlt.html
You can run all your tests in one browser instance if you like. You just have to pass your webdriver instance to each test. Like having a singelton WebDriver in a static class from where all your testcases can access the WebDriver. Works fine and is usefull if you got a session to keep
driver = new HtmlUnitDriver(true);
java.util.logging.Logger.getLogger("com.gargoylesoftware").setLevel(Level.OFF);