How to capture a screenshot with PHPUnit and Selenium2 when the test fails? - testing

I'm using PHPUnit 4.6 and PHPUnit Selenium 1.4.2 with PhantomJS. I want capture a screenshot with the last page when selenium test fails.
In PHPUnit Manual there is a example for Selenium 1, but I'm trying use with Selenium 2, because I need use GhostDriver.
WebTestCase.php
class WebTestCase extends PHPUnit_Extensions_Selenium2TestCase
{
protected $captureScreenshotOnFailure = TRUE;
protected $screenshotPath = '/../../screenshots';
protected $screenshotUrl = 'http://localhost:8080/screenshots';
protected function setUp() {
$this->setBrowser('phantomjs');
$this->setBrowserUrl("http://localhost:8080");
$this->setHost('localhost');
}
}
Test.php
class Test extends WebTestCase
{
public function testTitle()
{
$this->url('');
assertEquals($this->title(), "My App");
}
}
But this not capture a screenshot.
$ vendor/bin/phpunit
PHPUnit 4.6-ge85198b by Sebastian Bergmann and contributors.
Configuration read from /MyApp/phpunit.xml
F
Time: 231 ms, Memory: 5.50Mb
There was 1 failure:
1) Test::testTitle
Failed asserting that two strings are equal.
--- Expected
+++ Actual
## ##
-''
+'My App'
/MyApp/tests/functional/Test.php:7
FAILURES!
Tests: 1, Assertions: 1, Failures: 1.

Hmm. The difference between SeleniumTestCase and Selenium2TestCase is not really good documented in the PHPUnit Manual. Also there is no clear separation and not enough usage examples for common cases on Selenium2.
$captureScreenshotOnFailure does not exist on
PHPUnit_Extensions_Selenium2TestCase.
Anyway, let's try putting this together:
<?php
class Test extends PHPUnit_Extensions_Selenium2TestCase
{
protected function setUp() {
$this->setBrowser('phantomjs');
$this->setBrowserUrl("http://localhost:8080");
$this->setHost('localhost');
}
public function testEnterText()
{
$this->url("/");
try {
$this->assertEquals($this->title(), "My App");
} catch (Exception $e) {
$this->screenshot( __DIR__.'/'.$this->getName().'-'.time(). '.png');
}
}
public function screenshot($file)
{
$filedata = $this->currentScreenshot();
file_put_contents($file, $filedata);
}
}
The try-catch-block: in the try part the assertion is done, if the assertion fails, the exception is caught. The catch-block gives us a chance to (grab details of the exception or re-throw it or) make a screenshot.
The main function is $this->currentScreenshot(), which was used in this test
https://github.com/giorgiosironi/phpunit-selenium/blob/master/Tests/Selenium2TestCaseTest.php#L733
ScreenshotListener
Please note that there is a ScreenshotListener around, which might be worth looking at:
https://github.com/giorgiosironi/phpunit-selenium/blob/master/PHPUnit/Extensions/Selenium2TestCase/ScreenshotListener.php
With usage example over at https://github.com/giorgiosironi/phpunit-selenium/blob/master/Tests/Selenium2TestCase/ScreenshotListenerTest.php
This might be a cleaner implementation to grab test failures and make shots.

Combining the solutions from #Jens A. Koch and #John Joseph, we get this:
<?php
class homepageTest extends PHPUnit_Extensions_Selenium2TestCase {
private $listener;
public function setUp() {
// Your screenshots will be saved in '/var/www/vhosts/screenshots/'
$screenshots_dir = '/var/www/vhosts/screenshots/';
$this->listener = new PHPUnit_Extensions_Selenium2TestCase_ScreenshotListener($screenshots_dir);
$this->setBrowser('firefox');
$this->setBrowserUrl('https://netbeans.org');
}
public function testNetbeansContainsHorses() {
$this->url('https://netbeans.org');
$this->assertContains('Equestrian', $this->title()); // Will fail on NetBeans page.
}
public function onNotSuccessfulTest($e) {
$this->listener->addError($this, $e, microtime(true));
parent::onNotSuccessfulTest($e);
}
}

A way of doing this across all your web tests is to override one of the test failure functions from the parent test case class, and capture your screenshot there.
Example:
class MyBaseWebTests
{
$this->directory = '/some_path_to_put_screenshots_in/';
// Override PHPUnit_Extensions_Selenium2TestCase::onNotSuccessfulTest
public function onNotSuccessfulTest(Exception $e)
{
$filedata = $this->currentScreenshot();
$file = $this->directory . get_class($this) . '.png';
file_put_contents($file, $filedata);
parent::onNotSuccessfulTest($e);
}
}
Now, after any of your web tests fail, they will dump a screenshot in that folder with the name of the web test class as the filename.

Use this to save screenshot..very useful in case of headless browser.
$fp = fopen('path/35.png', 'wb');
fwrite($fp, $this->currentScreenshot());
fclose($fp);
sleep(1);

Related

How to Take Screenshot when TestNG Assert fails?

String Actualvalue= d.findElement(By.xpath("//[#id=\"wrapper\"]/main/div[2]/div/div[1]/div/div[1]/div[2]/div/table/tbody/tr[1]/td[1]/a")).getText();
Assert.assertEquals(Actualvalue, "jumlga");
captureScreen(d, "Fail");
The assert should not be put before your capture screen. Because it will immediately shutdown the test process so your code
captureScreen(d, "Fail");
will be not reachable
This is how i usually do:
boolean result = false;
try {
// do stuff here
result = true;
} catch(Exception_class_Name ex) {
// code to handle error and capture screen shot
captureScreen(d, "Fail");
}
# then using assert
Assert.assertEquals(result, true);
1.
A good solution will be is to use a report framework like allure-reports.
Read here:allure-reports
2.
We don't our tests to be ugly by adding try catch in every test so we will use Listeners which are using an annotations system to "Listen" to our tests and act accordingly.
Example:
public class listeners extends commonOps implements ITestListener {
public void onTestFailure(ITestResult iTestResult) {
System.out.println("------------------ Starting Test: " + iTestResult.getName() + " Failed ------------------");
if (platform.equalsIgnoreCase("web"))
saveScreenshot();
}
}
Please note I only used the relevant method to your question and I suggest you read here:
TestNG Listeners
Now we will want to take a screenshot built in method by allure-reports every time a test fails so will add this method inside our listeners class
Example:
#Attachment(value = "Page Screen-Shot", type = "image/png")
public byte[] saveScreenshot(){
return ((TakesScreenshot)driver).getScreenshotAs(OutputType.BYTES);
}
Test example
#Listeners(listeners.class)
public class myTest extends commonOps {
#Test(description = "Test01: Add numbers and verify")
#Description("Test Description: Using Allure reports annotations")
public void test01_myFirstTest(){
Assert.assertEquals(result, true)
}
}
Note we're using at the beginning of the class an annotation of #Listeners(listeners.class) which allows our listeners to listen to our test, please mind the (listeners.class) can be any class you named your listeners.
The #Description is related to allure-reports and as the code snip suggests you can add additional info about the test.
Finally, our Assert.assertEquals(result, true) will take a screen shot in case the assertion fails because we enabled our listener.class to it.

Infrom Selenium that feature file starts \ ends

I have to make some operations when a Feature file starts or ends.
But I didn't find any way that Selenium can know it.
Meanwhile I use a specific hook tag to catch the beginning and another one to catch the end. But Is there a way to know it in Selenium code?
You can use before and after hook to perform some actions, you can add extra tag to your cucumber scenario and check it by scenario.getSourceTagNames(). see the example below:
#Before
public void setUpScenario(Scenario scenario) {
List<String> tags = scenario.getSourceTagNames();
if (tags.contains(scenario_specific_tag)) {
System.out.println("Before your scenario running ...." );
}
}
#After
public void endUpScenario(Scenario scenario) {
List<String> tags = scenario.getSourceTagNames();
if (tags.contains(scenario_specific_tag)) {
System.out.println("After your scenario ...." );
}
}

Intellij 13 ultimate runs geb tests smoothly but they wait when run independently

I am using cucumber groovy with geb.
Here is my profile and driver
FirefoxProfile firefoxProfile = new FirefoxProfile();
firefoxProfile.setAcceptUntrustedCertificates(true);
firefoxProfile.setAssumeUntrustedCertificateIssuer(false);
firefoxProfile.setPreference("browser.helperApps.neverAsk.saveToDisk",
"text/csv,application/pdf,application/csv,application/vnd.ms-excel");
firefoxProfile.setPreference("browser.download.manager.showAlertOnComplete",false);
firefoxProfile.setPreference("browser.download.manager.showWhenStarting", false);
firefoxProfile.setPreference("browser.download.manager.useWindow", false);
firefoxProfile.setPreference("browser.helperApps.deleteTempFileOnExit", true);
firefoxProfile.setPreference("webdriver.load.strategy", "unstable")
driver = {
def driver = new FirefoxDriver(firefoxProfile)
driver
}
Here is my step definition (FYI, this is the firstmost step)
MyPage.setUrl(Globals.get(key))
to MyPage
waitFor(10,0) {
ExpectedConditions.presenceOfElementLocated(By.tagName("title"))
}
at MyPage
I noticed that if i put a breakpoint at "at MyPage" in intellij and debug then it breaks at that point and then i can resume. However if I simply run either from Intellij or using ./gradlew clean cucumber
then the page loads and waits for a long time. I don't think it ever proceeds (only waited for a minute to check)
Whats the issue here ?
Update 1
class MyPage extends Page{
static url = ""
static at = {
module1.attrib.value() != null
Globals.get(module1.attrib.value())
}
static content = {
module1 { module Module1 } // Simple Geb Module
module2 { fieldsMap -> module Module2, fieldsMap: fieldsMap }
}
}
class Module2 extends Module {
def fieldsMap
static content = {
textField { $("input", name: fieldsMap['textFieldName']) }
}
}
Try doing it in a more gebish way:
Set the url explicitly in a subClass of MyPage - you should really try to use one page object per page.
define an at condition and use it to test for the waitFor condition for the page to wait till its loaded.
By default an at condition will wait for its condition to be true.
and
do not use ExpectedConditions try something more Gebish like:
waitFor { title != "" }
In your at condition you have not properly specified a truth statement. Although I am not sure what Globals.get(module1.attrib.value()) returns? But it looks like you should remove this line and do this logic somewhere else.
Calling value returns a string.
Module 1
class MyPage extends Page{
static url = ""
static at = {
waitFor { module1.attrib.value() != "" }
}
}
MyPage.setUrl(Globals.get(key))
to MyPage
at MyPage
Just nit picking here:
driver = {
new FirefoxDriver(firefoxProfile)
}
Updated version of Selenium and firefox fixed the issue.
Closing this

Executing Specific Geb Tests according to environment

I have a set of Spec tests I am executing within a Grails Project.
I need to execute a certain set of Specs when I am on local, and another set of Spec when I run the pre-prod environment.
My current config is executing all my specs at the same time for both environements, which is something I want to avoid.
I have multiple environments, that I have configured in my GebConfig:
environments {
local {
baseUrl = "http://localhost:8090/myApp/login/auth"
}
pre-prod {
baseUrl = "https://preprod/myApp/login/auth"
}
}
You could use a spock config file.
Create annotations for the two types of tests - #Local and #PreProd, for example in Groovy:
import java.lang.annotation
#Retention(RetentionPolicy.RUNTIME)
#Target([ElementType.TYPE, ElementType.METHOD])
#Inherited
public #interface Local {}
Next step is to annotate your specs accordingly, for example:
#Local
class SpecificationThatRunsLocally extends GebSpec { ... }
Then create a SpockConfig.groovy file next to your GebConfig.groovy file with the following contents:
def gebEnv = System.getProperty("geb.env")
if (gebEnv) {
switch(gebEnv) {
case 'local':
runner { include Local }
break
case 'pre-prod':
runner { include PreProd }
break
}
}
EDIT: It looks like Grails is using it's own test runner which means SpockConfig.groovy is not taken into account when running specifications from Grails. If you need it to work under Grails then the you should use #IgnoreIf/#Require built-in Spock extension annotations.
First create a Closure class with the logic for when a given spec should be enabled. You could put the logic directly as a closure argument to the extension annotations but it can get annoying to copy that bit of code all over the place if you want to annotate a lot of specs.
class Local extends Closure<Boolean> {
public Local() { super(null) }
Boolean doCall() {
System.properties['geb.env'] == 'local'
}
}
class PreProd extends Closure<Boolean> {
public PreProd() { super(null) }
Boolean doCall() {
System.properties['geb.env'] == 'pre-prod'
}
}
And then annotate your specs:
#Requires(Local)
class SpecificationThatRunsLocally extends GebSpec { ... }
#Requires(PreProd)
class SpecificationThatRunsInPreProd extends GebSpec { ... }

Restart failed test case automatically in TestNG/Selenium

I am using Selenium webdriver, in Java with TestNG to run an X amount of test cases.
What I would like, is for any test case to automatically restart (either from starting or from point of failure), as soon as it fails.
I know TestNG framework has the following method
#Override
public void onTestFailure(ITestResult tr) {
log("F");
}
but I do not know how to find out which testcase it was and then how would I restart it.
I wanted to see an example with actual code in it and found it here:
Restarting Test immediately with TestNg
Observe how the below tests will each be re-run once as soon as the failure happens.
import org.testng.Assert;
import org.testng.IRetryAnalyzer;
import org.testng.ITestResult;
import org.testng.annotations.Test;
public class Retry implements IRetryAnalyzer {
private int retryCount = 0;
private int maxRetryCount = 1;
public boolean retry(ITestResult result) {
if (retryCount < maxRetryCount) {
retryCount++;
return true;
}
return false;
}
#Test(retryAnalyzer = Retry.class)
public void testGenX() {
Assert.assertEquals("james", "JamesFail"); // ListenerTest fails
}
#Test(retryAnalyzer = Retry.class)
public void testGenY() {
Assert.assertEquals("hello", "World"); // ListenerTest fails
}
}
From testng.org
Every time tests fail in a suite, TestNG creates a file called testng-failed.xml in the output directory. This XML file contains the necessary information to rerun only these methods that failed, allowing you to quickly reproduce the failures without having to run the entirety of your tests.
If you want to rerun the test exactly after the failure you need to call the method that failed. You can get that method name from ITestResult object.
If you want to rerun all the failed test cases together, then you can give the testng-failed.xml as input xml after the first execution.