How to Take Screenshot when TestNG Assert fails? - selenium

String Actualvalue= d.findElement(By.xpath("//[#id=\"wrapper\"]/main/div[2]/div/div[1]/div/div[1]/div[2]/div/table/tbody/tr[1]/td[1]/a")).getText();
Assert.assertEquals(Actualvalue, "jumlga");
captureScreen(d, "Fail");

The assert should not be put before your capture screen. Because it will immediately shutdown the test process so your code
captureScreen(d, "Fail");
will be not reachable
This is how i usually do:
boolean result = false;
try {
// do stuff here
result = true;
} catch(Exception_class_Name ex) {
// code to handle error and capture screen shot
captureScreen(d, "Fail");
}
# then using assert
Assert.assertEquals(result, true);

1.
A good solution will be is to use a report framework like allure-reports.
Read here:allure-reports
2.
We don't our tests to be ugly by adding try catch in every test so we will use Listeners which are using an annotations system to "Listen" to our tests and act accordingly.
Example:
public class listeners extends commonOps implements ITestListener {
public void onTestFailure(ITestResult iTestResult) {
System.out.println("------------------ Starting Test: " + iTestResult.getName() + " Failed ------------------");
if (platform.equalsIgnoreCase("web"))
saveScreenshot();
}
}
Please note I only used the relevant method to your question and I suggest you read here:
TestNG Listeners
Now we will want to take a screenshot built in method by allure-reports every time a test fails so will add this method inside our listeners class
Example:
#Attachment(value = "Page Screen-Shot", type = "image/png")
public byte[] saveScreenshot(){
return ((TakesScreenshot)driver).getScreenshotAs(OutputType.BYTES);
}
Test example
#Listeners(listeners.class)
public class myTest extends commonOps {
#Test(description = "Test01: Add numbers and verify")
#Description("Test Description: Using Allure reports annotations")
public void test01_myFirstTest(){
Assert.assertEquals(result, true)
}
}
Note we're using at the beginning of the class an annotation of #Listeners(listeners.class) which allows our listeners to listen to our test, please mind the (listeners.class) can be any class you named your listeners.
The #Description is related to allure-reports and as the code snip suggests you can add additional info about the test.
Finally, our Assert.assertEquals(result, true) will take a screen shot in case the assertion fails because we enabled our listener.class to it.

Related

How to persist/read-back Run Configuration parameters in Intellij plugin

I'm making a basic IntelliJ plugin that lets a user define Run Configuration (following the tutorial at [1]), and use said Run Configurations to execute the file open in the editor on a remote server.
My Run Configuration is simple (3 text fields), and I have it all working, however, after editing the Run Configuration, and click "Apply" or "OK" after changing values, the entered values are lost.
What is the correct way to persist and read-back values (both when the Run Configuration is re-opened as well as when the Run Configuration's Runner invoked)? It looks like I could try to create a custom persistence using [2], however, it seems like the Plugin framework should have a way to handle this already or at least hooks for when Apply/OK is pressed.
[1] https://www.jetbrains.org/intellij/sdk/docs/tutorials/run_configurations.html
[2] https://www.jetbrains.org/intellij/sdk/docs/basics/persisting_state_of_components.html
Hopefully, this post is a bit more clear to those new to IntelliJ plugin development and illustrates how persisting/loading Run Configurations can be achieved. Please read through the code comments as this is where much of the explanation takes place.
Also now that SettingsEditorImpl is my custom implementation of the SettingsEditor abstract class, and likewise, RunConfigurationImpl is my custom implementation of the RunConfigiration abstract class.
The first thing to do is to expose the form fields via custom getters on your SettingsEditorImpl (ie. getHost())
public class SettingsEditorImpl extends SettingsEditor<RunConfigurationImpl> {
private JPanel configurationPanel; // This is the outer-most JPanel
private JTextField hostJTextField;
public SettingsEditorImpl() {
super();
}
#NotNull
#Override
protected JComponent createEditor() {
return configurationPanel;
}
/* Gets the Form fields value */
private String getHost() {
return hostJTextField.getText();
}
/* Copy value FROM your custom runConfiguration back INTO the Form UI; This is to load previously saved values into the Form when it's opened. */
#Override
protected void resetEditorFrom(RunConfigurationImpl runConfiguration) {
hostJTextField.setText(StringUtils.defaultIfBlank(runConfiguration.getHost(), RUN_CONFIGURATION_HOST_DEFAULT));
}
/* Sync the value from the Form UI INTO the RunConfiguration which is what the rest of your code will interact with. This requires a way to set this value on your custom RunConfiguration, ie. RunConfigurationImpl##setHost(host) */
#Override
protected void applyEditorTo(RunConfigurationImpl runConfiguration) throws ConfigurationException {
runConfiguration.setHost(getHost());
}
}
So now, the custom SettingsEditor, which backs the Form UI, is set up to Sync field values In and Out of itself. Remember, the custom RunConfiguration is what is going to actually represent this configuration; the SettingsEditor implementation just represents the FORM (a subtle difference, but important).
Now we need a custom RunConfiguration ...
/* Annotate the class with #State and #Storage, which is used to define how this RunConfiguration's data will be persisted/loaded. */
#State(
name = Constants.PLUGIN_NAME,
storages = {#Storage(Constants.PLUGIN_NAME + "__run-configuration.xml")}
)
public class RunConfigurationImpl extends RunConfigurationBase {
// Its good to 'namespace' keys to your component;
public static final String KEY_HOST = Constants.PLUGIN_NAME + ".host";
private String host;
public RunConfigurationImpl(Project project, ConfigurationFactory factory, String name) {
super(project, factory, name);
}
/* Return an instances of the custom SettingsEditor ... see class defined above */
#NotNull
#Override
public SettingsEditor<? extends RunConfiguration> getConfigurationEditor() {
return new SettingsEditorImpl();
}
/* Return null, else we'll get a Startup/Connection tab in our Run Configuration UI in IntelliJ */
#Nullable
#Override
public SettingsEditor<ConfigurationPerRunnerSettings> getRunnerSettingsEditor(ProgramRunner runner) {
return null;
}
/* This is a pretty cool method. Every time SettingsEditor#applyEditorTo() is changed the values in this class, this method is run and can check/validate any fields! If RuntimeConfigurationException is thrown, the exceptions message is shown at the bottom of the Run Configuration UI in IntelliJ! */
#Override
public void checkConfiguration() throws RuntimeConfigurationException {
if (!StringUtils.startsWithAny(getHost(), "http://", "https://")) {
throw new RuntimeConfigurationException("Invalid host");
}
}
#Nullable
#Override
public RunProfileState getState(#NotNull Executor executor, #NotNull ExecutionEnvironment executionEnvironment) throws ExecutionException {
return null;
}
/* This READS any prior persisted configuration from the State/Storage defined by this classes annotations ... see above.
You must manually read and populate the fields using JDOMExternalizerUtil.readField(..).
This method is invoked at the "right time" by the plugin framework. You dont need to call this.
*/
#Override
public void readExternal(Element element) throws InvalidDataException {
super.readExternal(element);
host = JDOMExternalizerUtil.readField(element, KEY_HOST);
}
/* This WRITES/persists configurations TO the State/Storage defined by this classes annotations ... see above.
You must manually read and populate the fields using JDOMExternalizerUtil.writeField(..).
This method is invoked at the "right time" by the plugin framework. You dont need to call this.
*/
#Override
public void writeExternal(Element element) throws WriteExternalException {
super.writeExternal(element);
JDOMExternalizerUtil.writeField(element, KEY_HOST, host);
}
/* This method is what's used by the rest of the plugin code to access the configured 'host' value. The host field (variable) is written by
1. when writeExternal(..) loads a value from a persisted config.
2. when SettingsEditor#applyEditorTo(..) is called when the Form itself changes.
*/
public String getHost() {
return host;
}
/* This method sets the value, and is primarily used by the custom SettingEditor's SettingsEditor#applyEditorTo(..) method call */
public void setHost(String host) {
this.host = host;
}
}
To read these configuration values elsewhere, say for example a custom ProgramRunner, you would do something like:
final RunConfigurationImpl runConfiguration = (RunConfigurationImpl) executionEnvironment.getRunnerAndConfigurationSettings().getConfiguration();
runConfiguration.getHost(); // Returns the configured host value
See com.intellij.execution.configurations.RunConfigurationBase#readExternal as well as com.intellij.execution.configurations.RunConfigurationBase#loadState and com.intellij.execution.configurations.RunConfigurationBase#writeExternal

Infrom Selenium that feature file starts \ ends

I have to make some operations when a Feature file starts or ends.
But I didn't find any way that Selenium can know it.
Meanwhile I use a specific hook tag to catch the beginning and another one to catch the end. But Is there a way to know it in Selenium code?
You can use before and after hook to perform some actions, you can add extra tag to your cucumber scenario and check it by scenario.getSourceTagNames(). see the example below:
#Before
public void setUpScenario(Scenario scenario) {
List<String> tags = scenario.getSourceTagNames();
if (tags.contains(scenario_specific_tag)) {
System.out.println("Before your scenario running ...." );
}
}
#After
public void endUpScenario(Scenario scenario) {
List<String> tags = scenario.getSourceTagNames();
if (tags.contains(scenario_specific_tag)) {
System.out.println("After your scenario ...." );
}
}

How to capture a screenshot with PHPUnit and Selenium2 when the test fails?

I'm using PHPUnit 4.6 and PHPUnit Selenium 1.4.2 with PhantomJS. I want capture a screenshot with the last page when selenium test fails.
In PHPUnit Manual there is a example for Selenium 1, but I'm trying use with Selenium 2, because I need use GhostDriver.
WebTestCase.php
class WebTestCase extends PHPUnit_Extensions_Selenium2TestCase
{
protected $captureScreenshotOnFailure = TRUE;
protected $screenshotPath = '/../../screenshots';
protected $screenshotUrl = 'http://localhost:8080/screenshots';
protected function setUp() {
$this->setBrowser('phantomjs');
$this->setBrowserUrl("http://localhost:8080");
$this->setHost('localhost');
}
}
Test.php
class Test extends WebTestCase
{
public function testTitle()
{
$this->url('');
assertEquals($this->title(), "My App");
}
}
But this not capture a screenshot.
$ vendor/bin/phpunit
PHPUnit 4.6-ge85198b by Sebastian Bergmann and contributors.
Configuration read from /MyApp/phpunit.xml
F
Time: 231 ms, Memory: 5.50Mb
There was 1 failure:
1) Test::testTitle
Failed asserting that two strings are equal.
--- Expected
+++ Actual
## ##
-''
+'My App'
/MyApp/tests/functional/Test.php:7
FAILURES!
Tests: 1, Assertions: 1, Failures: 1.
Hmm. The difference between SeleniumTestCase and Selenium2TestCase is not really good documented in the PHPUnit Manual. Also there is no clear separation and not enough usage examples for common cases on Selenium2.
$captureScreenshotOnFailure does not exist on
PHPUnit_Extensions_Selenium2TestCase.
Anyway, let's try putting this together:
<?php
class Test extends PHPUnit_Extensions_Selenium2TestCase
{
protected function setUp() {
$this->setBrowser('phantomjs');
$this->setBrowserUrl("http://localhost:8080");
$this->setHost('localhost');
}
public function testEnterText()
{
$this->url("/");
try {
$this->assertEquals($this->title(), "My App");
} catch (Exception $e) {
$this->screenshot( __DIR__.'/'.$this->getName().'-'.time(). '.png');
}
}
public function screenshot($file)
{
$filedata = $this->currentScreenshot();
file_put_contents($file, $filedata);
}
}
The try-catch-block: in the try part the assertion is done, if the assertion fails, the exception is caught. The catch-block gives us a chance to (grab details of the exception or re-throw it or) make a screenshot.
The main function is $this->currentScreenshot(), which was used in this test
https://github.com/giorgiosironi/phpunit-selenium/blob/master/Tests/Selenium2TestCaseTest.php#L733
ScreenshotListener
Please note that there is a ScreenshotListener around, which might be worth looking at:
https://github.com/giorgiosironi/phpunit-selenium/blob/master/PHPUnit/Extensions/Selenium2TestCase/ScreenshotListener.php
With usage example over at https://github.com/giorgiosironi/phpunit-selenium/blob/master/Tests/Selenium2TestCase/ScreenshotListenerTest.php
This might be a cleaner implementation to grab test failures and make shots.
Combining the solutions from #Jens A. Koch and #John Joseph, we get this:
<?php
class homepageTest extends PHPUnit_Extensions_Selenium2TestCase {
private $listener;
public function setUp() {
// Your screenshots will be saved in '/var/www/vhosts/screenshots/'
$screenshots_dir = '/var/www/vhosts/screenshots/';
$this->listener = new PHPUnit_Extensions_Selenium2TestCase_ScreenshotListener($screenshots_dir);
$this->setBrowser('firefox');
$this->setBrowserUrl('https://netbeans.org');
}
public function testNetbeansContainsHorses() {
$this->url('https://netbeans.org');
$this->assertContains('Equestrian', $this->title()); // Will fail on NetBeans page.
}
public function onNotSuccessfulTest($e) {
$this->listener->addError($this, $e, microtime(true));
parent::onNotSuccessfulTest($e);
}
}
A way of doing this across all your web tests is to override one of the test failure functions from the parent test case class, and capture your screenshot there.
Example:
class MyBaseWebTests
{
$this->directory = '/some_path_to_put_screenshots_in/';
// Override PHPUnit_Extensions_Selenium2TestCase::onNotSuccessfulTest
public function onNotSuccessfulTest(Exception $e)
{
$filedata = $this->currentScreenshot();
$file = $this->directory . get_class($this) . '.png';
file_put_contents($file, $filedata);
parent::onNotSuccessfulTest($e);
}
}
Now, after any of your web tests fail, they will dump a screenshot in that folder with the name of the web test class as the filename.
Use this to save screenshot..very useful in case of headless browser.
$fp = fopen('path/35.png', 'wb');
fwrite($fp, $this->currentScreenshot());
fclose($fp);
sleep(1);

Execute some action when Spock test fails

I'd like to execute some action when Spock test fails. Specifically, take a screenshot. Is it possible? How to do it?
Create a listener class
class ExampleListener extends AbstractRunListener {
def void error(ErrorInfo error) {
println "Actual on error logic"
}
}
then add it to each specification using implementation of IGlobalExtension that is executed for each Spec
class GlobalSpecExtension implements IGlobalExtension {
#Override
void visitSpec(SpecInfo specInfo) {
specInfo.addListener(new ExampleListener())
}
}
and finally create file named org.spockframework.runtime.extension.IGlobalExtension in your META-INF/services directory (typically it will be under src/test/resources if you are using Maven) with the full name of your IGlobalExtension implementation e.g.
com.example.tests.GlobalSpecExtension
The best way to achieve this is to write a (global or annotation-driven) Spock extension that implements and registers an AbstractRunListener. For an example, see OptimizeRunOrderExtension. For how to register a global extension, see the IGlobalExtension descriptor.
There isn't much documentation on extensions because the APIs are still subject to change. If you want to play it safe (and can live with some restrictions), you can implement a JUnit Rule instead.
One problem that you may encounter in both cases is that they don't provide access to the current spec instance. If you need this, you may have to use both an AbstractRunListener (to be notified of the failure) and an IMethodInterceptor (to get hold of the spec instance), both registered by the same extension. (Shouldn't be this hard, but that's what's currently there.)
I've managed to do it this way:
class ExampleTest extends GebSpec{
static boolean success = false
def setup(){
success = false
}
def cleanup(){
assert success == true, someAction()
}
def someAction(){
}
def "TestCase"(){
expect:
/*What you expect here*/
(success = true) != null
}
}
Before each test case "success" is set to false by the setup() method.
At the end of each test case you add the "(success = true) != null" statement. Therefore "success" will only be true if the test case has passed.
After each test case the cleanup() method will verify if "success" is true. If it isn't the method someAction() will be called.
I can't upvote or comment on user3074543's answer, but it's simpler than creating an extension. I want easy. So I shortened user*'s a little (I don't mean the 1-line methods). You can make the logic simpler by recording failure instead of success, and reduce typing with a done() helper.
class Test extends spock.lang.Specification {
def fail
def setup(){ fail = true }
def done(){ !(fail = false) }
def cleanup(){ fail && doStuffWhenFail() }
def 'test things'(){
expect:
stuff
done()
}
}

Restart failed test case automatically in TestNG/Selenium

I am using Selenium webdriver, in Java with TestNG to run an X amount of test cases.
What I would like, is for any test case to automatically restart (either from starting or from point of failure), as soon as it fails.
I know TestNG framework has the following method
#Override
public void onTestFailure(ITestResult tr) {
log("F");
}
but I do not know how to find out which testcase it was and then how would I restart it.
I wanted to see an example with actual code in it and found it here:
Restarting Test immediately with TestNg
Observe how the below tests will each be re-run once as soon as the failure happens.
import org.testng.Assert;
import org.testng.IRetryAnalyzer;
import org.testng.ITestResult;
import org.testng.annotations.Test;
public class Retry implements IRetryAnalyzer {
private int retryCount = 0;
private int maxRetryCount = 1;
public boolean retry(ITestResult result) {
if (retryCount < maxRetryCount) {
retryCount++;
return true;
}
return false;
}
#Test(retryAnalyzer = Retry.class)
public void testGenX() {
Assert.assertEquals("james", "JamesFail"); // ListenerTest fails
}
#Test(retryAnalyzer = Retry.class)
public void testGenY() {
Assert.assertEquals("hello", "World"); // ListenerTest fails
}
}
From testng.org
Every time tests fail in a suite, TestNG creates a file called testng-failed.xml in the output directory. This XML file contains the necessary information to rerun only these methods that failed, allowing you to quickly reproduce the failures without having to run the entirety of your tests.
If you want to rerun the test exactly after the failure you need to call the method that failed. You can get that method name from ITestResult object.
If you want to rerun all the failed test cases together, then you can give the testng-failed.xml as input xml after the first execution.