I'm creating web tests in Selenium using MSTest and want to take a screenshot everytime a test fails but I don't want to take one every time a test passes.
What I wanted to do is put a screenshot function inside the [TestCleanup] method and run it if test failed but not if test passed. But how do I figure out if a last test passed?
Currently I'm doing bool = false on [TestInitialize] and bool = true if test runs through.
But I don't think that's a very good solution.
So basically I'm looking for a way to detect if last test true/false when doing [TestCleanup].
Solution
if (TestContext.CurrentTestOutcome != UnitTestOutcome.Passed)
{
// some code
}
The answer by #MartinMussmann is correct, but incomplete. To access the "TestContext" object you need to make sure to declare it as a property in your TestClass:
[TestClass]
public class BaseTest
{
public TestContext TestContext { get; set; }
[TestCleanup]
public void TestCleanup()
{
if (TestContext.CurrentTestOutcome != UnitTestOutcome.Passed)
{
// some code
}
}
}
This is also mentioned in the following post.
Related
I've been spotting a lot of these in my project's tests:
#Test
void someTest() throws IOException {
if (checkIfTestIsDisabled(SOME_FLAG)) return;
//... the test starts now
Is there an alternative to adding a line at the beginning of each test? For example in JUnit4 there is an old project that provides an annotation #RunIf(somecondition) and I was wondering if there is something similar in JUnit5?
Thank you for your attention.
Tests can be disabled with #DisabledIf and a custom condition.
#Test
#DisabledIf("customCondition")
void disabled() {
// ...
}
boolean customCondition() {
return true;
}
See also the user guide about custom conditions.
String Actualvalue= d.findElement(By.xpath("//[#id=\"wrapper\"]/main/div[2]/div/div[1]/div/div[1]/div[2]/div/table/tbody/tr[1]/td[1]/a")).getText();
Assert.assertEquals(Actualvalue, "jumlga");
captureScreen(d, "Fail");
The assert should not be put before your capture screen. Because it will immediately shutdown the test process so your code
captureScreen(d, "Fail");
will be not reachable
This is how i usually do:
boolean result = false;
try {
// do stuff here
result = true;
} catch(Exception_class_Name ex) {
// code to handle error and capture screen shot
captureScreen(d, "Fail");
}
# then using assert
Assert.assertEquals(result, true);
1.
A good solution will be is to use a report framework like allure-reports.
Read here:allure-reports
2.
We don't our tests to be ugly by adding try catch in every test so we will use Listeners which are using an annotations system to "Listen" to our tests and act accordingly.
Example:
public class listeners extends commonOps implements ITestListener {
public void onTestFailure(ITestResult iTestResult) {
System.out.println("------------------ Starting Test: " + iTestResult.getName() + " Failed ------------------");
if (platform.equalsIgnoreCase("web"))
saveScreenshot();
}
}
Please note I only used the relevant method to your question and I suggest you read here:
TestNG Listeners
Now we will want to take a screenshot built in method by allure-reports every time a test fails so will add this method inside our listeners class
Example:
#Attachment(value = "Page Screen-Shot", type = "image/png")
public byte[] saveScreenshot(){
return ((TakesScreenshot)driver).getScreenshotAs(OutputType.BYTES);
}
Test example
#Listeners(listeners.class)
public class myTest extends commonOps {
#Test(description = "Test01: Add numbers and verify")
#Description("Test Description: Using Allure reports annotations")
public void test01_myFirstTest(){
Assert.assertEquals(result, true)
}
}
Note we're using at the beginning of the class an annotation of #Listeners(listeners.class) which allows our listeners to listen to our test, please mind the (listeners.class) can be any class you named your listeners.
The #Description is related to allure-reports and as the code snip suggests you can add additional info about the test.
Finally, our Assert.assertEquals(result, true) will take a screen shot in case the assertion fails because we enabled our listener.class to it.
I have some parameterized tests
#ParameterizedTest
#CsvFileSource(resources = "testData.csv", numLinesToSkip = 1)
public void testExample(String parameter, String anotherParameter) {
// testing here
}
In case one execution fails, I want to ignore all following executions.
AFAIK there is no built-in mechanism to do this. The following does work, but is a bit hackish:
#TestInstance(Lifecycle.PER_CLASS)
class Test {
boolean skipRemaining = false;
#ParameterizedTest
#CsvFileSource(resources = "testData.csv", numLinesToSkip = 1)
void test(String parameter, String anotherParameter) {
Assumptions.assumeFalse(skipRemaining);
try {
// testing here
} catch (AssertionError e) {
skipRemaining = true;
throw e;
}
}
}
In contrast to a failed assertion, which marks a test as failed, an assumption results in an abort of a test. In addition, the lifecycle is switched from per method to per class:
When using this mode, a new test instance will be created once per test class. Thus, if your test methods rely on state stored in instance variables, you may need to reset that state in #BeforeEach or #AfterEach methods.
Depending on how often you need that feature, I would rather go with a custom extension.
Environment:
I have a program - named CSIS - which I need to run a lot of automated tests on in Visual Studio 2010 using C#. I have a series of functions which need to be run in many different orders but which all start and end at the same 'home screen' of CSIS. The tests will either be run on their own as a single CodedUITest (.cs filetype) or as an ordered test (.orderedtest filetype).
Goal:
The goal is to open to the CSIS homepage once no matter which of the unit tests is run first and then, after all CodedUITests are finished, no matter which unit test is last, the automated test will close CSIS. I don't want to create a separate unit test to open CSIS to the homepage and another to close CSIS as this is very inconvenient for testers to use.
Current Solution Development:
UPDATE: The new big question is how do I get '[ClassInitialize]' to work?
Additional Thoughts:
UPDATE: I now just need ClassInitialize to execute code at the beginning and ClassCleanUp to execute code at the end of a test set.
If you would like the actual code let me know.
Research Update:
Because of Izcd's answer I was able to more accurately research the answer to my own question. I've found an answer online to my problem.
Unfortunately, I don't understand how to implement it in my code. I pasted the code as shown below in the 'Code' section of this question and the test runs fine except that it executes the OpenWindow() and CloseWindow() functions after each test instead of around the whole test set. So ultimately the code does nothing new. How do I fix this?
static private UIMap sharedTest = new UIMap();
[ClassInitialize]
static public void ClassInit(TestContext context)
{
Playback.Initialize();
try
{
sharedTest.OpenCustomerKeeper();
}
finally
{
Playback.Cleanup();
}
}
=====================================================================================
Code
namespace CSIS_TEST
{
//a ton of 'using' statements are here
public partial class UIMap
{
#region Class Initializization and Cleanup
static private UIMap sharedTest = new UIMap();
[ClassInitialize]
static public void ClassInit(TestContext context)
{
Playback.Initialize();
try
{
sharedTest.OpenWindow();
}
finally
{
Playback.Cleanup();
}
}
[ClassCleanup]
static public void ClassCleanup()
{
Playback.Initialize();
try
{
sharedTest.CloseWindow();
}
finally
{
Playback.Cleanup();
}
}
#endregion
Microsoft's unit testing framework includes ClassInitialise and ClassCleanUp attributes which can be used to indicate methods that execute functionality before and after a test run.
( http://msdn.microsoft.com/en-us/library/ms182517.aspx )
Rather than try and make the unit tests aware of their position, I would suggest it might be better to embed the opening and closing logic of the home screen within the aforementioned ClassInitialise and ClassCleanUp marked methods.
I figured out the answer after a very long process of asking questions on StackOverflow, Googling, and just screwing around with the code.
The answer is to use AssemblyInitialize and AssemblyCleanup and to write the code for them inside the DatabaseSetup.cs file which should be auto-generated in your project. You should find that there already is a AssemblyInitialize function in here but it is very basic and there is no AssemblyCleanup after it. All you need to do is create a static copy of your UIMap and use it inside the AssemblyInitialize to run your OpenWindow() code.
Copy the format of the AssemblyInitialize function to create an AssemblyCleanup function and add your CloseWindow() function.
Make sure your Open/CloseWindow functions only contains basic code (such as Process.Start/Kill) as any complex variables such as forms have been cleaned up already and won't work.
Here is the code in my DatabaseSetup.cs:
using System.Data;
using System.Data.Common;
using System.Configuration;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using Microsoft.Data.Schema.UnitTesting;
using System.Windows.Input;
using Keyboard = Microsoft.VisualStudio.TestTools.UITesting.Keyboard;
using Mouse = Microsoft.VisualStudio.TestTools.UITesting.Mouse;
using MouseButtons = System.Windows.Forms.MouseButtons;
namespace CSIS_TEST
{
[TestClass()]
public class DatabaseSetup
{
static private UIMap uIMap = new UIMap();
static int count = 0;
[AssemblyInitialize()]
public static void InitializeAssembly(TestContext ctx)
{
DatabaseTestClass.TestService.DeployDatabaseProject();
DatabaseTestClass.TestService.GenerateData();
if(count < 1)
uIMap.OpenWindow();
count++;
}
[AssemblyCleanup()]
public static void InitializeAssembly()
{
uIMap.CloseWindow();
}
}
}
I currently have a test which tests the presenter I have in the MVP model. On my presenter I have a property which will call into my View, which in my test is mocked out. In the Initialization of my test, after I set my View on the Presenter to be the mocked View, I set my property on the Presenter which will call this method.
In my test I do not have an Expect.Call for the method I invoke, yet when I run I get this Rhino mock exception:
Rhino.Mocks.Exceptions.ExpectationViolationException: IView.MethodToInvoke(); Expected #1, Actual #0..
From what I understand with Rhino mocks, as long as I am invoking on the Mock outside the expecting block it should not be recording this. I would imagine the test to pass. Is there a reason it is not passing?
Below is some code to show my setup.
public class Presenter
{
public IView View;
public Presenter(IView view)
{
View = view
}
private int _property;
public int Property
get { return _property;}
set
{
_property = value;
View.MethodToInvoke();
}
}
... Test Code Below ...
[TestInitialize]
public void Initilize()
{
_mocks = new MockRepository();
_view = _mocks.StrictMock<IView>();
_presenter = new Presenter(_view);
_presenter.Property = 1;
}
[TestMethod]
public void Test()
{
Rhino.Mocks.With.Mocks(_mocks).Expecting(delegate
{
}).Verify(delegate
{
_presenter.SomeOtherMethod();
});
}
Why in the world would you want to test the same thing each time a test is run?
If you want to test that a specific thing happens, you should check that in a single test.
The pattern you are using now implies that you need to
- set up prerequisites for testing
- do behavior
- check that behavior is correct
and then repeat that several times in one test
You need to start testing one thing for each test, and that help make the tests clearer, and make it easier to use the AAA syntax.
There's several things to discuss here, but it certainly would be clearer if you did it something like:
[TestMethod]
ShouldCallInvokedMethodWhenSettingProperty()
{
var viewMock = MockRepository.GenerateMock<IView>()
var presenter = new Presenter(viewMock);
presenter.Property = 1;
viewMock.AssertWasCalled(view => view.InvokedMethod());
}
Read up more on Rhino Mocks 3.5 syntax here: http://ayende.com/Wiki/Rhino+Mocks+3.5.ashx
What exactly are you trying to test in the Test method?
You should try to avoid using strict mocks.
I suggest using the Rhino's AAA syntax (Arrange, Act, Assert).
The problem lied with me not understanding the record/verify that is going on with Strict mocks. In order to fix the issue I was having this is how I changed my TestInitilize function. This basicaly does a quick test on my intial state I'm setting up for all my tests.
[TestInitialize]
public void Initilize()
{
_mocks = new MockRepository();
_view = _mocks.StrictMock<IView>();
_presenter = new Presenter(_view);
Expect.Call(delegate { _presenter.View.InvokedMethod(); });
_mocks.ReplayAll();
_mocks.VerifyAll();
_mocks.BackToRecordAll();
_presenter.Property = 1;
}