I was wondering how one would create proper test evidence of Selenium UI tests.
I was thinking about screenshots but they do not actually cover everything you have done as it is difficult to determine when to screenshot. (Every click or every wait or every pageload).
Another option I thought about was screen recording, but this makes it rather difficult to work with parallelism as you record the whole screen and not a specific chrome window.
However, you could possibly also take a screenshot every second via the webdriver and turn this into a video. Then one would have to work with a separate thread which can be quite challenging considering the condition you have to provide to stop the thread from taking screenshots. Else the test will run forever.
As I was not able to draw a convincing conclusion based on my own thoughts about creating a test evidence report for UI tests I was hoping someone could explain to me how to properly do this.
I had similar issue and I've introduced in my automation framework ExtentReports + klov server with Testrail as tool for test-management.
I think that nobody would ask of You to show testcases via video or screenshot, but if that is necessary, You can check out several libraries for taking 'video', because this is not actual video, than rather bunch of screenshots that are mashed into one video.
What has actually proven really good investment of time is to take fallen tests cases screenshot and attach it in testcase result (Testrail, Bugzila, Extentreports whatever).
Actualy if using selenium/appium You can check this repo
[https://github.com/groupon/Selenium-Grid-Extras] they make 'video' like mentioned and stored on local hub/node.
But best menthod that has been real good method was report with detail steps of each testcase:
Screenshot testcase with detailed steps and action:
The way I handled the reporting in my application goes in the line of what Kovacic said.
I also used ExtentReports as a way to generate metrics and having a step by step record of what happened.
I created a method reponsible for recording a step ( clicked that, navigated there, asserting that... ) with the option of taking a screenshot if needed, and another one for starting a new test.
Then , it's a matter of calling those methods in the PageObject style testing framework and pretty much having those method called in every action made by your framework.
To better illustrate here are some implementation examples (c#) :
Log a step method
public void LogStep(Status status,string MessageToLog, bool hasScreenshot)
{
//we leave the possibility of taking the screenshot with the step or not
if (hasScreenshot)
{
Test.Log(logstatus, messageToLog)
.AddScreenCaptureFromPath(GetScreenshot());
}
else
Test.Log(logstatus, messageToLog);
}
Screenshot capture method
public static string GetScreenshot()
{
ITakesScreenshot ts;
//Browser.Driver here is the instance of the Driver you want to take screenshots with
ts = (ITakesScreenshot)Browser.Driver;
var screenshot = ts.GetScreenshot();
// Here just input the name you want your screenshot to have, with path
var screenshotPath = ScreenShotFolder + #"\" + _screenshotcount + ".bmp";
screenshot.SaveAsFile(screenshotPath);
// I've introduced a variable to keep track of the screenshot count (optional)
return (ScreenShotFolder.Substring(_reportRoot.Length) +"/"+ _screenshotcount + ".bmp");
}
Example of call in the framework
public void BlockAccount()
{
try
{
_blockAccBtn.Click();
_confirmBtn.Click();
ExtentReportGenerator.LogStep(Status.Info, "Blocking Account");
}
catch (NoSuchElementException)
{
ExtentReportGenerator.LogStep(Status.Fail, "Could not find block button", true);
}
}
A NunitTest using the whole system
[TestCase, Order(1)]
public void CanBlockCard()
{
//Creates a new test in the report
ExtentReportGenerator.Test = ExtentReportGenerator.Extent.CreateTest(GetCurrentMethod());
//Each one of these calls to the framework has logged steps
CashlessPages.CashlessAffiliationsPage.AccessAccount(1, 1);
CashlessPages.CashlessAccountsPage.BlockAccount();
Assert.IsTrue(CashlessPages.CashlessAccountsPage.IsAccBlocked());
}
Example of generated Report
Hope this helps
Related
I have seen and used applications that are "all in one". where just launching the application would be running application.exe, but when you launch the same application with a switch for example: application.exe /settings the settings page would be launched.
I do not have any code as I don't even know where to begin researching the hurdle.
I would like my program to launch a specific form (and maybe limit the user to THAT form) depending on the switch used.
FYI I was testing the new “Ask a Question Wizard” prototype, I didn't realise it was going to post an actual question.
Ah well I figured I'll leave it up, hopefully myself & others too, might learn something.
if you can make change in this using the switch then your requirement can be fulfilled easily...
static void Main()
{
Application.EnableVisualStyles();
Application.SetCompatibleTextRenderingDefault(false);
Application.Run(new YourStartupfFormHere());
}
I have integrated Sikuli with my Selenium project. For the sake of learning, I have used simple gmail login application to automate it using Sikuli. Well, I am able to execute script. Now let's say, I'm typing something in my Username field. And sometimes, the mouse is not hovered to the username field. So my test scripts failed. And it is intermittent behavior.
public static void main(String[] args) throws Exception {
Screen screen = new Screen();
Pattern pattern1 = new Pattern("E:\\Projects\\Java\\Demo\\Images\\UserName.PNG");
Pattern pattern2 = new Pattern("E:\\Projects\\Java\\Demo\\Images\\Password.PNG");
Pattern pattern3 = new Pattern("E:\\Projects\\Java\\Demo\\Images\\SignIn.PNG");
Pattern pattern4 = new Pattern("E:\\Projects\\Java\\Demo\\Images\\Next.PNG");
Pattern pattern5 = new Pattern("E:\\Projects\\Java\\Demo\\Images\\SignedIn.PNG");
Pattern pattern6 = new Pattern("E:\\Projects\\Java\\Demo\\Images\\SentMail.PNG");
Pattern pattern7 = new Pattern("E:\\Projects\\Java\\Demo\\Images\\SentMessage.PNG");
System.setProperty("webdriver.chrome.driver","E:\\Projects\\Java\\Demo\\Drivers\\chromedriver.exe");
WebDriver driver = new ChromeDriver();
driver.manage().timeouts().implicitlyWait(5,TimeUnit.SECONDS);
driver.navigate().to("https://www.gmail.com");
driver.manage().window().maximize();
screen.type(pattern1,"email id");
screen.click(pattern4);
screen.type(pattern2,"password");
screen.click(pattern5);
screen.click(pattern3);
screen.wait(pattern6,20);
screen.click(pattern6);
screen.wait(pattern7,5);
screen.click(pattern7);
}
Does anyone have an idea why this happens?
First of all, share your code.
Usually, intermittent behavior like you describe is caused by timeouts. Meaning that you are looking for an element that is not there yet or has not yet become stable.
A practical example in your scenario can be trying to detect the username field before the page has fully loaded. It will be useful to know how you have used both tools. What you used for navigation and what for elements identification?
Saying that, the quickest way to try and solve this problem is to put few seconds delay before you start searching for username element. See if that helps you.
EDIT
Now when you have posted your code, have a look at these two lines:
driver.manage().window().maximize();
screen.type(pattern1,"email id");
Here, you maximize the browser window and immediately try to find and type into the element described by pattern1. This is likely to be a problem since your driver instance does not wait for the window to become maximized and the next command will start executing immediately. You should allow some time to ensure that the window has finished resizing. just add a short sleep between these lines and see if that helps.
As it happens intermittently and occurs for the very first action in a newly drawn screen this looks like a timing problem.
The Sikuli solution here is to wait until your input field is available before you use it.
The statement to use is:
wait(pattern1[, seconds])
Insert just before:
screen.type(pattern1,"email id");
Reference:
http://doc.sikuli.org/region.html#Region.wait
Guys First of all I am totally new for Selenium.
I am having a automation project. In my project, I am creating a screenshot function to take screenshots of my event which I have created for my testcases. Now if my test cases passes then all screenshot should move to Pass folder, else fail folder.
I would like to know how to detect that my test case pass?
I know Nunit detects but I wanted to program it so that I cam place my screenshot as well as log file to pass or fail folder.
Program in C#
Selenium
Nunit to run my test case.
I think you meant was this. But there is work around for this. You need to add your code accordingly.
if (TestContext.CurrentContext.Result.Outcome.Equals(ResultState.Failure))
{
IntegrationTest.WriteInLog("FAILS");
}
else if (TestContext.CurrentContext.Result.Outcome.Equals(ResultState.Success))
{
IntegrationTest.WriteInLog("SUCESS");
}
Check status property and compare it with TestStatus enum at teardown method.
NUnit2:
TestContext.CurrentContext.Result.Status
NUnit3:
TestContext.CurrentContext.Result.Outcome.Status
I'm pretty new to Thucydides / JBehave, but haven't found any solutions posted to this one. This is the first time I've used Thucydides / JBehave, but have used Selenium before.
I have my .story file. The .story file lists 5 scenario's.
Each scenario is implemented in it's own POJO java class.
e.g
public class ManagerBypassLoginSteps
{
#Steps
ManagerSteps managerSteps;
#Given("the manager is not logged in")
public void mangerLogsIn()
{
managerSteps.start();
}
#When("the manager goes to a different page")
public void managerToDashboardPage()
{
managerSteps.goesToDashboardPage();
}
#Then("they should see the login page")
public void managerShouldSeeLoginPage()
{
managerSteps.verifyOnLoginPage();
managerSteps.close();
}
}
The ManagerSteps class extends net.thucydides.core.steps.ScenarioSteps, but is otherwise just a POJO. The ManagerSteps calls the page objects - all very normal stuff and as per the examples.
In the scenario above, the test fails as the code displays an error message instead of returning the user to the log in page - that's fine, the security code doesn't meet the specified requirements (yet) But, after the assert inside the ManagerSteps class fails, the test appears to stop. This means I have a browser instance just sitting there until I close it. When run as a series of tests, this means a broswer in the Selenium grid is now tied up.
I need a way to detect a test failure, to call to the page object to close / quit.
I can't find the equivalent of a #Before or #After that will always run, I could find it I could use the #After to close the page object's webDriver.
I am not using Thucydides to manage the WebDriver lifecycle as I couldn't find any examples of this that did not use the ThucydidesRunner.class.
Do I have to use the ThucydidesRunner.class ? My impression from the Thucydides manual for integrating with JBehave (http://thucydides.info/docs/thucydides/_writing_acceptance_tests_with_jbehave.html) didn't suggest it was required.
How does everyone else handle test failures in this situation ?
The real problem I had was closing the browser, after a test failure.
The best way to manage the WebDriver appear to be to use the Thucydides #ManagedDriver annotation to handle the lifecyle.
I ended up using a very simple JUnit runner like so
public class UserLogin extends ThucydidesJUnitStory
{
#Managed
public WebDriver webdriver;
#ManagedPages(defaultUrl = "http://localhost:8080/dc-bus-main-web")
public Pages pages;
}
When this is started, it maps to the user_login.story file, through JBehave's naming convention, then starts normally.
I'm using Behat with Mink.
I would like one of my step definitions to act differently depending on which driver is running.
Ideally, my code would look something like this
public function stepDefinition(){
if($this->getSession()->getDriver()->name == 'goutte'){
//code to run if using goutte
}else{
//code to run if selenium is running
}
}
So, though a little bit of delving into the code meant that I've found a solution to this. And seen as google was no help, hopefully this will be of help to someone else.
My code now looks like this
if( $this->getSession()->getDriver() instanceof Behat\Mink\Driver\Selenium2Driver){
// Selenium Code
}else{
//Goutte Code
}
I've just grabbed the driver object and have checked which driver class it's an extension of, simple.
Now I can run the same step definition if the #javascript tag is or isn't before my scenario.