I am using sbt test to run TestNG test cases:
I get a testing result like this
===============================================
xxxx-test
Total tests run: 2, Failures: 0, Skips: 2
Configuration Failures: 1, Skips: 1
===============================================
[info] Passed: Total 2, Failed 0, Errors 0, Passed 0, Skipped 2
[success] Total time: 8 s, completed May 11, 2019 11:44:59 AM
The configuration failure is caused by an error in #BeforeClass phase.
The problem is my CI treat this result as passed.
How can I mark the whole test failed if any of my test cases skipped?
How about trying to modify the test result to fail via IMethodListener or ITestListener?
Related
Exception while taking screenshot Cannot invoke method getScreenshotAs() on null object
Tests run: 2, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 0.907 sec <<< FAILURE! -
java.lang.IllegalStateException: The driver executable does not exist:/src/***/Chrome_Drivers/chromedriver_win32/chromedriver.exe
While running the selenium jobs in Jenkins, how should we specify the chrome driver path?
I am trying to run the code in windows and the same code is running fine in my local however when I try to run the same code with Jenkins(code in bitbucket) it does not find the path.
I tried adding bitbucket URL in the System.setProperty("webdriver.chrome.driver", "https://bitbucket.org///src/***/config/chromedriver.exe");
but nothing works.
Please help.
This post is in addition query to Is it possible to migrate from cucumber-jvm to QAF? Could you please help me know the approach to execute as testNG? For now I've configured the example from below link
https://github.com/qmetry/qaf/releases/download/2.1.9-RC2/QAFGherkin.zip
however when I try to execute the CucumberRunner.java It doesn't find any feature
log4j:WARN No such property [follow] in org.apache.log4j.FileAppender.
[ConfigurationManager] - ISFW build info: {qaf-Type=core, qaf-Revision=9-RC1, qaf-Version=2.1, qaf-Build-Time=23-Sep-2016 12:16:17}
[ConfigurationManager] - Resource dir: C:\Users\Kalyankar\Downloads\qaf-blank-project-maven-masterDec\qaf-blank-project-maven-master\resources. Found property files to load: 6
[ConfigurationManager] - Resource dir: C:\Users\Kalyankar\Downloads\qaf-blank-project-maven-masterDec\qaf-blank-project-maven-master\resources. Found property files to load: 0
[TestNG] Running:
C:\Users\Kalyankar\AppData\Local\Temp\testng-eclipse-1055468291\testng-customsuite.xml
No features found at [classpath:com/qmetry/qaf/examples/cucumber/runners]
0 Scenarios
0 Steps
0m0.000s
PASSED: run_cukes
Runs Cucumber Features
===============================================
Default test
Tests run: 1, Failures: 0, Skips: 0
===============================================
===============================================
Default suite
Total tests run: 1, Failures: 0, Skips: 0
===============================================
[TestNG] Time taken by org.testng.reporters.JUnitReportReporter#6e1567f1: 18 ms
[TestNG] Time taken by org.testng.reporters.EmailableReporter2#72d818d1: 18 ms
[TestNG] Time taken by [FailedReporter passed=0 failed=0 skipped=0]: 30 ms
[TestNG] Time taken by org.testng.reporters.XMLReporter#31a5c39e: 35 ms
[TestNG] Time taken by org.testng.reporters.jq.Main#59ec2012: 224 ms
[TestNG] Time taken by org.testng.reporters.SuiteHTMLReporter#42dafa95: 131 ms
I've ensured that application.properties values are also set as step.provider.pkg=com.qmetry.qaf.examples.steps.cucumber then later, I moved a feature file to com.qmetry.qaf.examples.cucumber.runners package & executed it. No luck. Any help would be much appreciated.
Thanks in advance
Please refer migration steps. Will you please share your project structure after migration and config file you are using?
Use QAF latest version and don't run with cucumber runner, instead run using QAF. Below is sample configuration file:
<test name="Gherkin-QAF-Test">
<parameter name="step.provider.pkg" value="your.steps.pkg" />
<parameter name="scenario.file.loc" value="your features file relative path" />
<classes>
<class name="com.qmetry.qaf.automation.step.client.gherkin.GherkinScenarioFactory" />
</classes>
</test>
You can run as TestNG test.
Default directory for .bdd or .feature file is <project_home>/scenarios directory. If it is not under /scenarios you need to set using scenario.file.loc.
I'm having issue with play framework and JUnit test with play 2.3
When running: "activator test" then it runs fine, but...
[info] Loading project definition from /.../test/toto/project
[info] Set current project to toto (in build file:/home/jenkins/.../)
[info] Compiling 6 Java sources to /home/jenkins/automated_tests/.../toto/target/scala-2.11/test-classes...
[warn] application - do execute tests in class utils.UtilsTest
[warn] application - do execute tests in class test_marketcity.BasicTest
[info] Passed: Total 14, Failed 0, Errors 0, Passed 14
[success] Total time: 29 s, completed Feb 12, 2015 9:48:51 AM
There's nothing about the test which have been run, only the full total of test (14 here). With play 2.2 I had a more detailled log containing also each test:
[info] x firstTest (x means here it failed)
[info] + secondTest (+ means here it succeeded)
Should I have to put a specific option here to get more details ?
Thx
It is a common issue with tests on activator. I usually run activator clean before tests to get the output. Sometimes running tests from sbt helps.
I'm using Selenium WebDriver in .NET environment. I'm using WebDriver inside NUnit test runner like that:
[Test]
public static void Should_register_user()
{
IWebDriver driver = new FirefoxDriver();
...
driver.Quit();
}
It works quite fine. If I run this test from MSBuild script like that: packages\NUnit.Runners.2.6.2\Tools\nunit-console.exe /nologo S022.SeleniumTests.dll /xml=NUnit.Report.xml (no matter if I'm using NUnit community task or exec task).
The test performs but it it fails the Firefox browser remains opened and the execution in script waits for me to close the browser window. When I do that the script executes further.
Here it is what Selenium says:
ProcessModel: Default DomainUsage: Single
Execution Runtime: net-3.5
.F
Tests run: 1, Errors: 0, Failures: 1, Inconclusive: 0, Time: 10,3118323 secon
ds
Not run: 0, Invalid: 0, Ignored: 0, Skipped: 0
Errors and Failures:
1) Test Failure : S022.SeleniumTests.RegistrationTests.Should_register
_user
Expected string length 6 but was 11. Strings differ at index 0.
Expected: "Log-in"
But was: "News"
-----------^
at S022.SeleniumTests.RegistrationTests.Should_register_user() in S022.SeleniumTests\T
ests\RegistrationTests.cs:line 22
If the test passes the execution goes along as expected.
What am I doing wrong?
You're creating and destroying the browser instance within the [Test] method. When the test case fails, an exception is thrown, and your call to driver.Quit() is never getting executed. You probably want to create the driver instance in the [SetUp] method, and call driver.Quit() in the [TearDown] method.
I have a maven project in Eclipse, a web-project using Seam. Now I wanted to use Seam Test for unit test. I dit not use SeamGen to create the project.
I followed this guide, but when I run my test I get this error. I get a feeling the wrong sax parser is being used. Does anyone recognize it?
[Parser] Running: :\Documents and Settings\PEJA715\Local Settings\Temp\testng-eclipse\testng-customsuite.xml
FAILED CONFIGURATION: #BeforeSuite startSeam
java.lang.RuntimeException: Unable to bootstrap:
at org.jboss.embedded.Bootstrap.bootstrapURL(Bootstrap.java:147)
at org.jboss.embedded.Bootstrap.bootstrap(Bootstrap.java:183)
at org.jboss.embedded.Bootstrap.bootstrap(Bootstrap.java:195)
at org.jboss.seam.mock.EmbeddedBootstrap.startAndDeployResources(EmbeddedBootstrap.java:11)
at org.jboss.seam.mock.AbstractSeamTest.startJbossEmbeddedIfNecessary(AbstractSeamTest.java:1024)
at org.jboss.seam.mock.AbstractSeamTest.startSeam(AbstractSeamTest.java:915)
at org.jboss.seam.mock.SeamTest.startSeam(SeamTest.java:58)
Caused by: org.jboss.xb.binding.JBossXBRuntimeException: Failed to create a new SAX parser
at org.jboss.xb.binding.UnmarshallerFactory$UnmarshallerFactoryImpl.newUnmarshaller(UnmarshallerFactory.java:100)
at org.jboss.kernel.plugins.deployment.xml.BeanXMLDeployer.deploy(BeanXMLDeployer.java:85)
at org.jboss.embedded.Bootstrap.deployBaseBootstrapUrl(Bootstrap.java:130)
at org.jboss.embedded.Bootstrap.bootstrapURL(Bootstrap.java:142)
... 21 more
Caused by: org.jboss.xb.binding.JBossXBException: Failed to create a new SAX parser
at org.jboss.xb.binding.parser.sax.SaxJBossXBParser.(SaxJBossXBParser.java:97)
at org.jboss.xb.binding.UnmarshallerImpl.(UnmarshallerImpl.java:55)
at org.jboss.xb.binding.UnmarshallerFactory$UnmarshallerFactoryImpl.newUnmarshaller(UnmarshallerFactory.java:96)
... 24 more
Caused by: javax.xml.parsers.ParserConfigurationException: Feature 'http://apache.org/xml/features/xinclude' is not recognized.
at org.apache.xerces.jaxp.SAXParserFactoryImpl.newSAXParser(Unknown Source)
at org.jboss.xb.binding.parser.sax.SaxJBossXBParser.(SaxJBossXBParser.java:92)
... 26 more
... Removed 15 stack frames
SKIPPED CONFIGURATION: #BeforeClass setupClass
SKIPPED CONFIGURATION: #BeforeMethod begin
SKIPPED CONFIGURATION: #AfterMethod end
SKIPPED CONFIGURATION: #AfterClass cleanupClass
SKIPPED: testRegister
===============================================
TestOfTest
Tests run: 1, Failures: 0, Skips: 1
Configuration Failures: 1, Skips: 4
===============================================
===============================================
effsort-web
Total tests run: 1, Failures: 0, Skips: 1
Configuration Failures: 1, Skips: 5
===============================================
[org.testng.internal.PoolService] Shutting down poolservice org.testng.internal.PoolService#1117a20 terminated:false
Hard to say from the stacktrace (showing your dependencies might help) but double check that you use the SAX parser used by JBoss 5.0.1.GA (and not xml-apis that you might get transitively).