I'm not sure whether the issue is with NUnit or SpecFlow but whenever I add the 'Run Functional Tests' task to my build, I get the following error when running a build in TFS2017...
2017-05-06T00:11:00.4676774Z ##[warning]DistributedTests: Test discovery started.
2017-05-06T00:11:00.4676774Z ##[warning]DistributedTests: Test Run Discovery Aborted . Test run id : 5
2017-05-06T00:11:00.4676774Z ##[warning]DistributedTests: UnExpected error occured during test execution. Try again.
2017-05-06T00:11:00.4676774Z ##[warning]DistributedTests: Error : NUnit Adapter 3.7.0.0: Test discovery complete
2017-05-06T00:11:00.4676774Z
2017-05-06T00:11:00.4676774Z ##[warning]DistributedTests: Test run aborted. Test run id: 5
2017-05-06T00:11:00.4676774Z ##[error]System.Exception: The test run was aborted, failing the task.
2017-05-06T00:11:00.5175379Z ##[error]PowerShell script completed with 1 errors.
Does anyone know what the problem is?
Many thanks,
Run Functional Tests task are used for below scenarios.
Typical scenarios include:
Tests that require additional installations on the test machines, such as different browsers for Selenium tests
Coded UI tests
Tests that require a specific operating system configuration
To execute a large number of unit tests more quickly by using multiple test machines
If you are using specflow you need to make sure that you implement using MS Test.
Regardless of the fact which unit test provider you use for SpecFlow, because all the major unit test frameworks provide the necessary adapter for it (this is the same adapter that you need to run the tests from the Visual Studio Test Explorer Window)
So you need to use Visual Studio Test task in this scenario(Also for handling standard unit tests). For details of the settings please refer this blog: SpecFlow Tips--Run only specific scenarios in TFS/VSTS build
Related
I am working on a node.js application and would like to know if there is a way I can run all the unit tests from all the sub modules even if there are some test failures to know how many tests are failing in total to start putting the fixes for them. We use mocha for our tests on the back-end and jest for the ui.
Thanks.
The default behavior for mocha is to run all the tests. If it is exiting after the first test failure, that would suggest that you are using the "bail" option typically enabled on the command line with either --bail or -b.
Relevant docs: https://mochajs.org/#-bail-b
It can also be caused by passing the option { bail: true } to mocha.setup(). Look in your test runner and in your package.json.
Lastly, the least likely of these possibilities is that it could also be caused by using this.bail() somewhere in the Mocha test runner.
I am using VSTest to run tests via Azure DevOps. Tests run successfully, but using the option rerun failed tests: true, an error appears during the restart phase.
NUnit 3.12.0 ;
NUnit Adapter 3.16.1.0 (Checked with 4.0.0.0)
vstest.console.exe "C:\agent2.172.2\_work\r1\a\UITest\drop\Tests.Web\bin\Release\netcoreapp3.1\Tests.Web.dll"
/Settings:"C:\agent2.172.2\_work\_temp\3utv233tymm.runsettings"
/Logger:"trx"
/TestAdapterPath:"C:\agent2.172.2\_work\r1\a\UITest\drop\Tests.Web\bin\Release\netcoreapp3.1"
/TestCaseFilter:"FullyQualifiedName=Tests.Web.Tests._5.CourierModuleTest.N1_SendingTest.Id_5_1_01_TransferToCourierModule(Chrome)|FullyQualifiedName=Tests.Web.Tests._3.IssuanceOfDocuments.InformationOnTheApplicationIdentificationOfRecipient.N2_RecipientIdentificationTest.Id_3_2_13_RegisterAddressByFiasTest(Chrome)"
NUnit Adapter 3.16.1.0: Test execution started
An exception occurred while invoking executor 'executor://nunit3testexecutor/': Incorrect format for TestCaseFilter Missing Operator '|' or '&'. Specify the correct format and try again. Note that the incorrect format can lead to no test getting executed.
How can I fix this error and successfully restart the tests in DevOps?
I tried googling for a similar issue but didn't find anything that would work. Any help is really appreciated
Here's similar discussion, according to comments from the Contributors of azure-pipelines-task repo:
1.You should update your VS and VSTest component within it to latest version since we don't support data driven tests for Rerun scenario. It will be available with VS 15.8 release and higher.
2.You should configure your VSTest task following this solution:
Recommended to give total number of your tests as input (Number of tests per batch).
Is there a way to execute TestNG.xml in dry run mode so that I can figure out what methods gets qualified for the test run. I am using Eclipse and intend to run the tests via testng.xml. How to configure Run Configurations for this.
Newbie to Selenium-TestNG and Eclipse
I tried to provide -Dtestng.mode.dryrun=true in Run Configurations -> Arguments tab under both Program argument and VM arguments
The run configurations had no effect on the execution. The tests were executed in normal fashion. I expected the configurations would just list test methods in the console
You are going to see all tests with no failures. That what you expect when you run testNg with the argument.
To check the dryrun argument works, make your test to fail. Then run your test with "-Dtestng.mode.dryrun=true". Add the argument in "VM arguments"
Also, check your version of testNG is 6.14 or higher
I check out the source code of Cassandra, and I want to run a unit test case in debug mode to understand how it works
Below is my JUnit run configuration set up. The code can compile correctly using ant. And I tried both targets build and build-test.
IntelliJ can pick up the class in the run configuration, but when I run this profile, I got.
Process finished with exit code 1
Class not found: "org.apache.cassandra.tools.SSTableExportTest"
Empty test suite.
What part do I need to change the configuration so that IntelliJ can run this unit test cases?
Since you're trying to debug Cassandra. You can reference https://wiki.apache.org/cassandra/RunningCassandraInIDEA. They have already set up a ant target to generate required configurations.
I have Groovy Script in my Test Case in SOAP UI.
There are one big script and a lot of test steps which are run by this script. I know about functionality of TestRunner but it run all steps in test case, but i need to run only my groovy script. How?
UPDATE
I disable all Test steps and left active only groovy script. Then I Launch TestRunner on Test Case, it return exception :
java.lang.Exception: TestCase [Case1] failed without assertions
at com.eviware.soapui.tools.SoapUITestCaseRunner.throwFailureException(SoapUITestCaseRunner.java:535)
at com.eviware.soapui.tools.SoapUITestCaseRunner.runRunner(SoapUITestCaseRunner.java:437)
at com.eviware.soapui.tools.AbstractSoapUIRunner.run(AbstractSoapUIRunner.java:162)
at com.eviware.soapui.tools.AbstractSoapUIRunner.runFromCommandLine(AbstractSoapUIRunner.java:93)
at com.eviware.soapui.tools.SoapUITestCaseRunner.main(SoapUITestCaseRunner.java:119)
Just disable the rest of the steps and run the test case. The disabled test steps are not run automatically by soapui, you can call those conditionally :)