Is it possible to run tests written in Concordion.Net using some form of Visual Studio addin? I am looking to start using Concordion.Net. However, I was used to running tests from Visual Studio directly through the Resharper plugin. Since Concordion.Net is a different concept, and uses HTML files to 'specify' the tests, one cannot use the Resharper plugin.
You can run them using the nUnit GUI, but it is a bit of a pain to use it and slow.
You can use TestDriven.NET (http://testdriven.net/) to execute Concordion.NET tests within Visual Studio. To run a test please use the "Run Test(s)" command of TestDriven.NET on your Concordion.NET fixture class (http://testdriven.net/quickstart.aspx).
Please, make sure that you have specified the [assembly: RequiredAddin("ConcordionNUnitAddin")] attribute in the AssemblyInfo.cs of your specification project as described in the how to "Run Concordion.NET Tests with NUnit" (http://concordion.org/dotnet/RunningTests.html).
When you run Concordion.NET tests with TestDriven.NET in Visual Studio, you should see an output similar to:
“------ Test started: Assembly: Concordion.Spec.dll ------
Processed specifications : C:\Users\myuser\AppData\Local\Temp\Concordion\Spec\Concordion\Command\AssertEquals\NonString\Boolean.html
1 passed, 0 failed, 0 skipped, took 0,67 seconds (NUnit 2.6.2).”
To be able to run Concordion.NET tests with TestDriven.NET you have to install the Concordion.NUnit.dll into the installation directory of TestDriven.NET (C:\Program Files (x86)\TestDriven.NET\NUnit\2.6\addins\Concordion.NUnit.dll).
Concordion.NET tests can be run with the help of NUnit (http://concordion.org/dotnet/RunningTests.html) based on the Concordion NUnit addin. When NUnit discovers a class containing Concordion.NET fixture code annotated with [ConcordionTest], the Concordion.NUnit addin loads the related HTML specification document and creates a test instance that is executed with the help of NUnit.
Unfortunatelly, ReSharper uses a different mechanism to find tests than NUnit(http://confluence.jetbrains.com/display/NETCOM/4.09+Test+Framework+Support+%28R8%29). Thus, it would be necessary to write a ReSharper plugin to recognize classes annotated with [ConcordionTest] and use the ReSharper NUnit support to execute them. NUnit 3.0 will probably comply with interfaces consumed by ReSharper.
On the other Hand, the current version of the NUnit Adapter for Visual Studio (http://nunit.org/index.php?p=vsTestAdapter&r=2.6.2) is not able to load NUnit addins appropriately. An update of NUnit is necessary, which is planned for NUnit 3.0 (https://github.com/nunit/nunit-vs-adapter/issues/9).
To run Concordion.NET tests in Visual Studio you can use TestDriven.NET as described above.
Related
I have a cross platform automation solution based on SpecFlow and NUnit 3.
This solution succeeds to run Web (Selenium) and Android (Xamarin.UITest) automated tests on Windows, unfortunately, it does not works on OS X in Visual Studio Mac.
When I use the command line NUnit console runner OR Rider (JetBrains IDE), tests seems to be working, but when I use VS Mac, it does not.
All my issues seems to be related to VS Mac, the first I had was because assemblies dynamically loaded by SpecFlow were not found.
Someone gave me a workaround on GitHub (TestRunnerSupportAssembly key to add in .csproj so that test runner can find assemblies).
And now that the assemblies are loaded, it is the App.config file which seems to be loaded but whose keys do not appear in AppSettings object.
Here is a part of the log I get from the IDE during test run :
Mono: Config attempting to parse: '/Users/.../Automation/AutomationMobile/bin/Debug/AutomationMobile.dll.config'.
I really think the issue is because of the test runner used by VS Mac, but I can't find how to configure it appropriately.
PS : I cannot migrate to .NET core, because few assemblies does not support it (Xamarin.UITest for instance).
I am running a CI build on TFS server using Visual Studio 2019. My project is based on .NET Core. The test results are not getting published after running the build.
First of all, you need to check your pipeline if that contain corresponding tasks.
You could choose to use Visual Studio Test task in your build definition/pipeline.
Use this task in a build or release pipeline to run unit and functional tests (Selenium, Appium, Coded UI test, and more) using the Visual Studio Test Runner. Other than MSTest-based tests, test frameworks that have a Visual Studio test adapter, such as xUnit, NUnit, Chutzpah, can also be executed.
Tests that target the .NET core framework can be executed by specifying the appropriate target framework value.
Another choice is using .NET Core CLI task with test command, and you need to combine with Publish Test Results task to show the results in the build summary.
If you already have the right configuration. Then check the logs of test related task. For example ,if it does not generate the test results, there should also no test result to publish. Also set system.debug=true to enable verbose debug mode to get a more detail result.
Besides, also attach an official doc show Build, test, and deploy .NET Core apps in case you need.
I've written tests using Selenium in NUnit Framework using C# language. I want to associate these tests as part of builds in the TFS. So whenever new build is generated. These tests would be able to run as part of builds and generate/email reports as well.
Recommend you to use the new build system vNext build. vNext builds are JSON based, and you can plugin tasks built in msbuild, powershell and varied other scripting languages.
About how to integrate NUnit Tests in TFS builds, this blog describes clearly how to do this: Running NUnit Tests in a TFS 2015 Build vNext
Simply summarized as follows:
Add Nuget Package for NUnit Test Adapter
Specify path of custom Test Adapter inside build definition
Copying adapters inside Visual Studio TestWindows folder
Specify Path to Custom Test Adapter with nunit packages
Some other tutorial for you reference:
xUnit or NUnit with Visual Studio Online Build
Running nUnit and Jasmine.JS unit tests in TFS/VSO vNext build
I'm writting test of user interface in VB.Net for my web application. I use Selenium to do it. I made one orderedtest and I want to export it in .exe.
But I don't find .exe when I compile my test unit project in VS. (I use Visual Studio 2012).
I want to export it because I want to launch this test suite when I deploy my application on one server.
This is simply not possible. Microsoft Test projects always produce (set of) assemblies.
You can execute the tests by deploying the TFS Test Agent on the system (or installing Visual Studio Premium), which installs the commandline tools that allow you to run these tests. Installation is enough, you don't need to configure it or even connect it to a TFS environment. The installation will deploy the tools required to run the tests.
You can run your ordered test using either the old or the new test runner:
The new: vstest.console.exe YourOrderedTest.orderedtest /logger:trx
The old: mstest.exe /TestContainer:YourOrderedTest.orderedtest
You would need to copy all items in the Bin folder of your Test project to your target system. You may need to include extra assemblies that have been installed on your development system and may not be available on your target server (references of the test project or projects referenced by the test project).
See
Running automated tests from the command line
MsTest command-line options
VsTest.Console command-line options
For running any ordered test using vstest.console following command can be used.
e.g. if you want to execute mytest.orderedtest then run following command
vstest.console mytest.orderedtest /Logger:trx
Run from the folder where the ordered test is present.
I need to run nUnit tests using a workflow template in TFS Server 2010.
I found all the reference bellow explaning how to configure the Build server:
http://morten.lyhr.dk/2008/05/using-nunit-and-ncover-with-tfs-build.html
How to run nunit with msbuild from VS2010
http://www.codewrecks.com/blog/index.php/2009/07/19/integrate-nunit-test-into-a-tfs-build/
http://msdn.microsoft.com/en-us/library/ee342461.aspx
Bellow is closest to what I need:
https://tfsbuildextensions.codeplex.com/wikipage?title=How%20to%20integrate%20the%20nUnit%20build%20activity
But the answer I need is for the question bellow:
"How to set a variable containing all the nUnit tests and how to configure a for each loop to run all the tests"
I do get all my build to run, but in the end, it does not run any nUnit test.
But which tool should I use to run the nUnit tests?:
Please let me know if you have any advice on how to get it done using the TFS 2010.
Thank you so much.
The post •https://tfsbuildextensions.codeplex.com/wikipage?title=How%20to%20integrate%20the%20nUnit%20build%20activity gives all the steps required to make use of this community TFS extensions. It is meant to be a step by step to get the activity working
Step 1 of the notes shows how to find all the assemblies that have been built, basically scan a for any file ending *.dll. I think this is the key to your question, you just pass a list of all DLLs produced by the build and let nUnit find the tests. If you want to do something else you can build the list of DLLs any way you want, but a scan seems the most simple, you don't have to worry over managing the list ypourself
Step 2a runs the nUnit test, the list of DLLs is passed into the nUnit activity (in effect the command line test runner). This will look in each DLL for the nUnit attributes and run the tests it finds.
Step 2b publishes the results back to TFS. This is done by taking the XML output of the test runer and publishing it with MSTEST to the TFS server.
If you are using TFS 2012 or later I would not use this activity, I would use the new test adaptor model built into TFS http://blogs.blackmarble.co.uk/blogs/rfennell/post/2012/03/27/Unit-testing-in-VS11Beta-and-getting-your-tests-to-run-on-the-new-TFSPreview-build-service.aspx and http://visualstudiogallery.msdn.microsoft.com/c8164c71-0836-4471-80ce-633383031099/
I was missing two steps answered in the link bellow:
Visual Studio Online CI Nunit Tests not found during build
1 - To add the nUnit adapter on each test project
2 - To add the nUnit 4.5 dlls to the build server and check it in