I've written tests using Selenium in NUnit Framework using C# language. I want to associate these tests as part of builds in the TFS. So whenever new build is generated. These tests would be able to run as part of builds and generate/email reports as well.
Recommend you to use the new build system vNext build. vNext builds are JSON based, and you can plugin tasks built in msbuild, powershell and varied other scripting languages.
About how to integrate NUnit Tests in TFS builds, this blog describes clearly how to do this: Running NUnit Tests in a TFS 2015 Build vNext
Simply summarized as follows:
Add Nuget Package for NUnit Test Adapter
Specify path of custom Test Adapter inside build definition
Copying adapters inside Visual Studio TestWindows folder
Specify Path to Custom Test Adapter with nunit packages
Some other tutorial for you reference:
xUnit or NUnit with Visual Studio Online Build
Running nUnit and Jasmine.JS unit tests in TFS/VSO vNext build
Related
I have a cross platform automation solution based on SpecFlow and NUnit 3.
This solution succeeds to run Web (Selenium) and Android (Xamarin.UITest) automated tests on Windows, unfortunately, it does not works on OS X in Visual Studio Mac.
When I use the command line NUnit console runner OR Rider (JetBrains IDE), tests seems to be working, but when I use VS Mac, it does not.
All my issues seems to be related to VS Mac, the first I had was because assemblies dynamically loaded by SpecFlow were not found.
Someone gave me a workaround on GitHub (TestRunnerSupportAssembly key to add in .csproj so that test runner can find assemblies).
And now that the assemblies are loaded, it is the App.config file which seems to be loaded but whose keys do not appear in AppSettings object.
Here is a part of the log I get from the IDE during test run :
Mono: Config attempting to parse: '/Users/.../Automation/AutomationMobile/bin/Debug/AutomationMobile.dll.config'.
I really think the issue is because of the test runner used by VS Mac, but I can't find how to configure it appropriately.
PS : I cannot migrate to .NET core, because few assemblies does not support it (Xamarin.UITest for instance).
I am running a CI build on TFS server using Visual Studio 2019. My project is based on .NET Core. The test results are not getting published after running the build.
First of all, you need to check your pipeline if that contain corresponding tasks.
You could choose to use Visual Studio Test task in your build definition/pipeline.
Use this task in a build or release pipeline to run unit and functional tests (Selenium, Appium, Coded UI test, and more) using the Visual Studio Test Runner. Other than MSTest-based tests, test frameworks that have a Visual Studio test adapter, such as xUnit, NUnit, Chutzpah, can also be executed.
Tests that target the .NET core framework can be executed by specifying the appropriate target framework value.
Another choice is using .NET Core CLI task with test command, and you need to combine with Publish Test Results task to show the results in the build summary.
If you already have the right configuration. Then check the logs of test related task. For example ,if it does not generate the test results, there should also no test result to publish. Also set system.debug=true to enable verbose debug mode to get a more detail result.
Besides, also attach an official doc show Build, test, and deploy .NET Core apps in case you need.
Is it possible to run tests written in Concordion.Net using some form of Visual Studio addin? I am looking to start using Concordion.Net. However, I was used to running tests from Visual Studio directly through the Resharper plugin. Since Concordion.Net is a different concept, and uses HTML files to 'specify' the tests, one cannot use the Resharper plugin.
You can run them using the nUnit GUI, but it is a bit of a pain to use it and slow.
You can use TestDriven.NET (http://testdriven.net/) to execute Concordion.NET tests within Visual Studio. To run a test please use the "Run Test(s)" command of TestDriven.NET on your Concordion.NET fixture class (http://testdriven.net/quickstart.aspx).
Please, make sure that you have specified the [assembly: RequiredAddin("ConcordionNUnitAddin")] attribute in the AssemblyInfo.cs of your specification project as described in the how to "Run Concordion.NET Tests with NUnit" (http://concordion.org/dotnet/RunningTests.html).
When you run Concordion.NET tests with TestDriven.NET in Visual Studio, you should see an output similar to:
“------ Test started: Assembly: Concordion.Spec.dll ------
Processed specifications : C:\Users\myuser\AppData\Local\Temp\Concordion\Spec\Concordion\Command\AssertEquals\NonString\Boolean.html
1 passed, 0 failed, 0 skipped, took 0,67 seconds (NUnit 2.6.2).”
To be able to run Concordion.NET tests with TestDriven.NET you have to install the Concordion.NUnit.dll into the installation directory of TestDriven.NET (C:\Program Files (x86)\TestDriven.NET\NUnit\2.6\addins\Concordion.NUnit.dll).
Concordion.NET tests can be run with the help of NUnit (http://concordion.org/dotnet/RunningTests.html) based on the Concordion NUnit addin. When NUnit discovers a class containing Concordion.NET fixture code annotated with [ConcordionTest], the Concordion.NUnit addin loads the related HTML specification document and creates a test instance that is executed with the help of NUnit.
Unfortunatelly, ReSharper uses a different mechanism to find tests than NUnit(http://confluence.jetbrains.com/display/NETCOM/4.09+Test+Framework+Support+%28R8%29). Thus, it would be necessary to write a ReSharper plugin to recognize classes annotated with [ConcordionTest] and use the ReSharper NUnit support to execute them. NUnit 3.0 will probably comply with interfaces consumed by ReSharper.
On the other Hand, the current version of the NUnit Adapter for Visual Studio (http://nunit.org/index.php?p=vsTestAdapter&r=2.6.2) is not able to load NUnit addins appropriately. An update of NUnit is necessary, which is planned for NUnit 3.0 (https://github.com/nunit/nunit-vs-adapter/issues/9).
To run Concordion.NET tests in Visual Studio you can use TestDriven.NET as described above.
I need to run nUnit tests using a workflow template in TFS Server 2010.
I found all the reference bellow explaning how to configure the Build server:
http://morten.lyhr.dk/2008/05/using-nunit-and-ncover-with-tfs-build.html
How to run nunit with msbuild from VS2010
http://www.codewrecks.com/blog/index.php/2009/07/19/integrate-nunit-test-into-a-tfs-build/
http://msdn.microsoft.com/en-us/library/ee342461.aspx
Bellow is closest to what I need:
https://tfsbuildextensions.codeplex.com/wikipage?title=How%20to%20integrate%20the%20nUnit%20build%20activity
But the answer I need is for the question bellow:
"How to set a variable containing all the nUnit tests and how to configure a for each loop to run all the tests"
I do get all my build to run, but in the end, it does not run any nUnit test.
But which tool should I use to run the nUnit tests?:
Please let me know if you have any advice on how to get it done using the TFS 2010.
Thank you so much.
The post •https://tfsbuildextensions.codeplex.com/wikipage?title=How%20to%20integrate%20the%20nUnit%20build%20activity gives all the steps required to make use of this community TFS extensions. It is meant to be a step by step to get the activity working
Step 1 of the notes shows how to find all the assemblies that have been built, basically scan a for any file ending *.dll. I think this is the key to your question, you just pass a list of all DLLs produced by the build and let nUnit find the tests. If you want to do something else you can build the list of DLLs any way you want, but a scan seems the most simple, you don't have to worry over managing the list ypourself
Step 2a runs the nUnit test, the list of DLLs is passed into the nUnit activity (in effect the command line test runner). This will look in each DLL for the nUnit attributes and run the tests it finds.
Step 2b publishes the results back to TFS. This is done by taking the XML output of the test runer and publishing it with MSTEST to the TFS server.
If you are using TFS 2012 or later I would not use this activity, I would use the new test adaptor model built into TFS http://blogs.blackmarble.co.uk/blogs/rfennell/post/2012/03/27/Unit-testing-in-VS11Beta-and-getting-your-tests-to-run-on-the-new-TFSPreview-build-service.aspx and http://visualstudiogallery.msdn.microsoft.com/c8164c71-0836-4471-80ce-633383031099/
I was missing two steps answered in the link bellow:
Visual Studio Online CI Nunit Tests not found during build
1 - To add the nUnit adapter on each test project
2 - To add the nUnit 4.5 dlls to the build server and check it in
I am trying to convert a build system setup with TeamCity and Nant scripts to use TFS2010 (We bought the license and might just as well make use of it) After some work I get the web project to build and deploy to the web-server. We have a domain, API, test and web project in our solution.
How do I configure TFS to run the unit tests that we have written so far? I did configure the build to look for ***.UnitTest.dll in(VS2010) Edit build definition>Process>Automated Tests
Now the build fails with a message that says:"Could not load file or assembly 'nunit.framework, Version=2.5.3.9345" Am I correct when I say that TFS is trying to run NUnit on the build server? I did install NUnit-2.5.3.9345 on that TFS2010 build server and still nothing?
Thank you
Jack
The build facility in TFS uses MSTest as test runner, with which it's tightly integrated.
If you want to run your unit tests with NUnit as part of your build, take a look at the NUnit for Team Build project on CodePlex.
The project started out for TFS 2008, however support for TFS 2010 has been added in version 2.0. Note that this feature is still in early stages of development, so your mileage may vary.
I'm late to the game, because I've had to deal with this issue recently. I found this article helpful for me in this. It didn't work right off the bat, but I found if I added it into my buildscript via the controls in a similar manner/pattern, it would work.
My only problem now has been getting it to actually error (right now it warns) even when flagging them to cause the build to error
Link: http://blog.gfader.com/2011/06/running-nunit-tests-in-tfs-2010.html