Has anyone successfully used the MSpec Test Adapter with TFS? If yes, can you please shed some light on how I would go on about doing this?
I noticed this post but it uses the console runner.
After lots of research and trial and error, I finally figured how to make this work. It's actually pretty easy.
Install the MSpec Test Adapter on the build server.
Download the MSpec Test Adapter and change the file type to .zip
Extract the files to a folder on your build server.
Point the "Version control path to custom assemblies" to the files in step 3 above. *Note: You can access the window to do this from Visual Studio by clicking Build > Manage Build Controllers > and selecting the controller.
Ensure in the build definition, you select Visual Studio Test Runner. This can be accessed from the test definition > Process tab > Test > Automated tests > Test source
When you run your build, you should see your tests run.
Related
I am having an issue getting selenium end to end tests to work after an automated deployment using visual studio team services (VSTS).
I have a build working that generates a build artefact. This is triggered from VSTS but runs on an on premises build server. I have a deployment working that deploys to an on premises development web server. All this works including unit tests running after the build.
When I try to add testing after the deployment is when I run into the problem. The tests are to be run on the build server and point to the dev server website. The deployment has two phases. A deploy and then an agent phase that runs a test assemblies task using the build agent on the build server. The problem seems to be that the test dll's are not being included in the build artifact and so are never found when the test process runs. Deploy setup us as follows.
I have a copy files before the publish artifact in the build definition that seems to copy the files in to the right place but they are not included in the zip file artefact. I've looked at several websites and posts on here but I still seem to be missing a vital bit of knowledge that will get this working.
Use of the log did help as recommended so thanks for that.
I have managed to get this working. I separated the selenium tests out into a separate solution and built that separately creating it's own build artefact. I then added this to the agent task in the deploy. This worked. The only thing I need to get sorted now is the correct search path to find the test DLL's. It's not quite as dynamic as I would like at the moment. I can play tunes on that until I get it right though.
I accept this is working around rather than solving the original problem but needs and timescales must. I think moving the end to end UI tests out of the main solution makes sense anyway but no doubt others may think differently.
Thanks for your help everyone
I am using Visual Studio Team Services to carry out an automated build, and using SonarQube to display Code Quality, Coverage, etc. I am also using a privately hosted build agent.
The build steps all work successfully with data being processed and populated through to SonarQube which is great. However, no code coverage is being displayed in SonarQube. After looking through the logs in VSTS I found that SonarQube is looking for the .trx file (which contains code coverage) in a different directory to the one that VSTS publishes the .trx file to.
So when VSTS builds the solution it creates the test results file here: C:\agent_work\3\s\TestResults
But SonarQube is trying to use test results from here: C:\agent_work\3\TestResults
On the build server, if I manually copy the .trx file into the correct location and then run the build again, the code coverage all works fine and processes through to SonarQube. So the issue is definitely the mismatch of the locations where the .trx is published and where it is to be picked up form.
I can't find a way of changing the publish location, or the SonarQube source location.
Please help!
This is a known issue, and will be fixed by the next release. See: https://jira.sonarsource.com/browse/SONARMSBRU-262.
We just announced the RC versions of the products that have this fix, you can give them a try. See this thread.
As I'm using Selenium/MSTest for UI tests, I got a problem: I need to deploy an ASP.NET site to a staging server just before the automated tests are executed during a TFS build (TFS 2012).
Although I thought I could do this configuration in the TFS Build process template (DefaultTemplate.xaml), I can't figure out how to change the order to execute a Build->Deploy->Test flow.
Note I've found some how-tos aiding in this goal when using TFS Lab Environment, but this isn't my case.
Default Template Unit Testing is different from UI Testing. Default template is designed to run Build -> Unit Test -> Deploy.
You are now in Build -> unit Test -> Deploy -> UI Test.
For UI Testing, You need separate Test Agent and set up build agent in interactive process.
These links may help
http://www.deliveron.com/blog/post/Configuring-a-TFS-2010-Team-Build-Server-to-Run-Coded-UI-Tests.aspx
http://blogs.infosupport.com/testing-your-web-application-with-selenium-and-mtm-part-1/
http://blogs.infosupport.com/testing-your-web-application-with-selenium-and-mtm-part-2/
Finally I've solved the problem chaining builds.
For example, when I queue a production build, this is the sequence:
Build, deploy to staging without UI tests
Build, no deploy, staging UI tests only
Build, deploy to production without UI tests
Build, no deploy, production UI tests only
I got the solution from this blog post:
http://blog.stangroome.com/2011/09/06/queue-another-team-build-when-one-team-build-succeeds/
Post's author posted the modified DefaultTemplate.xaml (build definition) in GitHub Gist:
https://gist.github.com/jstangroome/1196590/
Credits for the whole blogger!
One simple solution is to have an intermediate msbuild project file that is specified in your build definition and from there, call the actual solution file.
You can then do whatever is required pre- or post- build to ensure that the environment is up and running prior to TFS executing the tests. We do things like compile and deploy databases prior to TFS running our integration tests.
This is one solution that avoids having to manipulate the build template.
Our department uses Visual Studio 2008 Team System, and we have a build server that integrates with our TFS source control server. It pulls the source code, builds the solution and runs the unit tests, just as we might do from within VS, and emails a report. The build server is setup using MSBuild and MSTest as the primary tools. All very sweet.
On our development machines we also run a set of selenium unit tests, and I want to include this in the test suite on the build server. I have been told that 'this is not possible using MSBuild/MSTest', but I am at a loss to understand why.
Does any one have experience of running selenium tests (they are just conventional test methods written in C#) who might be able to advise me on whether this is possible and what the gotchas are? Thinking about it, apart from giving the browser access to the desktop when the server is not logged in, once MSBuild has handed off a test list to MSTest it's exactly the same process as on our develop machines.
TIA
I know it's 3 years on, but someone might drop in on this post and not see an answer. This is possible todo.
In a similar fashion how you'd run unit test, in the build definition using default template you need to specify the name of the UnitTest DLL and ensure Run Unit Test is not disabled. Also ensure that the build is building your Automated UI Test solution.
Simples.
I have been creating codedUI tests for a website in VS2010.
What I would like to do is add these tests to our build process as a form of gated checkin. So if new code produced failed tests then it this code rejected.
does anyone have any experience or know of any resouces regarding this setup.
Thanks in advance
If you are using TFS for your builds you can easily integrate the tests into your build process. If you are using something else for your build process you will need to have MSTEST installed on the build server and another machine to run the tests as an agent.