My team has several Selenium Unit Tests that we run locally. Selenium opens a browser and runs through our application. This works fine locally. Now, management wants to run the tests, on demand, via TFS. Should be easy, right? WRONG!
I am having trouble setting up our TFS 2015 server to run these Selenium UI tests.
Everything I read tells me to run the "Test Agent for Visual Studio 2015" tool. Problem is, the tool does not exist on my machine and I cannot understand why. I have tried many things, like installing the MS Visual Studio "Agents", with no luck.
When I try to open the freshly installed "Agents" I get a message saying: "Test Agent for Visual Studio 2015 has no configuration tool".
So the VS.NET "Agents" panel never opens. Seems bizarre to me that I installed something that needs further configuration, with no way to perform that configuration.
From further reading, I think I need to provision a group machine in the TFS web administration page, but am totally unsure if that's required for my situation, as I cannot even get past the basics here.
I see other people have had this problem, with very little response or help from Microsoft. I am a little baffled at why this seems so very difficult to do on a TFS server? It runs locally just fine, like a breeze. But TFS? It's like a giant puzzle.
Once this is working, I need to configure the test agent to run in "interactive mode" so it can run the browser but I cannot even begin to figure that out yet. Where do you set it to "run interactive" because I dont see any of those options. Am I missing TFS installation components? Do I have the wrong Visual Studio? Do I need the Ultimate edition of Visual Studio to be able to perform UI tests with the browser? We have the Professional edition.
Here is someone with the same problem:
https://connect.microsoft.com/VisualStudio/feedback/details/1712725/test-agent-for-visual-studio-2015-has-no-configuration-tool
Does anyone have any ideas or instructions on how to setup the "Test Agent" I need to run my team's existing Selenium UI tests in TFS2015? Seems very difficult when it should not be.
In the link you provided, Allen has explained that "There is no configuration UI with the test agent anymore. This is because we have simplified the existing remote testing scenario by doing the install and configuration for you when running via the build pipeline."
TFS 2015 Update2 now have a "Run Functional Test" task that you can use to run tests (included Coded UI Tests) against machine groups. So, first you have to upgrade your TFS 2015 to TFS 2015 Update2 if you haven't.
The tasks you need for test scenarios using Visual Studio 2015 and Visual Studio Team Services (VSTS) or Team Foundation Server (TFS) 2015:
Create environments from physical or virtual machines that you've already set up.
Set up your build to run your app and tests in the environments that you created.
After your build finishes, review your test results to start resolving problems that you found.
So, your build process template should look like the screenshot below. You need to specify every task to meet the requirement of your project. All TFS tasks can be found at this website, you can get more information for each task from it. Coded UI or Selenium tests that are running on full fidelity browsers would need Interactive Process checked.:
Related
We have TFS2015 and I am trialling Visual Studio Enterprise 2015 (no installation disk).
I am now looking for an ISO with Microsoft Test Agents for Visual Studio 2015.
Documentation for VS 2015 states that these ISOs should be available:
I can get the web installer (exe) for the Agents from here, but where can I get the ISO?
Please note: I need the ISO to get hold of Test Controller 2015 to run Load Tests, so just having the Agents is not enough.
Also, I have tried using Test Agents 2015 with Test Controller 2013, and my Agent configuration fails with a time out (whilst attempting to restart the service).
And finally, I have tried using Test Agents 2013 with TFS2015, which fails with a "Registry lookup failed" error on the "Agent Deployment" build step in our build definition (the powershell scripts used in this step assume that there is a path for an installed Test Agent 2015 in your registry). This problem is fixed by providing a path to Test Agent 2015 installer in "Test Agent Location" property of that build step.
So, I am now trying to locate the ISO with Test Agents for Visual Studio 2015 - does anybody know where I can get it from, please?
It was announced that Test Agent 2015 no longer needed a Test Controller, as the agent orchestration now is managed by the server (TFS or VSTS). You can also refer this post.
Q: Will the Test Agent 2015 support all the scenarios supported by
Test Controller and Test Agent of Visual Studio 2013?
A: We recommend you use Agents for Visual Studio 2015 in all the new automated testing scenarios. You can use the Deploy Test Agents task in a build definition to download and install the test agents on your machine. The following table shows the scenarios supported by Agents for Visual Studio 2013 and the alternatives for Team Foundation Server (TFS) 2015 and Team Services (TS).
For your situation, seems you are using on-premises Load Test , you need to use Test Controller/Test Agentes2013 Update5 . More information, please refer Using a Test Controller and Test Agents in a Load Test
As for ISO files, it seems agent for VS2013
are using the .iso including test agents and test controls in download center. But for agent for Vs2015 are just using .exe in download center.
Yes, of course, go to the page below:
https://www.visualstudio.com/vs/older-downloads/
at the scroll down the page you will see VS2015, after clicking download you will need to sign in,
then could choose the corresponding version you want to download:
do remember to sign in to Microsoft for downloading agents.
https://my.visualstudio.com/Downloads?q=Visual%20Studio%202015%20Update%203
I am having a Windows application which was created on VB.Net and Visual Studio 2008, The application is somewhat big and is around 10 years old. The repository we are using is TFS. I am having a task to create an automatic build for this Windows application and I choose the Jenkins CI for it.
My plan is to build the project solution using MS Build plugin and then publish it and deploy the solution to IIS path.
I have given the below MS Build query to build using Jenkins and get the solution code from TFS and the output was successful:
/t:AppProcSolution /p:Configuration=Release /maxcpucount
But I need publish the same AppProcSolution. Could I do it by passing any other parameters to the above script or should I need to use MS deploy etc. I am totally new to automatic integration. Is it possible for me get the published solution to a particular folder? Almost all the .Net integration using Jenkins tutorial available on the Internet is for deployment to GitHub etc. So if someone has any guidelines to help please provide me a solution.
MSDeploy is good for server application deployments like web applications/sites, SQL server databases or Windows Services.
When you say "Windows application" I assume you mean a WinForm or desktop client application. The best tool for this, in my opinion, is ClickOnce:
https://msdn.microsoft.com/en-us/library/142dbbz4(v=vs.90).aspx
You could also use a Setup project in VS:
https://msdn.microsoft.com/en-us/library/ms235317(v=vs.90).aspx
I am testing web based products, and I am new to Microsoft tools. I am trying to figure out how test plan creation works, or can work depending on what configuration of Visual Studio tools I am using. I find a lot of conflicting info online.
I have the three tools below installed right now, but I can request something different if I need it. (Feel free to suggest). Any advice from pros more experienced with this tool set would be GREAT.
I access Team Foundation Server 2012 via my browser, where I can view and add product backlog items, bugs, sprints, the board etc. I cannot add test plans here, but I can add test cases to them.
I have Visual Studio 2013 installed on my PC. This seems like pretty much the same thing as the browser accessible tool, and I don't use it much because I find it easier to work from the browser for testing web.
I have Microsoft Test Manager 2012 installed on my PC. This appears to be the only place where I can create test plans, cases, and can create and utilize shared test case steps.
Questions:
How can I create test plans using the browser tool?
Is it possible to update the TFS 2012 Browser Tool to 2013, and how is that done?
Additionally to the #jessehouwing answer I would highly recommend you this free ebook:Testing for Continuous Delivery with Visual Studio 2012
Team Foundation Server 2012 update 2 adds Web Based test Case Management to TFS 2012, this is a smaller step than an upgrade to TFS 2013 all the way. You cannot just upgrade the Web Access part of a TFS installation, it would require a complete upgrade of the cluster. You also cannot just install this to your machine, it has to be installed on all the TFS servers in the cluster.
Traditionally Test Manager is the place to go to to do Test Suite Management and to view the results of a test run. Microsoft has started to move the functionality traditionally found in Test manager to the Web portal, It seems you're not the only person who found that easier to use. This transition started with TFS 2012 update 3 and was further expanded with TFS 2013.
So your options are:
Use Test Manager (you'll have to use it to manage certain things you cannot yet do on the web anyway).
Ask your Server Administrator to upgrade TFS 2012 to at least Update 2 (I'd recommend he upgrades to TFS 2012 update 4 or TFS 2013 in any case).
Ask your Server Administrator to upgrade TFS 2012 to TFS 2013.
The following resources can be very helpful in understandign how to use the tools to their fullest potential:
Visual Studio ALM Rangers' Test Guidance
Patterns & Practices' Testing for Continuous Delivery with Visual Studio 2012 (as mentioned by #Elena)
Microsoft Virtual Academy Software Testing with Visual Studio 2012 (exam 70-497 jump start)
Please pardon a newbie's question about MS TFS and Visual Studio 2012, I hope I'm using the correct vocabulary.
Our shop has recently moved to MS TFS and Visual Studio 2012 to build internal web services for integration. Currently we have a TFS 2010 Server running on Server2008R2 and a Build server on a Windows 7 box, our dev application server is also running Server2008R2. I have been given the job of getting a nightly build and deployment running. I have the build definition working but the deployment portion is not functioning. I have installed Web Deploy on both the dev application server and the build server as per these articles:
http://www.kevingao.net/version-control/how-to-auto-deploy-web-application-with-tfs-build-server.html
and
http://www.iis.net/learn/install/installing-publishing-technologies/installing-and-configuring-web-deploy
We have configured an user, web_deploy_dev in active director and that user is a local admin on the dev app server.
The parameters added to the build definition are:
/p:DeployOnBuild=True
/p:DeployTarget=MsDeployPublish
/p:MSDeployServiceURL="https://PA-APPSRV1-DEV:8172/msdeploy.axd" /* double quotes not in original code */
/p:DeployIISAppPath="DW/DWServices" /* E:\Webroot\DW\DWServices */
/p:CreatePackageOnPublish=True
/p:MsDeployPublishMethod=WMSVC
/p:AllowUntrustedCertificate=True
/p:UserName=migp\web_deploy_dev
/p:Password=******
I suspect that MSBUILD is not even calling msdeploy.exe as I see no errors in the windows event and application logs, but I do not know all the places to look. I can use a web browser and connect to the service on the dev app server as the web_deploy_dev user from the build server, I receive a web page not found error after inputting user name and PW, but I assume that is to be expected.
I am at a loss as to where to look next, I've tried searching the web but nothing I try seems to work.
Thank you in advance for your help and input.
Roy
It sounds like you are experiencing a problem where your build completes and claims success, but you aren't getting any error and nothing is actually deployed. When I've experienced this with my TeamCity build server, the problem was that MSBuild was not set up on the machine that would do the building and deploying. These are my steps to resolve the problem I think you might be having:
Make sure that at least Visual Studio Express is installed. you can tell if a full enough version of Visual Studio is installed by navigating to the
C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\ directory. If you see folders like 10.0, 11.0, or '12.0', look inside them and verify that there is a Microsoft.WebApplications.targets file. Note the versions that pass that check.
In your Build Configuration, add a /p:VisualStudioVersion=12.0 or whatever version of Visual Studio is appropriate.
Let me know if this gets you anywhere or not. I'm really curious to hear if you are finding any errors or logs anywhere. Basically, in my experience using Web Deploy and MSBuild with TeamCity, if the machine that is building and deploying doesn't have the MSBuild installed with the Microsoft.WebApplication.targets file, it silently fails.
I have setup a tfs 2012 server with the build controller and the build agent. I have a simple visual studio 2012 solution with a windows console application project and a test project on a client machine. Test impact analysis is enabled in the build definition.
Simply put, the impacted tests list that should appear on each build performed on the tfs server is never populated. I have tried to change the test runner from vs runner to mstest to no avail.
Please advise. Thanks.
EDIT: I installed VS 2012 on the server. This enabled code coverage, but still no impact analysis.
Fortunately I fixed the 'problem'.
For TIA to work, you need to successfully complete a test case in test manager, passing all test steps. Only then does the analysis file get generated. Also, I had problems with the video recording module of the test controller, which apparently also prevents the file from being generated sometimes, though this could just be a coincidence.
After successfully generating the file, subsequent builds will now have a baseline to compare against.
Have fun.