I require my VSTS build server, or more accurately my build agent, to execute a task under a specific user account. Example:
One of my build steps tests the UI of an application. It requires to be logged in under a different account to be able to execute tests correctly. I cannot find anything in VSTS that allows me to configure that.
My current workaround is to run my tests in a different thread, started via a PowerShell command; however, this solution feels dirty, is fairly complicated, has limited live reporting and costs work to re-integrate the test results.
Is there any better way, maybe one that work straight out of the box via VSTS?
Note: Changing the build agent to run under a different account doesn't work because I need to run tests under multiple accounts long-term to test various authentication features.
You can run the test on remote machine by using Visual Studio Test Agent Deployment and Run Functional Tests tasks.
With this way, you can specify username and password that the test agent running on (Visual Studio Test Agent Deployment task).
Related
I have to test the project code using Jenkins. In Jenkins, I am using automation testing throw selenium. then my question is I have test scripts on my laptop and using Google Cloud virtual machine SHH i have to set up and test in every push on git. Here is my 2 demand-
1. on demand-whenever we want all the test run we should just trigger the test run.
2. Whenever we deploy something on staging then test all the test cases
Regarding (1), you are always able to trigger a job manually through the Jenkins UI. No special configurations there.
Regarding (2), you can install a plugin that will integrate webhooks functionality into Jenkins. In my case, I like to use Generic Webhook Trigger for this purpose, as it has the flexibility that I need on my setups.
In order to trigger the job on every deploy to staging, and assuming that your deploys are automated, you will need to add a final step on the deploy script, to make an HTTP request to the webhook URL (eg. JENKINS_URL/generic-webhook-trigger/invoke?token=<your-token>
I don't fully understand your setup with your machine and the VM on GCloud, in any case, I believe that the test code should be available to the machine running the tests, and not be stored in a location that might be unavailable when the tests need to be run (as your laptop might be).
There should be a way where user should be able to access test cases and select or customize them online before running. Or in simple terms is it easy to use feature files online.
I think what you want is Jenkins
After you set up your Jenkins server you can access it though a web page, create jobs, and run them. There are also reporting plug-ins like Cucumber Reports that make nice, easy to read, reports.
We have a TFS (2013) build defined which runs a project of Selenium tests against the web site in the solution (chromedriver). If run locally, by any developer, three of the tests currently fail. The build agent is on a separate box from the tfs build controller and running under a local admin account.
With the code all checked in, if the build is manually triggered by three of the developers, it works as expected - three tests fail. However, if the build is launched by either of the other two developers, almost all of the tests fail, either with a...
System.InvalidOperationException: unknown error: cannot focus element
or
OpenQA.Selenium.NoSuchElementException: no such element
...type of error.
All developers have the same rights in TFS. Can anyone suggest how the id of the requesting user should be able to influence the results of a build being run remotely and under different credentials? (Any solutions would also be appreciated!)
It would appear that this is as a result of screwed up AD Groups on the network.
Since the originating user's credentials are actually carried over to the build process, rather that the local admin credentials being used, we (eventually) discovered some different AD memberships of groups between the developers which impacted the permissions on some folders on the build box, stopping certain resources from being accessed.
Unfortunately the errors raised by Selenium were complete red herrings and hiding the actual cause - difficult to diagnose until we actually ran the builds locally on the build server under the various developers' credentials.
We have a CI setup using bamboo where one job was to run our automation, it checks out our Automation Solution from a repository, and run it using NUnit runner where it contains tests using selenium webdriver and some unit tests. It runs unit tests(not using webdriver commands) just fine, but not the webdriver tests.
We are sure that the solution is being checkout correctly because we can run the test using the command line referencing the created local repository, including tests that uses webdriver using the command below:
C:\pathToNUnitConsole\nunit-console C:\pathToCreatedLocalRepositoryInTheServer\NUnitTest.dll -include=FirefoxBrowserTest
We can see that the bamboo job never launched a browser, and we are getting test failure for not seeing browser elements. Also, we are getting error as we see in the logs:
10-Apr-2015 13:19:49 http://10.215.12.28:15871/cgi-bin/blockpage.cgi?ws-session=587410860
10-Apr-2015 13:20:42 JavaScript error: resource://gre/modules/CertUtils.jsm, line 168: NS_ERROR_ABORT: Certificate issuer is not built-in.
I don't know if this is your problem, but it sounds very similar to an issue I had trying to get NUnit runner to run tests built in SpecsFor.MVC, which is a wrapper that deals with instantiating an IIS Express instance, deploying a target web application to known location, then using Selenium/Chrome Driver to run the tests against the web app.
The problem I had was trying to run the Bamboo remote agent as a windows service. The Local Service account was not able to access the interactive desktop. Also, changing the service to run as a specific user account still failed, as the service was not allowed access to the user's isolated Session0.
I could login as that test user and run the test from the command line no trouble, but I couldn't get it to run when initiated by the remote agent windows service.
I eventually fixed it by using AlwaysUp (not a plug. I'm sure other similar solutions are available). This allowed me to set the machine running the tests to always log in as the test user on startup, would automatically restart the bamboo remote agent if it was closed for some reason and most importantly, to give the service access to the user's isolated session0.
Hope that might give you a few clues...
I'm a QA Engineer where I work and I want to enable all the other
team members to run the tests without me- i.e - even when I'm not here- they can for example get to some url, enter the url of the environment they want to test and play the tests, without installing anything on their computers.
what would be the best practice?
thanks.
Sounds like Jenkins will do what you want.
In that situation I would:
Install Jenkins on a dedicated machine (onsite or in the cloud), or try out a commercial Jenkins cloud provider
Configure a Jenkins job that executes your tests (you can configure the tests to run on the same machine where Jenkins is installed, or if you would rather have Jenkins run the tests on a second machine, that's possible too)
Show your team members how to access Jenkins and launch the test job (or, you can also configure your test job to run automatically at a certain time of day, or even better, when changes are made in your version control system)