We have a CI setup using bamboo where one job was to run our automation, it checks out our Automation Solution from a repository, and run it using NUnit runner where it contains tests using selenium webdriver and some unit tests. It runs unit tests(not using webdriver commands) just fine, but not the webdriver tests.
We are sure that the solution is being checkout correctly because we can run the test using the command line referencing the created local repository, including tests that uses webdriver using the command below:
C:\pathToNUnitConsole\nunit-console C:\pathToCreatedLocalRepositoryInTheServer\NUnitTest.dll -include=FirefoxBrowserTest
We can see that the bamboo job never launched a browser, and we are getting test failure for not seeing browser elements. Also, we are getting error as we see in the logs:
10-Apr-2015 13:19:49 http://10.215.12.28:15871/cgi-bin/blockpage.cgi?ws-session=587410860
10-Apr-2015 13:20:42 JavaScript error: resource://gre/modules/CertUtils.jsm, line 168: NS_ERROR_ABORT: Certificate issuer is not built-in.
I don't know if this is your problem, but it sounds very similar to an issue I had trying to get NUnit runner to run tests built in SpecsFor.MVC, which is a wrapper that deals with instantiating an IIS Express instance, deploying a target web application to known location, then using Selenium/Chrome Driver to run the tests against the web app.
The problem I had was trying to run the Bamboo remote agent as a windows service. The Local Service account was not able to access the interactive desktop. Also, changing the service to run as a specific user account still failed, as the service was not allowed access to the user's isolated Session0.
I could login as that test user and run the test from the command line no trouble, but I couldn't get it to run when initiated by the remote agent windows service.
I eventually fixed it by using AlwaysUp (not a plug. I'm sure other similar solutions are available). This allowed me to set the machine running the tests to always log in as the test user on startup, would automatically restart the bamboo remote agent if it was closed for some reason and most importantly, to give the service access to the user's isolated session0.
Hope that might give you a few clues...
Related
Currently using Selenium + ChromeDriver to execute automated functional tests through Azure DevOps VsTest task but whenever the task tries to execute tests pointing to a website in NetCore, it times out very quickly and sometimes (a lot of times) I get the standard error message from the registered screenshot on error of "This site can't be reached".
I tried running this locally and pointing to the same website but this issue doesn't seem to happen at all and I've got no clue on what's going on due to the big inconsistency between the results that the VsTest task provides.
I also tried opening Chrome on the machine that executes the task but it seems to open the URL fine there.
One thing to note is that the website I'm using uses Windows Authentication mechanism and I'm sending my credentials directly through the URL with https:\\credentialuserexample1:credentialpassexample2#thewebsiteaddress.com and it seems to open the page without issue there too.
I'm a QA Engineer where I work and I want to enable all the other
team members to run the tests without me- i.e - even when I'm not here- they can for example get to some url, enter the url of the environment they want to test and play the tests, without installing anything on their computers.
what would be the best practice?
thanks.
Sounds like Jenkins will do what you want.
In that situation I would:
Install Jenkins on a dedicated machine (onsite or in the cloud), or try out a commercial Jenkins cloud provider
Configure a Jenkins job that executes your tests (you can configure the tests to run on the same machine where Jenkins is installed, or if you would rather have Jenkins run the tests on a second machine, that's possible too)
Show your team members how to access Jenkins and launch the test job (or, you can also configure your test job to run automatically at a certain time of day, or even better, when changes are made in your version control system)
I'm working on a suite of Selenium WebDriver (not Selenium RC and not using Selenium-Server) tests that are being executed by a command-line call to MSTest. I've broken them into test classes for each browser we plan on testing (Chrome, Firefox and IE9, for now) and each test class has a startup method that logs in to the system under test and then executes the test(s) requested and a TearDown method that closes the browser using WebDriver.Quit().
On my development machine everything is hunky-dory, but on our build machine the class initialization/start-up method executes fine and logs into the site, but as soon as the first actual test starts the test runner closes with the error message:
OpenQA.Selenium.WebDriverException: Unexpected error. System.Net.WebException: Unable to connect to the remote server ---> System.Net.Sockets.SocketException: No connection could be made because the target machine actively refused it 127.0.0.1:59982
obviously the port changes depending on which browser's being tested; when I run the whole suite I get errors on different ports for the tests for IE, Firefox, and Chrome - two errors for each browser (as the suite currently has two tests for each browser)
Windows Firewall is off for the build machine, and checking netstat before running the tests shows that the port's not in use; if I debug the tests and stop after the startup code runs but before the test runs, of course the port is in use. 127.0.0.1 and localhost are the same as well.
I get the same error whether or not I call the test via the command line, run/debug it from within Visual Studio using the VS Test tools, or by executing my console app, which calls MSTest via Process.Start(). The exception is thrown on the first FindElement() call.
Where can I start looking to figure out why this is failing on the build machine and not on my dev box?
The issue here was that in trying to implement some more robust error handling during startup I place a call to the Teardown code in the finally block of a try/catch wrapped around the login code, so as soon as the test harness was done logging in I was closing and cleanin up the WebDriver instance, so when the test ran there wasn't a WebDriver there to communicate with.
I suspect when I as looking at the "same" code on my dev machine the libraries that were actually running didn't have that block in them.
When you run an application from Visual Studio it fires up a server listening on a certain port. In your case 59982.
You need to actually have your web application running on its own on the test environments before running your test against it.
On your dev machine your application is running on that port. I'm guessing on your remote machines that is not the case.
i integrated my selenium scripts (using junit) with Hudson. i invoked my job through ant . My problem was my scripts are running successfully but the fire fox browser is not getting opened. At this moment i am not using selenium grid .please provide any suggestion
Check whether you run Hudson as a service using your own account. If you want to see Firefox browser you must select Local System Account as a service runner (Log on option in service properties) Then there is a possibility to check interact with desktop checkbox and it makes browser "visible" while running your tests.
I'm running Hudson for my automated builds and love it.
I'd now like to create automated Web browser tests using either WaTiN (preferred) or Selenium.
As my Hudson runs as a Windows service (under Tomcat), how can I configure this to run? Will I be forced to log in as a user on the Hudson box so my automated browser tests will find a browser? Or is there some other sorcery with an interactive session that I can conjure up?
If possible I'd like the Hudson box itself to run the tests.
After some research I've found that WaTin won't work so well as I'd need to configure Hudson to run in an interactive Java session.
Selenium Grid, however, is perfect - there's a Hudson plug-in that can control it, too.
So that's what we'll use.
I'd now like to create automated Web
browser tests ... my Hudson runs as a
Windows service ... Will I be forced
to log in as a user on the Hudson box
so my automated browser tests will
find a browser? Or is there some other
sorcery with an interactive session
that I can conjure up? If possible I'd
like the Hudson box itself to run the
tests.
A Windows service can indeed run a browser, even when logged in as the LocalSystem account. We have that exact setup, with Hudson installed as a Windows service, although deploying it to Tomcat and running that as a service should work equally well. We use Sahi for our automated testing, which runs our test suite in both IE and Firefox.
You didn't say what problem you had with Watin, but in case it's relevant, Sahi needs to set the browser's HTTP proxy to a special host and port. This works no problems with Firefox because Sahi configures its proxy via a profile specified on the Firefox command line. IE is a little tricker, because its proxy settings are in the Windows registry, and when running as the LocalSystem account, these come from a special place in the registry (not HKCU as for a logged-in user). See this SF answer for the details. If that was the problem you were having, you may yet be able to achieve your goal of running everything on the Hudson box.
Have you checked here? This shows the Selenium Grid plug-in to Hudson.