I have tests written with TestCafe Studio (.testcafe) format. I want to execute my tests concurrently for the pipelines, where I'm using ** -c 2 ** with the node.js
"test": "NODE_TLS_REJECT_UNAUTHORIZED=0 testcafe -c 2 chrome --skip-js-errors --assertion-timeout 20000 --selector-timeout 20000 Tests/**/*.testcafe",
This will open up 2 instances of chrome browser and execute my tests parallel. Which is what I want exactly.
But when I execute tests on remote browser, I'm not sure how I need to configure the concurrency so that it will run my tests parallel.
I tried below with node.js (-c 2 remote:2), where I expect that I can execute my tests parallel in a remote browser
"test-ie": "NODE_TLS_REJECT_UNAUTHORIZED=0 testcafe -c 2 remote:2 --skip-js-errors --assertion-timeout 60000 --selector-timeout 60000 Tests/**/*.testcafe",
But when I execute this I just got one single connection URL, and its not starting the tests. ( But the connection details includes Connecting 2 remote browser(s)... )
`> NODE_TLS_REJECT_UNAUTHORIZED=0 testcafe -c 2 remote:2 --skip-js-errors --assertion-timeout 60000 --selector-timeout 60000 Tests/**/*.testcafe
Connecting 2 remote browser(s)...
Navigate to the following URL from each remote browser.
Connect URL: http://192.168.1.1:64828/browser/connect
CONNECTED Chrome 89.0.4389.90 / macOS 10.15.7`
How can I get this configured to run test parallel in a remote browser?
Thank you
Your command syntax is correct. TestCafe outputs a single URL to connect all remote instances. After you open this URL in the first browser window (and see the first CONNECTED message), open the second browser window and navigate to the same URL. The console will display the second CONNECTED message and tests will start.
Related
I have created the Jenkins job to execute the TestCafe tests, the job is working fine.
But I want to execute multiple jobs parallelly.
As TestCafe runs tests on the default port, it is not allowing me to execute jobs parallelly.
Can anyone please suggest how to achieve this?
I tried to execute the jobs parallelly, but its is giving an exception port already in use.
I am not able to change the port in Jenkins job
When starting TestCafe, you can set ports:
https://testcafe.io/documentation/402644/reference/testcafe-api/testcafe/createrunner
https://testcafe.io/documentation/402639/reference/command-line-interface#--ports-port1port2
Use different ports for each parallel task.
I have a test system (various web pages / web applications), that is hosted in an environment accessible only via machines with IP addresses that are white listed. I control the white list.
Our CI system is cloud hosted (Gitlab), so VMs are spun up dynamically as needed to run automated integration tests as a part of the build pipeline.
The tests in question use BrowserStack automation to run Selenium based tests, which means the source IP addresses of the BrowserStack automation driven requests that hit the test environment are dynamic, as BS is cloud hosted. Also the IP addresses of our test runner machines that call / invoke the BrowserStack automation are dynamic as well.
The whole system worked fine before the intro of IP white listing on the test environment. Since white listing was enabled, the BrowserStack tests can no longer access the environment URLs (due to not being able to white list the dynamic IPs).
I have been trying to get the CI driven tests working again using BS "Local Testing" feature, outlined here https://www.browserstack.com/local-testing.
I have set-up a dedicated Linux VM with a static IP address (cloud hosted). I have installed and am running the BrowserStackLocal.exe binary, using our BS key. It starts up fine and says it has connected to BrowserStack via a web socket. My understanding is this should cause all http(s) etc requests that come from my CI / BrowserStack automation driven tests to be routed through that stand-alone machine (via BS cloud), resulting in it's static IP address being the source of the requests seen at the test environment. This IP addr is white listed.
This is the command that is running on the dedicated / static IP machine:
BrowserStackLocal.exe --{access key} --verbose 3
I have also tried the below, but it made no apparent difference:
BrowserStackLocal.exe --{access key} --force-local --verbose 3
However, this does not seem to work? Either through "live" testing if I try and access the test env directly through BrowserStack, or through BS automate. In both cases the http(s) requests all time out and cannot access our test environment URLs. Also even with --verbose 3 logging level enabled on the BrowserStackLocal.exe process, I never see any request being logged on the stand-alone / static IP machine when I try to run the tests in various ways.
So I am wondering if this is the correct way to solve this problem? Am I misunderstanding how to do this? Do I need to run the BrowserStackLocal.exe perhaps on the same CI runner machine that is invoking the BS automation? This would be problematic as these have dynamic IPs as well (currently).
Thanks in advance for any help!
EDIT/UPDATE: I managed to get this to work!! (Sort of) - it's just a bit slow. If I run the following command on my existing dedicated / static IP server:
BrowserStackLocal.exe --key {mykey} --force-local --verbose 3
Then on another machine (like my dev laptop) if I hit the BS web driver server http://hub-cloud.browserstack.com/wd/hub, and access the site http://www.whatsmyip.org/ to see what IP address comes back, and it did (eventually) come back with my static IP machines address! The problem though is it was quite slow - 20-30 secs for that one site hit, so still looking at alternative solutions. Note for this to work your test code must set the "local" browserstack capability flag to 'true' - eg for Node.js:
// Input capabilities
var capabilities = {
'browserstack.local' : 'true'
}
UPDATE 2: Turning down the --verbose logging level on the local binary (or leaving that flag off completely) seemed to improve things - I am getting 5-10 sec response times now for each request. That might have to do. But this does work as described.
SOLUTION: I managed to get this to work - it's just a bit slow. If I run the following command on my existing dedicated / static IP server (note adding verbose logging seems to slow things down more, so no --verbose flag used now):
BrowserStackLocal.exe --key {mykey} --force-local
Then on another machine (like my dev laptop) if I hit the BS web driver server http://hub-cloud.browserstack.com/wd/hub, and access the site http://www.whatsmyip.org/ to see what IP address comes back, and it did come back with my static IP machines address. Note for this to work your test code must set the "local" browserstack capability flag to 'true' - eg for Node.js:
// Input capabilities
var capabilities = {
'browserstack.local' : 'true'
}
So while a little slow, that might have to do. But this does work as described.
I'm using Codeception to run some acceptance tests using WebDriver and a Docker Selenium standalone server. I noticed that after one of my tests threw an error I get this logged:
[Selenium browser Logs]
13:59:52.345 SEVERE - https://ssl.google-analytics.com/ga.js - Failed to load resource: net::ERR_CONNECTION_CLOSED
Funny thing is that there isn't any google analytics on the page at all (or the app for that matter). I'm guessing that there is some anonymous usage analytics but can't tell whose it is, whether it's Selenium, Codeception or it's included libs, or even the ChromeDriver for Selenium. I run the Selenium server with --net=host and maybe that has something to do with it.
Anyone know how to turn these analytics off or let Codeception ignore failed external resources?
add the following entry to your local hosts file,
127.0.0.1 www.google-analytics.com
127.0.0.1 google-analytics.com
127.0.0.1 ssl.google-analytics.com
This works for all browsers, regardless of whether they support plugins. As long as you don't run a web server on your local machine, these connections are instantly rejected and so don't take very long to fail.
You could also try 0.0.0.0 (never personally tested by me though).
0.0.0.0 www.google-analytics.com
0.0.0.0 google-analytics.com
0.0.0.0 ssl.google-analytics.com
PS: 0.0.0.0, a null route, is better. If you're running your own web server you'll start to get 404s if your route to localhost.
I have a docker setup for nightwatch.js to run selenium tests through selenium grid for an express server application. When I just use a docker-compose up and then run my nightwatch tests manually after the server starts everything appears to start properly. If I run them as part of a containers command (ie. in my app server containers command or in a new container based on it that just runs nightwatch) then I get a 503 bad gateway error. I think there is a race condition in my docker setup that is causing this. Is there a way to guarantee my app server starts properly before running my nightwatch tests?
Prefix your command with wait-for-it.sh script (download it a and put in the image):
command: /wait-for-it.sh theotherservice:PORT -- your-previous-command
wait-for-it.sh will wait to the specified server:port to execute the command that is after --, so you can avoid the race condition.
Pardon me for the lengthy description:
I have the following infrastructure where I am attempting to deploy the Selenium grid via Jenkins. For the sake of simplicity, I am using simplified IP addresses.
Machine 1 (Jenkins):
IP: 1.1.1.1
Machine 2 (The Test Script + The Firefox RC instance resides on this)
IP: 2.2.2.2
Machine 3 (The IE RC instance resides on this)
IP: 3.3.3.3
Now, I am starting instances of RC as follows:
Machine 2:
RC 1: ant -Dport=5555 -Denvironment=*firefox -Dhost=2.2.2.2 -DhubURL=http://1.1.1.1:4444 launch-remote-control
Machine 3:
RC 2: ant -Dport=5556 -Denvironment=*iexplore -Dhost=3.3.3.3 -DhubURL=http://1.1.1.1:4444 launch-remote-control
After doing this, I am able to establish connection on the Jenkins Selenium console for both Machine 2 and Machine 3.
As far as my test script (residing solely on Machine 2) is concerned -
For firefox,
selenium = new DefaultSelenium("1.1.1.1",4444,"*firefox","http://mytestURL.com")
For IE,
selenium = new DefaultSelenium("1.1.1.1",4444,"*iexplore","http://mytestURL.com")
The problem is that I unable to get the test script to execute on Machine 3, where the IE instance resides.
What settings do I have to make in the DefaultSelenium object to get the test script residing on Machine 2 to execute on Machine 3, via the Jenkins grid?
Thanks for looking.
The problem was with the host's IP address on Machine 3. This works as expected.