Once I've built my container with my Meteor app in it, I'd really like to be able to go
docker run me/myapp velocity test-app --ci --once --settings settings-test.json
And have it exit with 0 if successful, in which case I'll push it to docker hub, deploy it somewhere etc.
However when I try this it just hangs:
[velocity] is in continuous integration mode
[velocity] mocha is starting a mirror at http://localhost:56381/.
[velocity] *** Meteor Tools is installing ***
This takes a few minutes the first time.
[velocity] You can see the mirror logs at: tail -/app/.meteor/local/log/mocha.log
I'm using jasmine as per https://github.com/meteor-velocity/velocity-examples (I started with Mocha, but switched over to see if it made any difference).
Inspecting my .meteor/local/log files I find jasmine-client- unit.log has this at the bottom:
WARN [watcher]: [39m Pattern "/app/tests/jasmine/client/unit/**/*-+(stub|stubs|mock|mocks).+(js|coffee|litcoffee|coffee.md)" does not match any file.
WARN [karma]: [39m No captured browser, open http://localhost:9876/
INFO [karma]: [39m Karma v0.13.9 server started at http://localhost:9876/
INFO [launcher]: [39m Starting browser Chrome
ERROR [launcher]: [39m No binary for Chrome browser on your platform.
Please, set "CHROME_BIN" env variable
Parent process ( 725 ) is dead! Exiting jasmine-client-unit
Chrome clearly isn't going to be available in docker - should phantomjs be installed at this point and specified as a the running option? I would have expected this to be the case by default if the --ci option has been specified?
Thanks.
Ideally you should be using a real browser such as Chrome or Firefox to do automated testing, not PhantomJS. You can run browsers headlessly using Xvfb.
These might be useful:
http://codeutopia.net/blog/2013/07/13/headless-chromefirefox-testing-in-nodejs-with-selenium-and-xvfb/
http://elementalselenium.com/tips/38-headless
Selenium is the way to go for testing. IMO the best setup would be to use a CI server like Jenkins or TeamCity, with Xvfb installed, and have a test/deploy shell script in Jenkins such as:
#!/bin/sh
set -ex
cd $WORKSPACE
export VELOCITY_CI=1
meteor --test --settings $WORKSPACE/.deploy-staging/settings.json
cd .deploy-staging
mupx setup
mupx deploy
(Note that Xvfb is not implemented here, though). This test is not entirely working, I have yet to jump into Xvfb myself, though I know this is the right direction.
I'm using mupx to deploy my apps, which automatically creates a Docker instance on the remote server for me and handles deployment completely.
Related
I have installed Botium-Binding and prepared my machine to have installed with selenium-standalone server as per the documentation in https://github.com/codeforequity-at/botium-connector-webdriverio.
Once everything is setup i have added below capabilites to the botium.json file in samples/custom scripts folder.
"WEBDRIVERIO_OPTIONS": {
"capabilities": {
"browserName": "chrome",
}
}
Then i have run the command npm install && npm run mocha to trigger the runner.
All i could see is that the Webdriver session is being created and terminated with the error below:
WebdriverIO Plugin Sample
Starting ChromeDriver 92.0.4515.107 (87a818b10553a07434ea9e2b6dccf3cbe7895134-refs/branch-heads/4515#{#1634}) on port 44325
Only local connections are allowed.
Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
ChromeDriver was started successfully.
1) "before each" hook for "contractcancellation"
0 passing (9s)
1 failing
1) WebdriverIO Plugin Sample
"before each" hook for "contractcancellation":
Error: WebDriver error on startup: Failed to create session.
Make sure to connect to valid hostname:port or the port is not in use.
If you use a grid server
Make sure to add vendor prefix like "goog:", "appium:", "moz:", etc to non W3C capabilities.
See more https://www.w3.org/TR/webdriver/#capabilities
tried adding Hostname, port, platform to the capabilities but of no use. Looking for some help on this to get me unblocked.
You have to follow the instructions to install a chromedriver version that matches your Chrome version (as pcalkins wrote above).
It this doesnt help I recommend to enable verbose logging to get more insights on whats happening. You may post the log output here for further investigation.
Summary: How do I extract Chromedriver logs when running Selenium standalone Chrome instance? I.e. interacting via Selenium API commonly on port 4444.
Details:
We are using Protractor to connect to a container running the Docker image selenium/standalone-chrome Selenium "grid". Connection info is specified via the HUB_PORT_4444_TCP_ADDR environment variable. The connection URL ends up being http://localhost:4444/wd/hub. This works fine and our tests are running successfully in Jenkins.
For completeness I'd like to extract the Chromedriver logs and attach them to the build in case we need more info for debugging test failues. How can that be done?
This question seemed like a close match but they are running Chromedriver directly. I need to ask Selenium to provide them somehow.
Log properties of standalone chrome container can be configured using JAVA_OPTS.
You can add JAVA_OPTS environment variable to standalone chrome container
name: JAVA_OPTS
value: "-Dwebdriver.chrome.logfile=<Path to log file, with file name>"
We had a shared volume mounted, and gave path to that folder for putting log file.
Used yaml file for creating container template so used in above mentioned way.
If you are using CLI to launch container, same can be passed through CLI too.
Is it possible to make Jenkins use actual browser instead of headless browser? I a running some tests written in TestNG (using Selenium webdriver). When I run the testng.xml file in Eclipse, the browser starts and the tests run. But when I use Jenkins and run the tests with maven, it doesn't start any browsers.
If your jenkins is hosted in a Windows machine, there are some special configurations you should know about services that are allow to use the interface.
By the way, the easiest way to see the browsers running is starting jenkins using the command line:
java -jar jenkins.war
In linux you could use the same command or use xvfb plugin to run browsers in background.
Hope helps
In addition to this, the main reason for not launching the browser is JNLP (java network launch protocol) , when we execute the war we can interact with the desktop applications.
Using Selenium Grid will allow you to execute the test on Jenkins but open the browser on a remote slave.
To achieve this you need to create an instance RemoteWebdriver than ChromeDriver, IEDriver etc I
For linux. If jenkins is running as a daemon, you could specify active display to connect to and run your browser on it. Check what display you could connect to:
ps e | grep -Po " DISPLAY=[\.0-9A-Za-z:]* " | sort -u
My output is:
DISPLAY=:10.0
DISPLAY=:2
DISPLAY=:2.0
Then go to your jenkins project -> Configure -> Build and add the next string above your main build configuration through "Add build step -> Execute shell"
/bin/bash -c "export DISPLAY=:10"
Edited: I've encountered the issue again recently. To resolve it:
I've given for the jenkins' user ability to interact with the desktop of my current user:
xhost +si:localuser:jenkins
so if I connect to my linux system through ssh using jenkins' user credentials, export display of my current user (export DISPLAY=:10) and run, for example google-chrome or firefox inside of putty's terminal, they are launching on my current user's desktop.
After this I've checked If I could start "mvn test" command inside of workspace/MyTests folder from my putty so it will start browser and execute tests.
At the end I've created simple script in the root of my current user:
vi ~/.startup.sh
#!/bin/bash
xhost +si:localuser:jenkins
and added it to my Xfce4 GUI: Application -> Settings -> Session and Startup -> Application Autostart, specifying command field as:
sh -c $HOME/.startup.sh
It's because of this script should work only when desktop is loaded to share my current user's display with jenkins' user. After reboot and connecting to this server through RDC desktop loads with xhost command applied. And after this jenkins could interract with desktop even when I close the RDP connection but leaving current user's session alive.
I've removed Build step in my jenkins' project configuration that was stated "Invoke top-level Maven targets". It could not start my browsers.
I've changed my "Add build step -> Execute shell" to:
export DISPLAY=:10
cd /var/lib/jenkins/workspace/MyTests
mvn test
I've tried grid also, turning selenium-server-standalone -hub and -node into daemons. But it was slower than launching browsers with WebDriver in the such way.
I'm creating my tests (though I'm a beginner, learning) using Codeception. This includes acceptance and unit tests for now.
I want to add my repo to Travis CI so I can automate testing process after each commit and put build-status tag.
I would like to ask;
Can Travis-CI run codeception tests?
Can Travis-CI run codeception acceptance tests emulating browser?
If both answers are no, is there any other CI tool which can?
Thank you.
Yes, it is possible to run Codeception tests, including acceptance tests that run using WebDriver, on Travis CI.
It is possible to run your tests with a real browser on Travis, but it is easiest to use a headless browser, since Travis is running on a headless machine. PhantomJS is perfect for this, and it comes pre-installed with Travis CI's build bootstrap.
To run the tests with PhantomJS, you'll need to configure the WebDriver module like this in your .yml Codeception configuration file:
modules:
config:
WPWebDriver:
url: 'http://127.0.0.1:8888'
browser: phantomjs
The URL is important. I have found that attempting to use localhost instead of 127.0.0.1 will not work. Also, if you accidentally leave out the http://, that won't work either. You can use most any 8*** port, since most of them are open, but of course you'll need to have a web server running on that port to serve your static files or run your PHP application. The easiest way to do this, I find, is to use PHP's built-in webserver.
Your .travis.yml file might look something like this:
# Travis CI configuration file.
language: php
php:
- 5.6
- 7.0
before_script:
# Start up a web server.
- php -S 127.0.0.1:8888 -t /path/to/web/root >/dev/null 2>&1 &
# Start up the webdriver.
- phantomjs --webdriver=4444 >/dev/null 2>&1 &
# Install Codeception.
# Doing this last gives the webdriver and server time to start up.
- composer install --prefer-source
script:
- vendor/bin/codecept run
You will of course need to add Codeception to your project's composer.json file:
composer require --dev codeception/codeception
You'll also need to change path/to/web/root above to the path to the directory where you want the server's document root to be.
If you'd like to see a working demo running WebDriver tests against WordPress, you can check out this GitHub repo.
I'd think that it can be done, but gluing everything tohether is not going to be for the faint of heart. Reason why I think it can be done is that codeception, itself, is ci-ed on Travis. See https://travis-ci.org/Codeception/Codeception. I'd contact the people at codeception and ask for their thoughts.
Or you can take a peek at how they do it in the build logs, such as:
https://travis-ci.org/Codeception/Codeception/jobs/14432638
Looks like they're running headless with a downloaded standalone selenium server.
Travis-ci have some information on how to run GUI tests. In particular, they allow you to use a sauce labs account and run distributed selenium tests from there.
I ran into this problem today and I solved it by adding Codeception to my composer.json:
"require-dev": {
"codeception/codeception": "^2.1"
},
and referring to it on my .travis.yml:
install:
- composer self-update
- composer install
before_script:
- #Code that creates and seeds my database and so on
script: php vendor/codeception/codeception/codecept run
we are using Jenkins as CI in our project. We were running the CI from the command line using the following command
java -jar C:\\jenkins\\jenkins.war --httpPort=8085 --ajp13Port=8009
As the system needs to restart frequently, we change CI to start as a windows service.
Now we are facing the issues for Selenium test cases. Selenium test cases are not running after we make Jenkins as service. We are getting the following log and no more progress from that point
18:36:30,718 INFO [org.openqa.selenium.server.SeleniumDriverResourceHandler] Command request: getNewBrowserSession[*iexploreproxy, http://192.168.132.105:8080/, ] on session null
18:36:30,718 INFO [org.openqa.selenium.server.BrowserSessionFactory] creating new remote session
18:36:30,796 INFO [org.openqa.selenium.server.BrowserSessionFactory] Allocated session 80b95d0273ac4ea4a82860c79438f071 for http://192.168.132.105:8080/, launching...
18:36:30,796 INFO [org.openqa.selenium.server.browserlaunchers.WindowsProxyManager] Modifying registry settings...
18:36:31,781 INFO [org.openqa.selenium.server.browserlaunchers.InternetExplorerCustomProxyLauncher] Launching Internet Explorer...
Per hudson wiki, you should be running Hudson (or jenkins) as tasks rather than service for GUI testing. Check here. Look at the section GUI Testing in Windows
Following changes will resolve the problem
Update the selenium version.
Use *iexploreproxy or *piiexplore for IE instead of *iehta/*iexplore