I have my testing up and running with Travis/SauceLabs. Now I would like to add a SauceLabs test badge to my repo.
I added the badge markdown to my Readme file but how can I send the build pass/fail to SauceLabs? I found this instructions for Selenium,
Key: passed
Value type: bool
Example: "passed": true
but how/where do I add info in my files for my Grunt-Karma/Travis/SauceLabs testing?
The Karma-Sauce-Launcher was using the wrong id, and this was fixed on Github by this PR and released in NPM as version 0.2.5
Aditionally the saucelabs reporter has to be added to the gruntfile options, besides the existing ones or the default progress.
So the bugfix I added to my package.json:
"karma-sauce-launcher": "~0.2.5"
and added this:
reporters: ['progress', 'saucelabs'],
in the Karma options.
You have to use the REST API. What you would do is add code in a function that is executed at the very end of your test suite and knows the result of your test run. This code would have to perform a query equivalent to this curl command:
$ curl -H "Content-Type:text/json" -s -X PUT -d '{"passed": <status>}' http://<username>:<key>#saucelabs.com/rest/v1/<username>/jobs/<job-id>
(The identifiers in angular brackets have to be replaced with appropriate values.)
I've done it with Python but I don't have JavaScript code to share. And by the way, you have to do this for Selenium too because as the documentation states, when Selenium sends the job data to Sauce Labs, it cannot know yet what the test result is going to be.
If you're using Grunt already, you should use https://github.com/axemclion/grunt-saucelabs as it is the official plug-in that is worked on by devs from SauceLabs.
Related
I am trying to run my UI test using testcafe and saucelabs. I am facing this above error. Currently I am using testcafe v1.8.3 and testcafe-browser-provider-saucelabs v1.7.0
I have tried changing versions of browser provider also but still facing the above error. Pls help out with a solution for this as i am stuck with it for more than a week
So, it looks like the runner you are using (testcafe-browser-provider) is a very old one, there is a new runner you can use for testcafe tests called saucectl.
TLDR:
Install saucectl globally npm install -g saucectl
Set up saucectl within your project folder with saucectl init This will create a .sauce/config.yml file
Tweak the settings to run the spec files and OS/ browser of your choice
Use saucectl run
You can see an example proj here: https://github.com/saucelabs/saucectl-testcafe-example
It looks like your provider is installed locally, while you are using the global TestCafe installation. You also need to install TestCafe locally or both packages globally. After this, check your browser provider: testcafe -b saucelabs.
I am using testcafe v1.8.3 and testcafe-browser-provider-saucelabs v1.7.0
Please update your testcafe and testcafe-browser-provider-saucelabs versions to the latest ones.
I have been working in TestCafe recently and I need a custom reporter. I was looking for something like mocha reporter. But all I could find is generating a plugin and publish it to npm. Is there a way to have a single file reporter?
Is there a way to have a single file reporter?
You can redirect the reporter's output to the file.
testcafe chrome test.js -r spec:result.txt
I was looking for something like mocha reporter.
TestCafe ships with 5 reporters: spec, list, minimal, xUnit, JSON. Also, many custom reporters can be find in npm: https://www.npmjs.com/search?q=testcafe-reporter.
I’m trying to integrate Bugzilla Testopia with Jenkins with the aid of the Testopia Plugin for Jenkins.
The general configuration is probably fine as the connection between Testopia and Jenkins is well established (Jenkins log says: ‘Connecting to Testopia to retrieve automated test cases’ and no error occurs then). However I’m unable to retrieve any information concerning Test Runs/Test Cases etc. from Testopia.
Moreover I cannot perform any of ‘Iterative Test Build Steps’. If I want to ‘Execute Shell’ in ‘Iterative Test Build Steps’ with Testopia Plugin no operation is carried out (even if I try: echo 12345 etc.). If I use ‘Single Test Build Steps‘ then shell command is executed.
My goal is to retrieve test class name from Testopia (it is stored in Testopia Test Case’s field - Automation/Scripts ) and then run maven build from Jenkins with this class name set as the parameter. Afterwards depending on the Jenkins build success or failure status I’d like to update the Test Case Status in Testopia.
How to fetch any information from Testopia into Jenkins?
Why any of ‘Iterative Test Build Steps’ is not executed?
Any clues? - Testopia Plugin site example wasn't too helpful for me.
Both Bugzilla and Jenkins are hosted on the same Ubuntu 14.04. I've got the latest stable versions of Jenkins, Bugzilla and Testopia.
Thanks in advance,
M.
EDIT:
Well this debugging does not work for me. I added a new log recorder with the 'ALL' level chosen and I cannot see any additional log neither in job's console output nor in the newly created logger output.
Maybe sth is wrong with my Testopia installation? Some more details concerning my configuration:
I've got Testopia installed on the same machine (as Jenkins) and usually I access it through: http://'ip_address'/bugzilla
- in Testopia plugin configuration my URL to Testopia installation is: http://'ip_address'/bugzilla/xmlrpc.cgi
- I've got only one bugzilla account - these admin's credentials I use in Jenkins
- sometimes in Jenkins I can see a warning concerning improper reverse proxy configuraiton - maybe it has sth to do with the problem
After job execution all Testopia's fields are 0 - Run Id, Build Id etc. - which obviously indicates that no information was successfully retrieved from Testopia.
Any ideas how to check why I cannot retrieve any information from Testopia?
EDIT 2:
In the meantime I think I've found a clue in the jenkins.log file in jenkins installation directory:
Exceptions like these occur:
INFO: TESTOPIA_TEST_SUITE_3 #13 main build action completed: SUCCESS
org.apache.xmlrpc.XmlRpcException: The requested method 'TestRun.get' was not found.
at org.apache.xmlrpc.client.XmlRpcStreamTransport.readResponse(XmlRpcStreamTransport.java:197)
...
org.apache.xmlrpc.XmlRpcException: The requested method 'TestRun.get_test_cases' was not found.
at org.apache.xmlrpc.client.XmlRpcStreamTransport.readResponse(XmlRpcStreamTransport.java:197)
Shall I insert the full stacktrace?
It looks like that plugin logs a fair amount of information, though not all of it to the build console output itself.
To debug further, you could try adding a new log recorder for the logger jenkins.plugins.testopia (with log level "all"), run a build, then refresh the web page for the newly-created log recorder to see the output.
You should at least see "Filtering for automated test cases" after connection, information about each test case found, and then log output for each iterative build step as it's run on each test case.
I got the similiar problem when I use python xmlrpc to commnicate with my bugzilla-testopia server.
I chekc the error code and find XMLRPC.pm under my bugzilla install location : "./WebService/Server/XMLRPC.pm".
I know the error was thrown by this sub function, which checks login status and forward to modules where the moduel.function must be in PUBLIC_METHODS:
sub handle_login {
...
if (none { $_ eq $method } $class->PUBLIC_METHODS) {
ThrowCodeError('unknown_method', { method => $full_method });
}
...
}
I don't know why but TestCase.get could not be found in PUBLIC_METHODS, so I just marked them as comment and then it works. You can do this quick way to make sure your client setting is correct. Then you should solve the "PUBLIC_METHODS" problem next.
I'm creating my tests (though I'm a beginner, learning) using Codeception. This includes acceptance and unit tests for now.
I want to add my repo to Travis CI so I can automate testing process after each commit and put build-status tag.
I would like to ask;
Can Travis-CI run codeception tests?
Can Travis-CI run codeception acceptance tests emulating browser?
If both answers are no, is there any other CI tool which can?
Thank you.
Yes, it is possible to run Codeception tests, including acceptance tests that run using WebDriver, on Travis CI.
It is possible to run your tests with a real browser on Travis, but it is easiest to use a headless browser, since Travis is running on a headless machine. PhantomJS is perfect for this, and it comes pre-installed with Travis CI's build bootstrap.
To run the tests with PhantomJS, you'll need to configure the WebDriver module like this in your .yml Codeception configuration file:
modules:
config:
WPWebDriver:
url: 'http://127.0.0.1:8888'
browser: phantomjs
The URL is important. I have found that attempting to use localhost instead of 127.0.0.1 will not work. Also, if you accidentally leave out the http://, that won't work either. You can use most any 8*** port, since most of them are open, but of course you'll need to have a web server running on that port to serve your static files or run your PHP application. The easiest way to do this, I find, is to use PHP's built-in webserver.
Your .travis.yml file might look something like this:
# Travis CI configuration file.
language: php
php:
- 5.6
- 7.0
before_script:
# Start up a web server.
- php -S 127.0.0.1:8888 -t /path/to/web/root >/dev/null 2>&1 &
# Start up the webdriver.
- phantomjs --webdriver=4444 >/dev/null 2>&1 &
# Install Codeception.
# Doing this last gives the webdriver and server time to start up.
- composer install --prefer-source
script:
- vendor/bin/codecept run
You will of course need to add Codeception to your project's composer.json file:
composer require --dev codeception/codeception
You'll also need to change path/to/web/root above to the path to the directory where you want the server's document root to be.
If you'd like to see a working demo running WebDriver tests against WordPress, you can check out this GitHub repo.
I'd think that it can be done, but gluing everything tohether is not going to be for the faint of heart. Reason why I think it can be done is that codeception, itself, is ci-ed on Travis. See https://travis-ci.org/Codeception/Codeception. I'd contact the people at codeception and ask for their thoughts.
Or you can take a peek at how they do it in the build logs, such as:
https://travis-ci.org/Codeception/Codeception/jobs/14432638
Looks like they're running headless with a downloaded standalone selenium server.
Travis-ci have some information on how to run GUI tests. In particular, they allow you to use a sauce labs account and run distributed selenium tests from there.
I ran into this problem today and I solved it by adding Codeception to my composer.json:
"require-dev": {
"codeception/codeception": "^2.1"
},
and referring to it on my .travis.yml:
install:
- composer self-update
- composer install
before_script:
- #Code that creates and seeds my database and so on
script: php vendor/codeception/codeception/codecept run
http://code.google.com/p/selenium/wiki/WebDriverJs#In_the_Browser
Where I can get javascript file "webdriver.js" from
So I can test and use it. Has anyone have done it..
webdriver.js can be built from the selenium sources. You need java and svn to take the following steps:
Get the selenium sources:
svn checkout http://selenium.googlecode.com/svn/trunk/ selenium-read-only
Change to trunk:
cd selenium-read-only/trunk
Build webdriver.js:
./go webdriverjs
This step generates a single javascript file in build/javascript/webdriver/webdriver.js. You can then go ahead and use this file as described in selenium's wiki.