I'm using a post-build trigger script in an Xcode bot to run some tests through Appium. I've gotten it to run, but I can't find any way to mark the build as failed if the tests fail.
Is this possible, or should I just email the results for now and wait to move to a different CI system? Documentation for these bot trigger scripts is sketchy at best.
Related
We are working with a client that uses VeriFIX to test their FIX message flow. Whilst they have built up lots of tests in many suites, it is a manual process to run them and to collate the results.
On the VeriFIX website it says
Incorporate tests into nightly builds using VeriFIX’s command-line script player.
but I cannot find any details on how to to it. Does anyone have any experience in running VeriFIX tests in a continuous integration server (ideally a Jenkins pipeline).
Many thanks.
You can run VeriFIX playlists in batch mode from the command line:
"%VERIFIX_HOME%\verifixbatch\verifixbatch.exe" -version "FIX (x.y)" -playlist "myplaylist" -disablelogging "false"
If you have received the user manual with your installation of veriFIX, the details of how to integrate with CI are in there.
To integrate veriFIX with Jenkins you will create batch files containing tests and run the batch files as jobs in Jenkins.
The placement of your veriFIX installation is important. If your veriFIX is on a users machine, as is often the case, separate from the environment machine Jenkins resides on, there can be difficulties getting the tests to run.
If you have a centralised install of veriFIX things are much easier.
I would like to schedule task which is login to a partner site and prolongs a license. I found that it can do Selenium. After I faced issue with scheduler and found article here http://www.learnseleniumtesting.com/ about Jenkins configuration. Now I cannot understand how to run test from selenium in Jenkins. Is there need good knowledge in python or java ? Also I found that RC standalone selenium server is deprecated and avoided to be using now. Is there need deep diving into using these tools for executing 5-7 action on site ?
I feel that Selenium is great for doing any tasks that you may find yourself doing over and over again. For "prolonging a license" on a partner site I think this is the perfect tool for you. We setup Jenkins to run certain Selenium tests after each commit and other Selenium tests every night. If you need me to go into more detail about how the setup looks I would be glad to show you more details.
Adding Jenkins build information to help show how we did it.
1) The first step is to get a build in Jenkins that will compile any code you have and deploy a war if needed with something similar to the following: mvn -P release clean test package. We set ours to build #midnight shown in the picture below.
2) After this build finishes it triggers the next job to run this command shown in the image below: mvn -Dtest -P smokeTest clean verify
We are building apps for iOS and Android. Currently we are experimenting with TeamCity for CI.
Currently we have various tools to help us in the Test automation.
1. Selenium: For testing the web interface
2. SOAPUI: For testing the web services
3. Calabash-iOS: For testing iOS app
This is the scenario that we have done successfully with TeamCity:
1. Developer commits code to BitBucket
2. TeamCity runs calabash
3. If calabash pass, TeamCity will build and give the artifacts
This is the scenario that I am trying to do so that I can achieve an end-to-end automated testing:
When developer commits his code to our Bitbucket git repository,
TeamCity will trigger Calabash to run and check for the UI and the output of the details. On end,
TeamCity will trigger Selenium to add data in the Back Office web. On end,
TeamCity will trigger SOAP UI to check newly added data. On end,
TeamCity will trigger Calabash to check the newly added data
Has anyone tried this before or can give me an idea on how I can approach this?
You can setup different builds for each step since you can trigger builds via simple GET requests triggered by a script. TeamCity Accessing Server by HTTP
We run a similar setup. Basically we have one build kick off the Selenium/SoapUI scripts. That build doesn't wait for the job to finish, it just kicks it off. Then in our Selenium/SoapUI tests as the last step we hit the URL for the next build that picks up reports etc.
I was wondering if it is possible for Jenkins to be notified after TFS finishes a build.
I'm trying to make a system where, after the TFS build, Jenkins is used to run automated selenium tests.
Thanks in advance!
You have a few options:
You could use a GenericTest to trigger Jenkins and include as part of the Build Definition. This also allows your script to communicate pass/fail information back to the build.
You could just include an InvokeProcess workflow activity to trigger Jenkins. The process would be similar to this or this.
Or you could use a tool that would trigger after a Team Build has finished. Normally people would use tools that then deploy stuff. Most of these tools trigger a Powershell script at some point, which would allow you to trigger the test run:
Octopus Deploy
TFS Deployer
We use Hudson for our build/CI needs. In addition to unit tests (running during build) I have a staging environment that runs additional integration tests. Basically the build happens and then build artifacts are submitted the external system. I do not wish to block a Hudson build to wait for integration tests (as that locks the station into idle, and prevents it from building anything else). What I want is to update the the result of the build with the result of the external tests (and attach some logs back to the build, if possible).
Now because the staging environment is asynchronous to the build system (i.e. other systems/people can submit tests), Hudson can't be just monitoring what goes on there right after the build. Hudson build just goes into a test queue. So, I need to notify Hudson, it can't be polling something for updates.
Does Hudson support such behaviour, and if so, how can I achieve it?
I would suggest using the hudson users mailing list [1]
[1] http://java.net/projects/hudson/lists/users/archive
To solve the asynchronous wait issue, you can use the build triggering with an authentication token, and use it in a script.
The Build Triggers section has a Trigger Builds Remotely (e.g. from scripts), which when selected allows you to enter an Authentication Token (which has the descriptive text from your question).
You can thus, remotely trigger the build from a script, i.e., have it as part of the integration test script and trigger the build job using this authentication token.
..
You can have a downstream project just for result collection that integrates the results from various tests and report it to the master and aggregate them all. This project can be triggered using the authentication token, or if there is a single integration test job, you can tie it up as a downstream project.