I'm working with a Jenkins system, and I've recently started working to optimize the tests. There are almost a thousand Selenium tests and twice as many unit and integration tests.
I'm wondering if there is some way to find which tests are the most prone to failure so we can attack the worst offenders first. It would also be nice if there was an integrated way to track who is working on which test, and if a test has been fixed, what was done to fix it. I'm new to Jenkins, so please point me to some documentation or a plugin I can install that would help me with this.
I think what you are describing is JIRA which can tie SCMs to issues to releases to developers.
There is Jenkins to JIRA integration to create Jira issues out of Jenkins
Related
We are working with a client that uses VeriFIX to test their FIX message flow. Whilst they have built up lots of tests in many suites, it is a manual process to run them and to collate the results.
On the VeriFIX website it says
Incorporate tests into nightly builds using VeriFIX’s command-line script player.
but I cannot find any details on how to to it. Does anyone have any experience in running VeriFIX tests in a continuous integration server (ideally a Jenkins pipeline).
Many thanks.
You can run VeriFIX playlists in batch mode from the command line:
"%VERIFIX_HOME%\verifixbatch\verifixbatch.exe" -version "FIX (x.y)" -playlist "myplaylist" -disablelogging "false"
If you have received the user manual with your installation of veriFIX, the details of how to integrate with CI are in there.
To integrate veriFIX with Jenkins you will create batch files containing tests and run the batch files as jobs in Jenkins.
The placement of your veriFIX installation is important. If your veriFIX is on a users machine, as is often the case, separate from the environment machine Jenkins resides on, there can be difficulties getting the tests to run.
If you have a centralised install of veriFIX things are much easier.
I'm trying to understand what is the best practice for running remote e2e tests written in Selenium - for running different suites on different environments. the testing code is attached to git and basically we use TeamCity, but it doesn't have ti be used.
any suggestions would be great
thanks!
You can have a look at my protractor example project on GitHub. I'm not sure my examples would be considered "best practices" but it's how I currently do things :)
The examples cover:
page objects
Running on TravisCI
Running tests on Sauce Labs and Browserstack
Running multiple browsers at once
I have a set of +800 selenium tests and because of problems in our application (mostly asynchronicity bugs which we are aware but cannot fix now) they, sometimes, randomly, fail.
We use jenkins for CI and I'd like to know if there is a way to configure jenkins to see the past builds and warn of failing tests only if the same tests fails in the previous builds. This, albeit not a perfect solution, would help mitigate the randomness of tests and the effort to analyse only those that are real bugs.
Any ideias?
I have a bunch of acceptance tests that need the application to be running. It all works fine when I test from command line (thanks to some gradle magic) but I would like to be able to run these tests from IntelliJ IDEA without worrying about starting up Jetty.
Is there any clever way to achieve that automation? I do not even know where to begin.
Thank you very much.
You can do it with Maven/Ant, other Run configurations, but not Gradle at the moment, at least until this feature request is implemented.
For testing purposes it's generally a good idea to use jetty embedded. That way you can fully automate start/stop of jetty and it will work completely independent from build tools/ide.
It's really simple. With few lines of code you have a full featured jetty configured and running for testing.
This is one of the most beloved features of jetty. Have a look at this:
http://www.eclipse.org/jetty/documentation/current/advanced-embedding.html
Disclaimer: I'm not at all a skilled Jenkins user.
I was looking for a plugin to allow me to run only tests affected by the latest commits, in maven-built projects, but could find neither a maven plugin smart enough to not run all tests again, nor a jenkins plugin doing this. Is there a way to achieve this?
Say, for example, I have a project consisting of A.java, testA.java, B.java and testB.java. I have jenkins configured to run a build and test for each commit. I have just commited a change to A.java, but not to B.Java, and there's no dependency from testB.java and B.java to A.java. I don't want Jenkins to run testB.java when it checks out fresh sources. If ther3e was a dependency from B.java or testB.java to A.java, I'd like jenkins to figure this out, and run testB.java too.
Background: running all tests in a project is too time consuming. Running only a fixed set of tests leaves bug holes for bugs. Running all tests which might be broken by a change by hand is error prone.