I have a project 'ABC' with the main code and junit tests. I do have the requirement that i can execute the set unit tests against a older version of the product artefacts.
To solve this requirement i would create a maven project which only contains the junit tests.
Another maven product builds my product code and places the artifact into the repository.
Now i could launch my tests against any product build by changing the build dependency within the junit test project.
Is this a good solution? Are there perhaps better solutions to solve this requirement?
I think thats a pretty good approach. You could create a profile for each old version and then activate them via the profile name and test different old versions without having to change the pom file for each run. You could then also run the different profiles separately scheduled on a continuous integration server...
Related
Is there any good way to get Karate tests to test a Spring-Boot microservice in the "integration-test" phase of a Maven build? What I mean is: is there an anti-pattern/workaround that works well OR am I out of luck because this was an afterthought of the Karate development?
Facts I have gathered so far:
It says here: "the surefire plugin is not hardcoded into Karate"
Is there a way to run Karate test during maven's integration test phase?
Running Karate tests in context of maven-failsafe-plugin does not work at all. Easy to reproduce on my test project url below. Two problems foremost:
When running from maven-failsafe, Karate will generate .txt and .html files but does not generate .json Cucumber output files.
Therefore, you can get no HTML report from the cucumber-reporting plugin (which I require)
I created a project to demonstrate these facts: https://github.com/djangofan/karate-api-sample
Running integration tests requires that I start Spring Boot before the test suite and then stop the spring boot server afterwards. Maven SureFire does not have this capability BUT the maven failsafe plugin does: the maven build helper plugin has a pre-integration-test hook for it. Limited only by Maven surefire, I am out of luck.
I see an example in the karate/karate-demo project of starting the spring boot server from code. This is not easy to accomplish when I am trying to test a web service that is already established. Is this my only option: to use surefire with a class filter and code to bootstrap?
Given no good answer, I will just revert to using cucumber-java + resteasy, which I am pretty sure will work with maven failsafe: https://maven.apache.org/surefire/maven-failsafe-plugin/examples/cucumber.html
First, this is not Karate.js not sure where that came from :)
When running from maven-failsafe, Karate will generate .txt and .html files but does not generate .json Cucumber output files.
Sorry, cannot replicate.
For failsafe, just use the naming conventions and it will work, e.g. **/IT*.java
Maven SureFire does not have this capability BUT the maven failsafe plugin does:
So use failsafe as mentioned above. If you are not able to do that, please change your project to demonstrate the issue and we promise to fix it. Also refer these instructions: https://github.com/intuit/karate/wiki/How-to-Submit-an-Issue
Given no good answer, I will just revert to using cucumber-java + resteasy
That is of course your choice. At least in my (biased :) opinion you will miss all the JSON assertion value that Karate adds, and the parallel execution + aggregation of reports. I am sure you will help as far as possible so that we can make the experience better for all who need this solution - but else, hey - no worries :P
We are currently using bamboo to build and test our software. Now our build plans are just a bunch of task: execute this bat, execute that bat etc. Created with the Bamboo UI.
It happens that over months/years the build plan needs adjustments:
Parallelize jobs
Add extra jobs
Change some tasks
But this will break when we try to build an older version of the software. Some scripts (called from bamboo task) are not-existing in older versions.
At my previous employer we used Jenkins pipelines where the content of the build and test was just a file present in the sources repo.
Now with bamboo it appears you can use Bamboo Specs. From I read you create specs file and when you run this, it which will create build plan. But I don't see a relation to cater for changing build plans over time (changing steps).
For example the Bamboo Specs of develop are used to build all Plan Branches (e.g. Pull Requests). So if you want to change the build in a PullRequest, you first need to merge this into develop, the Bamboo Spec of develop updates the Build Plan. Not possible to test this before merging.
Question: How can you make scripted buildplans in Bamboo, where every branch of develop can a have possible other way of building?
We have it now setup as:
Buildplan 'Product A': plan branches: develop, release_x, release, y
Buildplan 'Product A PullRequest': plan branches: feature/*
Edit: supported in 7.0: https://confluence.atlassian.com/bamboo/enhanced-plan-branch-configuration-996709304.html
Old answer:
I found Atlassian documentation: https://jira.atlassian.com/browse/BAM-19620. They call it 'divergent plan branches'. No support, there is a feature request.
As of 15-4-2019:
Atlassian Update – [11 April 2019] Hi everyone,
Thank you for your votes and thoughts on this issue.
We fully understand that many of you are dependent on this
functionality.
After careful consideration, we've decided to prioritise [this
feature] on Bamboo roadmap. We hope to start development after our
current projects are completed.
Expect to hear an update on our progress within the next 6 months.
To learn more on how your suggestions are reviewed, see our updated
workflow for server feature suggestions.
Kind regards,
Bamboo Team
Question: How can you make scripted build plans in Bamboo?
To make scripted build plans in Bamboo, you have to use bamboo specs. Since you are already familiar with Jenkins, bamboo specs work exactly like Jenkinsfile by automating your pipeline. The benefit of using this is that it lives in your source code and the changes you make to this file in your source code automatically changes your plan(pipeline) when bamboo build is triggered.
This is how I script build plans in bamboo:
I add my bamboo.yml file under the root of my repo. But currently, I use git subtree and my bamboo specs live in there. But you don't have to do this. The below link provides you with a simple approach.
Link my repo to bamboo
Tell bamboo to scan for bamboo specs in the repo
Make commit and push
https://confluence.atlassian.com/bamboo/tutorial-bamboo-specs-yaml-stored-in-bitbucket-server-941616819.html
If I have to make changes to the plan in the future, I edit the bamboo specs file then commit and push.
I had the same problem and unfortunately had to go through an unpleasant choice
Backporting the build script
This is not necessarily feasible everywhere, but I managed to make it work somehow for my project.
The idea is: treat the build script as a C#/Java interface, or better as a contract.
As soon as your branches do not provide significant changes in building the software, e.g. your desktop app becomes a web app, or you switch from Ant to Gradle, you can handle this.
Assuming my application is always a web application to be released as a jar on JFrog Artifactory, I have identified the following steps that are common to all maintained versions:
Use javac to build the jar of all modules
Use gulp to build the Javascript resources
Run JUnit from the repository
Baptize 💒 the artifacts with a version number obtained with a tricky algorithm
Push the artifacts to JFrog Artifactory
So the idea is that I had taken my Ant build script and mostly rewrote it in order to do the same tasks on different versions of the application. I started doing the changes from an older version, not maintained anymore, as an excercise. In fact, my official Git branches look like release/x.y.z where semver is x.y.z.k and newer bugfix-builds are built from the head of any x.y.z release.
So I took release/3.10.0 branch and rewrote Ant. I am currently testing with a manually created Bamboo plan
Stage: Compile
ant clean ivy-retrieve compile jar #builds the jar in a job
ant gulp-install gulp-prod zip #creates javascript resources
Stage: Test
ant run-junit
Manual Stage: Release
ant baptize ivy-release #tags the artifact using ${bamboo.jira.version} and pushes to JFrog Artifactory
What I am going to do with Yaml
Since the build script is the same, but specific tasks (e.g. Java compiler version) may change in different versions, I can create a very single Yaml script that rules them versions all.
I will then merge release/3.10.0 => release/3.10.1 => release/3.10.2 ... release/3.11.2 by merging the conflicts
Personal experience
Tonight I am struggling in making the JUnit tests work as I also chose to backport my testing framework to the older version of the project. I accept that a few tests will fail because older and non-maintained versions contain bugs. For me this is a way to prove that the system works.
Indeed, diverging branches are a great idea, but I am forced to use Bamboo 6 in my office
I am developing an Eclipse plugin and I use maven to coordinate my source structure. In order to compile the plugin I use the tycho extension for maven. However, I was wondering how to execute unitests.
I want to use the surefire plugin for testing as I additionally use a sonar server for source code quality management. Unitests are applyed if I use eclipse-test-plugin as package target. However, I want to make use of the default surefire plugin for applying unitests.
Now I figured out that the src/test/java that contains my unittest packages is read and compiled correctly but written into the wrong output folder. I need to have the tests in target/test-classes. However they are compiled to target/classes.
As I am new to Eclipse plugin development and maven I could not find out how to write the tests to the correct output folder. I've already tried adding and and changing the build.properties of the eclipse-plugin project. It works also fine for other projects that aren't plugin projects and do not make use of tycho.
Any help appreciated.
Regards,
Florian
Unlike standard maven projects, the convention for eclipse plugins/OSGi bundles is to have tests reside in separate projects. This is because there is no such thing as a maven dependency scope "test" in OSGi.
Thus keeping your tests inside the same project as your code under test would force you to mix up test code/dependencies an productive code/dependencies.
As you mentioned, Tycho provides a separate maven packaging type "eclipse-test-plugin" which you should use for dedicated test plugins/fragments. See https://docs.sonatype.org/display/TYCHO/PackagingTypes
There is no support in Tycho for plain unit tests residing in the same project.
The past week or two I've been studying Maven, and I like it, but there are just a few things that I require Ant tasks for, rather than Maven's complicated and scarcely-documented POM file. However, I believe Maven has a great folder structure going for it, and I like that it natively supports tests, packages the project into a jar by default, and supports a 'resources' folder, from which everything is directly copied into the jar file.
Is there a sample Ant project out there that does the same sort of thing but with an Ant build.xml script? I want the placeholder folders and hello world app and test, just like Maven does when you first create a project from its default archetype:create goal (as demonstrated in the Maven in 5 Minutes page). Or, is there an even better Ant sample project out there that does more or suggests a better folder layout?
If no such thing exists, perhaps someone can help create it in a nice detailed answer? I would be willing to host a permanently-available zip file for anyone who finds this question in the future.
mvn archetype:create
mvn ant:ant
http://maven.apache.org/plugins/maven-ant-plugin/usage.html
... but if there are really "just a few things", you may want to check out the AntRun plugin instead. I'm not trying to sell you on Maven, believe me, but since it's not clear exactly what is stopping you from trying it with your project, I guess I'm suggesting you try to push the issue a little harder.
Refer this: Why you should use the Maven Ant Tasks instead of Maven or Ivy
I also wouldn't recommend Ivy, reasons at the link above.
Quoting the Maven - Frequently Asked Technical Questions and more precisely:
How can I use Maven features in an Ant build?
The Maven Ant Tasks allow many of the
features of Maven, such as dependency
management and repository deployment,
to be used in an Ant build.
Refer to the installation page and the usage page for instructions for installing and using the Maven Ant Tasks respectively. You'll find many links to samples in the usage page and a build.xml showing most of the features in action.
An alternative (direct competitor?) to Maven Ant Tasks would be Apache Ivy.
PS: While it's definitely a good idea to adopt maven standards, even partially, I'd really think about it twice before to drop Maven (but I live in the Maven jungle for a while now - and I like it - so I'm biased).
I recently discovered that Hudson was not the problem. In actuality it was Maven itself as the multi-module build was causing the build failure, not Hudson. I just hadn't noticed where the issue actually existed.
Leaving the original question here.
I'm using the failsafe-maven-plugin to run some integration tests. The difference between failsafe and surefire is that failsafe allows failures and does not fail the build.
On my nightly builds there are occasions that a service the integration tests use might be down. In normal builds, the failsafe plugin would let the build continue since the integration tests are allowed to fail. However, Hudson does not seem to respect this and stops the build and produces rain.
I tried to turn the failsafe tests off on nightly builds using -DskipITs. This appears to fail since I'm in a multi module build.
Any ideas on how to get Maven to respect that these tests can fail even though they're part of a specific module?
The project structure is as follows:
-parent
\-jar
\-jar (where integration tests run)
\-war
\-ear
You can use profiles to make builds a bit different for different environments (nightly builds, releases, normal developer builds and so on).
I'd also try updating the Maven version, there were recently few fixes related to multi-module builds.
I don't believe your original assumption that failsafe-maven doesn't fail the build is correct. A failed test does not stop the integration-test phase from completing, which is different from the surefire plugin that runs unit tests. This allows the post-integration-test phase to run, so the test environment can be torn down (app server shut down, etc.).
After this, the verify phase is run, which looks at the results of the integration tests. if one of these tests has failed, then Maven will return with a build failure, which Hudson will rightly pick up so your build can be flagged as broken.
Use a maven profile to turn on/off the verify goal of the maven failsafe plugin.