I'm using an Jenkins pipeline to trigger AWS CodeBuild, and in my buildspec I run some tests that I wish to publish as artefacts so that they can be downloaded and read by Jenkins.
When all of my tests pass, this works just fine. However, when one or more tests fail, it seems as though the artifacts phase is ignored, so there are no artefacts for Jenkins to download.
Though it's not what I require, I have also attempted to use the reports phase, but that behaves in the exact same way, which I find confusing as it seems crazy to fail on a test and then not publish the reports.
Is it possible to make CodeBuild execute the artifacts phase regardless of success or failure?
version: 0.2
env:
shell: bash
phases:
install:
runtime-versions:
python: latest
commands:
- pip install cfn-lint checkov
- ...
pre_build:
commands:
- cd myproj
- cfn-lint --template cloudformation/template.cfn.yaml --format junit > cfn-lint.xml
- checkov --directory cloudformation --framework cloudformation secrets --output=junitxml > checkov.xml
build:
commands:
- ...
post_build:
commands:
- ...
artifacts:
base-directory: myproj
files:
- cfn-lint.xml
- checkov.xml
The answer here is that the artefacts (and reports) phase is not run if the pre_build phase fails.
https://docs.aws.amazon.com/codebuild/latest/userguide/view-build-details.html#view-build-details-phases
While I've technically answered my question, this means that I've had to move tests into the build phase, which feels wrong because the post_build phase is run regardless of success or failure, so publishing of my artefact (outside of AWS) also fails.
When I run mvn javadoc:javadoc locally, it gives me a bunch of warnings—empty #return tags, unknown tags etc.—but eventually builds the Javadoc tree.
On GitLab CI, I have the following in .gitlab-ci-yml:
variables:
MAVEN_OPTS: "-Dhttps.protocols=TLSv1.2 -Dmaven.repo.local=$CI_PROJECT_DIR/.m2/repository -Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=WARN -Dorg.slf4j.simpleLogger.showDateTime=true -Djava.awt.headless=true"
# [...]
deploy:jdk8:
stage: deploy
script:
- if [ ! -f ci_settings.xml ];
then echo "CI settings missing\! If deploying to GitLab Maven Repository, please see https://docs.gitlab.com/ee/user/project/packages/maven_repository.html#creating-maven-packages-with-gitlab-cicd for instructions.";
fi
- 'mvn $MAVEN_CLI_OPTS deploy -s ci_settings.xml'
- 'mvn javadoc:javadoc'
- 'cp -r target/site/apidocs public/javadoc/dev'
artifacts:
paths:
- public
only:
- master
- dev
Here, Javadoc generation fails, with the previously mentioned warnings being reported as errors.
I am ultimately going to fix these things, but in the meantime, I would like Javadoc on CI to behave like its local counterpart. Where is the setting to accomplish that?
Changing the mvn javadoc:javadoc line as follows did the trick for me:
- 'mvn javadoc:javadoc -DadditionalJOption=-Xdoclint:none'
Still not sure why this sems to be the default behavior on my local Maven installation but not on GitLab, but at least this solves my issue for now.
I am using mvn for build and deploy my code in tomcat. I am using command for build clean install -DskipTests -DskipJangarooApp -pl :studio-webapp -am -T 4C where studio-webapp is my module name.
And after build i use tomcat:run command like this :
tomcat7:run -Dinstallation.host=192.168.252.100
where 192.168.252.100 is my host machine IP. This process is running well.
But the problem is when I change any java class in my IntelliJ I again run buil command then run. And its taking so much time. So that I want to apply tomcat configuration in IntelliJ but i am not able to find any option of "-Dinstallation.host" in this screen
I have some task:
I have github repostiroy with folder "docker" and in this folder have many sub-folders with names (app1, app2, app3, etc.) with only Dockerfile in subfolder.
Sometimes i add new subfolder with name app50 and i want to Jenkins build automatically this new dockerfile and if all ok push new images to private docker registry.
How I can automate to select sub-folder, enter to him and run docker build ? with cutsom tag and name.
For example i upload new folder "app70-1.2.3" with Dockerfile. Jenkins need enter to folder "docker/app70-1.2.3" and run "docker build -t app70:1.2.3 ."
I am using similar approach and the simplest way to achieve this in your case could be via Feature branching strategy. Just use your branch as folder name and then run something like this as Jenkins (shell) build step:
cd ${GIT_BRANCH}
docker build -t ${GIT_BRANCH} .
I can install an artifact by install:install-file,
but how can I download an artifact?
For example:
mvn download:download-file -DgroupId=.. -DartifactId=.. -Dversion=LATEST
You could use the maven dependency plugin which has a nice dependency:get goal since version 2.1. No need for a pom, everything happens on the command line.
To make sure to find the dependency:get goal, you need to explicitly tell maven to use the version 2.1, i.e. you need to use the fully qualified name of the plugin, including the version:
mvn org.apache.maven.plugins:maven-dependency-plugin:2.1:get \
-DrepoUrl=url \
-Dartifact=groupId:artifactId:version
UPDATE: With older versions of Maven (prior to 2.1), it is possible to run dependency:get normally (without using the fully qualified name and version) by forcing your copy of maven to use a given version of a plugin.
This can be done as follows:
1. Add the following line within the <settings> element of your ~/.m2/settings.xml file:
<usePluginRegistry>true</usePluginRegistry>
2. Add the file ~/.m2/plugin-registry.xml with the following contents:
<?xml version="1.0" encoding="UTF-8"?>
<pluginRegistry xsi:schemaLocation="http://maven.apache.org/PLUGIN_REGISTRY/1.0.0 http://maven.apache.org/xsd/plugin-registry-1.0.0.xsd"
xmlns="http://maven.apache.org/PLUGIN_REGISTRY/1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<useVersion>2.1</useVersion>
<rejectedVersions/>
</plugin>
</plugins>
</pluginRegistry>
But this doesn't seem to work anymore with maven 2.1/2.2. Actually, according to the Introduction to the Plugin Registry, features of the plugin-registry.xml have been redesigned (for portability) and the plugin registry is currently in a semi-dormant state within Maven 2. So I think we have to use the long name for now (when using the plugin without a pom, which is the idea behind dependency:get).
With the latest version (2.8) of the Maven Dependency Plugin, downloading an artifact from the Maven Central Repository is as simple as:
mvn org.apache.maven.plugins:maven-dependency-plugin:2.8:get -Dartifact=groupId:artifactId:version[:packaging[:classifier]]
where groupId:artifactId:version, etc. are the Maven coordinates
An example, tested with Maven 2.0.9, Maven 2.2.1, and Maven 3.0.4:
mvn org.apache.maven.plugins:maven-dependency-plugin:2.8:get -Dartifact=org.hibernate:hibernate-entitymanager:3.4.0.GA:jar:sources
(Thanks to Pascal Thivent for providing his wonderful answer in the first place. I am adding another answer, because it wouldn't fit in a comment and it would be too extensive for an edit.)
Here's what worked for me to download the latest version of an artifact called "component.jar" with Maven 3.1.1 in the end (other suggestions did not, mostly due to maven version changes I believe)
This actually downloads the file and copies it into the local working directory
From bash:
mvn dependency:get \
-DrepoUrl=http://.../ \
-Dartifact=com.foo.something:component:LATEST:jar \
-Dtransitive=false \
-Ddest=component.jar \
Regarding how to get the artifact binary, Pascal Thivent's answer is it, but to also get the artifact sources jar, we can use:
mvn dependency:get -Dartifact=groupId:artifactId:version:jar:sources
e.g.
mvn dependency:get -Dartifact=junit:junit:4.12:jar:sources
This works because the artifact parameter actually consists of groupId:artifactId:version[:packaging][:classifier]. Just the packaging and classifier are optional.
With jar as packaging and sources as classifier, the maven dependency plugin understands we're asking for the sources jar, not the artifact jar.
Unfortunately for now sources jar files cannot be downloaded transitively, which does make sense, but ideally I do believe it can also respect the option downloadSources just like the maven eclipse plugin does.
One could use dependency:copy (http://maven.apache.org/plugins/maven-dependency-plugin/copy-mojo.html) which takes a list of artifacts defined in the plugin configuration section and copies them to a specified location, renaming them or stripping the version if desired. This goal can resolve the artifacts from remote repositories if they don't exist in either the local repository or the reactor.
Not all the properties of the plugin could be used in maven CLI. The properties which have "User Property:" property defined could be specified. In the below example I am downloading junit to my temp folder and stripping the vesion from the jar file.
mvn org.apache.maven.plugins:maven-dependency-plugin:2.8:copy -Dartifact=junit:junit:4.11 -DoutputDirectory=/tmp -Dmdep.stripVersion=true
where
artifact=junit:junit:4.11 is the maven coordinates. And you specify artifcat as groupId:artifactId:version[:packaging[:classifier]]
(Thanks to Pascal Thivent for providing his https://stackoverflow.com/a/18632876/2509415 in the first place. I am adding another answer)
The usage from the official documentation:
https://maven.apache.org/plugins/maven-dependency-plugin/usage.html#dependency:get
For my case, see the answer below:
mvn dependency:get -Dartifact=$2:$3:$4:$5 -DremoteRepositories=$1 -Dtransitive=false
mvn dependency:copy -Dartifact=$2:$3:$4:$5 -DremoteRepositories=$1 -Dtransitive=false -DoutputDirectory=$6
#mvn dependency:get -Dartifact=com.huya.mtp:hynswup:1.0.88-SNAPSHOT:jar -DremoteRepositories=http://nexus.google.com:8081/repository/maven-snapshots/ -Dtransitive=false
#mvn dependency:copy -Dartifact=com.huya.mtp:hynswup:1.0.88-SNAPSHOT:jar -DremoteRepositories=http://nexus.google.com:8081/repository/maven-snapshots/ -Dtransitive=false -DoutputDirectory=.
Use the command mvn dependency:get to download the specific artifact and use
the command mvn dependency:copy to copy the downloaded artifact to the destination directory -DoutputDirectory.
one liner to download latest maven artifact without mvn:
curl -O -J -L "https://repository.sonatype.org/service/local/artifact/maven/content?r=central-proxy&g=io.staticcdn.sdk&a=staticcdn-sdk-standalone-optimizer&e=zip&v=LATEST"
maven command:
if you use maven, you can use dependency:copy to download the artifact to the local folder.
mvn dependency:copy -Dartifact=groupId:artifactId:version[:packaging[:classifier]] -DoutputDirectory=<your local path>. -U
Refer: https://maven.apache.org/plugins/maven-dependency-plugin/copy-mojo.html
Curl command:
# guide: https://help.sonatype.com/repomanager3/rest-and-integration-api/search-api
# https://msnexus.xxxx.com/service/rest/v1/search/assets?sort=version&repository=public&maven.groupId=<groupId>&maven.artifactId=<>&maven.baseVersion=1.46.0-SNAPSHOT&maven.extension=war
download_artifact() {
local host_url=$1
local group_id=$2
local artifact_id=$3
local artifact_type=$4
local artifact_version=$5
local final_name=$6
local location=$7
local search_version=$5
local prerelease="false"
if [[ "${artifact_version}" == *"SNAPSHOT" ]]; then
prerelease="true"
fi
if [[ "${artifact_version}" == "latest"* ]]; then
search_version="*"
fi
assets_url="${host_url}/service/rest/v1/search/assets?sort=version&repository=public&maven.groupId=${group_id}&maven.artifactId=${artifact_id}&maven.baseVersion=${search_version}&prerelease=${prerelease}&maven.extension=${artifact_type}"
echo "INFO: Assets url: $assets_url"
download_url=$(curl "$assets_url" -H "accept: application/json" | jq -r ".items[0].downloadUrl // empty")
echo "INFO: Downloading artifact from url: $download_url"
if [[ -z "$download_url" ]]; then
echo "ERROR: Artifact not exists in Nexus, please check your version [${version}] for [${service_name}]"
exit 1
fi
pre_dir=$(pwd)
if [[ ! -d "$location" ]]; then
mkdir -p $location
fi
cd $location
curl -o "${final_name}.${artifact_type}" "$download_url"
cd $pre_dir
}
Here's an example to get ASM-7 using Maven 3.6:
mvn dependency:get -DremoteRepositories=maven.apache.org -Dartifact=org.ow2.asm:7.0:sources:jar
Or you can download the jar from here: https://search.maven.org/search?q=g:org.ow2.asm%20AND%20a:asm and then
mvn install:install-file -DgroupId=org.ow2.asm -DartifactId=asm -Dversion=7.0 -Dclassifier=sources -Dpackaging=jar -Dfile=/path/to/asm-7.0.jar
To copy artifact in specified location use copy instead of get.
mvn org.apache.maven.plugins:maven-dependency-plugin:3.1.2:copy \
-DrepoUrl=someRepositoryUrl \
-Dartifact="com.acme:foo:RELEASE:jar" -Dmdep.stripVersion -DoutputDirectory=/tmp/
You can also do this using docker in PowerShell:
docker run -it --rm -v ${PWD}:/build/source -v ${HOME}/.m2:/build/.m2 --net=host aemdesign/centos-java-buildpack:jdk8 /bin/bash --login -c 'mvn dependency:get -Dmaven.repo.local=/build/.m2/repository -DrepoUrl=https://repo1.maven.org/maven2 -Dartifact=io.prometheus.jmx:jmx_prometheus_javaagent:LATEST -Ddest=/build/source/jmx_prometheus_javaagent.jar'
or in bash:
docker run -it --rm -v $PWD:/build/source -v $HOME/.m2:/build/.m2 --net=host aemdesign/centos-java-buildpack:jdk8 /bin/bash --login -c 'mvn dependency:get -Dmaven.repo.local=/build/.m2/repository -DrepoUrl=https://repo1.maven.org/maven2 -Dartifact=io.prometheus.jmx:jmx_prometheus_javaagent:LATEST -Ddest=/build/source/jmx_prometheus_javaagent.jar'
The command:
mvn install:install-file
Typically installs the artifact in your local repository, so you shouldn't need to download it. However, if you want to share your artifact with others, you will need to deploy the artifact to a central repository see the deploy plugin for more details.
Additionally adding a dependency to your POM will automatically fetch any third-party artifacts you need when you build your project. I.e. This will download the artifact from the central repository.
LATEST is deprecated, try with range [,)
./mvnw org.apache.maven.plugins:maven-dependency-plugin:3.1.1:get \
-DremoteRepositories=repoId::default::https://nexus/repository/maven-releases/ \
"-Dartifact=com.acme:foo:[,)"
Unfortunately maven-dependency-plugin:get do NOT support version ranges e.g. [2.17.1,) or [,)
If you need download a specific maven artifact but using version range, as I do, look here:
Download Maven artifact with version range