Similar to this issue but that is for Windows and there is no correct answer.
I want to do a simple deployment using Bamboo (cloud version). The idea is we checkout the data from a Bitbucket repository and then use SCP to publish content.
The problem is when SCP runs, I keep getting:
There were no files
I have set up our Bitbucket repo and Bamboo can connect fine. However, I'm not sure where Bamboo checks out the files? I didn't set up our Bamboo instance, so I found that in the config settings, the default path is:
/home/bamboo/bamboo-agent-home/xml-data/build-dir/
I would have thought this should be really straight forward. The repo gets checked out into /a/path/somewhere/ and SCP uploads from /a/path/somewhere. The problem is I don't know where the path is and I cannot find any documentation that tells me where it is.
My previous experience with Bamboo SCP Task have shown that it does not work well with Windows SSH servers. I tried freeSSHd and SolarWinds SCP server and I could not connect to any of them through SCP Task.
As a workaround I used Maven build task with the following ant-run configuration:
<configuration>
<tasks>
<scp todir="username:password#xxx.xxx.xxx.xxx:/" trust="true" failonerror="false">
<fileset dir="dirname">
<include name="**/*">
</fileset>
</scp>
</tasks>
</configuration>
I had this task in a different Bamboo stage. To pass files (artifacts) from one stage to another:
click Create definition under the Artifacts tab in the first job configuration (which generates the files you wanna pass)
click Create dependency under the Artifacts tab in the seconds job configuration which runs previously mentioned ant-run task
I think you want to use the SCP Task as one of the tasks (the final one) of your Deployment project. To use that task you'll need to share one or more artifacts that are produced by the associated Build project.
Related
How do I run several buildsteps after each other in IntelliJ? I think I want a mini CI/CD build system inside the editor.
For example, the project I work on now is a Spring boot and javascript web site. I need to build it with maven with mvn clean package -Pdockerimage. This copies files for building the Docker image to target/dockerimgbuild.
Then I want to build the docker image using docker build -t scheduling-ui-dev . and after that run it with docker compose docker-compose up --build from src/main/resources/docker-compose.
I have built one run configuration for each of these steps but how do I run them after each other? I have found that you can have before launch but the system is clunky and complains if target/dockerimgbuild doesn't exists even before it have run the maven step which creates it. Latest problem I stumbled on was that a file prevented maven from removing target/dockerimgbuild and all run steps was automatically removed from the run configurations.
There is a run configuration called compound but that runs everything in parallell and you can not specify order which is a problem.
I wonder if it is feasible to start TeamCity in a container, do anyone have a clue about that (is teamcity easy to configure, how to make it launch a docker-compose container on my host machine etc)?
My solution right now is to have several terminals (if this gets more permanent I will replace it with a script) where I just press up and enter to execute the steps manually. Seems stupid as I guess maven itself can do all of this...but I don't know how or how much work it is.
There is a compound Run/Debug configuration: https://www.jetbrains.com/help/idea/run-debug-configuration-compound-run-configuration.html
Also, there is a multi-run plugin: https://plugins.jetbrains.com/plugin/7248-multirun
My pom.xml contains:
<distributionManagement>
<repository>
<id>ssh-myown-repository</id>
<url>scpexe://user#host/path/to/repository</url>
</repository>
</distributionManagement>
So whenever I issue:
mvn deploy
My project is, well, deployed properly on my remote repository, under a name such as this:
http://host/repository/project/0.7.0-SNAPSHOT/project-0.7.0-20120518.212052-3.jar
I know file "maven-metadata.xml" contains information about which file is the latest release, but I'd very much like the mvn deploy stage to create a link to the latest jar file, say:
http://host/repository/project/0.7.0-SNAPSHOT/project-0.7.0-latest.jar
I understand "scpexe" has no way of doing this directly as links are part of the file system on which the web server is running (ie.: they are not represented as uploadable file content).
I have shell access on this remote server, so is there a way to hook on maven's deploy action to execute a script after deployment is complete ?
You know that the -SNAPSHOT is a thing like the "LATEST". Furthermore i recommend to use a repository manager.
I've looked at the usage, but haven't understood from it how to configure multiple servers. I added separate server elements to settings.xml - but I don't understand how to specify a different URL for every server.
The URL element belongs to the global plugin configuration. How do I configure multiple server URLs?
You could add multiple profiles to your pom.xml. One for each server. Check the Maven documentation on profiles for details!
If you examine the documentation of the Tomcat plugin you will see that it does not support multiple <configuration> sections. That should be a small addition because in the deploy phase you only copy the WAR file to the server using an HTTP PUT command. So I wonder why they have not added this capability to the plugin.
Anyhow, one possible workaround is to:
Make multiple copies of your pom.xml in the same directory but give them unique names, e.g. dev_1_pom.xml, or dev_<some_machine_name_or_IP>, qa_1_pom.xml ..... You can keep your development pom.xml file name the same because you will likely still run Maven from the command line. Personally, I prefer running the mvn command from my IDE (a button click away vs. typing an mvn command with arguments every time).
In each of the copies, change the <configuration> section under your Tomcat plugin to point to a different server that matches the name of your specific pom.xml. You will need corresponding sections in settings.xml
Create corresponding External Tools Configuration(s) (Eclipse, or other IDE) and each one call the corresponding POM file. Here is an example with Eclipse:
Open External Tools Configuration Dialog in Eclipse (either from the dropdown menu next to the button, or by going to the menu bar and clicking Run > External Tools > External Tools Configurations). Then on the Main Tab, provide values for the following fields
Location: C:\downloads\tools\apache-maven-3.0.3\bin\mvn.bat
Working directory: ${workspace_loc:/<project_name>} - replace <project_name> with the name of your project
Arguments: -f <pom_file_name> <other_arguments> - <other_arguments> could be tomcat7:redeploy
Now you can run these external tool launchers individually to deploy to different servers.
Optionally, extract the mvn commands from your launchers and create a shell script (batch or Unix bash script) that runs all of them. That way you can deploy to multiple servers at once. You can also run this script from Eclipse. Create a new External Tools Configuration launcher but this time your Location: field will point to cmd (Windows) or bash (Unix, Linux ...), not mvn
I am looking for a way to capture the unique SNAPSHOT build number that was generated during the 'mvn deploy' phase of the build. I would like to be able to pass this version (during a Hudson build) to another process used for deploying to an application server. The key here is being able to capture the exact Maven SNAPSHOT build number, such as:
foobar-0.4-20100707.060244-11.war
I have noticed that Hudson is capturing this information if you archive the maven build artifact, but it is not clear how I can expose this variable and pass it to another job (which is what I want to do). I can see this variable in the Hudson Home directory, like so:
/hudson/jobs/JOB_NAME/builds/24/org.ace.widgets$foobar/archive/org.ace.widgets/foobar/0.4-20100707.060244-11
Any Maven and/or Hudson experts out there that have any clue how to expose the SNAPSHOT build number? Hudson is doing it?
Check out my answer to this slightly different problem where I use GMaven to access the project metadata after deploy. What they have in common is that you have to access the unique build number. So you could adapt the script so that after it has read the project meta data (after deploy) it stores the unique version in the maven properties:
pom.properties['uniqueVersion'] = uniqueVersion
If the appserver-deploy-process is also a maven plugin, access this property, otherwise store it as a file using something like this:
new File(pom.build.directory, "uniqueVersion.txt").text = uniqueVersion
Now you can pick it up from target/uniqueVersion.txt using a shell script or so.
A little bit late, but I just noticed that you have exactly the same problem I was facing. I needed to be able to deploy arbitrary build artifacts produced on a Windows box on AIX machines. The deployment process needed to be run locally on the AIX boxes. So I defined slaves on the AIX boxes that download the build artifacts from the build job that run on the Windows box. The master is on the Windows box.
To put it in short. The build job archives the necessary artifacts and triggers the deployment job with its build URL as the parameter (it is actually a "run parameter", but string would work too). The deploy job uses wget to determine the artifact URL (it searches for a artifact ID that contains certain texts, e.g. the artifact name without the version), and downloads the artifact with wget again. wget saves it without version, so that all my deployment scripts can run on the unversioned name. You can use the first step also to find out about the artifact name as well.
wget uses the remote API (xml version). If you don't want to use wget, there is a command line tool that will do the connection for you.
You can test it by appending following strings to the run URL of the build job and use the resulting URL in your webbrowser.
#to find the path (URL) of the artifact
api/xml?xpath=*/artifact[contains(fileName,"MyApp")]/relativePath/text()
#to find the path (URL) of the artifact with more than 1 string to match
#the match must identify exactly one artifact, otherwise you will
#get an error message
api/xml?xpath=*/artifact[contains(fileName,"MyApp") and contains(fileName,".ear")]/relativePath/text()
#To download the artifact
#replace $relativePath with the actual output from one of the queries above
artifact/$relativePath
Based on Sean Patrick Floyds answer, even though this question is old, I'd like to share a full code example which writes the uniqueVersion into a maven property. Please keep in mind that Maven's internals are accessed using reflection, so this may easiliy breaks up in future maven versions. I tested it against Maven 3.2.5.
<build>
<plugins>
<plugin>
<groupId>org.codehaus.groovy.maven</groupId>
<artifactId>gmaven-plugin</artifactId>
<executions>
<execution>
<id>set-uniqueVersion-property</id>
<phase>deploy</phase>
<goals>
<goal>execute</goal>
</goals>
<configuration>
<source>
<![CDATA[
def uniqueVersion = null;
println "Grabbing uniqueVersion...";
project.artifact.metadataList.each{
if(it.class.simpleName=='ProjectArtifactMetadata'){
def afi = it.class.superclass.superclass.getDeclaredField('artifact');
afi.accessible = true;
uniqueVersion = it.artifact.version;
}
};
project.properties['uniqueVersion'] = uniqueVersion;
println("Unique Version grabbed: $uniqueVersion");
]]>
</source>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
If you want to use this property within other maven plugins, please make sure that their executions are made in the deploy phase after the set-uniqueVersion-property execution. If you prefer to write the unique version to a file, simply add
new File(pom.build.directory, "uniqueVersion.txt").text = uniqueVersion
as already stated by Sean Patrick.
Is it possible to use Ant scripts to checkin/checkout source code elements while using the CCRC plugin to eclipse? I am getting an error message saying that the element the script is attempting to check out is not part of the VOB, but of course it is there and I can check it out manually.
It should be possible to use those Ant ClearCase tasks with CCRC views ("web views" which are anologous to snapshot views)
A script like this one should work:
<project name="Testing ClearCase " default="CC" basedir=".">
<target name="CC">
<property name="FileSrc" value="MyView/MyVob/MyDir"/>
<property name="dist" value="dist"/>
<cccheckout viewpath="${FileSrc}/myFile"
reserved="false"
nowarn="true"
comment="Auto Build from script"
failonerr="false" />
<copy file="${dist}/myFile" tofile="${FileSrc}/myFile"/>
<cccheckin viewpath="${FileSrc}/myFile"
comment="Checked in by myFile.xml ANT script"
nowarn="false"
failonerr="false"
identical="true"/>
</target>
</project>
But you need to make sure your current directory is (in this script) just above where you update your web CCRC view "myView".
The only issues I know of are:
if CCRC try to checkout a file of a replicated Vob.
if the parent directory of a file to be checked-in was renamed from another view
The Ant ClearCase tasks in VonC's answer use the cleartool command (getClearToolCommand() in org.apache.tools.ant.taskdefs.optional.clearcase.ClearCase.java). When I invoke a cleartool operation, even from within or above the CCRC view, I get the error message from the question.
Now (as some years have passed since VonC's answer) there is a CCRC CLI that can be used instead (http://www-01.ibm.com/support/docview.wss?uid=swg24021929, setting CCSHARED to your top level \eclipse directory). The commands are similar to those provided by cleartool, although as it appears not to support UCM to solve your problem of doing a check out I first had to set an activity on the stream using the CCRC eclipse plugin.
To get the CCRC CLI to work with the ant ClearCase tasks would require changing the task to:
Call rcleartool rather than cleartool.
Since cleartool points to an .exe and rcleartool is a bat for loading a jar, ProcessBuilder won't be able to process the new command (I tested with rcleartool.bat and cmd \c rcleartool.bat) unless you convert the jar to an exe.