Force xmllint to ignore bad default xmlns - xmllint

I am trying to process a large number of xml files (maven poms) using xmllint --xpath. With some trial and error I figured out that it does not work as expected due to the bad default namespace declaration in these files, which is as follows:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
A simple command fails as follows:
$ echo $(xmllint --xpath '/project/modelVersion/text()' pom.xml )
XPath set is empty
If I get rid of the xmlns attribute, replacing the root element as follows:
<project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
The previous command gives the expected output:
$ echo $(xmllint --xpath '/project/modelVersion/text()' pom.xml )
4.0.0
Changing hundreds of pom files is not an option, especially since maven itself does not complain.
Is there a way for the xmllint to process the file with the bad xmlns?
UPDATE
Thanks to Damien I was able to make some progress:
$ ( echo setns x=http://maven.apache.org/POM/4.0.0; echo 'xpath /x:project/x:modelVersion/text()'; ) | xmllint --shell pom.xml
/ > setns x=http://maven.apache.org/POM/4.0.0
/ > xpath /x:project/x:modelVersion/text()
Object is a Node Set :
Set contains 1 nodes:
1 TEXT
content=4.0.0
But this does not quite do what I need. My follow up questions are as follows:
Is there a way to print only the text? I would like the output to contain on 4.0.0 in the above example
It seems the output gets truncated after about 30 characters. Is it possible to get complete output? This does not happen with xmllint --xpath

strip the namespace with sed
given in pom.xml:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
</project>
this:
cat pom.xml | sed '2 s/xmlns=".*"//g' | xmllint --xpath '/project/modelVersion' -
returns this:
<modelVersion>4.0.0</modelVersion>
if you have funky formatting (like, the xmlns attributes are on their own lines), run it through the formatter first:
cat pom.xml | xmllint --format - | sed '2 s/xmlns=".*"//g' | xmllint --xpath '/project/modelVersion' -

xmllint --xpath "/*[local-name() = 'project']/*[local-name() = 'parent']/*[local-name() = 'version']/text()" pom.xml
For a top level pom.xml:
xmllint --xpath "/*[local-name() = 'project']/*[local-name() = 'version']/text()" pom.xml
It ain't real pretty, but it avoids formatting assumptions and/or re-formatting the input pom.xml file.
If you need to strip off the "-SNAPSHOT" for some reason, pipe the result of the above through | sed -e "s|-SNAPSHOT||".

Related

WebDeploy: Change Package Output folder in Web Publishing Pipeline

I have a Visual Studio solution with multiple webapp projects. The build should create a web package for each project. The web packages should finally end up in a folder structor like this:
$(Outputfolder)
|
+-- Web
|
+-- <name package 1>
| |
| +-- ... package files ...
|
+-- <name package 2>
| |
| +-- ... package files ...
|
+-- ...
|
In order to change the destination folder for a web package I have added a .wpp.targets file to each web app project. Here I have adjusted the DefaultPackageOutputDir property:
<?xml version="1.0" encoding="utf-8" ?>
<Project ToolsVersion="12.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup>
<DefaultPackageOutputDir Condition=" '$(DefaultPackageOutputDir)'=='' ">$(OutFolder)Web\Webapp1\</DefaultPackageOutputDir>
</PropertyGroup>
</Project>
This is it how I call MSBuild. I simply hand over the output folder as a property:
<MSBuild Projects="#(ItemToBuild)"
Targets="Build"
Properties="Configuration=$(Configuration);
Platform=$(Platform);
DeployOnBuild=True;
DeployTarget=Package;
OutFolder=$(OutFolder)" />
This does the trick but I'm not completely satisfied. I want to make the build more general. It bothers me, that I have to name the webapp explicitly. My idea was to use the property DefaultMSDeployDestinationApplicationName instead:
<?xml version="1.0" encoding="utf-8" ?>
<Project ToolsVersion="12.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup>
<DefaultPackageOutputDir Condition=" '$(DefaultPackageOutputDir)'=='' ">$(OutFolder)Web\$(DefaultMSDeployDestinationApplicationName)\</DefaultPackageOutputDir>
</PropertyGroup>
</Project>
Unfortunately the property DefaultMSDeployDestinationApplicationName seems to be empty. The package files end up in the Web folder. I guess the property DefaultMSDeployDestinationApplicationName is not yet defined at the time the .wpp.targets file is readed.
Does somebody know a better place to define the property DefaultPackageOutputDir?
I was able to change the output directory for a Web Project by specifying the following options to MsBuild:
/p:OutDir="$(build.artifactstagingdirectory)\$(BuildConfiguration)" /p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true

xmllint: Remove <?xml version="1.0"?>

I am using xmllint to clean up my .xml file but for some reason the output adds <?xml version="1.0"?> to the first line. Is there a flag to have this be removed?
Both of the following seem to avoid insertion of <?xml version="1.0"?>:
xmllint --exc-c14n data.xml
xmllint --c14n data.xml
Hint:
xmllint --help
-or-
man xmllint

Maven property overloading

I have very simple maven descriptor which defined some properties:
<?xml version="1.0"?>
<project
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd"
xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<properties>
<it.port>8080</it.port>
</properties>
</project>
I can override it.port property with command:
$ mvn -Dit.port=8181 verify
But following command doesn't work as expected:
$ MAVEN_OPTS="-Dit.port=8181" mvn verify
This pass system variable to the JVM but maven refuse to override this property and default value given to test (8080). Original problem is that TeamCity (out CI server) pass system variables to the JVM in MAVEN_OPTS, so property overriding doesn't work.
Can I override maven properties with MAVEN_OPTS environment variable?
No you can't. You can:
Use settings.xml on your local machine to specify the property
Use a profile in the project pom
Use -D directly on the command line.

Maven: Using inherited property in dependency classifier causes build failure

Given three POM files:
C depends on B.
B inherits from A.
I can build A and B
C fails to build because of its dependency on B.
The full source-code and build output is included below for your review.
Here is A's POM:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.foo</groupId>
<artifactId>A</artifactId>
<packaging>pom</packaging>
<version>1.0-SNAPSHOT</version>
<name>A</name>
<repositories>
<repository>
<id>foo releases</id>
<name>libs-releases-local</name>
<layout>default</layout>
<url>http://foo.net/artifactory/libs-releases-local</url>
</repository>
</repositories>
<dependencies>
<dependency>
<groupId>org.eclipse.swt</groupId>
<artifactId>swt</artifactId>
<classifier>${swt.classifier}</classifier>
<version>3.6.1</version>
</dependency>
</dependencies>
<profiles>
<profile>
<id>windows-x86</id>
<properties>
<swt.classifier>win32-x86</swt.classifier>
</properties>
</profile>
</profiles>
</project>
Here is B's POM:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>com.foo</groupId>
<artifactId>A</artifactId>
<version>1.0-SNAPSHOT</version>
<relativePath>../A</relativePath>
</parent>
<artifactId>B</artifactId>
<packaging>jar</packaging>
<name>B</name>
<profiles>
<profile>
<id>windows-x86</id>
<properties>
<swt.classifier>win32-x86</swt.classifier>
</properties>
</profile>
</profiles>
</project>
Here is C's POM:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.foo</groupId>
<artifactId>C</artifactId>
<packaging>jar</packaging>
<version>1.0-SNAPSHOT</version>
<name>C</name>
<dependencies>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>B</artifactId>
<version>${project.version}</version>
</dependency>
</dependencies>
</project>
Here is the build output from C:
------------------------------------------------------------------------
Building C
task-segment: [install]
------------------------------------------------------------------------
[compiler:compile]
Nothing to compile - all classes are up to date
Downloading: http://foo.net/artifactory/libs-releases-local/org/eclipse/swt/swt/3.6.1/swt-3.6.1-${swt.classifier}.jar
[WARNING] Unable to get resource 'org.eclipse.swt:swt:jar:${swt.classifier}:3.6.1' from repository foo releases (http://foo.net/artifactory/libs-releases-local): Error transferring file: foo.net
Downloading: http://repo1.maven.org/maven2/org/eclipse/swt/swt/3.6.1/swt-3.6.1-${swt.classifier}.jar
Unable to find resource 'org.eclipse.swt:swt:jar:${swt.classifier}:3.6.1' in repository central (http://repo1.maven.org/maven2)
------------------------------------------------------------------------
[ERROR]BUILD ERROR
------------------------------------------------------------------------
Failed to resolve artifact.
Missing:
----------
1) org.eclipse.swt:swt:jar:${swt.classifier}:3.6.1
I know this issue is related to https://issues.apache.org/jira/browse/MNG-3228 but I'm not sure how to fix it. Please help!
UPDATE:
Adding a classifier to B helped. Now C builds so long as the repository only contains B's jar file. If I upload B's POM file alongside the JAR in the repository, C fails with the aforementioned error (${swt.classifier} not defined). Any ideas?
In a comment you write, "I am expecting SWT's classifier to get resolved at B's build-time, not C's build-time", but that is wrong - you need the classifier at C's build time because C has a dependency on swt (transitive via A). That dependency is only fully-determined by a property, so you have to have a way of evaluating the property in C's pom.
A depends on swt-${classifier}
C depends on A
therefore C depends on swt-${classifier}
therefore C's pom must define the property. It can be defined by a profile (as in A), or manually at runtime (bad for reproducibility), but you can't build C without it.
It's as simple (and puzzling) as that.
If you're expecting the property to somehow get fully "resolved" along the way, and be already well-defined by the time you build C, you are not understanding how Maven treats those properties. It leaves them alone. There was an attempt to do some different stuff in Maven 2.1 (the classifier property expression would be transformed to its value when you installed A), but it was not successful, caused many surprising behaviours, it was reverted for 2.2, and in fact caused 2.1 to be quickly deprecated. For more details, and some hints as to how complicated the problem really is, see the link below.
https://cwiki.apache.org/confluence/display/MAVENOLD/Artifact-Coordinate+Expression+Transformation
Until the Maven developers decide otherwise, I think we will continue to have the behaviour that has been around since 2.0: "Expressions in artifact coordinates are ignored. Users have plenty of rope with which to hang themselves"
Once you get used to it, though, it's not confusing anymore. It's only when you're trying to second-guess Maven that you get surprised.
Maven is trying to find the artifact org.eclipse.swt:swt:3.6.1 but the coordinates aren't being resolved correctly. The error is saying that ${swt.classifier} isn't being recognized from a <properties/> block in your POM.xml. Since that value shows up in a <profile/> block, can you verify what Maven command you are running?
Try this: mvn dependency:resolve -P windows-x86
Also, verify that both the version of SWT and the Classifier are actually correct. The latest version I see on Maven Central is not 3.6.0, but 3.3.0-v3346
This isn't possible as of Maven 3.1.0. Here is the relevant feature request: https://issues.apache.org/jira/browse/MNG-1388
I know this issue is related to https://issues.apache.org/jira/browse/MNG-3228 but I'm not sure how to fix it.
I'm not sure there is a link with this issue, I don't see anything related to profile activation in the pom.xml shown above.
Actually, I'm not even sure to understand the expected result. From where is the classifier supposed to come? Maybe I'm missing some parts but I think you should install/deploy a qualified version of B (with a fully resolved POM) and have C depend on this qualified version.
How would I need to modify B's POM to deploy a qualified version? I am expecting SWT's classifier to get resolved at B's build-time, not C's build-time.
Yes but at C's build-time, C needs B and B's dependencies so the installed/deployed .pom of B has to be fully resolved. At least, that's how I think things can work.
But I must admit I'm not sure how to handle this case exactly and after reading issues like MNG-4140 or the Artifact-Coordinate Expression Transformation page, I'm totally confused.
I suggest to post this on the maven users list for the right way (and I'll follow the thread closely because I think I have some broken POMs using profiles, properties and dependencies to fix now, thanks :)

How to configure a subproject dependency in Maven without deploying jars?

I googled this and it seems that no one has an answer, yet it seems like such an elementary thing that it should be possible.
I have the following project structure:
parent
---sub-project1
---sub-project2
sub-project2 needs to have sub-project1 as a dependency.
So I have this in sub-project2's pom:
<dependencies>
<dependency>
<artifactId>sub-project1</artifactId>
<groupId>mygroup</groupId>
<version>1.0-SNAPSHOT</version>
</dependency>
....
When I do this, Maven tries to dowload the sub-project1.jar file, which does not exist because it's not ready for the repo yet.
I tried to put a <scope>import</scope> in the dependency, but that didn't work either -- same result.
So what do I have to do to get Maven to look at sub-project1 when building sub-project2?
EDIT Here are some pom snippets:
Parent:
<project
xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<prerequisites>
<maven>2.0.9</maven>
</prerequisites>
<modules>
<module>sub-project1</module>
<module>sub-project2</module>
</modules>
....
sub-project1:
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<artifactId>parent</artifactId>
<groupId>mygroup</groupId>
<version>1.0-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>sub-project1</artifactId>
....
sub-project2:
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<artifactId>parent</artifactId>
<groupId>mygroup</groupId>
<version>1.0-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>sub-project1</artifactId>
<dependencies>
....
<dependency>
<artifactId>sub-project2</artifactId>
<groupId>mygroup</groupId>
<version>1.0-SNAPSHOT</version>
<scope>import</scope>
</dependency>
</dependencies>
The error I'm getting when I got mvn clean install on the parent is:
[ERROR] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Compilation failure
With a lot of classes/package not found errors
You should have a master pom at parent's level, in which you will list the modules of your project.
<modules>
<module>sub-project1</module>
<module>sub-project2</module>>
</modules>
In each subproject you have to reference your parent:
<parent>
<artifactId>parent</artifactId>
<groupId>mygroup</groupId>
<version>1.0-SNAPSHOT</version>
</parent>
And you specify the dependencies between the project just as you did. I think you've missed some of the steps I've described.
Edit: you should issue your mvn clean install at the parent level.
When I do this, Maven tries to dowload the sub-project1.jar file, which does not exist because it's not ready for the repo yet.
That's the normal behavior, Maven resolves dependencies through the local repository so you need to install sub-project1 first. Actually, the common way to deal with this kind of situation is to launch a reactor build (a multi-modules build) from the parent.
Assuming you are aggregating modules in the parent i.e. you have something like this declared in the "parent" pom.xml:
<modules>
<module>sub-project1</module>
<module>sub-project2</module>>
</modules>
Just cd into the parent directory and launch a reactor build:
$ cd parent
$ mvn install
Maven will then calculate the build order (deducted from the oriented graph made of modules and their dependencies) and run install on all modules in the calculated order (parent first, then sub-project1 and finally sub-project2 for your particular example).
But don't use a scope of type import, you are misusing it here. Remove it.
Update: The question has been updated while I was answering and the POMs shown do no illustrate the situation given in the original question (reversed dependency, probable mistake in the artifact id). But the suggested approach still applies. Remove the <scope>import</scope> on the dependency and start a reactor build from the parent.