Compiling gRPC,with both Kotlin and Java using Maven (various packages not found) - kotlin

I have a mixed kotlin/java project and I wanted to add gRPC to it. I configured the gRPC plugin as usual:
<extensions>
<extension>
<groupId>kr.motd.maven</groupId>
<artifactId>os-maven-plugin</artifactId>
<version>1.6.2</version>
</extension>
</extensions>
<plugins>
<plugin>
<groupId>org.xolstice.maven.plugins</groupId>
<artifactId>protobuf-maven-plugin</artifactId>
<version>0.6.1</version>
<configuration>
<protocArtifact>
com.google.protobuf:protoc:3.11.2:exe:${os.detected.classifier}
</protocArtifact>
<pluginId>grpc-java</pluginId>
<pluginArtifact>
io.grpc:protoc-gen-grpc-java:1.25.0:exe:${os.detected.classifier}
</pluginArtifact>
</configuration>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>compile-custom</goal>
</goals>
</execution>
</executions>
</plugin>
...
but when building from scratch
mvn clean && mvn package
the kotlin code would never find the generated sources. When building first from IntelliJ it would work fine - IntelliJ knows what to do.
I tried configuring build-helper-maven-plugin to include the generated-sources directories that the protobuf compiler and stub generator use, but that didn't seem to have any effect (and in fact seems unnecessary).

The answer it turns out is because of the order plugins execute. Generally for kotlin-java hybrid projects you want to compile the kotlin first so that the java compiler knows how to find kotlin classes. The problem is, if you do that the kotlin compiler doesn't see the protoc-generated java classes yet. It's a circular dependency.
It took me a while to figure it out, but to fix it you just have to tell the kotlin compiler where to find the protoc-generated sources like this:
<plugin>
<groupId>org.jetbrains.kotlin</groupId>
<artifactId>kotlin-maven-plugin</artifactId>
<version>${kotlin.version}</version>
<executions>
<execution>
<id>compile</id>
<goals> <goal>compile</goal> </goals>
<configuration>
<sourceDirs>
<sourceDir>${project.basedir}/src/main/java</sourceDir>
<sourceDir>${project.build.directory}/generated-sources/protobuf/java/</sourceDir>
<sourceDir>${project.build.directory}/generated-sources/protobuf/grpc-java/</sourceDir>
</sourceDirs>
</configuration>
</execution>
...
</plugin>
It took me entirely too long to figure that out, so I'm posting this as a self-answered question to perhaps save somebody else some time.
I saw a number of related questions but none exactly the same. Good luck.

Related

Duplicate class error when both axistools and clover plugin are included in maven project

We have a maven project which uses clover and axistools-wsdl2java plugin. Platfrom is windows.
We are using clover 2.4.0 plugin to get code coverage which integrated in project's pom.xml.
Configured the clover plugin as shown below.
<plugin>
<groupId>com.atlassian.maven.plugins</groupId>
<artifactId>maven-clover2-plugin</artifactId>
<version>2.4.0</version>
<configuration> <generatePdf>true</generatePdf>
<generateXml>true</generateXml>
<generateHtml>true</generateHtml>
<licenseLocation>C:/EcasSVNCO/ews/ews-mvn/ewsbase/src/test/resources/license/clover.license</licenseLocation>
<reportDescriptor>C:/EcasSVNCO/ews/ews-mvn/ewsbase/src/test/resources/descriptor/default-clover-report.xml</reportDescriptor>
<excludes>
<exclude>${basedir}/src/main/java/*.java</exclude>
</excludes>
</configuration>
<executions>
<execution>
<id>install</id>
<phase>install</phase>
<goals>
<goal>instrument</goal>
<goal>clover</goal>
</goals>
</execution>
<!--
<execution>
<id>test</id>
<phase>test</phase>
<goals>
<goal>instrument</goal>
<goal>clover</goal>
</goals>
</execution>
<execution>
<id>site</id>
<phase>pre-site</phase>
<goals>
<goal>instrument</goal>
<goal>clover</goal>
</goals>
</execution>
-->
</executions>
</plugin>
Also, there is axistools plugin used to generate classes using wsdl file, which is configured as below.
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>axistools-maven-plugin</artifactId>
<version>1.4</version>
<configuration>
<sourceDirectory>${base.dir}/src/main/resources/wsdl</sourceDirectory>
<outputDirectory>${base.dir}/src/main/java</outputDirectory>
<wsdlFiles>
<wsdlFiles>wsecas.wsdl</wsdlFiles>
</wsdlFiles>
<packageSpace>com.symantec.wsecas</packageSpace>
<testCases>true</testCases>
<serverSide>true</serverSide>
<phase>install</phase>
<subPackageByFileName>true</subPackageByFileName>
</configuration>
<executions>
<execution>
<goals>
<goal>wsdl2java</goal>
</goals>
</execution>
</executions>
</plugin>
When we execute a command 'mvn clean install' , compilation will go fine. Then first axistool's wsdl2java goal is ran and source files are generated in respective directory.
Next clover plugin tries to instrument the source code by running unit test cases, when this plugin places the instrumented source code under
{basedir}/target/clover/src/main/java/...
Then it fires "compile" goal to compile all the source code. While compiling the source code, two source paths are getting added i.e.
{basedir}/src/main/java/... and {basedir}/target/clover/src/main/java/... , both have same classes.
When maven compiler tries to compile these sources compilation is failing by throwing "Duplicate Class Error".
But when we comment out axistools plugin, clover instrumentation and report generation goes fine.
If any of you have come across similar issue 'Duplicate class error', please guide us on this regard.
Any suggestion and help will be greatly appreciated.
It seems that the problem is that your Axis plugin generates sources to src/main/java (it should be rather a target/src-generated).
When the Clover is invoked it will first take 'normal' sources from src/main/java, instrument them and put into target/clover/java. Next Clover will change list of source roots replacing src/main/java by target/clover/java.
Next the AXIS code generation comes into play and it generates new sources again into src/main/java adding also this directory as extra source root.
As a consequence a compiler sees the same set of sources (i.e. non-generated ones) in two locations and complains about it.
This case is very similar to the one described in Clover's Knowledge Base:
https://confluence.atlassian.com/display/CLOVERKB/Duplicate+class+errors+with+Clover+and+JAXB+or+JAXB2+plugin
https://confluence.atlassian.com/display/CLOVERKB/Duplicate+class+errors+with+Clover+and+jaxws-maven+plugin
you will find possible solutions in these articles.

Maven annotation processing with maven-compiler-plugin

I try to compile my code that contains annotations that generate source code. I use the maven-compiler-plugin and build-helper-maven-plugin. My POM is looking like that:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.2</version>
<configuration>
<source>1.6</source>
<target>1.6</target>
<generatedSourcesDirectory>${project.build.directory}/generated-sources/apt</generatedSourcesDirectory>
</configuration>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>1.5</version>
<executions>
<execution>
<phase>generate-sources</phase>
<goals>
<goal>add-source</goal>
</goals>
<configuration>
<sources>
<source>${project.build.directory}/generated-sources/apt</source>
</sources>
</configuration>
</execution>
</executions>
</plugin>
When I run mvn compile, ${project.build.directory}/generated-sources/apt is added as a source directory, and the generated sources are generated in the correct directory. But I get compiler errors because of missing references to the generated classes. It's like the generated source directory is not included in the compilation process.
I also try apt-maven-plugin which does not generate anything. And maven-annotation-plugin but it behaves as describe above.
The release 2.0.7-SNAPSHOT of maven-annotation-plugin should solve the problem
fwiw I just downgraded from 3.0 to 2.5.1 and fixed an issue I was seeing with APT processing not being executed after a few runs, using maven so just downgraded one version:
http://search.maven.org/#search%7Cgav%7C1%7Cg%3A%22org.apache.maven.plugins%22%20AND%20a%3A%22maven-compiler-plugin%22
Can you try with the latest version of maven compiler plugin (2.3.2)?
Also build-helper-maven-plugin is not required since it looks like you are using the maven-compiler-plugin itself to generate the sources from the annotation.

OSGI: generate bundle-classpath in maven-bundle-plugin

I am trying to add all the jars from web-inf/lib into the Bundle-ClassPath.
I saw several ways to do it, none of them is working:
1) add
<Embed-Dependency>*;scope=compile|runtime;inline=true</Embed-Dependency>
<Embed-Directory>WEB-INF/lib</Embed-Directory>
<Embed-Transitive>true</Embed-Transitive>
2) add
<Bundle-ClassPath>.,{maven-dependencies},WEB-INF/classes</Bundle-ClassPath>
Of course, writing jars one-by-one in "Bundle-ClassPath" solves the problem, but it doesn't sound like a reasonable solution.
thanks
In your first code snippet, wouldn't using <Embed-Dependency> as you have it written work? The examples at http://felix.apache.org/site/apache-felix-maven-bundle-plugin-bnd.html seem to indicate it would.
Also, what version of the bnd plugin are you using? These features are available as of 1.2.0+.
Working example for a classic webapp being OSGified
<plugin>
<groupId>org.apache.felix</groupId>
<artifactId>maven-bundle-plugin</artifactId>
<extensions>true</extensions>
<executions>
<execution>
<id>bundle-manifest</id>
<phase>process-classes</phase>
<goals>
<goal>manifest</goal>
</goals>
<configuration>
<instructions>
<Private-Package>org.example</Private-Package>
<Web-ContextPath>webappcontextpath</Web-ContextPath>
<Bundle-ClassPath>.,WEB-INF/classes,{maven-dependencies}</Bundle-ClassPath>
<Embed-Dependency>*;scope=compile|runtime</Embed-Dependency>
<Embed-Directory>WEB-INF/lib</Embed-Directory>
</instructions>
<supportedProjectTypes>
<supportedProjectType>war</supportedProjectType>
</supportedProjectTypes>
</configuration>
</execution>
</executions>
</plugin>
Note that Embed-Dependency is inside the instructions element

Maven-2: avoid default packaging?

My project uses many assemblies, hence I'm interested only in the assemblies.
On executing mvn install apart from the assemblies, I'm getting the default packaged .jar.
How can I avoid this?
I have a pom.xml similar to the one you have provided.
On executing mvn install, I'm getting App1.jar, App2.jar, and snapshot jar containing all contents
<build>
<plugins>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<version>2.2-beta-2</version>
<executions>
<execution>
<id>assemblyone</id>
<phase>compile</phase>
<goals>
<goal>single</goal>
</goals>
<configuration>
<finalName>App1</finalName>
<appendAssemblyId>false</appendAssemblyId>
<descriptors>
<descriptor>${basedir}/src/main/resources/assemblies/report.xml</descriptor>
</descriptors>
</configuration>
</execution>
<execution>
<id>assemblytwo</id>
<phase>compile</phase>
<goals>
<goal>single</goal>
</goals>
<configuration>
<finalName>App2</finalName>
<appendAssemblyId>false</appendAssemblyId>
<descriptors>
<descriptor>${basedir}/src/main/resources/assemblies/src.xml</descriptor>
</descriptors>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
How can I avoid this snapshot (not sure of the exact term) jar and ensure that only assemblies are created?
I can read your question two ways, I've outlined answers for both below. If neither is correct, can you modify your question with a bit more explanation please.
1) Do you mean you have a project with default (jar) packaging, and you want to avoid the creation of the jar when no assembly is defined? If this is the case, what is the build achieving if no assembly is defined?
2) Do you instead mean that you are running mvn assembly:assembly to generate the assembly and want to know how to get that assembly when running the install goal?
For option 2, you can bind the assembly-plugin to a lifecycle phase to ensure it is always run, if you specify that <appendAssemblyId> should be false, then the assembly will replace the default jar.
For example, this configuration will invoke the assembly plugin during the packaging phase and replace the default jar:
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<version>2.2-beta-2</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
<configuration>
<appendAssemblyId>false</appendAssemblyId>
<descriptors>
<descriptor>src/main/assembly/archive.xml</descriptor>
</descriptors>
</configuration>
</execution>
</executions>
</plugin>
For option 1, this is actually quite tricky to do. The best I can think of is to specify that the project has pom packaging and configure the project with the executions normally bound to the jar lifecycle in a profile. The lifecycle bindings you'd need to configure are listed in the introduction to the build lifecycle
I'm not sure that you can really do that in a really simple way.
A solution is to call the clean plugin once the build is achieved, by doing that:
<build>
<plugins>
<plugin>
<artifactId>maven-clean-plugin</artifactId>
<executions>
<execution>
<id>auto-clean</id>
<phase>package</phase>
<goals>
<goal>clean</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
...
</build>
This way, the JAR created in the target/ directory will be deleted at the end of the Maven2 execution.
However, you will have to define another directory to store the assemblies created by Maven2. Otherwise, it will be deleted by the call of the clean plugin... If you want to store them in the directory assemblies-target/, you can add that in the pom.xml file:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>2.2-beta-4</version>
<configuration>
...
<!-- Copy the ZIP in target/ of the ROOT directory. -->
<outputDirectory>assemblies-target</outputDirectory>
</configuration>
...
I think it would be much more clear if you showed us your whole POM and the artifacts that are being built. I can only guess as to what the problem is because your terminology is not what I am familiar with. This is my guess as to the problem: you have a POM configured to generated two assembly JARs, but you get a third JAR for the POM itself. In other words, if your POM's artifactId is MyApp, you are getting a MyApp-1.0.0.jar or similar in addition to the two JARs you actually want.
If that is the case, the problem boils down to that you are using Maven to create multiple artifacts from a single module. Maven is designed to produce only one primary artifact from each module. What I would do is change your POM to have a packaging type of "pom" and give it two modules in a <modules> section named App1 and App2. Create sub-directories under your module, one for each App. Give them each a POM configured for a single assembly, with a packaging type of "jar". Move the code/files/etc. as appropriate into each sub-module so there aren't any remaining in the parent module.
With Maven, if you find yourself generating two artifacts from one module (which you are), you should first consider that you are probably violating a Maven best-practice and rearrange things so you only produce one artifact per module.
Please let me know if this doesn't make sense and I will try to clarify.

Integrate Protocol Buffers into Maven2 build

I'm experimenting with Protocol Buffers in an existing, fairly vanilla Maven 2 project. Currently, I invoke a shell script every time I need to update my generated sources. This is obviously a hassle, as I would like the sources to be generated automatically before each build. Hopefully without resorting to shameful hackery.
So, my question is two-fold:
Long shot: is there a "Protocol Buffers plugin" for Maven 2 that can achieve the above in an automagic way? There's a branch on Google Code whose author appears to have taken a shot at implementing such a plugin. Unfortunately, it hasn't passed code review or been merged into protobuf trunk. The status of that plugin is thus unknown.
Probably more realistic: lacking an actual plugin, how else might I go about invoking protoc from my Maven 2 build? I suppose I may be able to wire up my existing shell script into an antrun invocation or something similar.
Personal experiences are most appreciated.
You'll find some information about the plugin available in the Protocol Buffers repository in the Protocol Buffers Compiler Maven Plug-In thread on the Protocol Buffers discussion group. My understanding is that it's usable but lacking tests. I'd give it a try.
Or you could just use the antrun plugin (snipet pasted from the thread mentioned above):
<build>
<plugins>
<plugin>
<artifactId>maven-antrun-plugin</artifactId>
<executions>
<execution>
<id>generate-sources</id>
<phase>generate-sources</phase>
<configuration>
<tasks>
<mkdir dir="target/generated-sources"/>
<exec executable="protoc">
<arg value="--java_out=target/generated-sources"/>
<arg value="src/main/protobuf/test.proto"/>
</exec>
</tasks>
<sourceRoot>target/generated-sources</sourceRoot>
</configuration>
<goals>
<goal>run</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-java</artifactId>
<version>2.0.3</version>
</dependency>
</dependencies>
The accepted answer encouraged me to get the Google-provided plugin to work. I merged the branch mentioned in my question into a checkout of 2.2.0 source code, built and installed/deployed the plugin, and was able to use it in my project as follows:
<build>
<plugins>
<plugin>
<groupId>com.google.protobuf.tools</groupId>
<artifactId>maven-protoc-plugin</artifactId>
<version>0.0.1</version>
<executions>
<execution>
<id>generate-sources</id>
<goals>
<goal>compile</goal>
</goals>
<phase>generate-sources</phase>
<configuration>
<protoSourceRoot>${basedir}/src/main/protobuf/</protoSourceRoot>
<includes>
<param>**/*.proto</param>
</includes>
</configuration>
</execution>
</executions>
<configuration>
<protocExecutable>/usr/local/bin/protoc</protocExecutable>
</configuration>
</plugin>
</plugins>
</build>
Note that I changed the plugin's version to 0.0.1 (no -SNAPSHOT) in order to make it go into my non-snapshot thirdparty Nexus repository. YMMV. The takeaway is that this plugin will be usable once it's no longer necessary to jump through hoops in order to get it going.
The accepted solution does not scale for multiple proto files. I had to come up with my own:
<build>
<plugins>
<plugin>
<artifactId>maven-antrun-plugin</artifactId>
<executions>
<execution>
<id>compile-protoc</id>
<phase>generate-sources</phase>
<configuration>
<tasks>
<mkdir dir="${generated.sourceDirectory}" />
<path id="proto.path">
<fileset dir="src/main/proto">
<include name="**/*.proto" />
</fileset>
</path>
<pathconvert pathsep=" " property="proto.files" refid="proto.path" />
<exec executable="protoc" failonerror="true">
<arg value="--java_out=${generated.sourceDirectory}" />
<arg value="-I${project.basedir}/src/main/proto" />
<arg line="${proto.files}" />
</exec>
</tasks>
</configuration>
<goals>
<goal>run</goal>
</goals>
</execution>
</executions>
</plugin>
</build>
There's also great plugin by Igor Petruk named protobuf-maven-plugin. It's in central repo now and plays nicely with eclipse (m2e-1.1 is recommended).
I just updated the maven plugin to work with 2.2.0 -- the updated pom are attached to the code review bug.
Here are the instructions to build the plugin yourself:
svn co http://protobuf.googlecode.com/svn/branches/maven-plugin/tools/maven-plugin
cd maven-plugin
wget -O pom.xml 'http://protobuf.googlecode.com/issues/attachment?aid=8860476605163151855&name=pom.xml'
mvn install
You can then use the maven config above.
I just tried a less official but very recent (v 0.1.7) fork from https://github.com/dtrott/maven-protoc-plugin and it worked very well, courtesy of David Trott. I tested it with a couple of Maven modules one of which contained DTO-style messages and the other a service depending on them. I borrowed the plugin configuration MaxA posted on Oct 16 '09, I had protoc on my PATH and I added
<temporaryProtoFileDirectory>${basedir}/target/temp</temporaryProtoFileDirectory>
right after
<protocExecutable>protoc</protocExecutable>.
What is really nice is that all I had to do is to declare a normal dependency from the service module on the DTO module. The plugin was able to resolve proto files dependencies by finding the proto files packaged with the DTO module, extracting them to a temporary directory and using while generating code for the service. And it was smart enough not to package a second copy of the generated DTO classes with the service module.
There is a maven plugin for protobuf. https://www.xolstice.org/protobuf-maven-plugin/usage.html
The minimal config
<plugin>
<groupId>org.xolstice.maven.plugins</groupId>
<artifactId>protobuf-maven-plugin</artifactId>
<version>0.5.0</version>
<configuration>
<protocExecutable>/usr/local/bin/protoc</protocExecutable>
</configuration>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>test-compile</goal>
</goals>
</execution>
</executions>
</plugin>
I think that using antrun to invoke non-Maven steps is the generally accepted solution.
You could also try the maven-exec-plugin.
I forked of the plugin from David Trott and have it compiling multiple languages which makes it a lot more useful. See the github project here and a tutorial on integrating it with a maven build here.