I'm building a simple reactive web application ( Following Josh long's tech talk ) Simply put I have reactive web, r2dbc and h2 as dependencies.
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-webflux</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-r2dbc</artifactId>
</dependency>
<dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.projectreactor</groupId>
<artifactId>reactor-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
So I expect spring would configure everything for me( It does for Josh ). But I get error saying not being able to connect to a database and there is a suggestion asking to include h2(which I already have). What am I doing wrong here?
Description:
Failed to configure a ConnectionFactory: 'url' attribute is not specified and no embedded database could be configured.
Reason: Failed to determine a suitable R2DBC Connection URL
Action:
Consider the following:
If you want an embedded database (H2), please put it on the classpath.
If you have database settings to be loaded from a particular profile you may need to activate it (no profiles are currently active).
Ok it was missing r2dbc-h2 dependency. This happened because I didn't add r2dbc when I created the project with start.spring.io then added it and inspect the pom but only copied spring-boot-starter-data-r2dbc.
<dependency>
<groupId>io.r2dbc</groupId>
<artifactId>r2dbc-h2</artifactId>
<scope>runtime</scope>
</dependency>
Still bit confusing though. Spring boot says it looks in to the class path and auto configure dependencies but seems like sometimes it need given combination of dependencies.
We are using solace as the Messaging system in our application and while writing the unit test classes (using JUNIT )for listners i have to start the solcae in my local.
Instead i was trying to mock the broker (apache ActiveMq) to use amqp protocl and send messages to the listeners.
https://github.com/apache/activemq/blob/activemq-5.15.x/activemq-amqp/src/test/java/org/apache/activemq/transport/amqp/AmqpTransformerTest.java
But when i try to build the maven project i see the error
package org.apache.activemq.transport.amqp.client does not exist.
I have added the below dependencies but i still facing the same issue. Please suggest
<dependency>
<groupId>org.apache.activemq</groupId>
<artifactId>activemq-broker</artifactId>
<version>5.15.12</version>
<!-- <scope>test</scope> -->
</dependency>
<!-- Testing Dependencies -->
<dependency>
<groupId>org.apache.qpid</groupId>
<artifactId>qpid-jms-client</artifactId>
<version>0.51.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.activemq</groupId>
<artifactId>activemq-kahadb-store</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.activemq</groupId>
<artifactId>activemq-jaas</artifactId>
<version>5.15.12</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.activemq</groupId>
<artifactId>activemq-broker</artifactId>
<version>5.15.12</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.activemq</groupId>
<artifactId>activemq-spring</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.activemq</groupId>
<artifactId>activemq-http</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.activemq</groupId>
<artifactId>activemq-mqtt</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.activemq</groupId>
<artifactId>activemq-leveldb-store</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.activemq.tooling</groupId>
<artifactId>activemq-junit</artifactId>
<version>5.15.12</version>
<scope>test</scope>
</dependency>
I am not able to resolve the below compilation issues.
org.apache.activemq.transport.amqp.client can not be resolved since the dependecy for this package is not found,But i have added the above dependencies in the maven project.
import org.apache.activemq.transport.amqp.client.AmqpClient;
import org.apache.activemq.transport.amqp.client.AmqpConnection;
import org.apache.activemq.transport.amqp.client.AmqpMessage;
import org.apache.activemq.transport.amqp.client.AmqpSender;
import org.apache.activemq.transport.amqp.client.AmqpSession;
Please suggest.
thank you experts.
Not entirely clear what your test is doing but the classes it can't find are those of the AMQP test client that is implemented in the ActiveMQ 5.x AMQP module's test jar so you definitely won't find them with the dependencies you have there.
The AMQP test client in the ActiveMQ broker is not meant for general use by anyone as is was built specifically to test the AMQP stack in the broker. If you remove the usage of that from your tests you should have better luck.
I have created simple seedstack web project through guideline mentioned on http://seedstack.org/docs/basics/
Undertow is also started with seedstack:run.
However, while accessing "hello" resource undertow throws below exception:
ERROR 2018-07-25 21:37:34,468 XNIO-1 task-2 io.undertow.request
UT005023: Exception handling request to
/api/seed-w20/application/configuration
null returned by binding at
org.seedstack.w20.internal.W20Module.configure(W20Module.java:51) (via
modules: com.google.inject.util.Modules$OverrideModule ->
io.nuun.kernel.core.internal.injection.KernelGuiceModuleInternal ->
org.seedstack.w20.internal.W20Module) but the 3rd parameter of
org.seedstack.w20.internal.FragmentManagerImpl.(FragmentManagerImpl.java:32)
is not #Nullable at
org.seedstack.w20.internal.W20Module.configure(W20Module.java:51) (via
modules: com.google.inject.util.Modules$OverrideModule ->
io.nuun.kernel.core.internal.injection.KernelGuiceModuleInternal ->
org.seedstack.w20.internal.W20Module) while locating
org.seedstack.w20.internal.ConfiguredApplication
for the 3rd parameter of org.seedstack.w20.internal.FragmentManagerImpl.(FragmentManagerImpl.java:32)
while locating org.seedstack.w20.internal.FragmentManagerImpl while
locating org.seedstack.w20.FragmentManager
for field at org.seedstack.w20.internal.rest.application.ApplicationConfigurationResource.fragmentManager(ApplicationConfigurationResource.java:38)
while locating
org.seedstack.w20.internal.rest.application.ApplicationConfigurationResource
Any help please?
This is a bug introduced recently into the w20-bridge, which occurs when no w20.app.json configuration file is present.
You can workaround it by creating an empty-object w20.app.json file at the root of the classpath:
{}
You can also update the version of all w20-bridge dependencies to 3.2.4 which has a fix for it. This can be done by using the dependencyManagement section of your POM:
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.seedstack</groupId>
<artifactId>seedstack-bom</artifactId>
<version>18.4.3</version>
<type>pom</type>
<scope>import</scope>
</dependency>
<dependency>
<groupId>org.seedstack.addons.w20</groupId>
<artifactId>w20-bridge-web</artifactId>
<version>3.2.4</version>
</dependency>
<dependency>
<groupId>org.seedstack.addons.w20</groupId>
<artifactId>w20-bridge-web-bootstrap-3</artifactId>
<version>3.2.4</version>
</dependency>
<dependency>
<groupId>org.seedstack.addons.w20</groupId>
<artifactId>w20-bridge-web-business-theme</artifactId>
<version>3.2.4</version>
</dependency>
<dependency>
<groupId>org.seedstack.addons.w20</groupId>
<artifactId>w20-bridge-web-components</artifactId>
<version>3.2.4</version>
</dependency>
<dependency>
<groupId>org.seedstack.addons.w20</groupId>
<artifactId>w20-bridge-rest</artifactId>
<version>3.2.4</version>
</dependency>
<dependency>
<groupId>org.seedstack.addons.w20</groupId>
<artifactId>w20-bridge-specs</artifactId>
<version>3.2.4</version>
</dependency>
</dependencies>
</dependencyManagement>
This fix will be included in the upcoming SeedStack 18.7.
I am writing a Spark structured streaming application in which data processed with Spark needs be sink'ed to elastic search.
This is my development environment, hence I have a standalone Elastic search.
I have tried following two ways to sink the data in the DataSet to ES.
1.ds.writeStream().format("org.elasticsearch.spark.sql").start("spark/orders");
2.ds.writeStream().format("es").start("spark/orders");
In both cases I am getting the following error:
Caused by:
java.lang.UnsupportedOperationException: Data source es does not support streamed writing
at org.apache.spark.sql.execution.datasources.DataSource.createSink(DataSource.scala:287) ~[spark-sql_2.11-2.1.1.jar:2.1.1]
at org.apache.spark.sql.streaming.DataStreamWriter.start(DataStreamWriter.scala:272) ~[spark-sql_2.11-2.1.1.jar:2.1.1]
at org.apache.spark.sql.streaming.DataStreamWriter.start(DataStreamWriter.scala:213) ~[spark-sql_2.11-2.1.1.jar:2.1.1]
pom.xml:
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.11</artifactId>
<version>2.1.1</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql_2.11</artifactId>
<version>2.1.1</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.mongodb.spark</groupId>
<artifactId>mongo-spark-connector_2.11</artifactId>
<version>2.0.0</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-streaming_2.11</artifactId>
<version>2.1.1</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-hive_2.11</artifactId>
<version>2.1.1</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-streaming-kafka_2.11</artifactId>
<version>1.6.2</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql-kafka-0-10_2.11</artifactId>
<version>2.1.1</version>
</dependency>
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch-spark-20_2.11</artifactId>
<version>5.6.1</version>
</dependency>
Appreciate any help in resolving this issue.
you can try
ds.write.format("org.elasticsearch.spark.sql").option("es.resource",ES_INDEX+"/"+ES_TYPE).option("es.mapping.id",ES_ID).mode("overwrite").save()
Elasticsearch sink does not support streamed writing which means you can't stream output to Elasticsearch.
You could write streaming output to kafka and using logstash to read from kafka to elasticsearch.
Update:
Streamed Writing is now supported in version Elasticsearch 6.x when using Spark 2.2.0.
Dependency:
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch-spark-20_2.11</artifactId>
<version>6.2.4</version>
</dependency>
writeStream code:
ds
.writeStream
.outputMode(OutputMode.Append()) // only append mode is currently supported
.format("es")
.option("checkpointLocation", "/my/checkpointLocation")
.option("es.mapping.id", "MY_OPTIONAL_ID_ATTRIBUTE")
.trigger(Trigger.ProcessingTime(5, TimeUnit.SECONDS))
.start("index/type")
I'm trying to create an intelliJ plugin that needs to execute maven targets on the current project. All the talk in the intertubes recommends using the MavenEmbedder. Good luck with that. The 2.0.4 version isn't well supported and there are no references for how to use it.
I gave it a whirl and ran into a wall where the embedder had not been initialized with all the fields it needs. Reflective private member injection? Awesome! Why would anyone need an obvious way to initialize an object?
It seems a few people are using a 2.1 version with some success. I have been unable to find that in a jar or even sources.
I went and checked out the 3.0 version of the embedder project: http://maven.apache.org/ref/3.0-beta-3/maven-embedder/ It does away with the MavenEmbedder object all together and seems to only support access through the main or doMain methods on MavenCli. Has anyone used these methods and can give some advice?
Yeah, the's not much in the way of documentation of MavenCli. The API is significatly simpler but i'd still like some examples. Here's one that works...
MavenCli cli = new MavenCli();
int result = cli.doMain(new String[]{"compile"},
"/home/aioffe/workspace/MiscMaven",
System.out, System.out);
System.out.println("result: " + result);
It takes a dir and runs the 'compile' phase...
Working maven configuration for maven 3.6.3
Code
MavenCli cli = new MavenCli();
System.setProperty("maven.multiModuleProjectDirectory", workingDirectory);
cli.doMain(new String[]{"compile"}, workingDirectory, System.out, System.err);
Dependencies
<dependencies>
<dependency>
<groupId>org.apache.maven</groupId>
<artifactId>maven-embedder</artifactId>
<version>3.6.3</version>
</dependency>
<!-- https://issues.apache.org/jira/browse/MNG-5995 -->
<dependency>
<groupId>org.apache.maven</groupId>
<artifactId>maven-compat</artifactId>
<version>3.6.3</version>
</dependency>
<!-- enable logging -->
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-simple</artifactId>
<version>1.7.30</version>
</dependency>
</dependencies>
The dependency matrix information for provided scopes and dynamically acquired components can be a bit confusing. It was for me, since it appeared to me that I got all the required items by direct or transitive dependency, but then remote resolution didn't work.
I wanted to jump to Maven 3.3.3 (latest as of 2015-05-25). I got it working without the sisu errors that presented when I tried to optimistically update to current versions of things specified here (and elsewhere). This is a project with a tag that worked with the example specified as of today using JDK8.
https://github.com/mykelalvis/test-maven-embedder/tree/20150525-working
Relevant deps (SLF4J is just so I can see the logs)
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-simple</artifactId>
<version>1.7.5</version>
</dependency>
<dependency>
<groupId>org.apache.maven</groupId>
<artifactId>maven-embedder</artifactId>
<version>3.3.3</version>
</dependency>
<dependency>
<groupId>org.eclipse.aether</groupId>
<artifactId>aether-connector-basic</artifactId>
<version>1.0.2.v20150114</version>
</dependency>
<dependency>
<groupId>org.eclipse.aether</groupId>
<artifactId>aether-transport-wagon</artifactId>
<version>1.0.2.v20150114</version>
</dependency>
<dependency>
<groupId>org.apache.maven.wagon</groupId>
<artifactId>wagon-http</artifactId>
<version>2.9</version>
</dependency>
<dependency>
<groupId>org.apache.maven.wagon</groupId>
<artifactId>wagon-provider-api</artifactId>
<version>2.9</version>
</dependency>
<dependency>
<groupId>org.apache.maven.wagon</groupId>
<artifactId>wagon-http-lightweight</artifactId>
<version>2.9</version>
</dependency>
Running this is:
rm -r ~/.m2/repository/org/apache/maven/plugins/maven-clean-plugin/
mvn exec:java
Probably should have made it a unit test of some sort.
If someone has a superior solution for embedded Maven 3.3.3 (i.e. came up with a smaller or more range-oriented set of required dependencies), please post them.
to build on the comment from #StevePerkins, and using maven version 3.1.0,
I had to exclude the transitive dependency from aether-connector-wagon to wagon-provider-api to get it working.
pom.xml:
(...)
<dependency>
<groupId>org.apache.maven</groupId>
<artifactId>maven-embedder</artifactId>
<version>3.1.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.eclipse.aether</groupId>
<artifactId>aether-connector-wagon</artifactId>
<version>0.9.0.M2</version>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>org.apache.maven.wagon</groupId>
<artifactId>wagon-provider-api</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.maven.wagon</groupId>
<artifactId>wagon-http</artifactId>
<version>2.5</version>
<scope>test</scope>
</dependency>
(...)
and here is a java example:
(...)
MavenCli cli = new MavenCli();
ByteArrayOutputStream baosOut = new ByteArrayOutputStream();
ByteArrayOutputStream baosErr = new ByteArrayOutputStream();
PrintStream out = new PrintStream(baosOut, true);
PrintStream err = new PrintStream(baosErr, true);
cli.doMain( new String[] { "clean" }, new File("."), out, err );
String stdout = baosOut.toString("UTF-8");
String stderr = baosErr.toString("UTF-8");
(...)
full example here
There is a dependency matrix for each version of maven-embedder, e.g. for 3.2.5: http://maven.apache.org/ref/3.2.5/maven-embedder/dependencies.html
Based on that I had to use org.apache.maven:maven-embedder:jar:3.2.5, org.apache.maven:maven-aether-provider:jar:3.2.5, and org.apache.maven.wagon:wagon-provider-api:jar:2.8.
It also fixes dependency on very old Guava library, since this version uses 18.0.
Dependency list for Maven Embedded 3.6.3 version that works in my Spring Boot 2.3 project (JDK8 or JDK 11 runtime):
<!-- Maven Embedder -->
<dependency>
<groupId>org.apache.maven</groupId>
<artifactId>maven-embedder</artifactId>
<version>3.6.3</version>
</dependency>
<dependency>
<groupId>org.apache.maven</groupId>
<artifactId>maven-compat</artifactId>
<version>3.6.3</version>
</dependency>
<dependency>
<groupId>org.apache.maven.wagon</groupId>
<artifactId>wagon-http</artifactId>
<version>3.3.4</version>
</dependency>
<dependency>
<groupId>org.eclipse.aether</groupId>
<artifactId>aether-connector-basic</artifactId>
<version>1.1.0</version>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.eclipse.aether</groupId>
<artifactId>aether-transport-wagon</artifactId>
<version>1.1.0</version>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-simple</artifactId>
<version>1.7.30</version>
</dependency>
<dependency>
<groupId>org.usefultoys</groupId>
<artifactId>slf4j-toys</artifactId>
<version>1.6.3</version>
</dependency>
The Maven CLI command looks like to:
// Maven CLI to execute Maven Commands
MavenCli cli = new MavenCli();
int result = cli.doMain(args, workingDirectory,
org.usefultoys.slf4j.LoggerFactory.getInfoPrintStream(LOGGER),
org.usefultoys.slf4j.LoggerFactory.getErrorPrintStream(LOGGER));
HTH