I am using a hadoop cluster with cloudera-6.3.2 distribution.
I have a requirement to read hive ACID table from spark (Java client). So native spark does not read hive acid table. Hence planning to use Hive WareHouse Connector. But getting below exceptions.Even could not read non acid table also. Any Thoughts?
20/03/27 21:26:46 INFO HiveWarehouseSessionImpl: Created a new HWC session: 9f46ffe3-c863-4fbf-82ef-9f730f0c0cfc
20/03/27 21:26:47 INFO LlapBaseInputFormat: Handle ID 6439738b-a20a-46cc-9ee3-63214b573660: query=select * from test
20/03/27 21:26:47 ERROR HiveConnection: Error opening session
org.apache.thrift.TApplicationException: Required field 'client_protocol' is unset! Struct:TOpenSessionReq(client_protocol:null, configuration:{set:hiveconf:hive.server2.thrift.resultset.default.fetch.size=1000, use:database=default/})
at org.apache.thrift.TApplicationException.read(TApplicationException.java:111)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:79)
Java Code
HiveWarehouseSession hive = HiveWarehouseSession.session(spark).build();
hive.executeQuery("select * from test").show();
Here is my pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>fw_analytics</groupId>
<artifactId>com.analytics</artifactId>
<version>1.0-SNAPSHOT</version>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>8</source>
<target>8</target>
</configuration>
</plugin>
</plugins>
</build>
<packaging>jar</packaging>
<dependencies>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.11</artifactId>
<version>2.4.0-cdh6.3.2</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-hive_2.11</artifactId>
<version>2.4.0-cdh6.3.2</version>
</dependency>
<dependency>
<groupId>com.hortonworks.hive</groupId>
<artifactId>hive-warehouse-connector_2.11</artifactId>
<version>1.0.0.7.0.3.0-79</version>
</dependency>
</dependencies>
<repositories>
<repository>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>true</enabled>
</snapshots>
<id>hortonworks.extrepo</id>
<name>Hortonworks HDP</name>
<url>http://repo.hortonworks.com/content/repositories/releases</url>
</repository>
<repository>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>true</enabled>
</snapshots>
<id>Cloudera.extrepo</id>
<name>Cloudera HDP</name>
<url>https://repository.cloudera.com/artifactory/cloudera-repos/</url>
</repository>
<repository>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>true</enabled>
</snapshots>
<id>hortonworks.other</id>
<name>Hortonworks Other Dependencies</name>
<url>http://repo.hortonworks.com/content/groups/public</url>
</repository>
</repositories>
</project>
Related
I am trying to deploy a custom policy to Exchange, but it results in:
[INFO] Scanning for projects...
[INFO]
[INFO] -----< 7de36ab5-1c25-4cd4-89e8-ec315f34b341:message-header-policy >-----
[INFO] Building message-header-policy 1.0.3
[INFO] ----------------------------[ mule-policy ]-----------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.446 s
[INFO] Finished at: 2018-10-16T15:01:04+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Could not find goal 'deploy-file' in plugin org.mule.tools.maven:mule-maven-plugin:3.1.6 among available goals clean, compile, deploy, generate-resources, generate-sources, generate-test-resources, generate-test-sources, initialize, package, process-classes, process-resources, process-sources, site, test-compile, undeploy, validate -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoNotFoundException
My pom file is:
http://maven.apache.org/maven-v4_0_0.xsd">
4.0.0
<groupId>organizationid</groupId>
<artifactId>message-header-policy</artifactId>
<version>1.0.3</version>
<packaging>mule-policy</packaging>
<name>message-header-policy</name>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<app.runtime>4.1.3</app.runtime>
<exchange.url>https://maven.eu1.anypoint.mulesoft.com/api/v1/organizations/organizationid/maven</exchange.url>
<mule.maven.plugin.version>3.1.6</mule.maven.plugin.version>
</properties>
<build>
<plugins>
<plugin>
<groupId>org.mule.tools.maven</groupId>
<artifactId>mule-maven-plugin</artifactId>
<version>${mule.maven.plugin.version}</version>
<executions>
<execution>
<id>upload-template</id>
<phase>deploy</phase>
<goals>
<goal>deploy-file</goal>
</goals>
<configuration>
<repositoryId>Repository</repositoryId>
<url>${exchange.url}</url>
<file>${project.basedir}/${project.artifactId}.yaml</file>
<generatePom>false</generatePom>
<artifactId>${project.artifactId}</artifactId>
<groupId>${project.groupId}</groupId>
<version>${project.version}</version>
<packaging>yaml</packaging>
<classifier>mule-policy</classifier>
</configuration>
</execution>
</executions>
<extensions>true</extensions>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>org.mule.connectors</groupId>
<artifactId>mule-http-connector</artifactId>
<version>1.3.1</version>
<classifier>mule-plugin</classifier>
</dependency>
<dependency>
<groupId>org.mule.connectors</groupId>
<artifactId>mule-sockets-connector</artifactId>
<version>1.1.1</version>
<classifier>mule-plugin</classifier>
</dependency>
<dependency>
<groupId>com.mulesoft.anypoint</groupId>
<artifactId>mule-http-policy-transform-extension</artifactId>
<version>1.1.0</version>
<classifier>mule-plugin</classifier>
</dependency>
</dependencies>
<repositories>
<repository>
<id>anypoint-exchange</id>
<name>Anypoint Exchange</name>
<url>https://maven.eu1.anypoint.mulesoft.com/api/v1/maven</url>
<layout>default</layout>
</repository>
<repository>
<id>mulesoft-releases</id>
<name>MuleSoft Releases Repository</name>
<url>https://repository.mulesoft.org/releases/</url>
<layout>default</layout>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>mulesoft-releases</id>
<name>mulesoft release repository</name>
<layout>default</layout>
<url>https://repository.mulesoft.org/releases/</url>
<snapshots>
<enabled>false</enabled>
</snapshots>
</pluginRepository>
</pluginRepositories>
<distributionManagement>
<repository>
<id>Repository</id>
<name>Corporate Repository</name>
<url>${exchange.url}</url>
<layout>default</layout>
</repository>
</distributionManagement>
organizationid is the GUID from our organisation ....
I have based this on the documentation found on this page:
https://docs.mulesoft.com/api-manager/2.x/policy-scope-size-concept
You need to add your Exchange credentials in maven settings.xml
<servers>
<server>
<id>Repository</id>
<username>myusername</username>
<password>mypassword</password>
</server>
Hope it helps
I was not able to get an Arquillian testcase running together with WLP 8.5.5.4. Is there any simple example project in place which demonstrates this to me? I did not find one.
You can find complex setup with creation of testing server in this redbook Configuring and Deploying Open Source with WebSphere Application Server Liberty Profile.
Here you have very basic setup:
Configure arquillian.xml file for WebSphere Liberty:
<arquillian xmlns="http://jboss.org/schema/arquillian" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://jboss.org/schema/arquillian http://jboss.org/schema/arquillian/arquillian_1_0.xsd">
<engine>
<property name="deploymentExportPath">target/</property>
</engine>
<container qualifier="wlp-managed-85" default="true">
<configuration>
<property name="wlpHome">C:/IBM/WebSphere/wlp</property>
<property name="serverName">defaultServer</property>
<property name="httpPort">9080</property>
<property name="appDeployTimeout">20</property>
<property name="appUndeployTimeout">20</property>
</configuration>
</container>
Configure required dependencies in the pom.xml:
<!-- Configure WASdev repository -->
<repositories>
<repository>
<releases>
<updatePolicy>never</updatePolicy>
</releases>
<snapshots>
<enabled>false</enabled>
<updatePolicy>never</updatePolicy>
</snapshots>
<id>ibm-maven-repo</id>
<name>ibm-maven-repo</name>
<url>http://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/wasdev/maven/repository/</url>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>ibm-maven-repo</id>
<name>ibm-maven-repo</name>
<url>http://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/wasdev/maven/repository/</url>
<layout>default</layout>
<snapshots>
<enabled>true</enabled>
</snapshots>
<releases>
<enabled>true</enabled>
</releases>
</pluginRepository>
</pluginRepositories>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.jboss.arquillian</groupId>
<artifactId>arquillian-bom</artifactId>
<version>1.1.8.Final</version>
<scope>import</scope>
<type>pom</type>
</dependency>
</dependencies>
.....
<dependency>
<groupId>com.ibm.tools.target</groupId>
<artifactId>was-liberty</artifactId>
<version>LATEST</version>
<type>pom</type>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.11</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.jboss.arquillian.junit</groupId>
<artifactId>arquillian-junit-container</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<!-- Arquillian WebSphere Liberty Profile support -->
<groupId>org.jboss.arquillian.container</groupId>
<artifactId>arquillian-wlp-managed-8.5</artifactId>
<version>1.0.0.Alpha2</version>
<scope>test</scope>
</dependency>
I'm getting the following error after importing the project from Maven POM to MuleStudio.
The attribute comments is undefined for the annotation type Generated
Please help to resolve this issue.
My POM file:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.hello</groupId>
<artifactId>hello-connector</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>mule-module</packaging>
<name>Hello Connector</name>
<parent>
<groupId>org.mule.tools.devkit</groupId>
<artifactId>mule-devkit-parent</artifactId>
<version>3.4.3</version>
</parent>
<properties>
<junit.version>4.9</junit.version>
<mockito.version>1.8.2</mockito.version>
<jdk.version>1.6</jdk.version>
<category>Community</category>
<licensePath>LICENSE.md</licensePath>
<devkit.studio.package.skip>false</devkit.studio.package.skip>
</properties>
<scm>
<connection>scm:git:git://github.com:mulesoft/hello-connector.git</connection>
<developerConnection>scm:git:git#github.com:mulesoft/hello-connector.git</developerConnection>
<url>http://github.com/mulesoft/hello-connector</url>
</scm>
<repositories>
<repository>
<id>mulesoft-releases</id>
<name>MuleSoft Releases Repository</name>
<url>http://repository.mulesoft.org/releases/</url>
<layout>default</layout>
</repository>
<repository>
<id>mulesoft-snapshots</id>
<name>MuleSoft Snapshots Repository</name>
<url>http://repository.mulesoft.org/snapshots/</url>
<layout>default</layout>
</repository>
</repositories>
<dependencies>
<dependency>
<groupId>org.eclipse.equinox</groupId>
<artifactId>common</artifactId>
<version>3.3.0-v20070426</version>
<scope>provided</scope>
</dependency>
</dependencies>
</project>
I had the same problem. I used mule-devkit-partent version 3.4.2 to make it work.
I am trying to deploy OSGI bundle to Jboss 7.1.1.Final. Everytime I compile and deploy project it gives me following error.
I have checked in M2, only 1.0-beta-2 version is present as we can see similar thing mentioned in repository too http://mvnrepository.com/artifact/org.codehaus.mojo/jasperreports-maven-plugin/1.0-beta-2
There is no version defined in POM, not sure from where it is trying to pull this version.
Command
mvn jboss-as:deploy -Pdomain -DskipTests -Dusername=manish -Dpassword=password -Dhost=x.x.x.x -Dport=9999 -X
POM Configuration
<repository>
<id>jboss-public-repository</id>
<name>JBoss Repository</name>
<url>https://repository.jboss.org/nexus/content/groups/public</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
<repository>
<id>jasperreports</id>
<url>http://jasperreports.sourceforge.net/maven2</url>
</repository>
</repositories>
...
<plugins>
<!-- Jasper Plugins -->
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>jasperreports-maven-plugin</artifactId>
<configuration>
<outputDirectory>${project.build.directory}/jasper</outputDirectory>
</configuration>
<executions>
<execution>
<goals>
<goal>compile-reports</goal>
</goals>
</execution>
</executions>
<dependencies>
<dependency>
<groupId>net.sf.jasperreports</groupId>
<artifactId>jasperreports</artifactId>
<version>4.7.0</version>
</dependency>
<dependency>
<groupId>org.codehaus.groovy</groupId>
<artifactId>groovy-all</artifactId>
<version>1.7.5</version>
</dependency>
</dependencies>
</plugin>
Error
Caused by: org.apache.maven.project.ProjectBuildingException: Error getting POM for 'org.codehaus.mojo:jasperreports-maven-plugin' from the repository: Failed to resolve artifact, possibly due to a repository list that is not appropriately equipped for this artifact's metadata.
org.codehaus.mojo:jasperreports-maven-plugin:pom:1.0-beta-3-SNAPSHOT
from the specified remote repositories:
central (http://repo1.maven.org/maven2),
jboss-public-repository (https://repository.jboss.org/nexus/content/groups/public)
for project org.codehaus.mojo:jasperreports-maven-plugin
at org.apache.maven.project.DefaultMavenProjectBuilder.findModelFromRepository(DefaultMavenProjectBuilder.java:592)
at org.apache.maven.project.DefaultMavenProjectBuilder.buildFromRepository(DefaultMavenProjectBuilder.java:251)
at org.apache.maven.project.artifact.MavenMetadataSource.retrieveRelocatedProject(MavenMetadataSource.java:163)
... 22 more
The jasperreport plugin is available via Maven central but not as a SNAPSHOT version. The question is if you need to use the SNAPSHOT version or the release version which can be found here.
Below are my maven settings from behind the proxy
User Settings:
<?xml version="1.0" encoding="UTF-8"?>
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd">
<localRepository>D:\path\mvn_repo</localRepository>
<pluginGroups></pluginGroups>
<proxies>
<proxy>
<active>true</active>
<protocol>https</protocol>
<username>myusername</username>
<password>password</password>
<host>myproxyhost</host>
<port>myproxyport</port>
</proxy>
<proxy>
<active>true</active>
<protocol>http</protocol>
<username>myusername</username>
<password>password</password>
<host>myproxyhost</host>
<port>myproxyport</port>
</proxy>
</proxies>
<servers></servers>
<mirrors></mirrors>
<profiles></profiles>
</settings>
Global Settings:
<?xml version="1.0" encoding="UTF-8"?>
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd">
<localRepository>D:\path\mvn_repo</localRepository>
<pluginGroups></pluginGroups>
<proxies></proxies>
<servers></servers>
<mirrors></mirrors>
<profiles>
<profile>
<id>jboss-public-repository</id>
<repositories>
<repository>
<id>jboss-public-repository-group</id>
<name>JBoss Public Maven Repository Group</name>
<url>https://repository.jboss.org/nexus/content/groups/public-jboss/</url>
<layout>default</layout>
<releases>
<enabled>true</enabled>
<updatePolicy>never</updatePolicy>
</releases>
<snapshots>
<enabled>true</enabled>
<updatePolicy>never</updatePolicy>
</snapshots>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>jboss-public-repository-group</id>
<name>JBoss Public Maven Repository Group</name>
<url>https://repository.jboss.org/nexus/content/groups/public-jboss/</url>
<layout>default</layout>
<releases>
<enabled>true</enabled>
<updatePolicy>never</updatePolicy>
</releases>
<snapshots>
<enabled>true</enabled>
<updatePolicy>never</updatePolicy>
</snapshots>
</pluginRepository>
</pluginRepositories>
</profile>
</profiles>
<activeProfiles>
</settings>
However I get 'Connection timed out' when I try to execute test goal.
This happens with both maven command line and M2Eclipse. What more settings do I require.
Maybe the proxy you are using requires NTLM authentication, which maven does not support.
You can try to use CNTLM:
http://cntlm.sourceforge.net/
JBoss repository has been moved in June 2011 causing many other users to experience problems like you describe. Most likely you need to change the URL in your pom.
For more details, refer to
How to configure jboss repository properly for my repository manager
and other answers listed as "Related" to above
JBoss wiki for maven users: Maven Getting Started - Users
PS. When doing the change, make sure to check all your poms that contain repository blocks because otherwise, broken JBoss URL may slip through these and kill your build