How can I change the user-agent in Maven? I need to be able to change this to get through the company firewall. I am using version 2.2.1 and I noticed an improvement in the 2.0.10 release notes:
[MNG-3652] - set a user agent for Maven HTTP requests.
Brett Porter posted a blog on Configuring Maven HTTP Connections that describes how you can do this and some other funky things:
<server>
<id>archiva.localhost</id>
<configuration>
<httpHeaders>
<property>
<name>User-Agent</name>
<value>Internal-Build-System/1.0</value>
</property>
</httpHeaders>
</configuration>
</server>
for command line version
try
"-Daether.connector.userAgent=your custom user agent"
Related
I have a yarn single node cluster setup in an Ubuntu VM.
When I am doing a spark-submit from the VM everything is working fine but when I am launching the same command from another VM with SSH the job is not working because it use a bad IP address for resource Manager.
The command I use in the yarn VM :
/home/namenode/spark/bin/spark-submit --master yarn --class Main --deploy-mode cluster /home/namenode/jars/data-transformation-service_2.11-0.1.0-SNAPSHOT.jar
The result :
Connecting to ResourceManager at /192.168.1.110:8032
And then my job is finishing well.
The command I use from another VM with SSH :
ssh namenode#192.168.1.110 '/home/namenode/spark/bin/spark-submit --master yarn --class Main --deploy-mode cluster /home/namenode/jars/data-transformation-service_2.11-0.1.0-SNAPSHOT.jar'
The result :
22/10/26 15:16:12 INFO DefaultNoHARMFailoverProxyProvider: Connecting to ResourceManager at /0.0.0.0:8032
22/10/26 15:16:13 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
And it loops again and again...
Do you have any idea how to fix this ? Thank you.
I finally managed to solved my issue by using this yarn-site :
<?xml version="1.0"?>
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
-->
<configuration>
<!-- Site specific YARN configuration properties -->
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.env-whitelist</name>
<value>JAVA_HOME,HADOOP_COMMON_HOME,HADOOP_HDFS_HOME,HADOOP_CONF_DIR,CLASSPATH_PREP END_DISTCACHE,HADOOP_YARN_HOME,HADOOP_MAPRED_HOME</value>
</property>
<property>
<name>yarn.resourcemanager.address</name>
<value>192.168.1.110:8032</value>
</property>
<property>
<name>yarn.resourcemanager.hostname</name>
<value>192.168.1.110</value>
</property>
</configuration>
Using Arquillian 1.1.4.Final and Tomee 1.6.0.2
Took the tomee-plus-remote profile setup from the Tomee information about arqullian adapters and put it into the Maven pom.xml (with activeByDefault true).
Goal is to deploy a MQ JCA rar into the remote Tomee and configure a connection factory to MQ.
Set the arqullian.xml initially to:
<container qualifier="tomee" default="true">
<configuration>
<property name="httpPort">-1</property>
<property name="stopPort">-1</property>
</configuration>
</container>
Running via JUnit not sure why the webprofile is initialized and started rather than plus (when I have tomee plus specified in Maven):
Info: Succeeded in installing singleton service
jun 11, 2014 11:07:52 FM org.apache.openejb.config.ConfigurationFactory init
Info: openejb configuration file is 'C:\Users\MYG\AppData\Local\Temp\arquillian-apache-tomee\apache-tomee-webprofile-1.6.0.2\conf\tomee.xml'
Another thing is how to load a tomee.xml configuration. Thought, the "serverXml" in the arquillian.xml (set to src/test/resources/tomee.xml) would work but then everything inside that xml is not recognized as a valid rule. Can't add directives like Deployments as one does with resources. So how to configure the remote tomee from arquillian?
Yeah, tomee.xml was not really designed for arquillian.xml since all its config can be passed to properties attribute of tomee container using properties format
By adding a conf property to the arquillian.xml to for example src/test/conf where there is a tomee.xml file then it is loaded. This must be Tomee thing that I didn't know about until now.
I'm looking for a solution similar to this:
httpOnly Session Cookie + Servlet 3.0 (e.g. Glassfish v3)
I want to turn the HTTP headers: http-only and Secure off so that authentication with https carries across to http requests.
However,
1 Relating to the other post. I am not using servlet 3.0. I may be willing to try to use servlet 3.0 if there was decent documentation somewhere on how to migrate an intellij project from serverl 2.5 to 3.0. It does not seem obvious how to use this. Changing my maven pom to javax.servlet 3.0 and then editing my web.xml version to 3.0 doesn't work. Gives a facet-error on the 3.0 version. Sigh.
This is my primary approach to solve this. I edited domains/domain1/config for default-web.xml and domain.xml to what I thought would work but it doesn't. Oracle's documentation does not seem very clear to me on this.
domain.xml
<web-container>
<session-config>
<session-manager>
<manager-properties></manager-properties>
<store-properties></store-properties>
</session-manager>
<session-properties>
<property name="cookieSecure" value="false"></property>
<property name="cookieHttpOnly" value="false"></property>
</session-properties>
</session-config>
</web-container>
....
<configs>
<config name="server-config">
<http-service sso-enabled="true">
<access-log></access-log>
<virtual-server id="server" sso-cookie-secure="false" sso-cookie-http-only="false" network-listeners="http-listener-1,http-listener-2"></virtual-server>
<virtual-server id="__asadmin" sso-cookie-http-only="false" network-listeners="admin-listener"></virtual-server>
</http-service>
default-web.xml
<session-config>
<session-timeout>60</session-timeout>
<cookie-config>
<http-only>false</http-only>
<secure>false</secure>
</cookie-config>
<cookie-properties>
<property name="cookieSecure" value="false" />
<property name="cookieHttpOnly" value="false" />
</cookie-properties>
<tracking-mode>COOKIE</tracking-mode>
</session-config>
Okay I feel a bit stupid but I generally have never written to a specific container before to make sure that my code was portable across containers. However, after some research, I discovered that you can put a WEB-INF/glassfish-web.xml file that looks like this to control your security cookie setting for each WAR file under glassfish.
http://blogs.oracle.com/jluehe/entry/ow_to_configure_the_security
<?xml version="1.0" encoding="UTF-8"?>
<glassfish-web-app>
<session-config>
<cookie-properties>
<b><property name="cookieSecure" value="[true|false|dynamic]"/></b>
</cookie-properties>
</session-config>
</glassfish-web-app>
I am trying to connect to an HBase node from a Java application. HBaseConfiguration is key, but the available Javadoc and documentation is really poor and insufficient.
Does anyone have proper examples of hbase-site.xml hbase-default.xml to use for remote connection?
Thanks!
There are only two variables you need to set from a clients point of view:
hbase.rootdir
hbase.zookeeper.quorum
Here are the steps from my setup doc about the hbase-site.xml. We don't make any changes to the hbase-default.xml as ... well... that's all the default settings. :)
edit hbase-site.xml. Copy the following to the file.
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl"
href="configuration.xsl"?>
<configuration>
<property>
<name>hbase.rootdir</name> <value>hdfs://PDHadoop1.corp.COMPANY.com:54310/usr/hbase</value>
<final>true</final> </property>
<property>
<name>hbase.zookeeper.quorum</name>
<value>PDHadoop1.corp.COMPANY.com,PDHadoop2.corp.COMPANY.com,PDHadoop3.corp.COMPANY.com,PDHadoop4.corp.COMPANY.com</value>
<final>true</final> </property>
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
<final>true</final> </property>
</configuration>
Save the file and quit the editor.
Please note that hbase.rootdir is pointing to PDHadoop1 as that is the name node in development environment. Similarly, hbase.zookeeper.quorum is pointing to all zookeeper servers in development environment. Please substitute these values with appropriate server names in your environment.
edit regionservers. Copy the following to the file.
PDHadoop3.corp.COMPANY.com
PDHadoop2.corp.COMPANY.com
PDHadoop1.corp.COMPANY.com
I apologize for the XML's lack of formatting.
These are the settings we use in production, I opened the file on my dev cluster to verify.
I hope that helps.
One major gotcha that I've encountered is that if your /etc/hosts contains an entry for that hostname pointing to the loopback address (127.0.0.1, 127.0.1.1, et cetra), then Hbase master will incorrectly register itself in Zookeeper with that loopback address -- which will not work when your client is not on the same machine as your master.
I wasted quite a bit of time to (first) get Hbase working. The solution is to remove the entry in /etc/hosts; but this requires that you override the "out of the box" behavior of the OS, at least on the ubuntu box that I've tested this on...
Can Maven Wagon plugin be configured to use a private key for ssh/scp? Everything I've tried still leaves maven to ask me for a password when it gets to the point of scp-ing.
You should be able to specify the path to the private key in the server element in your settings.xml:
The repositories for download and
deployment are defined by the
repositories and
distributionManagement elements of
the POM. However, certain settings
such as username and password should
not be distributed along with the
pom.xml. This type of information
should exist on the build server in
the settings.xml.
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0
http://maven.apache.org/xsd/settings-1.0.0.xsd">
...
<servers>
<server>
<id>server001</id>
<username>my_login</username>
<password>my_password</password>
<privateKey>${user.home}/.ssh/id_dsa</privateKey>
<passphrase>some_passphrase</passphrase>
<filePermissions>664</filePermissions>
<directoryPermissions>775</directoryPermissions>
<configuration></configuration>
</server>
</servers>
...
</settings>
id: This is the ID of the
server (not of the user to login as)
that matches the id element of the
repository/mirror that Maven tries to
connect to.
username, password: These elements appear as a pair denoting the login and password
required to authenticate to this
server.
privateKey,
passphrase: Like the previous two elements, this pair specifies a path
to a private key (default is
${user.home}/.ssh/id_dsa) and a
passphrase, if required. The
passphrase and password elements may
be externalized in the future, but for
now they must be set plain-text in the
settings.xml file.
filePermissions, directoryPermissions: When a repository file or directory is
created on deployment, these are the
permissions to use. The legal values
of each is a three digit number
corresponding to *nix file
permissions, ie. 664, or 775.
Note: If you use a private key to
login to the server, make sure you
omit the <password> element.
Otherwise, the key will be ignored.
Password Encryption
A new feature - server password and
passphrase encryption has been added
to 2.1.x and 3.0 trunks. See details
on this page.
Pay a special attention to the "note": If you use a private key to login to the server, make sure you omit the <password> element. Otherwise, the key will be ignored. So the final configuration will be close to:
<settings>
...
<servers>
<server>
<id>ssh-repository</id>
<username>your username in the remote system</username>
<privateKey>/path/to/your/private/key</privateKey>
<passphrase>sUp3rStr0ngP4s5wOrD</passphrase><!-- if required -->
<configuration>
...
</configuration>
</server>
</servers>
...
</settings>
I know this is an old thread, but it looks like the Wagon plugin is reading settings.xml (e.g. username) but not using all of the settings. I could not get it to stop asking for Kerberos username/password during scp. (Looks like there might have been changes to plugin late 2016 that affect this.)
Just adding this answer in case it helps someone else.
For me, the solution was even simpler: totally skip using 'settings.xml'
and simply specify 'scpexe' instead of 'scp' for protocol (like under distributionManagement section of pom.xml). This then uses your machine's default SSH configuration (unix settings under ~/.ssh).
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>wagon-maven-plugin</artifactId>
<version>1.0</version>
<executions>
<execution>
<id>upload-to-server</id>
<phase>deploy</phase>
<goals><goal>upload-single</goal></goals>
<configuration>
<fromFile>file-to-upload</fromfile>
<url>scpexe://username#serverName/dirname-to-copy-to
<toFile>file-to-upload</toFile>
</configuration>
</execution>
</executions>
</plugin>
I wanted to do the exact same thing today in conjunction with the maven-site-plugin (3.9.1) and was also hitting some roadblocks (specifically, the wagon-ssh plugin insisted on asking me for my Kerberos username and password).
What finally worked for me with wagon-ssh-3.4.3:
<!-- add scp support for mvn site:deploy -->
<dependency>
<groupId>org.apache.maven.wagon</groupId>
<artifactId>wagon-ssh</artifactId>
<version>3.4.3</version>
</dependency>
and in settings.xml:
<server>
<id>ssh-repository</id>
<username>pridkdev</username>
<privateKey>${user.home}/.ssh/pridkdev.ppk</privateKey>
<filePermissions>664</filePermissions>
<directoryPermissions>775</directoryPermissions>
<configuration>
<interactive>false</interactive>
<strictHostKeyChecking>no</strictHostKeyChecking>
<preferredAuthentications>publickey</preferredAuthentications>
</configuration>
</server>
I guess what was crucial is the <configuration> block and there especially the <preferredAuthentications> setting.
I found the necessary info here:
http://maven.apache.org/plugins/maven-deploy-plugin/examples/deploy-ssh-external.html