Can Maven Wagon plugin use a private key for scp? - maven-2

Can Maven Wagon plugin be configured to use a private key for ssh/scp? Everything I've tried still leaves maven to ask me for a password when it gets to the point of scp-ing.

You should be able to specify the path to the private key in the server element in your settings.xml:
The repositories for download and
deployment are defined by the
repositories and
distributionManagement elements of
the POM. However, certain settings
such as username and password should
not be distributed along with the
pom.xml. This type of information
should exist on the build server in
the settings.xml.
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0
http://maven.apache.org/xsd/settings-1.0.0.xsd">
...
<servers>
<server>
<id>server001</id>
<username>my_login</username>
<password>my_password</password>
<privateKey>${user.home}/.ssh/id_dsa</privateKey>
<passphrase>some_passphrase</passphrase>
<filePermissions>664</filePermissions>
<directoryPermissions>775</directoryPermissions>
<configuration></configuration>
</server>
</servers>
...
</settings>
id: This is the ID of the
server (not of the user to login as)
that matches the id element of the
repository/mirror that Maven tries to
connect to.
username, password: These elements appear as a pair denoting the login and password
required to authenticate to this
server.
privateKey,
passphrase: Like the previous two elements, this pair specifies a path
to a private key (default is
${user.home}/.ssh/id_dsa) and a
passphrase, if required. The
passphrase and password elements may
be externalized in the future, but for
now they must be set plain-text in the
settings.xml file.
filePermissions, directoryPermissions: When a repository file or directory is
created on deployment, these are the
permissions to use. The legal values
of each is a three digit number
corresponding to *nix file
permissions, ie. 664, or 775.
Note: If you use a private key to
login to the server, make sure you
omit the <password> element.
Otherwise, the key will be ignored.
Password Encryption
A new feature - server password and
passphrase encryption has been added
to 2.1.x and 3.0 trunks. See details
on this page.
Pay a special attention to the "note": If you use a private key to login to the server, make sure you omit the <password> element. Otherwise, the key will be ignored. So the final configuration will be close to:
<settings>
...
<servers>
<server>
<id>ssh-repository</id>
<username>your username in the remote system</username>
<privateKey>/path/to/your/private/key</privateKey>
<passphrase>sUp3rStr0ngP4s5wOrD</passphrase><!-- if required -->
<configuration>
...
</configuration>
</server>
</servers>
...
</settings>

I know this is an old thread, but it looks like the Wagon plugin is reading settings.xml (e.g. username) but not using all of the settings. I could not get it to stop asking for Kerberos username/password during scp. (Looks like there might have been changes to plugin late 2016 that affect this.)
Just adding this answer in case it helps someone else.
For me, the solution was even simpler: totally skip using 'settings.xml'
and simply specify 'scpexe' instead of 'scp' for protocol (like under distributionManagement section of pom.xml). This then uses your machine's default SSH configuration (unix settings under ~/.ssh).
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>wagon-maven-plugin</artifactId>
<version>1.0</version>
<executions>
<execution>
<id>upload-to-server</id>
<phase>deploy</phase>
<goals><goal>upload-single</goal></goals>
<configuration>
<fromFile>file-to-upload</fromfile>
<url>scpexe://username#serverName/dirname-to-copy-to
<toFile>file-to-upload</toFile>
</configuration>
</execution>
</executions>
</plugin>

I wanted to do the exact same thing today in conjunction with the maven-site-plugin (3.9.1) and was also hitting some roadblocks (specifically, the wagon-ssh plugin insisted on asking me for my Kerberos username and password).
What finally worked for me with wagon-ssh-3.4.3:
<!-- add scp support for mvn site:deploy -->
<dependency>
<groupId>org.apache.maven.wagon</groupId>
<artifactId>wagon-ssh</artifactId>
<version>3.4.3</version>
</dependency>
and in settings.xml:
<server>
<id>ssh-repository</id>
<username>pridkdev</username>
<privateKey>${user.home}/.ssh/pridkdev.ppk</privateKey>
<filePermissions>664</filePermissions>
<directoryPermissions>775</directoryPermissions>
<configuration>
<interactive>false</interactive>
<strictHostKeyChecking>no</strictHostKeyChecking>
<preferredAuthentications>publickey</preferredAuthentications>
</configuration>
</server>
I guess what was crucial is the <configuration> block and there especially the <preferredAuthentications> setting.

I found the necessary info here:
http://maven.apache.org/plugins/maven-deploy-plugin/examples/deploy-ssh-external.html

Related

Unable to send test reports via email using maven plugin

I unable to send test reports via email using maven plugin.
Error: 1. [ERROR] Sending the email to the following server failed : smtp.gmail.com:465: AuthenticationFailedException -> [Help 1]
Sending the email to the following server failed : smtp.gmail.com:995: Could not connect to SMTP host: smtp.gmail.com, port: 995, response: -1 -> [Help 1]
I tried with port 465, 587 as well but nothing would work. Any suggestion would really help as well as not getting any solution over the internet yet
**Code:**
<plugin>
<groupId>ch.fortysix</groupId>
<artifactId>maven-postman-plugin</artifactId>
<executions>
<execution>
<id>send a mail</id>
<phase>test</phase>
<goals>
<goal>send-mail</goal>
</goals>
<inherited>true</inherited>
<configuration>
<!-- From Email address -->
<from>test#totalitycorp.com</from>
<!-- Email subject -->
<subject>Yovo Test Automation Report</subject>
<!-- Fail the build if the mail doesnt reach -->
<failonerror>false</failonerror>
<!-- host -->
<mailhost>smtp.gmail.com</mailhost>
<!-- port of the host -->
<mailport>995</mailport>
<mailssl>true</mailssl>
<mailAltConfig>true</mailAltConfig>
<!-- Email Authentication(USername and Password) -->
<mailuser>test#gmail.com</mailuser>
<mailpassword>234aASD</mailpassword>
<receivers>
<!-- To Email address -->
<receiver>abhi.c74#gmail.com</receiver>
</receivers>
<fileSets>
<fileSet>
<!-- Report directory Path -->
<directory>/home/maverick/eclipse-workspace/YovoAndroidAutomation/test-output</directory>
<includes>
<!-- Report file name -->
<include>emailable-report.html</include>
</includes>
<!-- Use Regular Expressions like **/*.html if you want all the
html files to send -->
</fileSet>
</fileSets>
</configuration>
</execution>
</executions>
</plugin>
[ERROR] Sending the email to the following server failed : smtp.gmail.com:465: Authentication Required -> [Help 1]
This is very interesting issue. If we have to login in to Gmail from an external source—other than from the Gmail login—we need to enable the less secure option (i.e., set it to “ON”).
Here is how we do this:
Go to the “Less secure apps” section in My Account.
Next to “Access for less secure apps,” select "Turn on".
Note to Google Apps users: This setting is hidden if your administrator has locked less secure app account access.

Wildfly 10 - HSQL Datascource - null pointer excception testing connection to database

I would like to know if anybody is aware of the following problem in wildfly 10.
When trying to setup a datasource for HSQL, I was confronted with the problem where for a non xa data source driver, you would expect the configuration element connection-url to be of any use. In particular, this configuration element is of use when you attempt to connect to H2 or oracle.
However, when testing the following configuration I was systematically hitting a null pointer exception on HSQL getConnection.
The hsql modulue is added as jboss module with:
<module xmlns="urn:jboss:module:1.3" name="org.hsql">
<properties>
</properties>
<resources>
<resource-root path="hsqldb-2.3.2.jar"/>
</resources>
<dependencies>
<module name="javax.api"/>
<module name="javax.transaction.api"/>
</dependencies>
</module>
The original configuration of the datasource, the one that does not work was as follows:
<datasource jta="false" jndi-name="java:/jdbc/HSQL_NON_JTA_DS" pool-name="HSQL_NON_JTA_DS" enabled="true" use-ccm="true">
<connection-url>jdbc:hsqldb:hsql://localhost:9001/DATABSE</connection-url>
<datasource-class>org.hsqldb.jdbc.JDBCDataSource</datasource-class>
<driver>hsql</driver>
<security>
<user-name>USER</user-name>
<password>USER</password>
</security>
</datasource>
<driver name="hsql" module="org.hsql">
<datasource-class>org.hsqldb.jdbc.JDBCDataSource</datasource-class>
</driver>
With the following stack trace:
2017-03-01 18:08:47,083 WARN [org.jboss.jca.core.connectionmanager.pool.strategy.OnePool] (thread: management task-6) IJ000604: Throwable while attempting to get a new connection: null: javax.resource.ResourceException: IJ031084: Unable to create connection
at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createLocalManagedConnection(LocalManagedConnectionFactory.java:343)
at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:350)
at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:285)
at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreConcurrentLinkedDequeManagedConnectionPool.createConnectionEventListener(SemaphoreConcurrentLinkedDequeManagedConnectionPool.java:1319)
at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreConcurrentLinkedDequeManagedConnectionPool.getConnection(SemaphoreConcurrentLinkedDequeManagedConnectionPool.java:496)
at org.jboss.jca.core.connectionmanager.pool.AbstractPool.internalTestConnection(AbstractPool.java:1061)
at org.jboss.jca.core.connectionmanager.pool.strategy.OnePool.testConnection(OnePool.java:93)
at org.jboss.as.connector.subsystems.common.pool.PoolOperations$TestConnectionInPool.invokeCommandOn(PoolOperations.java:234)
at org.jboss.as.connector.subsystems.common.pool.PoolOperations$1.execute(PoolOperations.java:90)
at org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:890)
at org.jboss.as.controller.AbstractOperationContext.processStages(AbstractOperationContext.java:659)
at org.jboss.as.controller.AbstractOperationContext.executeOperation(AbstractOperationContext.java:370)
at org.jboss.as.controller.OperationContextImpl.executeOperation(OperationContextImpl.java:1344)
at org.jboss.as.controller.ModelControllerImpl.internalExecute(ModelControllerImpl.java:392)
at org.jboss.as.controller.ModelControllerImpl.execute(ModelControllerImpl.java:217)
at org.jboss.as.domain.http.server.DomainApiHandler.handleRequest(DomainApiHandler.java:212)
at io.undertow.server.handlers.encoding.EncodingHandler.handleRequest(EncodingHandler.java:72)
at org.jboss.as.domain.http.server.security.SubjectDoAsHandler$1.run(SubjectDoAsHandler.java:72)
at org.jboss.as.domain.http.server.security.SubjectDoAsHandler$1.run(SubjectDoAsHandler.java:68)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.jboss.as.controller.AccessAuditContext.doAs(AccessAuditContext.java:92)
at org.jboss.as.domain.http.server.security.SubjectDoAsHandler.handleRequest(SubjectDoAsHandler.java:68)
at org.jboss.as.domain.http.server.security.SubjectDoAsHandler.handleRequest(SubjectDoAsHandler.java:63)
at io.undertow.server.handlers.BlockingHandler.handleRequest(BlockingHandler.java:56)
at org.jboss.as.domain.http.server.DomainApiCheckHandler.handleRequest(DomainApiCheckHandler.java:95)
at io.undertow.security.handlers.AuthenticationCallHandler.handleRequest(AuthenticationCallHandler.java:52)
at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202)
at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:793)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
at org.hsqldb.jdbc.JDBCDataSource.getConnection(Unknown Source)
at org.hsqldb.jdbc.JDBCDataSource.getConnection(Unknown Source)
at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createLocalManagedConnection(LocalManagedConnectionFactory.java:312)
... 31 more
This null pointer exception happend, quite simply because.
In the JDBCDataSource.java of HSQL, the following method was getting a null URL.
Meaning, JBOSS had not picked up the URL on the connection-url and configured on the driver.
private Connection getConnection(String url,
Properties props) throws SQLException {
if (!url.startsWith("jdbc:hsqldb:")) {
url = "jdbc:hsqldb:" + url;
}
return JDBCDriver.getConnection(url, props);
}
NOTE: If you want to debug HSQL you need to use the -debug.jar files that HSQL releases internally.
In any case...
After finding out that the subsytem for data sources is provided by IRON JCAMAR, it was possible to hunt the documentation for extra configuration properties to work-around what should never have been a problem.
Here is the documentation on elements supported for the data-source.
http://www.ironjacamar.org/doc/userguide/1.1/en-US/html_single/index.html#deployingds_descriptor
Ultiamtely, to make the connection work It was necessar to enrich it with the following element.
<connection-property name="url">jdbc:hsqldb:hsql://localhost:9001/DATABSE</connection-property>
Does it make any sense that additional element was necessary? It should have been redundant and unecessary. It is a good thing that custome data source properties are to be supported.
In this case, Is the problem with HSQL or is the problem with JBOSS.
In eery other app server, specifying the connection URL has always worked fine.
To me this appars to be a particular behavior of Wildfly.
This quite a problem since your average documentation on data sources just makes it abundantly clear that the connection-url is the element you do need to configure. In this case, it really is not.
Many thanks for a reply on this.
Looks like configuration issue. Update module name to org.hsqldb in datasource and module.xml file to follow standard module naming conventions. Update directory name from hsql to hsqldb under modules directory. It's all up to you. You can use your custom module naming convention too.
Some of the drivers does not have getURL() method implemented in their datasource classes. So, we have to specify datasource configuration differently for them.
Like Postgres, does not have getURL method. So, we will specify properties like this in our stanalone.xml/domain.xml file.
<xa-datasource-property name="ServerName">DatabaseHostName</xa-datasource-property>
<xa-datasource-property name="PortNumber">DatabasePortName</xa-datasource-property>
<xa-datasource-property name="DatabaseName">DatabaseName</xa-datasource-property>
<xa-datasource-class>com.edb.xa.PGXADataSource</xa-datasource-class>
<driver>postgresql</driver>
<security>
<user-name>database.username</user-name>
<password>database.password</password>
</security>
Whereas in case of Oracle which has getURL method implemented, configuration given below works fine.
<xa-datasource-property name="URL">jdbc:oracle:thin:#database.host:database.port:database.name</xa-datasource-property>
<xa-datasource-class>oracle.jdbc.xa.client.OracleXADataSource</xa-datasource-class>
<driver>oracle</driver>
I am not sure about HSQL driver implementation of getURL API. You can try specifying properties like I mentioned for Postgres and check if it works for your use-case.

How to suppress/control logging of Wagon-FTP Maven extension?

I'm deploying Maven site by FTP, using Wagon-FTP. Works fine, but output is full of FTP connection/authentication details, which effectively expose logins and passwords to everybody (especially if the project is open source and its CI protocols are publicly accessible):
[...]
[INFO]
[INFO] --- maven-site-plugin:3.0-beta-3:deploy (default-deploy) # rempl ---
Reply received: 220---------- Welcome to Pure-FTPd [privsep] [TLS] ----------
220-You are user number 1 of 50 allowed.
220-Local time is now 09:08. Server port: 21.
220 You will be disconnected after 15 minutes of inactivity.
Command sent: USER ****
Reply received: 331 User **** OK. Password required
Command sent: PASS ********
Reply received: 230-User **** has group access to: ***
230 OK. Current restricted directory is /
[...]
Is it possible to suppress this logging? Or configure it... This is a section of my pom.xml, where Wagon-FTP is used:
[...]
<build>
<extensions>
<extension>
<groupId>org.apache.maven.wagon</groupId>
<artifactId>wagon-ftp</artifactId>
<version>1.0-beta-7</version>
</extension>
</extensions>
[...]
</build>
[...]
Not possible, and basically it is related to maven site plugin and not the wagon ftp (which is only a simple adapter for the apache-commons-net ftp client). See the source of AbstractDeployPlugin from line 310.
Debug debug = new Debug();
wagon.addSessionListener( debug );
wagon.addTransferListener( debug );
Where Debug is using the standard output.
IMHO the nice solution would be to use a more sophisticated SessionListener or a flag to avoid addSessionListener(debug) if not needed in the Wagon source.

Looking for proper hbase-site.xml hbase-default.xml config example for HBase client

I am trying to connect to an HBase node from a Java application. HBaseConfiguration is key, but the available Javadoc and documentation is really poor and insufficient.
Does anyone have proper examples of hbase-site.xml hbase-default.xml to use for remote connection?
Thanks!
There are only two variables you need to set from a clients point of view:
hbase.rootdir
hbase.zookeeper.quorum
Here are the steps from my setup doc about the hbase-site.xml. We don't make any changes to the hbase-default.xml as ... well... that's all the default settings. :)
edit hbase-site.xml. Copy the following to the file.
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl"
href="configuration.xsl"?>
<configuration>
<property>
<name>hbase.rootdir</name> <value>hdfs://PDHadoop1.corp.COMPANY.com:54310/usr/hbase</value>
<final>true</final> </property>
<property>
<name>hbase.zookeeper.quorum</name>
<value>PDHadoop1.corp.COMPANY.com,PDHadoop2.corp.COMPANY.com,PDHadoop3.corp.COMPANY.com,PDHadoop4.corp.COMPANY.com</value>
<final>true</final> </property>
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
<final>true</final> </property>
</configuration>
Save the file and quit the editor.
Please note that hbase.rootdir is pointing to PDHadoop1 as that is the name node in development environment. Similarly, hbase.zookeeper.quorum is pointing to all zookeeper servers in development environment. Please substitute these values with appropriate server names in your environment.
edit regionservers. Copy the following to the file.
PDHadoop3.corp.COMPANY.com
PDHadoop2.corp.COMPANY.com
PDHadoop1.corp.COMPANY.com
I apologize for the XML's lack of formatting.
These are the settings we use in production, I opened the file on my dev cluster to verify.
I hope that helps.
One major gotcha that I've encountered is that if your /etc/hosts contains an entry for that hostname pointing to the loopback address (127.0.0.1, 127.0.1.1, et cetra), then Hbase master will incorrectly register itself in Zookeeper with that loopback address -- which will not work when your client is not on the same machine as your master.
I wasted quite a bit of time to (first) get Hbase working. The solution is to remove the entry in /etc/hosts; but this requires that you override the "out of the box" behavior of the OS, at least on the ubuntu box that I've tested this on...

How can the user-agent be changed in Maven?

How can I change the user-agent in Maven? I need to be able to change this to get through the company firewall. I am using version 2.2.1 and I noticed an improvement in the 2.0.10 release notes:
[MNG-3652] - set a user agent for Maven HTTP requests.
Brett Porter posted a blog on Configuring Maven HTTP Connections that describes how you can do this and some other funky things:
<server>
<id>archiva.localhost</id>
<configuration>
<httpHeaders>
<property>
<name>User-Agent</name>
<value>Internal-Build-System/1.0</value>
</property>
</httpHeaders>
</configuration>
</server>
for command line version
try
"-Daether.connector.userAgent=your custom user agent"