GlassFish running on a host with a public ip address and jmx port 8986
On startup
JMXStartupService has started JMXConnector on JMXService URL service:jmx:rmi://test.anahata.net.au:8986/jndi/rmi://test.anahata.net.au:8986/jmxrmi]]
When I try to connect via JConsole using test.anahata.net.au:8986 + user + password from another box, I get:
non-JRMP server at remote endpoint
If I try to connect using service:jmx:rmi://test.anahata.net.au:8986/jndi/rmi://test.anahata.net.au:8986/jmxrmi
I get:
connection refused
domain.xml
> <admin-service system-jmx-connector-name="system"
> type="das-and-server">
> <jmx-connector address="0.0.0.0" port="8986" name="system" auth-realm-name="admin-realm">
> <ssl classname="com.sun.enterprise.security.ssl.GlassfishSSLImpl"
> client-auth="want" cert-nickname="s1as"></ssl>
> </jmx-connector>
> <property name="adminConsoleContextRoot" value="/admin"></property>
> <property name="adminConsoleDownloadLocation" value="${com.sun.aas.installRoot}/lib/install/applications/admingui.war"></property>
> <property name="ipsRoot" value="${com.sun.aas.installRoot}/.."></property>
> <das-config></das-config>
> </admin-service>
>
> ...
> <jvm-options>-Djava.rmi.server.hostname=test.anahata.net.au</jvm-options>
I have a smilar setup on a different vm running gf 4.1.1 and a real SSL certificate and I can connect to the it using host:port but If I try to connect using
service:jmx:rmi://host.com:8886/jndi/rmi://host.com:8886/jmxrmi
I get
java.rmi.ConnectIOException: non-JRMP server at remote endpoint
Has anyone got his head around this?
-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.local.only=false
-Dcom.sun.management.jmxremote.port=8686
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.authenticate=false
-Djava.rmi.server.hostname=myremoteip
Since glassfish 4 this is the only way.
Though I did managed to connect to glassfish's own jmx only before running asadmin enable-secure-admin.
turn on logging to FINE see if you got "LoginException during JMX auth"
javax.enterprise.system.tools.admin.security
Anyway my conclusion was lot of headache, just use the jvm jmx above.
When you get this error:
java.rmi.ConnectIOException: non-JRMP server at remote endpoint
you are missing the keystore.jks file from glassfish/payara domain.
Related
I have a Mule 4 application [App1] created on CloudHub. I tried to deploy the application's jar file onto CloudHub. This application has a Static IP [eg. 100.101.102.103] assigned to it in Runtime Manager. This IP address is whitelisted by customer to allow communication with their SFTP sites and APIs. My Mule application has APIs and some SFTP flows. When I try to deploy my mule application [App1], the deployment fails with below error:
Connectivity test failed for config 'SFTP_Config'. Application deployment will continue. Error was: Could not establish SFTP connection with host: 'sftp.hostname' at port: '22' - Error during login to 'sftpuser#sftp.hostname'.
The SFTP Config is:
<sftp:config name="SFTP_Config" doc:name="SFTP Config" doc:id="5d626288-5181-41d5-807d-2786ea4292d8" >
<sftp:connection host="${sftp.host}" port="${sftp.port}" username="${secure::sftp.username}" password="${secure::sftp.password}" connectionTimeoutUnit="MINUTES" connectionTimeout="2" responseTimeoutUnit="MINUTES" responseTimeout="2" workingDir="${sftp.peoplePosition.directory}">
<reconnection failsDeployment="false" >
<reconnect frequency="${sftp.retryInterval}" count="${sftp.retryAttempts}" />
</reconnection>
</sftp:connection>
</sftp:config>
I also tried using failsDeployment="false" in the SFTP configuration as recommended in this KB article
but it didn't work either.
The log shows:
[2023-02-16 05:59:00.754] ERROR
org.mule.extension.sftp.internal.connection.SftpConnectionProvider
[qtp1351434790-36]: Auth fail
com.jcraft.jsch.JSchException: Auth fail
[2023-02-16 05:59:00.824] WARN
org.mule.runtime.core.internal.connection.
PoolingConnectionManagementStrategy
[qtp1351434790-36]: Failed to create a connection while
applying the pool initialization policy.
org.mule.runtime.api.connection.ConnectionException:
Could not establish SFTP connection with host: 'sftphost' at port: '22'
- Error during login to sftpuser#sftphost
at
org.mule.runtime.core.internal.connection.ErrorTypeHandler
ConnectionProviderWrapper.lambda$connect$0(ErrorTypeHandler
ConnectionProviderWrapper.java:70)
at java.util.Optional.map(Optional.java:215)
I have verified the SFTP credentials, they are working fine with Winscp.
Is there any way a CloudHub worker can complete the deployment successfully or validate the SFTP configuration using Static IP instead of it's own IP address?
I'm trying to use JMX with activeMQ for monitoring so far I've been using this and this as a reference but so far I'm unable to connect to jmx remotely and also I don't see any mention of jmx url in activemq logs. I'm wondering if there is another way to make sure jmx is working? is it supposed to be indicated in activemq logs?
PS I'm using jdk1.7 and activeMQ 5.14.2.
Thanks in advance!
EDIT
I set useJmx="true" in my activemq.xml file:
<broker xmlns="http://activemq.apache.org/schema/core" brokerName="primary" useJmx="true" dataDirectory="${activemq.data}">
I tried two steps:
FIRST
I tried changing management context from createConnector="false" to :
<managementContext>
<managementContext createConnector="true" connectorPort="1099"/>
</managementContext>
FOR FIRST TIME THE PORT IS OPEN AND ACTIVEMQ RUNS FINE AND JMX URL GETS REPORTED IN LOGS ALTHOUGH I CAN NOT CONNECT IT TO IT REMOTLEY BUT IM ASSUMING ITS WORKING
SECOND
I reverted back the changes I made for managmentContext and I tried setting:
ACTIVEMQ_SUNJMX_START="-Dcom.sun.management.jmxremote.port=1099 -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.password.file=${ACTIVEMQ_BASE}/jmx.password -Dcom.sun.management.jmxremote.access.file=${ACTIVEMQ_BASE}/jmx.access"
in bin/activemq script and I set a username in conf/jmx.access file as:
admin readwrite
And also have set a password in conf/jmx.password:
admin activemq
NOW ACTIVEMQ IS NOT RUNNINT AT ALL BUT IT WILL RUN IF I SET
AUTHENTICATE=FALSE AND DELETE JMX.ACCESS AND JMX.PASSWORD
CONFIGURATION IN BIN/ACTIVEMQ FILE BUT I NEED USER NAME AND PASSWORD
FOR SECURITY REASONS
I found this post which has the exact same issue as mine. any ideas?
Password authentication for remote monitoring is enabled by default. To disable it, set the following system property when you start the JVM:
-Dcom.sun.management.jmxremote.authenticate=false like you done in second test but you need to add system property -Dcom.sun.management.jmxremote
Try to add these jvm param to env file and update host ip
-Djava.net.preferIPv4Stack=true -Djava.rmi.server.hostname=X.X.X.X
UPDATE
SO, to resume, i think that the FIRST step you tried is the best, for making it working these are the steps :
revert all jmx env file changes, like this :
# ACTIVEMQ_SUNJMX_START="$ACTIVEMQ_SUNJMX_START -Dcom.sun.management.jmxremote.port=1099 "
# ACTIVEMQ_SUNJMX_START="$ACTIVEMQ_SUNJMX_START -Dcom.sun.management.jmxremote.password.file=${ACTIVEMQ_CONF}/jmx.password"
# ACTIVEMQ_SUNJMX_START="$ACTIVEMQ_SUNJMX_START -Dcom.sun.management.jmxremote.access.file=${ACTIVEMQ_CONF}/jmx.access"
# ACTIVEMQ_SUNJMX_START="$ACTIVEMQ_SUNJMX_START -Dcom.sun.management.jmxremote.ssl=false"
ACTIVEMQ_SUNJMX_START="$ACTIVEMQ_SUNJMX_START -Dcom.sun.management.jmxremote"
<broker useJmx="true" ...
<managementContext>
<managementContext createConnector="true" connectorPort="1099" />
</managementContext>
verify that in AMQ logs you have
INFO | JMX consoles can connect to
service:jmx:rmi:///jndi/rmi://localhost:1099/jmxrmi |
org.apache.activemq.broker.jmx.ManagementContext | JMX connector
NOTE : Assuming that 10.10.10.16 is the IP of AMQ host.
try to connect with jconsole from another machine than AMQ host with url "service:jmx:rmi:///jndi/rmi://10.10.10.16:1099/jmxrmi" without user/pwd.
if you cannot connect, try like this :
<managementContext>
<managementContext createConnector="true" connectorPort="1099" connectorHost="10.10.10.16" />
</managementContext>
verify that in AMQ logs you have
INFO | JMX consoles can connect to
service:jmx:rmi:///jndi/rmi://10.10.10.16:1099/jmxrmi |
org.apache.activemq.broker.jmx.ManagementContext | JMX connector
retry to connect, step 4
at this step normally you can connect with jconsole.
if you want to add security and authorizations, use this :
<managementContext>
<managementContext createConnector="true" connectorPort="1099" connectorHost="10.10.10.16" >
<property xmlns="http://www.springframework.org/schema/beans" name="environment">
<map xmlns="http://www.springframework.org/schema/beans">
<entry xmlns="http://www.springframework.org/schema/beans" key="jmx.remote.x.password.file"
value="${activemq.conf}/jmx.password"/>
<entry xmlns="http://www.springframework.org/schema/beans" key="jmx.remote.x.access.file"
value="${activemq.conf}/jmx.access"/>
</map>
</property>
</managementContext>
</managementContext>
Please try these steps and let me know in which one you fails to connect and provide error message from jconsole.
A couple troubleshooting steps:
Start jconsole or visualvm on the same system and connect using the "pid" attach method. Browse the MBeans and confirm org.apache.activemq beans are present
Run netstat -na and confirm ports 1099 (and 44444) are in LISTEN
Look at logs and confirm you do not have any "java.net.BindException: Address already in use.." messages that indicate a port conflict with an already running Java process.
Edit bin/env to configure JMX (this disables requiring SSL, sets the port to 1099 and disables requiring username and password.
ACTIVEMQ_SUNJMX_START="$ACTIVEMQ_SUNJMX_START -Dcom.sun.management.jmxremote.port=1099 "
ACTIVEMQ_SUNJMX_START="$ACTIVEMQ_SUNJMX_START -Dcom.sun.management.jmxremote.ssl=false "
ACTIVEMQ_SUNJMX_START="$ACTIVEMQ_SUNJMX_START -Dcom.sun.management.jmxremote "
# ACTIVEMQ_SUNJMX_START="$ACTIVEMQ_SUNJMX_START -Dcom.sun.management.jmxremote.password.file=${ACTIVEMQ_CONF}/jmx.password"`
# ACTIVEMQ_SUNJMX_START="$ACTIVEMQ_SUNJMX_START -Dcom.sun.management.jmxremote.access.file=${ACTIVEMQ_CONF}/jmx.access"
I am trying to run in jboss instance in domain mode. While I do that I am getting the following issue......
[Host Controller] 12:45:56,535 WARN [org.jboss.as.host.controller] (Controller Boot Thread) JBAS010900: Could not connect to remote domain controller at remote://nnn.nn.nn.88:9999 -- java.net.ConnectException: JBAS012144: Could not connect to remote://nnn.nn.nn.88:9999. The connection timed out
I had ran two JBoss instance in domain mode after configuring...
First JBoss instance->
./domain.sh -b nnn.nn.nn.88 -Djboss.bind.address.management=nnn.nn.nn.88
Second JBoss Instance ->
./domain.sh -b nnn.nn.nn.89 -Djboss.domain.master.address=nnn.nn.nn.88 --host-config=host-slave.xml
nnn.nn.nn.88 host.xml configuration is as follows...
<domain-controller>
<local/>
</domain-controller>
nnn.nn.nn.89 host-slave.xml configuration is as follows...
<domain-controller>
<remote host="${jboss.domain.master.address}" port="${jboss.domain.master.port:9999}" security-realm="ManagementRealm"/>
<domain-controller>
I am able to telnet to port 9999 on host nnn.nn.nn.88 from 89..... as I configured by removing loopback ip for public & management port...... Although is it the implication that <domain-controller> has <local/>....
Please help me to solve this issue... JDK version is JDK 7 Update 80.... EAP 6.3....
In HC host.xml and if we use --host-config=host-slave.xml that particular xml has to connected with DC under <domain-controller> node....
jboss.domain.master.address should be Domain Controller address nnn.nn.nn.88....
<domain-controller>
<remote host="${jboss.domain.master.address:nnn.nn.nn.88}" port="${jboss.domain.master.port:9999}" security-realm="ManagementRealm"/>
<domain-controller>
As per the solution article from redhat....
https://access.redhat.com/solutions/218053#
I ran following command for the same configuration which I had while posting this question..... And I got succeeded.....
DC->
./domain.sh -b my-host-ip1 -bmanagement my-host-ip1
HC->
./domain.sh -Djboss.domain.master.address=my-host-ip1 -b my-host-ip2 -bmanagement my-host-ip2
Although is this way of configuring gives clustering capability to DC and HCs..... I had raised same question to Redhat on the same solution article..... The answer must be yes I hope....
https://access.redhat.com/solutions/218053#comment-975683
I have remote JBoss 7.1 server and I would like to connect to this server by using VisualVM or JConsole.
I googled a bit and found several threads/tutorials on how to connect to JBoss 7.1 by using VisualVM or JCoonsole, e.g.:
https://community.jboss.org/thread/171346?start=0&tstart=0
https://issues.jboss.org/browse/WFLY-895
https://community.jboss.org/wiki/UsingJconsoleToConnectToJMXOnAS7
http://blog.akquinet.de/2012/11/01/connecting-visualvm-with-a-remote-jboss-as-7-eap6-jvm-process/
Unfortunately I had no luck in connecting to my JBoss 7.1 over JMX.
Do you have any ideas how to connect to JBoss 7.1 with VisualVM?
EDIT:
I added -Dcom.sun.management.jmxremote.port=1090 -Dcom.sun.management.jmxremote.authenticate=false to standalone.conf.bat, but I got an exception:
Caused by: java.lang.IllegalStateException: The LogManager was not properly inst
alled (you must set the "java.util.logging.manager" system property to "org.jboss.logmanager.LogManager").
So, I added another option to JAVA_OPTS: -Dcom.sun.management.jmxremote.port=1090 -Dcom.sun.management.jmxremote.authenticate=false -Djava.util.logging.manager=org.jboss.logmanager.LogManager
but the exception I receive is this:
Could not load Logmanager "org.jboss.logmanager.LogManager"
java.lang.ClassNotFoundException: org.jboss.logmanager.LogManager
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
Found a solution. I needed to add the following params to JAVA_OPTS in standalone.conf.bat:
-Dcom.sun.management.jmxremote.port=1090 ^
-Dcom.sun.management.jmxremote.authenticate=false ^
-Dcom.sun.management.jmxremote ^
-Dcom.sun.management.jmxremote.ssl=false ^
-Djava.util.logging.manager=org.jboss.logmanager.LogManager ^
-Xbootclasspath/p:<JBOSS_PATH>/modules/org/jboss/logmanager/main/jboss-logmanager-1.2.2.GA.jar ^
-Xbootclasspath/p:<JBOSS_PATH>/modules/org/jboss/logmanager/log4j/main/jboss-logmanager-log4j-1.0.0.GA.jar ^
-Xbootclasspath/p:<JBOSS_PATH>/modules/org/apache/log4j/main/log4j-1.2.16.jar -Djboss.modules.system.pkgs=org.jboss.logmanager
Note 1: Use \ instead of ^ on Unix.
Note 2: Replace <JBOSS_PATH> with your JBoss 7.x installation path. Mine was c:/java/jboss-as-7.1.1.Final.
You can connect directly to the JBOSS JMX instead of the VM one (no need to modify JAVA_OPTS)
Just ensure you have the following config in standalone.xml (JMX subsystem active)
in standalone mode (listened port 9999):
<subsystem xmlns="urn:jboss:domain:jmx:1.1">
<show-model value="true"/>
<remoting-connector />
</subsystem>
in domain mode (listened port 4447)
<subsystem xmlns="urn:jboss:domain:jmx:1.1">
<show-model value="true"/>
<remoting-connector use-management-endpoint="false"/>
</subsystem>
Then we need a little hack : As JBoss JMX implementation is a bit specific we need to include some jboss lib to the classpath of JVisualVM
Get the file $JBOSS_HOME/bin/jconsole.sh/jconsole.bat and copy / rename it to jvisualvm.sh / jvisualvm.bat in the same directory.
Then replace the executable call to jconsole per jvisualvm using -cp:a "$CLASSPATH"instead of -J-Djava.class.path="$CLASSPATH"
Now just launch visualVM using the script and add a new JMX connection (file menu) using the folowing url service:jmx:remoting-jmx://hostname:port (the credentials are the one of management realm)
NB: think to ssh tunnel if you don't have direct access to JMX port
If you open a command shell and type "netstat -a", do you see something LISTENING on the JMX port 1090? If not, perhaps you have to check JBOSS configuration.
https://community.jboss.org/thread/171346?start=0&tstart=0
Add the JVM opts to run.conf (run.conf.bat)
JAVA_OPTS="$JAVA_OPTS -Dcom.sun.management.jmxremote.port=9999 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote"
In Visual VM console open a remote host. And Add JMX Connection and give port number as <9999>
I'm trying to use SSL with the JMX connector that Active MQ creates, but with no success. I'm able to get SSL working with the JVM platform JMX connector, but that requires storing keystore and truststore passwords plaintext, which is a no-go for our project.
Using the instructions here, I set up managementContext in activemq.xml as follows:
<managementContext>
<managementContext createConnector="true">
<property xmlns="http://www.springframework.org/schema/beans" name="environment">
<map xmlns="http://www.springframework.org/schema/beans">
<entry xmlns="http://www.springframework.org/schema/beans"
key="javax.net.ssl.keyStore"
value="${activemq.base}/conf/keystore.jks"/>
<entry xmlns="http://www.springframework.org/schema/beans"
key="javax.net.ssl.keyStorePassword"
value="${keystore.password}"/>
<entry xmlns="http://www.springframework.org/schema/beans"
key="javax.net.ssl.trustStore"
value="${activemq.base}/conf/truststore.jks"/>
<entry xmlns="http://www.springframework.org/schema/beans"
key="javax.net.ssl.trustStorePassword"
value="${truststore.password}"/>
</map>
</property>
</managementContext>
</managementContext>
This section seems to be completely ignored when the connector starts up. I can connect without credentials. I also tried using username and password authentication instead of ssl for JMX, as seen here, and that worked fine.
Has anyone seen this before? Any ideas? Thanks!
Have you enabled jmx ssl in the activemq launch scripts? On windows in the activemq-admin or activemq batch files, uncomment and modify the SUNJMX settings.
JMX authentiation is independent of whether ssl is used. It is controlled by the authenticate attribute. By default it will use the jmx access files in your jre, so re-point them with the system properties shown below. You may get an error message stating that the files themselves must be access controlled, so set them with chmod on unix or cacls on windows. I would suggest even turning off the ssl and getting the authentication to work first. You can test with jconsole with a remote connection to confirm that it wants credentials. Then follow-up with the ssl stuff.
set SUNJMX=-Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.port=1199 -Dcom.sun.management.jmxremote.authenticate=true -Dcom.sun.management.jmxremote.ssl=true -Dcom.sun.management.jmxremote.local.only=false -Dcom.sun.management.jmxremote.password.file=%ACTIVEMQ_BASE%/conf/access/jmx.password -Dcom.sun.management.jmxremote.access.file=%ACTIVEMQ_BASE%/conf/access/jmx.access
I had the same issue regarding the ActiveMQ SSL configuration (keystore & password) in the XML not working.
My requirement was to enable remote JMX monitoring of ActiveMQ with SSL and authentication through a firewall.
I resolved it using a custom JMX connector (via a Java Agent), rather than using the JMX connector that Active MQ creates.
see: JMX connectivity through a firewall for an example (JMXAgent.java)
The important entries for configuring SSL in the JMXAgent.java are:
Map<String, Object> env = new HashMap<String, Object>();
SslRMIClientSocketFactory csf = new SslRMIClientSocketFactory();
SslRMIServerSocketFactory ssf = new SslRMIServerSocketFactory();
env.put(RMIConnectorServer.RMI_CLIENT_SOCKET_FACTORY_ATTRIBUTE, csf);
env.put(RMIConnectorServer.RMI_SERVER_SOCKET_FACTORY_ATTRIBUTE, ssf);
You can also specify your authentication files in the env Map:
env.put("jmx.remote.x.password.file", System.getProperty("password.file","<default_path>"));
env.put("jmx.remote.x.access.file", System.getProperty("access.file","<default_path>"));
The Java Agent needs to be compiled and put into a jar with a valid manifest file as described here
Add the following to the activemq launch configuration (depending on activemq version/ environment and run ActiveMQ:
-javaagent:<full_path_to_agent_jar_file> \
-Dpassword.file=<full_path_to_jmx.password_file> \
-Daccess.file=<full_path_to_jmx.access_file> \
-Djavax.net.ssl.keyStore=<full_path_to_keystore_file> \
-Djavax.net.ssl.keyStorePassword=<password>
You should then be able to connect through jconsole (with correct security parameters)
The remote JMX connection URL will be something like:
service:jmx:rmi://<host>:<rmi_server_port>/jndi/rmi://<host>:<port>/jmxrmi
Note - ports can be configured in the Java Agent.