I am running Jmeter 5.4.1 using openjdk 11 across all Linux machines. I tried using ubuntu 18.04.5, ubuntu 20.04.2, and Debian 10 which all had the same issues. I am able to run tests with specified ports or RMI SSL enabled but not both together. Everything works until I set client.rmi.localport then it has errors. Currently I don't have the firewall enabled as I was trying to eliminate obstacles. Nothing is on the specified ports until Jmeter runs (1099, 4888, 5888, 5889, 5900 are free and open). These are the changes I made to the user.properties file.
Client (Master)
remote_hosts=172.xx.xxx.xx1,172.xx.xx.xx2
server.rmi.ssl.keystore.file=/home/user/rmi_keystore.jks
client.rmi.localport=5888
Server (Slave)
server.rmi.ssl.keystore.file=/home/user/rmi_keystore.jks
server.rmi.localport=4888
/etc/hosts file looks like this on each machine just with the last number of ip changed and hostname to reflect that.
127.0.0.1 localhost
127.0.1.1 xxx-l1.xxx.xxxx.xxx xxx-l1
172.xx.xxx.xx1 xxx-l1.xxx.xxxx.xxx xxx-l1
# The following lines are desirable for IPv6 capable hosts
::1 localhost ip6-localhost ip6-loopback
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
Currently trying to get it to work in GUI or command line but mostly been using the GUI to test with. I tried using either below to start the Client and run the tests.
./jmeter -Djava.rmi.server.hostname=172.xx.xxx.xx8
JVM_ARGS="-Xms1024m -Xmx4096m" ./jmeter -Djava.rmi.server.hostname=172.xx.xxx.xx8
If its a simple test only hitting google.com in the gui I'm able to run the distributed test once to success and then after that when rerunning the test it has these errors in the Client log and nothing generated in the Server log. If its a more complicated test or just including something as a view results tree on the first run it will show the errors. Command line also fails first run. The other log I have seen says Caused by: java.net.SocketException: Broken pipe (Write failed) inplace of the java.io.NotSerializableException but still shows same port messages.
2021-07-09 09:57:17,008 INFO o.a.j.s.FileServer: Default base='/home/user/Desktop/apache-jmeter-5.4.1/bin'
2021-07-09 09:57:17,011 INFO o.a.j.g.a.Load: Loading file: /home/user/Desktop/Summer2021/basictest.jmx
2021-07-09 09:57:17,012 INFO o.a.j.s.FileServer: Set new base='/home/user/Desktop/Summer2021'
2021-07-09 09:57:17,208 INFO o.a.j.s.SaveService: Testplan (JMX) version: 2.2. Testlog (JTL) version: 2.2
2021-07-09 09:57:17,226 INFO o.a.j.s.SaveService: Using SaveService properties version 5.0
2021-07-09 09:57:17,228 INFO o.a.j.s.SaveService: Using SaveService properties file encoding UTF-8
2021-07-09 09:57:17,230 INFO o.a.j.s.SaveService: Loading file: /home/user/Desktop/Summer2021/basictest.jmx
2021-07-09 09:57:17,255 INFO o.a.j.p.h.s.HTTPSamplerBase: Parser for text/html is org.apache.jmeter.protocol.http.parser.LagartoBasedHtmlParser
2021-07-09 09:57:17,255 INFO o.a.j.p.h.s.HTTPSamplerBase: Parser for application/xhtml+xml is org.apache.jmeter.protocol.http.parser.LagartoBasedHtmlParser
2021-07-09 09:57:17,255 INFO o.a.j.p.h.s.HTTPSamplerBase: Parser for application/xml is org.apache.jmeter.protocol.http.parser.LagartoBasedHtmlParser
2021-07-09 09:57:17,255 INFO o.a.j.p.h.s.HTTPSamplerBase: Parser for text/xml is org.apache.jmeter.protocol.http.parser.LagartoBasedHtmlParser
2021-07-09 09:57:17,255 INFO o.a.j.p.h.s.HTTPSamplerBase: Parser for text/vnd.wap.wml is org.apache.jmeter.protocol.http.parser.RegexpHTMLParser
2021-07-09 09:57:17,255 INFO o.a.j.p.h.s.HTTPSamplerBase: Parser for text/css is org.apache.jmeter.protocol.http.parser.CssParser
2021-07-09 09:57:17,383 INFO o.a.j.s.FileServer: Set new base='/home/user/Desktop/Summer2021'
2021-07-09 09:57:23,234 INFO o.a.j.e.DistributedRunner: Configuring remote engine: 172.xx.xxx.xx1
2021-07-09 09:57:24,033 INFO o.a.j.e.DistributedRunner: Starting distributed test with remote engines: [172.xx.xxx.xx1] # Fri Jul 09 09:57:24 MDT 2021 (1625846244032)
2021-07-09 09:57:24,033 INFO o.a.j.e.ClientJMeterEngine: running clientengine run method
2021-07-09 09:57:24,034 INFO o.a.j.e.u.CompoundVariable: Note: Function class names must contain the string: '.functions.'
2021-07-09 09:57:24,034 INFO o.a.j.e.u.CompoundVariable: Note: Function class names must not contain the string: '.gui.'
2021-07-09 09:57:24,254 INFO o.a.j.r.RmiUtils: Resolving by name the value of System property 'java.rmi.server.hostname': 172.xx.xxx.xx8
2021-07-09 09:57:24,254 INFO o.a.j.r.RmiUtils: Local IP address=172.xx.xxx.xx8
2021-07-09 09:57:24,256 INFO o.a.j.s.SampleEvent: List of sample_variables: []
2021-07-09 09:57:24,452 INFO o.a.j.r.SSLRMIServerSocketFactory: Created SSLSocket: [SSL: ServerSocket[addr=/172.xx.xxx.xx8,localport=5890]]
2021-07-09 09:57:24,455 INFO o.a.j.r.RmiUtils: Resolving by name the value of System property 'java.rmi.server.hostname': 172.xx.xxx.xx8
2021-07-09 09:57:24,455 INFO o.a.j.r.RmiUtils: Local IP address=172.xx.xxx.xx8
2021-07-09 09:57:24,655 INFO o.a.j.r.SSLRMIServerSocketFactory: Created SSLSocket: [SSL: ServerSocket[addr=/172.xx.xxx.xx8,localport=5889]]
2021-07-09 09:57:25,209 INFO o.a.j.e.ClientJMeterEngine: sent test to 172.xx.xxx.xx1 basedir='.'
2021-07-09 09:57:25,209 INFO o.a.j.e.ClientJMeterEngine: Sending properties {}
2021-07-09 09:57:25,212 INFO o.a.j.e.ClientJMeterEngine: sent run command to 172.xx.xxx.xx1
2021-07-09 09:57:25,213 INFO o.a.j.e.DistributedRunner: Remote engines have been started:[172.xx.xxx.xx1]
2021-07-09 09:57:25,214 INFO o.a.j.g.u.JMeterMenuBar: setRunning(true, 172.xx.xxx.xx1)
2021-07-09 09:57:25,445 INFO o.a.j.g.u.JMeterMenuBar: setRunning(false, 172.xx.xxx.xx1)
2021-07-09 09:57:29,906 INFO o.a.j.e.DistributedRunner: Configuring remote engine: 172.xx.xxx.xx1
2021-07-09 09:57:30,347 INFO o.a.j.e.DistributedRunner: Starting distributed test with remote engines: [172.xx.xxx.xx1] # Fri Jul 09 09:57:30 MDT 2021 (1625846250347)
2021-07-09 09:57:30,347 INFO o.a.j.e.ClientJMeterEngine: running clientengine run method
2021-07-09 09:57:30,348 INFO o.a.j.r.RmiUtils: Resolving by name the value of System property 'java.rmi.server.hostname': 172.xx.xxx.xx8
2021-07-09 09:57:30,348 INFO o.a.j.r.RmiUtils: Local IP address=172.xx.xxx.xx8
2021-07-09 09:57:30,550 ERROR o.a.j.e.ConvertListeners: RemoteException occurred while replacing Remotable item.
java.rmi.server.ExportException: Listen failed on port: 5890; nested exception is:
java.io.IOException: Could not bind to /172.xx.xxx.xx8 using port 5890
at sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:337) ~[?:?]
at sun.rmi.transport.tcp.TCPTransport.exportObject(TCPTransport.java:243) ~[?:?]
at sun.rmi.transport.tcp.TCPEndpoint.exportObject(TCPEndpoint.java:412) ~[?:?]
at sun.rmi.transport.LiveRef.exportObject(LiveRef.java:147) ~[?:?]
at sun.rmi.server.UnicastServerRef.exportObject(UnicastServerRef.java:234) ~[?:?]
at java.rmi.server.UnicastRemoteObject.exportObject(UnicastRemoteObject.java:470) ~[?:?]
at java.rmi.server.UnicastRemoteObject.exportObject(UnicastRemoteObject.java:381) ~[?:?]
at java.rmi.server.UnicastRemoteObject.<init>(UnicastRemoteObject.java:260) ~[?:?]
at org.apache.jmeter.samplers.RemoteSampleListenerImpl.<init>(RemoteSampleListenerImpl.java:43) ~[ApacheJMeter_core.jar:5.4.1]
at org.apache.jmeter.engine.ConvertListeners.addNode(ConvertListeners.java:77) ~[ApacheJMeter_core.jar:5.4.1]
at org.apache.jorphan.collections.HashTree.traverse(HashTree.java:975) ~[jorphan.jar:5.4.1]
at org.apache.jmeter.engine.ClientJMeterEngine.runTest(ClientJMeterEngine.java:136) ~[ApacheJMeter_core.jar:5.4.1]
at org.apache.jmeter.engine.DistributedRunner.start(DistributedRunner.java:134) ~[ApacheJMeter_core.jar:5.4.1]
at org.apache.jmeter.gui.action.RemoteStart.doAction(RemoteStart.java:80) ~[ApacheJMeter_core.jar:5.4.1]
at org.apache.jmeter.gui.action.ActionRouter.performAction(ActionRouter.java:87) ~[ApacheJMeter_core.jar:5.4.1]
at org.apache.jmeter.gui.action.ActionRouter.lambda$actionPerformed$0(ActionRouter.java:69) ~[ApacheJMeter_core.jar:5.4.1]
at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:313) [?:?]
at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:770) [?:?]
at java.awt.EventQueue$4.run(EventQueue.java:721) [?:?]
at java.awt.EventQueue$4.run(EventQueue.java:715) [?:?]
at java.security.AccessController.doPrivileged(Native Method) ~[?:?]
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:85) [?:?]
at java.awt.EventQueue.dispatchEvent(EventQueue.java:740) [?:?]
at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:203) [?:?]
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:124) [?:?]
at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:113) [?:?]
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:109) [?:?]
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101) [?:?]
at java.awt.EventDispatchThread.run(EventDispatchThread.java:90) [?:?]
Caused by: java.io.IOException: Could not bind to /172.xx.xxx.xx8 using port 5890
at org.apache.jmeter.rmi.SSLRMIServerSocketFactory.createServerSocket(SSLRMIServerSocketFactory.java:138) ~[ApacheJMeter_core.jar:5.4.1]
at sun.rmi.transport.tcp.TCPEndpoint.newServerSocket(TCPEndpoint.java:670) ~[?:?]
at sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:324) ~[?:?]
... 28 more
Caused by: java.net.BindException: Address already in use (Bind failed)
at java.net.PlainSocketImpl.socketBind(Native Method) ~[?:?]
at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:436) ~[?:?]
at java.net.ServerSocket.bind(ServerSocket.java:395) ~[?:?]
at java.net.ServerSocket.<init>(ServerSocket.java:257) ~[?:?]
at javax.net.ssl.SSLServerSocket.<init>(SSLServerSocket.java:181) ~[?:?]
at sun.security.ssl.SSLServerSocketImpl.<init>(SSLServerSocketImpl.java:78) ~[?:?]
at sun.security.ssl.SSLServerSocketFactoryImpl.createServerSocket(SSLServerSocketFactoryImpl.java:87) ~[?:?]
at org.apache.jmeter.rmi.SSLRMIServerSocketFactory.createServerSocket(SSLRMIServerSocketFactory.java:136) ~[ApacheJMeter_core.jar:5.4.1]
at sun.rmi.transport.tcp.TCPEndpoint.newServerSocket(TCPEndpoint.java:670) ~[?:?]
at sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:324) ~[?:?]
... 28 more
2021-07-09 09:57:30,556 INFO o.a.j.r.RmiUtils: Resolving by name the value of System property 'java.rmi.server.hostname': 172.xx.xxx.xx8
2021-07-09 09:57:30,556 INFO o.a.j.r.RmiUtils: Local IP address=172.xx.xxx.xx8
2021-07-09 09:57:30,751 ERROR o.a.j.e.ConvertListeners: Error replacing class org.apache.jmeter.threads.RemoteThreadsListenerTestElement by wrapper: class org.apache.jmeter.threads.RemoteThreadsListenerWrapper
java.rmi.server.ExportException: Listen failed on port: 5889; nested exception is:
java.io.IOException: Could not bind to /172.xx.xxx.xx8 using port 5889
at sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:337) ~[?:?]
at sun.rmi.transport.tcp.TCPTransport.exportObject(TCPTransport.java:243) ~[?:?]
at sun.rmi.transport.tcp.TCPEndpoint.exportObject(TCPEndpoint.java:412) ~[?:?]
at sun.rmi.transport.LiveRef.exportObject(LiveRef.java:147) ~[?:?]
at sun.rmi.server.UnicastServerRef.exportObject(UnicastServerRef.java:234) ~[?:?]
at java.rmi.server.UnicastRemoteObject.exportObject(UnicastRemoteObject.java:470) ~[?:?]
at java.rmi.server.UnicastRemoteObject.exportObject(UnicastRemoteObject.java:381) ~[?:?]
at java.rmi.server.UnicastRemoteObject.<init>(UnicastRemoteObject.java:260) ~[?:?]
at org.apache.jmeter.threads.RemoteThreadsListenerImpl.<init>(RemoteThreadsListenerImpl.java:58) ~[ApacheJMeter_core.jar:5.4.1]
at org.apache.jmeter.engine.ConvertListeners.addNode(ConvertListeners.java:63) ~[ApacheJMeter_core.jar:5.4.1]
at org.apache.jorphan.collections.HashTree.traverse(HashTree.java:975) ~[jorphan.jar:5.4.1]
at org.apache.jmeter.engine.ClientJMeterEngine.runTest(ClientJMeterEngine.java:136) ~[ApacheJMeter_core.jar:5.4.1]
at org.apache.jmeter.engine.DistributedRunner.start(DistributedRunner.java:134) ~[ApacheJMeter_core.jar:5.4.1]
at org.apache.jmeter.gui.action.RemoteStart.doAction(RemoteStart.java:80) ~[ApacheJMeter_core.jar:5.4.1]
at org.apache.jmeter.gui.action.ActionRouter.performAction(ActionRouter.java:87) ~[ApacheJMeter_core.jar:5.4.1]
at org.apache.jmeter.gui.action.ActionRouter.lambda$actionPerformed$0(ActionRouter.java:69) ~[ApacheJMeter_core.jar:5.4.1]
at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:313) [?:?]
at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:770) [?:?]
at java.awt.EventQueue$4.run(EventQueue.java:721) [?:?]
at java.awt.EventQueue$4.run(EventQueue.java:715) [?:?]
at java.security.AccessController.doPrivileged(Native Method) ~[?:?]
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:85) [?:?]
at java.awt.EventQueue.dispatchEvent(EventQueue.java:740) [?:?]
at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:203) [?:?]
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:124) [?:?]
at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:113) [?:?]
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:109) [?:?]
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101) [?:?]
at java.awt.EventDispatchThread.run(EventDispatchThread.java:90) [?:?]
Caused by: java.io.IOException: Could not bind to /172.xx.xxx.xx8 using port 5889
at org.apache.jmeter.rmi.SSLRMIServerSocketFactory.createServerSocket(SSLRMIServerSocketFactory.java:138) ~[ApacheJMeter_core.jar:5.4.1]
at sun.rmi.transport.tcp.TCPEndpoint.newServerSocket(TCPEndpoint.java:670) ~[?:?]
at sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:324) ~[?:?]
... 28 more
Caused by: java.net.BindException: Address already in use (Bind failed)
at java.net.PlainSocketImpl.socketBind(Native Method) ~[?:?]
at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:436) ~[?:?]
at java.net.ServerSocket.bind(ServerSocket.java:395) ~[?:?]
at java.net.ServerSocket.<init>(ServerSocket.java:257) ~[?:?]
at javax.net.ssl.SSLServerSocket.<init>(SSLServerSocket.java:181) ~[?:?]
at sun.security.ssl.SSLServerSocketImpl.<init>(SSLServerSocketImpl.java:78) ~[?:?]
at sun.security.ssl.SSLServerSocketFactoryImpl.createServerSocket(SSLServerSocketFactoryImpl.java:87) ~[?:?]
at org.apache.jmeter.rmi.SSLRMIServerSocketFactory.createServerSocket(SSLRMIServerSocketFactory.java:136) ~[ApacheJMeter_core.jar:5.4.1]
at sun.rmi.transport.tcp.TCPEndpoint.newServerSocket(TCPEndpoint.java:670) ~[?:?]
at sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:324) ~[?:?]
... 28 more
2021-07-09 09:57:30,770 ERROR o.a.j.e.ClientJMeterEngine: Error in rconfigure() method
java.rmi.MarshalException: error marshalling arguments; nested exception is:
java.io.NotSerializableException: org.apache.jmeter.threads.RemoteThreadsListenerTestElement
at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:160) ~[?:?]
at java.rmi.server.RemoteObjectInvocationHandler.invokeRemoteMethod(RemoteObjectInvocationHandler.java:217) ~[?:?]
at java.rmi.server.RemoteObjectInvocationHandler.invoke(RemoteObjectInvocationHandler.java:162) ~[?:?]
at com.sun.proxy.$Proxy24.rconfigure(Unknown Source) ~[?:?]
at org.apache.jmeter.engine.ClientJMeterEngine.runTest(ClientJMeterEngine.java:152) ~[ApacheJMeter_core.jar:5.4.1]
at org.apache.jmeter.engine.DistributedRunner.start(DistributedRunner.java:134) ~[ApacheJMeter_core.jar:5.4.1]
at org.apache.jmeter.gui.action.RemoteStart.doAction(RemoteStart.java:80) ~[ApacheJMeter_core.jar:5.4.1]
at org.apache.jmeter.gui.action.ActionRouter.performAction(ActionRouter.java:87) ~[ApacheJMeter_core.jar:5.4.1]
at org.apache.jmeter.gui.action.ActionRouter.lambda$actionPerformed$0(ActionRouter.java:69) ~[ApacheJMeter_core.jar:5.4.1]
at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:313) [?:?]
at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:770) [?:?]
at java.awt.EventQueue$4.run(EventQueue.java:721) [?:?]
at java.awt.EventQueue$4.run(EventQueue.java:715) [?:?]
at java.security.AccessController.doPrivileged(Native Method) ~[?:?]
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:85) [?:?]
at java.awt.EventQueue.dispatchEvent(EventQueue.java:740) [?:?]
at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:203) [?:?]
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:124) [?:?]
at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:113) [?:?]
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:109) [?:?]
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101) [?:?]
at java.awt.EventDispatchThread.run(EventDispatchThread.java:90) [?:?]
Caused by: java.io.NotSerializableException: org.apache.jmeter.threads.RemoteThreadsListenerTestElement
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1185) ~[?:?]
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:349) ~[?:?]
at java.util.HashMap.internalWriteEntries(HashMap.java:1839) ~[?:?]
at java.util.HashMap.writeObject(HashMap.java:1411) ~[?:?]
at jdk.internal.reflect.GeneratedMethodAccessor18.invoke(Unknown Source) ~[?:?]
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:1145) ~[?:?]
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1497) ~[?:?]
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1433) ~[?:?]
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1179) ~[?:?]
at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1553) ~[?:?]
at java.io.ObjectOutputStream.defaultWriteObject(ObjectOutputStream.java:442) ~[?:?]
at org.apache.jorphan.collections.HashTree.writeObject(HashTree.java:951) ~[jorphan.jar:5.4.1]
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:1145) ~[?:?]
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1497) ~[?:?]
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1433) ~[?:?]
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1179) ~[?:?]
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:349) ~[?:?]
at java.util.HashMap.internalWriteEntries(HashMap.java:1840) ~[?:?]
at java.util.HashMap.writeObject(HashMap.java:1411) ~[?:?]
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:1145) ~[?:?]
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1497) ~[?:?]
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1433) ~[?:?]
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1179) ~[?:?]
at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1553) ~[?:?]
at java.io.ObjectOutputStream.defaultWriteObject(ObjectOutputStream.java:442) ~[?:?]
at org.apache.jorphan.collections.HashTree.writeObject(HashTree.java:951) ~[jorphan.jar:5.4.1]
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:1145) ~[?:?]
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1497) ~[?:?]
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1433) ~[?:?]
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1179) ~[?:?]
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:349) ~[?:?]
at sun.rmi.server.UnicastRef.marshalValue(UnicastRef.java:293) ~[?:?]
at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:155) ~[?:?]
... 21 more
2021-07-09 09:57:30,773 INFO o.a.j.e.ClientJMeterEngine: Interrupting RMI Reaper
2021-07-09 09:57:30,773 ERROR o.a.j.u.JMeterUtils: Error in rconfigure() method java.rmi.MarshalException: error marshalling arguments; nested exception is:
java.io.NotSerializableException: org.apache.jmeter.threads.RemoteThreadsListenerTestElement
2021-07-09 09:57:33,605 INFO o.a.j.e.DistributedRunner: Remote engines have been started:[]
2021-07-09 09:57:33,606 ERROR o.a.j.e.DistributedRunner: The following remote engines have not started:[172.xx.xxx.xx1]
If anyone knows what I am doing wrong or need to change, it would be much appreciated.
If you want to override the client.rmi.localport setting make sure that it is the same on master and all the slaves
So my expectation is that if you add the next line:
client.rmi.localport=5888
to all the slaves user.properties file you setup should start working normally.
More information:
Remote hosts and RMI configuration
JMeter Distributed Testing with Docker
Apache JMeter Distributed Testing Step-by-step
I am trying to call SFMC Cloud endpoint and I need to connect through TLS. I created a trust store that contains 2 certificate one for smfc token authentication and another for the actual soap endpoint. I check the truststore and both certificates are already imported. However when I tried to deploy my application, it throws this error :
org.mule.soap.api.transport.DispatcherException: An error occured while sending the SOAP request
at com.mulesoft.connector.salesforce.marketing.internal.connection.provider.SalesforceMarketingTransportDispatcher.dispatch (SaleforceMarketingTransportDispatcher.java:58) ~[mule-sfdc-marketing-cloud-connector-3.1.6-mule-pluigin.jar:?]
at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:308) ~[cxf-core-3.3.5.jar:3.3.5]
at org.apache.cxf.endpoint.ClientImpl.doInvoker(ClientImpl.java:530) ~[cxf-core-3.3.5]
at org.apache.cxf.endpoint.ClientImpl.invoker(ClientImpl.java:441) ~[cxf-core-3.3.5]
at org.mule.soap.internal.client.AbstractSoapCxfClient.invoke(AbstractSoapCxfClient.java:116) ~[mule-soap-engine-1.3.2.jar:?]
at org.mule.soap.internal.client.AbstractSoapCxfClient.consume(AbstractSoapCxfClient.java:102) ~[mule-soap-engine-1.3.2.jar:?]
at com.mulesoft.connector.salesforce.marketing.internal.connection.SalesforceMarketingCloudConnection.execute(SalesforceMarketingCloudConnection.java:62) ~[mule-sfdc-marketing-cloud-connector-3.1.6-mule-plugin.jar:?]
at com.mulesoft.connector.salesforce.marketing.internal.connection.SalesforceMarketingCloudConnection.execute(SalesforceMarketingCloudConnection.java:56) ~[mule-sfdc-marketing-cloud-connector-3.1.6-mule-plugin.jar:?]
at com.mulesoft.connector.salesforce.marketing.internal.connection.SalesforceMarketingCloudConnection.validate(SalesforceMarketingCloudConnection.java:50) ~[mule-sfdc-marketing-cloud-connector-3.1.6-mule-plugin.jar:?]
at com.mulesoft.connectors.commons.template.connection.provider.ConnectorConnectionProvider.validate(ConnectorConnectionProvider.java:36) ~[mule-connector-commons-2.1.0.jar:?]
at com.mulesoft.connectors.commons.template.connection.provider.ConnectorConnectionProvider.validate(ConnectorConnectionProvider.java:16) ~[mule-connector-commons-2.1.0.jar:?]
at org.mule.runtime.module.extension.internal.runtime.connectivity.oauth.ExtensionOAuthUtils.validateOAuthConnection(ExtensionOAuthUtils.java:188) ~[mule-module-extensions-support-4.3.0-20201013:4.3.0-20201013]
at org.mule.runtime.module.extension.internal.runtime.connectivity.oauth.BaseOAuthConnectionProviderWrapper.validate(BaseOAuthConnectionProviderWrapper.java:42) ~[mule-module-extensions-support-4.3.0-20201013:4.3.0-20201013]
at org.mule.runtime.core.internal.connection.DefaultConnectionManager.doTestConnectivity(DefaultConnectionManager.java:186) ~[mule-core-4.3.0-20201013.jar:4.3.0-20201013]
at org.mule.runtime.core.internal.connection.DefaultConnectionManager.lambda$testConnectivity$1(DefaultConnectionManager.java:169) ~[mule-core-4.3.0-20201013.jar:4.3.0-20201013]
at org.mule.runtime.core.internal.connection.DefaultConnectionManager.doTestConnectivity(DefaultConnectionManager.java:176) [mule-core-4.3.0-20201013.jar:4.3.0-20201013]
at org.mule.runtime.core.internal.connection.DefaultConnectionManager.testConnectivity(DefaultConnectionManager.java:152) [mule-core-4.3.0-20201013.jar:4.3.0-20201013]
at org.mule.runtime.core.internal.connection.DelegateConnectionManagerAdapter$EagerConnectionManagerAdapter.testConnectivity(DelegateConnectionManagerAdapter.java:176) [mule-core-4.3.0-20201013.jar:4.3.0-20201013]
at org.mule.runtime.core.internal.connection.DelegateConnectionManagerAdapter.testConnectivity(DelegateConnectionManagerAdapter.java:98) [mule-core-4.3.0-20201013.jar:4.3.0-20201013]
at org.mule.runtime.core.internal.connection.CompositeConnectionManager.testConnectivity(CompositeConnectionManager.java:115) [mule-core-4.3.0-20201013.jar:4.3.0-20201013]
at org.mule.runtime.module..extension.internal.runtime.config.LifecycleAwareConfigurationInstance$1.doWork(LifecycleAwareConfigurationInstance.java:194) [mule-module-extensions-support-4.3.0-20201013.jar:4.3.0-20201013]
at org.mule.runtime.core.api.retry.policy.AbstractPolicyTemplate.execure(AbstractPolicyTemplate.java:52) [mule-core-4.3.0-20201013.jar:4.3.0-20201013]
at org.mule.runtime.core.internal.rety.async.RetryWorker.run(RetryWorker.java:56) [mule-core-4.3.0-20201013.jar:4.3.0-20201013]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
at java.util.concurrent.FutureTask.run(FutureTasl.java:264) [?:?]
at org.mule.service..scheduler.internal.AbstractRunnableFutureDecorator.doRun(AbstractRunnableFutureDecorator.java:111) [mule-service-scheduler-1.3.2/:?]
at org.mule.service..scheduler.internal.RunnableFutureDecorator.doRun(RunnableFutureDecorator.java:54) [mule-service-scheduler-1.3.2/:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:835) [?:?]
Caused by: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.ssl.Alert.createSSLException(Alert.java:131) ~[?:?]
at sun.security.ssl.TransportContext.fatal(TransportContext.java:320) ~[?:?]
at sun.security.ssl.TransportContext.fatal(TransportContext.java:263) ~[?:?]
at sun.security.ssl.TransportContext.fatal(TransportContext.java:258) ~[?:?]
at sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:641) ~[?:?]
at sun.security.ssl.CertificateMessage$T12CertificateConsumer.onCertificate(CertificateMessage360) ~[?:?]
at sun.security.ssl.SSLHandshake.consume(SSLHandshaker.java:392) ~[?:?]
at sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:443) ~[?:?]
at sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedActyion.run(SSLEngineImpl.java:1074) ~[?:?]
at sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedActyion.run(SSLEngineImpl.java:1061) ~[?:?]
at java.security.AccessController.doPrivileged(AccessController.java:689) ~[?:?]
at sun.security.ssl.SSLEngineImpl$DelegatedTask.run(SSLEngineImpl.java:1008) ~[?:?]
at org.glashfish.grizzly.ssl.SSLUtils.executeDelegatedTask(SSLUtils.java:274) ~[?:?]
at org.glashfish.grizzly.ssl.SSLBaseFilter.doHandshakeStep(SSLBaseFilter.java:723) ~[?:?]
at org.glashfish.grizzly.ssl.SSLFilter.doHandshakeStep(SSLFilter.java:345) ~[?:?]
at org.glashfish.grizzly.ssl.SSLBaseFilter.doHandshakeStep(SSLBaseFilter.java:637) ~[?:?]
at org.glashfish.grizzly.ssl.SSLBaseFilter.handleRead(SSLBaseFilter.java:349) ~[?:?]
at com.ning.http.client.providers.grizzly.SwitchingSSLFilter.handleRead(SwitchingSSLFilter.java:74) ~[?:?]
at org.glashfish.grizzly.filterchain.ExcecutorResolver$9.executer(ExcecutorResolver.java:119) ~[?:?]
at org.glashfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:284) ~[?:?]
at org.glashfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:201) ~[?:?]
at org.glashfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:133) ~[?:?]
at org.glashfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:112) ~[?:?]
at org.glashfish.grizzly.ProcessExecutor.executer(ProcessExecutor.java:77) ~[?:?]
at org.glashfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:540) ~[?:?]
at org.glashfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:112) ~[?:?]
at org.glashfish.grizzly.strategies.SameThreadIOStrategy.executeIoEvent(SameThreadIOStrategy.java:103) ~[?:?]
at org.glashfish.grizzly.strategies.AbstractIOStrategy.executeIoEvent(AbstractIOStrategy.java:89) ~[?:?]
at org.glashfish.grizzly.nio.SelectorRunner.iterateKeyEvents(SelectorRunner.java:415) ~[?:?]
at org.glashfish.grizzly.nio.SelectorRunner.iterateKeys(SelectorRunner.java:384) ~[?:?]
at org.glashfish.grizzly.nio.SelectorRunner.doSelect(SelectorRunner.java:348) ~[?:?]
at org.glashfish.grizzly.nio.SelectorRunner.run(SelectorRunner.java:279) ~[?:?]
... 7 more
I verified that the certificates are in the truststore using this command :
keytool -list -v -keystore truststore.jks
Here is the configuration for the salesforce marketing cloud :
<sfdc-marketing-cloud:config name="Salesforce_Marketing_Cloud_Config" doc:name="Salesforce Marketing Cloud Config" doc:id="ceb6487e-d6ac-4aee-b7b3-d292d9657b3d" >
<sfdc-marketing-cloud:oauth-client-credentials-connection serviceUrl="${api.sfmc.url}" >
<tls:context >
<tls:trust-store path="keystore.jks" password="test123" type="jks" />
</tls:context>
<sfdc-marketing-cloud:oauth-client-credentials clientId="${api.sfmc.client.id}" clientSecret="${api.sfmc.client.secret}" tokenUrl="${api.sfmc.token.url}" />
</sfdc-marketing-cloud:oauth-client-credentials-connection>
</sfdc-marketing-cloud:config>
I'm using this app runtime :
<properties>
<app.runtime>4.3.0-20201013</app.runtime>
<munit.version>2.3.1</munit.version>
<mule.maven.plugin.version>3.4.2</mule.maven.plugin.version>
</properties>
SFMC connector version :
<dependency>
<groupId>com.mulesoft.connectors</groupId>
<artifactId>mule-sfdc-marketing-cloud-connector</artifactId>
<version>3.1.4</version>
<classifier>mule-plugin</classifier>
</dependency>
Did I missed something? Thanks!
I am going through similar(http://activemq.2283324.n4.nabble.com/Sporadic-quot-Error-while-attempting-to-add-new-Connection-to-the-pool-nested-exception-is-javax-jms-td4741737.html) issues right now.
ActiveMQ version: 5.15.4
Environment: Windows
Karaf Version: 4.2.0
I am using ActiveMQ and Apache Karaf with camel and pax-jms. Multiple karaf
instances are using this one broker instance. My Karaf instances are
installed as a service, and so is ActiveMQ as well.
This is how my config looks like:
name=db-consumer
jms.url=tcp://localhost:61616
jms.username=<user>
jms.password=<password>
type=activemq
osgi.jndi.service.name=jms/consumer
Time and again, with high frequency of data, I am getting the following
error:
Uncategorized exception occurred during JMS processing; nested
exception is javax.jms.JMSException: Could not connect to broker URL:
tcp://localhost:61616. Reason: java.net.BindException: Address already
in use: connect
Do I need to configure something specific in the broker or in the
configuration to resolve this issue?
I tried pool=transx, but it starves for transx.
I saw that, there is a related post (ActiveMQ gives: "Could not connect to broker URL: tcp://localhost:61616" after around 10 000 messages) on stackoverflow, but this seems to be a problem in spring. I am looking for solution in osgi with pax-jms.
--Update 1: Added error dump--
org.springframework.jms.UncategorizedJmsException: Uncategorized exception occurred during JMS processing; nested exception is javax.jms.JMSException: Could not connect to broker URL: tcp://localhost:61616. Reason: java.net.BindException: Address already in use: connect
at org.springframework.jms.support.JmsUtils.convertJmsAccessException(JmsUtils.java:316) ~[115:org.apache.servicemix.bundles.spring-jms:4.3.14.RELEASE_1]
at org.springframework.jms.support.JmsAccessor.convertJmsAccessException(JmsAccessor.java:169) ~[115:org.apache.servicemix.bundles.spring-jms:4.3.14.RELEASE_1]
at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:487) ~[115:org.apache.servicemix.bundles.spring-jms:4.3.14.RELEASE_1]
at org.apache.camel.component.jms.JmsConfiguration$CamelJmsTemplate.send(JmsConfiguration.java:516) ~[52:org.apache.camel.camel-jms:2.20.3]
at org.apache.camel.component.jms.JmsProducer.doSend(JmsProducer.java:440) ~[52:org.apache.camel.camel-jms:2.20.3]
at org.apache.camel.component.jms.JmsProducer.processInOnly(JmsProducer.java:394) ~[52:org.apache.camel.camel-jms:2.20.3]
at org.apache.camel.component.jms.JmsProducer.process(JmsProducer.java:157) ~[52:org.apache.camel.camel-jms:2.20.3]
at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:148) ~[51:org.apache.camel.camel-core:2.20.3]
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201) ~[51:org.apache.camel.camel-core:2.20.3]
at org.apache.camel.processor.Pipeline.process(Pipeline.java:138) ~[51:org.apache.camel.camel-core:2.20.3]
at org.apache.camel.processor.Pipeline.process(Pipeline.java:101) [51:org.apache.camel.camel-core:2.20.3]
at org.apache.camel.processor.FatalFallbackErrorHandler.process(FatalFallbackErrorHandler.java:82) [51:org.apache.camel.camel-core:2.20.3]
at org.apache.camel.processor.RedeliveryErrorHandler.deliverToFailureProcessor(RedeliveryErrorHandler.java:1063) [51:org.apache.camel.camel-core:2.20.3]
at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:474) [51:org.apache.camel.camel-core:2.20.3]
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201) [51:org.apache.camel.camel-core:2.20.3]
at org.apache.camel.processor.Pipeline.process(Pipeline.java:138) [51:org.apache.camel.camel-core:2.20.3]
at org.apache.camel.processor.Pipeline.process(Pipeline.java:101) [51:org.apache.camel.camel-core:2.20.3]
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201) [51:org.apache.camel.camel-core:2.20.3]
at org.apache.camel.component.direct.DirectProducer.process(DirectProducer.java:62) [51:org.apache.camel.camel-core:2.20.3]
at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:148) [51:org.apache.camel.camel-core:2.20.3]
at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548) [51:org.apache.camel.camel-core:2.20.3]
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201) [51:org.apache.camel.camel-core:2.20.3]
at org.apache.camel.processor.ChoiceProcessor.process(ChoiceProcessor.java:117) [51:org.apache.camel.camel-core:2.20.3]
at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548) [51:org.apache.camel.camel-core:2.20.3]
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201) [51:org.apache.camel.camel-core:2.20.3]
at org.apache.camel.processor.Pipeline.process(Pipeline.java:138) [51:org.apache.camel.camel-core:2.20.3]
at org.apache.camel.processor.Pipeline.process(Pipeline.java:101) [51:org.apache.camel.camel-core:2.20.3]
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201) [51:org.apache.camel.camel-core:2.20.3]
at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:97) [51:org.apache.camel.camel-core:2.20.3]
at org.apache.camel.component.jms.EndpointMessageListener.onMessage(EndpointMessageListener.java:112) [52:org.apache.camel.camel-jms:2.20.3]
at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:719) [115:org.apache.servicemix.bundles.spring-jms:4.3.14.RELEASE_1]
at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:679) [115:org.apache.servicemix.bundles.spring-jms:4.3.14.RELEASE_1]
at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:649) [115:org.apache.servicemix.bundles.spring-jms:4.3.14.RELEASE_1]
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:317) [115:org.apache.servicemix.bundles.spring-jms:4.3.14.RELEASE_1]
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:255) [115:org.apache.servicemix.bundles.spring-jms:4.3.14.RELEASE_1]
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1166) [115:org.apache.servicemix.bundles.spring-jms:4.3.14.RELEASE_1]
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1158) [115:org.apache.servicemix.bundles.spring-jms:4.3.14.RELEASE_1]
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1055) [115:org.apache.servicemix.bundles.spring-jms:4.3.14.RELEASE_1]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:?]
at java.lang.Thread.run(Thread.java:748) [?:?]
Caused by: javax.jms.JMSException: Could not connect to broker URL: tcp://localhost:61616. Reason: java.net.BindException: Address already in use: connect
at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:36) ~[?:?]
at org.apache.activemq.ActiveMQConnectionFactory.createActiveMQConnection(ActiveMQConnectionFactory.java:374) ~[?:?]
at org.apache.activemq.ActiveMQConnectionFactory.createActiveMQConnection(ActiveMQConnectionFactory.java:304) ~[?:?]
at org.apache.activemq.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:244) ~[?:?]
at org.ops4j.pax.jms.activemq.ActiveMQConnectionFactoryFactory$1.createConnection(ActiveMQConnectionFactoryFactory.java:52) ~[?:?]
at Proxy907e4baf_5e06_4932_b3c9_aebc6215955c.createConnection(Unknown Source) ~[?:?]
at org.springframework.jms.support.JmsAccessor.createConnection(JmsAccessor.java:180) ~[?:?]
at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:474) ~[?:?]
... 38 more
Caused by: java.net.BindException: Address already in use: connect
at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method) ~[?:?]
at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:85) ~[?:?]
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) ~[?:?]
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) ~[?:?]
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) ~[?:?]
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172) ~[?:?]
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[?:?]
at java.net.Socket.connect(Socket.java:589) ~[?:?]
at org.apache.activemq.transport.tcp.TcpTransport.connect(TcpTransport.java:525) ~[?:?]
at org.apache.activemq.transport.tcp.TcpTransport.doStart(TcpTransport.java:488) ~[?:?]
at org.apache.activemq.util.ServiceSupport.start(ServiceSupport.java:55) ~[?:?]
at org.apache.activemq.transport.AbstractInactivityMonitor.start(AbstractInactivityMonitor.java:169) ~[?:?]
at org.apache.activemq.transport.InactivityMonitor.start(InactivityMonitor.java:52) ~[?:?]
at org.apache.activemq.transport.TransportFilter.start(TransportFilter.java:64) ~[?:?]
at org.apache.activemq.transport.WireFormatNegotiator.start(WireFormatNegotiator.java:72) ~[?:?]
at org.apache.activemq.transport.TransportFilter.start(TransportFilter.java:64) ~[?:?]
at org.apache.activemq.transport.TransportFilter.start(TransportFilter.java:64) ~[?:?]
at org.apache.activemq.ActiveMQConnectionFactory.createActiveMQConnection(ActiveMQConnectionFactory.java:354) ~[?:?]
at org.apache.activemq.ActiveMQConnectionFactory.createActiveMQConnection(ActiveMQConnectionFactory.java:304) ~[?:?]
at org.apache.activemq.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:244) ~[?:?]
at org.ops4j.pax.jms.activemq.ActiveMQConnectionFactoryFactory$1.createConnection(ActiveMQConnectionFactoryFactory.java:52) ~[?:?]
at Proxy907e4baf_5e06_4932_b3c9_aebc6215955c.createConnection(Unknown Source) ~[?:?]
at org.springframework.jms.support.JmsAccessor.createConnection(JmsAccessor.java:180) ~[?:?]
at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:474) ~[?:?]
... 38 more
Update:
To people who might encounter this issue, please remember to use connection pooling on the client side. I was using Camel Bundles. And, switching to connection pooling might not work on the fly. Restart the broker and restart the instances to see the effect. I guess, there is some kind of connection caching going on, because of which I was not able to see the changes immediately.
The pax-jms version of connection pooling config looks like:
name=db-consumer
jms.url=tcp://localhost:61616
jms.username=<user>
jms.password=<password>
type=activemq
pool=pooledjms
osgi.jndi.service.name=jms/consumer
I could try transx pool as well, but pooledjms was the easiest one and it works for my requirements.
I am trying to implement clustering in Mulesoft. My intention is to deploy an application to the cluster of application servers.
I have installed a Mule server each in two physical machines which I have binded to a cluster.
Then I created a Mule application which listens to HTTP endpoint to trigger a mail.
I have problem in deploying application to the created Mule cluster through MMC.
Click to see the cluster of servers
The application gets deployed to one of the servers in the cluster and the problem exists in deployment to the other server.
The error I am getting is:
*ERROR 2016-10-21 21:52:09,283 [qtp1451192493-104] org.mule.module.launcher.DefaultArchiveDeployer:
org.mule.module.launcher.DeploymentStartException: BindException: Address already in use: bind
at org.mule.module.launcher.application.DefaultMuleApplication.start(DefaultMuleApplication.java:156) ~[?:?]
at org.mule.module.launcher.artifact.ArtifactWrapper$4.execute(ArtifactWrapper.java:98) ~[?:?]
at org.mule.module.launcher.artifact.ArtifactWrapper.executeWithinArtifactClassLoader(ArtifactWrapper.java:129) ~[?:?]
at org.mule.module.launcher.artifact.ArtifactWrapper.start(ArtifactWrapper.java:93) ~[?:?]
at org.mule.module.launcher.DefaultArtifactDeployer.deploy(DefaultArtifactDeployer.java:26) ~[?:?]
at org.mule.module.launcher.DefaultArchiveDeployer.guardedDeploy(DefaultArchiveDeployer.java:310) ~[?:?]
at org.mule.module.launcher.DefaultArchiveDeployer.deployArtifact(DefaultArchiveDeployer.java:330) ~[?:?]
at org.mule.module.launcher.DefaultArchiveDeployer.deployPackagedArtifact(DefaultArchiveDeployer.java:155) ~[?:?]
at org.mule.module.launcher.MuleDeploymentService.deploy(MuleDeploymentService.java:192) ~[?:?]
at com.mulesoft.mmc.agent.service.impl.ApplicationServiceImpl.deploy(ApplicationServiceImpl.java:302) ~[?:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_77]
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[?:1.8.0_77]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[?:1.8.0_77]
at java.lang.reflect.Method.invoke(Unknown Source) ~[?:1.8.0_77]
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317) ~[?:?]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:201) ~[?:?]
at com.sun.proxy.$Proxy45.deploy(Unknown Source) ~[?:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_77]
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[?:1.8.0_77]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[?:1.8.0_77]
at java.lang.reflect.Method.invoke(Unknown Source) ~[?:1.8.0_77]
at org.springframework.remoting.support.RemoteInvocation.invoke(RemoteInvocation.java:212) ~[?:?]
at com.mulesoft.common.remoting.RemoteInvocationDecorator.invoke(RemoteInvocationDecorator.java:107) ~[?:?]
at org.springframework.remoting.support.DefaultRemoteInvocationExecutor.invoke(DefaultRemoteInvocationExecutor.java:39) ~[?:?]
at org.springframework.remoting.support.RemoteInvocationBasedExporter.invoke(RemoteInvocationBasedExporter.java:78) ~[?:?]
at com.mulesoft.common.remoting.StreamSupportingHttpInvokerServiceExporter.invokeAndCreateResult(StreamSupportingHttpInvokerServiceExporter.java:122) ~[?:?]
at org.springframework.remoting.httpinvoker.HttpInvokerServiceExporter.handleRequest(HttpInvokerServiceExporter.java:74) ~[?:?]
at com.mulesoft.mmc.agent.remoting.HttpInvokerServiceExporter.handleRequest(HttpInvokerServiceExporter.java:50) ~[?:?]
at org.springframework.web.servlet.mvc.HttpRequestHandlerAdapter.handle(HttpRequestHandlerAdapter.java:51) ~[?:?]
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:959) ~[?:?]
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:893) ~[?:?]
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:966) ~[?:?]
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:868) ~[?:?]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:595) ~[?:?]
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:842) ~[?:?]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:668) ~[?:?]
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:696) ~[?:?]
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1515) ~[?:?]
at com.mulesoft.mmc.agent.web.ConsoleClientCertFilter.doFilter(ConsoleClientCertFilter.java:138) ~[?:?]
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1495) ~[?:?]
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:519) ~[?:?]
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:138) ~[?:?]
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:564) ~[?:?]
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:213) ~[?:?]
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1097) ~[?:?]
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:448) ~[?:?]
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:175) ~[?:?]
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1031) ~[?:?]
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:136) ~[?:?]
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) ~[?:?]
at org.eclipse.jetty.server.Server.handle(Server.java:446) ~[?:?]
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:271) ~[?:?]
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:246) ~[?:?]
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.run(AbstractConnection.java:358) ~[?:?]
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:601) ~[?:?]
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:532) ~[?:?]
at java.lang.Thread.run(Unknown Source) [?:1.8.0_77]
Caused by: org.mule.api.lifecycle.LifecycleException: Failed to invoke lifecycle phase "start" on object: org.mule.module.http.internal.listener.DefaultHttpListenerConfig#30206033
at org.mule.lifecycle.phases.DefaultLifecyclePhase.applyLifecycle(DefaultLifecyclePhase.java:248) ~[?:?]
at org.mule.lifecycle.RegistryLifecycleCallback.doApplyLifecycle(RegistryLifecycleCallback.java:91) ~[?:?]
at org.mule.lifecycle.RegistryLifecycleCallback.onTransition(RegistryLifecycleCallback.java:67) ~[?:?]
at org.mule.lifecycle.RegistryLifecycleManager.invokePhase(RegistryLifecycleManager.java:140) ~[?:?]
at org.mule.lifecycle.RegistryLifecycleManager.fireLifecycle(RegistryLifecycleManager.java:111) ~[?:?]
at org.mule.registry.AbstractRegistryBroker.fireLifecycle(AbstractRegistryBroker.java:88) ~[?:?]
at org.mule.registry.MuleRegistryHelper.fireLifecycle(MuleRegistryHelper.java:141) ~[?:?]
at org.mule.lifecycle.MuleContextLifecycleManager$MuleContextLifecycleCallback.onTransition(MuleContextLifecycleManager.java:91) ~[?:?]
at org.mule.lifecycle.MuleContextLifecycleManager$MuleContextLifecycleCallback.onTransition(MuleContextLifecycleManager.java:87) ~[?:?]
at org.mule.lifecycle.MuleContextLifecycleManager.invokePhase(MuleContextLifecycleManager.java:69) ~[?:?]
at org.mule.lifecycle.MuleContextLifecycleManager.fireLifecycle(MuleContextLifecycleManager.java:61) ~[?:?]
at org.mule.DefaultMuleContext.start(DefaultMuleContext.java:294) ~[?:?]
at org.mule.module.launcher.application.DefaultMuleApplication.start(DefaultMuleApplication.java:134) ~[?:?]
... 56 more
Caused by: org.mule.api.DefaultMuleException: Address already in use: bind (java.net.BindException)
at org.mule.module.http.internal.listener.DefaultHttpListenerConfig.start(DefaultHttpListenerConfig.java:275) ~[?:?]
at sun.reflect.GeneratedMethodAccessor22.invoke(Unknown Source) ~[?:?]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[?:1.8.0_77]
at java.lang.reflect.Method.invoke(Unknown Source) ~[?:1.8.0_77]
at org.mule.lifecycle.phases.DefaultLifecyclePhase.applyLifecycle(DefaultLifecyclePhase.java:237) ~[?:?]
at org.mule.lifecycle.RegistryLifecycleCallback.doApplyLifecycle(RegistryLifecycleCallback.java:91) ~[?:?]
at org.mule.lifecycle.RegistryLifecycleCallback.onTransition(RegistryLifecycleCallback.java:67) ~[?:?]
at org.mule.lifecycle.RegistryLifecycleManager.invokePhase(RegistryLifecycleManager.java:140) ~[?:?]
at org.mule.lifecycle.RegistryLifecycleManager.fireLifecycle(RegistryLifecycleManager.java:111) ~[?:?]
at org.mule.registry.AbstractRegistryBroker.fireLifecycle(AbstractRegistryBroker.java:88) ~[?:?]
at org.mule.registry.MuleRegistryHelper.fireLifecycle(MuleRegistryHelper.java:141) ~[?:?]
at org.mule.lifecycle.MuleContextLifecycleManager$MuleContextLifecycleCallback.onTransition(MuleContextLifecycleManager.java:91) ~[?:?]
at org.mule.lifecycle.MuleContextLifecycleManager$MuleContextLifecycleCallback.onTransition(MuleContextLifecycleManager.java:87) ~[?:?]
at org.mule.lifecycle.MuleContextLifecycleManager.invokePhase(MuleContextLifecycleManager.java:69) ~[?:?]
at org.mule.lifecycle.MuleContextLifecycleManager.fireLifecycle(MuleContextLifecycleManager.java:61) ~[?:?]
at org.mule.DefaultMuleContext.start(DefaultMuleContext.java:294) ~[?:?]
at org.mule.module.launcher.application.DefaultMuleApplication.start(DefaultMuleApplication.java:134) ~[?:?]
... 56 more
Caused by: java.net.BindException: Address already in use: bind
at sun.nio.ch.Net.bind0(Native Method) ~[?:1.8.0_77]
at sun.nio.ch.Net.bind(Unknown Source) ~[?:1.8.0_77]
at sun.nio.ch.Net.bind(Unknown Source) ~[?:1.8.0_77]
at sun.nio.ch.ServerSocketChannelImpl.bind(Unknown Source) ~[?:1.8.0_77]
at sun.nio.ch.ServerSocketAdaptor.bind(Unknown Source) ~[?:1.8.0_77]
at org.glassfish.grizzly.nio.transport.TCPNIOBindingHandler.bindToChannelAndAddress(TCPNIOBindingHandler.java:131) ~[?:?]
at org.glassfish.grizzly.nio.transport.TCPNIOBindingHandler.bind(TCPNIOBindingHandler.java:88) ~[?:?]
at org.glassfish.grizzly.nio.transport.TCPNIOTransport.bind(TCPNIOTransport.java:248) ~[?:?]
at org.glassfish.grizzly.nio.transport.TCPNIOTransport.bind(TCPNIOTransport.java:228) ~[?:?]
at org.glassfish.grizzly.nio.transport.TCPNIOTransport.bind(TCPNIOTransport.java:219) ~[?:?]
at org.mule.module.http.internal.listener.grizzly.GrizzlyServer.start(GrizzlyServer.java:41) ~[?:?]
at org.mule.module.http.internal.listener.DefaultHttpListenerConfig.start(DefaultHttpListenerConfig.java:271) ~[?:?]
at sun.reflect.GeneratedMethodAccessor22.invoke(Unknown Source) ~[?:?]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[?:1.8.0_77]
at java.lang.reflect.Method.invoke(Unknown Source) ~[?:1.8.0_77]
at org.mule.lifecycle.phases.DefaultLifecyclePhase.applyLifecycle(DefaultLifecyclePhase.java:237) ~[?:?]
at org.mule.lifecycle.RegistryLifecycleCallback.doApplyLifecycle(RegistryLifecycleCallback.java:91) ~[?:?]
at org.mule.lifecycle.RegistryLifecycleCallback.onTransition(RegistryLifecycleCallback.java:67) ~[?:?]
at org.mule.lifecycle.RegistryLifecycleManager.invokePhase(RegistryLifecycleManager.java:140) ~[?:?]
at org.mule.lifecycle.RegistryLifecycleManager.fireLifecycle(RegistryLifecycleManager.java:111) ~[?:?]
at org.mule.registry.AbstractRegistryBroker.fireLifecycle(AbstractRegistryBroker.java:88) ~[?:?]
at org.mule.registry.MuleRegistryHelper.fireLifecycle(MuleRegistryHelper.java:141) ~[?:?]
at org.mule.lifecycle.MuleContextLifecycleManager$MuleContextLifecycleCallback.onTransition(MuleContextLifecycleManager.java:91) ~[?:?]
at org.mule.lifecycle.MuleContextLifecycleManager$MuleContextLifecycleCallback.onTransition(MuleContextLifecycleManager.java:87) ~[?:?]
at org.mule.lifecycle.MuleContextLifecycleManager.invokePhase(MuleContextLifecycleManager.java:69) ~[?:?]
at org.mule.lifecycle.MuleContextLifecycleManager.fireLifecycle(MuleContextLifecycleManager.java:61) ~[?:?]
at org.mule.DefaultMuleContext.start(DefaultMuleContext.java:294) ~[?:?]
at org.mule.module.launcher.application.DefaultMuleApplication.start(DefaultMuleApplication.java:134) ~[?:?]
... 56 more
Is there any specific configuration change to be done for clustering deployment?
Anyone please help me out.
Also suggestions related to BAM are most welcome
As the log says your HTTP Listener is trying to use a port that is already used
Caused by: java.net.BindException: Address already in use: bind
Use an OS tool to find out which process is using that port.
As you are deploying your application through Mule Management Console to frame a cluster.You should expose the application endpoint of HTTP inbound endpoint as "localhost:portnumber".Here in your scenario as both the servers are in same device.
TWO MULE SERVER INSTANCE IN A SINGLE MACHINE
The endpoint of serverA will be depicted as machineAIP:8080 and serverB will be also as machineIPA:8080.Here it arises the binding exception
TWO MULE SERVER INSTANCE IN TWO DIFFERENT MACHINES
The endpoint of serverA will be depicted as machineAIP:8080 and serverB will be also as machineIPB:8080.Here it differentiates with the help of machine IP.
Hence ,It is always good to go with two different servers.If you are creating two mule server instance with the same machine would result in issues related to exposing same endpoints.We need to use two different machines to enhance the high availability.It also ensures that crashing of the machine won't affect the functioning of clustered server.
Deploy the application using two different machines (Each machine has one mule server instance with it) and also the port number you are specifying in inbound endpoints are not in use.
If my answer gives you a solution. Happy coding :)
Based on the exception -
Caused by: java.net.BindException: Address already in use: bind
This could cause because of two reason -
the port used for the mule application through HTTP connector is already been used at the server level - Hence try to change the port and make sure that port is not in use.
If you are deploying an mule application using MMC in a cluster environment then you have to configure your HTTP connector with either localhost OR 0.0.0.0 port so that it will take the localhost ref while deploying in the server/nodes under the cluster.
Note, when you use localhost OR 0.0.0.0 then the apps will be deployed in the nodes and your load balancer will point to the node address (host dns) and then it will rediect the call to both nodes with the port you mentioned in the http connector.
Please try and let me know if it resolves the issue.
ref to my another answer - use this for reference
<http:listener-config name="HTTP_Listener_Configuration" host="0.0.0.0" port="8080" doc:name="HTTP Listener Configuration"/>
<flow name="sample-http-appFlow">
<http:listener config-ref="HTTP_Listener_Configuration" path="flow1" doc:name="HTTP"/>
<logger message="Payload is #[message.payload]" level="INFO" doc:name="Logger"/>
</flow>
<flow name="sample-http-appFlow1">
<http:listener config-ref="HTTP_Listener_Configuration" path="flow2" doc:name="HTTP"/>
<logger message="Payload is #[message.payload]" level="INFO" doc:name="Logger"/>
</flow>
I am using Jmeter 2.8 and trying to do distributed (master-slave) testing for the first time.
I setup everything fine
set Ip addresses in JMeter.properties
run Jmater-server.bat on all machines
but when I run test(a very simple request) it gets failed:
Request:
http://localhost:8080/web/login?credentials=<credential><userId>WillieWonka
</userId><passwd><![CDATA[WillieWonka]]></passwd></credential>
Error I am getting is
org.apache.http.conn.HttpHostConnectException: Connection to http://localhost:8080 refused
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:190)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:640)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:479)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl.sample(HTTPHC4Impl.java:284)
at org.apache.jmeter.protocol.http.sampler.HTTPSamplerProxy.sample(HTTPSamplerProxy.java:62)
at org.apache.jmeter.protocol.http.sampler.HTTPSamplerBase.sample(HTTPSamplerBase.java:1075)
at org.apache.jmeter.protocol.http.sampler.HTTPSamplerBase.sample(HTTPSamplerBase.java:1064)
at org.apache.jmeter.threads.JMeterThread.process_sampler(JMeterThread.java:426)
at org.apache.jmeter.threads.JMeterThread.run(JMeterThread.java:255)
at java.lang.Thread.run(Unknown Source)
Caused by: java.net.ConnectException: Connection refused: connect
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(Unknown Source)
at java.net.PlainSocketImpl.connectToAddress(Unknown Source)
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.SocksSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
While server is up and running (I tested this request running from localMachine only, It is working fine).
What is wrong?
What I am missing?
I have resolved this issue, replaced
localhost:8080
with
computerName:8080
and start application server via start.bat instead run.bat
now it is working fine.