I need to perform testing of certain scenarios of upload/download on SFTP server. For that i am using apache JMeter plugin SSH protocol support. I am able to test single file upload for a single user. I am struggling to perform scenarios such as Multiple file upload for single user OR Multiple file upload for multiple users using this plugin.
Any idea if there is any other plugin in order to achieve this or any alternative way?
Custom sampler code
import com.jcraft.jsch.*;
import java.io.*;
def jsch = new JSch()
def session = jsch.getSession(${__property(loginUserName)}, ${__property(hostURL)}, ${__property(port)})
session.setConfig("StrictHostKeyChecking", "no")
session.setPassword(${__property(loginUserPassword)})
def sftpSession = session.connect()
def channel = session.openChannel("sftp")
channel.connect();
def channelSftp = (ChannelSftp)channel;
log.info("SFTP Connection with host is acquired" + channelSftp)
channelSftp.cd(${__property(sftpDestinationFolder)});
for(i = 0; i <100; i++){
def f1 = new File(${__property(inputFileLocationOfBigFile)});
channelSftp.put(new java.io.FileInputStream(f1), f1.getName()+i);
}
session.disconnect()
Working fine on GUI mode of apache server but when running on the linux box or headless way it gives the below error.
2020-03-18 03:25:28,918 ERROR o.a.j.p.j.s.JSR223Sampler: Problem in JSR223 script JSR223 Sampler, message: javax.script.ScriptException: org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
Script1.groovy: 4: unable to resolve class JSch
# line 4, column 12.
def jsch = new JSch()
^
Script1.groovy: 12: unable to resolve class ChannelSftp
# line 12, column 19.
def channelSftp = (ChannelSftp)channel;
^
2 errors
javax.script.ScriptException: org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
Script1.groovy: 4: unable to resolve class JSch
# line 4, column 12.
def jsch = new JSch()
^
Script1.groovy: 12: unable to resolve class ChannelSftp
# line 12, column 19.
def channelSftp = (ChannelSftp)channel;
^
2 errors
at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.compile(GroovyScriptEngineImpl.java:183) ~[groovy-all-2.4.13.jar:2.4.13]
at org.apache.jmeter.util.JSR223TestElement.processFileOrScript(JSR223TestElement.java:215) ~[ApacheJMeter_core.jar:4.0 r1823414]
at org.apache.jmeter.protocol.java.sampler.JSR223Sampler.sample(JSR223Sampler.java:69) [ApacheJMeter_java.jar:4.0 r1823414]
at org.apache.jmeter.threads.JMeterThread.executeSamplePackage(JMeterThread.java:490) [ApacheJMeter_core.jar:4.0 r1823414]
at org.apache.jmeter.threads.JMeterThread.processSampler(JMeterThread.java:416) [ApacheJMeter_core.jar:4.0 r1823414]
at org.apache.jmeter.threads.JMeterThread.run(JMeterThread.java:250) [ApacheJMeter_core.jar:4.0 r1823414]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_241]
Caused by: org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
Script1.groovy: 4: unable to resolve class JSch
# line 4, column 12.
def jsch = new JSch()
^
Script1.groovy: 12: unable to resolve class ChannelSftp
# line 12, column 19.
def channelSftp = (ChannelSftp)channel;
^
2 errors
at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310) ~[groovy-all-2.4.13.jar:2.4.13]
at org.codehaus.groovy.control.CompilationUnit.applyToSourceUnits(CompilationUnit.java:958) ~[groovy-all-2.4.13.jar:2.4.13]
at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:605) ~[groovy-all-2.4.13.jar:2.4.13]
at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:554) ~[groovy-all-2.4.13.jar:2.4.13]
at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298) ~[groovy-all-2.4.13.jar:2.4.13]
at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268) ~[groovy-all-2.4.13.jar:2.4.13]
at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:254) ~[groovy-all-2.4.13.jar:2.4.13]
at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:211) ~[groovy-all-2.4.13.jar:2.4.13]
at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.getScriptClass(GroovyScriptEngineImpl.java:331) ~[groovy-all-2.4.13.jar:2.4.13]
at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.compile(GroovyScriptEngineImpl.java:181) ~[groovy-all-2.4.13.jar:2.4.13]
... 6 more
2020-03-18 03:25:28,927 INFO o.a.j.t.JMeterThread: Thread is done: Thread Group 1-1
2020-03-18 03:25:28,927 INFO o.a.j.t.JMeterThread: Thread finished: Thread Group 1-1
Any help on how to resolve this?
You cannot pass more than one file at a time over single SFTP connection, multiple files need to be queued up.
You can however open several connections by a single user or with different users, just make sure to properly parameterize the credentials and file names/locations.
For example:
Credentials can be parameterized using CSV Data Set Config
Files can be parameterized using Directory Listing Config
Related
I have a Apache Kafka (v. 2.13-3.0.0) installed on a remote Ubuntu server.
I follow this tutorial to secure my cluster:
https://medium.com/egen/securing-kafka-cluster-using-sasl-acl-and-ssl-dec15b439f9d
but when I try to start Kafka with jaas conf file with the commands:
export KAFKA_OPTS=-Djava.security.auth.login.config=<kafka-binary-
dir>/config/kafka_server_jaas.conf
./bin/kafka-server-start.sh ./config/server.properties
I receive the error:
[2021-11-12 10:30:47,864] INFO Registered kafka:type=kafka.Log4jController MBean (kafka.utils.Log4jControllerRegistration$)
[2021-11-12 10:30:48,089] INFO Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation (org.apache.zookeeper.common.X509Util)
[2021-11-12 10:30:48,099] ERROR Exiting Kafka due to fatal exception (kafka.Kafka$)
java.lang.ClassNotFoundException: kafka.security.auth.SimpleAclAuthorizer
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
at java.base/java.lang.Class.forName0(Native Method)
at java.base/java.lang.Class.forName(Class.java:398)
at org.apache.kafka.common.utils.Utils.loadClass(Utils.java:417)
at org.apache.kafka.common.utils.Utils.newInstance(Utils.java:406)
at kafka.security.authorizer.AuthorizerUtils$.createAuthorizer(AuthorizerUtils.scala:31)
at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:1583)
at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:1394)
at kafka.Kafka$.buildServer(Kafka.scala:67)
at kafka.Kafka$.main(Kafka.scala:87)
at kafka.Kafka.main(Kafka.scala)
These are the SSL config in server.properties file:
########### SECURITY using SCRAM-SHA-512 and SSL
listeners=PLAINTEXT://localhost:9092,SASL_PLAINTEXT://localhost:9093,SASL_SSL://localhost:9094
advertised.listeners=PLAINTEXT://localhost:9092,SASL_PLAINTEXT://localhost:9093,SASL_SSL://localhost:9094
security.inter.broker.protocol=SASL_SSL
ssl.endpoint.identification.algorithm=
ssl.client.auth=required
sasl.mechanism.inter.broker.protocol=SCRAM-SHA-512
sasl.enabled.mechanisms=SCRAM-SHA-512
# Broker security settings
ssl.truststore.location=/home/kafka/Downloads/kafka_2.13-3.0.0/config/truststore/kafka.truststore.jks
ssl.truststore.password=giuseppe
ssl.keystore.location=/home/kafka/Downloads/kafka_2.13-3.0.0/config/keystore/kafka.keystore.jks
ssl.keystore.password=giuseppe
ssl.key.password=giuseppe
# ACLs
authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer
super.users=User:admin
#zookeeper SASL
zookeeper.set.acl=false
########### SECURITY using SCRAM-SHA-512 and SSL
If I try to comment the 2 rows of ACL I receive the error:
[2021-11-12 11:05:29,301] INFO [ThrottledChannelReaper-
ControllerMutation]: Starting
(kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[2021-11-12 11:05:29,331] ERROR [KafkaServer id=0] Fatal error
during KafkaServer startup. Prepare to shutdown
(kafka.server.KafkaServer)
org.apache.kafka.common.KafkaException: Failed to acquire lock on
file .lock in /tmp/kafka-logs. A Kafka instance in another process
or thread is using this directory.
at kafka.log.LogManager.$anonfun$lockLogDirs$1(LogManager.scala:241)
at scala.collection.StrictOptimizedIterableOps.flatMap(StrictOptimizedIterableOps.scala:117)
at scala.collection.StrictOptimizedIterableOps.flatMap$(StrictOptimizedIterableOps.scala:104)
at scala.collection.mutable.ArraySeq.flatMap(ArraySeq.scala:37)
at kafka.log.LogManager.lockLogDirs(LogManager.scala:236)
at kafka.log.LogManager.<init>(LogManager.scala:112)
at kafka.log.LogManager$.apply(LogManager.scala:1283)
at kafka.server.KafkaServer.startup(KafkaServer.scala:254)
at kafka.Kafka$.main(Kafka.scala:109)
at kafka.Kafka.main(Kafka.scala)
What is the cause? May it be a wrong configuration?
Thanks.
Update:
Changing the row in:
# ACLs authorizer.class.name=org.apache.kafka.server.authorizer.Authorizer
there is this error: org.apache.kafka.common.KafkaException: Could not find
a public no-argument constructor for
org.apache.kafka.server.authorizer.Authorizer at org.apache.kafka.common.utils.Utils.newInstance(Utils.java:392)
I receive this new error:
[2021-11-12 16:51:57,613] ERROR Exiting Kafka due to fatal exception
(kafka.Kafka$)
org.apache.kafka.common.KafkaException: Could not find a public no-argument
constructor for org.apache.kafka.server.authorizer.Authorizer at org.apache.kafka.common.utils.Utils.newInstance(Utils.java:392)
at org.apache.kafka.common.utils.Utils.newInstance(Utils.java:406)
at kafka.security.authorizer.AuthorizerUtils$.createAuthorizer(AuthorizerUtils.scala:31)
at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:1583)
at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:1394)
at kafka.Kafka$.buildServer(Kafka.scala:67)
at kafka.Kafka$.main(Kafka.scala:87)
at kafka.Kafka.main(Kafka.scala)
Caused by: java.lang.NoSuchMethodException:
org.apache.kafka.server.authorizer.Authorizer.<init>()
at java.base/java.lang.Class.getConstructor0(Class.java:3508)
at java.base/java.lang.Class.getDeclaredConstructor(Class.java:2711)
at org.apache.kafka.common.utils.Utils.newInstance(Utils.java:390)
... 7 more
It just seems that if you change the
kafka.security.auth.SimpleAclAuthorizer
to
kafka.security.authorizer.AclAuthorizer
It should work; it worked for me.
Kafka 3.0 removed SimpleAclAuthorizer
Pull request - https://github.com/apache/kafka/commit/976e78e405d57943b989ac487b7f49119b0f4af4#diff-e0ccf1b5c964d2c303b6a69a8b8b67df5a6bfbae8aa514f580d353c4c6bf8e36
The blog seems to be using version 2.2.0.
I'm trying to publish message to a Tibco Queue on a SSL Tibco Server through JMeter 5.4.1 using JMS Point-to-Point Logic Controller.
JMS Point To Point Controller Config
But I'm getting the following error message:
2021-06-13 12:25:46,278 ERROR o.a.j.p.j.s.JMSSampler: Not permitted:
Failed to connect to any server at: ssl://[server-name]:7352,
ssl://[server-name]:7352 [Error: Failed to connect via SSL to
[ssl://[server-name]:7352]: Received fatal alert:
protocol_version: url that returned this exception =
SSL://[server-name]:7352 ]
javax.naming.AuthenticationException: Not permitted: Failed to connect
to any server at: ssl://[server-name]:7352,
ssl://[server-name]:7352 [Error: Failed to connect via SSL to
[ssl://[server-name]:7352]: Received fatal alert:
protocol_version: url that returned this exception =
SSL://[server-name] ] at
com.tibco.tibjms.naming.TibjmsContext.lookup(TibjmsContext.java:670)
~[tibjms.jar:8.0.0] at
com.tibco.tibjms.naming.TibjmsContext.lookup(TibjmsContext.java:491)
~[tibjms.jar:8.0.0] at
javax.naming.InitialContext.lookup(InitialContext.java:417)
~[?:1.8.0_291] at
org.apache.jmeter.protocol.jms.sampler.JMSSampler.threadStarted(JMSSampler.java:638)
[ApacheJMeter_jms.jar:5.4.1] at
org.apache.jmeter.threads.JMeterThread$ThreadListenerTraverser.addNode(JMeterThread.java:784)
[ApacheJMeter_core.jar:5.4.1] at
org.apache.jorphan.collections.HashTree.traverseInto(HashTree.java:993)
[jorphan.jar:5.4.1] at
org.apache.jorphan.collections.HashTree.traverse(HashTree.java:976)
[jorphan.jar:5.4.1] at
org.apache.jmeter.threads.JMeterThread.threadStarted(JMeterThread.java:752)
[ApacheJMeter_core.jar:5.4.1] at
org.apache.jmeter.threads.JMeterThread.initRun(JMeterThread.java:740)
[ApacheJMeter_core.jar:5.4.1] at
org.apache.jmeter.threads.JMeterThread.run(JMeterThread.java:252)
[ApacheJMeter_core.jar:5.4.1]
I tried:
openssl s_client -connect [server-name]:7352
It gave the following output:
SSL-Session:
Protocol : TLSv1.2
Cipher : ECDHE-RSA-AES256-GCM-SHA384
Session-ID:
Session-ID-ctx:
So added the following line in jmeter.properties file.
https.default.protocol=TLSv1.2
Also commented jdk.tls.disabledAlgorithms from java.security file for JDK (I'm using jdk1.8.0_291)
# jdk.tls.disabledAlgorithms=SSLv3, TLSv1, TLSv1.1, RC4, DES, MD5withRSA, \
# DH keySize < 1024, EC keySize < 224, 3DES_EDE_CBC, anon, NULL, \
# include jdk.disabled.namedCurves
But still I'm getting the same error. Someone please help.
I think you're using the wrong property (not only the wrong property but the wrong place as well), you're setting default protocol for HTTPS, while you need to set it for TLS, i.e. add the next line to system.properties file
jdk.tls.client.protocols=TLSv1.2
JMeter restart will be required to apply this property.
If it won't help or you will get different errors - consider adding the next line there as well:
javax.net.debug=all
and then check jmeter.log file and stdout for any suspicious entries
More information:
Configuring JMeter
Apache JMeter Properties Customization Guide
I resolved it by using the latest tibjms.jar in the lib directory in JMeter as the Tibco server was upgraded some hours before I raised this issue.
I am trying to connect to a secure FTP server created based on this link. I have not followed step 8 in the link. I am able to establish the connection and also change and print directories but am unable to create new directories. I am also unable to fetch files list.
Here is my code snippet:
import ssl
from ftplib import FTP_TLS
import sys
import os
import os.path
def connect():
ftp = FTP_TLS()
ftp.debugging = 2
ftp.connect('ipaddress', 21)
ftp.set_pasv(False)
ftp.login('user', 'passwd')
return ftp
ftps = connect()
destdir = "/"
try:
resp = ftps.pwd()
ftps.cwd(destdir)
except Exception:
ftps.mkd(destdir)
print(resp)
root = 'C:\\Users\\****\\****\\Logs' # local dir
for (dir, _, files) in os.walk(root):
newdir = destdir+dir[len(root):len(dir)].replace("\\", "/")
print(newdir)
try:
ftps.cwd(newdir)
except Exception:
ftps.mkd(newdir)
I am using python 3.7.3 and the corresponding ftplib. I would be happy to provide any other details required.
PS: I am able to connect with Filezilla and create directories.
This is the error after running.
I am able to create the directories successfully once I change the dir to /logs. I am getting an error "ftplib.error_perm: 500 Illegal PORT command." whenever I send cmd like retrlines or storbinary
, I get this error
I have searched about this and people have asked to set it to pasv mode. When I do that, I get this error. FYI, I have enabled pasv mode in the config file
I tried changing the port number to a number between pasv ports enabled in the config file (between 30000-31000). It does not connect also in this case. Error returned "ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it"
I am working the first time with an FTP server or for that matter any server communications, so my problems might seem trivial. I am sorry for your trouble.
Thanks.
I have two applications, one is deployed in jboss as 7, another is a java application (Camel).
The camel has an exposed JMX service:
service:jmx:rmi:///jndi/rmi://172.20.30.85:19999/jmxrmi/camel
now, I need to access it from a EJB(jboss as 7). I have tried the following two ways, but both are fialed.
A) works well with jboss as 5.x:
JMXServiceURL target = new JMXServiceURL("service:jmx:rmi:///jndi/rmi://" + host + ":" + port.toString() + "/jmxrmi/camel");
JMXConnector connector = JMXConnectorFactory.connect(target);
got exception:
Caused by: javax.naming.NameNotFoundException: rmi://172.20.30.85:19999/jmxrmi/camel -- service jboss.naming.context.java.rmi:."172.20.30.85:19999".jmxrmi.camel
at org.jboss.as.naming.ServiceBasedNamingStore.lookup(ServiceBasedNamingStore.java:103)
at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:197)
at org.jboss.as.naming.InitialContext$DefaultInitialContext.lookup(InitialContext.java:241)
at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:183)
at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:179)
at javax.naming.InitialContext.lookup(InitialContext.java:411)
at javax.naming.InitialContext.lookup(InitialContext.java:411)
at javax.management.remote.rmi.RMIConnector.findRMIServerJNDI(RMIConnector.java:1929)
at javax.management.remote.rmi.RMIConnector.findRMIServer(RMIConnector.java:1896)
at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:286)
B)
JMXServiceURL target = new JMXServiceURL("service:jmx:remoting-jmx://172.20.30.85:19999/jmxrmi/camel");
got exception:
ERROR [org.jboss.remoting.remote.connection] (Remoting "endpoint" read-1) JBREM000200: Remote connection failed: java.io.IOException: JBREM000202: Abrupt close on Remoting connection 79f4a60c to sha-lri-pc-161/172.20.30.85:19999
11:33:44,415 WARN [com.lombardrisk.reform.process.api.ProcessInspectorBean] (http-/127.0.0.1:8080-29) Could not connect to integration MBean server: java.lang.RuntimeException: Operation failed with status WAITING
at org.jboss.remotingjmx.RemotingConnector.internalRemotingConnect(RemotingConnector.java:235)
at org.jboss.remotingjmx.RemotingConnector.internalConnect(RemotingConnector.java:148)
at org.jboss.remotingjmx.RemotingConnector.connect(RemotingConnector.java:99)
at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
at com.lombardrisk.reform.process.api.ProcessInspectorBean.getIntegrationMBeanServerConnection(ProcessInspectorBean.java:328)
at com.lombardrisk.reform.process.api.ProcessInspectorBean.getRoutes(ProcessInspectorBean.java:486)
Is there anyone who can guide me? Thanks in advance.
My config file:
Host server
User new_user
HostName 10.0.1.193
Port 55555
LocalForward 3000 10.0.1.193:6000
IdentityFile ~/.ssh/server
Client.py
import xmlrpclib
s = xmlrpclib.ServerProxy('http://localhost:3000')
print s.pow(2,3) # Returns 2**3 = 8
print s.add(2,3) # Returns 5
print s.div(5,2) # Returns 5//2 = 2
# Print list of available methods
print s.system.listMethods()
Server.py
from SimpleXMLRPCServer import SimpleXMLRPCServer
from SimpleXMLRPCServer import SimpleXMLRPCRequestHandler
# Restrict to a particular path.
class RequestHandler(SimpleXMLRPCRequestHandler):
rpc_paths = ('/RPC2',)
# Create server
server = SimpleXMLRPCServer(("localhost", 6000),
requestHandler=RequestHandler)
server.register_introspection_functions()
# Register pow() function; this will use the value of
# pow.__name__ as the name, which is just 'pow'.
server.register_function(pow)
# Register a function under a different name
def adder_function(x,y):
return x + y
server.register_function(adder_function, 'add')
# Register an instance; all the methods of the instance are
# published as XML-RPC methods (in this case, just 'div').
class MyFuncs:
def div(self, x, y):
return x // y
server.register_instance(MyFuncs())
# Run the server's main loop
server.serve_forever()
My server.py is running fine, but when I run my client.py, it gives the following error:
Traceback (most recent call last):
File "client.py", line 4, in <module>
print s.pow(2,3) # Returns 2**3 = 8
File "/usr/lib/python2.7/xmlrpclib.py", line 1224, in __call__
return self.__send(self.__name, args)
File "/usr/lib/python2.7/xmlrpclib.py", line 1578, in __request
verbose=self.__verbose
File "/usr/lib/python2.7/xmlrpclib.py", line 1264, in request
return self.single_request(host, handler, request_body, verbose)
File "/usr/lib/python2.7/xmlrpclib.py", line 1292, in single_request
self.send_content(h, request_body)
File "/usr/lib/python2.7/xmlrpclib.py", line 1439, in send_content
connection.endheaders(request_body)
File "/usr/lib/python2.7/httplib.py", line 954, in endheaders
self._send_output(message_body)
File "/usr/lib/python2.7/httplib.py", line 814, in _send_output
self.send(msg)
File "/usr/lib/python2.7/httplib.py", line 776, in send
self.connect()
File "/usr/lib/python2.7/httplib.py", line 757, in connect
self.timeout, self.source_address)
File "/usr/lib/python2.7/socket.py", line 571, in create_connection
raise err
socket.error: [Errno 111] Connection refused
I have checked that my ssh if working and I can ssh into remote server with the given configuration i.e.
ssh server
works find. Can anyone explain what might be going wrong?
Your server runs and perhaps it does not complain, but this does not mean it "runs correctly" or more pointedly it doesn't mean the server is in a working state that the client expects.
The above is somewhat cryptic for a reason: something unknown has gone wrong, and even though you don't know yet what's broken, you want to start testing things you know should work and verify they are in fact working. This is a useful debugging skill even if the error is meaningless to you.
In this case, the client error message is "connection refused", meaning "refused [at the server]".
Try this:
on your "client" PC in a Terminal/DOS window, run:
telnet [your server ip] [your server port]
You should expect the same error - a connection refused. Perhaps the server is not actually opening the port. Or perhaps the server opened the port, but you can not see it remotely on another host due to a firewall on the server.
Also, running both client and server code on the same host can sometime reveal more clues (it should work but if it doesn't then there's maybe more than 1 problem).