If you set ss-enabled-components to all and then start the locator:
gfsh>start locator --name=IsLocator --bind-address=#.#.#.# --port=10334 --properties-file=..\config\gemfire.properties --security-properties-file=..\confi
g\gfsecurity.properties --J=-Dgemfire.jmx-manager-start=true --J=-Dgemfire.jmx-manager=true
the locator starts fine. So I try to connect the GFSH to the locator with an SSL connection:
gfsh>connect --locator=#.#.#.#[10334] --use-ssl
Please specify these SSL Configuration properties:
key-store: trusted.keystore
key-store-password: ********
trust-store: trusted.keystore
trust-store-password: ********
ciphers:
protocols:
Connecting to Locator at [host=#.#.#.#, port=10334] ..
There is an error
java.lang.ClassCastException: java.lang.Class cannot be cast to org.apache.geode.management.internal.JmxManagerLocatorResponse
Also, the locator logs show the Cluster configuration service start up completed successfully and is now running .... info, but then come repeated errors, all like:
[info 2017/09/22 14:35:10.213 BST IsLocator <locator request thread[3]> tid=0x58] Exception in processing request from #.#.#.#
javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection?
at sun.security.ssl.InputRecord.handleUnknownRecord(InputRecord.java:710)
at sun.security.ssl.InputRecord.read(InputRecord.java:527)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:983)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1385)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1413)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1397)
at org.apache.geode.internal.net.SocketCreator.configureServerSSLSocket(SocketCreator.java:1011)
at org.apache.geode.distributed.internal.tcpserver.TcpServer.lambda$processRequest$0(TcpServer.java:345)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
What is the locator trying to communicate with?
I think this was to do with not having set keyalg=EC in the SSL certificate per this question
Related
I have a Apache Kafka (v. 2.13-3.0.0) installed on a remote Ubuntu server.
I follow this tutorial to secure my cluster:
https://medium.com/egen/securing-kafka-cluster-using-sasl-acl-and-ssl-dec15b439f9d
but when I try to start Kafka with jaas conf file with the commands:
export KAFKA_OPTS=-Djava.security.auth.login.config=<kafka-binary-
dir>/config/kafka_server_jaas.conf
./bin/kafka-server-start.sh ./config/server.properties
I receive the error:
[2021-11-12 10:30:47,864] INFO Registered kafka:type=kafka.Log4jController MBean (kafka.utils.Log4jControllerRegistration$)
[2021-11-12 10:30:48,089] INFO Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation (org.apache.zookeeper.common.X509Util)
[2021-11-12 10:30:48,099] ERROR Exiting Kafka due to fatal exception (kafka.Kafka$)
java.lang.ClassNotFoundException: kafka.security.auth.SimpleAclAuthorizer
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
at java.base/java.lang.Class.forName0(Native Method)
at java.base/java.lang.Class.forName(Class.java:398)
at org.apache.kafka.common.utils.Utils.loadClass(Utils.java:417)
at org.apache.kafka.common.utils.Utils.newInstance(Utils.java:406)
at kafka.security.authorizer.AuthorizerUtils$.createAuthorizer(AuthorizerUtils.scala:31)
at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:1583)
at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:1394)
at kafka.Kafka$.buildServer(Kafka.scala:67)
at kafka.Kafka$.main(Kafka.scala:87)
at kafka.Kafka.main(Kafka.scala)
These are the SSL config in server.properties file:
########### SECURITY using SCRAM-SHA-512 and SSL
listeners=PLAINTEXT://localhost:9092,SASL_PLAINTEXT://localhost:9093,SASL_SSL://localhost:9094
advertised.listeners=PLAINTEXT://localhost:9092,SASL_PLAINTEXT://localhost:9093,SASL_SSL://localhost:9094
security.inter.broker.protocol=SASL_SSL
ssl.endpoint.identification.algorithm=
ssl.client.auth=required
sasl.mechanism.inter.broker.protocol=SCRAM-SHA-512
sasl.enabled.mechanisms=SCRAM-SHA-512
# Broker security settings
ssl.truststore.location=/home/kafka/Downloads/kafka_2.13-3.0.0/config/truststore/kafka.truststore.jks
ssl.truststore.password=giuseppe
ssl.keystore.location=/home/kafka/Downloads/kafka_2.13-3.0.0/config/keystore/kafka.keystore.jks
ssl.keystore.password=giuseppe
ssl.key.password=giuseppe
# ACLs
authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer
super.users=User:admin
#zookeeper SASL
zookeeper.set.acl=false
########### SECURITY using SCRAM-SHA-512 and SSL
If I try to comment the 2 rows of ACL I receive the error:
[2021-11-12 11:05:29,301] INFO [ThrottledChannelReaper-
ControllerMutation]: Starting
(kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[2021-11-12 11:05:29,331] ERROR [KafkaServer id=0] Fatal error
during KafkaServer startup. Prepare to shutdown
(kafka.server.KafkaServer)
org.apache.kafka.common.KafkaException: Failed to acquire lock on
file .lock in /tmp/kafka-logs. A Kafka instance in another process
or thread is using this directory.
at kafka.log.LogManager.$anonfun$lockLogDirs$1(LogManager.scala:241)
at scala.collection.StrictOptimizedIterableOps.flatMap(StrictOptimizedIterableOps.scala:117)
at scala.collection.StrictOptimizedIterableOps.flatMap$(StrictOptimizedIterableOps.scala:104)
at scala.collection.mutable.ArraySeq.flatMap(ArraySeq.scala:37)
at kafka.log.LogManager.lockLogDirs(LogManager.scala:236)
at kafka.log.LogManager.<init>(LogManager.scala:112)
at kafka.log.LogManager$.apply(LogManager.scala:1283)
at kafka.server.KafkaServer.startup(KafkaServer.scala:254)
at kafka.Kafka$.main(Kafka.scala:109)
at kafka.Kafka.main(Kafka.scala)
What is the cause? May it be a wrong configuration?
Thanks.
Update:
Changing the row in:
# ACLs authorizer.class.name=org.apache.kafka.server.authorizer.Authorizer
there is this error: org.apache.kafka.common.KafkaException: Could not find
a public no-argument constructor for
org.apache.kafka.server.authorizer.Authorizer at org.apache.kafka.common.utils.Utils.newInstance(Utils.java:392)
I receive this new error:
[2021-11-12 16:51:57,613] ERROR Exiting Kafka due to fatal exception
(kafka.Kafka$)
org.apache.kafka.common.KafkaException: Could not find a public no-argument
constructor for org.apache.kafka.server.authorizer.Authorizer at org.apache.kafka.common.utils.Utils.newInstance(Utils.java:392)
at org.apache.kafka.common.utils.Utils.newInstance(Utils.java:406)
at kafka.security.authorizer.AuthorizerUtils$.createAuthorizer(AuthorizerUtils.scala:31)
at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:1583)
at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:1394)
at kafka.Kafka$.buildServer(Kafka.scala:67)
at kafka.Kafka$.main(Kafka.scala:87)
at kafka.Kafka.main(Kafka.scala)
Caused by: java.lang.NoSuchMethodException:
org.apache.kafka.server.authorizer.Authorizer.<init>()
at java.base/java.lang.Class.getConstructor0(Class.java:3508)
at java.base/java.lang.Class.getDeclaredConstructor(Class.java:2711)
at org.apache.kafka.common.utils.Utils.newInstance(Utils.java:390)
... 7 more
It just seems that if you change the
kafka.security.auth.SimpleAclAuthorizer
to
kafka.security.authorizer.AclAuthorizer
It should work; it worked for me.
Kafka 3.0 removed SimpleAclAuthorizer
Pull request - https://github.com/apache/kafka/commit/976e78e405d57943b989ac487b7f49119b0f4af4#diff-e0ccf1b5c964d2c303b6a69a8b8b67df5a6bfbae8aa514f580d353c4c6bf8e36
The blog seems to be using version 2.2.0.
I have problems getting ListSFTP processor to work.
First I have tested accessing the sftp server using the commandline on the NiFi server. This works but I have to provide the pass phrase of the private ssh key file.
Then I made sure that I configured the ListSFTP processor using the same settings as the sftp commandline:
Hostname: ip address of the server
Port: 22
Username: Username with access to the server and bound to the key
Private Key Path: set to the full path of the file containing
Private Key Phrase: password for the key file
Remote path: .
Strict Host Key Checking: false
Host Key File: File hat contains the output of ssh-keyscan for the
given server
With these settings I get an error in the logfile. Any idea why NiFi cannot access the key file and access the sftp server. Cheers
2020-03-19 15:43:19,736 ERROR [Timer-Driven Process Thread-2]
o.a.nifi.processors.standard.ListSFTP
ListSFTP[id=d343fa19-0170-1000-3352-999829a448cc] Failed to perform
listing on remote host due to java.io.IOException: Failed to obtain c
onnection to remote host due to com.jcraft.jsch.JSchException: invalid
privatekey: [B#6d9b662: {} java.io.IOException: Failed to obtain
connection to remote host due to com.jcraft.jsch.JSchException:
invalid privatekey: [B#6d9b662
at org.apache.nifi.processors.standard.util.SFTPTransfer.getChannel(SFTPTransfer.java:515)
at org.apache.nifi.processors.standard.util.SFTPTransfer.getListing(SFTPTransfer.java:212)
at org.apache.nifi.processors.standard.util.SFTPTransfer.getListing(SFTPTransfer.java:175)
at org.apache.nifi.processors.standard.ListFileTransfer.performListing(ListFileTransfer.java:106)
at org.apache.nifi.processors.standard.ListSFTP.performListing(ListSFTP.java:146)
at org.apache.nifi.processor.util.list.AbstractListProcessor.listByTrackingTimestamps(AbstractListProcessor.java:472)
at org.apache.nifi.processor.util.list.AbstractListProcessor.onTrigger(AbstractListProcessor.java:414)
at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1162)
at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:209)
at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748) Caused by: com.jcraft.jsch.JSchException: invalid privatekey: [B#6d9b662
at com.jcraft.jsch.KeyPair.load(KeyPair.java:664)
at com.jcraft.jsch.KeyPair.load(KeyPair.java:561)
at com.jcraft.jsch.IdentityFile.newInstance(IdentityFile.java:40)
at com.jcraft.jsch.JSch.addIdentity(JSch.java:407)
at com.jcraft.jsch.JSch.addIdentity(JSch.java:388)
at org.apache.nifi.processors.standard.util.SFTPTransfer.getChannel(SFTPTransfer.java:485)
... 18 common frames omitted
I'm able connect IBM Queue directly but when u tried to connect from mule getting the below error and not able to deploy. I'm getting the below error
ERROR 2017-04-25 06:45:13,582
[main]org.mule.retry.notifiers.ConnectNotifier: Failed to connect/reconnect:
WebSphereMQConnector
{
name=WMQ2
lifecycle=initialise
this=5e7abaf7
numberOfConcurrentTransactedReceivers=4
createMultipleTransactedReceivers=true
connected=false
supportedProtocols=[wmq]
serviceOverrides=<none>
}
. Root Exception was: Connection timed out: connect. Type: class java.net.ConnectException
ERROR 2017-04-25 06:50:23,943 [main] org.mule.module.launcher.application.DefaultMuleApplication:
************************************************
Message : JMSWMQ0018: Failed to connect to queue manager 'RQACBRKB' with connection mode 'Client' and host name '172.11.11.11(6912)'.
JMS Code : JMSWMQ0018
Element : /WMQ2 # app:config.xml:14 (WMQ)
--------------------------------------------------------------------------------
Root Exception stack trace:
java.net.ConnectException: Connection timed out: connect
at java.net.TwoStacksPlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source)
at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source)
at java.net.AbstractPlainSocketImpl.connect(Unknown Source)
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.SocksSocketImpl.connect(Unknown Source)
com.ibm.mq.jmqi.JmqiException: CC=2;RC=2538;AMQ9213: A communications error for occurred [1=java.net.ConnectException[Connection timed out: connect],3=rbitbrka.apl.com] at com.ibm.mq.jmqi.remote.impl.RemoteTCPConnection.connnectUsingLocalAddress(RemoteTCPConnection.java:810) ~[?:?]
PFB connector details:
<wmq:connector name="WMQ5" hostName="${mq.host}" port="${mq.port}" queueManager="${mq.queue.manager}" channel="CLIENTS.SALES.CRM" username="${mq.user}" password="${mq.password}" transportType="CLIENT_MQ_TCPIP" specification="1.1" targetClient="JMS_COMPLIANT" validateConnections="false" doc:name="WMQ" maxRedelivery="-1">
<reconnect frequency="${mq.reconnection.period.ms}" count="${mq.reconnection.attempt}"/>
</wmq:connector>
When i telnet the ip and port getting below error:
C:\Users\111>telnet 172.11.11.11 6912
Connecting To 172.11.11.11...Could not open connection to the host, on port 6912: Connect failed
But when i ping getting responce
C:\Users\111>ping 172.11.11.11
The pertinent pieces of information from your provided error are:-
JMSWMQ0018: Failed to connect to queue manager 'RQACBRKB'
with connection mode 'Client' and host name '172.11.11.11(6912)'.
com.ibm.mq.jmqi.JmqiException: CC=2;RC=2538;
MQRC 2538 is MQRC_HOST_NOT_AVAILABLE which is explained in Knowledge Center. In there it mentions the most common reasons for this error:-
The listener has not been started on the remote system. (please check that your listener is running on port 6912 on the machine at IP address 172.11.11.11)
The connection name in the client channel definition is incorrect. (the connection name your client is using is '172.11.11.11(6912)' - is this correct?)
The network is currently unavailable.
A firewall blocking the port, or protocol-specific traffic.
The security call initializing the IBM MQ client is blocked by a security exit on the SVRCONN channel at the server.
"java.net.ConnectException: Connection timed out: connect" usually occurs when you have a config issue or you are unable to connect to the remote server. As mentioned above do you have an error on the MQ end and if not have you checked the connection properties within config. If these are correct, are you able to reach MQ from another client, such as SOAPUI?
Can you also post the connector and flow details?
I have followed the steps mentioned on https://github.com/PatrickCallaghan/datastax-ssl-secure-cluster/blob/master/README.md for setting up a secure SSL cassandra cluster. I receive the same error as you "Failed to setup secure pipeline". I overrode my cassandra.yaml cipher suites as mentioned by the website and I still get the same error.
My cassandra.yaml looks like this:
client_encryption_options:
enabled: true
# If enabled and optional is set to true encrypted and unencrypted connections are handled.
optional: false
keystore: ***/ssl/cassandra3_keystore.jks
keystore_password: ****
# require_client_auth: false
# Set trustore and truststore_password if require_client_auth is true
# truststore: conf/.truststore
# truststore_password: cassandra
# More advanced defaults below:
# protocol: TLS
# algorithm: SunX509
# store_type: JKS
cipher_suites: [TLS_RSA_WITH_AES_128_CBC_SHA]
Could someone guide me on what I could do?
Here is the full error trace:
Exception (java.lang.RuntimeException) encountered during startup: Failed to setup secure pipeline
java.lang.RuntimeException: Failed to setup secure pipeline
at org.apache.cassandra.transport.Server$AbstractSecureIntializer.<init>(Server.java:354)
at org.apache.cassandra.transport.Server$SecureInitializer.<init>(Server.java:411)
at org.apache.cassandra.transport.Server.start(Server.java:152)
at org.apache.cassandra.service.NativeTransportService$$Lambda$203.0000000040E88830.accept(Unknown Source)
at java.util.Collections$SingletonSet.forEach(Collections.java:4778)
at org.apache.cassandra.service.NativeTransportService.start(NativeTransportService.java:128)
at org.apache.cassandra.service.CassandraDaemon.startNativeTransport(CassandraDaemon.java:633)
at org.apache.cassandra.service.CassandraDaemon.start(CassandraDaemon.java:495)
at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:600)
at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:714)
Caused by: java.io.IOException: Error creating the initializing the SSL Context
at org.apache.cassandra.security.SSLFactory.createSSLContext(SSLFactory.java:170)
at org.apache.cassandra.transport.Server$AbstractSecureIntializer.<init>(Server.java:350)
... 9 more
Caused by: java.security.NoSuchAlgorithmException: SunX509 KeyManagerFactory not available
at sun.security.jca.GetInstance.getInstance(GetInstance.java:171)
at javax.net.ssl.KeyManagerFactory.getInstance(KeyManagerFactory.java:12)
at org.apache.cassandra.security.SSLFactory.createSSLContext(SSLFactory.java:146)
... 10 more
ERROR 15:36:01 Exception encountered during startup
java.lang.RuntimeException: Failed to setup secure pipeline
at org.apache.cassandra.transport.Server$AbstractSecureIntializer.<init>(Server.java:354) ~[apache-cassandra-3.7.jar:3.7]
at org.apache.cassandra.transport.Server$SecureInitializer.<init>(Server.java:411) ~[apache-cassandra-3.7.jar:3.7]
at org.apache.cassandra.transport.Server.start(Server.java:152) ~[apache-cassandra-3.7.jar:3.7]
at org.apache.cassandra.service.NativeTransportService$$Lambda$203.0000000040E88830.accept(Unknown Source) ~[na:na]
at java.util.Collections$SingletonSet.forEach(Collections.java:4778) ~[na:1.8.0-internal]
at org.apache.cassandra.service.NativeTransportService.start(NativeTransportService.java:128) ~[apache-cassandra-3.7.jar:3.7]
at org.apache.cassandra.service.CassandraDaemon.startNativeTransport(CassandraDaemon.java:633) [apache-cassandra-3.7.jar:3.7]
at org.apache.cassandra.service.CassandraDaemon.start(CassandraDaemon.java:495) [apache-cassandra-3.7.jar:3.7]
at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:600) [apache-cassandra-3.7.jar:3.7]
at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:714) [apache-cassandra-3.7.jar:3.7]
Caused by: java.io.IOException: Error creating the initializing the SSL Context
at org.apache.cassandra.security.SSLFactory.createSSLContext(SSLFactory.java:170) ~[apache-cassandra-3.7.jar:3.7]
at org.apache.cassandra.transport.Server$AbstractSecureIntializer.<init>(Server.java:350) ~[apache-cassandra-3.7.jar:3.7]
... 9 common frames omitted
Caused by: java.security.NoSuchAlgorithmException: SunX509 KeyManagerFactory not available
at sun.security.jca.GetInstance.getInstance(GetInstance.java:171) ~[na:1.8.0-internal]
at javax.net.ssl.KeyManagerFactory.getInstance(KeyManagerFactory.java:12) ~[na:8.0 build_20150122]
at org.apache.cassandra.security.SSLFactory.createSSLContext(SSLFactory.java:146) ~[apache-cassandra-3.7.jar:3.7]
... 10 common frames omitted
You can get round it by overriding the cipher suites for both node-to-node and client-node properties e.g.
cipher_suites: [TLS_RSA_WITH_AES_128_CBC_SHA]
This is because of the following problem in Oracle Java. http://www.pathin.org/tutorials/java-cassandra-cannot-support-tls_rsa_with_aes_256_cbc_sha-with-currently-installed-providers/
Once downloaded you can copy the files to the correct library on your server.
e.g.
scp * root#server:/usr/lib/jvm/java-7-oracle/jre/lib/security/
Getting this exception while connecting to Amazon S3 Bucket to download an object.
Exception in thread “main” com.amazonaws.SdkClientException: Unable to
execute HTTP request: Unrecognized SSL message, plaintext connection
The issue is due to:
Java 6 or earlier clients ignore SNI alerts but in Java 7/8 clients, SNI checking is enabled by default.
Resolution:
You can disable SNI alerts by specifying the JVM system parameter. Just add -Djsse.enableSNIExtension=false in eclipse.ini and restart eclipse.