I'm trying to setup a Kafka cluster and add SSL to it, but I'm getting always the same error:
INFO [SocketServer brokerId=0] Failed authentication with /XXX.XXX.XXX.XXX (SSL handshake failed) (org.apache.kafka.common.network.Selector)
I read several stackoverflow posts about this problem but I cannot fix it.
I'm using Kafka 2.5.0 on Ubuntu 18.04 with OpenJDK 14 and I used Let's Encrypt to generate certificates for my domain name (that are working perfectly with nginx).
First I created a pkcs12 file with the following command:
openssl pkcs12 -export -in fullchain.pem -inkey privkey.pem -out keystore.p12 -name kafka1 -CAfile chain.pem -caname root
...then I created the keystore (with the -ext option as described in kafka docs):
keytool -importkeystore -deststorepass 'STRONG_PASS' -destkeypass 'STRONG_PASS' -destkeystore keystore.jks -srckeystore keystore.p12 -srcstoretype PKCS12 -srcstorepass 'STRONG_PASS' -alias kafka1 -ext SAN=DNS:{FQDN}
...also I added the certificate to the truststore:
keytool -trustcacerts -keystore $JAVA_HOME/lib/security/cacerts -storepass changeit -noprompt -importcert -file /etc/letsencrypt/live/YOURDOMAIN/chain.pem
At this point, I believe that I have all of the necessary steps to have the keystore and the truststore correctly configured.
My broker config is:
broker.id=0
listeners=SSL://mydomain.com:9092
ssl.keystore.location=/path/to/keystore.jks
ssl.keystore.password=password
ssl.key.password=password
ssl.truststore.location=/path/to/jdk/lib/security/cacerts
ssl.truststore.password=changeit
ssl.secure.random.implementation=SHA1PRNG
security.inter.broker.protocol=SSL
ssl.endpoint.identification.algorithm= # I tried with and without this and the problem persists
advertised.listeners=SSL://mydomain.com:9092
num.network.threads=3
num.io.threads=8
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600
log.dirs=/path/to/kafka_data
num.partitions=1
num.recovery.threads.per.data.dir=1
offsets.topic.num.partitions=3
offsets.topic.replication.factor=2
transaction.state.log.replication.factor=1
transaction.state.log.min.isr=1
min.insync.replicas=2
default.replication.factor=2
log.retention.hours=168
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000
zookeeper.connect=IP1:2181,IP2:2181,IP3:2181
zookeeper.connection.timeout.ms=18000
group.initial.rebalance.delay.ms=3000
Then I run the broker and try to execute the following command to know if everything is working:
openssl s_client -debug -connect mydomain.com:9092 -tls1 | head -n 50
And the error is displayed:
INFO [SocketServer brokerId=0] Failed authentication with /XXX.XXX.XXX.XXX (SSL handshake failed) (org.apache.kafka.common.network.Selector)
My domain is not the same as my hostname in the machine, I don't know if it could be a problem. I only want to add security to the first broker and then repeat the same in the other two, but first have the first one working.
What is wrong with my config? Maybe the truststore? or the hostname and domain name?
I am using keycloak-8.0.1 in standalone configuration . I am trying to enable SSL/Https for Keycloak server running on my Test machine (A.B.C.D) , token request will be requested from machine (X.Y.Z.P).
(Will there be any CORS/CSRF issue for the generated token , looks like yes ? And will Keycloak over SSL help to solve this?) Nevertheless i require SSL enabling .
So went ahead with https://www.keycloak.org/docs/latest/server_installation/#enabling-ssl-https-for-the-keycloak-server
Step 1 Run command :
keytool -genkey -alias localhost -keyalg RSA -keystore keycloak.jks -validity 10950
...
two files server.key and keycloak.jks got created.
Question : should i used localhost here or better to use IP A.B.C.D of my Test machine? Though documentation says localhost so went ahead with that.
Step 2 Generate a certificate request :
$ keytool -certreq -alias yourdomain -keystore keycloak.jks > keycloak.careq
I can also generate the cert request using localhost/A.B.C.D.
Step 3: Send the cert req created in above step to CA and download the root cert from CA(root.crt) and import using command:
keytool -import -keystore keycloak.jks -file root.crt -alias root
Do i have to skip this step for localhost and if not how to generate root.crt for localhost.
step 4: last step is to import CA generated certificate to keystore
$ keytool -import -alias yourdomain -keystore keycloak.jks -file your-certificate.cer
Question: Now i have only two files generated in very first step "server.key" and keycloak.jks
and from where i should get root.crt and your-certificate.cer ? I tried uploading the ca request to CAcert.org but they dont create certificate for localhost DNS.
I already went through lot of links link1 , link2 , link3 link4 and getting confused
Please help.
Answer 1-
If you are testing this in your local machine you can use localhost but better would be if you are using IP-address or host name
Answer 3
Again if its for your testing you can use Self Sign Certificate only,No need to go to Ads Certificate authority.
Answer 4-
So for localhost or your machine IP simple Create Certificate with the help of keytool
keytool -genkey -alias initcert -ext san=ip:xxx.xxx.xxx.xx -keyalg RSA -keystore keycloak.jks-validity 365 -keysize 2048
Export the certificates
keytool -export -noprompt -trustcacerts -keystore keycloak.jks -alias initcert -file keycloak.cer -storepass changeit
Import Certificates into Client machine Java Keystore
keytool -import -noprompt -trustcacerts -alias "initcert" -file keycloak.cer -keystore /workspace/tools/jdk/java-1.7.0-openjdk-1.7.0.25.x86_64/jre/lib/security/cacerts
I have IIS server with pfx containing L1K cert. I need to request a new L1M cert for it, AND will need to also be able to import the returning cert to a java keystore as the URL in question will move from IIS to Apache Tomact.
Help!
I think I found the way to do this.
*credit to this site:
https://www.jamf.com/jamf-nation/discussions/4646/converting-a-windows-pfx-or-windows-pkcs12-keystore-to-a-jks-keystore
1 - use keytool to import PFX into JKS
keytool -importkeystore -srckeystore .pfx -srcstoretype pkcs12 -destkeystore .jks -deststoretype JKS
2 - get details such as Alias from PFX file
keytool -v -list -storetype pkcs12 -keystore .pfx
3 - generate CSR file from new JKS file
keytool -certreq -alias -keystore .jks -file .csr -storepass
So far the resulting CSR files are validated successfully by my CA Authority's online tool.
i have the following issue:
I obtain a free certificate from comodo (90 days) for my glassfish web application and then i have imported the certs into glassfish 3.1 by following http://javadude.wordpress.com/2010/04/06/getting-started-with-glassfish-v3-and-ssl/
I have also modify the domain.xml file by replacing the alias s1as with my certificate alias and the file keystore.jks with the server.keystore....but when i try to access my web application with https protocol i got the following log error:
[#|2012-10-12T14:41:18.828+0200|WARNING|glassfish3.1.2|com.sun.grizzly.config.Gr
izzlyServiceListener|_ThreadID=25;_ThreadName=http-thread-pool-443(1);|GRIZZLY00
07: SSL support could not be configured!
java.io.IOException: SSL configuration is invalid due to No available certificat
e or key corresponds to the SSL cipher suites which are enabled.
Please help me..i know that here i can find the solution to my issue...
Unfortunately I don`t have enough reputation to post images of glassfish console admin, but let me try to help somebody just using text.
NOTE1: The configuration was done on Ubuntu 12.04 server and glassfish 3.1.2
Comodo gives you 4 files
your_domain.key (your private key)
your_domain.crt (your public key)
PositiveSSLCA2.crt (CA public key)
AddTrustExternalCARoot.crt (CA public key)
Import every public key into the file cacerts.jks. To do that merge the public key files in one file:
NOTE2: The order of the files DOES matter.
cat your_domain.crt PositiveSSLCA2.crt AddTrustExternalCARoot.crt > all.crt
Now import them using keytool:
keytool -import -trustcacerts -alias tomcat -file all.crt -keystore cacerts.jks
Create a p12 file with your private key:
NOTE3: You can use the same password for every file to make things easier.
openssl pkcs12 -export -in all.crt -inkey your_domain.key -out your_domain.p12 - name your_alias -CAfile PositiveSSLCA2.crt -caname immed
NOTE4: Don`t forget you alias (your_alias), you will need to reference it in glassfish admin console later.
Now import the private key using keytool:
keytool -importkeystore -deststorepass changeit -destkeypass changeit -destkeystore keystore.jks -srckeystore your_domain.p12 -srcstoretype PKCS12 -srcstorepass changeit -alias your_alias
Now your keystore.jks (with your private keys) and your cacerts.jks (with you public key) are ready to me used. If you want to check if everything is ok run:
keytool -list -keystore keystore.jks
keytool -list -keystore cacerts.jks
Go to the glassfish admin console and find the session:
Configurations->server-config->HTTP Service->Http Listeners->http-listener-2
Go to the SSL tab and change the Certificate NickName to your_domain.
Restart Glassfish server.
Preconditions:
installed keytool and GlassFish 4.x (with default keystore password changeit)
your source keystore used to generate CSR
e.g. ~/mySourceKeystore.jks with password myPassword and private key with alias myAlias
your valid certificate (obtained from CA)
e.g. ~/myCertificate.crt with password myPassword and alias myAlias
certificate of CA (obtained from CA)
e.g. ~/AwesomeCA.crt
Here are all steps how to import SSL certificate into GlassFish:
Navigate to GLASSFISH-HOME/domains/domain1/config
Import your source keystore (with private key) into GlassFish keystore:
$ keytool -importkeystore -srckeystore ~/mySourceKeystore.jks -destkeystore keystore.jks`
Enter destination keystore password: changeit
Enter source keystore password: myPassword
Entry for alias server successfully imported.
Import command completed: 1 entries successfully imported, 0 entries failed or cancelled
Import certificate of CA into GlassFish keystore:
$ keytool -import -v -trustcacerts -alias AwesomeCA -file ~/AwesomeCA.crt -keystore keystore.jks
Enter keystore password: changeit
Certificate was added to keystore
[Storing keystore.jks]
Import obtained SSL certificate into GlassFish keystore:
$ keytool -import -v -trustcacerts -alias myAlias -file ~/myCertificate.crt -keystore keystore.jks
Enter keystore password: changeit
Enter key password for <myAlias>: myPassword
Certificate reply was installed in keystore
[Storing keystore.jks]
At this moment error java.security.UnrecoverableKeyException: Cannot recover key would occur during GlassFish startup because you have different keystore password and alias key password. To prevent this error you need to execute:
$ keytool -keypasswd -alias myAlias -new changeit -keystore keystore.jks
Enter keystore password: changeit
Enter key password for <myAlias>: myPassword
Change default alias (s1as) in GlassFish to your myAlias:
$ asadmin set configs.config.server-config.network-config.protocols.protocol.http-listener-2.ssl.cert-nickname=myAlias
(Optional) You can change default SSL port (8181) in GlassFish to well known 443:
$ asadmin set server.network-config.network-listeners.network-listener.http-listener-2.port=443
Restart GlassFish
For Glassfish 4.x you can follow this Comodo Guide
Here is the web archive if link expires.
I am trying to configure SSL on my Jetty.
I read this:
http://docs.codehaus.org/display/JETTY/How+to+configure+SSL
and created a key store.
Then, I jumped directly to section 4. But where is this configuration file I should configure Jetty?
I tried to serach for jetty.xml, but there is no such on my computer...
I had a lot of problems making it work but I finally foud out how to make it happend. I'm using ubuntu 10.04 with java 7. It may be possible to do it under windows but all the comands lines are bash commands, maybe possible to do the same with cigwin/mingw
I used Jetty 8.1.8. Download it from codehaus and choose the .tar.gz file for linux (.zip for windows).
Unzip the file in any directory you wish, this will be your {jetty} home folder for the sake of this article/answer.
Go to the {jetty}/etc directory.
Execute all the following command lines in order. Whenever a password is asked, input the same password all the time. The passwords are used to protect the key file, the key store and the certificate itself. Sometimes, a password will be asked to unlock the key store or to use a generated key. Once you will understand what everything is and how to use the passwords correctly, you may change those passwords when you feel ready (safer for production use). Otherwise, input the requested informations when asked.
openssl genrsa -des3 -out jetty.key
openssl req -new -x509 -key jetty.key -out jetty.crt
keytool -keystore keystore -import -alias jetty -file jetty.crt -trustcacerts
openssl req -new -key jetty.key -out jetty.csr
openssl pkcs12 -inkey jetty.key -in jetty.crt -export -out jetty.pkcs12
keytool -importkeystore -srckeystore jetty.pkcs12 -srcstoretype PKCS12 -destkeystore keystore
Now you have to edit {jetty}/etc/jetty-ssl.xml and configure your password to match the one you used during certificate generation. If you want to obfuscate your password, go back to the command line. Go tho your {jetty} home directory and execute the following:
java -cp lib/jetty-util-8.1.8.v20121106.jar org.eclipse.jetty.util.security.Password "{PASSWORD}"
Change {PASSWORD} for your actual password then past the obfuscated password, including the "OBF:" in all password fields found in jetty-ssl.xml. Note that a password obfuscated like that is hard to read for humans but easily unobfiscated programmatically. It just prevent developpers to know the password when they edit the file. All configuration files should be secured properly and their accesses be as restrictive as possible.
Edit {jetty}/start.ini and uncomment the line #etc/jetty-ssl.xml (just remove the #).
Start jetty:
java -jar start.jar
Now contact your server at: https://localhost:8443
Done!
Note that this answer is a quick way to enable SSL with jetty. To make it secure for production, you have to read some more on the subject.
Answer updated after more experience with keystores. I assure you this solution works perfectly with intermediate certificates (29/07/2015).
Note: PEM format means a readable file, certificates start with ---BEGIN CERTIFICATE--- and private keys start with -----BEGIN PRIVATE KEY----- line.
Here's an easy step by step guide. Start with an empty directory.
Skip to Step 2 if you have private key (PEM encoded .key)
Skip to Step 3 if you have certificate signing request (PEM encoded .csr)
Skip to Step 4 if you have your certificate (PEM encoded .crt or .pem)
Prepare (password-less) private key.
openssl genrsa -des3 -passout pass:1 -out domain.pass.key 2048
openssl rsa -passin pass:1 -in domain.pass.key -out domain.key
rm domain.pass.key
Prepare certificate signing request (CSR). We'll generate this using our key. Enter relevant information when asked. Note the use of -sha256, without it, modern browsers will generate a warning.
openssl req -key domain.key -sha256 -new -out domain.csr
Prepare certificate. Pick one:
a) Sign it yourself
openssl x509 -req -days 3650 -in domain.csr -signkey domain.key -out domain.crt
b) Send it to an authority
Your SSL provider will supply you with your certificate and their intermediate certificates in PEM format.
Add to trust chain and package it in PKCS12 format. First command sets a keystore password for convenience (else you'll need to enter password a dozen times). Set a different password for safety.
export PASS=LW33Lk714l9l8Iv
Pick one:
a) Self-signed certificate (no need for intermediate certificates)
openssl pkcs12 -export -in domain.crt -inkey domain.key -out domain.p12 -name domain -passout pass:$PASS
keytool -importkeystore -deststorepass $PASS -destkeypass $PASS -destkeystore domain.keystore -srckeystore domain.p12 -srcstoretype PKCS12 -srcstorepass $PASS -alias domain
b) Need to include intermediate certificates
Download intermediate certificates and concat them into one file. The order should be sub to root.
cat sub.class1.server.ca.pem ca.pem > ca_chain.pem
Use a -caname parameter for each intermediate certificate in chain file, respective to the order they were put into the chain file.
openssl pkcs12 -export -in domain.crt -inkey domain.key -out domain.p12 -name domain -passout pass:$PASS -CAfile ca_chain.pem -caname sub1 -caname root -chain
keytool -importkeystore -deststorepass $PASS -destkeypass $PASS -destkeystore domain.keystore -srckeystore domain.p12 -srcstoretype PKCS12 -srcstorepass $PASS -alias domain
Important note: Although keytool -list will only list one entry and not any intermediate certificates, it will work perfectly.
Configure jetty.
Move domain.keystore file to JETTY_HOME/etc/.
Pick one:
a) You're using new start.ini style configuration (Jetty 8+):
jetty.keystore=etc/domain.keystore
jetty.truststore=etc/domain.keystore
jetty.keystore.password=LW33Lk714l9l8Iv
jetty.keymanager.password=LW33Lk714l9l8Iv
jetty.truststore.password=LW33Lk714l9l8Iv
b) You're using old style configuration with .xml files (you should upgrade to new style!):
Edit JETTY_HOME/etc/jetty-ssl.xml file and change the part below. Replace password parts to match your password. We don't define KeyManagerPassword because our key has no password.
<Configure id="Server" class="org.eclipse.jetty.server.Server">
<New id="sslContextFactory" class="org.eclipse.jetty.http.ssl.SslContextFactory">
<Set name="KeyStore"><Property name="jetty.home" default="." />/etc/keystore</Set>
<Set name="KeyStorePassword">LW33Lk714l9l8Iv</Set>
<Set name="TrustStore"><Property name="jetty.home" default="." />/etc/keystore</Set>
<Set name="TrustStorePassword">LW33Lk714l9l8Iv</Set>
</New>
<Call name="addConnector">...</Call>
</Configure>
Edit start.ini file to include jetty-ssl.xml file.
(Re)start jetty.
Note that this keystore file can also be used with other containers like Tomcat. Good luck!
A default configuration file for Jetty and is located at $JETTY_HOME/etc/jetty.xml
If you are using maven's jetty plugin you will need to specify ssl keystore details in your pom.xml file. See this question for details
Just bought a cert from godaddy for mere $6/year. Great deal while it lasts. Here are the steps I followed to set it up on Amazon EC2/Ubuntu/Jetty based on these sites and Jean-Philippe Gravel's answer.
http://docs.codehaus.org/display/JETTY/How+to+configure+SSL
http://community.xmatters.com/docs/DOC-1228#.UgWsI1MU7lc
keytool -keystore keystore -alias jettykey -genkey -keyalg RSA
Note that "First and last name" must be your FQDN (without http://). On my first attempt I had dutifully put my first and last name, but godaddy has good warnings and rejected it.
Generate a CSR file for Godaddy:
keytool -certreq -alias jetty -keystore keystore -file jetty.csr
Submit this in the Godaddy form to create the certificate, including the BEGIN/END "NEW CERTIFICATE REQUEST".
(Godaddy requires you to verify its your site. There a couple methods for this and since I bought the domain name via a proxy, I found it easiest and quickest to verify by hosting an html page generated by godaddy.)
Download the zip containing both certificate and intermediary certificate from godaddy. There is a list of server types to choose from. I choose "other". Then combine cert with intermediary cert.
cat mydomain.com.crt gd_bundle.crt > certchain.txt
export my private key
keytool -importkeystore -srckeystore keystore -destkeystore intermediate.p12 -deststoretype PKCS12
openssl pkcs12 -in intermediate.p12 -out jettykey.pem -nodes
combine private key and certificate
openssl pkcs12 -export -inkey jettykey.pem -in certchain.txt -out jetty.pkcs12
import pkcs12 cert (alias becomes 1)
keytool -importkeystore -srckeystore jetty.pkcs12 -srcstoretype PKCS12 -destkeystore keystore
(I backed up the keystore then deleted the original key. I did this while troubleshooting and this may or may not be required by Jetty.)
keytool -delete -keystore keystore -alias jettykey
sudo cp keystore /usr/share/jetty/etc/
sudo vi /usr/share/jetty/etc/jetty-ssl.xml
Modify your.store.password, your.key.password, and your.trust.password accordingly. If you want to obfuscate it, use
java -cp /usr/share/jetty/lib/jetty.jar:/usr/share/jetty/lib/jetty-util.jar org.mortbay.jetty.security.Password <your.password>
Indicate to Jetty to load the jetty-ssl.xml file.
sudo echo "/etc/jetty/jetty-ssl.xml" >> /etc/jetty/jetty.conf
sudo /sbin/iptables -t nat -I PREROUTING -p tcp --dport 443 -j REDIRECT --to-port 8443
(Also modify Amazon EC2 security group to allow 443)
sudo service jetty start
If you happen to work with Jetty 9.3 then you should change configuration in start.d/ssl.ini:
jetty.sslContext.keyStorePath=mystore.jks
jetty.sslContext.keyStorePassword=X
jetty.sslContext.keyManagerPassword=X
jetty.sslContext.trustStorePath=mystore.jks
jetty.sslContext.trustStorePassword=X
Where:
mystore.jks is your store generated with the keytool
X is your password in plain text (I would recommend skipping obfuscation as it only gives you false security)
The store is exactly the same as you would generate for Tomcat. Even if you used different Java version to generate the keystore that should not be a problem.
When trying on Windows with Jetty as Maven plugin the following steps can help:
pom.xml
<plugin>
<groupId>org.mortbay.jetty</groupId>
<artifactId>jetty-maven-plugin</artifactId>
<version>8.1.11.v20130520</version>
<configuration>
<scanIntervalSeconds>10</scanIntervalSeconds>
<webApp>
<contextPath>/yourappcontext</contextPath>
</webApp>
<connectors>
<connector implementation="org.eclipse.jetty.server.nio.SelectChannelConnector">
<port>9090</port>
<maxIdleTime>1</maxIdleTime>
</connector>
<connector implementation="org.eclipse.jetty.server.ssl.SslSocketConnector">
<port>9443</port>
<keystore>src/test/resources/keystore</keystore>
<keyPassword>123456</keyPassword>
<password>123456</password>
</connector>
</connectors>
</configuration>
</plugin>
Generate key/certificate using the JDK tool keytool:
keytool -keystore keystore -alias jetty -genkey -keyalg RSA
This command will generate a file keystore which we need to put at the following (or what ever you like until it is configured in the keystore element) path src/test/resources/keystore.