I'd like to set up Spark Standalone's Web UI so it can be accessed through HTTPS.
Spark is being run on a cluster external to the computer I'm using to access the browser.
Here's what I've done so far, and it's not working:
Used OpenSSL to make a self-signed certificate and key and then keytool to make a keystore
Packaged the certificate and key as a .p12 file
In Chrome's settings under Manage Certificates, imported said p12 file
In spark-defaults.conf, added options
spark.ui.https.enabled true
spark.ui.ssl.server.keystore.location /path/to/spark.keystore
spark.ui.ssl.server.keystore.keypassword password
spark.ui.ssl.server.keystore.password password
In spark-env.sh, added
export SPARK_MASTER_OPTS="-Dspark.ui.https.enabled=true \
-Dspark.ui.ssl.server.keystore.location=/path/to/spark.keystore \
-Dspark.ui.ssl.server.keystore.keypassword=password \
-Dspark.ui.ssl.server.keystore.password=password"
export SPARK_WORKER_OPTS="-Dspark.ui.https.enabled=true \
-Dspark.ui.ssl.server.keystore.location=/path/to/spark.keystore \
-Dspark.ui.ssl.server.keystore.keypassword=password \
-Dspark.ui.ssl.server.keystore.password=password"
I've tried to connect to the server before, after, and in between each of these steps, and I keep getting the error "This site can’t provide a secure connection". What am I missing here?
According to this line I think that it's not possible to set up the Spark Standalone's web UI with HTTPS.
masterWebUiUrl = "http://" + masterPublicAddress + ":" + webUi.boundPort
My recommendation is to file an issue in Spark's JIRA and find a Spark developer to fix it.
Below config worked for me, try putting this in "spark-defaults.conf" and restart the Spark service. Also check logs for which port Spark UI is listening on, as "spark.ssl.ui.port" is set to "0". In my case, it was running on port 8480.
spark.ssl.enabled true
spark.ssl.ui.port 0
spark.ssl.keyStore <path_to_keystore>
spark.ssl.keyStorePassword <keystore_password>
spark.ssl.keyPassword <key_password>
spark.ssl.trustStore <path_to_truststore>
spark.ssl.trustStorePassword <truststore_password>
spark.ssl.enabledAlgorithms ECDHE-RSA-AES256-SHA,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_DHE_RSA_WITH_AES_256_CBC_SHA256
spark.ssl.protocol TLSv1.2
spark.ssl.trustStoreType JKS
See the screenshot for reference.
Related
I'm a little bit confused about https communication with influxdb. I am running a 1.8 Influxdb instance on a virtual machine with a public IP. It is an Apache2 server but for now I am not willing to use it as a webserver to display web pages to clients. I want to use it as a database server for influxdb.
I obtained a valid certificate from Let's Encrypt, indeed the welcome page https://datavm.bo.cnr.it works properly over encrypted connection.
Then I followed all the instructions in the docs in order to enable https: I put the fullchain.pem file in /etc/ssl directory, I set the file permissions (not sure about the meaning of this step though), I edited influxdb.conf with https-enabled = true and set the path for https-certificate and https-private.key (fullchain.pem for both, is it right?). Then, systemctl restart influxdb. When I run influxdb -ssl -host datavm.bo.cnr.it I get the following:
Failed to connect to https://datavm.bo.cnr.it:8086: Get https://datavm.bo.cnr.it:8086/ping: http: server gave HTTP response to HTTPS client
Please check your connection settings and ensure 'influxd' is running.
Any help in understanding what I am doing wrong is very appreciated! Thank you
I figured out at least a part of the problem. It was a problem related to permissions on the *.pem files. This thing looks weird because if I tip the following, as documentation says, it does not connect.
sudo chmod 644 /etc/ssl/<CA-certificate-file>
sudo chmod 600 /etc/ssl/<private-key-file>
If, instead, I tip the second line with 644 all works perfectly. But this way I'm giving to anyone the permission to read the private key! I'm not able to figure out this point.
UPDATE
If I put inside /etc/ssl/ the symlinks that point to the .pem files that live inside /etc/letsencrypt/live/hostname the connection is refused. Only if I put a copy of the files the ssl connection starts.
The reason I am willing to put the links inside /etc/ssl/ is the automatic renew of the certificates.
Anyone can help?
I am trying to send my app to a Google Cloud Cluster using the kubectl command behind a corporative proxy that needs a certificate ".crt" file to be used when doing HTTPS requests.
I already ran the gcloud container clusters get-credentials... command and it also asked for a certificate. I followed the given instructions by Google and I configured my certificate file without any issue and it worked.
But when I try the kubectl get pods I am getting the following message:
"Unable to connect to the server: x509: certificate signed by unknown authority"
How can I configure my certificate file to be used by the kubectl command?
I did a search about this subject but I found too difficult steps. Could I just run something like this:
kubectl --set_ca_file /path/to/my/cert
Thank you
The short answer up to what I know is no.
here[1] you can see the step by step of how to get this done in the easiest way I found so far, is not a one line way but is the closest to that.
after having your cert files you need to run this:
gcloud compute ssl-certificates create test-ingress-1 \ --certificate [FIRST_CERT_FILE] --private-key [FIRST_KEY_FILE]
then you need to create your YAML file with the configuration (in the link there are two examples)
run this command:
kubectl apply -f [NAME_OF_YOUR_FILE].yaml
[1] https://cloud.google.com/kubernetes-engine/docs/how-to/ingress-multi-ssl
I am working on setting up ssl to secure my end points. I got a test certificate created from my org
I have recieved a .pfx file
I converted it into .pem -----> .der -------> .jks format
We have basic infrastructure to put this .jks file in a folder called ssl and it gets picked up just by using a confg file.
Next I set up Jmeter to test this. Steps Followed
set up a test recorder --> set up an http get request that takes no parameters
2.changed protocol to https ---> given port no ---> set up host and path. This is correct as I have tested it with http and it returns fine.
Now when I try to test it I get an error Certificate_Unknown error.
I have tried searching the internet and StackFlow articles about testing SSL. I also stumbled upon an article which says I need to add the certificate to my java_home cacerts. I havenot been able to successfully test it. Any pointers to what I might be doing wrong or if I could test it in some other way would be very helpful.
I am comparatively new to SSL concepts and just learnt about formats, ssl etc.
Thanks in advance. :)
You don't need to convert .pfx file into .jks as .pfx files are basically PKCS 12 certificates and JMeter supports them out of the box
I fail to see where you "tell" JMeter to use the certificate. If your " basic infrastructure to put this .jks file in a folder called ssl and it gets picked up just by using a confg file" stanza is related to JMeter - you should address this question to the "infrastructure" providers. Otherwise you need to explicitly configure JMeter to use the certificate. Just add the next lines to system.properties file:
javax.net.ssl.keyStoreType=pkcs12
javax.net.ssl.keyStore=/path/to/certificate.pfx
javax.net.ssl.keyStorePassword=your certificate password
JMeter restart will be required to pick the properties up.
Firstly my exposure and experience with certificates is limited so I am trying to use this as a learning experience as well as obtaining an answer.
Using Jenkins I would like to be able to make a curl request as part of my build that uses a certificate (.p12) to authenticate.
Example
curl --cert /Users/Jenkins/.jenkins/workspace/develop-pipeline/../certificates/dev_cert.p12:password https://jira.dev.organisation.co.uk:443/rest/api/2/issue/MYSTATS-1234
So I have a Desktop certificate (private key) which I have saved onto the machine where Jenkins runs, but I also understand that I need a CA certificate to authorise this private key (hope this is correct so far).
When I have run the curl command from the terminal on the Jenkins machine I had a popup that asked me did I want to use the cert within the keychain to authorise and I clicked always allow, so command line curl requests always work.
However when I run this as part of a Jenkins build I get
(58) SSL: Can't load the certificate "/Users/Jenkins/.jenkins/workspace/develop-pipeline/../certificates/dev_cert.p12" and its private key: OSStatus -25308
The path to the cert is correct as the directory structure is
develop-pipeline
certificates
dev_cert.p12
What I don't understand is that if I don't provide a password
curl --cert /Users/Jenkins/.jenkins/workspace/develop-pipeline/../certificates/dev_cert.p12 https://jira.dev.organisation.co.uk:443/rest/api/2/issue/MYSTATS-1234
I get the message:
SSL The certificate "/Users/Jenkins/.jenkins/workspace/develop-pipeline/dev_cert.p12" requires a password
So it has found the certificate ?
Also providing an incorrect password yields
SSL: Incorrect password for the certificate "/Users/Jenkins/.jenkins/workspace/develop-pipeline/dev_cert.p12" and its private key.
What do I need to do to get this working?
This Github comment helped me sorting out the curl issue on OSX. So in my case installing curl via Homebrew with OpenSSL flags did the trick.
I am trying to deploy my web app remotely using the tomcat plugin from maven (mvn tomcat:deploy).
The Tomcat App Server Manager is at https://MyRemoteTomcatInstance/manager.
Firstly, I added the certificate to my trust store with this command:
keytool -v -alias mavensrv -import -file /Users/christianachilli/Desktop/KubePartnersLtd.p12 -keystore /Users/christianachilli/trust.jks
The file KubePartnersLtd.p12 is the result of exporting the site certificate with the Browser (I used FF 3.6).
Secondly, I added the
.mavenrc
file and here comes the pain for me! I am not sure of what should I put in it. At the moment I filled it in with:
MAVEN_OPTS="-Xmx512m -Djavax.net.ssl.trustStore=/Users/christianachilli/trust.jks \
-Djavax.net.ssl.trustStorePassword=123456 \
-Djavax.net.ssl.keyStore=/Users/christianachilli/.keystore \
-Djavax.net.ssl.keyStoreType= \
-Djavax.net.ssl.keyStorePassword="
This gives me an error:
No subject alternative names present
I tried many other combination using the brute force pattern but I couldn't succeed. I'd really appreciate if someone could point me in the right direction about what should be the parameters in
MAVEN_OPTS
Thanks and regards,
C
Check which CN is KubePartnersLtd.p12 for, e.g. localhost in the following example:
$ openssl verify tomcat-p12.crt
tomcat-p12.crt: /C=Unknown/ST=Unknown/L=Unknown/O=Unknown/OU=Unknown/CN=localhost
[...]
Make sure that such CN matches the DNS name you're using in the tomcat plugin configuration. In case they're not the same, you might consider adding an entry to your hosts file.