SSL Setup between Filebeat and Amazon MSK (Kafka) - ssl

I have successfully got Filebeat exporting logs to MSK in plaintext mode. When trying to do the same over SSL I’m getting error messages from the Filebeat side
Publish failed with circuit breaker is open
and SSL Handshake errors in the MSK Cloudwatch Logs.
I don’t need two way verification so I assume I just need to pass Filebeat the ACM-PCA from the MSK console however I still receive SSL handshake problems.
Could anyone please provide a sample Filebeat config for outputs to MSK.

You can create a private certifcate via the ACM Console.
Once done you can export the certificate from ACM and use the cert and key in your filebeat output.
Step One
certificate manager > request a certificate > request a private certificate > select your kafka private ca
For the DNS name wildcard your MSK host name, for example
*.my-kafka-cluster.h6q2kl.c4.kafka.eu-west-2.amazonaws.com
Once done request the certificate
Step Two
Export the Private certificate under actions in certificate manager, click through and you will get the cert, cert chain and key. You will only need the cert and key.
Step Three
Add your cert and key to your instance and configure your filebeat output to use the SSL certs.
(You may want to remove the password from the key) if not configure the filebeat keystore to avoid plaintext passwords
openssl rsa -in [original.key] -out [new.key]
output.kafka:
hosts: ["*******"]
topic: "*****"
partition.round_robin:
reachable_only: false
required_acks: 1
compression: gzip
max_message_bytes: 1000000
ssl.certificate: "/root/certs/kafka-broker.pem"
ssl.key: "/root/certs/kafka.key"
You should start to see your data in your topic after you restart filebeat

Related

Certbot SSL keys file names

I am setting up SSL for an IOT device that uses MQTT protocol acting as a client and connecting to a broker server. The server is using Certbot for SSL keys and registration. I configure the IOT device using AT commands and in the AT command I need the cacert, clientcert, and clientkey shown in the picture below. But I am wondering what are the names of these files on my server which is Ubuntu 20?
My best guess is one of the files below:
root#broker1:/etc/letsencrypt/keys# ls
0000_key-certbot.pem 0001_key-certbot.pem 0002_key-certbot.pem 0003_key-certbot.pem
or
root#broker1:/etc/letsencrypt/live/mydnsaddress# ls
README cert.pem chain.pem fullchain.pem privkey.pem
Unless you are using SSL Client authentication the only files you need on the client will be /etc/letsencrypt/live/<hostanme>/fullchain.pem
If you are using SSL Client authentication then you will need a certificate and private key specific to that client and that will not be issued by LetsEncrypt but by a private CA.

Get SSL certificate for Tableau

I can't connect with my Tableau Server from Tableau Desktop because it shows me that error: SSL certificate is not valid. I'm searching for a solution, but I'm only reading how to change the certificate with another, beacause I do not have any other certificate.
My question goes before that: how can I get a certificate? where I can get it from?
This is the guide from Tableau I use whenever I need to setup SSL for my servers. It includes obtaining an SSL Certificate from a certificate authority Steps for SSL Cert
To configure Tableau Server to use SSL, you must have an SSL certificate. To obtain the SSL certificate, complete the steps:
Set the OpenSSL configuration environment variable (optional)
Generate a key file
Create a Certificate Signing Request (CSR)
Send the CSR to a certificate authority (CA) to obtain an SSL certificate
Use the key and certificate to configure Tableau Server to use SSL

Re-autosign certificates in Puppet CA

I'm using an autosign script in Puppet to sign certificates. This is actually working, but I'm experimenting a problem when a machine try to request again a certificate (eg. ssl directory is removed). In this cases, a cached certificate is used and obviously does not match with the certificate created by the agent. Here is an example output:
Info: Creating a new SSL key for foo.bar.com
Info: Caching certificate for ca
Info: Caching certificate for foo.bar.com
Error: Could not request certificate: The certificate retrieved from the master does not match the agent's private key.
Certificate fingerprint: 41:B7:ED:3C:EC:A9:EF:A9:51:8C:6C:46:94:B1:30:09:72:2F:CC:D2:13:BA:A5:63:A7:2D:C5:FB:BD:DF:A5:B4
I don't want/can't remove certificates by hand executing puppet cert clean so I tried to use allow_duplicate_certs but seems to be buggy since... quite a lot time.
Do you know any other option to re-autosign a certificate when the host already have a signed certificate in the CA?

Installing SSL cert on Amazon EC2 ELB

I'm new to Amazon Web Services and have been trying to install a SSL certificate on the EC2 instance. I tried following AWS documentation, but found it baffling. I then followed the guide at http://www.robertbrewitz.com/2014/09/aws-and-setting-up-a-custom-ssl-certificate/.
I bought my SSL cert with Go Daddy, and generated 2 files using openssl:
server.key
server.csr
The guide said I should expect 3 certificates:
DigiCertCA.crt
TrustedRoot.crt
star_yourdomain_com.crt
Instead, I confusingly received 2 files named:
f6f65901b1708ae5.crt
gd_bundle-g2-g1.crt
I assume f6f65901b1708ae5.crt is my domain cert (but I'm not certain). Anyway, the guide says I need an Elastic Load Balancer in order to install an SSL cert so I created one.
I generated the private key with:
openssl rsa -in server.key -text
and the public key certficate with:
openssl x509 -inform PEM -in f6f65901b1708ae5.crt
I was also required to enter a Certificate Chain. I wasn't sure what this was and how to get it, so I guessed the command:
openssl x509 -inform PEM -in gd_bundle-g2-g1.crt
and entered the resulting certificate key that began with “-----BEGIN CERTIFICATE-----”
The guide continues, saying I then need to set up Cloudfront. I installed the aws command line tool and in order to generate the PEMs I ran:
openssl rsa -in server.key -text > aws_private.pem
openssl x509 -inform PEM -in f6f65901b1708ae5.crt > aws_public.pem
openssl x509 -inform PEM -in gd_bundle-g2-g1.crt > aws_public.pem
I uploaded the SSL cert with:
aws iam upload-server-certificate --server-certificate-name mydomain_com \
--certificate-body file://aws_public.pem --private-key file://aws_private.pem \
--certificate-chain file://aws_chain.pem --path /cloudfront/mydomain_com/
This was successful.
I then had to create a Cloudfront distribution, which I did, choosing the SSL cert.
However when I go to my https url (https://www.example.org/), it's not working. http://www.example.org/ however does work.
As there are an extraordinarily large amount of steps just for installing an SSL cert, I suspect I've made a mistake along the way. The problem is, I don't know where. Has anybody any pointers?
Also, are there not any simpler ways for installing an SSL cert? It seems to be ridiculously complex for something so common. I'd be willing to pay an expert to do it for me (I'm a software developer with almost no knowledge of anything SSL related), but its difficult to find anyone for such a task (and there is the problem of having to hand over login details etc). Any help greatly appreciated.
EDIT
The suggestion below is that I should use AWS Certificate Manager. I've had a look and this seems like a far more painless option. However, I did spend 86 euro on an SSL cert from Go Daddy so I'd prefer if that didn't go to waste. Is any of my work salvageable? Are there mean to resell SSL certs?
EDIT
I still haven't found a real solution to this. To clarify, I have a very niche site that will have very few visitors. I have the site on a EC2 instance. I followed the site above which advised using a Load Balancer and Cloudfront in order to encrypt with SSL. However, it's not working and it's probably overkill anyway. Can anyone help me with this? I'd like to use the SSL cert I paid for, but if not, should I use something like Lets Encrypt?
You mentioned the guide saying to expect 3 files, including a "DigiCertCA.crt" file; sounds like it was written with DigiCert as your cert provider in mind, rather than GoDaddy.
Certificates
First, to make sure that that "f6f65901b1708ae5.crt" contains your requested certificate (and lay to rest any doubts there). To do this, you can compare the data (e.g. Common Name (CN), DNS Subject Alternative Names (SANs), etc) in your "server.csr" file (that's the Certificate Signing Request) with what's in that "f6f65901b1708ae5.crt" file:
$ openssl req -noout -text < server.csr
This should display human-readable text about the details for your domain in the CSR file. Compare that with:
$ openssl x509 -noout -text < f6f65901b1708ae5.crt
This should display similar human-readable text, with more details/fields filled in. But they should roughly have what you expect. Note that if you see an error similar to this:
51299:error:0906D06C:PEM routines:PEM_read_bio:no start line:/SourceCache/OpenSSL098/OpenSSL098-52.40.1/src/crypto/pem/pem_lib.c:648:Expecting: TRUSTED CERTIFICATE
Then it suggests that your "f6f65901b1708ae5.crt" file is in DER format, not PEM format. If not, then you already have a PEM file, which is what AWS ELBs expect. If you have a DER formatted cert, it's easy to convert to PEM format, using:
$ openssl x509 -in f6f65901b1708ae5.crt -inform DER -out f6f65901b1708ae5.pem -outform PEM
I just wanting to be thorough by mentioning this part.
Assuming, now, that we know that that "f6f65901b1708ae5.crt" contains the PEM formatted certificate for your domain, we're ready to handle the "certificate chain" part.
I looked at GoDaddy's online certificate repository, to see if the "gd_bundle-g2-g1.crt" file you mentioned was there, and it was. (It's OK for these files to be publicly available, as they contain the public certificates which are meant for anyone/everyone to use.) Looking at that gd_bundle-g2-g1.crt file, I found that it contains multiple certificates. This is important.
See, a "certificate chain" is a list of files which provide a trust path (or "chain"), from that "f6f65901b1708ae5.crt" certificate you have to the trusted root GoDaddy CA certificate. Each certificate has a subject (who it was issued to), and an issuer (who issued it). This means that you can walk "backward", from your certificate to the issuer's certificate, to that certificate's issuer's certificate, etc. This walking backward is the "certificate chain".
The fact that that "gd_bundle-g2-g1.crt" file contains multiple certificates implies that that file contains the certificate chain you need. It also means that you do not want to do this:
$ openssl x509 -inform PEM -in gd_bundle-g2-g1.crt > aws_public.pem
because openssl x509 only reads the first certificate in the specified file, and you need all of them.
Given all of the above, you might need only the following (to make sure that your private key is PEM formatted):
$ openssl rsa -in server.key -text > aws_private.pem
And then, since we'll assume that "f6f65901b1708ae5.crt" is already PEM formatted (and if not, you know how to convert it above), and we know that "gd_bundle-g2-g1.crt" is already PEM formatted, we'll now be ready to upload these to the AWS ELB.
AWS ELB
To upload the certificates and key for use by the ELB, use something like this:
$ aws iam upload-server-certificate \
--server-certificate-name redmatterapp_com2 \
--certificate-body file://f6f65901b1708ae5.crt \
--private-key file://aws_private.pem \
--certificate-chain file://gd_bundle-g2-g1.crt \
--path /cloudfront/redmatterapp_com/
Note that I used a different --server-certificate-name, just to make sure it didn't override/conflict with your existing configuration. As a suggestion, you might include the date of when the cert being uploaded was created (or, better, when it will expire), as part of the name, as a hint to your future self of when that cert was added, e.g. "redmatterapp_com-2016-02-18".
Also note that if you are not using CloudFront, then you should not use the --path option. If you were using CloudFront and then removed it, I'd highly recommend doing the above aws iam upload-server-certificate command again, only with a different --server-certificate-name and no --path option (and removing the previous name). This may mean re-configuring any existing ELB HTTPS listeners to use the new certificate name, but it may be necessary, as that --path affects SSL handling.
Once the above is done, using the e.g. AWS Console, you should be able to configure your AWS ELB, and under the "Listeners" tab, click "Edit". Add a Listener e.g. for "https". When adding any SSL-capable listener, you'll see a "Change" link appear under the "SSL Certificate" column/tab. Click that "Change", and choose the "existing certificate" button. Under the "Certificate Name" dropdown, then, you should see an entry for the --server-certificate-name string/label you used above. Choose that entry, then click "Save". Now connections to that Listener, on your AWS ELB, should be properly configured for SSL/TLS connections that browsers will trust.
Thus the HTTPS Listener configuration would look something like:
Load Balancer Protocol: HTTPS
Load Balancer Port: 443
Instance Protocol: HTTP
Instance Port: 80
Cipher: (default policy)
SSL Certificate: (--server-certificate-name name)
Note that you do not want to use port 443 for the instance port as well; if you do, that says that you want HTTPS from the ELB to your instance, which is not usually needed. (Some secure sites want this, but that's a different topic/store.) Thus the above configures the ELB to handle the SSL termination; to your Node.js server on the instance, it only ever receives plain HTTP requests on port 80:
client ---> HTTP ---> ELB port 80 ---> HTTP ---> server port 80
client ---> HTTPS --> ELB port 443 --> HTTP ---> server port 80
If your server needs to know whether the original request was HTTP or HTTPS, look for the X-Forwarded-For (and other request headers) that the AWS ELB will automatically add to the request.
Now, once you have the ELB configured, it's a good step to verify that it's working properly. First, you can use openssl s_client to verify that the SSL handshake works:
$ openssl s_client -connect example.com:443
CONNECTED(00000003)
depth=3 /C=US/O=The Go Daddy Group, Inc./OU=Go Daddy Class 2 Certification Authority
verify error:num=19:self signed certificate in certificate chain
verify return:0
---
Certificate chain
0 s:/OU=Domain Control Validated/CN=redmatterapp.com
i:/C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com, Inc./OU=http://certs.godaddy.com/repository//CN=Go Daddy Secure Certificate Authority - G2
1 s:/C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com, Inc./OU=http://certs.godaddy.com/repository//CN=Go Daddy Secure Certificate Authority - G2
i:/C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com, Inc./CN=Go Daddy Root Certificate Authority - G2
2 s:/C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com, Inc./CN=Go Daddy Root Certificate Authority - G2
i:/C=US/O=The Go Daddy Group, Inc./OU=Go Daddy Class 2 Certification Authority
3 s:/C=US/O=The Go Daddy Group, Inc./OU=Go Daddy Class 2 Certification Authority
i:/C=US/O=The Go Daddy Group, Inc./OU=Go Daddy Class 2 Certification Authority
---
Server certificate
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
subject=/OU=Domain Control Validated/CN=example.com
issuer=/C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com, Inc./OU=http://certs.godaddy.com/repository//CN=Go Daddy Secure Certificate Authority - G2
---
No client certificate CA names sent
---
SSL handshake has read 4929 bytes and written 456 bytes
---
New, TLSv1/SSLv3, Cipher is AES128-SHA
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
Protocol : TLSv1
Cipher : AES128-SHA
...
Timeout : 300 (sec)
Verify return code: 0 (ok)
---
The above shows that we successfully completed an SSL handshake, and you can see that it shows the cert chain with the GoDaddy names; the "Server certificate" section should match the "f6f65901b1708ae5.crt" PEM file you loaded.
HTTPS and DNS
Now to test the HTTPS part; for this, I like to use curl, like so:
$ curl -kv https://example.com/
The -v option display more information, in particular whether the certificate matches the DNS name or not; the -k tells curl to ignore any certificate mismatches/issues, so that we can then see the verbose information about things.
This is where a little more configuration is necessary. If you use the auto-generated ELB DNS name in your curl command (or in your browser), then you will most likely see security warnings/issues. Why? Because part of HTTPS is verifying that the DNS name you use in your URL matches the domain/host name in your server certificate, either as the Common Name (CN) in your certificate, or as one of the DNS Subject Alternative Names (SANs). And you probably didn't include the ELB DNS name in your Certificate Signing Request (CSR) to GoDaddy.
This means, then, that the next step is to configure a DNS CNAME record that points to the AWS ELB name, e.g.:
www.example.com CNAME to aws-elb-1.elb.amazonaws.com
If you're using AWS for DNS, then you could do this using e.g. Route 53. Then you retry your curl command (or browser):
curl -kv https://www.example.com/
Another important thing to watch for (and you can see this in the curl -v output) is the contents of the Host request header. Some HTTP servers (i.e. the one running on your backend instance) are very picky about the value there; they want the Host header to match something in their configuration, and if it doesn't, they may refuse the request.
Another common gotcha is:
HTTP 503 Service Unavailable: Back-end server is at capacity
If you get this response from your ELB, it usually means that your backend servers are not responding or are failing to respond to the ELB healtcheck. Double-check that the port and path/URL used for the ELB healtcheck are correct, and that any firewalls or AWS Security Groups allow connectivity from the ELB to your instances.
So, now you should have an ELB which forwards port 80 (HTTP) and port 443 (HTTPS) to your backend instances. And you have a CNAME record for "www.example.com" in DNS that points to that ELB name. What's left?
HTTP Redirects
I highly recommend configuring your HTTP server to always redirect to the HTTPS equivalent of the same URL. Why?
For security of the data between your clients and your server, using SSL/TLS is now expected. So much so that more and more browsers automatically try HTTPS first, and only begrudgingly use HTTP as a fallback. Browsers like Chrome (and others) want to avoid this HTTP fallback so much so that they've introduced mechanisms like HTTP Strict Transport Security: a way for your site to tell the browser to only ever use HTTPS for the site, never HTTP. Plus it's always a better marketing story; in fact, you can expect to get negative reactions from your clients/users if you don't use HTTPS.
There's another thing that using HTTPS for all your site's traffic protects you from: someone else using their DNS name and your ELB. You have a CNAME record for "www.example.com" that points to "aws-elb-1.elb.amazonaws.com". But what if I also create my own CNAME for "www.evilco.com", which points to the same "aws-elb-1.elb.amazonaws.com"? Folks going to "www.evilco.com" will see your site, and think it's mine!
By forcing all traffic for your site to be HTTPS, you force all of the HTTPS clients to verify that the certificate presented by the server (i.e. your "f6f65901b1708ae5.crt" file) contains a Common Name (CN) or DNS Subject Alternative Name (SAN) that matches the domain name the client used in its URL. Obviously your certificate does not contain CN or DNS SANs for "www.evilco.com", and thus that verification process would fail -- and the end user would see that something is fishy. But if you allow HTTP traffic as well for your site, this phenomenon could happen -- and you, looking at your site's logs, would never know!
Postscript
I've not used the AWS Certificate Manager, or CloudFront, myself (so I can't comment on them), but I have used the above process, multiple times, for various certificate provisioning of multiple domains, at multiple jobs.
Hope this helps!

SSL client verification failure

I have a backend server with SSL only as defaut public interface. This server listens on 443 with cert, key and ca files on, ssl verify client is set to true.
On client side, I have client cert, key and a ca file made of 2 CA files catted together (intermediate CA, root CA) in a single ca.pem file. When I do curl request on my backend server, it works fine. I also tested the server / client cert and key with gnutls-serv and openssl s_server to ensure all was valid.
But when I create a keystore on Apigee (client cert and key file [pem format]) and a trustore (ca.pem file), there is an error:
- target.name fsbca-test
- Properties
- Expression ("fsbca-test" equals target.name)
- ExpressionResult true
- Tree TARGET_fsbca-test
- error The Service is temporarily unavailable
- error.cause General SSLEngine problem
- error.cause.cause General SSLEngine problem
- error.class com.apigee.messaging.adaptors.http.HttpAdaptorException
- state TARGET_REQ_FLOW
- type ErrorPoint
If I put SSL verify client to false on my backend, then the request is correctly diverted by Apigee and I get the response.
If I put IgnoreValidationErrors to true in the target endpoints property, then the request is diverted to my backend server but I can see an error in the server's log: "client sent no required SSL certificate while reading c...".
Any ideas about what could be wrong in what I am doing?
Additional track: could it be an issue on Apigee side with the CA file made of 2 certs (it may ignore trailing certificates found in a .pem). If wanted to test pkcs12 and jks but I failed to upload them to Apigee (the API doc page only describes .pem, JAR and cert action). I wrote a small Java client with pkcs12 keystore and jks trustore and it worked fine from my local workstation.
Thank you in advance for any piece of information that could help me.
Regards
Fr
You should upload your certificates separately (one per cert), and you need the entire trust chain of certificates to be stored in your truststore.
Here is the page about SSL to your backend.
EDIT:
Here is a method I know works:
1) Separate certs into separate PEM files. Do not put more than once cert in a file.
2) Validate each cert using openssl:
openssl x509 -noout -text -in <cert file name>
Validate that no certs are expired, and that the Subject and Issuer fields create a chain of all the certs, with identical names.
The server's certificate, the root certificate, and all certificates in between need to be in the truststore.