I have a test cluster where I want to connect to ETCD pod. It does not let me because of
"error":"tls: first record does not look like a TLS handshake"
I am not sure how to implement TLS for my client. Is there a simple way to disable TLS on ETCD?
According to these docs ETCD
can be set to accept TLS traffic, non-TLS traffic, or both TLS and non-TLS traffic.
Using the auto-tls flag did not help.
Related
This might be done by issuing openssl s_client -connect with -no_tls1_2 -no_tls1_1 -no_tls1 -no_ssl3 -no_ssl2 but the result seems inconclusive to me. The listener is configured with SASL/SCRAM authentication enabled, but I am not sure if Encrypt data in transit Between clients and brokers Plaintext Enabled MSK cluster setting applies to that listener or not. Another way would be to connect to it and use wireshark to investigate traffic, if data is actually encrypted or not. The fact that I do not need to specify explicitly truststore does not mean it cannot use default java truststore and actually perform one way ssl handshake, when I give it such client settings:
security.protocol=SASL_SSL
sasl.mechanism=SCRAM-SHA-512
Right?
I guess I have found the answer:
Plaintext does not apply to the listener with SASL/SCRAM control method.
when i use HAproxy as load balancer, at HTTP termination mode and i tail log of it
(tail -f /var/log/haproxy.log). There are 2 types of log appearing
[time] frontend_name/1: SSL handshake failure
and
[time] frontend_name~ message
frontend_name is name follow frontend keyword config in /etc/haproxy/haproxy.cfg
I don't know what /1 and ~ in log message is, and why SSL handshake failure appearing at log has ~
Can someone help me explain and fix this error?
Thanks!
~ after frontend name means connection has been established using SSL/TLS
You can find reference to it in %ft entry in the table at: https://cbonte.github.io/haproxy-dconv/2.4/configuration.html#8.2.4
About /1 in frontend_name/1: SSL handshake failure:
I can't find it in the docs, but by experimenting i found it's the number of port in frontend, to which connection was attempted and SSL handshake failed.
For config:
frontend frontend_name
bind *:443,*:444 ssl crt <path_to_cert>
bind *:445 ssl crt <path_to_cert> no-tlsv13
If i make TLS1.3 connection to port 445 (e.g. openssl s_client -connect 127.0.0.1:445 -tls1_3), i will get:
frontend_name/3: SSL handshake failure
because 445 is 3. port listed in this frontend.
[UPDATE]
I found a bit more. Error log format explains that /1 in frontend_name/1 is bind_name and can be declared:
bind *:443,*:443 ssl crt <path_to_cert> no-tlsv13 name bind_ssl_foo
will result in frontend-name/bind_ssl_foo: SSL handshake failure.
Unfortunately we can't change error log format.
To learn more we have to make that connection successful and that most likely requires us to lower security (FOR DEBUGGING ONLY!). Normal clients will still negotiate highest security they can, TLS 1.2 or 1.3.
bind *:443 ssl crt <path_to_cert> ssl-min-ver TLSv1.0
Since haproxy 2.2 default for ssl-min-ver is TLSv1.2.
Second step is to log SSL version, negotiated cipher and maybe whole cipherlist send by client by appending %sslv %sslc and maybe %[ssl_fc_cipherlist_str] to your log-format:
log-format "your_log_format_here %sslv %sslc %[ssl_fc_cipherlist_str]"
If you don't have your own log format you can extend HTTP format:
log-format "%ci:%cp [%tr] %ft %b/%s %TR/%Tw/%Tc/%Tr/%Ta %ST %B %CC %CS %tsc %ac/%fc/%bc/%sc/%rc %sq/%bq %hr %hs %{+Q}r %sslv %sslc %[ssl_fc_cipherlist_str]"
To use ssl_fc_cipherlist_str we need to set tune.ssl.capture-cipherlist-size 800 in global section, because default is 0.
sslv is SSL/TLS version client connected with.
sslc is SSL/TLS cipher client connected with.
ssl_fc_cipherlist_str is cipher list client offered when negotiating SSL/TLS connection. It can be long. Use if you are extra curious.
That will append to your logs info like this:
TLSv1 ECDHE-RSA-AES256-SHA ECDHE-ECDSA-AES256-SHA,ECDHE-RSA-AES256-SHA,DHE-RSA-AES256-SHA,ECDHE-ECDSA-AES128-SHA,ECDHE-RSA-AES128-SHA,DHE-RSA-AES128-SHA,AES256-SHA,AES128-SHA,TLS_EMPTY_RENEGOTIATION_INFO_SCSV
Match by IP previous errors with current entries and you will know what TLS version and ciphers they were using. Then decide whether to adjust your ciphers or force this client to upgrade their SSL software.
So all required changes below:
global
log /dev/log daemon
tune.ssl.capture-cipherlist-size 800
frontend frontend_name
bind *:443 ssl crt <path_to_cert> ssl-min-ver TLSv1.0
log-format "%ci:%cp [%tr] %ft %b/%s %TR/%Tw/%Tc/%Tr/%Ta %ST %B %CC %CS %tsc %ac/%fc/%bc/%sc/%rc %sq/%bq %hr %hs %{+Q}r %sslv %sslc %[ssl_fc_cipherlist_str]"
mode http
(...)
Again, lower security only for debugging if this connection error really is a problem for you.
AWS NLB supports TLS termination
https://aws.amazon.com/blogs/aws/new-tls-termination-for-network-load-balancers/
NLB being a Layer 4 load balancer I would expect it to work in a passthrough mode by directing the incoming packets to one of the backends without much of state maintenance (except for the flow tracking)
Are there any details available on how AWS implements the TLS termination in NLB ?
Is it possible to do it with open source tooling (like IPVS or haproxy) or AWS has some secret sauce here ?
The TLS termination itself is just what it says it is. TLS is a generic streaming protocol just like TCP one level up so you can unwrap it at the LB in a generic way. The magic is that they keep the IPs intact probably with very fancy routing magic, but it seems unlikely AWS will tell you how they did it.
In my SO question here, I have an example of how to terminate a TCP session in HAProxy and pass the unencrypted traffic to a backend.
In short, you need to use ssl in the frontend bind section and both frontend and backend configurations require use of tcp mode. Here is an example of terminating on port 443 and forwarding to port 4567.
frontend tcp-proxy
bind :443 ssl crt combined-cert-key.pem
mode tcp
default_backend bk_default
backend bk_default
mode tcp
server server1 1.2.3.4:4567
How can I achieve reverse SSL termination with ha proxy?
From my backend via HAproxy I need to a https enabled web service. How can I successfully proxy all traffic to that service via HAProxy?
Below results in Unable to communicate securely with peer: requested domain name does not match the server's certificate.
frontend foofront
bind 127.0.0.1:443
mode tcp
default_backend foo
backend fooback
mode tcp
balance leastconn
server foo foo.bar.com:443 check
With HAProxy you usually have two options for handling TLS-related scenarios. TLS Passthrough and TLS Termination.
TLS Passthrough
Looks like you're trying to do this in the example you gave.
In this mode, HAProxy does not touch traffic in any way, but is just forwarding it to the backend. When TLS is involved, that means that the backend has to have a proper certificate for a domain it's accessed from - if your HAProxy is handling traffic for myexample.com, backend servers will need to have appropriate certificates for myexample.com installed.
You can always check which certificate is served by using openssl s_client:
openssl s_client -connect localhost:443
TLS Termination
Alternatively, you can terminate TLS traffic on HAProxy itself. This will allow you to use any backend (both encrypted and unencrypted). In this case, HAProxy itself decrypts traffic for myexample.com and forwards it to backend.
In your case, configuration would look something like:
frontend foofront
bind 127.0.0.1:80
bind 127.0.0.1:443 ssl crt /path/to/cert/for/myexample.com
mode tcp
default_backend foo
backend foo
mode tcp
balance leastconn
server foo foo.bar.com:443 check ssl verify none # or verify all to enforce ssl checking
You can find more info on both approaches here.
Hope this helps.
Problem: I have an application connecting to several databases (Redis, Riak, etc) via TCP through HAProxy. Like this:
app -> HAProxy --TLS--> HAProxy -> Redis
Access to HAProxy on Redis side is managed by Chef via iptables. We already use CFSSL for mutual TLS authentication on Nginx<->Nginx connections because it's way faster than waiting until Chef updates configuration.
Wanna make HAProxy on application side use client-side TLS certificate when connecting to HAProxy on DB side.
Actually any other proxy will do on client side but we'd like to keep things simple.