I'm trying to haproxy work in ssl mode with 2 tomcat instances on different VMs (see my previous question: HAProxy load balancing with tomcat SSL).
Currently, I managed to get haproxy to work with the following configuration:
defaults
log 127.0.0.1 local0
option tcplog
frontend ft_test
mode http
bind 0.0.0.0:8443 ssl crt haproxy.pem
timeout client 120000ms
default_backend bk
backend bk
mode http
cookie JSESSIONID prefix
server s1 vm1:80 cookie JSESSIONID_SERVER_1 check maxconn 32
server s2 vm2:80 cookie JSESSIONID_SERVER_2 check maxconn 32
timeout connect 5000ms
timeout server 120000ms
haproxy.pem is a file I copied from here (for testing purposes): http://blog.exceliance.fr/2012/09/04/howto-ssl-native-in-haproxy/
When I change the port from 80 to 8443 on the VMs, and I try to access https://haproxy.vm.ip:8443 (same as I did with the 80 port), I get a: 502 Bad Gateway error in my browser.
I'm guessing this has something to do with the ssl certificates, but I'm not sure how to solve this (these issues are new to me, and I wasn't able to find much information).
The server.xml for each tomcat VM (they're the same):
<Connector port="8443" protocol="HTTP/1.1" SSLEnabled="true"
maxThreads="150" scheme="https" secure="true"
clientAuth="false" sslProtocol="TLS"
keystoreFile="${catalina.home}/conf/keystore"
keystorePass="password" />
Do I need to use the same certificate here as in the haproxy configuration? I tried that but it doesn't seem to work, and I didn't find an example for haproxy with a keystore file either.
So, help? Ideas? Thanks! :)
Had the same issue -
I was never able to get 1.5 to correctly and consistently process SSL traffic without receiving intermittent 502's and 503's.
We ended up using stunnel - http://www.stunnel.org/
Very simple to setup and configure. HAproxy just responds on port 80 and forwards over 443 so that internal traffic remains encrypted.
Here's a sample stunnel.conf file to start from:
sslVersion = all
options = NO_SSLv2
cert=/etc/stunnel/wildcard.pem
setuid = root
setgid = root
pid = /var/run/stunnel.pid
socket = l:TCP_NODELAY=1
socket = r:TCP_NODELAY=1
output = /var/log/stunnel.log
[service1]
accept = 10.0.0.10:443
connect = 10.0.0.10:80
TIMEOUTclose = 0
[service2]
accept = 10.0.0.11:443
connect = 10.0.0.11:80
TIMEOUTclose = 0
Related
I'm implementing a Frontend Loadbalancer which passthrough the traffic coming to port 80 and 443 to different backend ports. SSL termination is happening in the backend and HAproxy should not engage with anything other than forwarding the traffic coming to the frontend port 80 and 443 to the respective backend ports.
Port 80 forwarding seems fine and 443 is not working as expected and giving SSL handshake failure. Even my backend service is not coming up on the web browser with a warning saying this is not trusted. I have no clue why this is happening and my HAProxy experience is not that high and below is the current configuration. Please correct me if I'm wrong.
HAProxy is installed on Ubuntu 18.04.5 LTS
Config after the defaults section
frontend k8s_lb
mode tcp
bind x.x.x.x:80
default_backend kube_minions
frontend k8s_lb_https
mode tcp
bind x.x.x.x:443
default_backend kube_minions_https
backend kube_minions
mode tcp
balance roundrobin
server k8s_worker-01 x.x.x.x:32080
server k8s_worker-02 x.x.x.x:32080
backend kube_minions_https
mode tcp
balance roundrobin
server k8s_worker-01 x.x.x.x:32443
server k8s_worker-02 x.x.x.x:32443
The backend story:
I have a k8s cluster and traefik ingress which is running as a DaemonSet on each and every node, and minions are my backend servers. CertManager is in place to do the cert automation with Let's encrypt ACME protocol in the ingress resources, hence SSL termination should be happening through the ingress resources.
I have completed the certificates and everything seems perfect as I have already implemented a similar setup on AWS with a TCP loadbalancer and everything is perfectly working and running prod workloads.
So, I need to mention that backend services are all good and up and running. In this I replaced the AWS loadbalancer with HAProxy and need to implement the same.
Please assist me to fix this as I'm struggling with this and still no luck with the issue.
Thank you.
Sorry, I was able to figure it out and there is nothing to do with traefik and HAProxy for this SSL issue. My Client's DNS is configured in CloudFlare and they have enabled the universal SSL and it caused the issue.
I checked with a new DNS record from route53 working as expected so my HAProxy config do what I need.
I have successfully configured Apache (2.4.7) to require a client certificate and -- as a reverse proxy -- forward the information within the certificate to a Tomcat 8 server.
When trying to accomplish the same with Spring Boot however, it fails with
The proxy server received an invalid response from an upstream server
The proxy server could not handle the request GET /myapp
and returns a HTTP 502 error code.
The relevant, working Tomcat configuration is:
<Connector SSLEnabled="true" clientAuth="want" keyAlias="myalias"
keystoreFile="mystore.jks" keystorePass="mypassword" maxThreads="150"
port="8443" protocol="HTTP/1.1" scheme="https" secure="true"
sslProtocol="TLS" truststoreFile="mystore.jks" truststorePass="mypassword"/>
The relevant part of the Spring Boot application.properties file that won't work:
server.context-path=/myapp
server.port=8443
server.ssl.enabled=true
server.use-forward-headers=true
server.ssl.protocol=TLS
server.ssl.client-auth=need
server.ssl.key-alias=myalias
server.ssl.key-store=/path/to/mykeystore.jks
server.ssl.key-store-password=mypassword
server.ssl.key-password=mypassword
server.ssl.trust-store=/path/to/mykeystore.jks
server.ssl.trust-store-password=mypassword
server.tomcat.remote-ip-header=x-forwarded-for
server.tomcat.port-header=x-forwarded-port
Note, when accessing the app directly (i.e. requesting https://myapp.company.tld:12345/myapp) it works just fine, but using the reverse proxy (i.e. https://proxy-load-balancer.company.tld:12345/myapp) throws the error above.
The port difference (12345 vs the configured 8443) is due to an intermediate Docker layer: both the reverse proxy and the application runs in a container and their open ports (443 for Apache, 8443 for Tomcat/Spring Boot) are mapped to a different port, i.e. 12345.
Ok, so this all got sorted out. First, this Spring security setting
server.ssl.client-auth=need
will always force your embedded Tomcat to ask for a certificate, so in a client=>reverse proxy=>Tomcat situation you won't be able to authenticate unless maybe you use AJP, not sure about that.
But it turned out that using
server.ssl.client-auth=want
enables the further processing of your request and what people usually do is process the certificate at the reverse proxy level and forward some information to the backend server (Tomcat, Jetty, etc.).
In the end the developers had to adapt their Spring Boot application to handle this latter mode of operation, i.e. extract data from the forwarded HTTP request's header and proceed with the authentication based on that.
I used certbot to generate a Let's encrypt certificate for my website, but Yaws gives me an SSL accept failed: timeout error when I try to connect to it (after it times out of course). Interestingly it works when I redirect example.com to the local ip address of the server in the hosts file on my machine and connect to example.com:8080, but not when I connect to example.com without editing the hosts file or when I connect from my phone over 4G. Here's my webserver's configuration file (it is the only configuration file in conf.d):
<server www.example.com>
port = 8080
listen = 0.0.0.0
docroot = /usr/share/yaws
<ssl>
keyfile = /etc/letsencrypt/live/example.com/privkey.pem
certfile = /etc/letsencrypt/live/example.com/fullchain.pem
</ssl>
</server>
I made sure that the keyfile and the certificate are both readable by the yaws user. Next to the keyfiles is a README that contains the following:
`privkey.pem` : the private key for your certificate.
`fullchain.pem`: the certificate file used in most server software.
`chain.pem` : used for OCSP stapling in Nginx >=1.3.7.
`cert.pem` : will break many server configurations, and should not be used
without reading further documentation (see link below).
We recommend not moving these files. For more information, see the Certbot
User Guide at https://certbot.eff.org/docs/using.html#where-are-my-certificates.
So I'm relatively sure I've used the right file (the other ones gave me errors like badmatch and {tls_alert,"decrypt error"}). I also tried trivial things like writing https:// before the URL, but it didn't fix the issue, also, everything works fine when the server is running without SSL. The version of Erlang running on my server is Erlang/OTP 19. Also, if it's unclear, the domain isn't actually example.com.
Also, example.com is redirected via cname to examplecom.duckdns.org, if that matters.
UPDATE:
My server was listening on port 8080, that was forwarded from the external port 80, for https connections, when the default https port is port 443. My other mistake was connecting to http://example.com instead of https://example.com. Forwarding the external port 443 to the internal port 8443 and configuring yaws to listen on port 8443 fixed everything.
Just to be sure to understand, when you do something like curl -v https://example.com:8080, you get a timeout, that's it ? (here https protocol and port 8080 are mandatory of course)
SSL timeout during accept can be triggered when an unencrypted request is received on a SSL vhost.
Could you also provide the output of the following command:
echo -e "HEAD / HTTP/1.0\r\n\r\n" | openssl s_client -connect mysite.com:8080 -ign_eof
And finally, which version of Yaws are you running ? on which OS ?
I have developed a Rest service and deployed it in tomcat 8 server. It is working fine with http URL. I have a requirement to install SSL certificate for the server.
But there is already a service running on this server which has SSL certificate.
Now my questions are
1) do I need to install another SSL certificate for the same server ?
2) How do i find that previously installed certificate belongs to server or service ?
3) if I install new SSL certificate what configuration changes are to be included in server.xml for port redirect ?
Kindly help me techies.
1) do I need to install another SSL certificate for the same server ?
Probably not, depending of the kind of the SSL service running. You need to stablish a connector from the SSL service to Tomcat to forward SSL requests in a path. Usually in Tomcat is done using the AJP connector and is not needed extra configuration. Check the documentation of the SSL Service
2) How do i find that previously installed certificate belongs to server or service ?
An SSL certificate is issued to a host name (Or infrequently to an IP), so it will be valid for the entire server
3) if I install new SSL certificate what configuration changes are to be included in server.xml for port redirect ?
If the previous SSL service is running in the standard port 443, you will need a new port. Configure a new connector in server.xml with the port, ssl activated and the keystore with the certificate chain. See https://tomcat.apache.org/tomcat-8.0-doc/ssl-howto.html
<!-- Define a SSL Coyote HTTP/1.1 Connector on port 8443 -->
<Connector
protocol="org.apache.coyote.http11.Http11NioProtocol"
port="8443" maxThreads="200"
scheme="https" secure="true" SSLEnabled="true"
keystoreFile="${user.home}/.keystore" keystorePass="changeit"
clientAuth="false" sslProtocol="TLS"/>
How can I connect over ssl to a websocket served by GlassFish on an Amazon AWS EC2 instance through an ELB?
I am using Tyrus 1.8.1 in GlassFish 4.1 b13 pre-release as my websocket implementation.
Port 8080 is unsecured, and port 8181 is secured with ssl.
ELB dns name: elb.xyz.com
EC2 dns name: ec2.xyz.com
websocket path: /web/socket
I have successfully used both ws & wss to connect directly to my EC2 instance (bypassing my ELB). i.e. both of the following urls work:
ws://ec2.xyz.com:8080/web/socket
wss://ec2.xyz.com:8181/web/socket
I have successfully used ws (non-ssl) over my ELB by using a tcp 80 > tcp 8080 listener. i.e. the following url works:
ws://elb.xyz.com:80/web/socket
I have not, however, been able to find a way to use wss though my ELB.
I have tried many things.
I assume that the most likely way of getting wss to work through my ELB would be to create a tcp 8181 > tcp 8181 listener on my ELB with proxy protocol enabled and use the following url:
wss://elb.xyz.com:8181/web/socket
Unfortunately, that does not work. I guess that I might have to enable the proxy protocol on glassfish, but I haven't been able to find out how to do that (or if it's possible, or if it's necessary for wss to work over my ELB).
Another option might be to somehow have ws or wss run over an ssl connection that's terminated on the ELB, and have it continue unsecured to glassfish, by using an ssl > tcp 8080 listener. That didn't work for me, either, but maybe some setting was incorrect.
Does anyone have any modifications to my two aforementioned trials. Or does anyone have some other suggestions?
Thanks.
I had a similar setup and originally configured my ELB listeners as follows:
HTTP 80 HTTP 80
HTTPS 443 HTTPS 443
Although this worked fine for the website itself, the websocket connection failed. In the listener, you need to allow all secure TCP connection as opposed to SSL only to allow wss to pass through as well:
HTTP 80 HTTP 80
SSL (Secure TCP) 443 SSL (Secure TCP) 443
I would also recommend raising the Idle timeout of the ELB.
I recently enabled wss between my browser and an EC2 Node.js instance.
There were 2 things to consider:
in the ELB listeners tab, add a row for the wss port with SSL as load balancer protocol.
in the ELB description tab, set an higher idle timeout (connection settings), which is 60 sec by default. The ELB was killing the websocket connections after 1 minute, setting the idle timeout to 3600 (the max value) enables much longer communication.
It is obviously not the ultimate solution since the timeout is still there, but 1 hour is probably good enough for what we usually do.
hope this help