Apache ProxyPass HTTPS and remote server with SNI - apache

I'm wanting to front an AWS APIGateway URL with a reverse proxy in Apache. The reason is due to a process requiring a static IP to provision a service behind a strict firewall and that the current infrastructure has mod_proxy already in place. The solution I'm looking to implement is to simply to route https->https(apiGateway) via mod_proxy.
But.. AWS uses SNI and I can not get mod_proxy to handshake.
I have the following settings enabled
<IfModule mod_ssl.c>
<VirtualHost *:443>
ProxyPreserveHost On
SSLProxyEngine On
ProxyPass /api/1_0/ https://xxx.execute-api.eu-west-1.amazonaws.com/1_0/
ProxyPassReverse /api/1_0/ https://xxx.execute-api.eu-west-1.amazonaws.com/1_0/
The following logs are available in debug mode
proxy_util.c(2020): AH00942: HTTPS: has acquired connection for (xxx.execute-api.eu-west-1.amazonaws.com)
proxy_util.c(2610): AH00962: HTTPS: connection complete to 52.x.x.x:443 (xxx.execute-api.eu-west-1.amazonaws.com)
AH01964: Connection to child 0 established (server domain.com:443)
AH02003: SSL Proxy connect failed
SSL Library Error: error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure
AH01998: Connection closed to child 0 with abortive shutdown (server domain.com:443)
AH01997: SSL handshake failed: sending 502
If I connect using openssl, I can demonstrate a similar error
openssl s_client -tls1_2 -connect xxx.execute-api.eu-west-
1.amazonaws.com:443
CONNECTED(00000003)
140735866254216:error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert
handshake failure:s3_pkt.c:1494:SSL alert number 40
140735866254216:error:1409E0E5:SSL routines:ssl3_write_bytes:ssl handshake failure:s3_pkt.c:659:
adding -servername for SNI, results in a valid connection
SSL handshake has read 3601 bytes and written 489 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES128-GCM-SHA256
...
I therefore believe that mod_proxy and mod_ssl are not sending the servername to the remote https server and could be a bug.
I am running Ubuntu 14.04 with
Server version: Apache/2.4.7 (Ubuntu)
Server built: Sep 18 2017 16:37:54
OpenSSL 1.0.1f 6 Jan 2014
I have attempted to limit the SSLProxyProtocol to TLS1_2 and the cipher list too however the sslv3 alert handshake failure log still exists regardless.
Has any one come across this and know how to ensure that the SNI values are being sent or is this a restriction in the Apache modules?

This is due to ProxyPreserveHost On being set early in the configuration.
Setting ProxyPreserveHost Off under a proxy tag completes as expected:
<Proxy "https://xxx.execute-api.eu-west-1.amazonaws.com/1_0">
ProxyAddHeaders off
ProxyPreserveHost off
</Proxy>
Info on the directive:
When enabled, this option will pass the Host: line from the incoming
request to the proxied host, instead of the hostname specified in the
ProxyPass line.
This option should normally be turned Off. It is mostly useful in
special configurations like proxied mass name-based virtual hosting,
where the original Host header needs to be evaluated by the backend
server.

The answer above helped us. For the sake of "googling" it I would like to add the following scenario as well:
we use a cloud WAF infront of one of our services. Another service needs to forward specific queries to this host and pass through the WAF.
The working configuration looks like this:
<Proxy "balancer://qwertz">
ProxyPreserveHost off
BalancerMember "https://somehost.somewhere.de:443"
</Proxy>
ProxyPass "/special/webservices/" "balancer://qwertz/special/webservices/"
ProxyPassReverse "/special/webservices/" "balancer://qwertz/special/webservices/"
The apache is configured as reverse proxy and runs inside a docker container.
Note that the balancer has only one host, thus no balancing is done.
Before this answer we were getting SSL erros like:
Error during SSL Handshake with remote server returned by...
pass request body failed to...
or
server certificate does NOT include an ID which matches the server name
Like AWS the Cloud WAF also requires SNI.
Hope this post makes this answer more visible (or findable) to someone else.

Related

Apache as Proxy: how to configure TLS on the downstream side?

We want to use an Apache2 (v2.4.51) as a (reverse) proxy to access a downstream server (running Tomcat). That Tomcat is configured to accept only TLS v1.2 and a very limited set of ciphers (and that's not negotiable).
In the logs I found that our Apache tries to open the connection to that Tomcat using TLS v1.3 which causes the downstream server to immediately terminate the connection and no further communication happens.
How can I configure an Apache server to use a specific TLS version and cipher on an outgoing/ downstream connection? Everything I found re. Apache TLS configuration so far was dealing with the front-side, i.e. what Apache receives and accepts. But in my case I need to adjust the back-end-side, i.e. what Apache uses when it forwards a request.
How/where can one configure that?
Edit: Meanwhile I realized that the terms "upstream" & "downstream" are not always used consistently - so just in case: with "downstream" here I mean the connection (2) as sketched below:
{browsers/internet} --(1)--> [Apache reverse proxy] --(2)--> [Tomcat application server].
Edit 2: In Tomcat's log (catalina.out) I keep getting the below exception which seems to suggest that it is addressed using TLS v1.3 (which it can't handle):
Oct 06, 2022 5:22:06 PM org.apache.tomcat.util.net.NioEndpoint setSocketOptions
SEVERE:
javax.net.ssl.SSLHandshakeException: No appropriate protocol (protocol is disabled or cipher suites are inappropriate)
at sun.security.ssl.HandshakeContext.<init>(HandshakeContext.java:171)
at sun.security.ssl.ServerHandshakeContext.<init>(ServerHandshakeContext.java:62)
at sun.security.ssl.TransportContext.kickstart(TransportContext.java:220)
at sun.security.ssl.SSLEngineImpl.beginHandshake(SSLEngineImpl.java:97)
at org.apache.tomcat.util.net.SecureNioChannel.reset(SecureNioChannel.java:89)
at org.apache.tomcat.util.net.SecureNioChannel.<init>(SecureNioChannel.java:71)
at org.apache.tomcat.util.net.NioEndpoint.setSocketOptions(NioEndpoint.java:666)
at org.apache.tomcat.util.net.NioEndpoint$Acceptor.run(NioEndpoint.java:808)
at java.lang.Thread.run(Thread.java:750)
javax.net.ssl|FINE|B5|http-nio-8443-Acceptor-0|2022-10-06 17:22:07.539 CEST|HandshakeContext.java:304|No available cipher suite for TLS13
javax.net.ssl|SEVERE|B5|http-nio-8443-Acceptor-0|2022-10-06 17:22:07.540 CEST|TransportContext.java:316|Fatal (HANDSHAKE_FAILURE): Couldn't kickstart handshaking (
"throwable" : {
javax.net.ssl.SSLHandshakeException: No appropriate protocol (protocol is disabled or cipher suites are inappropriate)
at sun.security.ssl.HandshakeContext.<init>(HandshakeContext.java:171)
at sun.security.ssl.ServerHandshakeContext.<init>(ServerHandshakeContext.java:62)
at sun.security.ssl.TransportContext.kickstart(TransportContext.java:220)
at sun.security.ssl.SSLEngineImpl.beginHandshake(SSLEngineImpl.java:97)
at org.apache.tomcat.util.net.SecureNioChannel.reset(SecureNioChannel.java:89)
at org.apache.tomcat.util.net.SecureNioChannel.<init>(SecureNioChannel.java:71)
at org.apache.tomcat.util.net.NioEndpoint.setSocketOptions(NioEndpoint.java:666)
at org.apache.tomcat.util.net.NioEndpoint$Acceptor.run(NioEndpoint.java:808)
at java.lang.Thread.run(Thread.java:750)}
Edit 3: my /etc/apache2/conf.d/proxy.conf file now reads:
Listen 443
<VirtualHost *:443>
SSLEngine On
SSLCertificateFile -name-removed-
SSLProxyEngine On
SSLProtocol all -SSLv2 -SSLv3 -TLSv1 -TLSv1.1
SSLProxyMachineCertificateFile -name-removed-
ProxyPass /foobar https://-name-removed-:8443/foobar
ProxyPassReverse /foobar https://-name-removed-:8443/foobar
SSLProxyProtocol +TLSv1.2
<Proxy "*">
Require all granted
SSLProxyProtocol +TLSv1.2
</Proxy>
LogLevel debug
ErrorLog "-name-removed-"
</VirtualHost>
Note: "foobar" and "-name-removed-" represent values that I replaced for privacy.
Edit 4:
The nmap response was:
# nmap -sV --script ssl-enum-ciphers -p 8443 127.0.0.1
Starting Nmap 7.70 ( https://nmap.org ) at 2022-10-10 16:12 CEST
Nmap scan report for localhost (127.0.0.1)
Host is up (0.00011s latency).
PORT STATE SERVICE VERSION
8443/tcp open tcpwrapped
Service detection performed. Please report any incorrect results at https://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 3.67 seconds
#
What does that tell me?
I would suggest running a utility like nmap to assess the SSL capabilities of the backend server (Tomcat) in this instance. nmap nicely prints out the TLS protocols and ciphers that a given TLS server supports. Using this information, we could configure Apache HTTPD to use the same protocols and ciphers for the backend connection using SSLProxyProtocol and SSLProxyCipherSuite directives.
How to use nmap:
nmap -sV --script ssl-enum-ciphers -p
The issue has been resolved! I had a typo in the config file! Shame on me and sorry for the bandwidth consumed.

How to trust a remote servers self-signed SSL certificate in Apache

I'm setting up a Tomcat server behind an Apache proxy, everything is working fine except for the forward proxy when dealing with a remote server with a self-signed certificate. I can access other HTTPS sites, but not ones with self-signed certs. I've already added the CA using SSLProxyCACertificateFile, but it still doesn't work.
In the error log, I get "AH00898: Connect to remote machine blocked returned by server_hostname".
Apache 2.4.54 on Debian 11
I can wget the URL (Once I added the CA to Debian). I can also access the URL if I allow Tomcat to bypass the proxy (When the CA is in the Java key store). But I need all the traffic to go through the proxy.
<VirtualHost 10.10.10.6:8082>
ProxyRequests On
ProxyVia On
<Proxy "*">
Require ip 10.10.10
</Proxy>
ErrorLog ${APACHE_LOG_DIR}/error_forward_proxy_secure.log
CustomLog ${APACHE_LOG_DIR}/access_forward_proxy_secure.log combined
SSLProxyCACertificateFile /var/ssl/trusted_ca.crt
</VirtualHost>
What am I doing wrong?

SSL issue with Apache httpd reverse proxy scheme

Apologies this question is a mix of Chef and Apache-httpd, but ultimately I believe it's an Apache-httpd configuration issue.
I have a Chef client/node happily hitting a Chef Server w/ SSL.
Chef client/node - /etc/chef/client.rb
chef_server_url 'https://chefserverhostname/organizations/myOrg'
ssl_verify_mode :verify_peer
I'm trying to put a proxy in front of the Chef Server so I can re-point down the road if needed. Apache-httpd is installed fine (on the same box as Chef Server), all firewall and iptables are consistent with my below configuration.
Chef Server - /etc/httpd/conf/httpd.conf
Listen 8443
<VirtualHost *:8443>
ServerAdmin me#email
ServerName chefserverhostname
ServerAlias CDS-PROXY
ProxyPass / https://chefserverhostname:443/
ProxyPassReverse / https://chefserverhostname:443/
RewriteEngine On
</VirtualHost>
I reconfigure the Chef client/node's client.rb to point to https://chefserverhostname:8443/organizations/myOrg, and hope to have the request pass through the proxy. ChefServer/nginx is listening on 443 and I'm assuming that it's taking care of the SSL decrypt, and 8443 is just the 'pass-through'. But when I fire up my Chef client/node I'm getting an SSL error message:
ERROR: SSL Validation failure connecting to host: chefserverhostname - SSL_connect returned=1 errno=0 state=SSLv2/v3 read server hello A: unknown protocol
You are using HTTPS but your vhost does not have any configuration to enable mod_ssl or another TLS option. As such, you should be using http://hostname:8443 but this very not recommended.
Chef Server is an appliance at heart and this kind of extra proxy layer is possible but not how things are intended to operate. You should just use the default setup which already includes a fronting nginx proxy for Erchef and friends.
You can check https://github.com/chef-cookbooks/httpd this recipe and configure properly apache ssl and additional modules.
To me is also not ok to drop SSL connection on chef server - base on your current setup.
Since you have nginx on your machine you can extend the configuration there to support reverse proxy. Later on if you remove chefserver/nginx from this machine and you want only to leave proxy, install new nginx and apply the same conf for proxy configuration.
I rather drop apache from your setup.

Error during SSL Handshake with remote server

I have Apache2 (listening on 443) and a web app running on Tomcat7 (listening on 8443) on Ubuntu.
I set apache2 as reverse proxy so that I access the web app through port 443 instead of 8443. Besides, I need to have SSL communication not only between browser and apache2 but also between apache2 and tomcat7, thus I set SSL on both apache2 and tomcat7.
If I try to access the web app by directly contacting tomcat7, everything is fine.
The problem is that when I try to access the tomcat's web app through apache2 (reverse proxy), on the browser appears the error:
Proxy Error
The proxy server could not handle the request GET /web_app.
Reason: Error during SSL Handshake with remote server
The comment by MK pointed me in the right direction.
In the case of Apache 2.4 and up, there are different defaults and a new directive.
I am running Apache 2.4.6, and I had to add the following directives to get it working:
SSLProxyEngine on
SSLProxyVerify none
SSLProxyCheckPeerCN off
SSLProxyCheckPeerName off
SSLProxyCheckPeerExpire off
I have 2 servers setup on docker, reverse proxy & web server.
This error started happening for all my websites all of a sudden after 1 year.
When setting up earlier, I generated a self signed certificate on the web server.
So, I had to generate the SSL certificate again and it started working...
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout ssl.key -out ssl.crt
On a remote OEL (Oracle Enterprise Linux) 7.8 server, i have a backend web application running with HTTPS/8009. As its a third party app, I did not have choice to disable SSL or change port.
As i needed to access the web app from my local machine's browser, i thought of setting up a reverse proxy (HTTP to HTTPS mapping) using Apache httpd. Now i can access the web app from my local browser through below URL:
http://10.157.146.97:1234/
FYI, CURL commands working inside the Linux Machine were below ones:
curl http://10.157.146.97:1234/
curl -k https://localhost:8009/
Here is my reverse proxy setup :
/etc/httpd/conf/httpd.conf
Listen 1234
<VirtualHost *:1234>
SSLProxyEngine On
SSLProxyVerify none
SSLProxyCheckPeerCN off
SSLProxyCheckPeerName off
SSLProxyCheckPeerExpire off
ProxyPreserveHost On
ProxyPass / https://localhost:8009/
ProxyPassReverse / https://localhost:8009/
</VirtualHost>
One aspect i struggled a lot, earlier i was trying with url pattern (/sample) in ProxyPass/ProxyPassReverse but that was causing HTTP 404 (not found) for css/js files as web-app's welcome page contains indirect css/js paths (sample code below). So replacing url pattern (/sample) with (/) solved that problem too.
previous Not working config:
ProxyPass /sample https://localhost:8009/
ProxyPassReverse /sample https://localhost:8009/
<script defer src="abc.js"></script><link href="xyz.css" rel="stylesheet"></head>
Faced the same problem as OP:
Tomcat returned response when accessing directly via SOAP UI
Didn't load html files
When used Apache properties mentioned by the previous answer, web-page appeared but AngularJS couldn't get HTTP response
Tomcat SSL certificate was expired while a browser showed it as secure - Apache certificate was far from expiration. Updating Tomcat KeyStore file solved the problem.
Here is my variation on this theme, inspired by this Git gist. The server is a Docker container with an internal self-signed SSL certificate, reachable at https://localhost:8443. Proxied to server.example.org:443. Relevant config details:
<VirtualHost AAA.BBB.CCC.DDD:443>
ServerName server.example.org
ErrorLog ${APACHE_LOG_DIR}/error.log
CustomLog ${APACHE_LOG_DIR}/access.log combined
# These settings are definitely needed
SSLEngine On
SSLProxyEngine On
ProxyRequests Off
SSLProxyVerify none
SSLProxyCheckPeerCN off
SSLProxyCheckPeerName off
SSLProxyCheckPeerExpire off
# These may not be needed, depending on proxied application
ProxyPreserveHost on
RequestHeader set X-Forwarded-Proto https
ProxyPass "/" "https://localhost:8443/"
RewriteEngine on
RewriteCond %{HTTP:Upgrade} websocket [NC]
RewriteCond %{HTTP:Connection} upgrade [NC]
RewriteRule ^/?(.*) "wss://localhost:8443/$1" [P,L]
ProxyPassReverse "/" "https://localhost:8443/"
</VirtualHost>
The section between SSLEngine On and RequestHeader... I put together via Googling and trial and error. Maybe some of these settings are not needed, YMMV.
Note: the RewriteRule with "wss" was needed because the server uses secure websockets.
Platform: Ubuntu 20.04.3 LTS, Apache 2.4.41.
My problem was having one Apache server behind another and having mismatched certificates on each server.
It took me a while to realise that
SSLCertificateFile
and
SSLCertificateKeyFile
were pointing to files with the same names on each server, but that the first first server had an updated certificate and private key compared with the second one, so the content of the files was different.
Note that the error might also occur when your system have TLSv1 disabled. Like e.g Ubuntu 20.x have TLSv1.0 disabled by default. For example if you have something like this:
Apache 2.4.41 on Ubutntu20 (proxy) --[https]--> old Apache serving TLS v1.0
SSLProxyVerify etc will not help you.
What you need to do is to enable TLS 1.0 in openssl.conf. At least until you can update the old server 🙊...
Enabling old TLS on Ubuntu
So in Ubuntu 20.04.3 TLS to effectively enable TLSv1 change /etc/ssl/openssl.cnf. At the top of the file (before any sections) add:
# Added to enable TLS1.0
openssl_conf = default_conf
And on the very end of the file
##
# Added to enable TLS1.0
[default_conf]
ssl_conf = ssl_sect
[ssl_sect]
system_default = system_default_sect
[system_default_sect]
CipherString = DEFAULT#SECLEVEL=1
##
Comments are obviously not required 😉, but will help when you want to disable TLS1 again.
Once you restart / reboot everything should work fine.
Note that this is a global (system-wide) change. So it is not ideal, but it just works. See also: more notes about Ubuntu and default TLS versions.
If the host of your apache proxy
is different from the host
of the proxy target (or its sni name),
set -
ProxyPreserveHost Off

Error 107 (net::ERR_SSL_PROTOCOL_ERROR): SSL protocol error

I have configured SSL on my tomcat. I referred to the steps from the following site to configure SSL:
http://wiki.openbravo.com/wiki/How_To_Configure_SSL_For_Windows
I am using Win32 OpenSSL v0.9.8x Light installer and tomcat 7.0.22. But when I access https://server.ensarm.com:8843/ it gives the following error:
SSL connection error
Unable to make a secure connection to the server. This may be a problem with the server, or it may be requiring a client authentication certificate that you don't have.
Error 107 (net::ERR_SSL_PROTOCOL_ERROR): SSL protocol error.
I don't understand what could be the problem. :(
Check your Apache error log, see if it has a message similar to:
You configured HTTP(80) on the standard HTTPS(443) port!
This may indicate you have configured Apache to listen on port 443 while the SSLEngine is not on.
In httpd.conf only set ONE listen directive:
Listen *:80
Later in httpd.conf you should have something that looks like:
<IfModule ssl_module>
Include conf/httpd-ssl.conf
</IfModule>
In httpd-ssl.conf (or equivalent config file) make sure you have enabled the SSLEngine prior to the Listen directive:
SSLEngine on
listen *:443
Restart Apache and you should be good to go.