I am trying to configure httpd mod_proxy to serve as a reverse proxy in Google Cloud Platform. I set up a TCP load balancer and tried something like this in a VM apache config.
<Proxy "*">
</Proxy>
Listen 7001
Listen 7002
<VirtualHost *:7001>
ProxySourceAddress <LB IP 1 here>
</VirtualHost>
<VirtualHost *:7002>
ProxySourceAddress <LB IP 2 here>
</VirtualHost>
Apache logs show my attempts to GET through to yandex.ru via proxy:
AH01114: HTTP: failed to make connection to backend: www.yandex.ru
(70007)The timeout specified has expired: AH00957: HTTP: attempt to connect to 5.255.255.5:80 (*) failed
httpd version 2.4.6. OS Centos 7.
What am I missing here?
Related
Updates in the bottom, I kind of solved it but not sure if the solution is a correct one.
I have Apache running on CentOS with a proxy to localhost port 8080 where I have Flask app running using Gunicorn. This setup works on Apache port 80 (HTTP) and I can connect to it using my domain http://example.com with a browser but now I have tried to setup SSL/HTTPS and it just doesn't work.
Navigating to https://example.com tries to load the page for a while (like 30ish seconds) and then it shows 502 error page:
Proxy Error
The proxy server received an invalid response from an upstream server.
The proxy server could not handle the request GET /.
Reason: Error reading from remote server
Apache error log:
[proxy_http:error] [pid 30209] (103)Software caused connection abort: [client xx.xxx.xxx.xxx:60556] AH01102: error reading status line from remote server localhost:8080
[proxy:error] [pid 30209] [client xx.xxx.xxx.xxx:60556] AH00898: Error reading from remote server returned by /
Gunicorn error log (first 7 lines are from Gunicorn startup, no idea why this info is in error log, last 3 lines are when the HTTPS request returns 502 error):
[29478] [INFO] Listening at: http://127.0.0.1:8080 (29478)
[29478] [INFO] Using worker: sync
[29480] [INFO] Booting worker with pid: 29480
[29481] [INFO] Booting worker with pid: 29481
[29482] [INFO] Booting worker with pid: 29482
[29483] [INFO] Booting worker with pid: 29483
[29484] [INFO] Booting worker with pid: 29484
[29478] [CRITICAL] WORKER TIMEOUT (pid:29480)
[29480] [INFO] Worker exiting (pid: 29480)
[29554] [INFO] Booting worker with pid: 29554
Apache config which is working for HTTP (/etc/httpd/conf/httpd.conf):
Listen 80
#other default config values here
<VirtualHost *:80>
ProxyPass / http://localhost:8080/
ProxyPassReverse / http://localhost:8080/
</VirtualHost>
IncludeOptional conf.d/*.conf
Apache config which is not working for HTTPS (/etc/httpd/conf.d/ssl.conf):
Listen 443 https
#other default config values here
<VirtualHost *:443>
#other default config values here too
SSLCertificateFile /etc/pki/tls/certs/cert.pem
SSLCertificateKeyFile /etc/pki/tls/private/cert.key
SSLProxyEngine on
ProxyPass / https://localhost:8080/
ProxyPassReverse / https://localhost:8080/
</VirtualHost>
If I remove/comment the SSLProxyEngine, ProxyPass amd ProxyPassReverse lines and restart Apache I get the default Apache welcome page and HTTPS works just fine so clearly the problem is in the Proxy somehow?
The flask app is started with Gunicorn by:
gunicorn --config gunicorn_config.py app:app
gunicorn_config.py:
workers = 5
bind = '127.0.0.1:8080'
umask = 0o007
reload = True
accesslog = 'log_gunicorn_access.txt'
errorlog = 'log_gunicorn_error.txt'
app.py:
from flask import Flask
app = Flask(__name__)
#app.route('/')
def hello_world():
return 'Hello world!'
if __name__ == '__main__':
app.run(host='127.0.0.1', port=8080)
And once again, this works when navigating to my domain using HTTP but doesn't work when using HTTPS.
Any help?
UPDATE:
I managed to get another error. Now navigating to https://example.com loads instantly and shows 500 error page:
Proxy Error
The proxy server could not handle the request GET /.
Reason: Error during SSL Handshake with remote server
Apache error log:
[proxy:error] [pid 32385] (502)Unknown error 502: [client xx.xxx.xxx.xxx:50932] AH01084: pass request body failed to [::1]:8080 (localhost)
[proxy:error] [pid 32385] [client xx.xxx.xxx.xxx:50932] AH00898: Error during SSL Handshake with remote server returned by /
[proxy_http:error] [pid 32385] [client xx.xxx.xxx.xxx:50932] AH01097: pass request body failed to [::1]:8080 (localhost) from xx.xxx.xxx.xxx ()
No more any errors in Gunicorn error log.
I added these two lines to gunicorn_config.py:
keyfile = '/etc/pki/tls/private/cert.key'
certfile = '/etc/pki/tls/certs/cert.pem'
and made sure both files are accessible by the user running Gunicorn (chmod o+r cert.key/pem).
No idea if I was supposed to change it like this as I thought the traffic should go like: client --https--> Apache and then Apache --http--> Gunicorn.
Also HTTP (http://example.com) no longer works and gives the previous 502 error page but I guess running Gunicorn with the cert configs doesn't allow HTTP anymore and would need to run the app twice with different configs).
UPDATE 2:
I added more Apache logging by adding this line to /etc/httpd/conf.d/ssl.conf inside virtual host:
LogLevel info
And now I got additional info in Apache error log:
[ssl:info] [pid 3808] [remote 127.0.0.1:8080] AH02411: SSL Proxy: Peer certificate does not match for hostname localhost
Then I added new line to /etc/httpd/conf.d/ssl.conf inside virtual host:
SSLProxyCheckPeerName off
And now I got another Apache error:
[ssl:info] [pid 3999] [remote 127.0.0.1:8080] AH02005: SSL Proxy: Peer certificate CN mismatch: Certificate CN: example.com Requested hostname: localhost
Added new line to /etc/httpd/conf.d/ssl.conf inside virtual host:
SSLProxyCheckPeerCN off
Aaaand now navigating to https://example.com correctly works and I get "Hello world" back from the app!
Now I guess my question needs update as well: Is it bad practice, wrong or insecure to use SSLProxyCheckPeerName off and SSLProxyCheckPeerCN off in this context? Or is there a better way as I don't think there's a way to order an official SSL certificate on localhost?
You're using
ProxyPass / http://localhost:8080/
and
ProxyPass / https://localhost:8080/
(note the 1 letter difference).
Your localhost:8080 will serve either http or https. Based on your description (and common expectations) it's serving http. If you proxy even your :443 virtual host to http, it'll work better.
You might run into more issues, as the proxied application doesn't really know that it's actually served through https, but that's a different beast than this question.
I've got a somewhat unique (maybe not, but unique to me) architecture here for some home apps that variously require PHP 5.6 and PHP 7.1. This is the architecture recommended to me by /r/docker:
One container acting as Apache Reverse Proxy that will ingest a DNS request on port 80 and then pass it to one of the below containers' ports.
One container running Apache with PHP 5.6, listening on port 8081.
One container running Apache with PHP 7.1, listening on port 8080.
I've been able to get domainname.com to work this way, by setting up the following domainname.com.conf in /etc/apache2/sites-available:
<VirtualHost *:80>
ServerName domainname.com
<Proxy *>
Allow from localhost
</Proxy>
ProxyPass "/" "http://public.ip.address:8080/" retry=0
ProxyPreserveHost On
</VirtualHost>
However, one of my subdomains needs to push to an application on the PHP 5.6 side. So I've set up this subdomain.domainname.com.conf:
<VirtualHost *:80>
ServerName subdomain.domainname.com
<Proxy *>
Allow from localhost
</Proxy>
ProxyPass "/" "http://public.ip.address:8081/subdomain/" retry=0
ProxyPreserveHost On
</VirtualHost>
Indeed, going directly to http://public.ip.address:8081/subdomain/ loads the application just fine, but for whatever reason, going to http://subdomain.domain.com gives a 503. Originally it was just showing the Apache on Ubuntu page, which is when I realized I needed to a2ensite and reload Apache, which has since been done.
The logs show this:
[Wed Jul 10 21:36:59.049131 2019] [proxy:error] [pid 2298:tid 139882856969984] (113)No route to host: AH00957: HTTP: attempt to connect to public.ip.address:8081 (3.220.176.240) failed
[Wed Jul 10 21:36:59.049182 2019] [proxy_http:error] [pid 2298:tid 139882856969984] [client 128.229.4.2:58611] AH01114: HTTP: failed to make connection to backend: public.ip.address
We know that the system can access 8080 just fine, so I'm not sure why it's struggling with this traffic. As mentioned above, hitting the ProxyPass URL directly from the outside loads the page fine, so traffic inbound to that container is fine. SELinux is disabled. This is in AWS and Security Groups are configured properly as far as I can tell (see: traffic works from outside).
Using the public IP address is the only way I've been able to get inter-container traffic to work, because using Docker's bridge IPs didn't work previously.
On the old system, when all my apps were just PHP 5.6, I had a standard VirtualHost statement that just took in the subdomain.domain.com and pointed it to a specific DocumentRoot. I can't do that here since the reverse proxy container needs to hand off the traffic to the container with the right PHP version.
Many thanks for any tips or tricks here.
I need to forward all traffic from specified port to a Unix Domain Socket in Apache:
In sites-enabled/000-default.conf I have defined a VirtualHost:
<VirtualHost *:8091>
ProxyRequests Off
<Proxy *>
Order deny,allow
Allow from all
</Proxy>
ProxyPass / unix:/var/lib/jenkins/workspace/myproject/mysocket.sock|http://127.0.0.1/
ProxyPassReverse / unix:/var/lib/jenkins/workspace/myproject/mysocket.sock|http://127.0.0.1/
</VirtualHost>
In ports.conf I have:
Listen 80
Listen 8091
I have both proxy and proxy_http mods enabled:
krzysiek:/etc/apache2$ a2enmod proxy
Module proxy already enabled
krzysiek:/etc/apache2$ a2enmod proxy_http
Module proxy already enabled
I am using Apache 2.4.7 and offical documentation states that:
In 2.4.7 and later, support for using a Unix Domain Socket is available by using a target which prepends unix:/path/lis.sock|
When I reload Apache I get error:
krzysiek:/etc/apache2$ sudo service apache2 restart
* Restarting web server apache2 [fail]
* The apache2 configtest failed.
AH00526: Syntax error on line 64 of /etc/apache2/sites-enabled/000-default.conf:
ProxyPass URL must be absolute!
I tried changing:
ProxyPass / unix:/var/(...)
To:
ProxyPass / unix:///var/(...)
Then Apache reloads correctly but forwarding DOES NOT work, error.log says:
[proxy:warn] [pid 22436] [client 10.0.4.19:47662] AH01144: No protocol handler was valid for the URL /. If you are using a DSO version of mod_proxy, make sure the proxy submodules are included in the configuration using LoadModule.
Could you explain me what I'm doing wrong? Or how do I forward traffic from port to unix socket on Apache correctly?
Might be a bit late 🙂, but the following apache config works for me:
ProxyRequests off
Define SOCKET_NAME /path/to/my.sock
ProxyPass / unix:${SOCKET_NAME}|uwsgi://%{HTTP_HOST}/
ProxyPassReverse / unix:${SOCKET_NAME}|uwsgi://%{HTTP_HOST}/
with modules proxy and proxy_uwsgi loaded.
Important do not place your socket in /tmp (I think systemd prevents the apache process to directly access it for security reasons).
Also, watch out for the file permissions.
I'm trying to run an Apache Solr Service (on its emdedded jetty server) on a remote server. The admin has provided me following information:
DNS: my.server.com
IP: xxx.xxx.xxx
Server OS: 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux
Only Port 80 is accessible. On the server we want to deploy Apache Solr and a microservice which uses Solr as search engine. I want to use Apache Webserver to forward the HTTP-Request to the Solr Admin UI and to the microservice UI, but it doesn't seem to work, I use Apache Server version: Apache/2.4.10 (Debian)
Server built: Sep 15 2016 20:44:43.
I installed Apache and started the server, so far everything works as expected. I can access the admin view from Apache entering the DNS in my browser.
I enabled a few modules following this articel https://www.digitalocean.com/community/tutorials/how-to-use-apache-http-server-as-reverse-proxy-using-mod_proxy-extension:
a2enmod proxy
a2enmod proxy_http
a2enmod proxy_ajp
a2enmod rewrite
a2enmod deflate
a2enmod headers
a2enmod proxy_balancer
a2enmod proxy_connect
a2enmod proxy_html
Then I tried to configure a virtual host under /etc/apache2/sites-available/myconf.conf:
<VirtualHost *:80>
DocumentRoot /var/www/html
ErrorLog /var/log/apache2/error.log
CustomLog /var/log/apache2/access.log combined
ProxyPass /solr http://my.server.com:8983 retry=0 timeout=5
ProxyPassReverse /solr http://my.server.com:8983
ProxyPass /microservice http://my.server.com:6868 retry=0 timeout=5
ProxyPassReverse /microservice http://my.server.com:6868
LogLevel debug
</VirtualHost>
Solr uses its standard port 8983 and the microservice will be on port 6868. When I try to acces solr with http://my.server.com/solr I get an HTTP 503 Service unavailable.
I first tried this:
/usr/sbin/setsebool -P httpd_can_network_connect 1
But it changed nothing. I also had to install first:
apt-get install policycoreutils
to make this option available. The solr service seems to be ok:
solr status
Found 1 Solr nodes:
Solr process 14082 running on port 8983
{
"solr_home":"/etc/apache-solr/solr-6.2.0/server/solr",
"version":"6.2.0 764d0f19151dbff6f5fcd9fc4b2682cf934590c5 - mike - 2016-08-20 05:41:37",
"startTime":"2016-10-07T12:02:05.300Z",
"uptime":"0 days, 1 hours, 29 minutes, 55 seconds",
"memory":"29.7 MB (%6.1) of 490.7 MB"}
The Apache log keeps saying:
The timeout specified has expired: AH00957: HTTP: attempt to connect to xxx.xxx.xxx:8983 (my.server.com) failed
AH00959: ap_proxy_connect_backend disabling worker for (my.server.com) for 0s
AH01114: HTTP: failed to make connection to backend: my.server.com
Without my timeout setting everthing keeps the same but it takes ages before I get the 503 Error.
Any hints? After one day struggeling I'm depressed ... all I want is to finish the task.
Thanks in advance!
It turns out that I needed to append a slash to the urls:
ProxyPass /solr/ http://my.server.com:8983/ retry=0 timeout=5
ProxyPassReverse /solr/ http://my.server.com:8983/
ProxyPass /microservice/ http://my.server.com:6868/ retry=0 timeout=5
ProxyPassReverse /microservice/ http://my.server.com:6868/
Apache2 (CKAN) + TOMCAT6 + Mod Proxy Problems
Hello,
I have a server running CKAN over Apache2, this server also have a tomcat6 hosting some HTML pages. Apache is running in 8080 and Tomcat in 8081. Nginx is used to make Apache accessible through port 80.
I have a web in the tomcat served in
http://195.57.27.91:8081/arboles
I would like to make this web accessible through the port 80 so I use Mod Proxy from Apache2 to make a internal proxy that redirects the incoming request asking for /arboles to port 8081. This has been done by using http.conf like this:
LoadModule proxy_module /usr/lib/apache2/modules/mod_proxy.so
LoadModule proxy_http_module /usr/lib/apache2/modules/mod_proxy_http.so
ProxyRequests Off
ProxyPreserveHost On
<Proxy *>
Order deny,allow
Allow from all
</Proxy>
ProxyPass /arboles http://195.57.27.91:8081/arboles/
ProxyPassReverse /arboles http://195.57.27.91:8081/arboles/
In do know this Proxy is working because I have tried it in another server with another domain successfully.
PROBLEM:
When I try to resolve the URI:
http://195.57.25.27/arboles
It doesn't load the page. All I get is 504 Gateway Error Time-out.
I have checked the Apache2 logs and there is no track of error or misconfiguration, I don't know what else to try. Any ideas?
EDIT: I forgot to mention nginx log:
2014/01/31 10:39:28 [error] 13921#0: *321 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 158.49.219.193, server: , request: "GET /arboles/ HTTP/1.1", upstream: "http://127.0.0.1:8080/arboles/", host: "opendata.caceres.es"
Note: opendata.caceres.es is the domain name for the server.
Ok, finally I have solved the problem (props to Ross Jones for pointing out the idea).
Basically what I have done is redirect the URIS that have /arboles in their path to port 8081 by using nginx instead of Apache2. To do that I've modified my sites-enabled in /etc/nginx (I also could have add another site) to add the following redirection:
location /arboles {
proxy_pass http://127.0.0.1:8081/arboles/;
proxy_set_header Host $host;
proxy_cache_key $host$scheme$proxy_host$request_uri;
}
Hope this can be interesting for somebody in the future.
NOTE: if you don't have conf files in sites-enabled & sites-available you can make this changes in the conf.nginx file.
Jesús.