in apache2:
I want to set the AuthBasicAuthoritative Directive Off to pass the Authorization process to mod_python... But when it's set to Off the Apache raises an error "caught SIGTERM, shutting down"
can anyone tell me what to do please?
Related
Updates in the bottom, I kind of solved it but not sure if the solution is a correct one.
I have Apache running on CentOS with a proxy to localhost port 8080 where I have Flask app running using Gunicorn. This setup works on Apache port 80 (HTTP) and I can connect to it using my domain http://example.com with a browser but now I have tried to setup SSL/HTTPS and it just doesn't work.
Navigating to https://example.com tries to load the page for a while (like 30ish seconds) and then it shows 502 error page:
Proxy Error
The proxy server received an invalid response from an upstream server.
The proxy server could not handle the request GET /.
Reason: Error reading from remote server
Apache error log:
[proxy_http:error] [pid 30209] (103)Software caused connection abort: [client xx.xxx.xxx.xxx:60556] AH01102: error reading status line from remote server localhost:8080
[proxy:error] [pid 30209] [client xx.xxx.xxx.xxx:60556] AH00898: Error reading from remote server returned by /
Gunicorn error log (first 7 lines are from Gunicorn startup, no idea why this info is in error log, last 3 lines are when the HTTPS request returns 502 error):
[29478] [INFO] Listening at: http://127.0.0.1:8080 (29478)
[29478] [INFO] Using worker: sync
[29480] [INFO] Booting worker with pid: 29480
[29481] [INFO] Booting worker with pid: 29481
[29482] [INFO] Booting worker with pid: 29482
[29483] [INFO] Booting worker with pid: 29483
[29484] [INFO] Booting worker with pid: 29484
[29478] [CRITICAL] WORKER TIMEOUT (pid:29480)
[29480] [INFO] Worker exiting (pid: 29480)
[29554] [INFO] Booting worker with pid: 29554
Apache config which is working for HTTP (/etc/httpd/conf/httpd.conf):
Listen 80
#other default config values here
<VirtualHost *:80>
ProxyPass / http://localhost:8080/
ProxyPassReverse / http://localhost:8080/
</VirtualHost>
IncludeOptional conf.d/*.conf
Apache config which is not working for HTTPS (/etc/httpd/conf.d/ssl.conf):
Listen 443 https
#other default config values here
<VirtualHost *:443>
#other default config values here too
SSLCertificateFile /etc/pki/tls/certs/cert.pem
SSLCertificateKeyFile /etc/pki/tls/private/cert.key
SSLProxyEngine on
ProxyPass / https://localhost:8080/
ProxyPassReverse / https://localhost:8080/
</VirtualHost>
If I remove/comment the SSLProxyEngine, ProxyPass amd ProxyPassReverse lines and restart Apache I get the default Apache welcome page and HTTPS works just fine so clearly the problem is in the Proxy somehow?
The flask app is started with Gunicorn by:
gunicorn --config gunicorn_config.py app:app
gunicorn_config.py:
workers = 5
bind = '127.0.0.1:8080'
umask = 0o007
reload = True
accesslog = 'log_gunicorn_access.txt'
errorlog = 'log_gunicorn_error.txt'
app.py:
from flask import Flask
app = Flask(__name__)
#app.route('/')
def hello_world():
return 'Hello world!'
if __name__ == '__main__':
app.run(host='127.0.0.1', port=8080)
And once again, this works when navigating to my domain using HTTP but doesn't work when using HTTPS.
Any help?
UPDATE:
I managed to get another error. Now navigating to https://example.com loads instantly and shows 500 error page:
Proxy Error
The proxy server could not handle the request GET /.
Reason: Error during SSL Handshake with remote server
Apache error log:
[proxy:error] [pid 32385] (502)Unknown error 502: [client xx.xxx.xxx.xxx:50932] AH01084: pass request body failed to [::1]:8080 (localhost)
[proxy:error] [pid 32385] [client xx.xxx.xxx.xxx:50932] AH00898: Error during SSL Handshake with remote server returned by /
[proxy_http:error] [pid 32385] [client xx.xxx.xxx.xxx:50932] AH01097: pass request body failed to [::1]:8080 (localhost) from xx.xxx.xxx.xxx ()
No more any errors in Gunicorn error log.
I added these two lines to gunicorn_config.py:
keyfile = '/etc/pki/tls/private/cert.key'
certfile = '/etc/pki/tls/certs/cert.pem'
and made sure both files are accessible by the user running Gunicorn (chmod o+r cert.key/pem).
No idea if I was supposed to change it like this as I thought the traffic should go like: client --https--> Apache and then Apache --http--> Gunicorn.
Also HTTP (http://example.com) no longer works and gives the previous 502 error page but I guess running Gunicorn with the cert configs doesn't allow HTTP anymore and would need to run the app twice with different configs).
UPDATE 2:
I added more Apache logging by adding this line to /etc/httpd/conf.d/ssl.conf inside virtual host:
LogLevel info
And now I got additional info in Apache error log:
[ssl:info] [pid 3808] [remote 127.0.0.1:8080] AH02411: SSL Proxy: Peer certificate does not match for hostname localhost
Then I added new line to /etc/httpd/conf.d/ssl.conf inside virtual host:
SSLProxyCheckPeerName off
And now I got another Apache error:
[ssl:info] [pid 3999] [remote 127.0.0.1:8080] AH02005: SSL Proxy: Peer certificate CN mismatch: Certificate CN: example.com Requested hostname: localhost
Added new line to /etc/httpd/conf.d/ssl.conf inside virtual host:
SSLProxyCheckPeerCN off
Aaaand now navigating to https://example.com correctly works and I get "Hello world" back from the app!
Now I guess my question needs update as well: Is it bad practice, wrong or insecure to use SSLProxyCheckPeerName off and SSLProxyCheckPeerCN off in this context? Or is there a better way as I don't think there's a way to order an official SSL certificate on localhost?
You're using
ProxyPass / http://localhost:8080/
and
ProxyPass / https://localhost:8080/
(note the 1 letter difference).
Your localhost:8080 will serve either http or https. Based on your description (and common expectations) it's serving http. If you proxy even your :443 virtual host to http, it'll work better.
You might run into more issues, as the proxied application doesn't really know that it's actually served through https, but that's a different beast than this question.
This is due to non-existance of "/var/www/html" directory. run mkdir "/var/www/html" , hope it will solved.
I have installed a fresh copy of Centos 7. Then I restarted Apache but the Apache failed to start. I have 3 days stucked in this issue. Even the support can not figure out the error.
sudo service httpd start
Failed to start apache :
Job for httpd.service failed because the control process exited with error code. See "systemctl status httpd.service" and "journalctl -xe" for details.
httpd.service - The Apache HTTP Server
Loaded: loaded (/usr/lib/systemd/system/httpd.service; enabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Mon 2016-05-09 16:08:02 BST; 59s ago
Docs: man:httpd(8)
man:apachectl(8)
Main PID: 5710 (code=exited, status=1/FAILURE)
May 09 16:08:02 mike079.startdedicated.de systemd[1]: Starting The Apache HTTP Server...
May 09 16:08:02 mike079.startdedicated.de httpd[5710]: (98)Address already in use: AH00072: make_sock: could not bind to address 85.25.12.20:80
May 09 16:08:02 startdedicated.de httpd[5710]: no listening sockets available, shutting down
May 09 16:08:02 startdedicated.de httpd[5710]: AH00015: Unable to open logs
May 09 16:08:02 startdedicated.de systemd[1]: httpd.service: main process exited, code=exited, status=1/FAILURE
May 09 16:08:02.startdedicated.de kill[5712]: kill: cannot find process ""
May 09 16:08:02 .startdedicated.de systemd[1]: httpd.service: control process exited, code=exited status=1
May 09 16:08:02startdedicated.de systemd[1]: Failed to start The Apache HTTP Server.
May 09 16:08:02 startdedicated.de systemd[1]: Unit httpd.service entered failed state.
May 09 16:08:02 mike: httpd.service failed.
I got the same error because of a simple typo in vhost.conf.
Remember to make sure you don't have any errors in the config files.
apachectl configtest
From your output:
no listening sockets available, shutting down
what basically means, that any port in which one apache is going to be listening is already being used by another application.
netstat -punta | grep LISTEN
Will give you a list of all the ports being used and the information needed to recognize which process is so you can kill stop or do whatever you want to do with it.
After doing a nmap of your ip I can see that
80/tcp open http
so I guess you sorted it out.
In my case I got the error simply because I had changed the Listen 80 to listen 443 in the file
/etc/httpd/conf/httpd.conf
Since I had installed mod_ssl using the yum commands
yum -y install mod_ssl
there was a duplicate listen 443 directive in the file ssl.conf created during mod_ssl installation.
You can verify this if you have duplicate listen 80 or 443 by running the below command in linux centos (My linux)
grep '443' /etc/httpd/conf.d/*
below is sample output
/etc/httpd/conf.d/ssl.conf:Listen 443 https
/etc/httpd/conf.d/ssl.conf:<VirtualHost _default_:443>
/etc/httpd/conf.d/ssl.conf:#ServerName www.example.com:443
Simply reverting the listen 443 in httd.conf to listen 80 fixed my issue.
on command line type journalctl -xe and the results will be
SELinux is preventing /usr/sbin/httpd from name_bind access on the tcp_socket port 83 or 80
This means that the SELinux is running on your machine and you need to disable it. then edit the configuration file by type the following
nano /etc/selinux/config
Then find the line SELINUX=enforce and change to SELINUX=disabled
Then type the following and run the command to start httpd
setenforce 0
Lastly start a server
systemctl start httpd
Allow Apache Through the Firewall
Allow the default HTTP and HTTPS port, ports 80 and 443, through firewalld:
sudo firewall-cmd --permanent --add-port=80/tcp
sudo firewall-cmd --permanent --add-port=443/tcp
And reload the firewall:
sudo firewall-cmd --reload
Some other service may be using port 80: try to stop the other services: HTTPD, SSL, NGINX, PHP, with the command sudo systemctl stop and then use the command sudo systemctl start httpd
For me the issue was the following:
[Sat Sep 10 00:43:20.999998 2022] [auth_digest:error] [pid 123456] (2)No such file or directory: AH01762: Failed to create shared memory segment on file /run/httpd/authdigest_shm.128124
[Sat Sep 10 00:43:20.999999 2022] [auth_digest:error] [pid 123456] (2)No such file or directory: AH01760: failed to initialize shm - all nonce-count checking and one-time noncesdisabled
I have just tailed the error logs to get those two line above:
*ADMINSHELL* :/var/opt/custom_path/logs/httpd # tail error_log
So the problem was that this folder was missing: /run/httpd/
Solution:
mkdir /run/httpd
chown root:httpd /run/httpd
chmod 0710 /run/httpd
After this is done. You will be able to restart the service:
systemctl restart httpd.service
Double check:
systemctl status httpd.service
The error could be anywhere, configs, libraries or the binaries. Error in the control process is a general error thrown by the system when its not able to start/restart the service. In my case one of libraries linked to the exe in the .service file has a problem. Fixing that library solved the problem.
try this cmd to know the missing config or error in the file configuration
$ apachectl configtest
I had almost the same error, and I found the error on vhost.conf file, which is located on /etc/httpd/conf.d/vhost.conf.
If you were configuring the virtual host, I suggest you look at the following file and find if you have any errors there.
/etc/httpd/conf.d/vhost.conf
For me also the same error happens after adding dummy SSL
in /etc/httpd/conf.d/ssl.conf file it's created a line like,
SLCertificateFile /etc/pki/tls/certs/localhost.crt
I changed it from SLCertificateFile to SSLCertificateFile
SSLCertificateFile /etc/pki/tls/certs/localhost.crt
And after apache restart it started working fine.
<VirtualHost *:80>
ServerName www.YOURDOMAIN.COM
ServerAlias YOURDOMAIN.COM
DocumentRoot /var/www/YOURDOMAIN.COM/public_html
ErrorLog /var/www/YOURDOMAIN.COM/error.log
CustomLog /var/www/YOURDOMAIN.COM/requests.log combined
DocumentRoot /var/www/YOURDOMAIN.COM/public_html
<Directory />
Options FollowSymLinks
AllowOverride None
</Directory>
<Directory /var/www/YOURDOMAIN.COM/public_html>
Options Indexes FollowSymLinks MultiViews
AllowOverride All
Order allow,deny
allow from all
</Directory>
</VirtualHost>
I'm trying to integrate Tomcat with Apache. My aim is to redirect all the requests with
http://localhost/myapp to http://localhost:8080
I followed this guide: http://tomcat.apache.org/tomcat-8.0-doc/proxy-howto.html
My httpd.conf looks like this:
Include conf.modules.d/*.conf
LoadModule proxy_module modules/mod_proxy.so
ProxyPass /myapp http://localhost:8080 retry=0 timeout=5
ProxyPassReverse /myapp http://localhost:8080
My server.xml in apache-tomcat looks like this:
<Connector port="8080" protocol="HTTP/1.1" connectionTimeout="20000" redirectPort="8443" proxyPort="80" />
Now when I try the url http://localhost/myapp, it gives 503 Service Unavailable error.
Both Tomcat and Apache are up and running. The URL http://localhost:8080 works fine.
Can there be an issue with file permissions?
For tomcat the user and group are root/root and for httpd, the user and group are apache/apache
Am I missing something or am I doing it wrong?
Httpd version is 2.4.6 and Tomcat's version is 8.0
The httpd error logs:
[proxy:error] [pid 19905] (13)Permission denied: AH00957: HTTP: attempt to connect to 127.0.0.1:8080 (localhost) failed
[proxy:error] [pid 19905] AH00959: ap_proxy_connect_backend disabling worker for (localhost) for 0s
[proxy_http:error] [pid 19905] [client ::1:51615] AH01114: HTTP: failed to make connection to backend: localhost
Solved!
The answer is here: http://sysadminsjourney.com/content/2010/02/01/apache-modproxy-error-13permission-denied-error-rhel/
(Answered by the OP in a question edit. Converted to a community wiki answer. See Question with no answers, but issue solved in the comments (or extended in chat) )
The OP wrote:
The answer is here: http://sysadminsjourney.com/content/2010/02/01/apache-modproxy-error-13permission-denied-error-rhel/
Which is a link to a blog that explains:
SELinux on RHEL/CentOS by default ships so that httpd processes cannot initiate outbound connections, which is just what mod_proxy attempts to do.
If this is the problem, it can be solved by running:
/usr/sbin/setsebool -P httpd_can_network_connect 1
And for a more definitive source of information, see https://wiki.apache.org/httpd/13PermissionDenied
Resolve issue Immediate, It's related to internal security
We, SnippetBucket.com working for enterprise linux RedHat, found httpd server don't allow proxy to run, neither localhost or 127.0.0.1, nor any other external domain.
As investigate in server log found
[error] (13)Permission denied: proxy: AJP: attempt to connect to
10.x.x.x:8069 (virtualhost.virtualdomain.com) failed
Audit log found similar port issue
type=AVC msg=audit(1265039669.305:14): avc: denied { name_connect } for pid=4343 comm="httpd" dest=8069
scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r:port_t:s0 tclass=tcp_socket
Due to internal default security of linux, this cause, now to fix (temporary)
/usr/sbin/setsebool httpd_can_network_connect 1
Resolve Permanent Issue
/usr/sbin/setsebool -P httpd_can_network_connect 1
this worked for me by editing my *.conf file in apache:
ProxyRequests Off
ProxyPreserveHost On
RewriteEngine On
<Proxy http://localhost:8123>
Order deny,allow
Allow from all
</Proxy>
ProxyPass /node http://localhost:8123
ProxyPassReverse /node http://localhost:8123
On CentOS Linux release 7.5.1804, we were able to make this work by editing /etc/selinux/config and changing the setting of SELINUX like so:
SELINUX=disabled
I have been using xampp for a while now without any problem. All of a sudden I start xampp today and I am getting an "Error 403 Access Forbidden" when I try to go to my Localhost.
When I check the error logs for apache it says:
[Mon Mar 22 21:48:57 2010] [error] [client 127.0.0.1] client denied by server configuration: D:/xaamp
This is strange, it should first of all say "C:" and second of all say "Xampp" (not xaamp). Where would I change this is my config files. Have no idea how this could have happened.
I think this will be an Apache configuration problem. The configuration should be found in your httpd.conf at the location below:
\xampp\apache\conf\httpd.conf
Then find this property:
DocumentRoot
What does it say? It should say:
DocumentRoot "C:\xampp\htdocs"
My server was doing just fine up until yesterday. It was running Redmine, and it was the happiest little server until my "friend" imported a SQL table that my little guy couldn't take. Unfortunately, after an hour of trying to get the lil guy to respond, we had to power cycle him.
Now after restart, we get a 503 error when trying to visit the domain connected to Redmine. It's hooked up to a Mongrel daemon, and we use Apache Proxy to direct all connections to the port Redmine is running on.
Using Lynx on the server (http://localhost:8000) you can see the Ruby application working fine. But this bit is not working in my Apache configuration file:
<VirtualHost *:80>
ServerName sub.example.com
ProxyPass / http://localhost:8000
ProxyPassReverse / http://localhost:8000
ProxyPreserveHost on
LogLevel debug
</VirtualHost>
Here's the error log output for Apache:
[debug] mod_proxy_http.c(54): proxy: HTTP: canonicalising URL //localhost:8000
[debug] proxy_util.c(1335): [client 216.27.137.51] proxy: http: found worker http://localhost:8000 for http://localhost:8000/
[debug] mod_proxy.c(756): Running scheme http handler (attempt 0)
[debug] mod_proxy_http.c(1687): proxy: HTTP: serving URL http://localhost:8000/
[debug] proxy_util.c(1755): proxy: HTTP: has acquired connection for (localhost)
[debug] proxy_util.c(1815): proxy: connecting http://localhost:8000/ to localhost:8000
[debug] proxy_util.c(1908): proxy: connected / to localhost:8000
[debug] proxy_util.c(2002): proxy: HTTP: fam 2 socket created to connect to localhost
[error] (13)Permission denied: proxy: HTTP: attempt to connect to 127.0.0.1:8000 (localhost) failed
[error] ap_proxy_connect_backend disabling worker for (localhost)
[debug] proxy_util.c(1773): proxy: HTTP: has released connection for (localhost)
Apache will respond with 503's for at least 60 seconds any time it detects that the backend server is down. This is the default behavior. As in your example, if you restart your backend server (Rails in this example) and someone tries to access it through the Apache proxy before Rails is ready then Apache will return 503's for the next 60 seconds regardless if your backend is now 'up'. Please see the apache docs on ProxyPass where it states:
retry 60
Connection pool worker retry timeout in seconds. If the connection pool worker to the backend server is in the error state, Apache will not forward any requests to that server until the timeout expires. This enables to shut down the backend server for maintenance, and bring it back online later. A value of 0 means always retry workers in an error state with no timeout.
So if you set your Proxy Pass to include retry=0 you won't see the 503's when you restart your backend service. This is also useful when using Apache as reverse proxy during development! For example:
ProxyPass / http://localhost:8000 retry=0
Run following command
# /usr/sbin/setsebool httpd_can_network_connect 1
OR
# /usr/sbin/setsebool httpd_can_network_connect true
and after that restart httpd
# service httpd restart
Are you sure they're restarting in the correct order? I've had weird issues where Apache starts, then Mongrel starts and although Mongrel is running, Apache still throws the proxy error.
I've solved this in the past with various incantations and restarts of Apache and eventually the gods are happy. It seems that sometimes the Mongrel processes don't properly shut down so you have to manually kill them. Here's a link with some [possible] help.
I ended up adding a "kill" option to my /etc/init.d/ mongrel script because it happened so much. It stop Mongrel, killed all Mongrel sessions, started Mongrel and restarted Apache.
<snip>
kill)
echo "Stopping, killing, starting, and restarting Apache..."
mongrel_cluster_ctl stop -c $CONF_DIR --clean
killall -u mongrel
mongrel_cluster_ctl start -c $CONF_DIR --clean
/etc/init.d/httpd restart
RETVAL=$?
;;
</snip>
Probably not a very good solution but the evil went away.
Try running monit to monitor your mongrels behind Apache, and that way it can restart mongrels for you if they die or get too hungry for memory. If for any reason Apache still gets confused you may just have to gracefully restart apache and it should resolve itself, but for 99% of cases having monit watch over your mongrels should avoid this happening again. The other option is look into Phusion Passenger.
First check whether the port 8080 is listening or not by the following command
netstat -tlpn
If not than restart the jenkins server by the following command
sudo /etc/init.d/jenkins start
It should work now. Hope it helps.
Fist, you must install selinux: (SELinux stands for Security-Enhanced Linux.)
apt-get install selinux
After that, you can enable Security Policy of SElinux by follow command:
sed -i 's/SELINUX=.*/SELINUX=permissive/' /etc/selinux/config
Notice:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
Final,restart apache!