I'm not 100% sure that the real source of the problem is apache (could be php or other), but I'd like start from here as the only logs that provides me with information (with trace4 level) is apache log. No information appears from other logs.
I have:
- Linux 4.7.4-100.fc23.x86_64
- Server version: Apache/2.4.23 (Fedora)
- mysql Ver 15.1 Distrib 10.0.26-MariaDB, for Linux (x86_64) using readline 5.1
- PHP Version => 5.6.25
The problem: random internal server error 500. Random because, for example, if a page produces the error and do "reload" then it works. The same page may not work as well as work. Sometimes you can use the site for several minutes with no problems, then suddenly there is the error.
When error occurs, this is all that I get from the log file (with loglevel = trace4):
[...]
[Fri Sep 30 07:41:39.151052 2016] [rewrite:trace3] [pid 17957:tid 140241771534080] mod_rewrite.c(477): [client 151.25.206.200:38368] 151.25.206.200 - - [www.centrometeo.com/sid#55c3e3673400][rid#7f8c8000e9d0/subreq] [perdir /home/web/centrometeo.com/] applying pattern '.*' to uri 'index.php', referer: http://www.centrometeo.com/modelli-numerici/modello-wrf-nmm/559-wrf-mslp-6hprec-ita
[Fri Sep 30 07:41:39.151092 2016] [rewrite:trace1] [pid 17957:tid 140241771534080] mod_rewrite.c(477): [client 151.25.206.200:38368] 151.25.206.200 - - [www.centrometeo.com/sid#55c3e3673400][rid#7f8c8000e9d0/subreq] [perdir /home/web/centrometeo.com/] pass through /home/web/centrometeo.com/index.php, referer: http://www.centrometeo.com/modelli-numerici/modello-wrf-nmm/559-wrf-mslp-6hprec-ita
[Fri Sep 30 07:41:39.913003 2016] [core:trace4] [pid 17957:tid 140241771534080] util_script.c(567): [client 151.25.206.200:38368] Headers from script 'php.fcgi':, referer: http://www.centrometeo.com/modelli-numerici/modello-wrf-nmm/559-wrf-mslp-6hprec-ita
[Fri Sep 30 07:41:39.913097 2016] [core:trace4] [pid 17957:tid 140241771534080] util_script.c(568): [client 151.25.206.200:38368] Status: 500 Internal Server Error, referer: http://www.centrometeo.com/modelli-numerici/modello-wrf-nmm/559-wrf-mslp-6hprec-ita
[Fri Sep 30 07:41:39.913130 2016] [core:trace1] [pid 17957:tid 140241771534080] util_script.c(649): [client 151.25.206.200:38368] Status line from script 'php.fcgi': 500 Internal Server Error, referer: http://www.centrometeo.com/modelli-numerici/modello-wrf-nmm/559-wrf-mslp-6hprec-ita
[Fri Sep 30 07:41:39.913162 2016] [core:trace4] [pid 17957:tid 140241771534080] util_script.c(568): [client 151.25.206.200:38368] X-Powered-By: PHP/5.6.26, referer: http://www.centrometeo.com/modelli-numerici/modello-wrf-nmm/559-wrf-mslp-6hprec-ita
[Fri Sep 30 07:41:39.913191 2016] [core:trace4] [pid 17957:tid 140241771534080] util_script.c(568): [client 151.25.206.200:38368] P3P: CP="NOI ADM DEV PSAi COM NAV OUR OTRo STP IND DEM", referer: http://www.centrometeo.com/modelli-numerici/modello-wrf-nmm/559-wrf-mslp-6hprec-ita
[Fri Sep 30 07:41:39.913218 2016] [core:trace4] [pid 17957:tid 140241771534080] util_script.c(568): [client 151.25.206.200:38368] Content-Encoding: gzip, referer: http://www.centrometeo.com/modelli-numerici/modello-wrf-nmm/559-wrf-mslp-6hprec-ita
[Fri Sep 30 07:41:39.913246 2016] [core:trace4] [pid 17957:tid 140241771534080] util_script.c(568): [client 151.25.206.200:38368] Vary: Accept-Encoding, referer: http://www.centrometeo.com/modelli-numerici/modello-wrf-nmm/559-wrf-mslp-6hprec-ita
[Fri Sep 30 07:41:39.913275 2016] [core:trace4] [pid 17957:tid 140241771534080] util_script.c(568): [client 151.25.206.200:38368] Content-type: text/html; charset=UTF-8, referer: http://www.centrometeo.com/modelli-numerici/modello-wrf-nmm/559-wrf-mslp-6hprec-ita
[Fri Sep 30 07:41:39.913395 2016] [http:trace3] [pid 17957:tid 140241771534080] http_filters.c(1006): [client 151.25.206.200:38368] Response sent with status 500, headers:, referer: http://www.centrometeo.com/modelli-numerici/modello-wrf-nmm/559-wrf-mslp-6hprec-ita
[Fri Sep 30 07:41:39.913432 2016] [http:trace4] [pid 17957:tid 140241771534080] http_filters.c(835): [client 151.25.206.200:38368] X-Powered-By: PHP/5.6.26, referer: http://www.centrometeo.com/modelli-numerici/modello-wrf-nmm/559-wrf-mslp-6hprec-ita
[Fri Sep 30 07:41:39.913458 2016] [http:trace4] [pid 17957:tid 140241771534080] http_filters.c(835): [client 151.25.206.200:38368] P3P: CP=\\"NOI ADM DEV PSAi COM NAV OUR OTRo STP IND DEM\\", referer: http://www.centrometeo.com/modelli-numerici/modello-wrf-nmm/559-wrf-mslp-6hprec-ita
[Fri Sep 30 07:41:39.913481 2016] [http:trace4] [pid 17957:tid 140241771534080] http_filters.c(835): [client 151.25.206.200:38368] Content-Encoding: gzip, referer: http://www.centrometeo.com/modelli-numerici/modello-wrf-nmm/559-wrf-mslp-6hprec-ita
[...]
Note the use of fcgi. Apache is indeed configured with event/php-fpm/FastCGI, but the same thing happens with "normal" prefork / php!
Until a few days ago it did not happen, it started after updating the system (kernel, apache, php, mariadb).
I do not really know what to do.
Thank you very much.
Additional information:
/var/log/httpd/error_log, at "crash time":
[Fri Sep 30 19:09:03.897325 2016] [mpm_event:trace4] [pid 30339:tid 139796798162688] event.c(930): socket reached timeout in lingering-close state
Related
I make following the post https://www to https://no-www redirection.
I have finally managed to generate a wildcard certificate *.website.com which allows me with rewrite rules to get redirection to https://website.com from initially https://www.website.com.
Now, I am faced to another issue: my CGI scripts in cgi-bin directory are not working anymore like for example: https://website.com/cgi-bin/awstats.pl
I am using the following rewrite rules to get https://www.website.com to https://webiste.com (using zope framework behind apache) :
<VirtualHost *:443>
# Name
ServerAdmin admin#website.com
ServerName website.com
ServerAlias www.website.com
# LOG
CustomLog /var/log/apache2/access.log combined
# ACTIVATE SSL
SSLEngine On
SSLCertificateFile /etc/letsencrypt/live/website.com/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/website.com/privkey.pem
SSLCertificateChainFile /etc/letsencrypt/live/website.com/chain.pem
# REWRITE
RewriteEngine On
RewriteCond %{REQUEST_URI} !^/cgi-bin/awstats [NC]
RewriteCond %{HTTP_HOST} ^www\.(.*)$ [NC]
RewriteRule ^(.*)$ https://%1/$1 [R=301,L]
RewriteRule ^/(.*) https://localhost:8443/++vh++https:%{SERVER_NAME}:443/++/$1 [P,L]
SSLProxyEngine On
RequestHeader set Front-End-Https "On"
#CacheDisable *
ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/
Alias /awstatsclasses "/usr/share/awstats/lib/"
Alias /awstats-icon "/usr/share/awstats/icon/"
Alias /awstatscss "/usr/share/doc/awstats/examples/css"
<Directory "/usr/lib/cgi-bin/">
AllowOverride None
Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
Order allow,deny
Allow from all
SSLRequireSSL
</Directory>
</VirtualHost>
<VirtualHost *:80>
ServerAdmin admin#website.com
ServerName website.com
ServerAlias www.website.com
RewriteCond %{REQUEST_URI} ^/www\. [NC,OR]
RewriteCond %{REQUEST_URI} !^/podcast [NC]
# Rewrite below works : redirect 80 => https
RewriteRule ^/(.*) https://website.com/$1 [R=301,L]
# For Zope
RewriteRule ^/(.*) http://localhost:9674/++vh++http:%{SERVER_NAME}:80/++/$1 [P,L]
</IfModule>
</VirtualHost>
It's pretty tricky but the result is that if I type : https://website.com/cgi-bin/awstats.pl, I get the equivalent of a 404 error of Apache2 but coming from Zope.
How to make work my CGI scripts again ?
It's frustrating from previous post : I have fixed the redirection https://www.website.com to https://website.com but right now, these are the CGI scripts which are no longer accessible.
Before the modifications about the redirection https://www to https://no-www, the scripts were available. I don't understand where it could come from.
Update 1
Output of Apache2:
[Sun Mar 01 10:49:33.445944 2020] [ssl:debug] [pid 9866] ssl_engine_kernel.c(383): [client 91.171.129.151:7825] AH02034: Subsequent (No.7) HTTPS request received for child 7 (server website.com:443), referer: https://website.com/style/style2.css
[Sun Mar 01 10:49:33.445986 2020] [authz_core:debug] [pid 9866] mod_authz_core.c(846): [client 91.171.129.151:7825] AH01628: authorization result: granted (no directives), referer: https://website.com/style/style2.css
[Sun Mar 01 10:49:33.446022 2020] [proxy:debug] [pid 9866] mod_proxy.c(1249): [client 91.171.129.151:7825] AH01143: Running scheme https handler (attempt 0), referer: https://website.com/style/style2.css
[Sun Mar 01 10:49:33.446032 2020] [proxy:debug] [pid 9866] proxy_util.c(2316): AH00942: HTTPS: has acquired connection for (*)
[Sun Mar 01 10:49:33.446041 2020] [proxy:debug] [pid 9866] proxy_util.c(2369): [client 91.171.129.151:7825] AH00944: connecting https://localhost:8443/++vh++https:website.com:443/++/images/up-arrow.png to localhost:8443, referer: https://website.com/style/style2.css
[Sun Mar 01 10:49:33.446204 2020] [proxy:debug] [pid 9866] proxy_util.c(2578): [client 91.171.129.151:7825] AH00947: connected /++vh++https:website.com:443/++/images/up-arrow.png to localhost:8443, referer: https://website.com/style/style2.css
[Sun Mar 01 10:49:33.446288 2020] [proxy:debug] [pid 9866] proxy_util.c(3047): AH02824: HTTPS: connection established with 127.0.0.1:8443 (*)
[Sun Mar 01 10:49:33.446307 2020] [proxy:debug] [pid 9866] proxy_util.c(3215): AH00962: HTTPS: connection complete to 127.0.0.1:8443 (localhost)
[Sun Mar 01 10:49:33.446320 2020] [ssl:info] [pid 9866] [remote 127.0.0.1:8443] AH01964: Connection to child 0 established (server website.com:443)
[Sun Mar 01 10:49:33.454637 2020] [proxy:debug] [pid 9865] proxy_util.c(2331): AH00943: *: has released connection for (*)
[Sun Mar 01 10:49:33.454721 2020] [ssl:debug] [pid 9865] ssl_engine_io.c(1106): [remote 127.0.0.1:8443] AH02001: Connection closed to child 0 with standard shutdown (server website.com:443)
[Sun Mar 01 10:49:33.454772 2020] [proxy:debug] [pid 9865] proxy_util.c(3154): [remote 127.0.0.1:8443] AH02642: proxy: connection shutdown
[Sun Mar 01 10:49:33.459030 2020] [proxy:debug] [pid 9851] proxy_util.c(2331): AH00943: *: has released connection for (*)
[Sun Mar 01 10:49:33.459109 2020] [ssl:debug] [pid 9851] ssl_engine_io.c(1106): [remote 127.0.0.1:8443] AH02001: Connection closed to child 0 with standard shutdown (server website.com:443)
[Sun Mar 01 10:49:33.459144 2020] [ssl:debug] [pid 9866] ssl_engine_kernel.c(1740): [remote 127.0.0.1:8443] AH02275: Certificate Verification, depth 0, CRL checking mode: none (0) [subject: CN=website.com / issuer: CN=Let's Encrypt Authority X3,O=Let's Encrypt,C=US / serial: 033E19116893A728CDC809BA511D98069F7E / notbefore: Jun 29 23:22:00 2017 GMT / notafter: Sep 27 23:22:00 2017 GMT]
[Sun Mar 01 10:49:33.459161 2020] [proxy:debug] [pid 9851] proxy_util.c(3154): [remote 127.0.0.1:8443] AH02642: proxy: connection shutdown
[Sun Mar 01 10:49:33.459193 2020] [ssl:debug] [pid 9866] ssl_engine_kernel.c(1740): [remote 127.0.0.1:8443] AH02275: Certificate Verification, depth 0, CRL checking mode: none (0) [subject: CN=website.com / issuer: CN=Let's Encrypt Authority X3,O=Let's Encrypt,C=US / serial: 033E19116893A728CDC809BA511D98069F7E / notbefore: Jun 29 23:22:00 2017 GMT / notafter: Sep 27 23:22:00 2017 GMT]
[Sun Mar 01 10:49:33.463339 2020] [ssl:debug] [pid 9866] ssl_engine_kernel.c(2235): [remote 127.0.0.1:8443] AH02041: Protocol: TLSv1, Cipher: AES256-SHA (256/256 bits)
[Sun Mar 01 10:49:33.463411 2020] [proxy:debug] [pid 9853] proxy_util.c(2331): AH00943: *: has released connection for (*)
[Sun Mar 01 10:49:33.463486 2020] [ssl:debug] [pid 9853] ssl_engine_io.c(1106): [remote 127.0.0.1:8443] AH02001: Connection closed to child 0 with standard shutdown (server website.com:443)
[Sun Mar 01 10:49:33.463534 2020] [proxy:debug] [pid 9853] proxy_util.c(3154): [remote 127.0.0.1:8443] AH02642: proxy: connection shutdown
[Sun Mar 01 10:49:33.471527 2020] [proxy:debug] [pid 9866] proxy_util.c(2331): AH00943: *: has released connection for (*)
[Sun Mar 01 10:49:33.471590 2020] [ssl:debug] [pid 9866] ssl_engine_io.c(1106): [remote 127.0.0.1:8443] AH02001: Connection closed to child 0 with standard shutdown (server website.com:443)
[Sun Mar 01 10:49:33.471627 2020] [proxy:debug] [pid 9866] proxy_util.c(3154): [remote 127.0.0.1:8443] AH02642: proxy: connection shutdown
[Sun Mar 01 10:49:33.511179 2020] [ssl:debug] [pid 9853] ssl_engine_kernel.c(383): [client 91.171.129.151:7821] AH02034: Subsequent (No.8) HTTPS request received for child 4 (server website.com:443)
[Sun Mar 01 10:49:33.511249 2020] [authz_core:debug] [pid 9853] mod_authz_core.c(846): [client 91.171.129.151:7821] AH01628: authorization result: granted (no directives)
[Sun Mar 01 10:49:33.511303 2020] [proxy:debug] [pid 9853] mod_proxy.c(1249): [client 91.171.129.151:7821] AH01143: Running scheme https handler (attempt 0)
[Sun Mar 01 10:49:33.511332 2020] [proxy:debug] [pid 9853] proxy_util.c(2316): AH00942: HTTPS: has acquired connection for (*)
[Sun Mar 01 10:49:33.511343 2020] [proxy:debug] [pid 9853] proxy_util.c(2369): [client 91.171.129.151:7821] AH00944: connecting https://localhost:8443/++vh++https:website.com:443/++/favicon.ico to localhost:8443
[Sun Mar 01 10:49:33.511551 2020] [proxy:debug] [pid 9853] proxy_util.c(2578): [client 91.171.129.151:7821] AH00947: connected /++vh++https:website.com:443/++/favicon.ico to localhost:8443
[Sun Mar 01 10:49:33.511670 2020] [proxy:debug] [pid 9853] proxy_util.c(3047): AH02824: HTTPS: connection established with 127.0.0.1:8443 (*)
[Sun Mar 01 10:49:33.511696 2020] [proxy:debug] [pid 9853] proxy_util.c(3215): AH00962: HTTPS: connection complete to 127.0.0.1:8443 (localhost)
[Sun Mar 01 10:49:33.511713 2020] [ssl:info] [pid 9853] [remote 127.0.0.1:8443] AH01964: Connection to child 0 established (server website.com:443)
[Sun Mar 01 10:49:33.512494 2020] [ssl:debug] [pid 9853] ssl_engine_kernel.c(1740): [remote 127.0.0.1:8443] AH02275: Certificate Verification, depth 0, CRL checking mode: none (0) [subject: CN=website.com / issuer: CN=Let's Encrypt Authority X3,O=Let's Encrypt,C=US / serial: 033E19116893A728CDC809BA511D98069F7E / notbefore: Jun 29 23:22:00 2017 GMT / notafter: Sep 27 23:22:00 2017 GMT]
[Sun Mar 01 10:49:33.512541 2020] [ssl:debug] [pid 9853] ssl_engine_kernel.c(1740): [remote 127.0.0.1:8443] AH02275: Certificate Verification, depth 0, CRL checking mode: none (0) [subject: CN=website.com / issuer: CN=Let's Encrypt Authority X3,O=Let's Encrypt,C=US / serial: 033E19116893A728CDC809BA511D98069F7E / notbefore: Jun 29 23:22:00 2017 GMT / notafter: Sep 27 23:22:00 2017 GMT]
[Sun Mar 01 10:49:33.517345 2020] [ssl:debug] [pid 9853] ssl_engine_kernel.c(2235): [remote 127.0.0.1:8443] AH02041: Protocol: TLSv1, Cipher: AES256-SHA (256/256 bits)
[Sun Mar 01 10:49:33.525382 2020] [proxy:debug] [pid 9853] proxy_util.c(2331): AH00943: *: has released connection for (*)
[Sun Mar 01 10:49:33.525443 2020] [ssl:debug] [pid 9853] ssl_engine_io.c(1106): [remote 127.0.0.1:8443] AH02001: Connection closed to child 0 with standard shutdown (server website.com:443)
[Sun Mar 01 10:49:33.525476 2020] [proxy:debug] [pid 9853] proxy_util.c(3154): [remote 127.0.0.1:8443] AH02642: proxy: connection shutdown
[Sun Mar 01 10:49:34.109743 2020] [watchdog:debug] [pid 9869] mod_watchdog.c(567): AH02980: Watchdog: nothing configured?
[Sun Mar 01 10:49:34.109885 2020] [proxy:debug] [pid 9869] proxy_util.c(1924): AH00925: initializing worker proxy:reverse shared
[Sun Mar 01 10:49:34.109901 2020] [proxy:debug] [pid 9869] proxy_util.c(1981): AH00927: initializing worker proxy:reverse local
[Sun Mar 01 10:49:34.109955 2020] [proxy:debug] [pid 9869] proxy_util.c(2032): AH00931: initialized single connection worker in child 9869 for (*)
[Sun Mar 01 10:49:34.110492 2020] [watchdog:debug] [pid 9870] mod_watchdog.c(567): AH02980: Watchdog: nothing configured?
[Sun Mar 01 10:49:34.110610 2020] [proxy:debug] [pid 9870] proxy_util.c(1924): AH00925: initializing worker proxy:reverse shared
[Sun Mar 01 10:49:34.110625 2020] [proxy:debug] [pid 9870] proxy_util.c(1981): AH00927: initializing worker proxy:reverse local
[Sun Mar 01 10:49:34.110674 2020] [proxy:debug] [pid 9870] proxy_util.c(2032): AH00931: initialized single connection worker in child 9870 for (*)
[Sun Mar 01 10:49:48.437276 2020] [ssl:debug] [pid 9864] ssl_engine_io.c(1106): [client 91.171.129.151:7823] AH02001: Connection closed to child 5 with standard shutdown (server website.com:443)
[Sun Mar 01 10:49:48.438985 2020] [ssl:debug] [pid 9849] ssl_engine_io.c(1106): [client 91.171.129.151:7822] AH02001: Connection closed to child 0 with standard shutdown (server website.com:443)
[Sun Mar 01 10:49:48.467248 2020] [ssl:debug] [pid 9865] ssl_engine_io.c(1106): [client 91.171.129.151:7824] AH02001: Connection closed to child 6 with standard shutdown (server website.com:443)
[Sun Mar 01 10:49:48.470814 2020] [ssl:debug] [pid 9851] ssl_engine_io.c(1106): [client 91.171.129.151:7820] AH02001: Connection closed to child 2 with standard shutdown (server website.com:443)
[Sun Mar 01 10:49:48.478015 2020] [ssl:debug] [pid 9866] ssl_engine_io.c(1106): [client 91.171.129.151:7825] AH02001: Connection closed to child 7 with standard shutdown (server website.com:443)
[Sun Mar 01 10:49:48.539212 2020] [ssl:debug] [pid 9853] ssl_engine_io.c(1106): [client 91.171.129.151:7821] AH02001: Connection closed to child 4 with standard shutdown (server website.com:443)
[Sun Mar 01 10:49:56.282123 2020] [ssl:info] [pid 9852] [client 127.0.0.1:49482] AH01964: Connection to child 3 established (server website.com:443)
[Sun Mar 01 10:49:56.282356 2020] [ssl:debug] [pid 9852] ssl_engine_kernel.c(2319): [client 127.0.0.1:49482] AH02043: SSL virtual host for servername website.com found
[Sun Mar 01 10:49:56.282407 2020] [ssl:debug] [pid 9852] ssl_engine_kernel.c(2319): [client 127.0.0.1:49482] AH02043: SSL virtual host for servername website.com found
[Sun Mar 01 10:49:56.282418 2020] [core:debug] [pid 9852] protocol.c(2314): [client 127.0.0.1:49482] AH03155: select protocol from , choices=h2,http/1.1 for server website.com
[Sun Mar 01 10:49:56.296616 2020] [ssl:debug] [pid 9852] ssl_engine_kernel.c(2235): [client 127.0.0.1:49482] AH02041: Protocol: TLSv1.2, Cipher: ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)
[Sun Mar 01 10:49:56.296936 2020] [ssl:debug] [pid 9852] ssl_engine_kernel.c(383): [client 127.0.0.1:49482] AH02034: Initial (No.1) HTTPS request received for child 3 (server website.com:443)
[Sun Mar 01 10:49:56.297023 2020] [authz_core:debug] [pid 9852] mod_authz_core.c(846): [client 127.0.0.1:49482] AH01628: authorization result: granted (no directives)
[Sun Mar 01 10:49:56.297087 2020] [proxy:debug] [pid 9852] mod_proxy.c(1249): [client 127.0.0.1:49482] AH01143: Running scheme https handler (attempt 0)
[Sun Mar 01 10:49:56.297101 2020] [proxy:debug] [pid 9852] proxy_util.c(2316): AH00942: HTTPS: has acquired connection for (*)
[Sun Mar 01 10:49:56.297113 2020] [proxy:debug] [pid 9852] proxy_util.c(2369): [client 127.0.0.1:49482] AH00944: connecting https://localhost:8443/++vh++https:website.com:443/++/index.html to localhost:8443
[Sun Mar 01 10:49:56.297467 2020] [proxy:debug] [pid 9852] proxy_util.c(2578): [client 127.0.0.1:49482] AH00947: connected /++vh++https:website.com:443/++/index.html to localhost:8443
[Sun Mar 01 10:49:56.297696 2020] [proxy:debug] [pid 9852] proxy_util.c(3047): AH02824: HTTPS: connection established with 127.0.0.1:8443 (*)
[Sun Mar 01 10:49:56.297722 2020] [proxy:debug] [pid 9852] proxy_util.c(3215): AH00962: HTTPS: connection complete to 127.0.0.1:8443 (localhost)
[Sun Mar 01 10:49:56.297739 2020] [ssl:info] [pid 9852] [remote 127.0.0.1:8443] AH01964: Connection to child 0 established (server website.com:443)
[Sun Mar 01 10:49:56.298590 2020] [ssl:debug] [pid 9852] ssl_engine_kernel.c(1740): [remote 127.0.0.1:8443] AH02275: Certificate Verification, depth 0, CRL checking mode: none (0) [subject: CN=website.com / issuer: CN=Let's Encrypt Authority X3,O=Let's Encrypt,C=US / serial: 033E19116893A728CDC809BA511D98069F7E / notbefore: Jun 29 23:22:00 2017 GMT / notafter: Sep 27 23:22:00 2017 GMT]
[Sun Mar 01 10:49:56.298625 2020] [ssl:debug] [pid 9852] ssl_engine_kernel.c(1740): [remote 127.0.0.1:8443] AH02275: Certificate Verification, depth 0, CRL checking mode: none (0) [subject: CN=website.com / issuer: CN=Let's Encrypt Authority X3,O=Let's Encrypt,C=US / serial: 033E19116893A728CDC809BA511D98069F7E / notbefore: Jun 29 23:22:00 2017 GMT / notafter: Sep 27 23:22:00 2017 GMT]
[Sun Mar 01 10:49:56.303513 2020] [ssl:debug] [pid 9852] ssl_engine_kernel.c(2235): [remote 127.0.0.1:8443] AH02041: Protocol: TLSv1, Cipher: AES256-SHA (256/256 bits)
[Sun Mar 01 10:49:56.312046 2020] [proxy:debug] [pid 9852] proxy_util.c(2331): AH00943: *: has released connection for (*)
[Sun Mar 01 10:49:56.312139 2020] [ssl:debug] [pid 9852] ssl_engine_io.c(1106): [remote 127.0.0.1:8443] AH02001: Connection closed to child 0 with standard shutdown (server website.com:443)
[Sun Mar 01 10:49:56.312204 2020] [proxy:debug] [pid 9852] proxy_util.c(3154): [remote 127.0.0.1:8443] AH02642: proxy: connection shutdown
[Sun Mar 01 10:49:56.312461 2020] [ssl:debug] [pid 9852] ssl_engine_io.c(1106): [client 127.0.0.1:49482] AH02001: Connection closed to child 3 with standard shutdown (server website.com:443):%s/do
And output of Zope:
127.0.0.1 - - [01/Mar/2020:10:49:01 +0200] "GET /++vh++https:www.website.com:443/++/cgi-bin/awstats.pl HTTP/1.1" 404 102 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:73.0) Gecko/20100101 Firefox/73.0"
Update 2
Some interesting results to fix my issue:
If I do: 1)
<VirtualHost *:443>
...
# REWRITE
RewriteEngine On
RewriteCond %{REQUEST_URI} !^/cgi-bin/awstats [NC]
RewriteCond %{HTTP_HOST} ^www\.(.*)$ [NC]
RewriteRule ^/(.*) https://website.com/$1 [R=301,L]
RewriteRule ^/(.*) https://localhost:8443/++vh++https:%{SERVER_NAME}:443/++/$1 [P,L]
...
</VirtualHost>
Then, the redirection from https://www to https:// is well achieved but CGI scripts generates a Zope error.
If I do: 2) remove the line:
`RewriteRule ^/(.*) https://website.com/$1 [R=301,L]` )
i.e :
<VirtualHost *:443>
...
# REWRITE
RewriteEngine On
RewriteCond %{REQUEST_URI} !^/cgi-bin/awstats [NC]
RewriteCond %{HTTP_HOST} ^www\.(.*)$ [NC]
RewriteRule ^/(.*) https://localhost:8443/++vh++https:%{SERVER_NAME}:443/++/$1 [P,L]
...
</VirtualHost>
Then the redirection from https://www to https:// is not achieved but CGI scripts are available by typing in browser https://website.com/cgi-bin/awstats.pl.
How could I combine these 2 different configurations in order to have in the same time redirection and CGI scripts available ?
What you're missing in your workarounds is that the RewriteCond's only associate with the single RewriteRule that immediately follows.
If you want to skip the redirect to zope when the CGI is requested, exclude that particular RewriteRule by preceding it with a condition:
RewriteCond %{REQUEST_URI} !^/cgi-bin/awstats
# existing rule from Question
RewriteRule ^/(.*) https://localhost:8443/++vh++https:%{SERVER_NAME}:443/++/$1 [P,L]
I'm trying to create a generic apache2 webserver as an authentication "gateway".
Scenario:
Someone browses to spn-servername.active-directory.int/secure, apache should try to use kerberos to verify the user (best case with SSO) and redirect him to a backend / another webservice.
It works to the point where the auth and SSO are successful but I don't know how to generalize it for different containers on OpenShift and use the same Active Directory user.
The problem is, if I change the servername of the container and the apache conf servername, auth still works but SSO fails. I guess it's because the SPN of the active directory user and the servername are different?
I want to deploy different applications with different servernames without changing the user/keytab.
What is the best practice to configure multiple apache authentication gateways with different hostnames but with the same Active Directory user?
000default.conf
<VirtualHost *:80>
ServerName generic-hostname.active-directory.int
DocumentRoot "/var/www/html"
<IfModule !mod_auth_kerb.c>
LoadModule auth_gssapi_module /usr/lib/apache2/modules/mod_auth_gssapi.so
</IfModule>
LimitRequestFieldSize 32768
<Location "/secure">
AuthType GSSAPI
AuthName "GSSAPILogin"
GssapiBasicAuth On
GssapiCredStore keytab:/etc/http.keytab
require valid-user
</Location>
LogLevel debug
ErrorLog /var/log/apache2/sso.test.local-error.log
CustomLog /var/log/apache2/sso.test.local-access.log combined
</VirtualHost>
keytab generation:
ktpass -princ HTTP/spn-servername.active-directory.int#active-directory.int -mapuser sysaccount99#active-directory.int -pass mysecret -crypto RC4-HMAC-NT -ptype KRB5_NT_PRINCIPAL -out c:\Temp\http.keytab
Active Directory user:
displayName: sysaccount99
sAMAccountName: sysaccount99
userPrincipalName: HTTP/spn-servername.active-directory.int#active-directory.int
servicePrincipalName: : HTTP/spn-servername.active-directory.int#active-directory.int and HTTP/spn-servername.active-directory.int
/var/log/apache2/sso.test.local-error.log if SSO not working:
[Wed Jan 08 14:00:11.964555 2020] [core:trace5] [pid 871:tid 139656674920192] protocol.c(653): [client 192.168.56.1:55607] Request received from client: GET /secure/ HTTP/1.1
[Wed Jan 08 14:00:11.964643 2020] [http:trace4] [pid 871:tid 139656674920192] http_request.c(394): [client 192.168.56.1:55607] Headers received from client:
[Wed Jan 08 14:00:11.964649 2020] [http:trace4] [pid 871:tid 139656674920192] http_request.c(398): [client 192.168.56.1:55607] Host: generic-hostname.active-directory.int
[Wed Jan 08 14:00:11.964652 2020] [http:trace4] [pid 871:tid 139656674920192] http_request.c(398): [client 192.168.56.1:55607] User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:60.0) Gecko/20100101 Firefox/60.0
[Wed Jan 08 14:00:11.964655 2020] [http:trace4] [pid 871:tid 139656674920192] http_request.c(398): [client 192.168.56.1:55607] Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
[Wed Jan 08 14:00:11.964658 2020] [http:trace4] [pid 871:tid 139656674920192] http_request.c(398): [client 192.168.56.1:55607] Accept-Language: de,en-US;q=0.7,en;q=0.3
[Wed Jan 08 14:00:11.964661 2020] [http:trace4] [pid 871:tid 139656674920192] http_request.c(398): [client 192.168.56.1:55607] Accept-Encoding: gzip, deflate
[Wed Jan 08 14:00:11.964664 2020] [http:trace4] [pid 871:tid 139656674920192] http_request.c(398): [client 192.168.56.1:55607] Connection: keep-alive
[Wed Jan 08 14:00:11.964667 2020] [http:trace4] [pid 871:tid 139656674920192] http_request.c(398): [client 192.168.56.1:55607] Upgrade-Insecure-Requests: 1
[Wed Jan 08 14:00:11.964707 2020] [authz_core:debug] [pid 871:tid 139656674920192] mod_authz_core.c(809): [client 192.168.56.1:55607] AH01626: authorization result of Require valid-user : denied (no authenticated user yet)
[Wed Jan 08 14:00:11.964713 2020] [authz_core:debug] [pid 871:tid 139656674920192] mod_authz_core.c(809): [client 192.168.56.1:55607] AH01626: authorization result of <RequireAny>: denied (no authenticated user yet)
[Wed Jan 08 14:00:11.964728 2020] [auth_kerb:debug] [pid 871:tid 139656674920192] src/mod_auth_kerb.c(1971): [client 192.168.56.1:55607] kerb_authenticate_user entered with user (NULL) and auth_type Kerberos
[Wed Jan 08 14:00:11.964734 2020] [core:trace3] [pid 871:tid 139656674920192] request.c(119): [client 192.168.56.1:55607] auth phase 'check user' gave status 401: /secure/
[Wed Jan 08 14:00:11.964796 2020] [http:trace3] [pid 871:tid 139656674920192] http_filters.c(1129): [client 192.168.56.1:55607] Response sent with status 401, headers:
[Wed Jan 08 14:00:11.964804 2020] [http:trace5] [pid 871:tid 139656674920192] http_filters.c(1136): [client 192.168.56.1:55607] Date: Wed, 08 Jan 2020 14:00:11 GMT
[Wed Jan 08 14:00:11.964807 2020] [http:trace5] [pid 871:tid 139656674920192] http_filters.c(1139): [client 192.168.56.1:55607] Server: Apache/2.4.18 (Ubuntu)
[Wed Jan 08 14:00:11.964810 2020] [http:trace4] [pid 871:tid 139656674920192] http_filters.c(958): [client 192.168.56.1:55607] WWW-Authenticate: Negotiate
[Wed Jan 08 14:00:11.964813 2020] [http:trace4] [pid 871:tid 139656674920192] http_filters.c(958): [client 192.168.56.1:55607] WWW-Authenticate: Basic realm=\\"
[Wed Jan 08 14:00:11.964816 2020] [http:trace4] [pid 871:tid 139656674920192] http_filters.c(958): [client 192.168.56.1:55607] Content-Length: 479
[Wed Jan 08 14:00:11.964819 2020] [http:trace4] [pid 871:tid 139656674920192] http_filters.c(958): [client 192.168.56.1:55607] Keep-Alive: timeout=5, max=100
[Wed Jan 08 14:00:11.964822 2020] [http:trace4] [pid 871:tid 139656674920192] http_filters.c(958): [client 192.168.56.1:55607] Connection: Keep-Alive
[Wed Jan 08 14:00:11.964824 2020] [http:trace4] [pid 871:tid 139656674920192] http_filters.c(958): [client 192.168.56.1:55607] Content-Type: text/html; charset=iso-8859-1
[Wed Jan 08 14:00:11.974410 2020] [core:trace5] [pid 871:tid 139656658134784] protocol.c(653): [client 192.168.56.1:55607] Request received from client: GET /secure/ HTTP/1.1
[Wed Jan 08 14:00:11.974456 2020] [http:trace4] [pid 871:tid 139656658134784] http_request.c(394): [client 192.168.56.1:55607] Headers received from client:
[Wed Jan 08 14:00:11.974469 2020] [http:trace4] [pid 871:tid 139656658134784] http_request.c(398): [client 192.168.56.1:55607] Host: generic-hostname.active-directory.int
[Wed Jan 08 14:00:11.974473 2020] [http:trace4] [pid 871:tid 139656658134784] http_request.c(398): [client 192.168.56.1:55607] User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:60.0) Gecko/20100101 Firefox/60.0
[Wed Jan 08 14:00:11.974476 2020] [http:trace4] [pid 871:tid 139656658134784] http_request.c(398): [client 192.168.56.1:55607] Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
[Wed Jan 08 14:00:11.974479 2020] [http:trace4] [pid 871:tid 139656658134784] http_request.c(398): [client 192.168.56.1:55607] Accept-Language: de,en-US;q=0.7,en;q=0.3
[Wed Jan 08 14:00:11.974482 2020] [http:trace4] [pid 871:tid 139656658134784] http_request.c(398): [client 192.168.56.1:55607] Accept-Encoding: gzip, deflate
[Wed Jan 08 14:00:11.974484 2020] [http:trace4] [pid 871:tid 139656658134784] http_request.c(398): [client 192.168.56.1:55607] Connection: keep-alive
[Wed Jan 08 14:00:11.974487 2020] [http:trace4] [pid 871:tid 139656658134784] http_request.c(398): [client 192.168.56.1:55607] Upgrade-Insecure-Requests: 1
[Wed Jan 08 14:00:11.974490 2020] [http:trace4] [pid 871:tid 139656658134784] http_request.c(398): [client 192.168.56.1:55607] Authorization: Negotiate TlRMLLVNTUAABAAKKl4II4gAAAAAAAABBBBBBBBAAAAGA4AlAAAADw==
[Wed Jan 08 14:00:11.974524 2020] [authz_core:debug] [pid 871:tid 139656658134784] mod_authz_core.c(809): [client 192.168.56.1:55607] AH01626: authorization result of Require valid-user : denied (no authenticated user yet)
[Wed Jan 08 14:00:11.974529 2020] [authz_core:debug] [pid 871:tid 139656658134784] mod_authz_core.c(809): [client 192.168.56.1:55607] AH01626: authorization result of <RequireAny>: denied (no authenticated user yet)
[Wed Jan 08 14:00:11.974561 2020] [auth_kerb:debug] [pid 871:tid 139656658134784] src/mod_auth_kerb.c(1971): [client 192.168.56.1:55607] kerb_authenticate_user entered with user (NULL) and auth_type Kerberos
[Wed Jan 08 14:00:11.974598 2020] [auth_kerb:debug] [pid 871:tid 139656658134784] src/mod_auth_kerb.c(1722): [client 192.168.56.1:55607] Verifying client data using KRB5 GSS-API
[Wed Jan 08 14:00:11.974671 2020] [auth_kerb:debug] [pid 871:tid 139656658134784] src/mod_auth_kerb.c(1738): [client 192.168.56.1:55607] Client didn't delegate us their credential
[Wed Jan 08 14:00:11.974676 2020] [auth_kerb:debug] [pid 871:tid 139656658134784] src/mod_auth_kerb.c(1766): [client 192.168.56.1:55607] Warning: received token seems to be NTLM, which isn't supported by the Kerberos module. Check your IE configuration.
[Wed Jan 08 14:00:11.974681 2020] [auth_kerb:debug] [pid 871:tid 139656658134784] src/mod_auth_kerb.c(1159): [client 192.168.56.1:55607] GSS-API major_status:00010000, minor_status:00000000
[Wed Jan 08 14:00:11.974688 2020] [auth_kerb:error] [pid 871:tid 139656658134784] [client 192.168.56.1:55607] gss_accept_sec_context() failed: An unsupported mechanism was requested (, Unknown error)
[Wed Jan 08 14:00:11.974696 2020] [core:trace3] [pid 871:tid 139656658134784] request.c(119): [client 192.168.56.1:55607] auth phase 'check user' gave status 401: /secure/
[Wed Jan 08 14:00:11.974712 2020] [http:trace3] [pid 871:tid 139656658134784] http_filters.c(1129): [client 192.168.56.1:55607] Response sent with status 401, headers:
[Wed Jan 08 14:00:11.974716 2020] [http:trace5] [pid 871:tid 139656658134784] http_filters.c(1136): [client 192.168.56.1:55607] Date: Wed, 08 Jan 2020 14:00:11 GMT
[Wed Jan 08 14:00:11.974718 2020] [http:trace5] [pid 871:tid 139656658134784] http_filters.c(1139): [client 192.168.56.1:55607] Server: Apache/2.4.18 (Ubuntu)
[Wed Jan 08 14:00:11.974722 2020] [http:trace4] [pid 871:tid 139656658134784] http_filters.c(958): [client 192.168.56.1:55607] WWW-Authenticate: Basic realm=\\"
[Wed Jan 08 14:00:11.974725 2020] [http:trace4] [pid 871:tid 139656658134784] http_filters.c(958): [client 192.168.56.1:55607] Content-Length: 479
[Wed Jan 08 14:00:11.974731 2020] [http:trace4] [pid 871:tid 139656658134784] http_filters.c(958): [client 192.168.56.1:55607] Keep-Alive: timeout=5, max=99
[Wed Jan 08 14:00:11.974734 2020] [http:trace4] [pid 871:tid 139656658134784] http_filters.c(958): [client 192.168.56.1:55607] Connection: Keep-Alive
[Wed Jan 08 14:00:11.974737 2020] [http:trace4] [pid 871:tid 139656658134784] http_filters.c(958): [client 192.168.56.1:55607] Content-Type: text/html; charset=iso-8859-1
In my apache web-server there is a path where clients must authenticate with a valid certificate.
Sometimes there is a client (a soap - webservice) that can't connect, my apache return 403 "sslv3 alert bad certificate (SSL alert number 42) -- Subject CN in certificate not server name or identical to CA!?" and I need to check why and which certificate it is using.
I setup trace3 loglevel for mod_ssl and a customlog like this:
CustomLog /var/log/httpd-ssl.log "%t %h \"%{User-agent}i\" %{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%r\" %>s \"%{SSL_CLIENT_S_DN_CN}x\" <<<%{SSL_CLIENT_CERT}x>>>"
<IfModule mod_ssl.c>
ErrorLog /var/log/apache2/ssl_engine.log
LogLevel trace3
</IfModule>
In the first file log I can see all the informations of client that can connect but when the client fail there aren't the useful information:
[16/Feb/2019:11:01:43 +0100] XXX.XXX.XXX.XXX "IBM WebServices/1.0" - - "POST MYSECRETPATH HTTP/1.1" 403 "-" <<<->>>
In the second one I can see some information like:
[Thu Feb 21 13:57:55.288418 2019] [ssl:debug] [pid 99609] ssl_engine_kernel.c(359): [client xxx.xxx.xxx.xxx:56892] AH02034: Initial (No.1) HTTPS request received for child 5 (server XXX.XXX.XXX:443)
[Thu Feb 21 13:57:55.288591 2019] [ssl:debug] [pid 99609] ssl_engine_kernel.c(743): [client xxx.xxx.xxx.xxx:56892] AH02255: Changed client verification type will force renegotiation
[Thu Feb 21 13:57:55.557866 2019] [ssl:info] [pid 99609] [client xxx.xxx.xxx.xxx:56892] AH02221: Requesting connection re-negotiation
[Thu Feb 21 13:57:55.557902 2019] [ssl:debug] [pid 99609] ssl_engine_kernel.c(970): [client xxx.xxx.xxx.xxx:56892] AH02260: Performing full renegotiation: complete handshake protocol (client does support secure renegotiation)
[Thu Feb 21 13:57:55.557919 2019] [ssl:trace3] [pid 99609] ssl_engine_kernel.c(1988): [client xxx.xxx.xxx.xxx:56892] OpenSSL: Handshake: start
[Thu Feb 21 13:57:55.557932 2019] [ssl:trace3] [pid 99609] ssl_engine_kernel.c(1996): [client xxx.xxx.xxx.xxx:56892] OpenSSL: Loop: SSL renegotiate ciphers
[Thu Feb 21 13:57:55.557948 2019] [ssl:trace3] [pid 99609] ssl_engine_kernel.c(1996): [client xxx.xxx.xxx.xxx:56892] OpenSSL: Loop: SSLv3 write hello request A
[Thu Feb 21 13:57:55.557978 2019] [ssl:trace3] [pid 99609] ssl_engine_kernel.c(1996): [client xxx.xxx.xxx.xxx:56892] OpenSSL: Loop: SSLv3 flush data
[Thu Feb 21 13:57:55.557986 2019] [ssl:trace3] [pid 99609] ssl_engine_kernel.c(1996): [client xxx.xxx.xxx.xxx:56892] OpenSSL: Loop: SSLv3 write hello request C
[Thu Feb 21 13:57:55.557996 2019] [ssl:info] [pid 99609] [client xxx.xxx.xxx.xxx:56892] AH02226: Awaiting re-negotiation handshake
[Thu Feb 21 13:57:55.558005 2019] [ssl:trace3] [pid 99609] ssl_engine_kernel.c(1988): [client xxx.xxx.xxx.xxx:56892] OpenSSL: Handshake: start
[Thu Feb 21 13:57:55.558016 2019] [ssl:trace3] [pid 99609] ssl_engine_kernel.c(1996): [client xxx.xxx.xxx.xxx:56892] OpenSSL: Loop: before accept initialization
[Thu Feb 21 13:57:55.590106 2019] [ssl:debug] [pid 99609] ssl_engine_kernel.c(2141): [client xxx.xxx.xxx.xxx:56892] AH02645: Server name not provided via TLS extension (using default/first virtual host)
[Thu Feb 21 13:57:55.590134 2019] [ssl:trace3] [pid 99609] ssl_engine_kernel.c(1996): [client xxx.xxx.xxx.xxx:56892] OpenSSL: Loop: SSLv3 read client hello A
[Thu Feb 21 13:57:55.590146 2019] [ssl:trace3] [pid 99609] ssl_engine_kernel.c(1996): [client xxx.xxx.xxx.xxx:56892] OpenSSL: Loop: SSLv3 write server hello A
[Thu Feb 21 13:57:55.590177 2019] [ssl:trace3] [pid 99609] ssl_engine_kernel.c(1996): [client xxx.xxx.xxx.xxx:56892] OpenSSL: Loop: SSLv3 write certificate A
[Thu Feb 21 13:57:55.590190 2019] [ssl:trace3] [pid 99609] ssl_engine_kernel.c(1996): [client xxx.xxx.xxx.xxx:56892] OpenSSL: Loop: SSLv3 write certificate request A
[Thu Feb 21 13:57:55.590217 2019] [ssl:trace3] [pid 99609] ssl_engine_kernel.c(1996): [client xxx.xxx.xxx.xxx:56892] OpenSSL: Loop: SSLv3 flush data
[Thu Feb 21 13:57:55.887495 2019] [ssl:trace3] [pid 99609] ssl_engine_kernel.c(2001): [client xxx.xxx.xxx.xxx:56892] OpenSSL: Read: SSLv3 read client certificate A
[Thu Feb 21 13:57:55.887530 2019] [ssl:trace3] [pid 99609] ssl_engine_kernel.c(2020): [client xxx.xxx.xxx.xxx:56892] OpenSSL: Exit: failed in SSLv3 read client certificate A
[Thu Feb 21 13:57:55.887538 2019] [ssl:error] [pid 99609] [client xxx.xxx.xxx.xxx:56892] AH02261: Re-negotiation handshake failed
[Thu Feb 21 13:57:55.887567 2019] [ssl:error] [pid 99609] SSL Library Error: error:14094412:SSL routines:SSL3_READ_BYTES:sslv3 alert bad certificate (SSL alert number 42) -- Subject CN in certificate not server name or identical to CA!?
[Thu Feb 21 13:57:55.887578 2019] [core:trace3] [pid 99609] request.c(117): [client xxx.xxx.xxx.xxx:56892] auth phase 'check access (with Satisfy All)' gave status 403: /my/secret/path
[Thu Feb 21 13:57:55.887611 2019] [http:trace3] [pid 99609] http_filters.c(1003): [client xxx.xxx.xxx.xxx:56892] Response sent with status 403
But there isn't something real useful.
I want to log/write the certificate that apache is reading and not accepting. How can I log it ?
You can enable SSL debugging logs in your Application Server JVM config by adding the following JVM command line parameter and restart the Application Server:
-Djavax.net.debug=all
Depending on your WAS version, adding the above parameter is typically done by navigating to WAS Admin Console > Servers > Application Servers > YourServer > Process Management > Java Virtual Machine > Generic JVM arguments
I have been trying to get AD auth on a virtualhost page working for the past several days, to no avail. Help...
CentOS 7
Apache 2.4.6
mod_ldap and mod_authnz_ldap installed and loading
<VirtualHost *:80>
DocumentRoot /var/www/wwwtest/public
ServerName wwwtest.example.com
ErrorLog logs/wwwtest.example.com-error_log
CustomLog logs/wwwtest.example.com-access_log common
<Directory /var/www/wwwtest/public>
Allow from all
Order Allow,Deny
Options Indexes MultiViews FollowSymLinks
AllowOverride None
AuthType Basic
AuthName "login"
AuthBasicProvider ldap
AuthLDAPBindDN ldapuser#EXAMPLE.COM
AuthLDAPBindPassword ldappassword
AuthLDAPURL "ldap://ldap01.example.com:3268/ou=employees,ou=users,dc=example,dc=com?sAMAccountName?sub?(objectClass=user)"
AuthLDAPBindAuthoritative off
Require valid-user
</Directory>
</VirtualHost>
I have trace8 enabled in /etc/httpd/conf/httpd.conf
And this is what I see in /var/log/httpd/wwwtest.example.com-error.log
[Wed Oct 21 12:12:56.213178 2015] [http:trace4] [pid 20648] http_request.c(301): [client 172.16.250.250:49559] Headers received from client:
[Wed Oct 21 12:12:56.213263 2015] [http:trace4] [pid 20648] http_request.c(305): [client 172.16.250.250:49559] Host: wwwtest.example.com
[Wed Oct 21 12:12:56.213278 2015] [http:trace4] [pid 20648] http_request.c(305): [client 172.16.250.250:49559] User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:41.0) Gecko/20100101 Firefox/41.0
[Wed Oct 21 12:12:56.213284 2015] [http:trace4] [pid 20648] http_request.c(305): [client 172.16.250.250:49559] Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
[Wed Oct 21 12:12:56.213289 2015] [http:trace4] [pid 20648] http_request.c(305): [client 172.16.250.250:49559] Accept-Language: en-US,en;q=0.5
[Wed Oct 21 12:12:56.213293 2015] [http:trace4] [pid 20648] http_request.c(305): [client 172.16.250.250:49559] Accept-Encoding: gzip, deflate
[Wed Oct 21 12:12:56.213297 2015] [http:trace4] [pid 20648] http_request.c(305): [client 172.16.250.250:49559] DNT: 1
[Wed Oct 21 12:12:56.213301 2015] [http:trace4] [pid 20648] http_request.c(305): [client 172.16.250.250:49559] Connection: keep-alive
[Wed Oct 21 12:12:56.213305 2015] [http:trace4] [pid 20648] http_request.c(305): [client 172.16.250.250:49559] Cache-Control: max-age=0
[Wed Oct 21 12:12:56.213309 2015] [http:trace4] [pid 20648] http_request.c(305): [client 172.16.250.250:49559] Authorization: Basic RTAxMDEwMTAxOkNvbmNvcmRpYTIwMTU=
[Wed Oct 21 12:12:56.213530 2015] [authz_core:debug] [pid 20648] mod_authz_core.c(809): [client 172.16.250.250:49559] AH01626: authorization result of Require valid-user : denied (no authenticated user yet)
[Wed Oct 21 12:12:56.213556 2015] [authz_core:debug] [pid 20648] mod_authz_core.c(809): [client 172.16.250.250:49559] AH01626: authorization result of <RequireAny>: denied (no authenticated user yet)
[Wed Oct 21 12:12:56.213644 2015] [authnz_ldap:debug] [pid 20648] mod_authnz_ldap.c(501): [client 172.16.250.250:49559] AH01691: auth_ldap authenticate: using URL ldap://ldap01.example.com:3268/ou=employees,ou=users,dc=example,dc=edu?sAMAccountName?sub?(objectClass=user)
[Wed Oct 21 12:12:56.213705 2015] [authnz_ldap:trace1] [pid 20648] mod_authnz_ldap.c(522): [client 172.16.250.250:49559] auth_ldap authenticate: final authn filter is (&(objectClass=user)(sAMAccountName=TESTUSER))
[Wed Oct 21 12:12:56.215123 2015] [ldap:debug] [pid 20648] util_ldap.c(372): AH01278: LDAP: Setting referrals to On.
[Wed Oct 21 12:12:56.216479 2015] [ldap:trace2] [pid 20648] util_ldap.c(591): [client 172.16.250.250:49559] ldap_simple_bind() failed with server down (try 1)
[Wed Oct 21 12:12:56.217336 2015] [ldap:trace2] [pid 20648] util_ldap.c(591): [client 172.16.250.250:49559] ldap_simple_bind() failed with server down (try 2)
[Wed Oct 21 12:12:56.217358 2015] [ldap:trace2] [pid 20648] util_ldap.c(606): [client 172.16.250.250:49559] attempt to re-init the connection
[Wed Oct 21 12:12:56.217398 2015] [ldap:debug] [pid 20648] util_ldap.c(372): AH01278: LDAP: Setting referrals to On.
[Wed Oct 21 12:12:56.218332 2015] [ldap:trace2] [pid 20648] util_ldap.c(591): [client 172.16.250.250:49559] ldap_simple_bind() failed with server down (try 3)
[Wed Oct 21 12:12:56.219355 2015] [ldap:trace2] [pid 20648] util_ldap.c(591): [client 172.16.250.250:49559] ldap_simple_bind() failed with server down (try 4)
[Wed Oct 21 12:12:56.219392 2015] [ldap:trace2] [pid 20648] util_ldap.c(606): [client 172.16.250.250:49559] attempt to re-init the connection
[Wed Oct 21 12:12:56.219430 2015] [ldap:debug] [pid 20648] util_ldap.c(372): AH01278: LDAP: Setting referrals to On.
[Wed Oct 21 12:12:56.219444 2015] [authnz_ldap:debug] [pid 20648] mod_authnz_ldap.c(539): [client 172.16.250.250:49559] AH01694: auth_ldap authenticate: user TESTUSER authentication failed; URI / [LDAP: ldap_simple_bind() failed][Can't contact LDAP server] (not authoritative)
[Wed Oct 21 12:12:56.219454 2015] [auth_basic:error] [pid 20648] [client 172.16.250.250:49559] AH01618: user TESTUSER not found: /
[Wed Oct 21 12:12:56.219469 2015] [core:trace3] [pid 20648] request.c(119): [client 172.16.250.250:49559] auth phase 'check user' gave status 401: /
[Wed Oct 21 12:12:56.219530 2015] [http:trace3] [pid 20648] http_filters.c(992): [client 172.16.250.250:49559] Response sent with status 401, headers:
[Wed Oct 21 12:12:56.219532 2015] [http:trace5] [pid 20648] http_filters.c(999): [client 172.16.250.250:49559] Date: Wed, 21 Oct 2015 19:12:56 GMT
[Wed Oct 21 12:12:56.219534 2015] [http:trace5] [pid 20648] http_filters.c(1002): [client 172.16.250.250:49559] Server: Apache/2.4.6 (CentOS)
[Wed Oct 21 12:12:56.219536 2015] [http:trace4] [pid 20648] http_filters.c(835): [client 172.16.250.250:49559] WWW-Authenticate: Basic realm=\\”login\\”
[Wed Oct 21 12:12:56.219538 2015] [http:trace4] [pid 20648] http_filters.c(835): [client 172.16.250.250:49559] Content-Length: 381
[Wed Oct 21 12:12:56.219540 2015] [http:trace4] [pid 20648] http_filters.c(835): [client 172.16.250.250:49559] Keep-Alive: timeout=5, max=100
[Wed Oct 21 12:12:56.219541 2015] [http:trace4] [pid 20648] http_filters.c(835): [client 172.16.250.250:49559] Connection: Keep-Alive
[Wed Oct 21 12:12:56.219542 2015] [http:trace4] [pid 20648] http_filters.c(835): [client 172.16.250.250:49559] Content-Type: text/html; charset=iso-8859-1
I can do ldapsearch with these credentials and it returns user objects from our DC, so the credentials are correct. I ran Wireshark on the DC. It never saw any LDAP packets from this web server. I ran tcpdump on the web server and it never sent any LDAP packets when I attempted to auth...
We got AD auth via PHP working in like 10 minutes, but I had previously been working on this for days...so sure, it auth works now, but I want to know why mod_ldap and mod_authnz_ldap aren't working...or...what isn't working.
Also, I'm kinda new with Apache...so the problem is more than likely something I'm misunderstanding.
Thanks in advance.
UPDATE: Apparently it works just fine in Debian. (Apache 2.2.22, bu still) sigh
SOLVED: Clearly I'm still new at Linux as well.
It was, of course, an issue with SELinux. Even though I had set it from Enforcing to Permissive (and then eventually to Disabled), I didn't know that the only way to make that change is apparently by rebooting (or, setenforce 0). Rebooted, and it all worked fine because SELinux was now disabled. I then found that SELinux logs are at /var/log/audit/audit.log. There, were a bunch of:
type=AVC msg=audit(1445466425.176:1849): avc: denied { name_connect } for pid=21184 comm="httpd" dest=389 scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r:ldap_port_t:s0 tclass=tcp_socket
So to allow httpd to access ldap, I followed this post which says:
# getsebool -a | grep ldap
authlogin_nsswitch_use_ldap --> off
httpd_can_connect_ldap --> off
# setsebool httpd_can_connect_ldap 1
# getsebool -a | grep ldap
authlogin_nsswitch_use_ldap --> off
httpd_can_connect_ldap --> on
After that, auth worked perfectly with Firewalld enabled and with SELinux Enforcing. That also explains why a tcpdump showed no ldap bind attempts.
So yeah, hopefully this helps out anyone else that may have been stuck.
Bottom line: learn more about SELinux.
I have PHP 5.3.3 with PHP-FPM running at 172.17.0.163:20533. I tested it with nginx and with cgi-fcgi:
$ SCRIPT_FILENAME=/www/localhost/test.php REQUEST_METHOD=GET cgi-fcgi -bind -connect 172.17.0.163:20533
returns
X-Powered-By: PHP/5.3.3
Content-type: text/html
hello, it works!
So, PHP-FPM is working.
Unfortunately, Apache 2.4 has some issues with PHP-FPM:
<VirtualHost *>
UseCanonicalName Off
VirtualDocumentRoot "/www/%0"
RewriteEngine On
RewriteRule ^/(.*\.php(/.*)?)$ fcgi://172.17.0.163:20533/www/%{SERVER_NAME}/$1 [P]
</VirtualHost>
Opening http:// localhost/test.php returns a "404 Not found" error. Non PHP files are working. Looking at the apache error logs, everything looks fine.
[Fri Nov 15 18:53:00.426776 2013] [mpm_event:info] [pid 1959:tid 140474380953408] AH00490: Server built: Nov 13 2013 14:23:31
[Fri Nov 15 18:53:00.426787 2013] [core:notice] [pid 1959:tid 140474380953408] AH00094: Command line: '/usr/local/sbin/httpd'
[Fri Nov 15 18:53:00.426917 2013] [proxy:debug] [pid 3028:tid 140474380953408] proxy_util.c(1694): AH00925: initializing worker proxy:reverse shared
[Fri Nov 15 18:53:00.426950 2013] [proxy:debug] [pid 3028:tid 140474380953408] proxy_util.c(1734): AH00927: initializing worker proxy:reverse local
[Fri Nov 15 18:53:00.427010 2013] [proxy:debug] [pid 3028:tid 140474380953408] proxy_util.c(1769): AH00930: initialized pool in child 3028 for (*) min=0 max=25 smax=25
[Fri Nov 15 18:53:00.427101 2013] [proxy:debug] [pid 3030:tid 140474380953408] proxy_util.c(1694): AH00925: initializing worker proxy:reverse shared
[Fri Nov 15 18:53:00.427421 2013] [proxy:debug] [pid 3029:tid 140474380953408] proxy_util.c(1694): AH00925: initializing worker proxy:reverse shared
[Fri Nov 15 18:53:00.427445 2013] [proxy:debug] [pid 3029:tid 140474380953408] proxy_util.c(1734): AH00927: initializing worker proxy:reverse local
[Fri Nov 15 18:53:00.427488 2013] [proxy:debug] [pid 3029:tid 140474380953408] proxy_util.c(1769): AH00930: initialized pool in child 3029 for (*) min=0 max=25 smax=25
[Fri Nov 15 18:53:00.427129 2013] [proxy:debug] [pid 3030:tid 140474380953408] proxy_util.c(1734): AH00927: initializing worker proxy:reverse local
[Fri Nov 15 18:53:00.428326 2013] [proxy:debug] [pid 3030:tid 140474380953408] proxy_util.c(1769): AH00930: initialized pool in child 3030 for (*) min=0 max=25 smax=25
[Fri Nov 15 18:53:01.627599 2013] [rewrite:trace2] [pid 3028:tid 140474150618880] mod_rewrite.c(468): [client 172.17.42.1:57951] 172.17.42.1 - - [localhost/sid#7fc2bd82e7f8][rid#7fc2bd7a10a0/initial] init rewrite engine with requested uri /test.php
[Fri Nov 15 18:53:01.627664 2013] [rewrite:trace3] [pid 3028:tid 140474150618880] mod_rewrite.c(468): [client 172.17.42.1:57951] 172.17.42.1 - - [localhost/sid#7fc2bd82e7f8][rid#7fc2bd7a10a0/initial] applying pattern '^/(.*\\.php(/.*)?)$' to uri '/test.php'
[Fri Nov 15 18:53:01.627718 2013] [rewrite:trace2] [pid 3028:tid 140474150618880] mod_rewrite.c(468): [client 172.17.42.1:57951] 172.17.42.1 - - [localhost/sid#7fc2bd82e7f8][rid#7fc2bd7a10a0/initial] rewrite '/test.php' -> 'fcgi://172.17.0.163:20533/www/localhost/test.php'
[Fri Nov 15 18:53:01.627747 2013] [rewrite:trace2] [pid 3028:tid 140474150618880] mod_rewrite.c(468): [client 172.17.42.1:57951] 172.17.42.1 - - [localhost/sid#7fc2bd82e7f8][rid#7fc2bd7a10a0/initial] forcing proxy-throughput with fcgi://172.17.0.163:20533/www/localhost/test.php
[Fri Nov 15 18:53:01.627759 2013] [rewrite:trace1] [pid 3028:tid 140474150618880] mod_rewrite.c(468): [client 172.17.42.1:57951] 172.17.42.1 - - [localhost/sid#7fc2bd82e7f8][rid#7fc2bd7a10a0/initial] go-ahead with proxy request proxy:fcgi://172.17.0.163:20533/www/localhost/test.php [OK]
[Fri Nov 15 18:53:01.627776 2013] [proxy_fcgi:trace1] [pid 3028:tid 140474150618880] mod_proxy_fcgi.c(90): [client 172.17.42.1:57951] canonicalising URL //172.17.0.163:20533/www/localhost/test.php
[Fri Nov 15 18:53:01.627776 2013] [proxy_fcgi:debug] [pid 3028:tid 140474150618880] mod_proxy_fcgi.c(120): [client 172.17.42.1:57951] AH01060: set r->filename to proxy:fcgi://172.17.0.163:20533/www/localhost/test.php
[Fri Nov 15 18:53:01.628070 2013] [proxy:trace2] [pid 3028:tid 140474150618880] proxy_util.c(1857): [client 172.17.42.1:57951] *: found reverse proxy worker for fcgi://172.17.0.163:20533/www/localhost/test.php
[Fri Nov 15 18:53:01.628082 2013] [proxy:debug] [pid 3028:tid 140474150618880] mod_proxy.c(1100): [client 172.17.42.1:57951] AH01143: Running scheme fcgi handler (attempt 0)
[Fri Nov 15 18:53:01.628096 2013] [proxy_fcgi:debug] [pid 3028:tid 140474150618880] mod_proxy_fcgi.c(944): [client 172.17.42.1:57951] AH01076: url: fcgi://172.17.0.163:20533/www/localhost/test.php proxyname: (null) proxyport: 0
[Fri Nov 15 18:53:01.628107 2013] [proxy_fcgi:debug] [pid 3028:tid 140474150618880] mod_proxy_fcgi.c(954): [client 172.17.42.1:57951] AH01078: serving URL //172.17.0.163:20533/www/localhost/test.php
[Fri Nov 15 18:53:01.628134 2013] [proxy:debug] [pid 3028:tid 140474150618880] proxy_util.c(2020): AH00942: FCGI: has acquired connection for (*)
[Fri Nov 15 18:53:01.628147 2013] [proxy:debug] [pid 3028:tid 140474150618880] proxy_util.c(2072): [client 172.17.42.1:57951] AH00944: connecting //172.17.0.163:20533/www/localhost/test.php to 172.17.0.163:20533
[Fri Nov 15 18:53:01.628224 2013] [proxy:debug] [pid 3028:tid 140474150618880] proxy_util.c(2194): [client 172.17.42.1:57951] AH00947: connected /www/localhost/test.php to 172.17.0.163:20533
[Fri Nov 15 18:53:01.628248 2013] [proxy:trace2] [pid 3028:tid 140474150618880] proxy_util.c(2446): FCGI: fam 2 socket created to connect to *
[Fri Nov 15 18:53:01.629453 2013] [proxy_fcgi:trace4] [pid 3028:tid 140474150618880] util_script.c(521): [client 172.17.42.1:57951] Headers from script 'test.php':
[Fri Nov 15 18:53:01.629552 2013] [proxy_fcgi:trace4] [pid 3028:tid 140474150618880] util_script.c(522): [client 172.17.42.1:57951] Status: 404 Not Found
[Fri Nov 15 18:53:01.629583 2013] [proxy_fcgi:trace1] [pid 3028:tid 140474150618880] util_script.c(599): [client 172.17.42.1:57951] Status line from script 'test.php': 404 Not Found
[Fri Nov 15 18:53:01.629595 2013] [proxy_fcgi:trace4] [pid 3028:tid 140474150618880] util_script.c(522): [client 172.17.42.1:57951] X-Powered-By: PHP/5.3.3
[Fri Nov 15 18:53:01.629608 2013] [proxy_fcgi:trace4] [pid 3028:tid 140474150618880] util_script.c(522): [client 172.17.42.1:57951] Content-type: text/html
[Fri Nov 15 18:53:01.629680 2013] [proxy:debug] [pid 3028:tid 140474150618880] proxy_util.c(2035): AH00943: FCGI: has released connection for (*)
It seems as if mod_proxy_fcgi is not sending the script path correctly?! Has anyone an idea?
UPDATE 16 Nov 2013
I tested it with Apache 2.2.25 and mod_fastcgi 2.4.6:
<VirtualHost *>
UseCanonicalName Off
VirtualDocumentRoot "/www/%0"
AddHandler php5-fastcgi .php
FastCgiExternalServer /www/localhost -host 172.17.0.163:20533
</VirtualHost>
Works like a charm. I guess Apache 2.4.6 with mod_proxy_fcgi is buggy.
Update 17 Nov 2013
I tested it with Apache 2.4.6 and mod_proxy_fcgi and PHP 5.4.21. It works. So, there seems to be a problem with PHP 5.3.3 together with Apache 2.4.6 and mod_proxy_fcgi.
Using PHP 5.3.27 fixes all the issues.