This setup was working on an ubuntu 14.04 until a recent update (probably the update/ change in package itself is old).
An internal DNS resolves "piwik.7l" to 192.168.1.17 .
192.168.1.17 serves an apache with piwik on it and following conf
<VirtualHost *:80>
ServerName "piwik.7l"
ServerAlias www.piwik.7l
</VirtualHost>
The setup now does work if addressed by IP, but responds with a 400 Bad Request if browsed to via its domain (piwik.7l). The 400 page does contain
Apache/2.4.7 (Ubuntu) Server at piwik.7l Port 80 and after setting the LogLevel to debug, I see following messages in the error.log:
[Mon May 22 15:12:33.566249 2017] [core:debug] [pid 1559] vhost.c(794): [client 192.168.1.112:38587] AH02415: [strict] Invalid host name 'piwik.7l', problem near: .7l
[Mon May 22 15:12:33.566316 2017] [core:debug] [pid 1559] vhost.c(889): [client 192.168.1.112:38587] AH00550: Client sent malformed Host header: piwik.7l
[Mon May 22 15:12:33.566326 2017] [core:debug] [pid 1559] protocol.c(1356): [client 192.168.1.112:38587] AH00569: client sent HTTP/1.1 request without hostname (see RFC2616 section 14.23): /
In scroll-friendly
[strict] Invalid host name 'piwik.7l', problem near: .7l
AH00550: Client sent malformed Host header: piwik.7l
AH00569: client sent HTTP/1.1 request without hostname (see RFC2616 section 14.23): /
I tried various combinations of IP, port and domain name usage in the VirtualHost/ServerName/ServerAlias blocks and disabled all other VirtualHost-definitions.
To me it sounds as if Apache would try to validate the "tld" part of the domain (7l) in a strict way. Is there any way I can disable that or debug it further?
Edit
The relevant parts of apache2ctl -S:
VirtualHost configuration:
*:80 piwik.7l (/etc/apache2/sites-enabled/000-default.conf:1)
Edit 2
ping piwik.7l does resolve fine from the webserver and from my machine.
Thanks to #savedarios answer here: https://serverfault.com/questions/658537/apache-virtualhost-error-invalid-host-name/841984#841984
If found the solution to be the following change to my apache2.conf:
HttpProtocolOptions Unsafe
Related
I have two apache servers setup on two separate physical machines. My current setup is:
Apache 1 (Reverse Proxy) <===> Apache 2
Both apache server versions are Apache/2.4.29 (Ubuntu) running on Ubuntu 18.04.4 LTS and their /etc/apache2/apache.conf files are identical.
Apache 1 sites-enabled config:
<VirtualHost *:80>
ServerName subdomain.domain.tld
ServerAlias www.subdomain.domain.tld
ServerAdmin webmaster#domain.tld
DocumentRoot /var/www/html
ErrorLog ${APACHE_LOG_DIR}/error.log
CustomLog ${APACHE_LOG_DIR}/access.log combined
<Proxy *>
Order deny,allow
Allow from all
</Proxy>
ProxyRequests off
ProxyPreserveHost On
ProxyPass /maintenance_page !
ProxyPass / http://[apache2-ip-address]:27300/
ProxyPassReverse / http://[apache2-ip-address]:27300/
</VirtualHost>
Apache 2 sites-enabled config:
<VirtualHost *:27300>
ServerName subdomain.domain.tld
ServerAlias www.subdomain.domain.tld
ServerAdmin webmaster#domain.tld
DocumentRoot /var/www/html
ErrorLog ${APACHE_LOG_DIR}/error.log
CustomLog ${APACHE_LOG_DIR}/access.log combined
ErrorDocument 400 /notfound.html
ProxyRequests off
ProxyPreserveHost on
</VirtualHost>
If I directly hit http://[apache2-ip-address]:27300/ from the web browser the apache server landing page comes up fine. If I enter http://subdomain.domain.tld into the browser I get a proxy error:
The proxy server received an invalid response from an upstream server.
The proxy server could not handle the request
I logged a trace on both apache servers. Apache server 2 is receiving the proxied request from apache server 1 and is returning a 200 status response perfectly fine to apache server 1. The flow breaks at apache server 1 where I am seeing the following logs:
[Sat Jul 11 20:34:08.671267 2020] [proxy:debug] [pid 32275:tid 140388069250816] proxy_util.c(3075): AH00962: HTTP: connection complete to [apache2-ip-address]:27300 ([apache2-ip-address])
[Sat Jul 11 20:34:08.671333 2020] [core:trace6] [pid 32275:tid 140388069250816] core_filters.c(525): [remote [apache2-ip-address]:27300] core_output_filter: flushing because of FLUSH bucket
[Sat Jul 11 20:34:08.677508 2020] [proxy_http:error] [pid 32275:tid 140388069250816] (104)Connection reset by peer: [client xx.xxx.xxx.xx:39014] AH01102: error reading status line from remote server [apache2-ip-address]:27300
[Sat Jul 11 20:34:08.677575 2020] [proxy_http:debug] [pid 32275:tid 140388069250816] mod_proxy_http.c(1324): [client xx.xxx.xxx.xx:39014] AH01105: NOT Closing connection to client although reading from backend server [apache2-ip-address]:27300 failed.
[Sat Jul 11 20:34:08.677624 2020] [proxy:error] [pid 32275:tid 140388069250816] [client xx.xxx.xxx.xx:39014] AH00898: Error reading from remote server returned by /
[Sat Jul 11 20:34:08.677681 2020] [proxy:debug] [pid 32275:tid 140388069250816] proxy_util.c(2192): AH00943: HTTP: has released connection for ([apache2-ip-address])
[Sat Jul 11 20:34:08.677724 2020] [http:trace3] [pid 32275:tid 140388069250816] http_filters.c(1128): [client xx.xxx.xxx.xx:39014] Response sent with status 502, headers:
Things I've tried, from few other discussions I could find online, are the following changes to apache server 1 sites-enabled config :
SetEnv proxy-initial-not-pooled 1
SetEnv force-proxy-request-1.0 1
SetEnv proxy-nokeepalive 1
ProxyTimeout 600
ProxyPass / http://[apache2-ip-address]:27300/ timeout=600
ProxyPass / http://[apache2-ip-address]:27300/ nocanon
I've pretty much bruteforced the situation with several combinations of the above settings, but nothing seems to work. Any help is appreciated.
An additional check I ran is, if I run a nodejs application or python flask service on the same machine as either apache servers and proxy the service using ProxyPass / http://localhost:[port]/, the setup works properly. So both apache servers are running fine and are able to proxy services on their respective localhosts. Whatever is breaking has to do with the communication between the two apache servers.
UPDATE : Upon further triaging using curl with a networking person, the issue seems to be that the org firewall is only allowing inbound traffic to apache server 2 and blocking outbound traffic which may be causing 502 errors on apache server 1. This didn't seem like the issue up until I realized that my laptop was VPN'ed into the org network all along while testing and apache server 1 is sitting outside the org network. If this turns out to be the issue it's going to be a real bummer.
Adding following parameter in the http.conf file solves my issue of "proxy: error reading status line from remote server":
SetEnv proxy-initial-not-pooled 1
I go the reference from Apache URL https://httpd.apache.org/docs/2.4/mod/mod_proxy_http.html
Note: restart http server and try again.
In my case, an error with the database's connection was triggering this Apache's Reverse Proxyng error.
I have an apache, which works as reverse proxy for a Netscale Gateway. Sometimes it works, but in many cases the request ends with a 500. The log always give the same error type:
[Mon Aug 20 12:51:24.541905 2018] [http:error] [pid 4919:tid 140600024221440] [client 192.168.22.194:40187] AH02430: Response header 'Set-Cookie' value of 'NSC_TASS=\x0fi\xd4a\xbd\x8e\xcf\xdek\x18\xcd:\x01\xc6d\xf1\xe6;HttpOnly;Path=/;Secure' contains invalid characters, aborting request
[Mon Aug 20 13:03:09.550947 2018] [http:error] [pid 5023:tid 140354590320384] [client 192.168.22.194:24541] AH02430: Response header 'Set-Cookie' value of 'NSC_TASS=le8M1TpPxu5GG1h8nEom8vsA\xe3\x06\x87\x8fnId=&janusWebEvent=PDClass.getJanusServerPage_webEvent_nextPhaseGC2%2C114078_pid_pdPreview_imgPath_res133136%2Fimg%2F_cid_10_clName_ADV_oidHi_10_oidLow_114073;HttpOnly;Path=/;Secure' contains invalid characters, aborting request
[Mon Aug 20 13:09:15.239058 2018] [http:error] [pid 5330:tid 140134346917632] [client 192.168.22.194:40606] AH02430: Response header 'Set-Cookie' value of 'NSC_TASS=\x95\x15\xa91e\xf8\xc8\x96\xdfI\x02\x89\xf4y\x05\xf2&;HttpOnly;Path=/;Secure' contains invalid characters, aborting request
I think it is the backslash in the cookie value, because backslashes are not allowed.
Can apache skip the validating of this or can I write a rule to rewrite the cookie value to a url-encoding\utf-8... value?
My config:
<VirtualHost *:80> # a balancer managed the ssl
ServerName ng.subdomain.domain.tld
SSLProxyEngine On
SSLProxyVerify none
SSLProxyCheckPeerCN off
SSLProxyCheckPeerName off
SSLProxyCheckPeerExpire off
ProxyRequests Off
#ProxyPreserveHost On
ProxyPass / https://ng.domain2.tld/
ProxyPassReverse / https://ng.domain2.tld/
#ProxyPass / wss://ng.domain2.tld/
#ProxyPassReverse / wss://ng.domain2.tld/
ProxyPassReverseCookieDomain ng.subdomain.domain.tld ng.domain2.tld
</VirtualHost>
Perhaps you can set the HttpProtocolOptions to LenientMethods, it might be set at Strict by default. This will only work if your proxy targter returns 501 for invalid restuls.
For more information see the Apache documentation on HttpProtocolOptions.
An easy fix would be to simply have the netscaler encrypt its session cookies.
https://support.citrix.com/article/CTX220162
To Encrypt session cookies as suggested above:
set lb parameter -useSecuredPersistenceCookie Enabled-cookiePassphrase
Example :
set lb parameter -useSecuredPersistenceCookie Enabled-cookiePassphrase test
I configured apache server as a reverse proxy and it works fine if I point a backend server as HTTP. That is:
I configured virtual host 443 like:
ProxyPass /primary/store http://localhost:9763/store/
ProxyPassReverse /primary/store http://localhost:9763/store/
Here users will access the server like https://localhost/primary/store
And this works fine... But I want to configure HTTP server like;
ProxyPass /primary/store https://localhost:9443/store/
ProxyPassReverse /primary/store https://localhost:9443/store/
When I configure like apache server gives 500 internal server error. What am I doing wrong here?
The error I get is:
The server encountered an internal error or misconfiguration and was unable to complete your request.
Please contact the server administrator, you#example.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.
More information about this error may be available in the server error log.
In the apache error log it states;
nt: SSLProxyEngine]
[Mon Aug 04 00:03:26 2014] [error] proxy: HTTPS: failed to enable ssl support for [::1]:9443 (localhost)
[Mon Aug 04 00:03:31 2014] [error] [client ::1] SSL Proxy requested for localhost:443 but not enabled [Hint: SSLProxyEngine]
[Mon Aug 04 00:03:31 2014] [error] proxy: HTTPS: failed to enable ssl support for [::1]:9443 (localhost)
[Mon Aug 04 00:03:51 2014] [error] [client ::1] SSL Proxy requested for localhost:443 but not enabled [Hint: SSLProxyEngine]
[Mon Aug 04 00:03:51 2014] [error] proxy: HTTPS: failed to enable ssl support for [::1]:9443 (localhost)
How to configure http server to talk to HTTPS server?
Your server tells you exactly what you need : [Hint: SSLProxyEngine]
You need to add that directive to your VirtualHost before the Proxy directives :
SSLProxyEngine on
ProxyPass /primary/store https://localhost:9763/store/
ProxyPassReverse /primary/store https://localhost:9763/store/
See the doc for more detail.
In my case, my server was configured to work only in https mode, and error occured when I try to access http mode. So changing http://my-service to https://my-service helped.
For background information: (Question at bottom)
I'm trying to connect to a client, who has 8 servers, all of which have unique IP addresses. The client uses the same SSL certificate on all the servers (for this example, cert name == www.all_servers.com). The client only allows incoming requests over https.
I'm trying to create an apache proxy using mod_proxy that maps different URI mappings to different servers. For example:
https://PROXY_SERVER/SERVER1/{REQUEST}
This would send {REQUEST} to server1
https://PROXY_SERVER/SERVER2/{REQUEST}
would send {REQUEST} to server2.
So far, pretty simple.
In Apache 2.2, this could be achieved by using the IP addresses like so:
SSLProxyEngine On
ProxyPass /server1 https://1.1.1.1/
ProxyPassReverse /server1 https://1.1.1.1/
ProxyPass /server2 https://1.1.1.2/
ProxyPassReverse /server2 https://1.1.1.2/
This was due to Apache 2.2 not checking if the certificate matched (1.1.1.1 != www.all_servers.com)
However, in Apache 2.4, I'm now getting certificate issues (rightly so). (This exact code works on an apache 2.2 box)
[Thu Oct 10 12:01:48.571246 2013] [proxy:error] [pid 13282:tid 140475667224320] (502)Unknown error 502: [client 192.168.1.1:48967] AH01084: pass request body failed to 1.1.1.1:443 (1.1.1.1)
[Thu Oct 10 12:01:48.571341 2013] [proxy:error] [pid 13282:tid 140475667224320] [client 192.168.1.1:48967] AH00898: Error during SSL Handshake with remote server returned by /server1/asd
[Thu Oct 10 12:01:48.571354 2013] [proxy_http:error] [pid 13282:tid 140475667224320] [client 192.168.1.1:48967] AH01097: pass request body failed to 1.1.1.1:443 (1.1.1.1) from 192.168.1.1 ()
I can't use /etc/hosts, as one server would work, using:
1.1.1.1 www.all_servers.com
SSLProxyEngine On
ProxyPass /server1 https://www.all_servers.com/
ProxyPassReverse /server1 https://www.all_servers.com/
But many servers wouldn't
So, to the actual question:
Is there a way to force mod_proxy to ignore miss-matching certificates. Or, is there a better way to do this.
Thanks for any help with this!
You can set the SSLProxy* options on your Apache server (which is a client as far as the reverse proxy connections are concerned).
This was done with SSLProxyCheckPeerCN (off by default in 2.2, but on by default in 2.4), but I'm not sure how this is going to work with IP addresses (since having IP addresses in the CN is not standard). There's a new option in Apache Httpd 2.4 for checking SANs (SSLProxyCheckPeerName), but I'm not sure how it behaves for IP addresses either.
Having IP addresses in DNS SAN extensions or in the CN is not standard compliant with HTTPS:
If a subjectAltName extension of type dNSName is present, that MUST be used as the identity. Otherwise, the (most specific) Common
Name field in the Subject field of the certificate MUST be used.
Although the use of the Common Name is existing practice, it is
deprecated and Certification Authorities are encouraged to use the
dNSName instead.
[...]
In some cases, the URI is specified as an IP address rather than a hostname. In this case, the iPAddress subjectAltName must be present
in the certificate and must exactly match the IP in the URI.
Here are just a few lines from my Apache 2.0 error_log:
[Sun Nov 25 08:22:04 2012] [error] [client 64.34.195.190] File does not exist: /var/www/vhosts/default/htdocs/admin
[Sun Nov 25 14:14:32 2012] [error] [client 96.254.171.2] File does not exist: /var/www/vhosts/default/htdocs/azenv.php
[Wed Nov 28 03:02:01 2012] [error] [client 91.205.189.15] File does not exist: /var/www/vhosts/default/htdocs/user
[Wed Nov 28 03:44:35 2012] [error] [client 66.193.171.223] File does not exist: /var/www/vhosts/default/htdocs/vtigercrm
[Mon Dec 03 00:09:16 2012] [error] [client 82.223.239.68] File does not exist: /var/www/vhosts/default/htdocs/jmx-console
[Mon Dec 03 20:48:44 2012] [error] [client 221.2.209.46] File does not exist: /var/www/vhosts/default/htdocs/manager
[Thu Dec 06 07:37:04 2012] [error] [client 116.254.203.24] File does not exist: /var/www/vhosts/default/htdocs/w00tw00t.at.blackhats.romanian.anti-sec:)
[Thu Dec 06 07:37:05 2012] [error] [client 116.254.203.24] File does not exist: /var/www/vhosts/default/htdocs/phpMyAdmin
[Thu Dec 06 07:37:05 2012] [error] [client 116.254.203.24] File does not exist: /var/www/vhosts/default/htdocs/phpmyadmin
[Thu Dec 06 07:37:06 2012] [error] [client 116.254.203.24] File does not exist: /var/www/vhosts/default/htdocs/pma
[Thu Dec 06 07:37:06 2012] [error] [client 116.254.203.24] File does not exist: /var/www/vhosts/default/htdocs/myadmin
[Thu Dec 06 07:37:07 2012] [error] [client 116.254.203.24] File does not exist: /var/www/vhosts/default/htdocs/MyAdmin
[Thu Dec 13 02:19:53 2012] [error] [client 96.254.171.2] File does not exist: /var/www/vhosts/default/htdocs/judge.php
The most common errors are requests for the "phpMyAdmin" file, and "w00tw00t.at.blackhats.romanian.anti-sec:)".
I can see the IP address that the requests are coming from. But who is "client"?
Thanks,
Shane.
This is just an automatic script deployed by many Script Kiddies looking for a security breach in your apache version/configuration. The signature w00tw00t is usually left by DFind.
Just use a program like fail2ban configured such as this example explains to avoid being flooded by these requests :
https://web.archive.org/web/20160617020600/http://www.userdel.com/post/18618537324/block-w00tw00t-scans-with-fail2ban
This does not necessarily mean you've been hacked, but the server has been scanned for vulnerabilities. However, if you use any of the software that you saw in those logs and it is an older version having known vulnerabilities, you should check your server for unusual files and login activities.
Requests for this are usually sent without a server header. Just create a default virtual host for requests that don't have a server header you expect and blackhole it. Also fun to log broken traffic and do reverse DNS to see if it's coming from another webserver (compromised?) and contact the owner based on whois database. You never know who's running silly scripts from a publicly identifiable server to scan for vulnerabilities and later exploit them over ToR tunnel. Use burner contact information if you don't want to bring attention to yourself.
To follow up on the answer given by #user823629, here is a default virtual host configuration I use on Apache 2.4:
<VirtualHost *:80>
# Default vhost for requests not matching IP or Host of other vhosts
ServerName blackhole
ErrorLog logs/error_log_default
CustomLog logs/access_log_default combined
Redirect 404 /
</VirtualHost>
<VirtualHost *:443>
# Default vhost for requests not matching IP or Host of other vhosts
ServerName blackhole
ErrorLog logs/ssl_error_log_default
CustomLog logs/ssl_access_log_default combined
CustomLog logs/ssl_request_log_default "%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%r\" %b"
SSLEngine on
SSLCertificateFile /etc/pki/tls/certs/localhost.crt
SSLCertificateKeyFile /etc/pki/tls/private/localhost.key
Redirect 404 /
</VirtualHost>
It redirects all requests to the default 404 page. SSL requests that do not match any other site will end up at the second VirtualHost definition, and will of course result in a certificate error, but this is expected and fine.
I put this in conf.d and give it a name of conf.d/0_default.conf so that it comes before other vhosts definitions and it is the default virtual host. This can be verified via:
apachectl -t -D DUMP_VHOSTS
or on Redhat/Fedora/CentOS distros:
httpd -t -D DUMP_VHOSTS
Other virtual hosts will match before this default vhost if:
Their IP address and port matches the VirtualHost definition more explicitly (IP-based virtual host), or
The request contains a Host header that matches the request (name-based virtual host). Otherwise, the request will fall back to the default blackhole virtual host defined above.
Be careful with VirtualHost definitions with IP addresses specified. Since these match before the blackhole, the wrong configuration can become the default for that IP. List the specific IPs in the blackhole if necessary.
See http://httpd.apache.org/docs/current/vhosts/details.html for more details on virtual host matching.
Unless you actually use /var/www/vhosts/default/ for hosting a website, this means you have requests going to the default host that are not being caught by your virtualhosts setup.
Disregard for a moment that these are malicious requests, because the underlying reason for these vhosts/default/ errors is you probably have SSL disabled for a virtualhost, & these are HTTPS requests caught up in the default server config.
You can add %v %V %p to your Apache access logging parameters in httpd.conf to see more info on what these requests are & what virtualhost/servername is handling them (%v %V) & on what port (%p) the requests are being made through (typically port 443 if it's HTTPS).
To fix the HTTPS aspect, I'd enable SSL & then put in a RewriteRule to send HTTPS requests to HTTP (if that's the intended behavior). More info about how to do that here.
Otherwise to help with script kiddies, the blackhole mentioned above is the way to go. Just make sure you aren't mistakenly sending legitimate web crawler/spiders requesting HTTPS to the same galactic death -- for instance Googlebot tests legitimate pages via HTTPS since that's the direction Google wants the web to head in.