We have a server running Apache providing services via a simple API. We now stumbled upon the problem that we cannot access the API using a third-party library, altough the resulting HTTP request are ALMOST the same. The only difference - as far as we can tell from Wireshark - is the presence or absence of the explicit information about port 80. For example:
curl -d "..." http://www.example.com/foo/bar/
curl -d "..." http://www.example.com:80/foo/bar/
Both work, and Wireshark shows Host: www.example.com, i.e., without the port 80. As far as I understand cURL as well as browser or most other clients remove port 80. So far, all fine.
Now, a third-party library to make requests requires to set a port, and we need to set it to 80. If the library makes a request, Wiresharks now shows Host: www.example.com:80 - note the additional port information. This request fails, and as far as we can see in Wiresharks, this failing request only differs with respect to the host field.
Can this be a configuration issue of Apache? We currently have no direct access to the server to check the conf files. Or are we missing something completely different here.
From rfc 2616:
Host = "Host" ":" host [ ":" port ] ; Section 3.2.2
So "Host: www.example.com:80" is perfectly legitimate. But I have never seen port 80 (or 443 in the case of HTTPS) in the host field of a HTTP request. It is obviously required where the request is routed via a proxy to a non-standard port.
This would give me some concerns as to the quality of the "third-party library". My first of port of call in resolving this would be to speak to the providers of the component - they have presumably come across the problem before.
You did not mention what access you have to the library - did you check that this is not a configurable option? Do you have access to the source code, and the permission to modify it? (if not, that would imply it is commercial, paid-for software - which should give you the right to some support).
I don't know what the solution is, but some obvious things to try would be:
configure the URL at the default vhost for webserver rather than explicitly for www.example.com
or use mod_headers to rewrite the host field
or put a forward proxy in front of the webserver e.g. squid and add a url rewriter (if squid does not automatically strip the port from the host field)
Apache performs string matching with the Host field. So when the :80 is attached, the string matching will fail and Apache will consider it a URL it does not handle and reject it. That is why curl stripped it.
You can read more about the ServerName field here, which is the setting in which Apache matches against Host.
Update
So the :80 has no effect and the string matching still works.
On my production server, I did not change Apache's configuration. I wrote some quick PHP to send out the GET request on a socket, and Apache still responded correctly with the :80 attached to the Host: field.
I also checked on the server itself and see the request come in with the errant :80 attached to it and Apache answers with the status of 200 and presents the HTML.
There is something else wrong with the third party software's request.
Related
As a newbie set up mediawiki in apache httpd on my linux pc. In the corresponding .conf file "require all granted'" is set. The pc was assigned static IP 192.168.0.2 by the router. The port 80 of the pc is opened for any device. I visited http://192.168.0.2/mediawiki/index.php from the host. It worked as expected. Now I tried to visit the same address from my android phone connected to the same network and I was presented with the message "127.0.0.1 refused to connect" by chrome. I noticed that the part http://192.168.0.2 had been replaced by https://127.0.0.1 in the address bar. The same thing also occurred on Firefox on my phone. I can still visit http://192.168.0.2 from my phone and it works just fine. So what is happening here and how to solve this problem?
EDIT: The problem is solved. Mediawiki has this LocalSettings.php file which gets automatically created during installation. It contains a field "$wgServer" which is the "The protocol and server name to use in fully-qualified URLs" as shown in the documentation. This value was automatically set to "http://127.0.0.1" during the installation. Thereby whenever I was trying to connect to mediawiki from my phone I was actually getting redirected to that same phone. I changed the value to "http://192.168.0.2" and now everything is working fine!
Looks like something happened at the connection from your mobile phone to the pc hosting the web server.
Your configuration looks fine even tho you provided almost no details at all: according to your description, I suspect the problem is on your phone or on your router because the domain and url switched from 192.168.0.2 to 127.0.0.1 and the protocol switched from http to https.
You should try the same test from another pc connected to the same network.
You can have a better understanding of what's happening, testing the connection from command line using curl:
$ curl -v http://192.168.0.2/mediawiki/index.php
The output will give you some details of what's going on if the redirect was requested by the webserver or not.
My guess is that your apache is configured to listen to 127.0.0.1 or localhost (it's the default). Open httpd.conf or your virtual host configuration file and look for Listen. Here's some snippets from the official documentation:
The Listen directive tells the server to accept incoming requests only on the specified port(s) or address-and-port combinations. If only a port number is specified in the Listen directive, the server listens to the given port on all interfaces. If an IP address is given as well as a port, the server will listen on the given port and interface. Multiple Listen directives may be used to specify a number of addresses and ports to listen on. The server will respond to requests from any of the listed addresses and ports.
For example, to make the server accept connections on both port 80 and port 8000, on all interfaces, use:
Listen 80
Listen 8000
To make the server accept connections on port 80 for one interface, and port 8000 on another, use
Listen 192.0.2.1:80
Listen 192.0.2.5:8000
More information at
https://httpd.apache.org/docs/2.4/bind.html
Trying to get shibboleth working on a Docker container, using Apache as a reverse proxy. Problems occur because I am using port 80 for the public Apache instance and port 8000 for the internal instance, which confuses shibboleth:
2017-10-03 07:34:23 ERROR OpenSAML.MessageDecoder.SAML2POST [5]: POST targeted at (https://dashboard.hpc.unimelb.edu.au/Shibboleth.sso/SAML2/POST), but delivered to (https://dashboard.hpc.unimelb.edu.au:8000/Shibboleth.sso/SAML2/POST)
The first URL is the external URL which the end user sees. The second URL is what the docker container sees when it gets the proxied request (with the HTTP host name forwarded).
Note I used "ShibURLScheme https" on the internal apache instance to ensure it believes https is being used, as the reverse proxy ensures all requests are https.
Is there anyway I call tell Shibboleth that this is OK, the URLs really are the same? Maybe by rewriting the URL shibboleth sees or something?
Thanks
There are two things I would check:
Make sure the ServerName directive in your Apache conf file is set to
https://dashboard.hpc.unimelb.edu.au:80. Notice the :80. Omitting
the port number completely may work too. You want Apache and vis-a-vis, shibd to see the ServerName that the client is using. You should also make sure you have a UseCanonicalName On directive as part of this.
This is likely less of a problem, but make sure your sp-metadata given to the IdP is correct. You should add entries for the FQDN(s) that the client sees. Please note that most of that documentation page isn't applicable to your scenario, but adding the correct metadata entries is vital.
It looks like your ServerName in Apache isn't correct, so I'd start there.
I have hosting with godaddy with a private IP address. I am able to access my websites and others via the domain name, but not via IP Address.
I am now trying to access the HTTP via port 80 but it gives an error that the website is not setup.
Are there any workarounds on turning allowing accessing the website and other folders through the IP address over http?
This sounds to me like you were on a managed webpack that grants you access to the to-be-served content but not to the webserver or its config itself.
What you are experiencing there is an effect of a technique called virtual-hosts in which multiple sites are tied to one IP address. For this to work in the realm of HTTP, the Host header got introduced in HTTP/1.1. It allows clients to send a domain name for which they want to receive content.
If you request a raw IP, though, the Host header won't look like Host: example.com but rather like Host: 123.45.67.89. The server will not be able to associate a domainname with this as it is unlikely to be found elsewhere in the request. Therefore, it will be forced to serve content out of its default document directory which usually contains further documentations regarding the further configuration of the webserver in question. Linux distributions like to add additional informations specific to them. Check if you see any mentioning of e.g. Debian, RedHat, or Centos.
I am not sure if this is fixable. Usually webservers like Apache support IP-based virtual hosts. If a masshoster like godaddy is really going to address this is uncertain. Try to contact their support and see.
1]If you are using host headers, make sure you have a DNS entry pointing your URL to your IP
2]Telnet port 80 from a machine outside your network.If it fails then check with your ISP that port 80 should not block in firewall.
In my apache instillation, I keep seeing the following line in my access logs:
"POST http://yourinfo.allrequestsallowed.net/ HTTP/1.1" 200
It's really freaking me out because this site is not being hosted on my server (I checked the IP just to be 100% sure). I added a "Deny all" line since the site is still in development, and now the HTTP 200 response changed to 403, like the domain is being hosted on my server.
I'm incredibly confused and scared. Does anybody know what's going on? Can I Deny all to this domain that's apparently pointing to my server?
You may want to check to make sure you don't have ProxyRequests On set anywhere where it's not supposed to. Typically a request like that is for a forward proxy and the troubling bit is that you returned a 200 response which could indicate that the request was successfully proxied.
Take a look at this wiki page about Proxy abuse.
My server is properly configured not to proxy, so why is Apache returning a 200 (Success) status code?
That status code indicates that Apache successfully sent a response to the client, but not necessarily that the response was retrieved from the foreign website.
RFC2616 section 5.1.2 mandates that Apache must accept requests with absolute URLs in the request-URI, even for non-proxy requests. This means that even when proxying is turned off, Apache will accept requests that look like proxy requests. But instead of retrieving the content from the foreign site, Apache will serve the content at the corresponding location on your website. Since the hostname probably doesn't match a name for your site, Apache will look for the content on your default host.
But it's probably worthwhile to check that you aren't proxying. Otherwise, it's not really that big of a deal.
After Jon Lin pointed me in the right direction, I figured it out.
After disabling mod_proxy and enabling mod_security, I added the following to my virtual host configuration:
SecRuleEngine On
SecRule REQUEST_LINE "://" drop,phase:1
And then restarted apache. It quits the connection and returns any amount of data, which uses less resources and bandwidth during Brute Force and DDOS attacks.
Also, it shows as an HTTP 404 Response in the access logs.
EDIT: I updated the rule to drop all types or proxies (https,https,ftp). I don't know how many protocols can be used this way, but I'd rather be safe than sorry.
What are some good ways to do this? Is it even possible to do cleanly?
Ideally I'd like to use packet headers to decide which server should handle requests. However, if there is an easier/better way let me know.
It's impossible for both servers to listen on the same port at the same IP address: since a single socket can only be opened by a single process, only the first server configured for a certain IP/port combination will successfully bind, and the second one will fail.
You will thus need a workaround to achieve what you want. Easiest is probably to run Apache on your primary IP/port combination, and have it route requests for IIS (which should be configured for a different IP and/or port) to it using mod_rewrite.
Keep in mind that the alternative IP and port IIS runs on should be reachable to the clients connecting to your server: if you only have a single IP address available, you should take care to pick an IIS port that isn't generally blocked by firewalls (8080 might be a good option, or 443, even though you're running regular HTTP and not SSL)
P.S. Also, please note that you do need to modify the IIS default configuration using httpcfg before it will allow other servers to run on port 80 on any IP address on the same server: see Micky McQuade's answer for the procedure to do that...
I found this post which suggested to have two separate IP addresses so that both could listen on port 80.
There was a caveat that you had to make a change in IIS because of socket pooling. Here are the instructions based on the link above:
Extract the httpcfg.exe utility from the support tools area on the Win2003 CD.
Stop all IIS services: net stop http /y
Have IIS listen only on the IP address I'd designated for IIS: httpcfg set iplisten -i 192.168.1.253
Make sure: httpcfg query iplisten (The IPs listed are the only IP addresses that IIS will be listening on and no other.)
Restart IIS Services: net start w3svc
Start the Apache service
For people with only one IP address and multiple sites on one server, you can configure IIS to listen on a port other than 80, e.g 8080 by setting the TCP port in the properties of each of its sites (including the default one).
In Apache, enable mod_proxy and mod_proxy_http, then add a catch-all VirtualHost (after all others) so that requests Apache isn't explicitly handling get "forwarded" on to IIS.
<VirtualHost *:80>
ServerName foo.bar
ServerAlias *
ProxyPreserveHost On
ProxyPass / http://127.0.0.1:8080/
</VirtualHost>
Now you can have Apache serve some sites and IIS serve others, with no visible difference to the user.
Edit: your IIS sites must not include their port number in any URLs within their responses, including headers.
You need at least mod_proxy and mod_proxy_http which both are part of the distribution (yet not everytime built automatically). Then you can look here: http://httpd.apache.org/docs/2.2/mod/mod_proxy.html
Simplest config in a virtualhost context is:
ProxyPass /winapp http://127.0.0.1:8080/somedir/
ProxyPassReverse /winapp http://127.0.0.1:8080/somedir/
(Depending on your webapp, the actual config might become more sophisticated. )
That transparently redirects every request on the path winapp/ to the windows server and transfers the resulting output back to the client.
Attention: Take care of the links in the delivered pages: they aren't rewritten, so you can save yourself lotsa hassle if you generally use relative links in your app, like
<a href=../pics/mypic.jpg">
instead of the usual integration nightmare of every link being absolute:
<a href="http://myinternalhostname/somedir/crappydesign.jpg">
THE LATTER IS BAD ALMOST EVERY SINGLE TIME!
For rewriting links in pages there's mod_proxy_html (not to confuse with mod_proxy_http!) but that's another story and a cruel one as well.
Either two different IP addresses (like recommended) or one web server is reverse-proxying the other (which is listening on a port <>80).
For instance: Apache listens on port 80, IIS on port 8080. Every http request goes to Apache first (of course). You can then decide to forward every request to a particular (named virtual) domain or every request that contains a particular directory (e.g. http://www.example.com/winapp/) to the IIS.
Advantage of this concept is that you have only one server listening to the public instead of two, you are more flexible as with two distinct servers.
Drawbacks: some webapps are crappily designed and a real pain in the ass to integrate into a reverse-proxy infrastructure. A working IIS webapp is dependent on a working Apache, so we have some inter-dependencies.
I see this is quite an old post, but came across this looking for an answer for this problem. After reading some of the answers they seem very long winded, so after about 5 mins I managed to solve the problem very simply as follows:
httpd.conf for Apache leave the listen port as 80 and 'Server Name' as FQDN/IP :80.
Now for IIS go to Administrative Services > IIS Manager > 'Sites' in the Left hand nav drop down > in the right window select the top line (default web site) then bindings on the right.
Now select http > edit and change to 81 and enter your local IP for the server/pc and in domain enter either your FQDN (www.domain.com) or external IP close.
Restart both servers ensure your ports are open on both router and firewall, done.
This sounds long winded but literally took 5 mins of playing about. works perfectly.
System:
Windows 8, IIS 8, Apache 2.2
Installing Windows 10 I had this problem: apache(ipv4) and spooler service(ipv6) listening the same 80 port.
I resolved editing apache httpd.conf file changing the line
Listen 80
to
Listen 127.0.0.1:80
That's not quite true. E.g. for HTTP Windows supports URL based port sharing, allowing multiple processes to use the same IP address and Port.
You will need to use different IP addresses. The server, whether Apache or IIS, grabs the traffic based on the IP and Port, which ever they are bound to listen to. Once it starts listening, then it uses the headers, such as the server name to filter and determine what site is being accessed. You can't do it will simply changing the server name in the request