Iptables : How to block (handle) X-Forwarded-For addresses? - reverse-proxy

(I'm in fact, on the CentOS behind AWS ELB. But i don't want this question to be specific for AWS ELB only, but for the general ground.)
I'm working on a CentOS 6.5 box, which is behind a loadbalancer which only passes the X-Forwarded-For IPs to me. As a web server, I know the bad IPs coming in, and I need to block them, from by server itself. (Assuming there is no any IPS/IDS/Firewall in front, to rely on.)
So far, I don't know (I can't) block those bad IPs by my iptables because there's no real IP, but only X-Forwarded-For IPs passed to me. (So what I do is, I am using the .htaccess to block. And it works)
How to achieve this by the proper firewall level please?

I'd like to do the same thing. I've got a very long list of IP addresses to block in CIDR format, and converting it to regular expressions to use in a .htaccess file just doesn't seem like the right thing to do. And you know that in terms of processor load, regular expressions in .htaccess aren't even in the same galaxy as the integer bit-fiddling that iptables can do. But I don't believe that it's possible to use iptables for this. Iptables runs in the kernel, and it blocks the incoming IP addresses at a low level, before any header is read.
In my case, I'm only using the load balancer as a convenient way to handle https requests, I don't really need to balance a heavy load across multiple webserver instances. So what I've been considering is running a separate instance with nginx reverse proxy to handle https for my apache webserver, adding the X_FORWARDED headers exactly like the AWS load balancer does. That way I can use iptables on the instance running nginx, and I don't have to touch my apache configuration or webapps that have been running behind the load balancer.
You lose the redundancy of multiple IP addresses for the load balancer itself, as well as integration with AWS Cloud Front to balance backend load, but you gain ability to use iptables and you can offload processing of static content from apache, perhaps improving your response time. Since nginx is said to be much lighter-weight than apache for simple request processing, you shouldn't need much muscle on that instance. I wonder if AWS load balancers are actually just instances running nginx. If you look at the pricing, the hourly cost of a load balancer is roughly the same as a t2.small linux instance.
I haven't tried this yet myself, as nginx configuration is brand new to me, and it would require buying and installing an SSL certificate instead of using the wonderfully simple and convenient certificate manager.
I wonder if AWS would consider user feature-requests to be able to configure load balancers with iptables...
UPDATE: I just posted this in the AWS EC2 forum.
UPDATE 2: My feature request to AWS asking for a feature to configure iptables for the load balancer got answered with an explanation of how to use a network ACL to block requests originating from any CIDR in a list from reaching the load balancer. To me, that's just as good a solution. The OP was looking for a solution not specific to AWS, and this doesn't meet that criterion. If you have this problem with some server that is behind a reverse proxy, it simply isn't possible to use that server's iptables-style firewall to block incoming requests based on the original IP address - the firewall needs to decide whether to block a request long before it reads the headers, which is the only place that the original requesting address can be found. If you're on AWS, you can use a network ACL. Otherwise you'd need to have full control over the server performing the reverse proxy, and put the firewall rules on that server.

Looks like you can do pattern matching with IP Tables: http://wiztelsys.com/Article_iptables_bob2.html
So you'd have to do that for example:
iptables -I INPUT 1 -p tcp --dport <port> -m string --string "X-Forwarded-For: <ip>" --algo bm -j DROP
"-m": matching type = string
"--string": what string
"--algo bm": Boyer-Moore algorithm for pattern matching

Since you mention .htaccess it sounds like you're using Apache Webserver, so I'd recommend adding these rules there instead of in iptables. The basic way of blocking based on X-Forwarded-For in Apache is this:
RewriteCond %{HTTP:X-FORWARDED-FOR} ^171.42.6.123$
RewriteRule .* - [F,L]
Since the IP address is just a string being matched against then you can specify all sorts of regular expressions in your condition. See this question for an example of doing that.

Related

GCP load balancing ("internal" traffic over HTTPS)

I have a GCP instance group with 2 instances. Both are up and running. I want to configure a load balancer (HTTPS) to manage the traffic.
I've set up a forwarding rule with the HTTP-protocol and a certificate managed by google. This all works, but only when the traffic between the load balancer and the backend (the instances) is plain HTTP.
Steps I did so far
I create a template and this template is just a normal N1 series machine. I checked the boxes to create firewall rules for allowing http and https traffic.
I create a firewall rule named "allow-ports". This firewall rule targets all instances in the network, has a 0.0.0.0/0 IP-range and allow port tcp = 80, 443. How I see this, this firewall rule should open both the http (80) and https (443) port.
I create an instance group with port mapping. "http-port" = 80, "https-port" = 443. I use the template I just created.
When the instance group is created, I check if this is running. With SSH, I get access to the instances and install apache (sudo apt-get install -y apache2) on the both. When navigating to their external IP's in the browser, I see them both.
I create a HTTP(S) load balancer, with the option "From internet to my VMs". For backend configuration, I add a backend service with my instance group, protocol HTTP, named port "http-port". For frontend configuration, I set up the HTTPS protocol, create an IPv4 IP address, create a google-managed ssl certificate, and I'm done. I also added health checks btw.
Now... these steps work (after a few minutes). With the cloud DNS, I have set up a domain name which points to the IP address of the load balancer. When going to , I see the apache page.
What doesn't work?
When I change the backend configuration to HTTPS (and named port "https-port"), I get a 502 server error. So it seems to me that there is some connection, but there is an error. Could this be an apache error?
I have spent a whole day, creating and recreating instance groups, firewall rules, load balancers, ... but nothing seems to work. I'm surely missing something, probably something dumb, but I have no clue what it could be.
What do I want to achieve?
I do not only want a secure (HTTPS) connection between the client and my load balancer, I also want a secure connection between the load balancer and the backend service (the instance group). Because GCP offers the option to use the HTTPS protocol when creating a backend service, I feel that this could be done.
To be honest: I'm reading some articles about the fact that the internal traffic is secured, so a HTTPS connection is not necessary. But that doesn't matter to me, I really want to know how this works!
EDIT
I'm using the correct VPC (default). I also edited the firewall rule from 0.0.0.0/0 to 130.211.0.0/22 and 35.191.0.0/16 (see: https://cloud.google.com/compute/docs/tutorials/globally-autoscaling-a-web-service-on-compute-engine?hl=nl#configure_the_load_balancer).
In addition to my previous comment. I followed your steps at my test project to find out the cause of your issue. I installed the same configuration and checked it with HTTP at the back-end. As it was expected, I found no errors. After that, I installed SSL certificates to the back-end and to the load balancer. Then I switched my back-end, load balancer and health checks to HTTPS and disabled HTTP at the back-end. At this point, I found no errors also.
So, I decided to get 502 error in my test configuration in some way. I switched my health check at the load balancer to HTTP. A few minutes later I tried to reach my test service again and got 502 error. When I switched back my health check to HTTPS 502 error gone away.
During this test, I didn't change firewall rules, but allowed HTTP and HTTPS traffic in my instance template and I used default network.

If two Apache HTTP servers are installed in RedHat, how to make them not disturbing each other

I have already installed an Apache HTTP server in my RedHat system, now I need to install a Bitnami application package which contains another Apache. So I am wondering how to make them not disturbing each other?
I guess I need to configure different ports for the two HTTP server. But what if one has 8080 and another has 9090, will we visit http://[ServerName]:8080/something.html and http://[ServerName]:9090/something.html? I think this way is quite inconvenient. Am I wrong or any better idea?
My advice would be to do something like this.
Have one Apache instance listen in port 80 and the other one in port 8080 for example. The Apache instance that listens in port 80 can act as a proxy to the other Apache (port 8080) using the ProxyPass and ProxyPassReverse directives.
https://httpd.apache.org/docs/2.4/mod/mod_proxy.html
You would need to define prefixes or virtual hosts and inside them add ProxyPass directives.
I don't know to what kind of user those applications are targeted to but the usual end-user is not used to enter ports when browsing the web.
If you like to use the ports, go for it, but I would recommend using Name-based Virtual Host
so you could use different domains or subdomains to each application.
In addition to the example provided by the docs (in where they just point to different folders) in this digitalocean page they document how to make redirects to different urls.
I completely agree with EndermanAPM that usual end-user is not used to enter ports when browsing the web. Therefore, I would only allow port 80 to be accessed by the end-users.
Additional to the current solutions I see another one:
avoid messing up the settings of the Apache servers in order to not end-up with some malfunctions of your websites
leave the Apache servers listen on their designated ports (8080 respectively 9090)
install a dedicated proxy in front of the Apache servers. The proxy would listen on port 80 and would define redirect rules that would parse the request and would redirect it to the proper Apache server. (see the attached picture)
I recommend you HA Proxy. It is a very fast and reliable http and tcp proxy. I've been using it in production for years, in front of application servers, web servers and even database servers. Once you get used with its syntax, it is pretty easy to use.
I am aware that introducing a new component into the equation might add another source of potential issues. But I think that the architecture is cleaner. Besides, the two Apache servers will not be disturbing each other as you requested. You can shut down any one of the two and the other one would properly work further.

Apache - multiple sites on one IP, domain and port with SSL

Is it possible to run multiple apache sites on the same IP, domain and port (meaning the <VirtualHost> tags are exactly identical and no ServerName is given) while using SSL (not sure whether SSL makes a difference here)?
I would like to separate my web services into files in etc/apache2/sites-available to be able to activate or deactivate them on demand. Basically Apache should just take all files and string them together internally, but leave me the possibility to a2dissite certain parts.
Further clarification:
By "sites", I mean files in the etc/apache2/sites-available directory. "Web services" in this context are certain application like phpMyAdmin or an Etherpad which run on the Apache and whose configuration (e.g. Alias or ProxyPass) I want to write into its own configuration file ("site"),
The short answer is no.
SSL operates at a level between TCP and HTTP. But the virtual host name is sent via HTTP. So how does SSL know which certificate to use for a virtual host?
There is a way to do it - basically start up the HTTP over TCP then switch to SSL after the virtual host name (in the Host header) is sent. However this is complex, error prone and generally considered a bad idea.
Best practice is to have one IP per SSL. One machine can have multitple IP addresses, even a single network port can have mulitple IP addresses.

Proxy Protocol on Elastic Load Balancing non-terminated SSL connection

For reasons we're not going to change, our application needs to handle the SSL connection, and not the ELB. The goal of using the Proxy Protocol is to get the client's IP address over an SSL connection.
http://aws.typepad.com/aws/2013/07/elastic-load-balancing-adds-support-for-proxy-protocol.html?ref_=9 indicates "Alternatively, you can use it if you are sending HTTPS requests and do not want to terminate the SSL connection on the load balancer. For more information, please visit the Elastic Load Balancing Guide."
Unfortunately, it appears the guide that's linked to doesn't actually elaborate on this, and the basic documentation for the Proxy Protocol ( http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/enable-proxy-protocol.html ) fails in our environment when configured as described.
Does anyone have steps or a link for this?
The proxy protocol (version 1) injects a single line into the data stream at the beginning of the connection, before SSL is negotiated by your server. You don't get this information "over" an SSL connection; you get the information prior to SSL handshaking. Your server has to implement this capability and specifically be configured so that it can accept and understand it. For an IPv4 connection, it looks like this:
PROXY TCP4 source-ip dest-ip source-port dest-port\r\n
The standard for the protocol is here:
http://haproxy.1wt.eu/download/1.5/doc/proxy-protocol.txt
Additional info in the ELB docs here:
http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/TerminologyandKeyConcepts.html#proxy-protocol
Regarding Apache support, at least at the time AWS announced support for the proxy protocol...
“neither Apache nor Nginx currently support the Proxy Protocol header inserted by the ELB”
— http://aws.typepad.com/aws/2013/07/elastic-load-balancing-adds-support-for-proxy-protocol.html?ref_=9
That is subject to change, of course, but I didn't successfully google for any Apache support of the proxy protocol. Of course, since Apache is open source, you could presumably hack it in there, though I am unfamiliar with the Apache source code.
Realizing that you don't want to change what you're doing now, I still would suggest that depending on your motivation for not wanting to change, there may still be a relatively simple solution. It's a change, but not involving SSL on ELB. What about running HAProxy behind ELB to terminate the SSL in front of Apache? Since HAProxy 1.5 can terminate SSL and appears to be able to translate the proxy protocol string from ELB into an X-Forwarded-For header, as well as generate X-SSL headers to give your application information about the client's SSL cert (perhaps that's your motivation for terminating SSL at the app server instead of on the ELB?) ... so this might be an alternative.
Otherwise, I don't have suggestions unless Apache implements support in the future, or we can find some documentation to indicate that they already have.
For the newer Network Load Balancers which allow your application servers to terminate the TLS connections, you can still get the real IP addresses of your clients and avoid all the work of configuring proxy protocol on the ELBs and in the web server config by simply configuring the target groups to use the servers' instance ids rather than their IP addresses. Regardless of which web server you use, the real IPs of the clients will show up in the logs with no translation needed.
Just to follow up on Michael - sqlbot's answer discussing the AWS support for proxy protocol on EC2 instances behind classic TCP elastic load balancers, the Apache module to use that implements the proxy protocol is mod_remoteip. Enabling it and updating the configuration properly will correct the problem of logging IP addresses of users rather than the elastic load balancer's IPs.
To enable proxy protocol on the elastic load balancer you could use these aws cli commands described in the aws documentation:
aws elb create-load-balancer-policy --load-balancer-name my-elb-name --policy-name my-elb-name-ProxyProtocol-policy --policy-type-name ProxyProtocolPolicyType --policy-attributes AttributeName=ProxyProtocol,AttributeValue=true
aws elb set-load-balancer-policies-for-backend-server --load-balancer-name my-elb-name --instance-port 443 --policy-names my-elb-name-ProxyProtocol-policy
aws elb set-load-balancer-policies-for-backend-server --load-balancer-name my-elb-name --instance-port 80 --policy-names my-elb-name-ProxyProtocol-policy
To enable use of proxy protocol in apache, in a server-wide or VirtualHost context, follow the mod_remoteip documentation such as below:
<IfModule mod_remoteip.c>
RemoteIPProxyProtocol On
RemoteIPHeader X-Forwarded-For
# The IPs or IP range of your ELB:
RemoteIPInternalProxy 192.168.1.0/24
# The IPs of hosts that may need to connect directly to the web server, bypassing the ELB (if applicable):
RemoteIPProxyProtocolExceptions 127.0.0.1
</IfModule>
You'll need to update the LogFormat wherever you have those defined (e.g. httpd.conf) to use %a rather than %h else the load balancer IP addresses will still appear.

Applying IP rules to HTTP only (and not HTTPS) with .htaccess

I have been setting up an IP blocklist reciently and I was wondering is it possible to block an IP that is connecting via HTTP and not to block them if they connect via HTTPS. There was a post on SO .Htaccess rules to redirect respective HTTP links to HTTP and HTTPS to HTTPS? which is similar but uses mod_rewrite which I have had horrible experience with and has only given me 500 errors in the past . Is there any way to do it with the standard format?
order allow,deny
allow from 192.168.1.0/24
deny from all
I need support for IPv6 addresses too. If the rewrite method is the only option, in your answer could you include a link that I could look at to perform my task properly? Many thanks!
I am using Apache/2.2.20 (Ubuntu)
What you desire isn't built into Apache's .htaccess mechanism. Simply: no protocol level commands are supported by mod_auth or mod_access. Furthermore, what you seek breaks the expected assumption that if you provide a resource over HTTP, that same path will work over HTTPS. This will cause surprising results for people using HTTPS enforcers.
But, if you're dead set on doing something like this, I would recommend Squid. You can use it to do all kinds of nifty things, like denying access to the cache from certain protocols on a per-file basis, and otherwise fiddling with data coming off your Apache server before you serve it to your users.