I was assign a task by my direct manager to make sure that all the websites in the company will have "X-Forwarded-For" HTTP Header set, in order to receive the original IP of the users for our Web Application Firewall logs.
I am not a developers, but I need to make sure our developers do that, and they seem to not understand what needs to be in the value of the header.
Because looking at some examples, it seemed that some people put specific IP like this:
X-Forwarded-For: <client>, <proxy1>, <proxy2>
which doesn't make any sense to me, because how can u type the IP in the value when it is completely random for each one?
Basically, I need that our logs will contain the real IP from each computer which surf behind a proxy or a load balancer.
Would like for some help : )
Thanks!
If you have one reverse proxy (or load balancer) between the client and the application server, then the proxy should add the header:
X-Forwarded-For: <client>
before forwarding the request on to the application server. The application server receives the request from the proxy IP but can deduce the client's IP from the value of the header.
If you have two reverse proxies (or load balancers) between the client and the application server, the first proxy (the one nearest the client) acts the same as above.
The second proxy receives the request from proxy1's IP and also receives the X-Forwarded-For header from proxy1. It then appends the IP address from where the request was receives (proxy1) and passes the updated header to the application server as:
X-Forwarded-For: <client>, <proxy1>
Each proxy or load balancer is responsible for creating the header if it does not already exist, and appending the IP address of the from where the request was received (i.e. the previous step in the chain).
Only the first IP address is necessary to identify the client, the remaining IP addresses are necessary to ensure that the header has not been faked.
Related
I have a few questions about cloudflare service before I use it.
Are there any difference on http request header between a web server use cf-connecting-ip,x-forwarded-for etc and a web don't use these kinds of thing?
Can/How client knows that a web server uses cf-connecting-ip, x-forwarded-for etc?
These request headers are specific to webproxies, and appended after the request was received by Cloudflare, when it's relaying to your origin. The user cannot see it, unless your webserver for some reason sends it back.
x-forwarded-for: https://en.wikipedia.org/wiki/X-Forwarded-For
Used by proxy servers to tell the origin any HTTP servers involved in relaying the request between the user and the origin. you may see 1 or multiple sets of IP addresses in this header.
cf-connecting-ip:
Relays the IP of the user connecting to Cloudflare to the origin webserver
From: https://support.cloudflare.com/hc/en-us/articles/200170986-How-does-Cloudflare-handle-HTTP-Request-headers-
CF-Connecting-IP Provides the client (visitor) IP address (connecting
to Cloudflare) to the origin web server. cf-connecting-ip contains a
special Cloudflare IP 2a06:98c0:3600:0:0:0:0:103 when the request
originates from a Cloudflare Workers subrequest instead of the
visitor's true IP.
Example:
CF-Connecting-IP: 203.0.113.1
X-Forwarded-For Maintains proxy server and original visitor IP
addresses. If there was no existing X-Forwarded-For header in the
request sent to Cloudflare, X-Forwarded-For has an identical value to
the CF-Connecting-IP header:
Example:
X-Forwarded-For: 203.0.113.1 If an X-Forwarded-For header was already
present in the request to Cloudflare, Cloudflare appends the IP
address of the HTTP proxy to the header:
Example:
X-Forwarded-For: 203.0.113.1,198.51.100.101,198.51.100.102 In the
examples above, 203.0.113.1 is the original visitor IP address and
198.51.100.101 and 198.51.100.102 are proxy server IP addresses provided to Cloudflare via the X-Forwarded-For header.
So for advice, review your application and see if it depends on the IP of the user, if so you will need to modify your web-server configuration to relay the correct IP.
To restore original visitor IP addresses at your origin web server,
Cloudflare recommends your logs or applications look at
CF-Connecting-IP or True-Client-IP instead of X-Forwarded-For since
CF-Connecting-IP and True-Client-IP have a consistent format
containing only one IP.
I came across an IP address / unknown bot that made four HTTP requests, managing to request four different domain names in the following fashion without the first character being a forward slash /:
"GET www.example.com
When I make test the request http://localhost/www.example.com I see the following in Apache:
"GET /www.example.com
All other requests start with a forward slash. How did the bot manage to make such a request and how can I reproduce this to determine how to handle such requests?
Quoted Apache logs reduced to request method and URL to avoid off-topic comments.
Based on the way HTTP requests work, this can be achieved by sending a raw HTTP request to your IP address and specifying both the GET and Host headers as described on the linked page above:
The most common form of Request-URI is that used to identify a
resource on an origin server or gateway. In this case the absolute
path of the URI MUST be transmitted (see section 3.2.1, abs_path) as
the Request-URI, and the network location of the URI (authority) MUST
be transmitted in a Host header field. For example, a client wishing
to retrieve the resource above directly from the origin server would
create a TCP connection to port 80 of the host "www.w3.org" and send
the lines:
GET /pub/WWW/TheProject.html HTTP/1.1
Host: www.w3.org
followed by the remainder of the Request. Note that the absolute path cannot be empty; if none is present in the
original URI, it MUST be given as "/" (the server root).
This can be done on Windows using PuTTY, or on Linux/Mac using nc (see answer here for more details: https://stackoverflow.com/a/3620596/1038813)
How do I set up another Host Header in the cloudflare worker?
For example, I have set up a 1.2.3.4 ip for my site's www record
By default www requests are sent with the header www.ex.com but I want to send the www requests with the new.ex.com header
You need to configure a DNS record for new.ex.com so that it points to the same IP address. Then, you can make a fetch() request with new.ex.com in the URL.
If you cannot make new.ex.com point at the right IP, another alternative is to make a fetch() request using the resolveOverride option to specify a different hostname's IP address to use:
fetch("https://new.ex.com", {cf: {resolveOverride: "www.ex.com"}});
Note that this only works if both hostnames involved are under your zone. Documentation about resolveOverride can be found here.
You cannot directly set the Host header because doing so could allow bypassing of security settings when making requests to third-party servers that also use Cloudflare.
// Parse the URL.
let url = new URL(request.url)
// Change the hostname.
url.hostname = "check-server.example.com"
// Construct a new request
request = new Request(url, request)
Note that this will affect the Host header seen by the origin
(it'll be check-server.example.com). Sometimes people want the Host header to remain the same.
// Tell Cloudflare to connect to `check-server.example.com`
// instead of the hostname specified in the URL.
request = new Request(request,
{cf: {resolveOverride: "check-server.example.com"}})
We run a website which is hosted using WCF.
The website is hosted on: https://foo.com and the ssl certicate is registered using the following command:
netsh http add sslcert hostnameport=foo.com:443
When we browse the website on the server, all is fine, and the certificate is valid.
There is a loadbalance in front of the server which listens to bar.com and then redirects the request to our server.
The loadbalancer doesn't rewrite the get URL, but only the Host Header.
The rewritten header looks like this:
GET https://foo.com/ HTTP/1.1
Host: bar.com
Connection: keep-alive
Now we have some issues which indicates that the ssl certificate is invalid in this case.
The Loadbalancer itself has a certificate registered listening to https://bar.com
Questions:
Is it ok/allowed that the get URL and the Host in the http header are different?
If it is ok to have different values in the header, under which url should we run the site? get URL or Host url?
Well, referencing the RFC2616:
If Request-URI is an absolute URI, the host is part of the
Request-URI. Any Host header field value in the request MUST be
ignored.
So, back to your questions:
It is allowed but a bad idea as it will create confusion, better to use relative path. i.e.
GET /path HTTP/1.1
instead of
GET https://foo.com/path HTTP/1.1.
Modify the loadbalance configuration to do so. Or make the both values the same.
If Host header has a value different than the request URI, then the URI is taking priority over the Hosts header.
I have come to know that x forwarded for gives the ip address o the client who actually sent the request(getremoteaddress() returns the ip address of the machine that sent the reques to the server which can be a proxy). Anyway my question is where is the x forwarded for included in the request. How to extract the client address from it?
I would recommend reading the fine Wikipedia article first:
http://en.wikipedia.org/wiki/X-Forwarded-For
The method for extracting the information from the headers will depend on your platform (which you haven't provided in your question).
Please be aware that X-Forwarded-For headers can be spoofed, and that the machine sending the request may simply pretend to be a proxy and put a fake IP address into the X-Forwarded-For header.
You should only trust X-Forwarded-For headers from your own proxies, not remote ones.