How to use ProxyPass to send a query parameter without encoding it? - apache

In my apache configuration I am matching on some url and trying to redirect that to another location. For the most part this configuration is working. The part that is not is the query parameter on the proxy pass url. All my research is leading me to a RewriteRule, but that makes no sense to me since I am completely throwing away the incoming url as it doesn't matter after it's used to match the Location block.
<Location "/src">
ProxyErrorOverride Off
ProxyPass https://example.com/dst/v1234?user=me
ProxyPassReverse https://example.com/dst/v1234?user=me
Header set Cache-Control "max-age=604800, public"
</Location
When I am testing I can see the following:
Web Server: POST http://localhost/src -d { "data": true }
App Server: POST https://example.com/dst/v1234%3fuser=me -d { "data": true }
The App Server doesn't know how to interpret the %3f even though that's url encoding for the question mark (?). The App Server is not under my control and I can't change that code to decode this for me. How can I get Apache to send the question mark (?) instead of the url encoded variant?

After much research and another very helpful Stack Overflow post. I found a solution!
ProxyPass /webservice balancer://api/webservice nocanon
Apache mod_proxy url encoding
The answer is to use nocanon.
<Location "/src">
ProxyErrorOverride Off
ProxyPass https://example.com/dst/v1234?user=me nocanon
ProxyPassReverse https://example.com/dst/v1234?user=me
Header set Cache-Control "max-age=604800, public"
</Location
Normally, mod_proxy will canonicalise ProxyPassed URLs. But this may be incompatible with some backends, particularly those that make use of PATH_INFO. The optional nocanon keyword suppresses this and passes the URL path "raw" to the backend. Note that this keyword may affect the security of your backend, as it removes the normal limited protection against URL-based attacks provided by the proxy.
https://httpd.apache.org/docs/2.4/mod/mod_proxy.html

Related

Apache ProxyPass not forwarding original http headers to workers

I am sending a request to httpd with a setHeader, named.
I see that header if I do not use ModProxy, using getRequestHeader with the header name.
However I need the header information in the ProxyPass workers.
How can I get Apache to forward the headers in ProxyPass.
I have searched for an answer for this for quite a while but I cannot find something that works.
My ProxyPass directive:
<IfModule mod_proxy_fcgi.c>
#No PATH_INFO with mod_proxy_fcgi unless this is set
SetEnvIf Request_URI . proxy-fcgi-pathinfo=1
ProxyPass "/gas30/" "fcgi://192.168.1.5:6394/"
Alias "/gas30" "/opt/fourjs30/gst/gas/bin/fastcgidispatch enablereuse=on"
ProxyPass "/gas31/" "fcgi://192.168.1.5:6395/"
Alias "/gas31" "/opt/fourjs31/gst/gas/bin/fastcgidispatch enablereuse=on"
</IfModule>
I am using Apache/2.4.6 (CentOS 7)
Thanks in advance
Johan
I use tcpdump to display the header received on Listen port 6380
Just to make sure it is there.
http_auth: Y2FuY3VuQG1ic2JiYW5rLmNvbScgJzExMTExMTExJyAnJDJhJDEyJDJwMlFOMVNZVURldUlpcWw2R3ZWMk83Mi45QUtXSlROcUlnWDRFQVUvZnJjU01SZnJNQXVL
That is the correct header.
I have tried a lot of things to forward that to port 6395.
I am totally stuck on that though.
The last thing I tried was to hard code it under Location.
But I do not see it in tcpdump
<Location /gas31>
Header add http_auth "Y2FuY3VuQG1ic2JiYW5rLmNvbScgJzExMTExMTExJyAnJDJhJDEyJDJwMlFOMVNZVURldUlpcWw2R3ZWMk83Mi45QUtXSlROcUlnWDRFQVUvZnJjU01SZnJNQXVL"
RequestHeader set http_auth "Y2FuY3VuQG1ic2JiYW5rLmNvbScgJzExMTExMTExJyAnJDJhJDEyJDJwMlFOMVNZVURldUlpcWw2R3ZWMk83Mi45QUtXSlROcUlnWDRFQVUvZnJjU01SZnJNQXVL"
Order Deny,Allow
Deny from all
Allow from all
</Location>
I have come to the end of the line with this one, I tried everything and I cannot get the request header to pass through.
.
I can either put my header in the JSON messages I send or move to nginx.
Since I have been using httpd for years I will take the first option.
I saw this post but I cannot add a comment due to reputation.
Is this relevant ?
If it is where do I put these instructions ?
It seems this line preserves the Header for reasons I don't quite understand:
SetEnvIf HTTP_MY_HEADER "(.*)" MY_HEADER=$0
The reason I don't understand this is that I am setting an Env var here, not a header -- are Env vars automatically turned into headers?
I though I might have to do this also, but was unnecessary:
RequestHeader set HTTP_MY_HEADER "${MY_HEADER}e"
I suppose this is an answer as "it works", although I would love to know why...

How to set up a seamless proxy in Apache to get around my ISP's firewall?

I'm really hoping someone can help me out with this because I've been at it for several days and I think I'm going crazy!
I'm trying to do what to me sounds like a stupidly simple thing. I want to set up a proxy server using Apache on a dedicated machine that I rent so that I can get around my ISPs nonsense firewall. I am aware that I could use a VPN, I don't want to do that for reasons that should hopefully become clear after I explain the details of what I want.
First of all, I don't want the proxy server to be used for every request. Only for the sites that are blocked by my ISP.
Suppose I try to access blockedsite.com/path/to/resource and it fails. I then simply want to change the URL in the address bar to proxy.myserver.com/proxy/blockedsite.com/path/to/resource and have Apache handle everything to provide me with a seamless experience. That means,
ProxyPassReverse should modify the response headers to use to the proxy server.
All URLs in the response body should be modified to use the proxy
Here's what I have so far:
<VirtualHost *:80>
ServerName proxy.myserver.com
ProxyRequests off
ProxyPass /proxy/ http://
ProxyPassReverse /proxy/ http://
ProxyPassReverse /proxy/ https://
ProxyHTMLURLMap http:// /proxy/
ProxyHTMLURLMap https:// /proxy/
<Location /proxy/>
ProxyPassReverse /
AddOutputFilterByType SUBSTITUTE text/html
Substitute "s|</title>|</title><meta name='referrer' content='no-referrer' />|ni"
ProxyHTMLEnable On
#ProxyHTMLURLMap / /app1/
RequestHeader unset Accept-Encoding
Order allow,deny
Allow from all
</Location>
</VirtualHost>
This setup works beautifully for URLs that don't try to redirect me elsewhere. But if for example I try to access proxy.myserver.com/proxy/facebook.com I am still being redirected on the client side to https://www.facebook.com instead of https://proxy.myserver.com/proxy/www.facebook.com as I would like. The extra weird thing is that when I set up my own test site which does nothing except redirect me to an HTTPS address, the ProxyPassReverse rule for HTTPS does actually seem to work... but not when I try to access sites like Facebook or Google.
I see no reason to ramble on about my issues, what I'm looking for is astoundingly simple: a transparent, seamless experience! Aside from sticking proxy.myserver.com/proxy/ in front of the URL in the address bar, I shouldn't have to do anything else for it to work. Yet that is not the case and despite over a week of searching, I have found nothing online to help me with this. It's as if I'm the only person in the universe to want to create a simple proxy with Apache that actually works as a firewall-get-arounder.
Please can someone lend me a hand here?! Even just to tell me I'm going about this all wrong and should give up and install Squid or something??
Your last paragraph contains the right answer. You should indeed just "install Squid or something". In particular, I'd recommend Apache Traffic Server - http://trafficserver.apache.org/ - this is exactly what it's made for.
While Apache httpd can do proxying, it's not it's primary function, and so there are always things that will end up being frustrating with it. We could get your above scenario working, but it's really not the right tool for the job.

Null Value for REQUEST_URI from Apache Web Server

My goal is to obtain the original request URL before proxying to another URL. I'm using Apache Http Server 2.4 in conjunction with Jetty. I'm creating a custom header called X-Forwarded-Uri in my httpd.conf file that provides me with the original request URI.
<VirtualHost *:80>
...
RequestHeader set X-Forwarded-Uri %{REQUEST_URI}e
ProxyPass /foo http://localhost:8080/foo
ProxyPassReverse /foo http://localhost:8080/foo
...
</VirtualHost>
However, when I make a request for http://localhost/foo and try to retrieve the value of X-Forwarded-Uri from my server side code, I consistently get back null. The request URI is supposed to be the path that comes after the host name and port number. Since I'm supplying /foo, I would expect to get back /foo.
Do I have a configuration error?
%{REQUEST_URI}e is not available to the internal var lookup stuff in mod_headers, since it is not always set, and when it is, it is set too late in request processing.
Use the expr="%{REQUEST_URI}" flavor of expressions instead. It knows how to answer that w/o the dependency on the environment variable by the same name.

How can LocationMatch and ProxyPassMatch be Combined?

I am setting up an Apache 2.4.6 server on an internal machine for testing purposes. One of the things that Apache server is supposed to do is act as a reverse-proxy for another server found on localhost:3030.
The server on localhost:3030 expects one out of a few dataset names on its first path level (for now, the set comprises only of the dataset experimental, but some more will be added later on), so I am trying to pass that through from the requested path.
In my vhost, this works:
<Location /experimental/>
ProxyPass http://localhost:3030/experimental/
ProxyPassReverse /
</Location>
For additional datasets, I could copy that and replace experimental with the other dataset names. Obviously, that leads to a lot of code duplication/redundancy, which is both a source of errors and a maintenance horror.
Therefore, I would like to become somewhat more flexible and treat several datasets in a single such block. This should be possible with the LocationMatch directive.
As indicated by this comment and this page, I need to replace ProxyPass ProxyPassMatch when using that inside a LocationMatch block. Essentially, the docs state the same:
The same will occur inside a LocationMatch section, however ProxyPass does not interpret the regexp as such, so it is necessary to use ProxyPassMatch in this situation instead.
The LocationMatch docs explain:
From 2.4.8 onwards, named groups and backreferences are captured and written to the environment with the corresponding name prefixed with "MATCH_" and in upper case. This allows elements of URLs to be referenced from within expressions and modules like mod_rewrite. In order to prevent confusion, numbered (unnamed) backreferences are ignored. Use named groups instead.
That information is only valid as of Apache 2.4.8, which is presumeably why the following does not work on my 2.4.6 installation:
<LocationMatch /(?<dataset>experimental)/>
ProxyPassMatch http://localhost:3030/%{env:MATCH_DATASET}/
ProxyPassReverse /
</LocationMatch>
On the other hand, this page and that posting imply that the numerical group index ($1) can be used (as the help text is valid only as of httpd 2.4.8, my suspicion / hope is that the numerical reference works before 2.4.8 (?)
In any case, I have tried this:
<LocationMatch "/(experimental)/">
ProxyPassMatch http://localhost:3030/$1/
ProxyPassReverse /
</LocationMatch>
yet according to the logs, the internal call invokes http://localhost:3030/$1/ instead of http://localhost:3030/experimental/ when requesting the experimental path on the vhost URL.
The ProxyPassMatch docs only say:
When used inside a LocationMatch section, the first argument is omitted and the regexp is obtained from the LocationMatch.
However, the text does not bother to provide an example for how to combine LocationMatch and ProxyPassMatch. What am I doing wrong?
The doc also states When the URL parameter doesn't use any backreferences into the regular expression, the original URL will be appended to the URL parameter., which seems to be your case.
Further more, you are missing the host in your ProxyPassReverse directive.
This should work just fine:
<LocationMatch "^/experimental/.*$">
ProxyPassMatch http://localhost:3030
ProxyPassReverse http://localhost:3030
</LocationMatch>
Got this working on Apache 2.4.29:
<LocationMatch "/fruit/(?:apple|banana|pear)">
ProxyPass http://localhost:8080
ProxyPassReverse http://localhost:8080
</LocationMatch>
The URL called by Apache is for example
http://localhost:8080/fruit/apple
The (?: is crucial when you are using parentheses in this example.

Apache - Reverse Proxy and HTTP 302 status message

My team is trying to setup an Apache reverse proxy from a customer's site into one of our web applications.
http://www.example.com/app1/some-path maps to http://internal1.example.com/some-path
Inside our application we use struts and have redirect = true set on certain actions in order to provide certain functionality. The 302 status messages from these re-directs cause the user to break out of the proxy resulting in an error page for the end user.
HTTP/1.1 302 Found
Location: http://internal.example.com/some-path/redirect
Is there any way to setup the reverse proxy in apache so that the redirects work correctly?
http://www.example.com/app1/some-path/redirect
There is an article titled Running a Reverse Proxy in Apache that seems to address your problem. It even uses the same example.com and /app1 that you have in your example. Go to the "Configuring the Proxy" section for examples on how to use ProxyPassReverse.
The AskApache article is quite helpful, but in practice I found a combination of Rewrite rules and ProxyPassReverse to be more flexible. So in your case I'd do something like this:
<VirtualHost example>
ServerName www.example.com
ProxyPassReverse /app1/some-path/ http://internal1.example.com/some-path/
RewriteEngine On
RewriteRule /app1/(.*) http://internal1.example.com/some-path$1 [P]
...
</VirtualHost>
I like this better because it gives you finer-grained control over the paths you're proxying for the internal server. In our case we wanted to expose only part of third-party application. Note that this doesn't address hard-coded links in HTML, which the AskApache article covers.
Also, note that you can have multiple ProxyPassReverse lines:
ProxyPassReverse / http://internal1.example.com/some-path
ProxyPassReverse / http://internal2.example.com/some-path
I mention this only because another third-party app we were proxying was sending out redirects that didn't include their internal host name, just a different port.
As a final note, keep in mind that Firebug is extremely useful when debugging the redirects.
Basically, ProxyPassReverse should take care of rewriting the Location header for you, as Kevin Hakanson pointed out.
One pitfall I have encountered is missing the trailing slash in the url argument. Make sure to use:
ProxyPassReverse / http://internal1.example.com/some-path/
(note the trailing slash!)
Try using the AJP connector instead of reverse proxy. Certainly not a trivial change, but I've found that a lot of the URL nightmares go away when using AJP instead of reverse proxy.