Nginx proxy_pass having problems with querystring for an API - api

I'm trying to proxy a certain rest endpoint on my linux api box to my windows box. Here's what I have right now.
My linux api box
...
location ~ ^/api/v0/roslyn(.*)$ {
resolver 8.8.8.8;
proxy_pass $scheme://my-windows-box.com/roslyn$1;
}
For example, I'd like to proxy the following url
http://my-linux-box.com/api/v0/roslyn?q=5
to
http://my-windows-box.com/roslyn?q=5
However, it seems to be missing the querystring, so the regex is failing?

I don't think you can match the args by regex, try this instead
location /api/v0/roslyn {
resolver 8.8.8.8;
proxy_pass $scheme://my-windows-box.com/roslyn$is_args$query_string;
}

Related

Nginx - proxy_pass to google storage bucket static page VueJS sub paths cause 404 error, VueJS router not kicking in

I'm hosting VueJS at google cloud storage bucket, app works only when using domain name without subpath: www.domain.com when using URL like: www.domain.com/sub/path I'm getting 404 error as it seem that NGINX is looking for this path in the bucket instead of let VueJS router take over.
I tried to follow older thread but in my case would not help.
Any ideas how to fix this?
location = / {
proxy_pass https://gcs/mygoogle-cloud-bucket/main.html;
proxy_set_header Host storage.googleapis.com;
}
location / {
rewrite /(.*) /$1 break;
proxy_pass https://gcs/mygoogle-cloud-bucket/$1$is_args$args;
proxy_redirect off;
index main.html;
proxy_set_header Host storage.googleapis.com;
}
It seems like what you need to do is to create a Static Website using Cloud Storage and VueJS.
With this bieng the case, there are a few things that needs to be clarified:
Cloud Storage doesn't support HTTPs, so yo uneed to use a Load Balancer.
Make sure the objects in your bucket are public.
Build the Vue project With Relative Path.
It is also recomended to set the special pages, but this is not necessary.
Set up your load balancer and the SSL certificate as it is mentioned here.
Configure routing rules.
Make sure you have connected your custom domain to your load balancer
This should get you going with your site. If you would like to check a worknig example, you can take a look at this one.
Your code should look something like:
location / {
rewrite /$ $uri$index_name;
proxy_set_header Host storage.googleapis.com;
proxy_pass https://gs/$bucket_name$uri;
proxy_http_version 1.1;
proxy_set_header Connection "";
}

nginx - proxy facebook/bots to a different server without changing canonical URL

TLDR;
How can I make it so that all scraper/bot requests reaching my frontend https://frontend.example.test/any/path/here are fed the data from https://backend.example.test/prerender/any/path/here without changing the canonical URL?
I have a complex situation where I have a Vue app that pulls data from a php API to render data. These are hosted in China so niceties like netlify prerender and prerender.io are not an option.
Initially I tried:
if ($http_user_agent ~* "googlebot|bingbot|yandex|baiduspider|twitterbot|facebookexternalhit|rogerbot|linkedinbot|embedly|quora link preview|showyoubot|outbrain|pinterest\/0\.|pinterestbot|slackbot|vkShare|W3C_Validator|whatsapp") {
rewrite ^/(.*)$ https://backend.example.test/prerender/$1 redirect;
}
which workd but Facebook used backend.example.text the canonical URL frontend.example.test.
Setting the og:url to the frontend app caused problems due to a redirect loop. I tried then setting the og:url to the frontend with a query param that skipped the nginx forward, but for some reason this wasn't working properly on the live server and I imagine facebook would still end up pulling the data from the final url anyhow.
Thus I imagine the only solution is to use proxy_pass but it is not permitted with a URI inside an if statement (and I have read the if is evil article).
I feel like all I need is something like a functioning version of:
location / {
if ($http_user_agent ~* "googlebot|bingbot|yandex|baiduspider|twitterbot|facebookexternalhit|rogerbot|linkedinbot|embedly|quora link preview|showyoubot|outbrain|pinterest\/0\.|pinterestbot|slackbot|vkShare|W3C_Validator|whatsapp") {
proxy_pass https://backend.example.test/prerender;
}
...
}
(I am of course aware of the contradiction of having to have Facebook sharing work in China, but the client is requesting this for their international users as well).
Here is the solution for your problem:
https://www.claudiokuenzler.com/blog/934/nginx-proper-way-different-upstream-user-agent-based-reverse-proxying-rule-without-if
I'm copying here the main parts in case the link breaks:
Create a dynamic target upstream with the map directive:
map "$http_user_agent" $targetupstream {
default http://127.0.0.1:8080;
"~^mybot" http://127.0.0.1:8090;
}
Here "~^mybot" is a regular expression, if the user agent matches that expression it will use that upstream server.
If the user-agent does not match any entries, Nginx will use the "default" entry (saving http://127.0.0.1:8080 as $targetupstream variable).
Then you just have to use a that upstream in a proxy pass setting:
location / {
include /etc/nginx/proxy.conf;
proxy_set_header X-Forwarded-Proto https;
proxy_pass $targetupstream;
}
Now, you could use one upstream pointing to locahost at a port that is being used by nginx to serve static files (for client only) and another port for the server renderer.

Nginx URL Encoding Issue

I'm using Nginx in front of Apache and have an issue with URL encoding.
Requests for :
GET /forum/index.php%3Ftopic%3D14454.3775%3Bwap2
Produces this error in Apache (when proxy_passed from Nginx) :
/forum/index.php?topic=14454.3775;wap2' not found or unable to stat
When I request the url above (not encoded) it gets processed just fine.
The solution I found is I need to change:
proxy_pass http://127.0.0.1:80;
to
proxy_pass http://127.0.0.1:80/;
When the forward slash is excluded, the original request will not be decoded. When I add the forward slash, this is the error message I receive :
"proxy_pass" cannot have URI part in location given by regular expression, or inside named location, or inside "if" statement, or inside "limit_except" block
My nginx config block is :
location ~* ^/forum/index.php
{
charset ISO-8859-1;
include /etc/nginx/proxypass.conf;
proxy_pass http://127.0.0.1:80;
}
How can I fix the proxy_pass directive and include the forward slash so the request url is decoded before it's passed over to Apache?
EDIT:
Request:
GET /index.php HTTP/1.1" 200 16481 "http://www.example.com/forum/index.php%3Ftopic%3D14454.3775%3Bwap2
Error:
script '/www/example.com/httpdocs/forum/index.php?topic=14454.3775;wap2' not found or unable to stat
I managed to solve the issue by doing this.
If there is a better way to handle this, please let me know.
location ~* /forum/index.php(.*)
{
include /nginx/proxypass.conf;
if ($args) {
proxy_pass http://127.0.0.1:80/forum/index.php?$args;
}
proxy_pass http://127.0.0.1:80/forum/index.php$1;
}

Nginx use placeholder in location (nginx.conf)

I want to use nginx as a revers ssl proxy.
I have a reset service with multiple apis and I want to have a place holder in the location configuration and use it in the proxy_pass
location /myapp/(\s+)/(\s+) {
proxy_pass http://localhost:9000/$1/$2;
}
Any idea how do I do it ?

Laravel routes behind reverse proxy

Ok, so for development purposes, we have a dedicated web server. It's not currently connected directly to the internet, so I've setup an apache reverse proxy on another server, which forwards to the development server.
This way, I can get web access to the server.
The problem is, the routes in Laravel are now being prefixed with the internal server IP address, or the servers computer name.
For example, I go to http://subdomain.test.com but all the routes, generated using the route() helper, are displaying the following url: http://10.47.32.22 and not http://subdomain.test.com.
The reverse proxy is setup as such:
<VirtualHost *:80>
ServerName igateway.somedomain.com
ProxyRequests Off
<Proxy *>
Order deny,allow
Allow from all
</Proxy>
ProxyPass / http://10.47.32.22:80/
ProxyPassReverse / http://10.47.32.22:80/
<Location />
Order allow,deny
Allow from all
</Location>
</VirtualHost>
I have set the actual domain name in config\app.php.
Question
How can I set the default URL to use in routing? I don't want it using the internal addresses, because that defeats the point of the reverse proxy.
I've tried enclosing all my routes in a Route::group(['domain' ... group, which doesn't work either.
I ran into the same (or similar problem), when a Laravel 5 application was not aware of being behind an SSL load-balancer.
I have the following design:
client talks to an SSL load balancer over HTTPS
SSL load balancer talks to a back-end server over HTTP
That, however, causes all the URLs in the HTML code to be generated with http:// schema.
The following is a quick'n'dirty workaround to make this work, including the schema (http vs. https):
Place the following code on top of app/Http/routes.php
In latest version of laravel, use web/routes.php
$proxy_url = getenv('PROXY_URL');
$proxy_schema = getenv('PROXY_SCHEMA');
if (!empty($proxy_url)) {
URL::forceRootUrl($proxy_url);
}
if (!empty($proxy_schema)) {
URL::forceSchema($proxy_schema);
}
then add the following line into .env file:
PROXY_URL = http://igateway.somedomain.com
If you also need to change schema in the generated HTML code from http:// to https://, just add the following line as well:
PROXY_SCHEMA = https
In latest version of laravel forceSchema method name has changed to forceScheme and the code above should look like this:
if (!empty($proxy_schema)) {
URL::forceScheme($proxy_schema);
}
Ok, so I got it. Hopefully this will help someone in the future.
It seems like Laravel ignores the url property in the config\app.php file for http requests (it does state it's only for artisan), and it instead uses either HTTP_HOST or SERVER_NAME provided by apache to generate the domain for URLs.
To override this default url, go to your routes.php file and use the following method:
URL::forceRootUrl('http://subdomain.newurl.com');
This will then force the URL generator to use the new url instead of the HTTP_HOST or SERVER_NAME value.
Go to app/Http/Middleware/TrustProxies.php and change the protected variable $proxies like this:
protected $proxies = ['127.0.0.1'];
Just this! Be happy!
Because laravel route is created not from the config/app itself rather than from the server. My solution is adding the proxy_set_header Host to the nginx's config.
server {
listen 80;
server_name my.domain.com;
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host my.domain.com;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://127.0.0.1:8000;
}
}
i'm using laravel 8 with nginx docker inside a running nginx on the machine, so yeah it's double nginx
add this code in App\Providers\AppServiceProvider class
if (Str::contains(Config::get('app.url'), 'https://')) {
URL::forceScheme('https');
}
Seems the Laravel have more convinient solution.. Check answer there: How do I configure SSL with Laravel 5 behind a load balancer (ssl_termination)?
Following up #TimeLord's solution:
In latest version of laravel the name for forced schema has changed and now it is:
URL::forceScheme()
I know this topic is old a.f but I've been solving this issue by replacing the following line in my DatabaseSessionHandler.pdf [#Illuminate/Session]:
protected function ipAddress()
{
return $_SERVER['HTTP_X_FORWARDED_FOR'];
// return $this->container->make('request')->ip();
}
Of course you need to migrate the sesssion table first and set up the config
(.env Variable SESSION_DRIVER=database)
For nginx, you don't need to do anything extra in Laravel. The fix can be done at from nginx config;
server {
listen 80;
listen [::]:80 ipv6only=on;
server_name sub.domain.dev;
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host sub.domain.dev;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://127.0.0.1:8000;
}
}
Figured out a way that much cleaner and only does exactly what the loadbalancer tells it to, add this function to your RouteServiceProvider
protected function enforceProtocol()
{
if(request()->server->has('HTTP_X_FORWARDED_PROTO')){
URL::forceScheme(request()->server()['HTTP_X_FORWARDED_PROTO']);
}
}
and in the boot section, simple call it like so
public function boot()
{
$this->enforceProtocol();
//other stuff
}