For single page apps, some advanced rewrite rules need to be implemented in your server conf to proxy web crawlers and social media bots to cached pre-rendered versions of the JavaScript SPA content.
Using a service like http://prerender.io
You will notice the various server configuration rules templated here that demonstrate this proxy:
https://prerender.io/getting-started#install-it
Using the https://www.firebase.com/docs/hosting/guide/url-redirects-rewrites.html does Firebase support this level of sophistication?
For example - how would I implement this nginx config using Firebase rewrite-rules:
server {
listen 80;
server_name example.com;
root /path/to/your/root;
index index.html;
location / {
try_files $uri #prerender;
}
location #prerender {
#proxy_set_header X-Prerender-Token YOUR_TOKEN;
set $prerender 0;
if ($http_user_agent ~* "baiduspider|twitterbot|facebookexternalhit|rogerbot|linkedinbot|embedly|quora link preview|showyoubot|outbrain|pinterest") {
set $prerender 1;
}
if ($args ~ "_escaped_fragment_") {
set $prerender 1;
}
if ($http_user_agent ~ "Prerender") {
set $prerender 0;
}
if ($uri ~ "\.(js|css|xml|less|png|jpg|jpeg|gif|pdf|doc|txt|ico|rss|zip|mp3|rar|exe|wmv|doc|avi|ppt|mpg|mpeg|tif|wav|mov|psd|ai|xls|mp4|m4a|swf|dat|dmg|iso|flv|m4v|torrent)") {
set $prerender 0;
}
if ($prerender = 1) {
rewrite .* /$scheme://example.com$request_uri? break;
proxy_pass http://service.prerender.io;
}
if ($prerender = 0) {
rewrite .* /index.html break;
}
}
}
As a side note - I think it's great that you guys have support now for doing things like:
"rewrites": [ {
"source": "**",
"destination": "/index.html"
}]
But find this really only solving half the battle that SPA's face.
Firebase core developer here
Firebase announced basic SEO support which makes it work with Googlebot at ng-conf 2015 in March. See this presentation around the 16:30 mark for the announcement.
Firebase is still aiming to get working with pre-rendering tools like prerender.io and Brombone at some point as well, to allow even more sophisticated options for SEO. But this should "just work" if you upgrade to the latest version of the Firebase client (2.2.4 at the time of this post).
As of October 10, 2014, Firebase seems to officially say "no": https://github.com/firebase/firebase-tools/issues/33
An alternative is Divshot hosting. They offer a Prerender solution that's very easy to implement: http://docs.divshot.com/services/prerender
Related
I have an Express API that's being called from a VueJS app. The app is located at a domain similar to dev.example.com. Part of my API call checks for subdomains using the built in req.subdomains. In this case it should return an array containing "dev", however, it's returning an empty array. Both the VueJS app and Express API are running on an AWS EC2 instance.
EC2 Nginx File
server {
listen 80 default_server;
server_name _;
#Vue App & Frontend Files
location / {
root /var/www/frontend-app/dist;
try_files $uri /index.html;
}
#Node API Reverse Proxy
location /api/ {
proxy_pass http://localhost:3000/;
proxy_set_header Host $host; //Fix: Passes down the requesting host information
}
}
Frontend API Call
submit() {
axios.post('/api/users/login', {
email:'test#example.com',
password: 'password'
})
}
Express API Sample Route
router.post('/users/login', async(req, res) => {
*Login code omitted*
res.send(req.subdomains); //Returns [], expected result ['dev']
})
I haven't been able to find any solution and was hoping someone would have an idea. Thanks!
Please bear with me as I don't have much experience configuring Nginx.
I want to create micro-services as subfolders. eg: www.example.com/users
USERS is the micro-service so USERS will be the name of the subfolder as well.
The website can be hosted at /var/www/html/ or /var/www/html/example.com/public_html (Assuming this one for this question)
So inside the main folder, /var/www/html/example.com/public_html there are many folders, each representing a micro-service.
/var/www/html/example.com/public_html/users/
/var/www/html/example.com/public_html/students/
/var/www/html/example.com/public_html/teachers/
/var/www/html/example.com/public_html/parents/
Each micro-service will have a latest folder which holds the code to be executed.
/var/www/html/example.com/public_html/users/latest/
Each latest folder has 2 folders - APP which holds the SPA front end code and API which holds the API code
/var/www/html/example.com/public_html/users/latest/app/
/var/www/html/example.com/public_html/users/latest/api/
I would like to add and delete folders as required without making changes to the Nginx config file.
If the folder exists, i would like the relevant front end code to be displayed. Else 404
I am aware that I can add the folders as required in Nginx location but that is not what I want.
(DON'T WANT TO DO THE BELOW CODE)
location /users {
alias /var/www/html/example.com/public_html/users/latest/app
}
location /api/users {
alias /var/www/html/example.com/public_html/users/latest/api
}
I am trying to figure out if I can configure NGINX to point to the relevant folder based on the url.
This is what I have so far but it just shows as 404 for everything I am trying. I am working towards writing the API in Go-lang but since I have experience using PHP I am taking that route first before switching to Go.
server {
listen 80;
listen [::]:80;
server_name _;
root /var/www/html/example.com/public_html;
index index.html;
location ^/<service>/<additional-optional-parameters-can-go-here>$ {
root /var/www/html/example.com/public_html/<service>/latest/app;
try_files $uri $uri/ =404
}
location ^/api/<service>/<additional-optional-parameters-can-go-here>$ {
root /var/www/html/example.com/public_html/<service>/latest/api;
try_files $uri $uri/ =404
location ~ \.php$ {
include snippets/fastcgi-php.conf;
fastcgi_param SCRIPT_FILENAME $request_filename;
fastcgi_pass unix:/var/run/php/php7.2-fpm.sock;
}
}
location ~ /\.ht {
deny all;
}
}
I think you can use regex in the location and server lines.
Perhaps that will help
server_name ~^(?<subdomain>.+)\.domain\.tdl$ ~^(?<subdomain>.+)\.domain2\.tdl$ domain.tdl domain2.tdl;
location ~ ^/sitename/[0-9a-z]+/index.php$ {
fastcgi_pass phpcgi;
}
location ~ \.php$ {
return 404;
}
location ~ ^/a/b/(?<myvar>[a-zA-Z]+) {
# use variable $myvar here
if ($myvar = "sth") { ... }
}
# Example using the subdomain var created in last steps
sub_filter_once off;
sub_filter 'Site Text' '$subdomain';
sub_filter 'Site Text' '$myvar';
Personally I like to create different conf files for each service, then issue a "service nginx reload". Perhaps you could also automate the conf file generation.
Just incase anyone else is trying a similar approach.
I went on Freelancer.com and hired someone to help me.
location ~ ^/api/([a-z]+)/.*?$ {
alias $folder_path/$1/$api_path;
rewrite ^/api/([a-z]+)/.*?$ /$1/$api_path last;
}
location ~ ^/([a-z]+)/(?:(?!\blatest\b).)*$ {
alias $folder_path/$1/$app_path;
rewrite ^/([a-z]+)/(?:(?!\blatest\b).)*$ /$1/$app_path last;
}
I have two web applications (node.js express apps), web1 and web2. These web apps expect to be hosted on sites that are typically something like http://www.web1.com and http://www.web2.com. I'd like to host them behind an nginx reverse proxy as https://www.example.com/web1 and https://www.example.com/web2. I do not want to expose the two web apps as two subdomains on example.com.
Here is a snippet of my nginx configuration (without SSL termination details) that I had hoped would accomplish this:
server {
listen 443;
server_name .example.com;
location /web1 {
proxy_pass http://www.web1.com:80;
}
location /web2 {
proxy_pass http://www.web2.com:80;
}
}
This works, except for the relative links that the web apps use. So web app web1 will have a relative link like /js/script.js which won't be handled correctly.
What is the best/standard way to accomplish this?
You should be able to do this by checking the $http_referer, something like:
location / {
if ($http_referer ~ ^http://(www.)?example.com/web1) {
proxy_pass http://www.web1.com:80;
}
if ($http_referer ~ ^http://(www.)?example.com/web2) {
proxy_pass http://www.web2.com:80;
}
}
The browser would be setting the referer to http://example.com/web1/some/page when it requests /js/script.js so the apps shouldn't need to change, unless they need to process or care about the referer internally.
The $http_referer does not seem to be easy to find in nginx docs, but is mentioned in a few sites:
http://nginx.2469901.n2.nabble.com/HTTP-Referer-Module-td3356604.html
https://stackoverflow.com/a/18917016/1422492
I think something like this:
server {
listen 443;
server_name .example.com;
location /web1 {
proxy_pass http://www.web1.com:80;
}
location /web2 {
proxy_pass http://www.web2.com:80;
}
location / {
if ($http_referer ~* (/web1) ) {
proxy_pass http://www.web1.com:80;
}
if ($http_referer ~* (/web2) ) {
proxy_pass http://www.web2.com:80;
}
}
How about using cookie and ngx_http_map_module?
Add add_header Set-Cookie "site=web1;Path=/;Domain=.example.com;"; to location /web1 {...} (web2 too).
Add map to under http
map $cookie_site $site {
default http://www.web1.com:80;
"web2" http://www.web2.com:80;
}
Default location is this
location / {
proxy_pass $site;
}
You can pass the value of cookie to proxy_pass directly. But, using map is more secure way.
My website have three different pages:
www.abc.com/
www.abc.com/node
www.abc.com/memeber
If I type www.abc.com in the browser,it goes to www.abc.com/, how can I change it to www.abc.com/node when I type www.abc.com and keep the www.abc.com/memeber normal as usual?
Just adding a small note, use return instead of rewrite for redirects, check nginx pitfalls
location = / {
return 301 $scheme://$server_name/node
}
location / {
# normal location handling, using try_files for example
}
Following worked for me provided I was only serving static files.
server {
listen 80;
server_name www.abc.com;
root /var/www/abc;
if ($request_uri = '/') {
rewrite ^ /node break;
}
}
location = / {
rewrite ^ /node;
}
I've got Varnish (3.0.3) sitting as a load-balancer/static cache in front of two web servers. I've got a CDN set up using Original Pull method. If I grab a URL from an image on my site manually, drop in the CDN address, I can verify that original pull is working and the image is pulled to the CDN and served.
My application is fairly complex and I'm testing this CDN to see if it significantly speeds up the web app, so I don't want to rewrite any of my php code to use the CDN images just yet.
What I'd like to do is set Varnish up to rewrite requests received for image files and pull them through the CDN instead of from the two Apache servers directly in my cluster.
I've read through the Varnish documentation and a couple howto's online about doing something similar, but I just can't get it to work properly and need a little help here.
Here are a couple different ways I tried doing this (edited for brevity):
sub vcl_recv {
#if request is image, redirect to CDN
if (req.url ~ "\.(gif|ico|jpg|jpeg|png)$") {
set req.http.host = "cdn.domain.com/";
error 750 req.http.host + req.url;
}
}
sub vcl_error {
if (obj.status == 750) {
set obj.status = 302;
set obj.http.Location = obj.response;
return(deliver);
}
}
That didn't work. It resulted in broken images everywhere, and anything that did show up was using the .webp extension, so it wasn't being processed by the condition above.
So I tried this:
backend cdn {
.host = "cdn.domain.com";
.port = "80";
}
sub vcl_recv {
#if request is image, redirect to CDN
if (req.url ~ "\.(gif|ico|jpg|jpeg|png)$") {
set req.backend = cdn;
return(lookup);
}
}
This showed some images on the page, but when viewing their source, they looked to be coming from the Apache servers (the domain name wasn't that of the CDN) and only about half the images were displaying...probably browser cache.
I'd love some input here, thanks guys.
Is there no way to use Varnish for this kind of redirect? Would I be better off setting nginx up in front of Varnish to rewrite requests to the cdn?
UPDATE:
Using both answers given below, I have the redirect working and an ACL in place to allow the CDN to pull images directly vs redirecting to itself. However, though I verified the ACL is allowing connection through by using my own external IP, the CDN isn't pulling new images from the server. It gives a 502 error (odd<) instead of pulling the image from the local server to the CDN and serving it. This is what the block of my vcl_recv looks like now:
acl cdn {
"ip.of.CDN";
}
sub vcl_recv {
#if request is image, redirect to CDN
if (req.url ~ "\.(gif|ico|jpg|jpeg|png)$") {
if(!client.ip ~ cdn){
error 750 "http://cdn.domain.com" + req.url;
}
}
}
sub vcl_error {
if (obj.status == 750) {
set obj.status = 302;
set obj.http.Location = obj.response;
return(deliver);
}
}
You can definitely do this with Varnish quite easily - no need to setup nginx or anything. Actually your first solution is very close to doing the trick. It just needs a few modifications.
sub vcl_recv {
#if request is image, redirect to CDN
if (req.url ~ "\.(gif|ico|jpg|jpeg|png)$") {
error 750 "http://cdn.domain.com" + req.url;
}
}
sub vcl_error {
if (obj.status == 750) {
set obj.status = 302;
set obj.http.Location = obj.response;
return(deliver);
}
}
You forgot "http://" from your CDN URL, and you can omit the last slash from the host as all req.urls begin with /.
You also need to make sure that the vcl_error code is the first one that is run in vcl_error(). I.e. if you have multiple definitions of vcl_error, make sure that none of them get to deliver any output before the if (obj.status == 750) check is reached.
Bear in mind that this solution causes all client browsers to query your server first and then make another request to the CDN after the 302 redirect. This adds a significant delay to each image load, and is probably not the best way of determining if CDN improves your app performance.
Update: Regarding your problems with CDN showing 502 errors when trying to pull content from your origin. Relying on the remote IP address for determining the redirection is quite risky, as the CDN could very well use a number of servers to do the pull, and the addresses could change over time. That would make the VCL very laborious and error-prone to maintain.
Would it be possible setting up a unique virtual host for the CDN to use? For instance originpull.domain.com and setup the CDN so that it pulls content from that address instead of your primary www.domain.com address?
You could then modify the vcl_recv() as follows:
sub vcl_recv {
#if request is image and request is not made from CDN, redirect to CDN
if (req.http.host != "originpull.domain.com" &&
req.url ~ "\.(gif|ico|jpg|jpeg|png)$") {
error 750 "http://cdn.domain.com" + req.url;
}
}
That would ensure that the requests from CDN will never be redirected.
Assuming you have the CDN pulling it's copy of the images from the site, and your not manually pushing images to the CDN. Aren't you missing a simple exclusion of the CDN network, from either your rewrite, or backend proxy? As the CDN needs to be able to directly pull a copy of the images, from your site to populate it's caches.
Been a while since I played with Varnish, and never an expert, but something along the following lines may work:
# Defnine the IP ranges of the CDN server.
acl cdn {
"localhost";
"11.22.33.0"/24;
}
...
#if request is image, redirect to CDN, unless from the CDN
if (req.url ~ "\.(gif|ico|jpg|jpeg|png)$") {
if (!client.ip ~ cdn) {
error 750 "http://cdn.domain.com" + req.url;
}
}
...