I have a webpage hosted on Amazon S3 but I don't want the http response code to be 200. The page is a maintenance page that I'll redirect traffic to when I take our main website down for maintenance.
I want the Amazon S3 page to include a response header of:
HTTP/1.1 503 Service unavailable
Amazon give the ability to add some metadata to the S3 Object but there is nothing for the http status code.
Is it possible?
Not sure which browsers, or crawlers, that supports this. But you could potentially use the meta http-equiv status meta tag to accomplish this.
<meta http-equiv="status" content="503 Service Unavailable" />
The specification says to treat it in the same way as if 503 had been sent as the status code.
I believe you can get Cloudfront to do this. I haven't tested it yet, but try this:
http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/custom-error-pages.html
You cannot customize the status code for S3 responses.
You can use API Gateway as a proxy to your S3 website error page where you can customize status codes returned.
Until Amazon allow a custom status code from S3, here is a workaround using nginx.
We watch for the existence of a specific file, that acts as a "ON switch" for maintenance mode. If found, we proxy_pass requests to S3 - The trick is to return 503 but redirect processing of 503 status codes to a nginx "named location".
Example nginx conf file (just the relevant bits shown):
server {
...
# Redirect processing of 503 status codes to a nginx "named location".
error_page 503 #maintenance;
# "Maintenance Mode" is off by default - Use a nginx variable to track state.
set $maintenance off;
# Switch on "Maintenance Mode" if a certain file exists.
if (-f /var/www/app/maintenanceON) {
set $maintenance on;
}
if ($maintenance = on) {
# For Maintenance mode Google recommend using status code: "503 Service unavailable".
return 503;
}
...
location #maintenance {
# Redirect the request to a static maintenance page hosted in Amazon S3.
# Note: Use proxy_pass instead of rewrite so we keep the 503 code (otherwise nginx serves a 302 code)
rewrite ^(.*)$ /index.html break;
proxy_pass http://bucketname.s3-website-us-east-1.amazonaws.com;
}
}
Related
I'm working on an nginx s3 reverse proxy container image to proxy frontend files (Angular apps) from s3 behind an Application Load Balancer. The frontend files are located in the specific folder of the given app name in the s3 bucket. These are angular apps which are built using standard angular commands. The dist contents are uploaded to s3 and then the ALB route paths, along with the nginx locations map to those app folders in s3. For example, here is my nginx conf file:
server {
listen 80;
listen 443 ssl;
ssl_certificate /etc/ssl/nginx-server.crt;
ssl_certificate_key /etc/ssl/nginx-server.key;
server_name timemachine.com;
sendfile on;
default_type application/octet-stream;
resolver 8.8.8.8;
server_tokens off;
location ~ ^/app1/(.*) {
proxy_http_version 1.1;
proxy_buffering off;
proxy_ignore_headers "Set-Cookie";
proxy_hide_header x-amz-id-2;
proxy_hide_header x-amz-request-id;
proxy_hide_header x-amz-meta-s3cmd-attrs;
proxy_hide_header Set-Cookie;
proxy_set_header Authorization "";
proxy_intercept_errors on;
rewrite ^/app1/?$ /app1/index.html;
proxy_pass https://<s3 bucket name here>;
break;
}
}
So there is a corresponding bucket folder /app1 in s3 which has the dist contents and is serving up the index.html. And on the ALB, there are two route paths. The first is /app1 which redirects to https:{port}//app1/ and then the second route path /app1/* which just forwards to the nginx reverse proxy container deployed via ECS Fargate.
This is not using cloudfront. The bucket is proxied internally on https and specific permissions are set on the bucket to be accessible w/in the given VPC.
The angular apps have specific modules, but the issue is since Im not saving any of this content in the container, I can't just do a try_files, or set an index to make this work, since all of this is proxied from s3 and the content is accessed differently.
I can access the app at with the given proxy configuration above, but for other paths, say when I navigate to the part of the apps where its /app1/account and then do a refresh, the page throws an access denied on the bucket and I just get the standard xml page in the browser.
How do I get this to work with all of those other relative paths without having to add each of those paths to nginx or the ALB routes? In other words, I dont want to have to add
location /app1/account {
}
and so on, or something like that. Yes, Im sort of new to nginx, so im still figuring things out.
I was expecting the above proxy to work with all paths on /app1 but im unsure what other route paths need to be added to the ALB or if the regex is off, or what else needs to be added to the nginx conf file.
All that to say, when I enter this
https://timemachine.com/app1
or this,
https://timemachine.com/app1/
both work and just rewrite to the index.html which is good.
After this, when I click on another icon in the UI that directs to another path on /app1/, I get directed to the page correctly at...
https://timemachine.com/app1/news
but then on a refresh on this path, instead of hitting url https://timemachine.com/app1/news, with all the data shown when I accessed this through UI, the url stays at https://timemachine.com/app1/news but the page defaults to s3 bucket access denied on that route(.xml).
The goal is just to be able to reload on the pages I can already access without the UI blowing up and defaulting to the access denied message. So I would like to be able to just enter https://timemachine.com/app1/news, which will display the content, then do a refresh and see the content again.
The are various modules within the angular apps and so these are relative paths, which may be part of the problem.
NOTE: All files, aside from assets folder, are in the base app1 bucket folder. So https://<s3_bucket_name>/app1 (with app1 being the folder).
Angular's docs indicate to use the Frontend Controller pattern for static files like so:
Use try_files, as described in Front Controller Pattern Web Apps,
modified to serve index.html:
try_files $uri $uri/ /index.html;
Obviously, that won't work here (since the files aren't local to nginx) so my understanding is we're looking for equivalent logic to that for when the files are hosted elsewhere.
Route not-assets to index.html
All assets are in the /assets/ folder - so the simplest solution is to look for anything starting with not-that and proxy those requests to the html file for the response:
server {
location ~ /app1/ {
rewrite ^/app1/(?!assets/) /index.html;
proxy_pass https://domain/bucket/app1/;
}
}
That regex means that:
/app1/assets/some.css gets proxied to https://domain/bucket/app1/assets/some.css
/app1/ gets proxied to https://domain/bucket/app1/index.html
/app1/something/else gets proxied to https://domain/bucket/app1/index.html
etc.
Do note that this is going to make your app respond HTTP 200 OK with html to almost any url - which may be confusing.
If there are any problems setting this up, enable the nginx debug log to see to what url requests are being proxied, and determine the difference from what's desired.
I want to display pdf from s3 in the browser by using pdfjs - https://mozilla.github.io/pdf.js/
In place using of s3 URL, I have reverse proxy it like this
URL www.my-site-url.com/public/s3-presinged-url-bucket-part-with-sign-info
NGINX Block
location /public {
proxy_pass https://XXXX.s3.us-west-2.amazonaws.com/;
}
But S3 throws error
<Code>SignatureDoesNotMatch</Code>
<Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message>
How to reverse proxy it correctly?
I ran into a similar situation when proxying to presigned S3 urls. Everything worked on development machines, but failed in production because CloudFront added additional headers which changed the signature. In my case because I already had valid presigned URLs provided headers were unmolested, I added proxy_pass_request_headers off; to make the proxy request roughly equivalent to a direct GET request.
I have an nginx configuration which listens to any subdomain *.mydomain.com,
and I want to use subdomain as variable, to proxy request to other site.
Here is my nginx configuration
server {
listen 80;
server_name "~^(?<subdomain>.*).mydomain.com";
location / {
resolver 1.1.1.1 1.0.0.1 ipv6=off;
proxy_pass http://hosting.mydomain.com/$subdomain/;
proxy_redirect off;
access_log /var/log/nginx/proxy.log;
}
}
As I request the site directly and it loads perfectly
Site placed on AWS S3, and bucket static website address cnamed to mydomain
However, when I try to access via user1.mydomain.com, the page didn't load images, and css
This is the same site
And in browser network panel shows
Difference between direct and proxy access
This issue is made, because I have many sites stored in S3 bucket and located in different folders (the folder name is used as subdomain).
And I want to use a single domain to access all of them via subdomains.
Thanks in advance
You forgot to proxy pass the URI, you're serving user1/index.html for every request, including for JS and CSS requests, it's why all of responses are the same size (2kb, the size of user1/index.html), and it's also why you're getting Uncaught SyntaxError: Unexpected token < in the first line of Enterprise_skeleton.bundle.js because it's returning an HTML document that starts with <!doctype html> instead of the actual JS bundle.
Change
location / {
proxy_pass http://hosting.mydomain.com/$subdomain/;
}
to
location / {
proxy_pass http://hosting.mydomain.com/$subdomain$uri;
}
In a setting of apache + mod_wsgi and nginx + uwsgi, what could be the way to setup web-server to proxy big "intranet" files requests?
What I am thinking about is a way a la x-sendfile, but where the wsgi application points to a file URL "intranet" location in its response, web-server downloads and uploads the file to the original requester without revealing it's "intranet" location. Of course, nothing happens if there is no authentication and access rights check on wsgi application side.
It's very hard to find this kind of setup by googling, not even sure what term to use.
By "intranet" I mean files, accessible via HTTPS requests from the proxy server, which may have its own credentials to them, but not from public internet or by local filesystem (like is the use case with x-sendfile)
If using mod_wsgi in daemon mode, you can return an empty HTTP 200 response with Location response header and when that is seen by the Apache process proxying to the mod_wsgi daemon process, it will evaluate that as a sub request. The path in that could be mapped to a new URL handler in Apache configuration which is actually a proxy setup which sends the request to another downstream backend server. The response from that will then be proxied back to the client. If you don't want that secondary URL handler to be visible outside, ie., someone can't request it direct if they work out the URL path, you need to use a mod_rewrite rule to reject any request if it isn't a sub request.
So you might have something like:
RewriteCond %{IS_SUBREQ} false
RewriteRule ^/hidden/stuff/ - [F]
ProxyPass /hidden/stuff/ http://backend.example.com/
The WGSI response would then be empty HTTP 200 response with Location header of:
Location: /hidden/stuff/some-file-name
The sub request request would end up being:
http://backend.example.com/some-file-name
against backend server with response proxied back to client.
I have site using cloudflare.com's free plan as reverse proxy.
I had one script which took more than 60 seconds to execute , and server threw 504 Gateway time-out , but i didnt get my web servers page, instead i got the custom page setup by cloudflare,
I have double checked and my Smart-errors application in cloudflare is turned OFF , so why cloudflare is still giving their custom made page ?
How can i switch off cloudflare's all custom pages and just use as reverse proxy.?
So my question is how can i turn off all the custom error pages from cloudflare ?
I am using nginx latest server on cent os 7. 64 bit.
with php-fpm latest.
update :
This is what i got , when i go the 504 error.
Error 504 Ray ID: 1fb6a3feef7c17c8 • 2015-06-24 07:16:17 UTC
Gateway time-out
You
Browser
Working
Singapore
CloudFlare
Working
mydomain.com
Host
Error
What happened?
The web server reported a gateway time-out error.
What can I do?
Please try again in a few minutes.
CloudFlare Ray ID: 1fb6a3feef7c17c8 • Your IP: myip • Performance & security by CloudFlare
Update : as suggested by damoncloudflare, i would like to add more details.,
its a dedicated server, with linux cent os 7, 64 bit , latest nginx , php-fpm,
in nginx conf i have already specified how to handle 504 errors by
error_page 500 502 503 504 /50x.html;
How to replicate it .
code
<?php
http_response_code(504);
?>
or
<?php
sleep(61);
echo 'i am done sleeping';
?>
where 61 is more than max_execution_time which is 60
More details , as why originally this happened.
i was downloading huge file and the download took more than 60 seconds which is max_execution_time set in php.ini file.
So i received the cloudflare error.,
i would like to add there is nothing wrong with my server.
i have one more query , will i be able to disable cloudlfare 504 error and show my web servers custom page, if i upgrade from free to pro ?
update 2
i guess , now i understand the issue .,
Error 504
Gateway time-out
is triggered when server is not reachable and thats where cloudflare shows their page,
obviously if server not reachable then , its not possible for server to show 504 page.
thanks for your reply.
You need a premium Cloudflare plan to customize error pages for certain response codes. From the Cloudflare dashboard (domain name > Custom Pages tab):
500 Class Errors [...] Upgrade to Pro [...] Once you publish your custom page, Cloudflare will use your customized page instead of serving our standard 500 Class Errors page to your visitors on 502, 504, and 52x errors.
Even if NGINX sends your error page to Cloudflare, Cloudflare will see the 5xx response code being sent back along with your error page and will respond with their error page instead of yours.
There exists a workaround. From the NGINX documentation on the error_page directive:
Furthermore, it is possible to change the response code to another using the “=response” syntax, for example:
error_page 404 =200 /empty.gif;
In NGINX, before responding to Cloudflare we map the 'offending' 5xx error to a different response code. That is, instead of sending the offending 5xx response code along with our error page we change the response code to one for which Cloudflare will not override our pages. It is a good idea to map the offending 5xx error to another 5xx class error (500). Although less information is conveyed, this preserves the general meaning of the 5xx class error.
error_page 500 503 /50x.html;
error_page 502 504 =500 /50x.html;
location = /50x.html {
root /usr/share/nginx/html;
internal;
}
I should add - if you are getting your error page for the 504 error, then NGINX is being reached. This error would then mean, in the reverse proxy context, that NGINX passed the request on successfully but the upstream service (web server) took too long to respond.
A 504 or 502 Gateway error is actually indicative of an issue with your back end server & I would recommend contacting your hosting provider/checking your server logs.
"So my question is how can i turn off all the custom error pages from cloudflare ?"
Customers on a paid plan (Pro or above) can customize error pages.
"I have double checked and my Smart-errors application in cloudflare is turned OFF , so why cloudflare is still giving their custom made page ?"
SmartErrors is an entirely different feature. It is designed to only trigger when we get a 404 on a URL on your site.