I would like to serve some Unreal-generated HTML5 files using Google Cloud storage. Some of the files are gzip-encoded, and one of the javascript files checks that every file which ends with gz is returned with Content-Encoding: gzip. I don't want to change these files.
In GC UI, I set the Content-Encoding (and Content-type) fields in file metadata. To prevent decompressive transcoding I also set Cache-Control: no-transform. Despite that, the Content-Encoding header is still missing. (full header below). Is there anything else I can do to make Google Cloud respond with Content-Encoding?
HTTP/1.0 200 OK
X-GUploader-UploadID: some hash
Date: Sun, 24 Feb 2019 01:06:05 GMT
Cache-Control: no-transform
Expires: Mon, 24 Feb 2020 01:06:05 GMT
Last-Modified: Sun, 24 Feb 2019 00:52:29 GMT
x-goog-generation: 1550969549021835
x-goog-metageneration: 3
x-goog-stored-content-encoding: gzip
x-goog-stored-content-length: 99750206
Content-Type: application/octet-stream
x-goog-hash: crc32c=oobjmg==
x-goog-hash: md5=fN6srk43sVaYDVNgpXtdLQ==
x-goog-storage-class: REGIONAL
Accept-Ranges: none
Server: UploadServer
Vary: Accept-Encoding
Related
I set cache-control key on bucket with this value: public, max-age=86400. This shows up in the response header, but the images are not being cached. Images are coming back with 200. I'm using active storage, s3, and CloudFront.
/// response header below ///
Accept-Ranges: bytes
Cache-Control: public, max-age=86400
Content-Disposition: inline; filename="xhbtr_ba6c7e14-cab0-44c2-8701-e48aa75ab3f7_w1200.jpg"; filename*=UTF-8''xhbtr_ba6c7e14-cab0-44c2-8701-e48aa75ab3f7_w1200.jpg
Content-Length: 1444501
Content-Type: image/jpeg
Date: Mon, 14 Sep 2020 15:15:21 GMT
ETag: "18c45e75803b2a82c0a85dc4dde7bba4"
Last-Modified: Mon, 14 Sep 2020 14:00:44 GMT
Server: AmazonS3
x-amz-id-2: /6ynYC2FE4QwOaVJe1uJOBeCsJhfKFzNbMu+X7r0L5pRyax2JoxzJ5qoyO0Sb7dr09yFxZYj5/iE=
x-amz-request-id: 415DC4198F2ED5F1
Server: AmazonS3
You are querying S3. S3 has no cache. (The cache-control directive is meant for the client, not the server.)
What you probably want is to use Cloudfront as an edge cache. Cloudfront loosely adheres on the cache-control directive for edge caching (depending on how you configure it). Since you mention Cloudfront in your question, I assume you have already set it up. So instead of calling the S3 URL, use the URL you’ve set up in the Cloudfront configuration.
I have configured apache http with some cache settings. However, it does not seem to work properly and the browser seems to keep fetching the file as if it's not cached.
I tried to debug it by fetching the header.
$ curl -skI https://whatever-url-it-is/some-script.js
HTTP/1.1 200 OK
Date: Sun, 15 Sep 2019 05:35:08 GMT
Server: Apache/2.4.39 (Amazon) OpenSSL/1.0.2k-fips PHP/5.6.40
Last-Modified: Sun, 02 Jul 2017 10:52:25 GMT
ETag: "152ba-5535372162c40"
Accept-Ranges: bytes
Content-Length: 86714
Cache-Control: max-age=2592000, public
Expires: Tue, 17 Sep 2019 05:35:08 GMT
Vary: Accept-Encoding,User-Agent
Content-Type: text/javascript
The Cache-Control header seems to be correct. The Expires header looks a little odd but according to this page (https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Expires) the Expire will be ignored given that Cache-Control is there.
What else can I do to debug this issue? Any idea is very well welcome. Thanks
Never mind. I didn't realized that "Disable Cache" was turned on in Chrome. User error :-)
I've put this in my head section. It appears in the page source in the browser.
<meta http-equiv="Cache-Control" content="max-age=1209600">
However, when I look in the Chrome extension Live HTTP Headers, it says the following.
Cache-Control: max-age=0
Content-Encoding: gzip
Content-Length: 5849
Content-Type: text/html; charset=utf-8
Date: Sat, 05 Apr 2014 04:29:16 GMT
Expires: Sat, 05 Apr 2014 04:29:16 GMT
Last-Modified: Sat, 05 Apr 2014 03:33:19 GMT
The max-age isn't registering. I've emptied the browser cache but it makes no difference.
Any explanations? This is the site, incidentally.
UPDATES:
Firebug similarly records Cache-Control: max-age=0.
Google also makes clear here that max-age overrides the Expires header (which I don't set) and that you don't need both.
When you use tools like Live HTTP Headers, they show you the actual HTTP headers sent by the browser. What they do with meta tags used to simulate HTTP headers is a different issue. We can expect any conflict to be resolved in favor of the actual headers. (This has been normatively specified in HTML specs for Content-Type headers.)
To control cacheing, you should (at least primarily) use server configuration. See Caching Tutorial for Web Authors and Webmasters.
I noticed that chrome cached a video file. I replaced it with another one on the server and chrome kept serving the old one from cache (using JW flash player 5)
The headers of the request look like this:
joe#joe-desktop:~$ wget -O - -S --spider http://www.2xfun.de/files_geheimhihi14/20759.mp4
Spider mode enabled. Check if remote file exists.
--2011-05-15 22:40:56-- http://www.2xfun.de/files_geheimhihi14/20759.mp4
Resolving www.2xfun.de... 213.239.214.112
Connecting to www.2xfun.de|213.239.214.112|:80... connected.
HTTP request sent, awaiting response...
HTTP/1.1 200 OK
Date: Sun, 15 May 2011 20:40:56 GMT
Server: Apache
Last-Modified: Sun, 15 May 2011 20:37:59 GMT
ETag: "89b38-3bb227-4a35683b477c0"
Accept-Ranges: bytes
Content-Length: 3912231
Cache-Control: max-age=29030400, public, must-revalidate
Expires: Sun, 15 Apr 2012 20:40:56 GMT
Connection: close
Content-Type: video/mp4
Length: 3912231 (3.7M) [video/mp4]
Remote file exists.
I am using mod_headers and mod_expires in apache2 like this:
<FilesMatch "\.(flv|ico|pdf|avi|mov|ppt|doc|mp3|wmv|wav|mp4)$">
ExpiresDefault A29030400
Header append Cache-Control "public, must-revalidate"
</FilesMatch>
Did I spell revalidate wrong or something?
edit:
To make the use case clear: I want the files to be cached, because they are rather big and I want to save bandwidth. But on the other hand I want the files to be re-validated. So the client does a HEAD request and checks whether the content has changed (thats what the etag is for), and only re-fetches if necessary.
Your problem is that must-revalidate only kicks in once a cache entry is no longer fresh, but you've marked the response as cacheable for 29 million seconds. 'Cache-Control: max-age=0, must-revalidate' may be closer to what you want, if you want to allow caching but require revalidation on each use.
I'm convinced that certain images on my site are not caching properly. I have set the headers as best I can, but it still seems like they download again every time I hit the refresh button.
For example, a particular image always takes a bit over 1 second to download. This is even after it should be cached. Here are the response headers:
HTTP/1.1 200 OK
Date: Sun, 06 Mar 2011 12:51:52 GMT
Server: Apache/2.2.17 (Unix) mod_ssl/2.2.17 OpenSSL/0.9.7a mod_auth_passthrough/2.1 mod_bwlimited/1.4 FrontPage/5.0.2.2635 PHP/5.2.16
Last-Modified: Thu, 01 Jan 1970 00:00:00 GMT
Accept-Ranges: bytes
Content-Length: 19211
Cache-Control: max-age=630323456, public
Expires: Wed, 03 Mar 2021 12:51:52 GMT
Keep-Alive: timeout=5, max=98
Connection: Keep-Alive
Content-Type: image/png
Is there anything wrong with this? Thanks.
UPDATE
<FilesMatch "\.(htm|html|php)$">
Header set Expires "Thu, 01 Jan 1970 00:00:00 GMT"
</FilesMatch>
Your Last-Modified says 1970, and your max-age is 630323456 seconds (19 years). So the file has been 'expired' since 1989, and must be re-downloaded. The browser is doing what it should be doing.
Solution:
Change the Last-Modified to the real Last-Modified (probably some time in the past few years)
Change the max-age to
Remove the Expires header; it is overridden when you also have max-age. See RFC2616 section 14.9.3. Alternatively, delete the Cache-Control header and keep only the Expires header. Either one is fine, but only use one, not both of them.