Cloudflare does not cache static assets with proper headers - cloudflare

I've read in the docs that cloudflare caches js and css files. (https://support.cloudflare.com/hc/en-us/articles/200172516-Which-file-extensions-does-Cloudflare-cache-for-static-content-)
But in my case the header cf-cache-status has the value MISS.
One example would be a css reasources like this:
https://example.com/style.a574b.css
with the following response headers:
cache-control: public, max-age=31536000
cf-cache-status: MISS
cf-ray: 405badb4ec2297bc-FRA
content-encoding: br
content-type: text/css
date: Tue, 03 Apr 2018 12:53:31 GMT
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
expires: Wed, 03 Apr 2019 12:53:31 GMT
last-modified: Tue, 03 Apr 2018 01:14:20 GMT
pragma:
server: cloudflare
status: 200
vary: Accept-Encoding
via: 1.1 vegur
x-application-context: ourder:prod,heroku:3829
x-content-type-options: nosniff
x-xss-protection: 1; mode=block
The original response (without cloudflare between is:)
Accept-Ranges: bytes
Cache-Control: max-age=31536000, public
Content-Encoding: gzip
Content-Length: 948
Content-Type: text/css
Date: Tue, 03 Apr 2018 19:17:46 GMT
Expires:
Last-Modified: Tue, 03 Apr 2018 13:19:21 GMT
Pragma:
Server: Cowboy
Via: 1.1 vegur
X-Application-Context: ourder:prod,heroku:20249
X-Content-Type-Options: nosniff
X-Xss-Protection: 1; mode=block

With the Cloudflare Support Help i was able to solve the problem. The empty Expires Header causes the problem.
See this ticket for more details:
https://support.cloudflare.com/hc/en-us/requests/1508118?page=1

Related

Content-encoding in the Header of index page of Apache 2.4

I have Apache 2.4.37 without Nginx and with PHP 7.2.24.
When I download header of my site:
$ curl -I -H "Accept-Encoding: gzip" https://mysite/
HTTP/2 200
server: Apache/2.4.37 (centos) OpenSSL/1.1.1k
x-powered-by: PHP/7.2.24
expires: Thu, 19 Nov 1981 08:52:00 GMT
cache-control: no-store, no-cache, must-revalidate
pragma: no-cache
content-type: text/html; charset=utf-8
I can't see Content Encoding gzip in the Header, but if I download index page of my site via:
curl -o index.html -H "Accept-Encoding: gzip" https://mysite/
index.html is a gzip archive.
When I download header of my site CSS page(https://mysite/file.css):
$ curl -I -H "Accept-Encoding: gzip" https://mysite/
HTTP/2 200
server: Apache/2.4.37 (centos) OpenSSL/1.1.1k
accept-ranges: bytes
vary: Accept-Encoding
content-encoding: gzip
cache-control: max-age=31536050
content-length: 5135
content-type: text/css
So, where is "content-encoding: gzip" in the Header of php page of my site?
I tried to add header('Content-Encoding: gzip'); in the Php code of index.php, but Browser said that invalid Encoding.
Thanks.

Varnish, SSL and HTTP headers - No X-Cache after reload with SSL?

I noticed a strange behaviour of my Varnish/Apache setup which makes me wonder wether the pages are at any time properly cached or not.
I am running Apache on Port 8080 and Varnish on port 80 on the same machine. Also I'm using Apache for SSL termination with Varnish like described in this article
I am testing to open a webpage in four different ways and I would expect the HTTP X-Cache headers to appear in all four instances to prove that varnish is properly running:
Call a fresh page using http (Port 80) or force a fresh copy by using a force reload to get a Status 200 OK
Call the same page again in Port 80 (http) with a simple reload to get HTTP Status 304 Not modified
Call the page using https (Port 443) and force Status 200 ok with a force reload
Reload the page using https and get a Status 304.
I'm using Chrome DevTools to check the headers. Here are the results:
Option 1: Force fresh page, http
HTTP/1.1 200 OK
Date: Fri, 15 Mar 2019 22:00:10 GMT
Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.2k-fips PHP/5.5.38
X-Content-Type-Options: nosniff
X-Powered-By: PHP/5.5.38
X-Drupal-Cache: MISS
Expires: Sun, 19 Nov 1978 05:00:00 GMT
Cache-Control: public, max-age=21600
X-Content-Type-Options: nosniff
Content-Language: de
X-Frame-Options: SAMEORIGIN
X-UA-Compatible: IE=edge,chrome=1
Last-Modified: Fri, 15 Mar 2019 22:00:10 GMT
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Type: text/html; charset=utf-8
X-Varnish: 323946989 325583205
Age: 2444
Via: 1.1 varnish-v4
X-Cache: HIT
X-Cache-Hits: 235
Content-Length: 39753
Connection: keep-alive
Accept-Ranges: bytes
Option 2: Reload page, http
HTTP/1.1 304 Not Modified
Date: Fri, 15 Mar 2019 22:00:10 GMT
Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.2k-fips PHP/5.5.38
X-Content-Type-Options: nosniff
X-Powered-By: PHP/5.5.38
X-Drupal-Cache: MISS
Expires: Sun, 19 Nov 1978 05:00:00 GMT
Cache-Control: public, max-age=21600
X-Content-Type-Options: nosniff
Content-Language: de
X-Frame-Options: SAMEORIGIN
X-UA-Compatible: IE=edge,chrome=1
Etag: "1552687210-1"
Last-Modified: Fri, 15 Mar 2019 22:00:10 GMT
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Type: text/html; charset=utf-8
X-Varnish: 195923610 325583205
Age: 2616
Via: 1.1 varnish-v4
X-Cache: HIT
X-Cache-Hits: 250
Connection: keep-alive
Option 1: Force fresh page, https
HTTP/1.1 200 OK
Date: Fri, 15 Mar 2019 22:00:10 GMT
Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.2k-fips PHP/5.5.38
Expires: Sun, 19 Nov 1978 05:00:00 GMT
Cache-Control: public, max-age=21600
Vary: Accept-Encoding
X-Content-Type-Options: nosniff
X-Powered-By: PHP/5.5.38
X-Drupal-Cache: MISS
X-Content-Type-Options: nosniff
Content-Language: de
X-Frame-Options: SAMEORIGIN
X-UA-Compatible: IE=edge,chrome=1
Etag: "1552687210-1"
Last-Modified: Fri, 15 Mar 2019 22:00:10 GMT
Content-Encoding: gzip
Content-Type: text/html; charset=utf-8
X-Varnish: 482447464 325583205
Age: 1410
Via: 1.1 varnish-v4
X-Cache: HIT
X-Cache-Hits: 146
Content-Length: 39753
Accept-Ranges: bytes
Option 4: Page reload, https
HTTP/1.1 304 Not Modified
Date: Fri, 15 Mar 2019 22:00:10 GMT
Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.2k-fips PHP/5.5.38
Connection: Keep-Alive
Keep-Alive: timeout=5, max=100
Etag: "1552687210-1"
Expires: Sun, 19 Nov 1978 05:00:00 GMT
Cache-Control: public, max-age=21600
Vary: Accept-Encoding
Why is the header in the last request completely different? And how can I check wether the page is really delivered by Varnish?
The header is different because it's a 304 http response, it means the file is not transferred again and delivered from your browser cache directly, so you don't have Varnish headers in it. I don't know however why the first reload without SSL has Varnish headers.
Another thing, you should use Hitch as a ssl proxy, it's a lot better than Apache for this and it's using the PROXY protocols which have several advantages.

Instagram Authentication no longer shows sign in inputs just serves instagram.com

So I know that instagram just did some API updates which I am trying to implement. But the very first step no longer seem to work:
https://api.instagram.com/oauth/authorize/?client_id=CLIENT-ID&redirect_uri=REDIRECT-URI&response_type=code
This very first step used to send you to a sign in page where you input your username and password, if you were already signed in then it would send you to a page confirming that you want the app to tap into your profile.
Today this no longer does so, despite already having tried to clear local cache etc...
So here is a curl response on the matter. Any lights on this would be amazing and please note that the client ID and secret are correct:
curl -IL https://api.instagram.com/oauth/authorize/\?client_id\=3bfds87f9asd6f8sd76f44a6390b\&redirect_uri\=http://localhost:3000/auth/instagram\&response_type\=code
HTTP/1.1 302 FOUND
Content-Language: en
Expires: Sat, 01 Jan 2000 00:00:00 GMT
Vary: Cookie, Accept-Language
Location: https://api.instagram.com/accounts/login/?force_classic_login=&next=/publicapi/oauth/authorize/%3Fclient_id%3D3b44dsf798asdf68d7s6f8d7a6390b%26redirect_uri%3Dhttp%3A//localhost%3A3000/auth/instagram%26response_type%3Dcode
Pragma: no-cache
Cache-Control: private, no-cache, no-store, must-revalidate
Date: Fri, 12 Jun 2015 18:45:33 GMT
Content-Type: text/html; charset=utf-8
Set-Cookie: csrftoken=164f0e68ds9f6ds8f76s7df6382e07ee3; expires=Fri, 10-Jun-2016 18:45:33 GMT; Max-Age=31449600; Path=/
Connection: keep-alive
Content-Length: 0
HTTP/1.1 302 FOUND
Content-Language: en
Expires: Sat, 01 Jan 2000 00:00:00 GMT
Vary: Accept-Language
Location: https://instagram.com/
Pragma: no-cache
Cache-Control: private, no-cache, no-store, must-revalidate
Date: Fri, 12 Jun 2015 18:45:33 GMT
Content-Type: text/html; charset=utf-8
Connection: keep-alive
Content-Length: 0
HTTP/1.1 200 OK
Cache-Control: private, no-cache, no-store, must-revalidate
Content-Language: en
Content-Length: 10411
Content-Type: text/html
Date: Fri, 12 Jun 2015 18:45:34 GMT
Expires: Sat, 01 Jan 2000 00:00:00 GMT
Pragma: no-cache
Set-Cookie: csrftoken=e6c925f87d6f87ds6f877a36c70ad9; expires=Fri, 10-Jun-2016 18:45:34 GMT; Max-Age=31449600; Path=/
Set-Cookie: mid=VXsozafsdjhjkadf7TPuiiFJi; expires=Thu, 07-Jun-2035 18:45:34 GMT; Max-Age=630720000; Path=/
Vary: Cookie, Accept-Language, Accept-Encoding
X-Frame-Options: SAMEORIGIN
Connection: keep-alive
I'm glad it was a bug. Everything is normal now.
Instagram API bug. It's back up and running.

Trying to view website after IP change in Chrome

I've recently changed the IP on a website as I added a SSL certificate.
Now going back to view the site I get the "Apache is working on your cPanel and WHM Server" message.
http://domain.com/cgi-sys/defaultwebpage.cgi
This looks like a cache issue with Chrome as I can see the page on other devices and when using Incognito browsing in Chrome also in other browsers on the same PC.
I've cleared every history entry in Chrome from my domain and also removed cookies. I still can't view my site. It has been 6 days since I changed the IP.
Any ideas? I'd rather be using Chrome by the way .
Thanks for any suggestions :)
The real URL will take you to the site more than likely with no problems www.tshirtsofoz.com.au
It looks like a configuration issue at the server. I don't believe its at the client. Below, the point of interest is the response:
HTTP/1.1 302 Moved Temporarily
Date: Wed, 27 Aug 2014 08:01:52 GMT
Server: Apache/2.2.25 (Unix) mod_ssl/2.2.25 OpenSSL/0.9.8e-fips-rhel5 mod_bwlimited/1.4
X-Powered-By: PHP/5.4.21
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
X-Pingback: https://tshirtsofoz.com.au/xmlrpc.php
Set-Cookie: PHPSESSID=9690b2302154849c9f0f4b1583cd6175; path=/
Location: https://tshirtsofoz.com.au/wp-admin/
Content-Length: 0
Connection: close
Content-Type: text/html; charset=UTF-8
And I get no response when following the redirect (i.e., GET /wp-admin/ ...). If I follow without the trailing slash, I get:
$ echo -e "GET /wp-admin HTTPS/1.1\r\n" | openssl s_client -connect www.tshirtsofoz.com.au:443 -ign_eof
...
HTTP/1.1 301 Moved Permanently
Date: Wed, 27 Aug 2014 08:07:45 GMT
Server: Apache/2.2.25 (Unix) mod_ssl/2.2.25 OpenSSL/0.9.8e-fips-rhel5 mod_bwlimited/1.4
Location: https://tshirtsofoz.com.au/wp-admin/
Content-Length: 386
Connection: close
Content-Type: text/html; charset=iso-8859-1
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>301 Moved Permanently</title>
</head><body>
<h1>Moved Permanently</h1>
<p>The document has moved here.</p>
<hr>
<address>Apache/2.2.25 (Unix) mod_ssl/2.2.25 OpenSSL/0.9.8e-fips-rhel5 mod_bwlimited/1.4 Server at tshirtsofoz.com.au Port 443</address>
</body></html>
closed
Also, OpenSSL 0.9.8e has a number of vulnerablities.
$ echo -e "GET / HTTPS/1.1\r\n" | openssl s_client -connect www.tshirtsofoz.com.au:443 -ign_eof
CONNECTED(00000003)
depth=1 C = US, ST = Illinois, L = Chicago, O = "Trustwave Holdings, Inc.", CN = "Trustwave Organization Validation CA, Level 2", emailAddress = ca#trustwave.com
verify error:num=20:unable to get local issuer certificate
verify return:0
---
Certificate chain
0 s:/CN=tshirtsofoz.com.au/O=T-shirts of Oz/L=Adelaide/ST=SA/C=AU
i:/C=US/ST=Illinois/L=Chicago/O=Trustwave Holdings, Inc./CN=Trustwave Organization Validation CA, Level 2/emailAddress=ca#trustwave.com
1 s:/C=US/ST=Illinois/L=Chicago/O=Trustwave Holdings, Inc./CN=Trustwave Organization Validation CA, Level 2/emailAddress=ca#trustwave.com
i:/C=US/O=SecureTrust Corporation/CN=SecureTrust CA
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIFYTCCBEmgAwIBAgITBlERJ/3dcNwVHxcP+I8iPTg4jTANBgkqhkiG9w0BAQUF
ADCBrjELMAkGA1UEBhMCVVMxETAPBgNVBAgTCElsbGlub2lzMRAwDgYDVQQHEwdD
aGljYWdvMSEwHwYDVQQKExhUcnVzdHdhdmUgSG9sZGluZ3MsIEluYy4xNjA0BgNV
BAMTLVRydXN0d2F2ZSBPcmdhbml6YXRpb24gVmFsaWRhdGlvbiBDQSwgTGV2ZWwg
MjEfMB0GCSqGSIb3DQEJARYQY2FAdHJ1c3R3YXZlLmNvbTAeFw0xNDA4MjExNzM0
MDJaFw0xNTA4MjEyMzM0MDJaMGMxGzAZBgNVBAMMEnRzaGlydHNvZm96LmNvbS5h
dTEXMBUGA1UECgwOVC1zaGlydHMgb2YgT3oxETAPBgNVBAcMCEFkZWxhaWRlMQsw
CQYDVQQIDAJTQTELMAkGA1UEBhMCQVUwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAw
ggEKAoIBAQC+9GB9oA1AlmI0BbIqDxpwVwWamGUwfGcc91tHoqkgXDQ/GztGTLb9
OLJD2ZYDt8PWe1NQHlpOo1LZkaqhoUEakvcI0DA10KUYGG77qNGG6jvseue/ndKn
T6UXsptFdnAyDBCbICRcWjcjgPq4MAjkMDCTZSyrQ7fKigjVlAiw0bqh3twnpp+Q
CLk51J3tDto9pYdgUWkEZoL4bdvxdP7iat8cd7b6JQuNWPU28SAagytUNj0IrU1q
nLm8YNJf38QvoWHonXl3dBdrkhHGFE2i7RRirGG6LJHb1KMpmJwXWlX+nHqK2yXk
nBAjsf8pBmSLuLZlPhVzjDGCHbDXS7eTAgMBAAGjggHAMIIBvDALBgNVHQ8EBAMC
BaAwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMBMB0GA1UdDgQWBBR+2SN4
zqcGpgpVrM1f36uxdTINBTAfBgNVHSMEGDAWgBRd2ZaaQMcnyyybouzPGavIr8yG
SDBIBgNVHSAEQTA/MD0GDysGAQQBge0YAwMDAwQEAzAqMCgGCCsGAQUFBwIBFhxo
dHRwczovL3NzbC50cnVzdHdhdmUuY29tL0NBMFsGA1UdEQRUMFKCEnRzaGlydHNv
Zm96LmNvbS5hdYIPdHNoaXJ0c29mb3ouY29tghZ3d3cudHNoaXJ0c29mb3ouY29t
LmF1ghN3d3cudHNoaXJ0c29mb3ouY29tMDUGA1UdHwQuMCwwKqAooCaGJGh0dHA6
Ly9jcmwudHJ1c3R3YXZlLmNvbS9PVkNBX0wyLmNybDBwBggrBgEFBQcBAQRkMGIw
JgYIKwYBBQUHMAGGGmh0dHA6Ly9vY3NwLnRydXN0d2F2ZS5jb20vMDgGCCsGAQUF
BzAChixodHRwOi8vc3NsLnRydXN0d2F2ZS5jb20vaXNzdWVycy9PVkNBX0wyLmNy
dDANBgkqhkiG9w0BAQUFAAOCAQEAPm7pdeMacdxfke86QEo/lR8lFlzcpalx/sSE
2GdmWLINFu2k1UbbhBNfqD9QaleKK7UaqxvKNkziJb0QGvQu4zQVjgTgW3X7xEdh
tHRINpAD5Pqn6snhtGLsuEWXngJSVjj8HvEpnZBtEHwsWuoQJhv0JUtHU5hrh96y
AvUrqLCXg52Rb3h6IZxKgsO6yhBtpSHf5W4zW8PCF7v7BD8jRvhn421vQM2MDQUt
0EW8NWu2WU780tBCIGgUDAudDcuv2sDlvSrCl4qw9JvZtxEjvYhxetPWUccTbLz+
uK8T3Pf6/zSE3hLU4w+kslmKflrRzI4aCAAHekPd1+r3RAgt5w==
-----END CERTIFICATE-----
subject=/CN=tshirtsofoz.com.au/O=T-shirts of Oz/L=Adelaide/ST=SA/C=AU
issuer=/C=US/ST=Illinois/L=Chicago/O=Trustwave Holdings, Inc./CN=Trustwave Organization Validation CA, Level 2/emailAddress=ca#trustwave.com
---
No client certificate CA names sent
---
SSL handshake has read 3217 bytes and written 506 bytes
---
New, TLSv1/SSLv3, Cipher is DHE-RSA-AES256-SHA
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
Protocol : TLSv1
Cipher : DHE-RSA-AES256-SHA
Session-ID: 94A2EBDFF8EBA3B60B0CBCE497B21FB5753DD07FABFBD21CC2642C2F0665EE34
Session-ID-ctx:
Master-Key: 74BBCA1A44D5A8AAD5FD916C761E349B7FAD876D35B03AB499E20ADBC8170C643917DAF414D8AAE46D5574349DBCBB07
Key-Arg : None
PSK identity: None
PSK identity hint: None
SRP username: None
Start Time: 1409126533
Timeout : 300 (sec)
Verify return code: 20 (unable to get local issuer certificate)
---
HTTP/1.1 302 Moved Temporarily
Date: Wed, 27 Aug 2014 08:01:52 GMT
Server: Apache/2.2.25 (Unix) mod_ssl/2.2.25 OpenSSL/0.9.8e-fips-rhel5 mod_bwlimited/1.4
X-Powered-By: PHP/5.4.21
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
X-Pingback: https://tshirtsofoz.com.au/xmlrpc.php
Set-Cookie: PHPSESSID=9690b2302154849c9f0f4b1583cd6175; path=/
Location: https://tshirtsofoz.com.au/wp-admin/
Content-Length: 0
Connection: close
Content-Type: text/html; charset=UTF-8
closed

403 Forbidden when using access token from OAuth2ServiceAccount - Error Site must be added to your account and verified

403 Forbidden when using access token from OAuth2ServiceAccount
My request is inserting sitemap.xml to webmaster tool
here the response from google
HTTP/1.1 403 Forbidden
Content-length: 48
X-xss-protection: 1; mode=block
X-content-type-options: nosniff
X-google-cache-control: remote-fetch
-content-encoding: gzip
Server: GSE
Reason: Forbidden
Via: HTTP/1.1 GWA
Cache-control: private, max-age=0
Date: Tue, 29 Jan 2013 11:03:08 GMT
X-frame-options: SAMEORIGIN
Content-type: text/html; charset=UTF-8
Expires: Tue, 29 Jan 2013 11:03:08 GMT
Site must be added to your account and verified.