Refused to get unsafe header "Content-Range" - http-headers

For some reason I cannot access the response header of "Content-Range" anymore... Therefore it's impossible to determine the file size of a resource using XHR.
I get error Refused to get unsafe header "Content-Range" on Chrome in this line:
var cr = this.getResponseHeader('Content-Range');
Here's the CORS config:
<?xml version="1.0" ?>
<CorsConfig>
<Cors>
<Origins>
<Origin>*</Origin>
</Origins>
<Methods>
<Method>GET</Method>
<Method>HEAD</Method>
<Method>DELETE</Method>
</Methods>
<ResponseHeaders>
<ResponseHeader>x-goog-meta-foo1</ResponseHeader>
<ResponseHeader>origin</ResponseHeader>
<ResponseHeader>range</ResponseHeader>
<ResponseHeader>Content-Range</ResponseHeader>
<ResponseHeader>Content-Length</ResponseHeader>
</ResponseHeaders>
<MaxAgeSec>1800</MaxAgeSec>
</Cors>
CURL output:
$ curl -H "Origin: http://peer5.com" http://commondatastorage.googleapis.com/peer5_vod/wind2_orig.mp4 -s -D - -o /dev/null
HTTP/1.1 200 OK
Server: HTTP Upload Server Built on May 8 2013 16:51:19 (1368057079)
Expires: Mon, 13 May 2013 09:47:40 GMT
Date: Mon, 13 May 2013 08:47:40 GMT
Cache-Control: public, max-age=3600, no-transform
Last-Modified: Fri, 22 Mar 2013 17:09:47 GMT
ETag: "755232ae8fef22bc7b4e9510a68a646e"
x-goog-generation: 1363972188238000
x-goog-metageneration: 2
Content-Type: video/mp4
x-goog-hash: crc32c=pZmS2Q==
x-goog-hash: md5=dVIyro/vIrx7TpUQpopkbg==
Accept-Ranges: bytes
Content-Length: 15535795
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: Content-Length, Date, Server, Transfer-Encoding

This appears to be a bug on our side. Only the last header value in the ResponseHeaders list is returned in the Access-Control-Expose-Headers header. We are working on rolling out a fix, but as a workaround, if you only need the Content-Range header (Content-Length is considered a simple header by the CORS spec and is added automatically), please try setting your CORS config to this:
<?xml version="1.0" ?>
<CorsConfig>
<Cors>
<Origins>
<Origin>*</Origin>
</Origins>
<Methods>
<Method>GET</Method>
<Method>HEAD</Method>
<Method>DELETE</Method>
</Methods>
<ResponseHeaders>
<ResponseHeader>Content-Range</ResponseHeader>
</ResponseHeaders>
<MaxAgeSec>1800</MaxAgeSec>
</Cors>
</CorsConfig>
UPDATE: This bug has been fixed.

Related

AWS S3 + CloudFront: Fonts not loading (CORS Problem)

I added some custom fonts to my website and uploaded them to AWS S3 + CloudFront.
A lot of topics here describe this problem but non of them are solving my issue.
Using curl I get this output:
curl --head https://cdn.mzguru.de/fonts/sourcesanspro/source-sans-pro-v12-latin-ext_latin-700.woff2
HTTP/1.1 200 OK
Content-Type: binary/octet-stream
Content-Length: 25348
Connection: keep-alive
Date: Tue, 22 Oct 2019 11:54:18 GMT
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET
Access-Control-Max-Age: 3000
Last-Modified: Fri, 12 Apr 2019 10:54:26 GMT
ETag: "639c2738552a0376c91e7d485e476fda"
Cache-Control: max-age=62208000
Accept-Ranges: bytes
Server: AmazonS3
X-Cache: Hit from cloudfront
Via: 1.1 bae3e24625567f5728a5caa96d6b7669.cloudfront.net (CloudFront)
X-Amz-Cf-Pop: FRA53
X-Amz-Cf-Id: iAy-QTfuV9ZqwmaRjXE0ramVSgsZkA6MtRmQOKDSonf6I8OabrpLZA==
Age: 12818
Within Chrome I get this error:
Access to font at 'https://cdn.mzguru.de/fonts/sourcesanspro/source-sans-pro-v12-latin-ext_latin-700.woff2' from origin 'https://www.monteurzimmerguru.de' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.
This is the point where I do not understand the problem. The error message says: "No 'Access-Control-Allow-Origin' header is present"
But in the curl request I see this header. What is wrong?
Thank you
EDIT
I have attached a screenshot with the error messages.
EDIT 2: AWS Interface changed (2022)
Please take a look at #James Dean post.
1.) Do I need to tick the options box?
2.) I can not find the settings you describe. I guess the UI changed in the meanwhile.
Your S3 CORS configuration is correct based on below output:
>curl -vk "https://cdn.mzguru.de/fonts/sourcesanspro/source-sans-pro-v12-latin-ext_latin-700.woff2" -H "Origin: https://www.monteurzimmerguru.de"
< HTTP/2 200
< content-type: binary/octet-stream
< content-length: 25348
< date: Thu, 24 Oct 2019 12:19:41 GMT
< access-control-allow-origin: *
< access-control-allow-methods: HEAD, GET
< access-control-max-age: 3000
< last-modified: Fri, 12 Apr 2019 10:54:26 GMT
< etag: "639c2738552a0376c91e7d485e476fda"
< cache-control: max-age=62208000
< accept-ranges: bytes
< server: AmazonS3
< x-cache: Hit from cloudfront
However, Chrome/Browser is making OPTIONS/Preflight request on CloudFront and options request is not allowed on cloudfront currently. Only Head and GET are allowed.
curl -X OPTIONS "https://cdn.mzguru.de/fonts/sourcesanspro/source-sans-pro-v12-latin-ext_latin-700.woff2" -H "Origin: https://www.monteurzimmerguru.de"
>This distribution is not configured to allow the HTTP request method that was used for this request
To fix this , you need to do it:
In the CloudFront cache behaviour, you need to allow GET,HEAD and OPTIONS
In Cache behaviour, cache based on selected header, you should select Origin
Invalidate cache once and test it again.
You have to update,
Query String Forwarding and Caching to Forward all, cache based on all
in cloudfront Cache Behavior Settings ( cloudfont -> select one -> edit )

icecast returns text/html instead of audio/mpeg

I've build a simple internet radio using ESP8266+VS1053. It works.
The query sent to icecast server looks like:
GET / HTTP/1.1
User-Agent: Windows-Media-Player/10.00.00.1111
Host: 66.85.88.242
Icy-Metadata:1
Accept: audio/mpeg,audio/aacp
Connection: keep-alive
SOME stations works fine, while others returns html instead of stream data. For example:
SheRadio: http://66.85.88.242:8136/1139_128 -- returns
HTTP/1.1 200 OK
Content-Length: 501
Content-Type: text/html
Date: Fri, 15 Jun 2018 22:05:43 GMT
Server: Icecast 2.4.0-kh3
Cache-Control: no-cache, no-store
Pragma: no-cache
Access-Control-Allow-Origin: *
Access-Control-Allow-Headers: Origin, Accept, X-Requested-With, Content-Type
Access-Control-Allow-Methods: GET, OPTIONS, HEAD
Connection: Close
Expires: Mon, 26 Jul 1997 05:00:00 GMT
DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html>
<head>
<link rel="stylesheet" type="text/css" href="/style.css" />
<title>Icecast Streaming Media Server</title>
(and so on)
instead of:
HTTP/1.0 200 OK
icy-notice1:<BR>This stream requires Winamp<BR>
icy-notice2:SHOUTcast DNAS/posix(linux x64) v2.5.5.733<BR>
Accept-Ranges:none
Access-Control-Allow-Origin:*
Cache-Control:no-cache,no-store,must-revalidate,max-age=0
Connection:close
icy-name:Goldradio Network
icy-genre:60s, 70s, 80s, Top 40, Oldies
icy-br:24
icy-sr:16000
icy-url:http://www.goldradio.org.uk
icy-pub:1
content-type:audio/mpeg
icy-metaint:8192
X-Clacks-Overhead:GNU Terry Pratchett
sent by other stations.
I was completely unable to understand why some worked and some not.
Thanks in advance for any help/ideas.
Please note that you MUST provide the proper path in your HTTP GET request!
GET / HTTP/1.1 is NOT requesting 'http://66.85.88.242:8136/1139_128', but 'http://66.85.88.242:8136/'. The latter is 'text/html' content.
This can be easily verified using:
curl -v --user-agent "Windows-Media-Player/10.00.00.1111" -H "Icy-Metadata: 1" -H "Accept: audio/mpeg,audio/aacp" http://66.85.88.242:8136/ >/dev/null
vs.
curl -v --user-agent "Windows-Media-Player/10.00.00.1111" -H "Icy-Metadata: 1" -H "Accept: audio/mpeg,audio/aacp" http://66.85.88.242:8136/1139_128 >/dev/null
Not sure how it is for ESP8266, but at least for the ESP32, libcurl builds perfectly fine and would be my preferred option for handling HTTP requests instead of error prone hand crafting.

GET Bucket op response + AWS S3 + Content-Length header

Just wanted to know if the GET Bucket op response ever skips the Content-Length header. I tested this and i saw that there was no Content-Length header in the response for GET Bucket op.
How does an application reading the response understand where the body of the response ends if the response doesn't contain Content-Length header?
Request-Response Snippet:
GET /?max-keys=1000&prefix&delimiter=%2F HTTP/1.1
Date: Sat, 09 Apr 2016 18:27:23 GMT
x-amz-request-payer: requester
Authorization: AWS AKIAIP3KAUILC4GG7A2A:UG3bGvIjayrxrkxEX1mfrvETy/M=
Connection: Keep-Alive
User-Agent: Cyberduck/4.9.19632 (Mac OS X/10.10.5) (x86_64)
HTTP/1.1 200 OK
x-amz-id-2: yg76HSq5j0mi0oR6dXF8ZfGq722kHBWiMQmNvXPqiLxr1S4nGj5GVn1RVrPQrOUfNynxxaMSYEY=
x-amz-request-id: B4468E68E10B6AEF
Date: Sat, 09 Apr 2016 18:27:25 GMT
x-amz-bucket-region: us-east-1
Content-Type: application/xml
Server: AmazonS3
Connection: close
<?xml version="1.0" encoding="UTF-8"?>
<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">......</ListBucketResult>
Thanks!
The Content-Length header is optional in response. And it may not reflect the real content-length even if it presents. Think about gzipped response. So to answer the question: When no Content-Length is received, the client keeps reading until the server closes the connection.
In Java, keep calling InputStream.read() until it returns -1.
Is the Content-Length header required for a HTTP/1.0 response?

What configuration is necessary for Amazon s3 and CORS?

As far as I am able to tell, the only configuration which should be necessary to get s3 to return the Access-Control-Allow-Origin header is setting a CORS policy and sending the Origin header.
Here is my CORS policy:
<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<CORSRule>
<AllowedOrigin>*</AllowedOrigin>
<AllowedMethod>GET</AllowedMethod>
<AllowedMethod>HEAD</AllowedMethod>
<AllowedHeader>*</AllowedHeader>
</CORSRule>
</CORSConfiguration>
When I send this request:
GET /dots.png HTTP/1.1
Host: dta-test.s3.amazonaws.com
Origin: https://www.test.com
Cache-Control: no-cache
I get a response with these headers:
Accept-Ranges →bytes
Cache-Control →public
Content-Length →893
Content-Type →image/png
Date →Thu, 09 Jan 2014 22:27:20 GMT
ETag →"5345e98a3abcb1057d1a427551d8d702"
Last-Modified →Thu, 09 Jan 2014 19:27:56 GMT
Server →AmazonS3
x-amz-id-2 →2AFNGowXCkLUJa09ZNERXqzxn5IwnygTKCXut0m0gvpapjxXn/kAPYQNvv4pYvVy
x-amz-request-id →71B183AF938A075D
As far as I am aware, the Access-Control-Allow-Origin header should be included in this response. What am I missing?
Edit:
This is what I'm missing: I was using the Postman Chrome extension. Chrome sanitizes the Origin header when making requests, even from extensions. It throws the error Refused to set unsafe header "Origin", which I missed.
We can see with curl that this is actually working:
curl -H "Origin: https://www.test.com" -I "https://dta-test.s3.amazonaws.com/dots.png" -s
HTTP/1.1 200 OK
x-amz-id-2: NnRAOdCOpMCtZ8Jk1bpnuRASb0K/gzM0Vv/6D28C6grcbEZoe2OC0cuu3SMwDaXe
x-amz-request-id: 70110890812CAC58
Date: Thu, 09 Jan 2014 23:03:41 GMT
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET, HEAD
Vary: Origin, Access-Control-Request-Headers, Access-Control-Request-Method
Cache-Control: public
Last-Modified: Thu, 09 Jan 2014 19:27:56 GMT
ETag: "5345e98a3abcb1057d1a427551d8d702"
Accept-Ranges: bytes
Content-Type: image/png
Content-Length: 893
Server: AmazonS3

Cliqset.com Error 403 while trying to update any data through API

I used both Fiddler and my test C# application to update my profile information in Cliqset.com through their API but I always get 403 error. It works well when I just want to get my profile info, not to update. I use all necessary HTTP headers, like CliqsetConsumerKey and Authorization.
Thank you.
When I try to get my status, I use this:
GET https://api.cliqset.com/user/cosmorocket/status HTTP/1.1
Authorization: Basic Y**********rZXQ6ZnVsY3J1bTA=
CliqsetConsumerKey: 11**********3c2ccd26c4ca946e3d8e
Host: api.cliqset.com
And it returns this:
HTTP/1.1 200 OK
Date: Fri, 10 Jul 2009 05:17:37 GMT
Server: Apache-Coyote/1.1
X-Powered-By: Servlet 2.4; JBoss-4.2.3.GA (build: SVNTag=JBoss_4_2_3_GA date=200807181417)/JBossWeb-2.0
Content-Type: application/xml
Content-Length: 153
Set-Cookie: csp=appInst2; path=/
Connection: close
<?xml version="1.0" encoding="UTF-8" standalone="yes"?><StatusEntry><Updated>2009-06-09T18:09:29.315Z</Updated><Status>cliqsetting all day long</Status></StatusEntry>
But if I use the same credentials for updating with this request:
PUT https://api.cliqset.com/200905/user/cosmorocket/status HTTP/1.1
I get this:
HTTP/1.1 403 Forbidden
Date: Fri, 10 Jul 2009 05:25:04 GMT
Vary: Accept-Encoding
Content-Length: 9
Connection: close
Content-Type: text/html; charset=iso-8859-1
Error 403
I contacted tech support and solved the problem. Each time a new user uses my registered application it's necessary to make an authenticated GET
request to 'https://api.cliqset.com/200905/user/useridentifier'. Notice that you don't change 'useridentifier' to any other login name, just leave it as it is. This
will setup the appropriate permissions to update data.
I did this request and now it works well.