My CORS rule doesn't fix my CORS error - amazon-s3

I have some CORS rules on my S3 bucket.
This is what it looks like:
<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<CORSRule>
<AllowedOrigin>https://prod-myapp.herokuapp.com/</AllowedOrigin>
<AllowedMethod>POST</AllowedMethod>
<AllowedHeader>*</AllowedHeader>
</CORSRule>
<CORSRule>
<AllowedOrigin>http://prod-myapp.herokuapp.com/</AllowedOrigin>
<AllowedMethod>POST</AllowedMethod>
<AllowedHeader>*</AllowedHeader>
</CORSRule>
</CORSConfiguration>
When I am in my app, and I try to upload a file (aka...do a POST request) in my JS console, I get this error:
XMLHttpRequest cannot load https://myapp.s3.amazonaws.com/. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://prod-myapp.herokuapp.com' is therefore not allowed access. The response had HTTP status code 403.
I attempted to do a POST from my CLI and I got this:
$ curl -v -H "Origin: http://prod-myapp.herokuapp.com" -X POST https://myapp.s3.amazonaws.com
* Rebuilt URL to: https://myapp.s3.amazonaws.com/
* Trying XX.XXX.XX.153...
* Connected to myapp.s3.amazonaws.com (XX.XXX.XX.153) port 443 (#0)
* TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
* Server certificate: *.s3.amazonaws.com
* Server certificate: VeriSign Class 3 Secure Server CA - G3
* Server certificate: VeriSign Class 3 Public Primary Certification Authority - G5
> POST / HTTP/1.1
> Host: myapp.s3.amazonaws.com
> User-Agent: curl/7.43.0
> Accept: */*
> Origin: http://prod-myapp.herokuapp.com
>
< HTTP/1.1 412 Precondition Failed
< x-amz-request-id: SOME_ID
< x-amz-id-2: SOME_ID_2
< Content-Type: application/xml
< Transfer-Encoding: chunked
< Date: Thu, 17 Sep 2015 04:43:28 GMT
< Server: AmazonS3
<
<?xml version="1.0" encoding="UTF-8"?>
* Connection #0 to host myapp.s3.amazonaws.com left intact
<Error><Code>PreconditionFailed</Code><Message>At least one of the pre-conditions you specified did not hold</Message><Condition>Bucket POST must be of the enclosure-type multipart/form-data</Condition><RequestId>SOME_ID</RequestId><HostId>SOME_HOST_ID</HostId></Error>
I just added the CORS rule that applies to the domain I am trying from about 10 - 15 minutes ago. But I was under the impression that it should happen immediately.
Is there some remote cache that I need to bust to get my browser to work? I tried it both in normal mode and in Incognito Mode.
Also, based on the results from curl, it seems as if I am no longer getting an Access-Control-Allow-Origin header error, right? So, theoretically, it should be working in my browser.
Am I misreading what is happening at the command-line?
What else am I missing?

This is a slightly solution that what I have done.
I set up the policy in S3 to allow put content to bucket by only the restrict domain as referer
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AddPerm",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::myapp/*",
"Condition": {
"StringLike": {
"aws:Referer": "http://prod-myapp.herokuapp.com/*"
}
}
}
]
}
so you can test the PUT method by
curl -v -H "Referer: http://prod-myapp.herokuapp.com/index.php" -H "Content-Length: 0" -X PUT https://myapp.s3.amazonaws.com/testobject.jpg

The error message from curl says:
At least one of the pre-conditions you specified did not hold
Bucket POST must be of the enclosure-type multipart/form-data
You can make curl use the content-type "multipart/form-data" by using the -F option (e.g. "-F name=value"). You can use this multiple times to add all of the form parameters that you need. This page lists the parameters expected by S3:
http://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectPOST.html
Specifying "file" and "key" gets you to the point where it fails with an "Access Denied" error. I assume that you've set it to be private, so you probably need the "access-key-id" or similar to get beyond this point.
curl -v -H "Origin: http://prod-myapp.herokuapp.com" -X POST \
https://myapp.s3.amazonaws.com -F key=wibble -F file=value
Also, based on the results from curl, it seems as if I am no longer getting an Access-Control-Allow-Origin header error, right? So, theoretically, it should be working in my browser.
It seems actually to make no difference whether you specify the -H origin option, so I'm not sure if your CORS setting is actually having any effect.

Check which requests you send to the server, before POST request can be sent OPTIONS request (chrome do it)
I got Precondition Failed error for CORS because only POST method was allowed, allowing OPTIONS method resolved this problem.

Related

Letting upstream handle cors requests

I'm trying to setup a service that already handles CORS requests and would like to keep it that way instead of handling the CORS request on the Edge Proxy.
Leaving the cors field blank didn't help at all.
Is there anyway to achieve this with Ambassador?
Ambassador will not handle CORS in anyway unless you set the cors parameter in a Mapping or Module config.
Even if that is set, the way Envoy handles CORS seems to be the behavior you are searching for.
Taking a look at the linked comment in this issue https://github.com/envoyproxy/envoy/issues/300#issuecomment-296796675, we can see how Envoy chose to implement it's CORS filter. Specifically:
Assign values to the CORS headers in the repsponse: For each of the headers specified in Table 1 above:
a. let value be the option for the header config
b. if value is not defined, continue to the next header
c. else, write the response header for the specified config option
This means that Envoy will first take the value of the headers set by the upstream service and only write them with the configured values if they are not set in the response.
You can test this by creating a route to the httpbin.org (which handles CORS) and setting cors parameter in the Mapping.
---
apiVersion: getambassador.io/v2
kind: Mapping
metadata:
name: cors-httpbin
spec:
prefix: /httpbin/
service: httpbin.org
cors:
origins:
- http://foo.example
methods:
- POST
- OPTIONS
The Mapping above should configure Envoy to set the access-control-allow-origins and access-control-allow-methods headers to http://foo.example.com and POST respectively. However, after sending a test request to this endpoint, we can see that we are instead getting very different CORS headers back in the response:
curl https://aes.example.com/httpbin/headers -v -H "Origin: http://bar.example.com" -H "Access-Control-Request-Method: GET" -X OPTIONS
* Trying 34.74.58.157:443...
* Connected to aes.example.com (10.11.12.100) port 443 (#0)
* TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate: aes.example.com
* Server certificate: Let's Encrypt Authority X3
* Server certificate: DST Root CA X3
> OPTIONS /httpbin/headers HTTP/1.1
> Host: aes.example.com
> User-Agent: curl/7.69.0
> Accept: */*
> Origin: http://bar.example.com
> Access-Control-Request-Method: GET
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< date: Thu, 19 Mar 2020 13:25:48 GMT
< content-type: text/html; charset=utf-8
< content-length: 0
< server: envoy
< allow: HEAD, OPTIONS, GET
< access-control-allow-origin: http://bar.example.com
< access-control-allow-credentials: true
< access-control-allow-methods: GET, POST, PUT, DELETE, PATCH, OPTIONS
< access-control-max-age: 3600
< x-envoy-upstream-service-time: 33
<
* Connection #0 to host aes.example.com left intact
This is because the httpbin.org upstream is setting these headers in the response and so Envoy is defaulting to using them instead of forcing the CORS configuration we gave it. In this way, Envoy really acts as a default for CORS settings and allows upstreams to set more or less restrictive configurations as they see fit.
This behavior can be confusing and caused me a lot of headaches trying to figure it out. I hope I helped clear it up for you.

Kong responds 404 when accessed vi Postman

I have set up an API and Kong. I have successfully installed Kong. In fact, i can access my APIs via curl on cli. However, when I use Postman, which I use all of the time, I get the response
{
"request_path": "/api/v1/",
"message": "API not found with these values",
"request_host": [
"192.168.33.13"
]
}
Ps. Im new to kong so please bear with me.
$ kong start
[INFO] Kong 0.7.0
[INFO] Using configuration: /etc/kong/kong.yml
[INFO] Setting working directory to /usr/local/kong
[INFO] database...........cassandra keyspace=kong ssl=verify=false enabled=false replication_factor=1 contact_points=127.0.0.1:9042 replication_strategy=SimpleStrategy timeout=5000 data_centers=
[INFO] dnsmasq............address=127.0.0.1:8053 dnsmasq=true port=8053
[INFO] serf ..............-profile=wan -rpc-addr=127.0.0.1:7373 -event-handler=member-join,member-leave,member-failed,member-update,member-reap,user:kong=/usr/local/kong/serf_event.sh -bind=0.0.0.0:7946 -node=precise64_0.0.0.0:7946 -log-level=err
[INFO] Trying to auto-join Kong nodes, please wait..
[WARN] Cannot auto-join the cluster because no nodes were found
[WARN] ulimit is currently set to "1024". For better performance set it to at least "4096" using "ulimit -n"
[INFO] nginx .............admin_api_listen=0.0.0.0:8001 proxy_listen=0.0.0.0:8000 proxy_listen_ssl=0.0.0.0:8443
[OK] Started
$ curl -i -X POST --url http://localhost:8001/apis/ --data 'name=geospatial' --data 'upstream_url=http://192.168.33.10/' --data 'request_host=192.168.33.10'
HTTP/1.1 201 Created
Date: Fri, 08 Apr 2016 14:38:22 GMT
Content-Type: application/json; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
Access-Control-Allow-Origin: *
Server: kong/0.7.0
{"upstream_url":"http:\/\/192.168.33.10\/","id":"240e6cc6-626f-4629-9551-0d341a57adba","name":"geospatial","created_at":1460126302000,"request_host":"192.168.33.10"}
When I curl the api...
$ curl -i -X GET -H "Host: 192.168.33.10" "http://192.168.33.13:8000/api/v1/"
HTTP/1.1 200 OK
Date: Fri, 08 Apr 2016 14:56:04 GMT
Content-Type: application/json
Content-Length: 70
Connection: keep-alive
Server: Werkzeug/0.11.4 Python/2.7.6
X-Kong-Upstream-Latency: 4
X-Kong-Proxy-Latency: 0
Via: kong/0.7.0
{"status": 200, "message": 200, "data": "Hello World!", "error": null}
How ever when I try to use Postman, I get a 404 status response.
with this body
{
"request_path": "/api/v1/",
"message": "API not found with these values",
"request_host": [
"192.168.33.13"
]
}
Am I missing something?
Disclaimer:
I am using Postman because I am lazy and because my clients will mostly be using Postman as well. So dont tell me to to use curl instead :)
Just encountered the same problem.
In order to send to Kong you need to add Host header, but Postman has some restricted headers that are blocked and Host is one of them.
You need to download Chrome Inspector to send the Host header.
For more details -
https://www.getpostman.com/docs/requests
You need to setup the header parameter named "Host" in order to Kong find your API.
You can see that your curl command have this parameter
-H "Host: 192.168.33.10"

Office 365 REST API to create contacts using CURL gets HTTPCode 400 Bad Request

I am testing Office 365 REST API using CURL following this link:
Contacts REST API in Office 365 APIs Preview
I can obtain correctly one contact using curl command in Windows like this:
curl --no-sessionid --insecure --basic --user "user#domain.com:password" -H "Accept: application/json" "https://outlook.office365.com/EWS/OData/Me/Contacts?$orderby=DisplayName+asc&$top=1"
And following documentation on this link if I try to create one contact using CURL with minimum required options for testing:
curl -X POST -d "{\"#odata.type\": \"#Microsoft.Exchange.Services.OData.Model.Contact\",\"GivenName\": \"TestContact\",\"EmailAddress1\": \"test#test.com\",\"BusinessPhone1\": \"123-456-7890\"}" https://outlook.office365.com/ews/odata/Me/Contacts --header "Content-Type:application/json" --insecure --verbose --user "user#domain.com:password"
I receive following error:
* About to connect() to outlook.office365.com port 443 (#0)
* Trying 157.56.250.178...
* connected
* Connected to outlook.office365.com (157.56.250.178) port 443 (#0)
* SSLv3, TLS handshake, Client hello (1):
...
* SSL connection using ECDHE-RSA-AES256-SHA
* Server certificate:
...
* Server auth using Basic with user 'user#domain.com'
> POST /ews/odata/Me/Contacts HTTP/1.1
...
> Content-Length: 157
>
* upload completely sent off: 157 out of 157 bytes
< HTTP/1.1 400 Bad Request
...
< Content-Length: 82
<
{"error":{"code":"ErrorInvalidRequest","message":"Cannot read the request body."}}* Connection #0 to host outlook.office365.com left intact
* Closing connection #0
* SSLv3, TLS alert, Client hello (1):
I have searched the internet and in stackoverflow but similar questions have no answer I'm looking for.
How could I create on contact using CURL?, I have tested it on Linux too but have the same results.
The following request works when I try it from Fiddler. Can you please try this out using CURL?
POST https://outlook.office365.com/ews/odata/Me/Contacts HTTP/1.1
Authorization: Basic <XXXX>
Content-Type: application/json
{
"GivenName" : "John",
"EmailAddresses" : [
{ "Address": "John#contoso.com", "Name" : "John" }
],
"BusinessPhones" : [
"123-456-7890"
]
}
I made a few changes to your request. You don't need to specify the OData.type as we infer that you are adding a Contact because you are sending a POST to Contacts collection. We need to fix our documentation as it lists the entity type as required. We have updated our namespace to Microsoft.OutlookServices and hence the type definitions have changed. To make the API easier to use, we have replaced EmailAddress1, EmailAddress2 etc. with a collection of EmailAddresses. Similarly, we have also changed BusinessPhones, HomePhones etc. to collections as well.
As I just explained in another post, the issues you are seeing are from some changes being rolled out to our preview APIs and our documentation is in the process of being updated. The current set of changes include versioning support, and this won't be an issue going forward.
Please let me know if you have any questions or need more info.
Thanks,
Venkat
Using following CURL command worked perfect:
curl -X POST -d "{\"GivenName\":\"John\",\"EmailAddresses\":[{\"Address\":\"John#contoso.com\",\"Name\":\"John\"}],\"BusinessPhones\":[\"123-456-7890\"]}" https://outlook.office365.com/ews/odata/Me/Contacts --header "Content-Type:application/json" --insecure --verbose --user "user#domain.com:password"
Now we can export contacts (this is not allowed from OWA) and import it using simple utilities like curl.
Thanks for your help Venkat.
Your original post showed the following namespace:
Microsoft.Exchange.Services.OData.Model.Contact
That has been changed to:
Microsoft.Office365.OutlookServices.Contact
Since the namespace was wrong, it couldn't read the request body.

Github API 502 error

I'm trying to add a user to a Github repository via their API, but I always get a 502 Bad Gateway error.
With curl I send a request like this (<...> replaced by a real owner, repo, etc.):
curl -i -H 'Authorization: token xxxxxxxxxx' -XPUT https://api.github.com/repos/<owner>/<repo>/collaborators/<username>
I also tried it with this url:
curl -i -H 'Authorization: token xxxxxxxxxx' -XPUT https://api.github.com/teams/<id>/members/<username>
As token I used a newly created Personal Access Tokens
But both times I get this back
HTTP/1.0 502 Bad Gateway
Cache-Control: no-cache
Connection: close
Content-Type: text/html
<html><body><h1>502 Bad Gateway</h1>
The server returned an invalid or incomplete response.
</body></html>
A GET on each URL works fine but a DELETE doesn't work either. So maybe it has to do with curl.
Quoting the reply from GitHub's support with changes in italic:
You're just getting trolled by HTTP and curl.
When you make a PUT request with no body, curl doesn't explicitly set a Content-Length header for that request. However, PUT requests with no Content-Length confuse servers and they respond in weird ways.
Can you please try explicitly setting the Content-Lenght header to 0, or supplying an empty body when making that request (so that curl can set the header for you)? You can accomplish that adding -d "" in your command.

Does Amazon S3 need time to update CORS settings? How long?

Recently I enabled Amazon S3 + CloudFront to serve as CDN for my rails application. In order to use font assets and display them in Firefox or IE, I have to enable CORS on my S3 bucket.
<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<CORSRule>
<AllowedOrigin>*</AllowedOrigin>
<AllowedMethod>GET</AllowedMethod>
<AllowedMethod>POST</AllowedMethod>
<AllowedMethod>PUT</AllowedMethod>
<MaxAgeSeconds>3000</MaxAgeSeconds>
<AllowedHeader>*</AllowedHeader>
</CORSRule>
</CORSConfiguration>
Then I used curl -I https://small-read-staging-assets.s3.amazonaws.com/staging/assets/settings_settings-312b7230872a71a534812e770ec299bb.js.gz, I got:
HTTP/1.1 200 OK
x-amz-id-2: Ovs0D578kzW1J72ej0duCi17lnw+wZryGeTw722V2XOteXOC4RoThU8t+NcXksCb
x-amz-request-id: 52E934392E32679A
Date: Tue, 04 Jun 2013 02:34:50 GMT
Cache-Control: public, max-age=31557600
Content-Encoding: gzip
Expires: Wed, 04 Jun 2014 08:16:26 GMT
Last-Modified: Tue, 04 Jun 2013 02:16:26 GMT
ETag: "723791e0c993b691c442970e9718d001"
Accept-Ranges: bytes
Content-Type: text/javascript
Content-Length: 39140
Server: AmazonS3
Should I see 'Access-Control-Allow-Origin' some where? Does S3 take time to update CORS settings? Can I force expiring headers if its caching them?
Try sending the Origin header:
$ curl -v -H "Origin: http://example.com" -X GET https://small-read-staging-assets.s3.amazonaws.com/staging/assets/settings_settings-312b7230872a71a534812e770ec299bb.js.gz > /dev/null
The output should then show the CORS response headers you are looking for:
< Access-Control-Allow-Origin: http://example.com
< Access-Control-Allow-Methods: GET
< Access-Control-Allow-Credentials: true
< Vary: Origin, Access-Control-Request-Headers, Access-Control-Request-Method
Additional information about how to debug CORS requests with cURL can be found here:
How can you debug a CORS request with cURL?
Note that there are different types of CORS requests (simple and preflight), a nice tutorial about the differences can be found here:
http://www.html5rocks.com/en/tutorials/cors/
Hope this helps!
Try these:
Try to scope-down the domain names you want to allow access to. S3 doesn't like *.
CloudFront + S3 doesn't handle the CORS configuration correctly out of the box. A kludge is to append a query string containing the name of the referring domain, and explicitly enable support for query strings in your CloudFront distribution settings.
To answer the actual question in the title:
No, S3 does not seem to take any time to propagate the CORS settings. (as of 2019)
However, if you're using Chrome (and maybe others), then CORS settings may be cached by the browser so you won't necessarily see the changes you expect if you just do an ordinary browser refresh. Instead right click on the refresh button and choose "Empty Cache and Hard Reload" (as of Chrome 73). Then the new CORS settings will take effect within <~5 seconds of making the change in the AWS console. (It may be much faster than that. Haven't tested.) This applies to a plain S3 bucket. I don't know how CloudFront affects things.
(I realize this question is 6 years old and may have involved additional technical issues that other people have long since answered, but when you search for the simple question of propagation times for CORS changes, this question is what pops up first, so I think it deserves an answer that addresses that.)
You have a few problems with the way you test CORS.
Your CORS configuration does not have a HEAD method.
Your curl command does not have -H header.
I am able to get your data by using curl like following. However they dumped garbage on my screen because your data is compressed binary.
curl --request GET https://small-read-staging-assets.s3.amazonaws.com/staging/assets/settings_settings-312b7230872a71a534812e770ec299bb.js.gz -H "http://google.com"