How to download AWS S3 object with no CORS [duplicate] - amazon-s3

Question

I'm trying to fetch some data from the REST API of HP Alm. It works pretty well with a small curl script—I get my data.
Now doing that with JavaScript, fetch and ES6 (more or less) seems to be a bigger issue. I keep getting this error message:
Fetch API cannot load . Response to preflight request doesn't
pass access control check: No 'Access-Control-Allow-Origin' header is
present on the requested resource. Origin 'http://127.0.0.1:3000' is
therefore not allowed access. The response had HTTP status code 501.
If an opaque response serves your needs, set the request's mode to
'no-cors' to fetch the resource with CORS disabled.
I understand that this is because I am trying to fetch that data from within my localhost and the solution should be using Cross-Origin Resource Sharing (CORS). I thought I actually did that, but somehow it either ignores what I write in the header or the problem is something else.
So, is there an implementation issue? Am I doing it wrong? I can't check the server logs unfortunately. I'm really a bit stuck here.
function performSignIn() {
let headers = new Headers();
headers.append('Content-Type', 'application/json');
headers.append('Accept', 'application/json');
headers.append('Access-Control-Allow-Origin', 'http://localhost:3000');
headers.append('Access-Control-Allow-Credentials', 'true');
headers.append('GET', 'POST', 'OPTIONS');
headers.append('Authorization', 'Basic ' + base64.encode(username + ":" + password));
fetch(sign_in, {
//mode: 'no-cors',
credentials: 'include',
method: 'POST',
headers: headers
})
.then(response => response.json())
.then(json => console.log(json))
.catch(error => console.log('Authorization failed : ' + error.message));
}
I am using Chrome. I also tried using that Chrome CORS Plugin, but then I am getting another error message:
The value of the 'Access-Control-Allow-Origin' header in the response
must not be the wildcard '*' when the request's credentials mode is
'include'. Origin 'http://127.0.0.1:3000' is therefore not allowed
access. The credentials mode of requests initiated by the
XMLHttpRequest is controlled by the withCredentials attribute.

Answer 1

This answer covers a lot of ground, so it’s divided into three parts:
How to use a CORS proxy to avoid “No Access-Control-Allow-Origin header” problems
How to avoid the CORS preflight
How to fix “Access-Control-Allow-Origin header must not be the wildcard” problems
How to use a CORS proxy to avoid “No Access-Control-Allow-Origin header” problems
If you don’t control the server your frontend code is sending a request to, and the problem with the response from that server is just the lack of the necessary Access-Control-Allow-Origin header, you can still get things to work—by making the request through a CORS proxy.
You can easily run your own proxy with code from https://github.com/Rob--W/cors-anywhere/.
You can also easily deploy your own proxy to Heroku in just 2-3 minutes, with 5 commands:
git clone https://github.com/Rob--W/cors-anywhere.git
cd cors-anywhere/
npm install
heroku create
git push heroku master
After running those commands, you’ll end up with your own CORS Anywhere server running at, e.g., https://cryptic-headland-94862.herokuapp.com/.
Now, prefix your request URL with the URL for your proxy:
https://cryptic-headland-94862.herokuapp.com/https://example.com
Adding the proxy URL as a prefix causes the request to get made through your proxy, which:
Forwards the request to https://example.com.
Receives the response from https://example.com.
Adds the Access-Control-Allow-Origin header to the response.
Passes that response, with that added header, back to the requesting frontend code.
The browser then allows the frontend code to access the response, because that response with the Access-Control-Allow-Origin response header is what the browser sees.
This works even if the request is one that triggers browsers to do a CORS preflight OPTIONS request, because in that case, the proxy also sends the Access-Control-Allow-Headers and Access-Control-Allow-Methods headers needed to make the preflight succeed.
How to avoid the CORS preflight
The code in the question triggers a CORS preflight—since it sends an Authorization header.
https://developer.mozilla.org/docs/Web/HTTP/Access_control_CORS#Preflighted_requests
Even without that, the Content-Type: application/json header will also trigger a preflight.
What “preflight” means: before the browser tries the POST in the code in the question, it first sends an OPTIONS request to the server, to determine if the server is opting-in to receiving a cross-origin POST that has Authorization and Content-Type: application/json headers.
It works pretty well with a small curl script - I get my data.
To properly test with curl, you must emulate the preflight OPTIONS the browser sends:
curl -i -X OPTIONS -H "Origin: http://127.0.0.1:3000" \
-H 'Access-Control-Request-Method: POST' \
-H 'Access-Control-Request-Headers: Content-Type, Authorization' \
"https://the.sign_in.url"
…with https://the.sign_in.url replaced by whatever your actual sign_in URL is.
The response the browser needs from that OPTIONS request must have headers like this:
Access-Control-Allow-Origin: http://127.0.0.1:3000
Access-Control-Allow-Methods: POST
Access-Control-Allow-Headers: Content-Type, Authorization
If the OPTIONS response doesn’t include those headers, the browser will stop right there and never attempt to send the POST request. Also, the HTTP status code for the response must be a 2xx—typically 200 or 204. If it’s any other status code, the browser will stop right there.
The server in the question responds to the OPTIONS request with a 501 status code, which apparently means it’s trying to indicate it doesn’t implement support for OPTIONS requests. Other servers typically respond with a 405 “Method not allowed” status code in this case.
So you’ll never be able to make POST requests directly to that server from your frontend JavaScript code if the server responds to that OPTIONS request with a 405 or 501 or anything other than a 200 or 204 or if doesn’t respond with those necessary response headers.
The way to avoid triggering a preflight for the case in the question would be:
if the server didn’t require an Authorization request header but instead, e.g., relied on authentication data embedded in the body of the POST request or as a query param
if the server didn’t require the POST body to have a Content-Type: application/json media type but instead accepted the POST body as application/x-www-form-urlencoded with a parameter named json (or whatever) whose value is the JSON data
How to fix “Access-Control-Allow-Origin header must not be the wildcard” problems
I am getting another error message:
The value of the 'Access-Control-Allow-Origin' header in the response
must not be the wildcard '*' when the request's credentials mode is
'include'. Origin 'http://127.0.0.1:3000' is therefore not allowed
access. The credentials mode of requests initiated by the
XMLHttpRequest is controlled by the withCredentials attribute.
For requests that have credentials, browsers won’t let your frontend JavaScript code access the response if the value of the Access-Control-Allow-Origin header is *. Instead the value in that case must exactly match your frontend code’s origin, http://127.0.0.1:3000.
See Credentialed requests and wildcards in the MDN HTTP access control (CORS) article.
If you control the server you’re sending the request to, a common way to deal with this case is to configure the server to take the value of the Origin request header, and echo/reflect that back into the value of the Access-Control-Allow-Origin response header; e.g., with nginx:
add_header Access-Control-Allow-Origin $http_origin
But that’s just an example; other (web) server systems have similar ways to echo origin values.
I am using Chrome. I also tried using that Chrome CORS Plugin
That Chrome CORS plugin apparently just simplemindedly injects an Access-Control-Allow-Origin: * header into the response the browser sees. If the plugin were smarter, what it would be doing is setting the value of that fake Access-Control-Allow-Origin response header to the actual origin of your frontend JavaScript code, http://127.0.0.1:3000.
So avoid using that plugin, even for testing. It’s just a distraction. To test what responses you get from the server with no browser filtering them, you’re better off using curl -H as above.
As far as the frontend JavaScript code for the fetch(…) request in the question:
headers.append('Access-Control-Allow-Origin', 'http://localhost:3000');
headers.append('Access-Control-Allow-Credentials', 'true');
Remove those lines. The Access-Control-Allow-* headers are response headers. You never want to send them in requests. The only effect of that is to trigger a browser to do a preflight.

Answer 2

This error occurs when the client URL and server URL don't match, including the port number. In this case you need to enable your service for CORS which is cross origin resource sharing.
If you are hosting a Spring REST service then you can find it in the blog post CORS support in Spring Framework.
If you are hosting service using a Node.js server then
Stop the Node.js server.
npm install cors --save
Add following lines to your server.js
const cors=require("cors");
const corsOptions ={
origin:'*',
credentials:true, //access-control-allow-credentials:true
optionSuccessStatus:200,
}
app.use(cors(corsOptions)) // Use this after the variable declaration

Answer 3

The problem arose because you added the following code as the request header in your front-end:
headers.append('Access-Control-Allow-Origin', 'http://localhost:3000');
headers.append('Access-Control-Allow-Credentials', 'true');
Those headers belong to the response, not request. So remove them, including the line:
headers.append('GET', 'POST', 'OPTIONS');
Your request had 'Content-Type: application/json', hence triggered what is called CORS preflight. This caused the browser sent the request with the OPTIONS method. See CORS preflight for detailed information.
Therefore in your back-end, you have to handle this preflighted request by returning the response headers which include:
Access-Control-Allow-Origin : http://localhost:3000
Access-Control-Allow-Credentials : true
Access-Control-Allow-Methods : GET, POST, OPTIONS
Access-Control-Allow-Headers : Origin, Content-Type, Accept
Of course, the actual syntax depends on the programming language you use for your back-end.
In your front-end, it should be like so:
function performSignIn() {
let headers = new Headers();
headers.append('Content-Type', 'application/json');
headers.append('Accept', 'application/json');
headers.append('Authorization', 'Basic ' + base64.encode(username + ":" + password));
headers.append('Origin','http://localhost:3000');
fetch(sign_in, {
mode: 'cors',
credentials: 'include',
method: 'POST',
headers: headers
})
.then(response => response.json())
.then(json => console.log(json))
.catch(error => console.log('Authorization failed: ' + error.message));
}

Answer 4

In my case, I use the below solution.
Front-end or Angular
post(
this.serverUrl, dataObjToPost,
{
headers: new HttpHeaders({
'Content-Type': 'application/json',
})
}
)
back-end (I use PHP)
header("Access-Control-Allow-Origin: http://localhost:4200");
header('Access-Control-Allow-Methods: GET, POST, OPTIONS');
header("Access-Control-Allow-Headers: Content-Type, Authorization");
$postdata = file_get_contents("php://input");
$request = json_decode($postdata);
print_r($request);

Answer 5

Using dataType: 'jsonp' worked for me.
async function get_ajax_data(){
var _reprojected_lat_lng = await $.ajax({
type: 'GET',
dataType: 'jsonp',
data: {},
url: _reprojection_url,
error: function (jqXHR, textStatus, errorThrown) {
console.log(jqXHR)
},
success: function (data) {
console.log(data);
// note: data is already json type, you
// just specify dataType: jsonp
return data;
}
});
} // function

Answer 6

Just my two cents... regarding How to use a CORS proxy to get around “No Access-Control-Allow-Origin header” problems
For those of you working with php at the backend, deploying a "CORS proxy" is as simple as:
create a file named 'no-cors.php' with the following content:
$URL = $_GET['url'];
echo json_encode(file_get_contents($URL));
die();
on your front end, do something like:
fetch('https://example.com/no-cors.php' + '?url=' + url)
.then(response=>{*/Handle Response/*})`

Answer 7

If your API is written in ASP.NET Core, then please follow the below steps:
Install the Microsoft.AspNetCore.Cors package.
Add the below line in the ConfigureServices method in file Startup.cs:
services.AddCors();
Add the below line in the Configure method in file startup.cs:
app.UseCors(options =>
options.WithOrigins("http://localhost:8080")
.AllowAnyHeader()
.AllowAnyMethod());
Make sure you add this after - app.UseRouting();
Refer to the below image(from MSDN) to see the middleware order:
https://i.stack.imgur.com/vQ4yT.png

Answer 8

Possible causes of CORS issues
Check your server-side access headers: Refer to this link
Check what request header is received from the server in the browser. The below image shows the headers
If you are using the fetch method and trying to access the cross-origin request make sure mode:cors is there. Refer to this link
Sometimes if there is an issue in the program also you are getting the CORS issue, so make sure your code is working properly.
Make sure to handle the OPTION method in your API.

Answer 9

Adding mode:no-cors can avoid CORS issues in the API.
fetch(sign_in, {
mode: 'no-cors',
credentials: 'include',
method: 'POST',
headers: headers
})
.then(response => response.json())
.then(json => console.log(json))
.catch(error => console.log('Authorization failed : ' + error.message));
}

Answer 10

In December 2021, Chrome 97, the Authorization: Bearer ... is not allowed unless it is in the Access-Control-Allow-Headers preflight response (ignores *). It produced this warning:
[Deprecation] authorization will not be covered by the wildcard symbol (*)
See: Chrome Enterprise release notes, Chrome 97
It also appears to enforce the same restriction on * on Access-Control-Allow-Origin. If you want to revive *-like behavior now that it is blocked, you'll likely have to read the requester's origin and return it as the allowed origin in the preflight response.
In some cases, a library may drop the Access-Control-Allow-Origin response header when there is some other invalid credential (example: an expired JWT). Then, the browser shows the "No 'Access-Control-Allow-Origin' header is present" error instead of the actual error (which in this example could be an expired JWT). Be sure that your library doesn't drop the header and confuse the client.

Answer 11

Faced this issue in my react/express app. Adding the below code in server.js (or your server file name) fixed the issue for me. Install cors and then
const cors = require('cors');
app.use(cors({
origin: 'http://example.com', // use your actual domain name (or localhost), using * is not recommended
methods: ['GET', 'POST', 'PUT', 'DELETE', 'PATCH', 'HEAD', 'OPTIONS'],
allowedHeaders: ['Content-Type', 'Origin', 'X-Requested-With', 'Accept', 'x-client-key', 'x-client-token', 'x-client-secret', 'Authorization'],
credentials: true
}))
Now you can make straightforward API calls from your front-end without having to pass any additional parameters.

Answer 12

With Node.js, if you are using routers, make sure to add CORS before the routers. Otherwise, you'll still get the CORS error. Like below:
const cors = require('cors');
const userRouter = require('./routers/user');
expressApp = express();
expressApp.use(cors());
expressApp.use(express.json());
expressApp.use(userRouter);

Answer 13

In case you are using Node.js and Express.js as the back-end and React & Axios as the front-end within a development environment in macOS, you need to run both sides under HTTPS. Below is what finally worked for me (after many hours of deep dive and testing):
Step 1: Create an SSL certificate
Just follow the steps from How to get HTTPS working on your local development environment in 5 minutes.
You will end up with a couple of files to be used as credentials to run the HTTPS server and React web:
server.key & server.crt
You need to copy them in the root folders of both the front and back ends (in a production environment, you might consider copying them in folder ./ssh for the back-end).
Step 2: Back-end setup
I read a lot of answers proposing the use of 'cors' package or even setting ('Access-Control-Allow-Origin', '*'), which is like saying: "Hackers are welcome to my website". Just do like this:
import express from 'express';
const emailRouter = require('./routes/email'); // in my case, I was sending an email through a form in React
const fs = require('fs');
const https = require('https');
const app = express();
const port = 8000;
// CORS (Cross-Origin Resource Sharing) headers to support Cross-site HTTP requests
app.all('*', (req, res, next) => {
res.header("Access-Control-Allow-Origin", "https://localhost:3000");
next();
});
// Routes definition
app.use('/email', emailRouter);
// HTTPS server
const credentials = {
key: fs.readFileSync('server.key'),
cert: fs.readFileSync('server.crt')
};
const httpsServer = https.createServer(credentials, app);
httpsServer.listen(port, () => {
console.log(`Back-end running on port ${port}`);
});
In case you want to test if the https is OK, you can replace the httpsServer constant by the one below:
https.createServer(credentials, (req: any, res: any) => {
res.writeHead(200);
res.end("hello world from SSL\n");
}).listen(port, () => {
console.log(`HTTPS server listening on port ${port}...`);
});
And then access it from a web browser: https://localhost:8000/
Step 3: Front-end setup
This is the Axios request from the React front-end:
await axios.get(`https://localhost:8000/email/send`, {
params: { /* Whatever data you want to send */ },
headers: {
'Content-Type': 'application/json',
}
})
And now, you need to launch your React web in HTTPS mode using the credentials for SSL we already created. Type this in your macOS terminal:
HTTPS=true SSL_CRT_FILE=server.crt SSL_KEY_FILE=server.key npm start
At this point, you are sending a request from an HTTPS connection at port 3000 from your front-end, to be received by an HTTPS connection at port 8000 by your back-end. CORS should be happy with this ;)

Answer 14

For those using ASP.NET Core:
In my case, I was using JavaScript to make a blob from an image stored on the API (the server), so the URL was pointing to that resource. In that API's program.cs class, I already had a CORS policy, but it didn't work.
After I read the Microsoft documentation (read the first paragraph) about this issue, it is said that if you want to access a resource on the server, by using JavaScript (which is what I was trying to do), then you must call the app.UseCors(); before the app.UseStaticFiles(); which is typically the opposite.
My program.cs file:
const string corsPolicyName = "ApiCORS";
builder.Services.AddCors(options =>
{
options.AddPolicy(corsPolicyName, policy =>
{
policy.WithOrigins("https://localhost:7212");
});
});
...
var app = builder.Build();
app.UseSwagger();
app.UseSwaggerUI(settings =>
{
settings.DisplayRequestDuration();
settings.EnableTryItOutByDefault();
});
app.UseHttpsRedirection();
app.UseCors(corsPolicyName); // 👈 This should be above the UseStaticFiles();
app.UseStaticFiles(); // 👈 Below the UseCors();
app.UseAuthentication();
app.UseAuthorization();
app.UseApiCustomExceptionHandler();
app.MapControllers();
app.Run();

Answer 15

Remove this:
credentials: 'include',

Answer 16

For a Node.js and Express.js backend I use this :)
app.use(function(req, res, next) {
res.header("Access-Control-Allow-Origin", "YOUR-DOMAIN.TLD"); // Update to match the domain you will make the request from
res.header("Access-Control-Allow-Headers", "Origin, X-Requested-With, Content-Type, Accept");
next();
});
For more details: CORS on ExpressJS

Answer 17

I have encountered this error several times over the past few years -- seemingly showing up out of the blue in a previously functioning website.
I determined that Chrome (and possibly other browsers) can return this error when there is some unrelated error that occurs on the server that prevents it from processing the CORS request (and prior to returning an HTTP 500 error).
These all occurred in a .NET Core environment, and I am not sure if it would happen in other environments.
Anyway, if your code has functioned before, and seems correct, consider debugging to find if there is some other error that is firing before you go crazy trying to solve an error that isn't really there.

Answer 18

In my case, the web server prevented the "OPTIONS" method
Check your web server for the options method
Apache: https://www-01.ibm.com/support/docview.wss?uid=ibm10735209
web tier: 4.4.6 Disabling the Options Method https://docs.oracle.com/cd/E23943_01/web.1111/e10144/getstart.htm#HSADM174
nginx: https://medium.com/#hariomvashisth/cors-on-nginx-be38dd0e19df
I'm using "webtier"
/www/webtier/domains/[domainname]/config/fmwconfig/components/OHS/VCWeb1/httpd.conf
<IfModule mod_rewrite.c>
RewriteEngine on
RewriteCond %{REQUEST_METHOD} ^OPTIONS
RewriteRule .* . [F]
</IfModule>
change to
<IfModule mod_rewrite.c>
RewriteEngine off
RewriteCond %{REQUEST_METHOD} ^OPTIONS
RewriteRule .* . [F]
</IfModule>

Answer 19

In my case, the solution was dumb as hell... Your allowed origin shouldn't have a slash at the end.
E.g., https://example.com/ -> https://example.com

Answer 20

In my case, I had to add a custom header middleware below all the existing middleware. I think some middleware might conflict with the Access-Control-Allow-Origin Header and try to set it according to their needs.
So the code would be something like this:
app.use(cors());
....all other middleware here
app.use(function (req, res, next) {
res.header("Access-Control-Allow-Origin", "http://localhost:3000");
res.header("Access-Control-Allow-Headers", "Origin, X-Requested-With, Content-Type, Accept");
next();
});
...your routes

Answer 21

I make this mistake a lot of times, and because of it, I've made a "check-list" to all of you.
Enable CORS on your project: If you're using Node.js (by example) you can use:
npm install cors;
import cors from 'cors';
app.use(cors());
You can manually set the headers like this (if you want it):
app.use((req, res, next) => {
res.setHeader('Access-Control-Allow-Origin', '*');
res.setHeader('Access-Control-Allow-Headers', 'Origin, X-Requested-With, Content-Type, Accept, Authortization');
res.setHeader('Acces-Control-Allow-Methods', 'GET, POST, PATCH, DELETE');
Remember to add http:// to your API link in your frontend project, some browsers like Chrome do not accept a request using CORS if the request URL isn't HTTP or HTTPS:
http://localhost:3000/api
Check if your project is using a proxy.config.js file. See Fixing CORS errors with Angular CLI proxy.

Answer 22

When the client used to call our backend service from his host username.companyname.com, he used to get the above error
Two things are required:
while sending back the response, send the header whose key is Access-Control-Allow-Origin and value is *:
context.Writer.Header()["Access-Control-Allow-Origin"] = []string{"*"} // Important to avoid a CORS error
Use the Go CORS library to set AllowCredentials to false and AllowAllOrigins to true.

Answer 23

Use the below npm module. This has virtually saved lives.
https://www.npmjs.com/package/local-cors-proxy
You're getting a CORS error, for example like the below URL
https://www.google.co.in/search/list
After successfully installed(local-cors-proxy) global npm install -g local-cors-proxy and set proxy URL that CORS URL.
For example, here the below CORS issue getting in localhost. So you need to add the domain name(https://www.google.co.in) and port(--port 8010) for the CORS issue domain.
For more please check the link
https://www.npmjs.com/package/local-cors-proxy
lcp --proxyUrl https://www.google.co.in --port 8010
After successfully set, it will generate the local proxy URL like below.
http://localhost:8010/proxy
Use that domain name in your project API URL.
API full URL:
http://localhost:8010/proxy/search/list
To get without a CORS issue response in your local project.

Answer 24

Using WebAPI build in .Net Core 6.0
None of the above worked for me... This did it
// global cors policy
app.UseCors(x => x
.AllowAnyMethod()
.AllowAnyHeader()
.SetIsOriginAllowed(origin => true) // allow any origin
.AllowCredentials());
credit: https://stackoverflow.com/a/70660054/8767516

Answer 25

Try adding all these headers in this code below Before every route, you define in your app, not after the routes
app.use((req, res, next) =>{
res.setHeader('Access-Control-Allow-Origin', '*');
res.setHeader('Access-Control-Allow-Headers','Origin, X-Requested-With, Content-Type,Accept, Authortization');
res.setHeader('Acces-Control-Allow-Methods','GET, POST, PATCH, DELETE');

Answer 26

If you are getting this error while deploying React app to netlify, use these steps.
step 1: Create netlify.toml file in the root folder of your react app.
step 2: Copy paste this code:
`[[redirects]]
from = "/cors-proxy/*"
to = ":splat"
status = 200
force = true`
step3: update your fetch/axios api this way:
It took me a while to figure this out.