I have an express server that creates requests to another server, which returns data as text.
For most get-requests I get the complete data as text (expected). But for some other requests
I get binary data and in the response header has a property "transfer-encoding":"chunked".
It seems the data comes in chunks.
Why is axios not handling chunked data communication out of the box. Do I have to set something to get it working? Or do I have to code the handle the chunked data. If yes, is there a sample how to do this?
My code in my express server "proxy.ts":
const app: express.Application = express();
app.get('/wrappedrequest', (req: Request, res: Response) => {
var hostname="http://myhost/myrequest:myport"
const endpoint = hostname + req.originalUrl
axios.get(endpoint).then(response => {
console.debug("response headers: " + JSON.stringify(response.headers))
res.send(response.data)
}).catch(error => {
res.send('error: ' + error)
})
})
app.listen(3001, () => console.log('Server running'))
In my console I can see "transfer-encoding":"chunked":
{"server":"Apache-Coyote/1.1","last-modified":"Wed, 30 Nov 2022 07:42:24 GMT","cache-control":"no-store, no-cache, must-revalidate, pre-check=0,
post-check=0, max-age=0","pragma":"no-cache","expires":"Thu, 01 Jan 1970 00:00:00 GMT","content-type":"text/xml",
"transfer-encoding":"chunked","date":"Wed, 30 Nov 2022 07:42:24 GMT","connection":"close"}
Any help how to fetch the data, when server responds with chunks?
Related
I realized that a response header would be present in the Network tab of the Chrome console even if that header wasn't set in express. I found this answer suggesting disallowing caching. What confuses me is why the cached response is still used even if a request is made to the server.
request from react
const baseURL = 'http://localhost:3001'
const axiosClient = axios.create({
baseURL,
withCredentials: true,
})
let accessToken
axiosClient.interceptors.response.use((response) => {
const { data, headers } = response
//store access token in memory
accessToken = headers['x-access-token']
console.log(accessToken)
// if (accessToken) axiosClient.defaults.headers['X-Access-Token'] = accessToken
return data
})
async me() {
return await axiosClient.get('/auth/me')
}
request reaches route
router.get('/me', (req, res) => {
// res.set('X-Access-Token', 'test 4')
res.send('me')
})
vscode debug console
res.getHeaders()
{x-powered-by: 'Express', access-control-allow-origin: 'http://localhost:3000', vary: 'Origin', access-control-allow-credentials: 'true', access-control-expose-headers: 'X-Access-Token'}
req.headers
{host: 'localhost:3001', connection: 'keep-alive', sec-ch-ua: '"Not?A_Brand";v="8", "Chromium";v="108", "Google Chrome";v="108"', accept: 'application/json, text/plain, */*', sec-ch-ua-mobile: '?0', …}
old token still appears in chrome
Since Chrome made a request to the server instead of just using the cached response (First, since there is no need to deliver the request to the origin server, then the closer the client and cache are, the faster the response will be), why isn't Chrome using the received response where the token header isn't present?
My API server doesn't change or overwrite the Access-Control-Allow-Origin header.
I have an Angular 2+ application with an Express/Swagger-UI API.
The Access-Control-Allow-Origin header in the API cannot be '*' because I use 'withCredentials' in my requests and it's trigger an error on the browser.
I've used 'cors' for setting the CORS headers. Like this:
'use strict';
var configuration = require('./configuration');
var SwaggerExpress = require('swagger-express-mw');
var app = require('express')();
var mongoose = require('mongoose');
var bluebird = require('bluebird');
var bodyParser = require('body-parser');
var jwt = require('jsonwebtoken');
var compression = require('compression');
var cors = require('cors');
app.use(bodyParser.json({ limit: '50mb' }));
app.use(compression());
SwaggerExpress.create(config, function(err, swaggerExpress) {
if (err) { throw err; }
swaggerExpress.register(app);
var port = process.env.PORT || 10010;
mongoose.Promise = bluebird;
mongoose.connect(appConfig.dbConection());
mongoose.connection.on('error', console.error.bind(console, 'connection error: '));
mongoose.set('debug', true);
mongoose.connection.once('open', function() {
app.listen(port);
})
});
const swaggerUi = require('swagger-ui-express');
const YAML = require('yamljs');
const swaggerDocument = YAML.load('./api/swagger/swagger.yaml');
app.use('/api-docs', swaggerUi.serve, swaggerUi.setup(swaggerDocument));
var allowedOrigins = ['http://localhost:4200', 'http://127.0.0.1:10010'];
app.use(cors({
origin: (origin, callback) => {
if (!origin) return callback(null, true);
if (allowedOrigins.indexOf(origin) === -1) {
return callback(new Error('The CORS policy for this site does not allow the specified Origin'), false);
}
return callback(null, true);
},
exposedHeaders: ['Origin', 'X-Requested-With', 'Content-Type', 'Accept', 'Authorization', 'api_key', 'x-api-key'],
credentials: true
}));
The Access-Control-Allow-Origin should match the array of allowed hosts.
When I call the API from the browser, it first make an OPTIONS request, that look correct, like this:
Request URL: http://localhost:10010/v1/loginApp
Request Method: OPTIONS
Status Code: 204 No Content
Remote Address: [::1]:10010
Referrer Policy: no-referrer-when-downgrade
Access-Control-Allow-Credentials: true
Access-Control-Allow-Headers: x-api-key
Access-Control-Allow-Methods: GET,HEAD,PUT,PATCH,POST,DELETE
Access-Control-Allow-Origin: http://localhost:4200
Access-Control-Expose-Headers: Origin,X-Requested-With,Content-Type,Accept,Authorization,api_key,x-api-key
Connection: keep-alive
Content-Length: 0
Date: Thu, 04 Apr 2019 08:25:53 GMT
Vary: Origin, Access-Control-Request-Headers
X-Powered-By: Express
But then it makes the GET/POST request, and then it has the default headers again:
Request URL: http://localhost:10010/v1/loginApp
Request Method: GET
Status Code: 200 OK
Remote Address: [::1]:10010
Referrer Policy: no-referrer-when-downgrade
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: Origin,X-Requested-With,Content-Type,Accept,Authorization,api_key,x-api-key
Content-Length: 770
Content-Type: application/json; charset=utf-8
Date: Thu, 04 Apr 2019 08:25:53 GMT
ETag: W/"302-BwX5DjK/lUY+ueP6UZ9TgWOTw/s"
Vary: Origin
X-Powered-By: Express
I think something (probably swagger-UI) is overwriting my headers configuration, but I can't figure it out.
You need to register your CORS middleware before the Swagger GET/POST handlers.
The OPTIONS request is succeeding because Swagger doesn't register an OPTIONS handler, so the request reaches the middleware.
The GET/POST requests are failing before the Swagger handlers accept the request before it reaches the CORS middleware.
Move app.use(cors({... so it is before app.use('/api-docs', swaggerUi....
It seems as if swagger-express-mw comes with CORS support built-in, see swagger_controllers in your config/default.yaml. I had the same issue and fixed it by removing cors from there.
Like you, I suspect it's overwriting the Access-Control-Allow-Origin header but didn't have time to check it properly.
I have an Ionic 4 app that uses a lambda API hosted on AWS. CORS is enabled on the API Gateway. The following snippet is from a curl request to the API.
< content-type: application/json
< content-length: 42
< date: Sat, 16 Feb 2019 02:19:25 GMT
< x-amzn-requestid: 47a5fcac-3191-11e9-af42-d387861aa6ad
< access-control-allow-origin: *
< access-control-allow-headers: Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token
< x-amz-apigw-id: VK7vFGc4oAMFTqg=
< access-control-allow-methods: POST,OPTIONS
This post discusses a few possible workarounds (change content type, etc.), but they don't work.
Changing the Content-Type header to text/plain or removing that header altogether makes no difference.
The following error is also presented on the Ionic console
Cross-Origin Read Blocking (CORB) blocked cross-origin response
https://mycoolapi.com/GetLegal with MIME type application/json.
See https://www.chromestatus.com/feature/5629709824032768 for more details.
The following is my service code.
getLegal(data: any) {
return new Promise((resolve, reject) => {
let httpHeaders = new HttpHeaders().set('Content-Type', 'application/json');
this.httpClient.post(this.apiUrl+'/GetLegal', JSON.stringify(data), {
headers: httpHeaders,
})
.subscribe(res => {
resolve(new LegalResponse(res));
}, (err) => {
console.log("Oops, there has been an error")
reject(err);
});
});
}
Help?
This ended up being a bug on the Amazon side. The curl snippet was from a GET method, which was sending the CORS headers. The POST method was not. After redeploying the API without changing anything, the GET method was no longer sending the CORS headers and the POST method was. The application is working, for now.
I'm trying to authenticate express API back-end using Axios HTTP request call.
I was able to see 'Set-Cookie' in the response header, but cookie was not set. Is it possible to set cookies through Axios HTTP calls?
Access-Control-Allow-Origin: *
Connection: keep-alive
Content-Length: 355
Content-Type: application/json; charset=utf-8
Date: Fri, 28 Sep 2018 05:59:01 GMT
ETag: W/"163-PAMc87SVHWkdimTJca7oRw"
Set-Cookie: token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...; Max-Age=3.6; Path=/; Expires=Fri, 28 Sep 2018 05:59:04 GMT; HttpOnly
X-Powered-By: Express
Try this out!
axios.get('your_url', {withCredentials: true}); //for GET
axios.post('your_url', data, {withCredentials: true}); //for POST
axios.put('your_url', data, {withCredentials: true}); //for PUT
axios.delete('your_url', data, {withCredentials: true}); //for DELETE
For more information on this from the axios docs:
"withCredentials indicates whether or not cross-site Access-Control requests should be made using credentials" - https://github.com/axios/axios
More detail on withCredentials:
https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequest/withCredentials
Yes you can set cookies by Axios. The cookies need to be passed into the headers object. You can send cookies in a get/post/put/delete/etc. request:
As suggested by Aaron:
axios.get('URL', {
withCredentials: true
});
axios.post('URL', data, {
withCredentials: true
});
axios.put('URL', data, {
withCredentials: true
});
axios.delete('URL', data, {
withCredentials: true
});
Or you may also try this:
axios.get(url, {
headers: {
Cookie: "cookie1=value; cookie2=value; cookie3=value;"
}
}).then(response => {
console.log(response);
});
In case anyone else faces the problem I've had,
Here's a repost of my answer on a similar question https://stackoverflow.com/a/62821342/8479303
In my case, the network panel showed that the response had the 'Set-Cookie' header, but in axios the header wouldn't show up, and the cookie was being set.
For me, the resolution was setting the Access-Control-Expose-Headers header.
For explanation, from this comment on an issue in the axios repository I was directed to this person's notes which led me to set the Access-Control-Expose-Headers header -- and now the cookie is properly setting in the client.
So, in Express.js, I had to add the exposedHeaders option to my cors middleware:
const corsOptions = {
//To allow requests from client
origin: [
"http://localhost:3001",
"http://127.0.0.1",
"http://104.142.122.231",
],
credentials: true,
exposedHeaders: ["set-cookie"],
};
...
app.use("/", cors(corsOptions), router);
It was also important that on the axios side I use the withCredentials config in following axios requests that I wanted to include the cookies.
ex/
const { data } = await api.get("/workouts", { withCredentials: true });
I tried setting withCredentials: true but was still getting this error:
Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at http://localhost:4000/users/register. (Reason: CORS request did not succeed).
CORS was configured to allow requests from the frontend port.
I had to change the default options for axios like so:
axios.defaults.withCredentials = true
And the issue was solved. No error and Set-Cookie working as expected.
cookie can't be touched, the thing is it gets bundled to request object after appended to the response object.
function sign(req,res){
res.cookie("x-token", signed, { maxAge: (new JWTService().jwtExpirySeconds *
1000) });
}
client after receiving this response just have to continue with requests, set-cookie in the name of "Cookie " will be bundled to those request, like this
caveat: when http cookie expires its is automatically removed and not bundled to request there after.
My middleware code:
app.use(cors());
app.use(function (req, res, next) {
res.header("Access-Control-Allow-Origin", "*");
res.header("Access-Control-Allow-Methods", "GET, PUT, POST, DELETE, OPTIONS");
res.header("Access-Control-Allow-Headers", "Origin, X-Requested-With, Content-Type, Accept, Authorization");
if (req.method === 'OPTIONS') {
res.send(200);
} else {
next();
}
});
Every time that I've found someone with the same error message, the response that rectifies their problem involves telling them to set "Access-Control-Allow-Methods" just like I have it.
My delete method on the front end:
deleteSaved(url) {
// return axios.delete("https://api.mlab.com/api/1/databases/etc.", {
// params: {
// "url": url
// }
// })
// .then(function(results) {
// console.log("axios results", results);
// return results;
// });
return fetch("/saved", {
method: 'DELETE',
body: JSON.stringify({url: url}),
mode: 'cors',
cache: 'default'
}).then(function(response) {
return response;
}).then(function(data){
console.log(data);
return data;
});
}
I have both fetch and axios there because I thought I was having trouble with the axios delete method. In the development server, both calls work properly.
In the axios method, I have my db API URL for the sake of simplicity (to make it easier to get things to work); in the fetch method, I have '/saved' because when I asked this question last time, someone commented and suggested that I proxy all DB requests through my express server rather than having the browser connect directly. I tried a few things (setting "proxy" and "https_proxy" to my mlab api url in the package.json associated with the create-react-app front end), but I don't think I really understand what that means. The proxy didn't work (the "/saved" would result in an error for a GET request at the apps URL + /saved rather than for the mlab API url) and I simply can't find anything on google about proxying requests to the DB through the express server.
My response headers:
Access-Control-Allow-Credentials:true
Access-Control-Allow-Methods:POST,GET,OPTIONS,PUT
Access-Control-Allow-Origin:https://nyt-mern-app.herokuapp.com
Access-Control-Max-Age:1728000
Allow:POST,GET,OPTIONS,PUT
Cache-Control:no-store, no-cache, must-revalidate, max-age=0, post-check=0, pre-check=0
Connection:Keep-Alive
Content-Length:0
Content-Type:text/html;charset=UTF-8
Date:Thu, 26 Oct 2017 19:34:08 GMT
Expires:Tue, 01 Feb 2000 08:00:00 GMT
Keep-Alive:timeout=5, max=99
Last-Modified:Thu, 26 Oct 2017 19:34:08 GMT
Pragma:no-cache
Server:Apache/2.4.7 (Ubuntu)
X-Frame-Options:DENY
Like I said, everything works in a local development environment. How do I get this to work in deployment?