Express 4.17 req.body is being printed as buffer - express

Something that I wanted clarification on is when console.logging req.body in my express app. I end up with a buffer/string or hexadecimal of some sort when I'm expecting an JSON object. I'm using postman to send a raw json body.
Here are some visuals of the source code and terminal/postman results.
const express = require('express');
const bodyParser = require('body-parser');
const { randomBytes } = require('crypto');
const app = express();
app.use(express.raw({type: "application/json"}));
app.use(express.json({strict: false}));
app.use(express.urlencoded({extended: false}));
const posts = {};
app.get('/posts', (req, res) => {
res.send(posts);
});
app.post('/posts', (req, res) => {
const id = randomBytes(4).toString('hex');
const { title } = req.body;
console.log(req.body)
posts[id] = {
id,
title: title
};
res.status(201).send(posts[id]);
});
app.listen(4000, () => {
console.log('Listening on 4000');
});
Console.log terminal of buffer/hex/string
Postman body
Postman Raw String

app.use(express.raw({type: "application/json"})); is going to give you a Buffer object and since that's your first middleware that might handle that mime type, that's what you're going to get for any application/json request.
Right from the Express doc for express.raw():
This is a built-in middleware function in Express. It parses incoming
request payloads into a Buffer
It's unclear why you're using express.raw() as that is not typical for JSON payloads, but when doing so, you are going to get a Buffer - that's how it works. One would more typically use express.json() for JSON payloads and let it parse your JSON so that req.body contains an actual Javascript object.
If you remove the app.use(express.raw({type: "application/json"})); line of code and let the app.use(express.json()); line of code right after it handle the application/json payloads, then you will get your parsed data in req.body.
Keep in mind that when using middleware they are processed in the order declared and for this specific type of middleware, the first one that matches and reads the body from the incoming stream takes precedence and none of the others after it will be able to do their job (since the incoming stream has already been read).

Related

How to sign JSON responses in Express

I'm developing an Express.js server; since it manages sensible data, I'd like to add a cryptographic signature to every response, using a private key. The signature part is already OK, the problem is catching the JSON string just before is sent to the client to sign it.
Ideally I'd like to add a custom response header, e.g. X-signature, so clients could verify the received payload against the public key exposed by the service.
In Express, how can I intercept the response body after the JSON.stringify() call but before the headers are sent?
I've copied what is done in the compression() middleware, replacing the send() method of Response with my own and calling the original one after I calculate the signature. Here's a simplified solution just to show the concept, in the real word I've taken care of body's possible types. The fact that calculateSignature() is an async function is a little troublesome, but everything seems to work well now.
Any comment is appreciated!
import express, { RequestHandler } from 'express';
declare const calculateSignature = (payload: string) => Promise<string>;
const responseSignature = (): RequestHandler => (_, res, next) => {
const _send = res.send;
res.send = (body) => {
if (isPlaintextType(body)) {
calculateSignature(body)
.then((signature) => res.setHeader('X-Signature', signature))
.catch((e) => console.error(e))
.finally(() => _send.call(res, body));
return res;
} else {
return _send.call(res, body);
}
};
next();
};
const app = express();
app.use(responseSignature());
app.listen(3000);

Key passed but validation fails in postman

I have this patch request where I update certain information depending on object of a document. When I send request using Postman(form-data) I get validation error saying "_id" is required.
Here's how it looks in Postman,
The route looks similar to this,
router.patch("/update", async (req, res) => {
try{
await updateValidation(req.body);
// whatever stuff processed with the data
} catch (err) {
res.status(400).send({ message: err.details[0].message });
}
}
The validation function looks like this,
const updateValidation = (data) => {
const schema = Joi.object({
_id: Joi.string().required(),
// other whatever validation possible
});
return schema.validateAsync(data);
};
Am I missing something here? I think I do, please point it out.
Besides using var bodyParser = require('body-parser'); in server side, it will still return empty req.body that will lead to errors because you have validation in place. The reason it is returning empty req.body when you sending PATCH request using form-data in Postman is because body-parser can't handle multipart/form-data. You need a package that can handle multipart/form-data like multer.
at the first to install the body-parser and multer, go to your terminal and use −
npm install --save body-parser multer
so add this code to in server.js
var express = require('express');
var bodyParser = require('body-parser');
var multer = require('multer');
var upload = multer();
var app = express();
at the next time use this middelware:
// for parsing application/json
app.use(bodyParser.json());
// for parsing application/xwww-
app.use(bodyParser.urlencoded({ extended: true }));
//form-urlencoded
// for parsing multipart/form-data
app.use(upload.array());
app.use(express.static('public'));
After importing the body parser and multer, we will use the body-parser for parsing json and x-www-form-urlencoded header requests, while we will use multer for parsing multipart/form-data.

Cant get body from axios.delete server side

I am having trouble accessing the data I am sending with my axios.delete() function, to do things on the backend.
This is my axios call:
export const deleteBooking = (email, bookingObjId) => {
console.log("Parameters are both strings and received properly"
, email, bookingObjId)
const response = axios.delete(`/users`,
{data: {userId: email, bookingId: bookingObjId}})
return response
}
This is my code receiving it:
bookingRouter.delete('/', async(req, res) => {
body = req.body
console.log("body receives nothing, it's just {}", body)
var id = mongoose.Types.ObjectId(body.bookingId)
const deleted = await User.updateOne({email: body.userId},
{$pull: {'bookings': {'_id':id}}}, {safe: true, multi: true})
res.status(200).json(deleted)
})
I have tried req.params as well, and that doesn't work. This seems like a simple fix but I can't find it after looking for hours. I know it's not anything mongoose or react related though, it can't be.
Here is the solution I find right after posting this after searching for hours:
const response = axios.request({
method: 'delete',
url,
data
})
works, as per: https://github.com/axios/axios/issues/3220#issuecomment-688566578

Forward decoded POST data to other URL using Express

Some background information:
I have a temperature sensor which is connected via LoRaWAN.
The data from the sensor is encoded, but the manufacturer of the sensor provides a decoder (node app).
I want to show the data in Thingsboard.
Now I was thinking to build a simple express API (I'm new to Express) as a kind of middleware.
The only purpose is to receive the encoded data, decode it (output is in JSON format), and pass it on to Thingsboard.
With the last step I need some help. This is my route:
const decoder = require('../helpers/decoder');
module.exports = app => {
app.post('/', (req, res) => {
const data = {
type: req.body.productType,
payload: req.body.payloadValue
};
const jsonData = decoder.decode(data.type, data.payload);
// jsonData needs to be forwarded/redirected Thingsboard
});
};
Can anyone point me in the right direction, how to pass on the "handled" POST data and pass it on to another URL (Thingsboard HTTP endpoint URL)
Thanks in advance!
💡 This is not the best practice, but if you want to read some data from some route in another route without passing them when the route was called then you can try this code below: 👇
const express = require('express');
const app = express();
app.post('/', (req, res) => {
app.locals['decoder'] = { data: "This is my decoder data"};
res.send('Successfully to add decoder data');
});
app.get('/', (req, res) => {
res.send(app.locals['decoder']);
} )
app.listen(3000, () => {
console.log('Server is up');
});
⚠️ It should be noted that the code above is only an example of how to read data from one route on another route.
Another option:
👨‍🏫 You can create a middleware that is used to decode your payload and use the middleware on each route that requires the results of the decoded payload.
⚠️ But, it should be noted that for this problem, you must leave a payload on each route that requires decoded results, so that it can be decoded.
Updated: Passing Data To External Endpoint
💡 If You want to pass the data to an external endpoint or another website, see code below: 👇
const axios = require('axios');
app.post('/', (req, res) => {
const decoder = { data: "This is my decoder data"};
// method of external-endpoint: e.g: POST
axios.post('https://www.example-website.com/api/v1/example-endpoint', decoder)
.then(response => {
console.log(response);
}).catch(ex => {
console.log(ex);
})
})
You can use that route above in the example express server in addition to this answer.
I hope it's can help you 🙏.

Here-API 401 : "Invalid app_id app_code combination"

I am using an Angular front-end with a Nodejs backend. Im currently proxying all my front-end requests through my express server. However when I make my http request to the Here API I am rejected due to an invalid combination of app_id and app_code.
angular service
import { Injectable } from '#angular/core';
import { HttpClient } from '#angular/common/http'
import { HttpParams } from '#angular/common/http'
#Injectable({
providedIn: 'root'
})
export class GetReqPlaces {
constructor(private http: HttpClient) { }
getPlaces(wLong,sLat,eLong,nLat){
// let obj = {params: {westLong: wLong, southLat: sLat, eastLong:eLong, northLat:nLat }};
let params = new HttpParams().set("westLong" , '-97.783').set("southLat", '30.231').set( "eastLong" , '-97.740').set("northLat", '30.329');
return this.http.get( 'api/find/places', { params : params}).subscribe(res=>console.log(res))
}
}
server.js
const express = require("express")
const bodyParser = require("body-parser")
const cors = require("cors")
const path = require("path")
const app = express();
const request = require("request")
const environment= require('./keys')
app.use(cors());
app.use(bodyParser.urlencoded({extended: true}));
app.use(bodyParser.json());
let reqPath = __dirname.substring(0,__dirname.length-7)
app.use(express.static(path.join(reqPath, '/dist/angular-places-search')));
app.get('/api/find/places', (req, res) => {
let appId = environment.environment.appId;
let appCode = environment.environment.appCode;
let URL= `https://places.cit.api.here.com/places/v1/discover/search?app_id={${appId}}&app_code={${appCode}}&in=${req.query.westLong},${req.query.southLat},${req.query.eastLong},${req.query.northLat}&pretty`;
console.log(URL)
request(URL, function (error, response, body) {
let data={
body:body,
};
console.log(error,response)
res.send(data);
});
});
app.get('/test', (req, res) => res.send('Well this route was a hit! Bada....tsss'));
// CATCH ALL
app.get('*', (req, res) => {
res.sendFile(path.join(reqPath, 'dist/angular-places-search/index.html'));
});
app.listen(4000, () => console.log(`Express server running on port 4000`));
Before this I was running into CORS and request issues but I think I sorted those out. Based on my research on this same error code (In the context of the framework that Im working in), people overwhelmingly suggest to wait for tokens to register with Here API. Waiting two days is enough I think, still doesnt work. Then there is the very popular solution of just scratching the Here freemium and starting a new project, which I did, and which did not solve my issue. Very few things I have 100% certainty on but I did copy my keys correctly and the URL path built is according to the required Here syntax.
If anyone has any insight you will be my Hero, and also the catalyst for my continued learning :D. Happy Sunday!
In addition the incoming message I get through express is :
method: 'GET',
path: '/places/v1/discover/search?app_id=%notmyid%7D&app_code=%normycode%7D&in=-97.783,30.231,-97.740,30.329&pretty'
However i dont know why it is setting the app_id=% instead of using {}, when i console log the URL it is correct, with my app_id and app_code
The %7D is the url encoded value of the symbol } (urlencoding) which is done by most libraries. For using the HERE API you should not enclose the app_id/app_code between {}. They should be provided directly as strings, check the examples