Forward decoded POST data to other URL using Express - express

Some background information:
I have a temperature sensor which is connected via LoRaWAN.
The data from the sensor is encoded, but the manufacturer of the sensor provides a decoder (node app).
I want to show the data in Thingsboard.
Now I was thinking to build a simple express API (I'm new to Express) as a kind of middleware.
The only purpose is to receive the encoded data, decode it (output is in JSON format), and pass it on to Thingsboard.
With the last step I need some help. This is my route:
const decoder = require('../helpers/decoder');
module.exports = app => {
app.post('/', (req, res) => {
const data = {
type: req.body.productType,
payload: req.body.payloadValue
};
const jsonData = decoder.decode(data.type, data.payload);
// jsonData needs to be forwarded/redirected Thingsboard
});
};
Can anyone point me in the right direction, how to pass on the "handled" POST data and pass it on to another URL (Thingsboard HTTP endpoint URL)
Thanks in advance!

💡 This is not the best practice, but if you want to read some data from some route in another route without passing them when the route was called then you can try this code below: 👇
const express = require('express');
const app = express();
app.post('/', (req, res) => {
app.locals['decoder'] = { data: "This is my decoder data"};
res.send('Successfully to add decoder data');
});
app.get('/', (req, res) => {
res.send(app.locals['decoder']);
} )
app.listen(3000, () => {
console.log('Server is up');
});
⚠️ It should be noted that the code above is only an example of how to read data from one route on another route.
Another option:
👨‍🏫 You can create a middleware that is used to decode your payload and use the middleware on each route that requires the results of the decoded payload.
⚠️ But, it should be noted that for this problem, you must leave a payload on each route that requires decoded results, so that it can be decoded.
Updated: Passing Data To External Endpoint
💡 If You want to pass the data to an external endpoint or another website, see code below: 👇
const axios = require('axios');
app.post('/', (req, res) => {
const decoder = { data: "This is my decoder data"};
// method of external-endpoint: e.g: POST
axios.post('https://www.example-website.com/api/v1/example-endpoint', decoder)
.then(response => {
console.log(response);
}).catch(ex => {
console.log(ex);
})
})
You can use that route above in the example express server in addition to this answer.
I hope it's can help you 🙏.

Related

Frontend to Backend POST request is not yielding the data I want

I'm currently working on a project using a React frontend and an Express backend. Currently, when I make a GET request to retrieve data from the backend, everything is working fine. However, I'm unable to POST data to the backend and gain access to the data that's being sent. I'm getting an OK message so the request is going through, but when I log the request data in the backend, I get a message like this which is a jumble of random fields.
Here is the code snippit in the front end for the POST request
const makePost = (data) => {
fetch('http://localhost:5000/api', {
method: 'POST',
headers: {"Content-Type": "application/json", "Access-Control-Allow-Origin": "*"},
body: JSON.parse(JSON.stringify(data))
}).then(function(response){
console.log(response.text())
})
}
Here is my backend which handles the POST request
const express = require('express');
const app = express();
const cors = require('cors');
app.use(cors({
origin: '*'
}));
app.get('/api', (req,res) => {
res.json(menuItems);
});
app.post('/api', (req,res) => {
console.log(req)
})
app.listen(5000, () => console.log("server started on port 5000"));
In the code snippit above, console.log(req) is what was logged in the screenshot linked above.
In your Express server POST API, you are not returning any data, it may cause problems. This is a sample POST API using Axios, Express, React, and MongoDB.Hope it would help you.
//POST API
app.post('/services',async(req,res)=>{
const service = req.body;
const result = await servicesCollection.insertOne(service);
console.log(result);
res.send(result)
});
In client-side POST api:
const onSubmit = data => {
axios.post('http://localhost/services', data)
.then(res=>{
if(res.data.insertedId){
alert('data added successfully');
reset();
}
})
sample post API:
app.post('/book', (req, res) => {
const book = req.body;
// Output the book to the console for debugging
console.log(book);
books.push(book);
res.send('Book is added to the database');
});
Pls take a look at this link: https://riptutorial.com/node-js/example/20967/post-api-using-express

Market data routing to frontend: Alpaca WebSocket -> Node.js WebSocket -> React Frontend

I'm trying to use the websocket example from:
https://github.com/alpacahq/alpaca-trade-api-js/blob/master/examples/websocket_example_datav2.js
In order to connect to the Alpaca V2 data stream. Currently, my stream is working but I'm trying to route my data to the client side using Server Sent Events. My data flow seems like it should be:
Alpaca Data Stream API -> My Node.js server -> React Frontend.
The issue I have is using the DataStream object in the example in order to route the data to the frontend. Since, with the object alone, I don't have any route to subscribe to via Server Sent Events, does this mean that I should also be using either express, socket.io, or ws? Since the all of the ".on_xyz" methods are defined within the DataStream object, I'm not sure how to set up the endpoint properly to allow my frontend to subscribe to it. If anyone knows how to route this datastream information forward it would be greatly appreciated- I'm particularly trying to work with the .onStockQuote method but any of them is fine! I'm simply trying to use Node as an inbetween router so that I don't have to subscribe directly from the frontend (and not use the sdk), because that limits scalability of the API's use.
"use strict";
/**
* This examples shows how to use tha alpaca data v2 websocket to subscribe to events.
* You should use the alpaca api's data_steam_v2, also add feed besides the other parameters.
* For subscribing (and unsubscribing) to trades, quotes and bars you should call
* a function for each like below.
*/
import express from 'express';
const app = express()
const Alpaca = require("#alpacahq/alpaca-trade-api");
const API_KEY = "XYZ_Key";
const API_SECRET = "XYZ_Secret";
const PORT = 3000;
// Add a new message and send it to all subscribed clients
const addMessage = (req, res) => {
const message = req.body;
// Return the message as a response for the "/message" call
res.json(message);
return ;
};
class DataStream {
constructor({ apiKey, secretKey, feed }) {
this.alpaca = new Alpaca({
keyId: apiKey,
secretKey,
feed,
});
const socket = this.alpaca.data_stream_v2;
socket.onConnect(function () {
console.log("Connected");
socket.subscribeForQuotes(["AAPL"]);
// socket.subscribeForTrades(["FB"]);
// socket.subscribeForBars(["SPY"]);
// socket.subscribeForStatuses(["*"]);
});
socket.onError((err) => {
console.log(err);
});
socket.onStockTrade((trade) => {
console.log(trade);
});
socket.onStockQuote((quote) => {
console.log(quote);
});
socket.onStockBar((bar) => {
console.log(bar);
});
socket.onStatuses((s) => {
console.log(s);
});
socket.onStateChange((state) => {
console.log(state);
});
socket.onDisconnect(() => {
console.log("Disconnected");
});
socket.connect();
// unsubscribe from FB after a second
// setTimeout(() => {
// socket.unsubscribeFromTrades(["FB"]);
// }, 1000);
}
}
app.post("/message", addMessage);
let stream = new DataStream({
apiKey: API_KEY,
secretKey: API_SECRET,
feed: "sip",
paper: false,
});
app.listen(PORT, () => {
console.log(`App listening on port ${PORT}`);
});

Express 4.17 req.body is being printed as buffer

Something that I wanted clarification on is when console.logging req.body in my express app. I end up with a buffer/string or hexadecimal of some sort when I'm expecting an JSON object. I'm using postman to send a raw json body.
Here are some visuals of the source code and terminal/postman results.
const express = require('express');
const bodyParser = require('body-parser');
const { randomBytes } = require('crypto');
const app = express();
app.use(express.raw({type: "application/json"}));
app.use(express.json({strict: false}));
app.use(express.urlencoded({extended: false}));
const posts = {};
app.get('/posts', (req, res) => {
res.send(posts);
});
app.post('/posts', (req, res) => {
const id = randomBytes(4).toString('hex');
const { title } = req.body;
console.log(req.body)
posts[id] = {
id,
title: title
};
res.status(201).send(posts[id]);
});
app.listen(4000, () => {
console.log('Listening on 4000');
});
Console.log terminal of buffer/hex/string
Postman body
Postman Raw String
app.use(express.raw({type: "application/json"})); is going to give you a Buffer object and since that's your first middleware that might handle that mime type, that's what you're going to get for any application/json request.
Right from the Express doc for express.raw():
This is a built-in middleware function in Express. It parses incoming
request payloads into a Buffer
It's unclear why you're using express.raw() as that is not typical for JSON payloads, but when doing so, you are going to get a Buffer - that's how it works. One would more typically use express.json() for JSON payloads and let it parse your JSON so that req.body contains an actual Javascript object.
If you remove the app.use(express.raw({type: "application/json"})); line of code and let the app.use(express.json()); line of code right after it handle the application/json payloads, then you will get your parsed data in req.body.
Keep in mind that when using middleware they are processed in the order declared and for this specific type of middleware, the first one that matches and reads the body from the incoming stream takes precedence and none of the others after it will be able to do their job (since the incoming stream has already been read).

Here-API 401 : "Invalid app_id app_code combination"

I am using an Angular front-end with a Nodejs backend. Im currently proxying all my front-end requests through my express server. However when I make my http request to the Here API I am rejected due to an invalid combination of app_id and app_code.
angular service
import { Injectable } from '#angular/core';
import { HttpClient } from '#angular/common/http'
import { HttpParams } from '#angular/common/http'
#Injectable({
providedIn: 'root'
})
export class GetReqPlaces {
constructor(private http: HttpClient) { }
getPlaces(wLong,sLat,eLong,nLat){
// let obj = {params: {westLong: wLong, southLat: sLat, eastLong:eLong, northLat:nLat }};
let params = new HttpParams().set("westLong" , '-97.783').set("southLat", '30.231').set( "eastLong" , '-97.740').set("northLat", '30.329');
return this.http.get( 'api/find/places', { params : params}).subscribe(res=>console.log(res))
}
}
server.js
const express = require("express")
const bodyParser = require("body-parser")
const cors = require("cors")
const path = require("path")
const app = express();
const request = require("request")
const environment= require('./keys')
app.use(cors());
app.use(bodyParser.urlencoded({extended: true}));
app.use(bodyParser.json());
let reqPath = __dirname.substring(0,__dirname.length-7)
app.use(express.static(path.join(reqPath, '/dist/angular-places-search')));
app.get('/api/find/places', (req, res) => {
let appId = environment.environment.appId;
let appCode = environment.environment.appCode;
let URL= `https://places.cit.api.here.com/places/v1/discover/search?app_id={${appId}}&app_code={${appCode}}&in=${req.query.westLong},${req.query.southLat},${req.query.eastLong},${req.query.northLat}&pretty`;
console.log(URL)
request(URL, function (error, response, body) {
let data={
body:body,
};
console.log(error,response)
res.send(data);
});
});
app.get('/test', (req, res) => res.send('Well this route was a hit! Bada....tsss'));
// CATCH ALL
app.get('*', (req, res) => {
res.sendFile(path.join(reqPath, 'dist/angular-places-search/index.html'));
});
app.listen(4000, () => console.log(`Express server running on port 4000`));
Before this I was running into CORS and request issues but I think I sorted those out. Based on my research on this same error code (In the context of the framework that Im working in), people overwhelmingly suggest to wait for tokens to register with Here API. Waiting two days is enough I think, still doesnt work. Then there is the very popular solution of just scratching the Here freemium and starting a new project, which I did, and which did not solve my issue. Very few things I have 100% certainty on but I did copy my keys correctly and the URL path built is according to the required Here syntax.
If anyone has any insight you will be my Hero, and also the catalyst for my continued learning :D. Happy Sunday!
In addition the incoming message I get through express is :
method: 'GET',
path: '/places/v1/discover/search?app_id=%notmyid%7D&app_code=%normycode%7D&in=-97.783,30.231,-97.740,30.329&pretty'
However i dont know why it is setting the app_id=% instead of using {}, when i console log the URL it is correct, with my app_id and app_code
The %7D is the url encoded value of the symbol } (urlencoding) which is done by most libraries. For using the HERE API you should not enclose the app_id/app_code between {}. They should be provided directly as strings, check the examples

github apps - webhook request doesn't have payload

First, here are my github apps events:
And I set up my webhook URL(blahblah.com/watch) and express server like this:
const express = require("express");
const app = express();
app.post("/watch", (req, res) => {
console.log(req);
});
app.listen(3000, () => console.log("opened"));
The problem is that when I create an issue comment, the request of webhook arrives in my express server but has no body or payload.
The request is here:
LOOK AT THAT! THERE IS NO PAYLOAD!
What should I do? What am I doing wrong?
I thought req must have a property body or payload. How can I access the payload of the request?
Actually, the problem is not because of webhook request, but my express code!
I've resolved the problem by modifying my code like this:
app.post("/watch", (req, res) => {
let body = "";
req.on("readable", () => {
body += req.read();
});
req.on("end", () => {
console.log(body);
})
});
I didn't know that there is readable event and end event to handle a request!
I've got an idea from this answer.