I have a little problem with my application.
My architecture:
Angular 6 (front)
NodeJS + Express + MongoDB (back)
There is a part, in my NodeJS application, that communicates with a REST API.
When an user clicks on button on the Angular WebSite, I send a request to Express.
In my Express application, I send another request to the API to retrieve information.
But this process takes a lot of time. My request timeout
What is the best solution to keep my process working after the Express reply has been sent?
Should I do otherwise?
Assuming the problem is with timeout, you can increase the default timeout:
You may set the timeout either globally for entire server. Read more here:
var server = app.listen(app.get('port'), function() {
//Server is running
});
server.timeout = 1000; //1000 is one second.
or just for specific route. Read more here:
app.post('/xxx', function (req, res) {
req.setTimeout(500000);
});
You can also change the timeout for the other API request you are making. Read more here
//Assuming you are using request module.
var options = {
url: 'http://url',
timeout: 120000
}
request(options, function(err, resp, body) {});
Related
I am trying to get a long lived access token from facebook... according to the documentation, I need to fetch from the server side for security. If I fetch from the client side it works fine - but I should follow the guidelines.. Here is what I am doing - can you tell me why I get the server side error "Reference error: fetch is not defined"?
First I created a route from the front end to request the long lived token. To get the long lived token I need to use the short lived token so I send the short lived token in the req params. The variable userAccessTokenShort is valid and defined.
const newLongLivedUserAccessToken = await fetch(`/api/autoquotegenerators/longLivedUserToken/${userAccessTokenShort}`, {
method: 'GET',
headers: {
Authorization: `Bearer ${token}`,
"Content-Type": "application/json; charset=UTF-8",
},
})
const newLongUserTokenRes = await newLongLivedUserAccessToken.json()
console.log(newLongUserTokenRes)
Then, on my express server, I made a route that handles the fetch request to the Facebook Graph API. This is where I get the reference error.
//Get Long Lived User Access Token on Backend /express route
router.get('/longLivedUserToken/:shortLived', (req, res) => {
try {
const getLongToken = fetch(`https://graph.facebook.com/v7.0/oauth/access_token?grant_type=fb_exchange_token&client_id=#############&client_secret=################&fb_exchange_token=${req.params.shortLived}`)
.then(token => res.send(token))
} catch (error) {
console.log(error)
}
})
If all goes well, the new token should be sent back to the client.
Thanks for your help!
There is no fetch() function built into node.js. If you want something that looks pretty much like fetch() in the browser, you can look at the node-fetch module in NPM.
Or, there are lots of modules you can choose from in node.js. My favorite is got().
FYI, the built-in interface for fetching http requests is http.request() or http.get(), but they are lower level and require more lines of code to do something simple than any of the above named modules.
There are several moving parts, so it's difficult to know what to debug here.
I have a web application on one localhost port, and a simple helper on another localhost running an express NodeJS application with a couple of endpoints.
The basic issue I'm seeing is that my cookie session on the express application is empty for subsequent calls, and I don't even see it being sent back in the first response.
The setup
The client makes basic GET ajax calls (jQuery at the moment) to the express application.
I have set http allowance for session cookies:
app.use(cookieSession({
name: 'session',
keys: ['abcdefg'],
maxAge: 24 * 60 * 60 * 1000, // 24 hours,
secure: false
}))
I have set cross-origin requests on the express application:
app.use((req, res, next) => {
const corsWhitelist = [
'http://localhost:8000',
'http://localhost:8777'
];
if (corsWhitelist.indexOf(req.headers.origin) !== -1) {
res.header('Access-Control-Allow-Origin', req.headers.origin);
res.header('Access-Control-Allow-Headers', 'Origin, X-Requested-With, Content-Type, Accept');
}
next();
});
And the requests are completed seemingly without issue, as the response bodies are what I expect.
The meat of the request handler is:
app.get('/initialize', (req, res) => {
console.log(req.session);
//if we have a session with verified status
if (req.session.hasOwnProperty("userId") && req.session.userId){
res.send({result: 'OK', message: 'good - 1'});
return;
}
const id = uuid.v4();
req.session.userId = id;
res.send({result: 'OK', message: 'good - 2'});
return;
});
I always always get the second response 'good - 2' from the ajax call. The log always shows the session as {}
It's probably worth noting that Chrome devtools shows "Provisional headers are shown" for the request headers, and set-cookie is not shown in the response headers. The AJAX is a simple GET to an endpoint with one parameter passed in.
Update
Just now occurred to me to try without using the AJAX call. Hitting the URL directly gets the cookie and keeps the session as expected. That will probably change the dynamic of the issue.
The fix was to use jsonp request/response to get cookies to pass around the ajax call. Nothing to do with express really.
https://samueleresca.net/2015/07/json-and-jsonp-requests-using-expressjs/
Pretty new to Node.js. I'm working on a project where I need access to Instagram's subscription/streaming API. Stuck on the handshake authorization, where my server needs to reply with a challenge code sent by Instagram.
The code to get and log the challenge code:
var http = require('http'),
url = require ('url');
http.createServer(function (req, res) {
var parts = url.parse(req.url, true);
var parts = parts.query['hub.challenge'];
console.log(parts);
}).listen(8080, '0.0.0.0');
And the code to send the subscribe request to Instagram:
Instagram = require('instagram-node-lib');
Instagram.set('client_id', 'myClientID');
Instagram.set('client_secret', 'myClientSecret');
Instagram.set('callback_url', 'http://www.myLocalTunnelAddress/callback.js')
Instagram.media.subscribe({ lat: 48.858844300000001, lng: 2.2943506, radius: 1000 });
I can get the challenge code from Instagram's GET request, but have been working at this for a while and haven't been able to send it back. Does anyone have a suggestions as to how to do this? Thank you!
Check out this node.js library
https://github.com/mckelvey/instagram-node-lib/blob/master/lib/class.instagram.subscriptions.js
and how he implemented his handshake method:
https://github.com/mckelvey/instagram-node-lib/blob/master/lib/class.instagram.subscriptions.js
I'm trying to write a front end to an API service with Node JS.
I'd like to be able to have a user point their browser at my node server and make a request. The node script would modify the input to the request, call the api service, then modify the output and pass back to the user.
I like the solution here (with Express JS and node-http-proxy) as it passes the cookies and headers directly from the user through my site to the api server.
proxy request in node.js / express
I see how to modify the input to the request, but i can't figure out how to modify the response. Any suggestions?
transformer-proxy could be useful here. I'm the author of this plugin and I'm answering here because I found this page when looking for the same question and wasn't satisfied with harmon as I don't want to manipulate HTML.
Maybe someone else is looking for this and finds it useful.
Harmon is designed to plug into node-http-proxy https://github.com/No9/harmon
It uses trumpet and so is stream based to work around any buffering problems.
It uses an element and attribute selector to enable manipulation of a response.
This can be used to modify output response.
See here: https://github.com/nodejitsu/node-http-proxy/issues/382#issuecomment-14895039
http-proxy-interceptor is a middleware I wrote for this very purpose. It allows you to modify the http response using one or more transform streams. There are tons of stream-based packages available (like trumpet, which harmon uses), and by using streams you can avoid buffering the entire response.
var httpProxy = require('http-proxy');
var modifyResponse = require('http-proxy-response-rewrite');
var proxy = httpProxy.createServer({
target:'target server IP here',
});
proxy.listen(8001);
proxy.on('error', function (err, req, res) {
res.writeHead(500, {
'Content-Type': 'text/plain'
});
res.end('Something went wrong. And we are reporting a custom error message.');
});
proxy.on('proxyRes', function (proxyRes, req, res) {
modifyResponse(res, proxyRes.headers['content-encoding'], function (body) {
if (body && (body.indexOf("<process-order-response>")!= -1)) {
var beforeTag = "</receipt-text>"; //tag after which u can add data to
// response
var beforeTagBody = body.substring(0,(body.indexOf(beforeTag) + beforeTag.length));
var requiredXml = " <ga-loyalty-rewards>\n"+
"<previousBalance>0</previousBalance>\n"+
"<availableBalance>0</availableBalance>\n"+
"<accuruedAmount>0</accuruedAmount>\n"+
"<redeemedAmount>0</redeemedAmount>\n"+
"</ga-loyalty-rewards>";
var afterTagBody = body.substring(body.indexOf(beforeTag)+ beforeTag.length)+
var res = [];
res.push(beforeTagBody, requiredXml, afterTagBody);
console.log(res.join(""));
return res.join("");
}
return body;
});
});
I am trying to use Socket.IO in Node.js, and am trying to allow the server to give an identity to each of the Socket.IO clients. As the socket code is outside the scope of the http server code, it doesn't have easy access to the request information sent, so I'm assuming it will need to be sent up during the connection. What is the best way to
1) get the information to the server about who is connecting via Socket.IO
2) authenticate who they say they are (I'm currently using Express, if that makes things any easier)
Use connect-redis and have redis as your session store for all authenticated users. Make sure on authentication you send the key (normally req.sessionID) to the client. Have the client store this key in a cookie.
On socket connect (or anytime later) fetch this key from the cookie and send it back to the server. Fetch the session information in redis using this key. (GET key)
Eg:
Server side (with redis as session store):
req.session.regenerate...
res.send({rediskey: req.sessionID});
Client side:
//store the key in a cookie
SetCookie('rediskey', <%= rediskey %>); //http://msdn.microsoft.com/en-us/library/ms533693(v=vs.85).aspx
//then when socket is connected, fetch the rediskey from the document.cookie and send it back to server
var socket = new io.Socket();
socket.on('connect', function() {
var rediskey = GetCookie('rediskey'); //http://msdn.microsoft.com/en-us/library/ms533693(v=vs.85).aspx
socket.send({rediskey: rediskey});
});
Server side:
//in io.on('connection')
io.on('connection', function(client) {
client.on('message', function(message) {
if(message.rediskey) {
//fetch session info from redis
redisclient.get(message.rediskey, function(e, c) {
client.user_logged_in = c.username;
});
}
});
});
I also liked the way pusherapp does private channels.
A unique socket id is generated and
sent to the browser by Pusher. This is
sent to your application (1) via an
AJAX request which authorizes the user
to access the channel against your
existing authentication system. If
successful your application returns an
authorization string to the browser
signed with you Pusher secret. This is
sent to Pusher over the WebSocket,
which completes the authorization (2)
if the authorization string matches.
Because also socket.io has unique socket_id for every socket.
socket.on('connect', function() {
console.log(socket.transport.sessionid);
});
They used signed authorization strings to authorize users.
I haven't yet mirrored this to socket.io, but I think it could be pretty interesting concept.
I know this is bit old, but for future readers in addition to the approach of parsing cookie and retrieving the session from the storage (eg. passport.socketio ) you might also consider a token based approach.
In this example I use JSON Web Tokens which are pretty standard. You have to give to the client page the token, in this example imagine an authentication endpoint that returns JWT:
var jwt = require('jsonwebtoken');
// other requires
app.post('/login', function (req, res) {
// TODO: validate the actual user user
var profile = {
first_name: 'John',
last_name: 'Doe',
email: 'john#doe.com',
id: 123
};
// we are sending the profile in the token
var token = jwt.sign(profile, jwtSecret, { expiresInMinutes: 60*5 });
res.json({token: token});
});
Now, your socket.io server can be configured as follows:
var socketioJwt = require('socketio-jwt');
var sio = socketIo.listen(server);
sio.set('authorization', socketioJwt.authorize({
secret: jwtSecret,
handshake: true
}));
sio.sockets
.on('connection', function (socket) {
console.log(socket.handshake.decoded_token.email, 'has joined');
//socket.on('event');
});
The socket.io-jwt middleware expects the token in a query string, so from the client you only have to attach it when connecting:
var socket = io.connect('', {
query: 'token=' + token
});
I wrote a more detailed explanation about this method and cookies here.
Here is my attempt to have the following working:
express: 4.14
socket.io: 1.5
passport (using sessions): 0.3
redis: 2.6 (Really fast data structure to handle sessions; but you can use others like MongoDB too. However, I encourage you to use this for session data + MongoDB to store other persistent data like Users)
Since you might want to add some API requests as well, we'll also use http package to have both HTTP and Web socket working in the same port.
server.js
The following extract only includes everything you need to set the previous technologies up. You can see the complete server.js version which I used in one of my projects here.
import http from 'http';
import express from 'express';
import passport from 'passport';
import { createClient as createRedisClient } from 'redis';
import connectRedis from 'connect-redis';
import Socketio from 'socket.io';
// Your own socket handler file, it's optional. Explained below.
import socketConnectionHandler from './sockets';
// Configuration about your Redis session data structure.
const redisClient = createRedisClient();
const RedisStore = connectRedis(Session);
const dbSession = new RedisStore({
client: redisClient,
host: 'localhost',
port: 27017,
prefix: 'stackoverflow_',
disableTTL: true
});
// Let's configure Express to use our Redis storage to handle
// sessions as well. You'll probably want Express to handle your
// sessions as well and share the same storage as your socket.io
// does (i.e. for handling AJAX logins).
const session = Session({
resave: true,
saveUninitialized: true,
key: 'SID', // this will be used for the session cookie identifier
secret: 'secret key',
store: dbSession
});
app.use(session);
// Let's initialize passport by using their middlewares, which do
//everything pretty much automatically. (you have to configure login
// / register strategies on your own though (see reference 1)
app.use(passport.initialize());
app.use(passport.session());
// Socket.IO
const io = Socketio(server);
io.use((socket, next) => {
session(socket.handshake, {}, next);
});
io.on('connection', socketConnectionHandler);
// socket.io is ready; remember that ^this^ variable is just the
// name that we gave to our own socket.io handler file (explained
// just after this).
// Start server. This will start both socket.io and our optional
// AJAX API in the given port.
const port = 3000; // Move this onto an environment variable,
// it'll look more professional.
server.listen(port);
console.info(`🌐 API listening on port ${port}`);
console.info(`🗲 Socket listening on port ${port}`);
sockets/index.js
Our socketConnectionHandler, I just don't like putting everything inside server.js (even though you perfectly could), especially since this file can end up containing quite a lot of code pretty quickly.
export default function connectionHandler(socket) {
const userId = socket.handshake.session.passport &&
socket.handshake.session.passport.user;
// If the user is not logged in, you might find ^this^
// socket.handshake.session.passport variable undefined.
// Give the user a warm welcome.
console.info(`⚡︎ New connection: ${userId}`);
socket.emit('Grettings', `Grettings ${userId}`);
// Handle disconnection.
socket.on('disconnect', () => {
if (process.env.NODE_ENV !== 'production') {
console.info(`⚡︎ Disconnection: ${userId}`);
}
});
}
Extra material (client):
Just a very basic version of what the JavaScript socket.io client could be:
import io from 'socket.io-client';
const socketPath = '/socket.io'; // <- Default path.
// But you could configure your server
// to something like /api/socket.io
const socket = io.connect('localhost:3000', { path: socketPath });
socket.on('connect', () => {
console.info('Connected');
socket.on('Grettings', (data) => {
console.info(`Server gretting: ${data}`);
});
});
socket.on('connect_error', (error) => {
console.error(`Connection error: ${error}`);
});
References:
I just couldn't reference inside the code, so I moved it here.
1: How to set up your Passport strategies: https://scotch.io/tutorials/easy-node-authentication-setup-and-local#handling-signupregistration
This article (http://simplapi.wordpress.com/2012/04/13/php-and-node-js-session-share-redi/) shows how to
store sessions of the HTTP server in Redis (using Predis)
get these sessions from Redis in node.js by the session id sent in a cookie
Using this code you are able to get them in socket.io, too.
var io = require('socket.io').listen(8081);
var cookie = require('cookie');
var redis = require('redis'), client = redis.createClient();
io.sockets.on('connection', function (socket) {
var cookies = cookie.parse(socket.handshake.headers['cookie']);
console.log(cookies.PHPSESSID);
client.get('sessions/' + cookies.PHPSESSID, function(err, reply) {
console.log(JSON.parse(reply));
});
});
use session and Redis between c/s
Server side
io.use(function(socket, next) {
// get here session id
console.log(socket.handshake.headers.cookie); and match from redis session data
next();
});
this should do it
//server side
io.sockets.on('connection', function (con) {
console.log(con.id)
})
//client side
var io = io.connect('http://...')
console.log(io.sessionid)