Socket.io redis How data stored and cleared - redis

i am hosting an app on heroku which is using socket.io. it is using sockets and i am using heroku 4 standard 1X dynos . So for this i used redistogo service and socket.io-redis plugin. it's working great but i want to know does socket.io-redis also clear the data from redis db when socket disconnected. Heroku redis goto service providing only 20MB data storage. .Please guideline How socket.io-redis inserting and clearing the data in redis database.

Assuming that you are referring to https://github.com/Automattic/socket.io-redis/blob/master/index.js, it appears that the plugin uses Redis' PubSub functionality. PubSub does not maintain state in the Redis database so there's no need to clear any data.

The session store is responsible for session clean up upon socket disconnection. I use https://github.com/tj/connect-redis for my session store.
Here is an example of cleaning up the socket connection properly upon disconnecting.
const websocket = require('socket.io')(app.get('server'), {
transports: process.env.transports
})
websocket.setMaxListeners(0)
websocket.adapter(require('socket.io-redis')({
host: process.env.redis_host,
port: process.env.redis_port,
key: 'socket_io',
db: 2
}))
websocket.use((socket, next) => {
app.get('session')(socket.request, socket.request.res || {}, next)
})
websocket.on('connection', socket => {
var sess = socket.request.session
socket.use((packet, next) => {
if(!socket.rooms[sess.id]) {
socket.join(sess.id, () => {
websocket.of('/').adapter.remoteJoin(socket.id, sess.id, err => {
delete socket.rooms[socket.id]
next()
})
})
}
})
socket.on('disconnecting', () => {
websocket.of('/').adapter.remoteDisconnect(sess.id, true, err => {
delete socket.rooms[sess.id]
socket.removeAllListeners()
})
})
})

Related

How can I persist a login forever, from a different browser?

Background:
I have an app (node, expressjs, express-session, redis) which, despite not setting maxAge or expires, seems to timeout logins after a while.
Intention:
Via the admin panel, I want admins to be able to view current sessions (already working, by grabbing sess:* from redis), and click a persist button on a session to make it last indefinitely.
I need a reliable way of going from the data stored against the sess:... to a unique identifier that I can reference to the browser instance. Probably, storing something at login that is signed, and then saving that into a permanent db that is checked alongside the session check (and sets up a new authenticated session if needed).
EDIT: To clarify 'from a different browser' - this is for the admin function. So the goal is to have a list of current sessions (achieved already), with a "persist this login" button on each.
This would mean that the user in question then doesn't get logged out.
EDIT: This is my current code for session handling:
const express = require('express') // v4.17.1
const app = express()
const redis = require('redis') // v4.5.1
const expressSession = require('express-session') // v1.17.2
const RedisStore = require('connect-redis')(expressSession) // v3.4.2
const redisClient = redis.createClient({ legacyMode: true })
redisClient.connect()
const session = expressSession({
store: new RedisStore({ client: redisClient }),
saveUninitialized: true,
secret: ‘xxxxxxx’,
resave: true
})
app.use(session)
Answering first part:
How can I persist a login forever, from a different browser? ... I have an app (node, expressjs, express-session, redis) which, despite not setting maxAge or expires, seems to timeout logins after a while.
If you don't set express-session cookie.maxAge or cookie.expires then redis-connect will set a default TTL of one day:
If the session cookie has a expires date, connect-redis will use it as the TTL. Otherwise, it will expire the session using the ttl option (default: 86400 seconds or one day). -- https://github.com/tj/connect-redis#ttl
You can disable that by setting redis-connect disableTTL option to true. It is not recommended you do this (see next part).
(Note, It could also be something completely different like your users are deleting your cookies when they close browser, or you have a SPA and session aren't tracking properly across refresh etc).
Answering second part:
Summarizing my understanding on what you want:
A user can have many login sessions, 1 per device and upto N active sessions (what N is or how it is imposed is beyond the scope).
Some of those sessions you want to dynamically make forever sessions using this hypothetical button on the hypothetical admin panel.
So what you seem to want to do is uniquely identify each user's devices and be able to dynamically toggle the session lifetime for the given device between some default and forever. It's device id that matters, session ids are kind of incidental here.
If you want to do this with express-session I would suggest not trying to use disableTTL, but just setting a really large cookie.maxAge for these sessions (5 years is practically forever in web apps, and you can touch the session every time the user is active to reset TTL as well).
Below is proof of concept code to do that. Note code uses latest versions at time of writing - will probably break with other versions. You'll need to fill in the details:
Store the flag indicating which devices are forever devices in your user DB.
Figure out how to uniquely identify each device the user logs in with (I would suggest just grouping device types and only allow one session on each type but I don't know enough about what your trying to do to give more specific advice).
Update active session when the admin toggles the button on the admin panel or whatever.
const express = require('express');
const redis = require('redis');
const session = require('express-session'); // https://github.com/expressjs/session
const app = express();
const port = 3000;
const RedisStore = require('connect-redis')(session);
const redisClient = redis.createClient({
url: 'redis://localhost',
legacyMode: true,
});
redisClient.connect()
.then(() => { console.log('Redis client connected'); })
.catch(console.error);
const DAY_IN_MS = 60 * 60 * 24 * 1000;
const YEAR_IN_MS = 365 * DAY_IN_MS;
// Mock user DB.
const users = {
bob: {
id: 'bob',
forever_devices: ['foo_device', 'bah_device']
}
};
app.use(session({
store: new RedisStore({
client: redisClient,
// disableTTL: true, // Dont need this.
}),
secret: 'keyboards',
saveUninitialized: false,
resave: true,
cookie: {
maxAge: 3*DAY_IN_MS // Default 3 days.
},
}));
app.listen(port, () => console.log(`App listening on port ${port}!`));
app.use((req, res, next) => {
console.log(req.query, req.sessionID, req.session);
next();
});
// Mock login accepting a mock user id and device_id. Obviously insecure ..
app.get('/login', (req, res) => {
const { id, device_id } = req.query;
if(!(id in users)) {
return res.send('Denied').status(401);
}
req.session.loggedIn = true;
if(users[id].forever_devices?.includes(device_id)) {
req.session.cookie.maxAge = YEAR_IN_MS*1000;
}
res.send('OK');
});
app.get('/logout', (req, res) => {
req.session.destroy();
res.send('OK');
});
app.get('/', async function (req, res) {
if(req.session.loggedIn) {
redisClient.v4.ttl(`sess:${req.sessionID}`).then((d) => {
res.send(`Hi ${req.sessionID} your TTL is ${d}`);
});
} else {
res.send('Not logged in');
}
});
Testing with curl (you'll need to start redis on localhost with no credentials):
> curl -b cjar -c cjar 127.0.0.1:3000/login?id=bob
OK
> curl -b cjar -c cjar 127.0.0.1:3000/
Hi sukXf3ddjx3AMakVGIfChvlI-_KhaPqQ your TTL is 259196
> curl -b cjar -c cjar 127.0.0.1:3000/logout
OK
> curl -b cjar -c cjar "127.0.0.1:3000/login?id=bob&device_id=foo_device"
OK
> curl -b cjar -c cjar 127.0.0.1:3000/
Hi 81JZL-PxCRhNclktuYxHjTeHibaoNP9G your TTL is 31535999997
> curl -b cjar -c cjar 127.0.0.1:3000/logout
OK
> curl -b cjar -c cjar 127.0.0.1:3000/
Not logged in

Close express server when client emits to server with Socket.io

Goal: close server when client emits message.
I can confirm client sends response as it logs: 'hit server
Server code for express is:
io.on('connection', (socket) => {
console.log('this is connected');
socket.on('message', function(data){
console.log('hit server');
server.close();
})
});
io.on('error', function() {
console.log('there is an error');
})
server.listen(3000, () => {
console.log('listen on *:3000');
});
To stop an Express server that you're using socket.io with, you first call:
server.close();
That stops it from accepting new http connections. You can then iterate existing socket.io connections and close each of them:
const sockets = await io.fetchSockets();
for (let socket of sockets) {
socket.disconnect(true);
}
This could, in theory, leave a few regular http connections that are still in process, but will eventually complete or timeout.
server.getConnections(callback)
will tell you if there are any more outstanding connections or not.
If you just want to shut everything down, including your app, you can call:
process.exit()

Redis performance with a Digital Ocean managed instance

I'm using nodeJS to connect my a Digital Ocean droplet (Ubuntu 20.04) to a Digital Ocean managed Redis instance. I'm using the ioredis npm library.
Consider the simple trivial code below. This code works perfectly with the public network name albeit taking around 400ms. If I use the private network name the entire script hangs. I've also tried the private IP 10...* but that doesn't work either.
Does anyone have any experience here or insight as to how to connect directly with the VPC? Is there a specific way to use the private network name?
const Redis = require("ioredis");
(async () => {
// Spin up a redis client
const redis = new Redis({
host: "db-redis-**************-0.b.db.ondigitalocean.com",
port: *****,
username: "******",
password: "**********",
tls: {
key: "",
cert: "",
},
});
console.time("Total time to write/read a 10 character string to redis");
// Generate a random string
const generateRandomString = (length = 6) =>
Math.random().toString(20).substr(2, length);
// Save data to the redis server with a TTL of 2 miniutes
redis.set("redisTest", generateRandomString(10), "EX", 120);
// Now read it back
await redis.get("redisTest", function (err, result) {
if (err) {
console.error(err);
} else {
console.log("Data retrieved: ", result);
}
});
// Done
console.log("Done.");
console.timeEnd("Total time to write/read a 10 character string to redis");
})();
If using the private network address hangs during opening the connection, it's likely because your Droplet is not in the same VPC as your Redis database. In your case, it turned out that the Droplet and Redis were in different regions, so moving them to the same region (and ensuring they're in the same VPC within that region) should resolve the issue.

Calling express-session backed by connect-redis inside websockets/ws

Okay so here goes. I am trying to to use express-session backed by connect-redis together with websockets/ws. The vast majority of example code I have seen usually includes the use of Express to serve the client a websocket script of some type and then configuring the server app to use the express-session middleware to work with session data...with websockets further down the process chain.
My client side app on the other hand immediately initiates an Opening Handshake/Upgrade request, which bypasses Express and heads straight into websockets/ws. For this reason I am not able to work with session data through the commonly used app.use(session(...) middleware setup.
The below code uses a setup that allows me to call express-session inside websocket/ws and to put the connection request through express-session so I can get at the session data. This works as is shown by the output also below. What does not work however is backing express-session by connect-redis inside this setup. To check I call a redis client directly to query my redis box after express-session has run, no keys are returned.
I have three questions:
1. Is what I am doing below the 'correct' way of integrating express-session with websockets/ws for the client setup described?
2. If not what should I be doing?
3. If this is a good way, is it possible to get connect-redis working as session store?
Your insights are very welcome.
Server-side code:
const express = require('express');
const Session = require('express-session');
const http = require('http');
const ws = require('ws');
const util = require('util');
const redis = require('redis');
const client = redis.createClient(6379, '10.0.130.10', {no_ready_check: true});
const RedisStore = require('connect-redis')(Session);
const SessionStore = new RedisStore({host: '10.0.130.10', port: '6379', ttl: 60, logErrors: true});
//const SessionStore = new Session.MemoryStore();
var session = Session({
store: SessionStore,
cookie: {secure: true, maxAge: 3600, httpOnly: true},
resave: false,
saveUninitialized: true,
secret: '12345'
});
// Define Express and WS servers
const app = express();
const server = http.createServer(app);
const wss = new ws.Server({ server });
// WS Websocket
wss.on('connection', function connection(ws, req) {
session(req, {}, function(){
let sessionId = req.sessionID;
console.log('SessionID: ' + sessionId);
let sessionCookie = req.session.cookie;
console.log('SessionCookie: ' + JSON.stringify(sessionCookie));
});
client.keys('*', function(err, reply) {
// reply is null when the key is missing
console.log('Redis reponse: ' + reply);
});
});
server.listen(10031, function listening() {
console.log('Listening on: ' + server.address().port);
});
Server-side console.log output (note the Redis response):
Listening on: 10031
SessionID: Oi8AdsoZTAm3hRLmxPfGo43Kmmj_Yd6F
SessionCookie: {"originalMaxAge":3599,"expires":"2017-05-29T17:45:54.467Z","secure":true,"httpOnly":true,"path":"/"}
Redis reponse:
Reference:
express-session, connect-redis and einaros/ws
Reference:
ExpressJS & Websocket & session sharing

Socket.IO Authentication

I am trying to use Socket.IO in Node.js, and am trying to allow the server to give an identity to each of the Socket.IO clients. As the socket code is outside the scope of the http server code, it doesn't have easy access to the request information sent, so I'm assuming it will need to be sent up during the connection. What is the best way to
1) get the information to the server about who is connecting via Socket.IO
2) authenticate who they say they are (I'm currently using Express, if that makes things any easier)
Use connect-redis and have redis as your session store for all authenticated users. Make sure on authentication you send the key (normally req.sessionID) to the client. Have the client store this key in a cookie.
On socket connect (or anytime later) fetch this key from the cookie and send it back to the server. Fetch the session information in redis using this key. (GET key)
Eg:
Server side (with redis as session store):
req.session.regenerate...
res.send({rediskey: req.sessionID});
Client side:
//store the key in a cookie
SetCookie('rediskey', <%= rediskey %>); //http://msdn.microsoft.com/en-us/library/ms533693(v=vs.85).aspx
//then when socket is connected, fetch the rediskey from the document.cookie and send it back to server
var socket = new io.Socket();
socket.on('connect', function() {
var rediskey = GetCookie('rediskey'); //http://msdn.microsoft.com/en-us/library/ms533693(v=vs.85).aspx
socket.send({rediskey: rediskey});
});
Server side:
//in io.on('connection')
io.on('connection', function(client) {
client.on('message', function(message) {
if(message.rediskey) {
//fetch session info from redis
redisclient.get(message.rediskey, function(e, c) {
client.user_logged_in = c.username;
});
}
});
});
I also liked the way pusherapp does private channels.
A unique socket id is generated and
sent to the browser by Pusher. This is
sent to your application (1) via an
AJAX request which authorizes the user
to access the channel against your
existing authentication system. If
successful your application returns an
authorization string to the browser
signed with you Pusher secret. This is
sent to Pusher over the WebSocket,
which completes the authorization (2)
if the authorization string matches.
Because also socket.io has unique socket_id for every socket.
socket.on('connect', function() {
console.log(socket.transport.sessionid);
});
They used signed authorization strings to authorize users.
I haven't yet mirrored this to socket.io, but I think it could be pretty interesting concept.
I know this is bit old, but for future readers in addition to the approach of parsing cookie and retrieving the session from the storage (eg. passport.socketio ) you might also consider a token based approach.
In this example I use JSON Web Tokens which are pretty standard. You have to give to the client page the token, in this example imagine an authentication endpoint that returns JWT:
var jwt = require('jsonwebtoken');
// other requires
app.post('/login', function (req, res) {
// TODO: validate the actual user user
var profile = {
first_name: 'John',
last_name: 'Doe',
email: 'john#doe.com',
id: 123
};
// we are sending the profile in the token
var token = jwt.sign(profile, jwtSecret, { expiresInMinutes: 60*5 });
res.json({token: token});
});
Now, your socket.io server can be configured as follows:
var socketioJwt = require('socketio-jwt');
var sio = socketIo.listen(server);
sio.set('authorization', socketioJwt.authorize({
secret: jwtSecret,
handshake: true
}));
sio.sockets
.on('connection', function (socket) {
console.log(socket.handshake.decoded_token.email, 'has joined');
//socket.on('event');
});
The socket.io-jwt middleware expects the token in a query string, so from the client you only have to attach it when connecting:
var socket = io.connect('', {
query: 'token=' + token
});
I wrote a more detailed explanation about this method and cookies here.
Here is my attempt to have the following working:
express: 4.14
socket.io: 1.5
passport (using sessions): 0.3
redis: 2.6 (Really fast data structure to handle sessions; but you can use others like MongoDB too. However, I encourage you to use this for session data + MongoDB to store other persistent data like Users)
Since you might want to add some API requests as well, we'll also use http package to have both HTTP and Web socket working in the same port.
server.js
The following extract only includes everything you need to set the previous technologies up. You can see the complete server.js version which I used in one of my projects here.
import http from 'http';
import express from 'express';
import passport from 'passport';
import { createClient as createRedisClient } from 'redis';
import connectRedis from 'connect-redis';
import Socketio from 'socket.io';
// Your own socket handler file, it's optional. Explained below.
import socketConnectionHandler from './sockets';
// Configuration about your Redis session data structure.
const redisClient = createRedisClient();
const RedisStore = connectRedis(Session);
const dbSession = new RedisStore({
client: redisClient,
host: 'localhost',
port: 27017,
prefix: 'stackoverflow_',
disableTTL: true
});
// Let's configure Express to use our Redis storage to handle
// sessions as well. You'll probably want Express to handle your
// sessions as well and share the same storage as your socket.io
// does (i.e. for handling AJAX logins).
const session = Session({
resave: true,
saveUninitialized: true,
key: 'SID', // this will be used for the session cookie identifier
secret: 'secret key',
store: dbSession
});
app.use(session);
// Let's initialize passport by using their middlewares, which do
//everything pretty much automatically. (you have to configure login
// / register strategies on your own though (see reference 1)
app.use(passport.initialize());
app.use(passport.session());
// Socket.IO
const io = Socketio(server);
io.use((socket, next) => {
session(socket.handshake, {}, next);
});
io.on('connection', socketConnectionHandler);
// socket.io is ready; remember that ^this^ variable is just the
// name that we gave to our own socket.io handler file (explained
// just after this).
// Start server. This will start both socket.io and our optional
// AJAX API in the given port.
const port = 3000; // Move this onto an environment variable,
// it'll look more professional.
server.listen(port);
console.info(`🌐 API listening on port ${port}`);
console.info(`🗲 Socket listening on port ${port}`);
sockets/index.js
Our socketConnectionHandler, I just don't like putting everything inside server.js (even though you perfectly could), especially since this file can end up containing quite a lot of code pretty quickly.
export default function connectionHandler(socket) {
const userId = socket.handshake.session.passport &&
socket.handshake.session.passport.user;
// If the user is not logged in, you might find ^this^
// socket.handshake.session.passport variable undefined.
// Give the user a warm welcome.
console.info(`⚡︎ New connection: ${userId}`);
socket.emit('Grettings', `Grettings ${userId}`);
// Handle disconnection.
socket.on('disconnect', () => {
if (process.env.NODE_ENV !== 'production') {
console.info(`⚡︎ Disconnection: ${userId}`);
}
});
}
Extra material (client):
Just a very basic version of what the JavaScript socket.io client could be:
import io from 'socket.io-client';
const socketPath = '/socket.io'; // <- Default path.
// But you could configure your server
// to something like /api/socket.io
const socket = io.connect('localhost:3000', { path: socketPath });
socket.on('connect', () => {
console.info('Connected');
socket.on('Grettings', (data) => {
console.info(`Server gretting: ${data}`);
});
});
socket.on('connect_error', (error) => {
console.error(`Connection error: ${error}`);
});
References:
I just couldn't reference inside the code, so I moved it here.
1: How to set up your Passport strategies: https://scotch.io/tutorials/easy-node-authentication-setup-and-local#handling-signupregistration
This article (http://simplapi.wordpress.com/2012/04/13/php-and-node-js-session-share-redi/) shows how to
store sessions of the HTTP server in Redis (using Predis)
get these sessions from Redis in node.js by the session id sent in a cookie
Using this code you are able to get them in socket.io, too.
var io = require('socket.io').listen(8081);
var cookie = require('cookie');
var redis = require('redis'), client = redis.createClient();
io.sockets.on('connection', function (socket) {
var cookies = cookie.parse(socket.handshake.headers['cookie']);
console.log(cookies.PHPSESSID);
client.get('sessions/' + cookies.PHPSESSID, function(err, reply) {
console.log(JSON.parse(reply));
});
});
use session and Redis between c/s
Server side
io.use(function(socket, next) {
// get here session id
console.log(socket.handshake.headers.cookie); and match from redis session data
next();
});
this should do it
//server side
io.sockets.on('connection', function (con) {
console.log(con.id)
})
//client side
var io = io.connect('http://...')
console.log(io.sessionid)