I have an application where I want to avoid robots to try to use my socket.io endpoint.
My socket.io sits on top of express:
const app = require('express')();
app.use(blockRobots);
const io = require('socket.io')(app{path: '/socket'});
If I access this server to any path except /socket, the middleware is executed.
However, doing a (GET) request to /socket does not trigger the middleware.
Any ideas?
Without delving into the code, I assume that socket.io attaches a listener to the HTTP server that gets triggered before Express gets to handle the request at all.
You can use the allowRequest option for socket.io to reject unwanted requests:
const io = require('socket.io')(app, {
path: '/socket',
allowRequest: (req, callback) => {
if (CHECK_FOR_ROBOT) {
return callback(null, false);
} else {
return callback(null, true);
}
}
});
Related
I'm trying to use the websocket example from:
https://github.com/alpacahq/alpaca-trade-api-js/blob/master/examples/websocket_example_datav2.js
In order to connect to the Alpaca V2 data stream. Currently, my stream is working but I'm trying to route my data to the client side using Server Sent Events. My data flow seems like it should be:
Alpaca Data Stream API -> My Node.js server -> React Frontend.
The issue I have is using the DataStream object in the example in order to route the data to the frontend. Since, with the object alone, I don't have any route to subscribe to via Server Sent Events, does this mean that I should also be using either express, socket.io, or ws? Since the all of the ".on_xyz" methods are defined within the DataStream object, I'm not sure how to set up the endpoint properly to allow my frontend to subscribe to it. If anyone knows how to route this datastream information forward it would be greatly appreciated- I'm particularly trying to work with the .onStockQuote method but any of them is fine! I'm simply trying to use Node as an inbetween router so that I don't have to subscribe directly from the frontend (and not use the sdk), because that limits scalability of the API's use.
"use strict";
/**
* This examples shows how to use tha alpaca data v2 websocket to subscribe to events.
* You should use the alpaca api's data_steam_v2, also add feed besides the other parameters.
* For subscribing (and unsubscribing) to trades, quotes and bars you should call
* a function for each like below.
*/
import express from 'express';
const app = express()
const Alpaca = require("#alpacahq/alpaca-trade-api");
const API_KEY = "XYZ_Key";
const API_SECRET = "XYZ_Secret";
const PORT = 3000;
// Add a new message and send it to all subscribed clients
const addMessage = (req, res) => {
const message = req.body;
// Return the message as a response for the "/message" call
res.json(message);
return ;
};
class DataStream {
constructor({ apiKey, secretKey, feed }) {
this.alpaca = new Alpaca({
keyId: apiKey,
secretKey,
feed,
});
const socket = this.alpaca.data_stream_v2;
socket.onConnect(function () {
console.log("Connected");
socket.subscribeForQuotes(["AAPL"]);
// socket.subscribeForTrades(["FB"]);
// socket.subscribeForBars(["SPY"]);
// socket.subscribeForStatuses(["*"]);
});
socket.onError((err) => {
console.log(err);
});
socket.onStockTrade((trade) => {
console.log(trade);
});
socket.onStockQuote((quote) => {
console.log(quote);
});
socket.onStockBar((bar) => {
console.log(bar);
});
socket.onStatuses((s) => {
console.log(s);
});
socket.onStateChange((state) => {
console.log(state);
});
socket.onDisconnect(() => {
console.log("Disconnected");
});
socket.connect();
// unsubscribe from FB after a second
// setTimeout(() => {
// socket.unsubscribeFromTrades(["FB"]);
// }, 1000);
}
}
app.post("/message", addMessage);
let stream = new DataStream({
apiKey: API_KEY,
secretKey: API_SECRET,
feed: "sip",
paper: false,
});
app.listen(PORT, () => {
console.log(`App listening on port ${PORT}`);
});
How can I define routes for Websockets using the ws library within an ExpressJS app? It's very easy to set it up the two layers in parallel, but the Websocket layer will not be able to benefit from ExpressJS middlewares (such as authentication). The only implementation I could find was express-ws, which is severely buggy due to not being up to date, and heavily relies on monkeypatching in order to work.
Partially modified from this answer. Modify your entry file to include this:
/* index.ts */
import http from 'http';
import express from 'express';
import exampleRouter from './exampleRouter';
// set up express and create a http server listen for websocket requests on the same port
const app = express();
const server = http.createServer(app);
// listen for websocket requests, which are simple HTTP GET requests with an upgrade header
// NOTE: this must occur BEFORE other middleware is set up if you want the additional ws.handled functionality to close unhandled requests
server.on('upgrade', (req: Request & { ws: { socket: Socket, head: Buffer, handled: Boolean } }, socket: Socket, head: Buffer) => {
// create a dummy response to pass the request into express
const res = new http.ServerResponse(req);
// assign socket and head to a new field in the request object
// optional **handled** field lets us know if there a route processed the websocket request, else we terminate it later on
req.ws = { socket, head, handled: false };
// have Express process the request
app(req, res);
});
/* whatever Express middlewares you want here, such as authentication */
app.use('/example', exampleRouter);
// set up a middleware to destroy unhandled websocket requests and returns a 403
// NOTE: this must occur AFTER your other middlewares but BEFORE the server starts listening for requests
app.use((req: Request & { ws?: { socket: Socket, head: Buffer, handled: Boolean } }, res: Response, next: NextFunction): void => {
if (req.ws && req.ws.handled === false) {
req.ws.socket.destroy();
res.status(404).json('404: Websocket route not found');
}
next();
});
const port = process.env.PORT || 8080;
server.listen(port);
Example of a Express Router with ws functionality, but the logic can be extracted to be used for one-offs
/* RouterWithWebSockets.ts */
// this is just a simple abstraction implementation so the you can set up multiple ws routes with the same router
// without having to rewrite the WSS code or monkeypatch the function into the Express Router directly
import express from 'express';
import { WebSocketServer, WebSocket } from 'ws';
class RouterWithWebSockets {
router;
constructor(router = express.Router()) {
this.router = router;
}
ws = (path: string, callback: (ws: WebSocket) => void, ...middleware: any): void => {
// set up a new WSS with the provided path/route to handle websockets
const wss = new WebSocketServer({
noServer: true,
path,
});
this.router.get(path, ...middleware, (req: any, res, next) => {
// just an extra check to deny upgrade requests if the path/route does not match
// you can process this route as a regular HTTP GET request if it's not a websocket upgrade request by replacing the next()
if (!req.headers.upgrade || path !== req.url) {
next();
} else {
req.ws.handled = true;
wss.handleUpgrade(req, req.ws.socket, req.ws.head, (ws: WebSocket) => {
callback(ws);
});
}
});
};
}
export default RouterWithWebSockets;
Finally, here is an example router with the Websocket routes
/* exampleRouter.ts */
const routerWithWebSockets = new RouterWithWebSockets();
routerWithWebSockets.router.get('/nonWSRoute', doSomething1); // processed as HTTP GET request
routerWithWebSockets.router.get('/wsRoute1', doSomething2); // processed as HTTP GET request
routerWithWebSockets.ws('/wsRoute1', (ws) => doSomethingWithWS1); // processed as Websocket upgrade request
routerWithWebSockets.ws('/wsRoute2', (ws) => doSomethingWithWS2); // processed as Websocket upgrade request
export default routerWithWebSockets.router;
I have a local (Angular) client running on port 4200 (http://localhost:4200) and a local (express) server on port 5000 (http://localhost:5000). Whenever I try to connect to my server, I get this message.
Access to XMLHttpRequest at 'http://localhost:5000/socket.io/?EIO=4&transport=polling&t=NU7H' from origin
'http://localhost:4200' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.
Here is the code to start my local server
#injectable()
export default class App {
app: express.Application;
constructor() {
this.app = express();
this.config();
this.bindRoutes();
}
// Middlewares config
private config(): void {
this.app.use(cors());
this.app.use(express.json());
this.app.use(express.urlencoded({ extended: true }));
}
bindRoutes(): void {
this.app.use('/', router);
}
}
Here is the code where I set up my socket
private _ioServer: SocketIO.Server;
initSocket(server: http.Server) {
this._ioServer = new SocketIO.Server(server);
this.connectChat(); // Chat namespace
this.connectStream(); // Game board streaming namespace
}
I tried with Postman, everything is working.
Thank you!
Any malicious site can take advantage of your cookies stored in the system called Cross-site request forgery
Any browser tries to prevent you from these attacks so they disable CORS.
Shorthand Fix [Not recommended] : There are many plugins out there you can use for your local testing that disables these checks on browser.
Proper Fix: Use an Express middleware to apply Access-Control-Allow-Origin: * in your header when response is returned back from the server.
Gist is that when browser sends the request to your server it will append Origin: http://localhost:3000 to the headers. Reacting to this request from browser, server should return a Access-Control-Allow-Origin header to specify which origins can access the server's resources.
You can be strict here to return Access-Control-Allow-Origin: http://localhost:4200 or open your gates by sending Access-Control-Allow-Origin: *.
Here is the quick code to have an express middleware:
const express = require('express');
const request = require('request');
const app = express();
app.use((req, res, next) => {
res.header('Access-Control-Allow-Origin', '*');
next();
});
app.get('/jokes/random', (req, res) => {
request(
{ url: 'https://joke-api-strict-cors.appspot.com/jokes/random' },
(error, response, body) => {
if (error || response.statusCode !== 200) {
return res.status(500).json({ type: 'error', message: err.message });
}
res.json(JSON.parse(body));
}
)
});
const PORT = process.env.PORT || 3000;
app.listen(PORT, () => console.log(`listening on ${PORT}`));
Source: https://medium.com/#dtkatz/3-ways-to-fix-the-cors-error-and-how-access-control-allow-origin-works-d97d55946d9
P.S, this is a very good read for your understanding of CORS.
In the index.js file of your middleware add:
app.use(cors())
I have a very simple user backend up and running (node, express, mongoose, mongo, etc) and with postman can verify when I add a user it works, when I request a login it works and get a token, and if I put in the wrong details it rejects it,
Now I used this git hub repo https://github.com/christiannwamba/vue-auth-vuex to spin up a simple frontend for this. Which I thought was all working fine as it appeared to be logging in until I found it was accepting whatever details I put in for the email and password as correct!
The backend server kept responding ok when I hit it with the vue app, but on closer inspection when I console logged what it was getting, which was null and returning user not found. So again I don't think there is anything wrong here.
Something I have noticed in chrome dev tools network, it is sending two versions of authenticate, first is empty and then the next one has responses.
I'm at a bit of a loss why it's sending empty requests first time and why it allows the login when it's getting a bad return.
Server.js file:
const express = require('express');
const logger = require('morgan');
const movies = require('./routes/movies') ;
const users = require('./routes/users');
const bodyParser = require('body-parser');
const mongoose = require('./config/database'); //database configuration
var jwt = require('jsonwebtoken');
var cors = require('cors')
const app = express();
// Add cors
app.use(cors());
app.options('*', cors()); // enable pre-flight
app.set('secretKey', 'nodeRestApi'); // jwt secret token
// connection to mongodb
mongoose.connection.on('error', console.error.bind(console, 'MongoDB connection error:'));
app.use(logger('dev'));
app.use(bodyParser.urlencoded({extended: false}));
app.get('/', function(req, res){
res.json({"api" : "User API"});
});
// public route
app.use('/users', users);
// private route
app.use('/movies', validateUser, movies);
app.get('/favicon.ico', function(req, res) {
res.sendStatus(204);
});
function validateUser(req, res, next) {
jwt.verify(req.headers['x-access-token'], req.app.get('secretKey'), function(err, decoded) {
if (err) {
res.json({status:"error", message: err.message, data:null});
}else{
// add user id to request
req.body.userId = decoded.id;
next();
}
});
}
// express doesn't consider not found 404 as an error so we need to handle 404 it explicitly
// handle 404 error
app.use(function(req, res, next) {
let err = new Error('Not Found');
err.status = 404;
next(err);
});
// handle errors
app.use(function(err, req, res, next) {
console.log(err);
if(err.status === 404)
res.status(404).json({message: "Not found"});
else
res.status(500).json({message: "Something looks wrong :( !!!"});
});
app.listen(3000, function(){
console.log('Node server listening on port 3000');
});
Update:
I have added in under my CORS bit in server.js:
app.options('/users/authenticate', function(req, res){
res.header('Access-Control-Allow-Origin', '*');
res.header('Access-Control-Allow-Methods', 'POST');
res.end();
});
In network I now only get the one request. Under form data it appears to be there but it's saying in the response that data is null, and even more odd the vuejs is still logging in and allowing access to the restricted pages.
Temporarily comment out the line where you set the headers to application/x-www-form-urlencoded. Then add app.use(bodyParser.json()) to your server.js and see if it works. What's happening is your request object is malformed, which is why the server cannot parse the request correctly.
Looks like CORS issue. If you run UI using a different server and your backend is running by itself, then your browser will send pre-flight request first which is an options request. That is the reason you see 2 authenticate requests in the developer tools. You can read more about this here
Why is an OPTIONS request sent and can I disable it?
I'm looking for a clean way to have my express app return 405 Method Not Allowed if a client sends a request that matches a mapped url route but does not match the mapped HTTP method.
My current implementation is to have a default "catch-all" handler that tries to match the url against the register routes, ignoring the HTTP method. If there is a match, then we know to return a 405, otherwise we let express do its default 404 behavior.
I'm hoping there is a better way that doesn't involve running all the route matching twice (once by express, once by my handler).
Here is an approach that I have used successfully with multiple Django applications and now with Node and Express. It is also follows RFC 2616 (HTTP/1.1) that says the following about HTTP 405:
The response MUST include an Allow header containing a list of valid
methods for the requested resource.
So, the key point is to route the requests to the same handler without regard to methods.
app.all('/page/:id', page.page);
app.all('/page/:id/comments', page.comments);
app.all('/page/:id/attachments', page.attachments);
...
The next point is to validate the method in the handler function 'comments'. Note that the handler is responsible for handling all the methods. In Django's world this is the only way to go because the framework forces you to separate the routing of the URLs from the actual action about to be performed against the resource the URL represents.
In the handler you could check the method like this...
exports.comments = function (req, res) {
if (req.route.method === 'get') {
res.send(200, 'Hello universe.');
} else {
res.set('Allow', 'GET');
res.send(405, 'Method Not Allowed');
}
}
...but as you can expect the code will quickly become repetitious and not nice to read especially when you have many handler functions and many different sets of allowed methods.
Therefore I prepared a shortcut function named restful for the job. Define the function wherever you want. I personally would place it in helpers.js under the same directory where the handler functions are implemented.
var restful = function (req, res, handlers) {
//
// This shortcut function responses with HTTP 405
// to the requests having a method that does not
// have corresponding request handler. For example
// if a resource allows only GET and POST requests
// then PUT, DELETE, etc requests will be responsed
// with the 405. HTTP 405 is required to have Allow
// header set to a list of allowed methods so in
// this case the response has "Allow: GET, POST" in
// its headers [1].
//
// Example usage
//
// A handler that allows only GET requests and returns
//
// exports.myrestfulhandler = function (req, res) {
// restful(req, res, {
// get: function (req, res) {
// res.send(200, 'Hello restful world.');
// }
// });
// }
//
// References
//
// [1] RFC-2616, 10.4.6 405 Method Not Allowed
// https://www.rfc-editor.org/rfc/rfc2616#page-66
//
// [2] Express.js request method
// http://expressjs.com/api.html#req.route
//
var method = req.route.method; // [2]
if (!(method in handlers)) {
res.set('Allow', Object.keys(handlers).join(', ').toUpperCase());
res.send(405);
} else {
handlers[method](req, res);
}
}
With restful it is now quite painless to handle 405 responses automatically and having proper Allow header being set. Just give a function for each method you allow and restful does the rest.
So lets modify the previous example:
exports.comments = function (req, res) {
restful(req, res, {
get: function (req, res) {
res.send(200, 'Hello restful universe.');
}
});
}
Why the name restful? In RESTful web it is quite essential for the API to obey the conventions like responsing with HTTP 405 to the request having non-supported method. Many of those conventions could be integrated to restful when needed. Therefore the name is restful and not something like auto405 or http405handler.
Hope this helps. Any thoughts?
Method 1: Use .route() and .all()
// Your route handlers
const handlers = require(`./handlers.js`);
// The 405 handler
const methodNotAllowed = (req, res, next) => res.status(405).send();
router
.route(`/products`)
.get(handlers.getProduct)
.put(handlers.addProduct)
.all(methodNotAllowed);
This works because requests are passed to the handlers in the order they are attached to the route (the request "waterfall"). The .get() and .put() handlers will catch GET and PUT requests, and the rest will fall through to the .all() handler.
Method 2: Middleware
Create middleware which checks for allowed methods, and returns a 405 error if the method is not whitelisted. This approach is nice because it allows you to see and set the allowed methods for each route along with the route itself.
Here's the methods.js middleware:
const methods = (methods = ['GET']) => (req, res, next) => {
if (methods.includes(req.method)) return next();
res.error(405, `The ${req.method} method for the "${req.originalUrl}" route is not supported.`);
};
module.exports = methods;
You would then use the methods middleware in your routes like this:
const handlers = require(`./handlers.js`); // route handlers
const methods = require(`./methods.js`); // methods middleware
// allows only GET or PUT requests
router.all(`/products`, methods([`GET`, `PUT`]), handlers.products);
// defaults to allowing GET requests only
router.all(`/products`, methods(), handlers.products);
Due to ambiguity, there really is no other way. Personally, I would do something like this:
var route = '/page/:id/comments'
app.get(route, getComments)
app.all(route, send405)
function send405(req, res, next) {
var err = new Error()
err.status = 405
next(err)
}
Either way, you have to check the routes twice.
Kinda old question but here is what i did. I just put this after all my routes but before my 400 handler
// Handle 405 errors
app.use(function(req, res, next) {
var flag = false;
for (var i = 0; i < req.route.stack.length; i++) {
if (req.method == req.route.stack[i].method) {
flag = true;
}
}
if (!flag) {
err = new Error('Method Not Allowed')
err.status = 405;
return next(err)
}
next();
});
I have been doing it this way:
Say if you have GET and POST method handlers for /. You can wrap the path with app.route or router.route and assign the handlers accordingly.
app.route("/").get((req, res) => {
/* DO SOMETHING*/
}).post((req, res) => {
/* DO SOMETHING*/
}).all((req, res) => {
res.status(405).send();
});
http://expressjs.com/en/4x/api.html#app.route
http://expressjs.com/en/4x/api.html#router.route
A request will get matched to the route and filtered through the handlers. If a handler is present, it will get handled as usual. Else, it will reach the all handler that will set the status code to 405 and ending the request.
I fixed it like this :
/*paths here*/
router.get('/blah/path1', blah.do_something );
router.post('/blah/path2', blah.do_something_else );
/* if we get here we haven't already gone off down another path */
router.all('/*', (req,res) => { res.status(405),
res.json({'status':405,
'message':req.method + ' not allowed on this route'})
});
/* simples */
I thought this was a pretty interesting problem and so I dove deeeeeep down into the depths of the express app function and found a way to dynamically build a 405 error that includes all of the possible routes (without having to manually update anything when you add a new route).
app.use("", (req, _, next) => {
const err = buildError(app, req);
if (!err) return next();
return next(err);
});
For those interested you can find the npm package here https://www.npmjs.com/package/express-ez-405, and below is a quick example of what it looks like to use it.
const express = require("express");
const { buildError } = require("express-ez-405");
const app = express();
app.use(express.json());
const userRouter = require("./routes/user");
const mainRouter = require("./routes/main");
const nestedRouter = require("./routes/nested");
// // Routes
app.use("/main", mainRouter);
app.use("/user", userRouter);
app.use("/nested/route", nestedRouter);
// Routes
// 405 & 404 error catcher
app.use("", (req, _, next) => {
const err = buildError(app, req);
if (!err) return next();
return next(err);
});
// 405 & 404 error catcher
// Error handling
app.use((err, _, res, __) =>
res.status(err.status).json({ message: err.message })
);
// Error handling
const PORT = 3000;
app.listen(PORT, () => {
console.log(`Server listening on port: ${PORT}...`);
});
Once this in in there you never have to worry about updating it regardless of whether you add, remove, or change routing.