How does ExpressJS determine if a requested resource is a "static" file? - express

I'm using JSPM to manage my client side dependencies and serving files using ExpressJS
My Directory structure is
node_modules
routes
views
app.js
public
css
images
js
main.js
jspm_packages
system.js
npm
angular2#2.0.0-beta.7.js
I have static route setup in my Express app.js as follows:
app.use(express.static(path.join(__dirname, 'public')));
As expected when I request for GET /jspm_packages/system.js
it serves the file correctly
however when I request GET /jspm_packages/npm/angular2#2.0.0-beta.7.js
It gives me a 404 - not found.
I suspect some of those special characters in the file name are messing up express from resolving the request as a "static" file and using the correct static route.
How can I test if express is marking the request as "static"?
How can I overwrite the express regex (or whatever mechanism) express is using to mark a request as "static"?
How can I write a custom middleware using my own regex and forward the request to static instead?
thanks.

When working with express, you must make sure right middleware is registered in right order.
As per your question, How does express identifies a resource as static?, Actually express does not determine if its static or not, it does not even understand request types, what express does is execute proper middleware for given request.
When express receives a request, It goes and starts executing matching middleware in sequence until it runs out of them.
So in practice, you'll always register your static middleware first (just after request parsers and all), before your dynamic routes. Like shown below ...
var express = require('express');
var path = require('path');
var cookieParser = require('cookie-parser');
var bodyParser = require('body-parser');
var routes = require('./routes/index');
var app = express();
// view engine setup
app.set('views', path.join(__dirname, 'views'));
app.set('view engine', 'jade');
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: false }));
app.use(cookieParser());
//Register your static middleware
app.use(express.static(path.join(__dirname, 'public')));
//Other cool code
app.use('/', routes);
Now, request will first go through express.static middleware, if it does not find a matching file, it will call next internally and pass request to next middleware in chain.
Meaning, suppose you have a static file in public directory named users and you have a route named routes.get('/users' ..... Now when user requests /users, the request will first pass to express.static and if it finds users file (which it does in this case), our route that we registered using routes.get will never get called. Now if you do the same but just remove the file named users from the public folder, then express.static middleware won't be able to find a matching file and will pass request to next middleware in chain i.e. routes.get(/users' ...`
Express does not assume or identify a resource as static by matching or using regular-expressions. If express.static middleware finds it, it will serve it else it will pass on the request to the next middleware in queue.
Though I was pretty sure, I tried to reproduce your issue, and I was served with the file with all the special characters in it, just as shown in following image.
Please reconfirm following:
Your express.static middleware gets registered first.
The resource you are trying to access exists, i.e. physical path exsists. (resource url are case insensitive, i.e. /USERS and /users both will match a file /public/users if it exists.
Make sure you don't have a typo.
If this does not resolve your issue, please share your app.js file, some content if you can't share all of it.
Hope this helps! Let me know if you need further assistance ... :)

Related

Next.js serving static files that are not included in the build or source code

I have files that are not stored in a CDN and would like to serve them with Next.js. These files are not intended to be integrated into Next.js and should not be placed in the public folder. More files will be added and I want to avoid using a custom Next.js server to do simple file serving for images that are not available during building. Additionally, this application will only be deployed locally and using a CDN is overkill for this situation.
Currently, I use Express.js and a Next.js custom server to use express.static to serve files, but this ends up slowing down Next.js and adds lots of unnecessary complexity to my stack. I'd rather just use the Next.js CLI to run my app instead of reinventing the wheel.
Is there a simple way I can serve static files within Next.js and outside the public directory?
I posted this question and my own answer here on StackOverflow because I was unable to find a good tutorial on how to do this. Nearly every google search says to use a custom server or to just put your files in the public folder, which is not what I was looking for. Hopefully, others who are looking for the same thing may find it here.
Disclaimer: I do not use Vercel to publish my applications, and I do not know if this answer will be applicable to Next.js on Vercel.
Next.js allows API routes to be customized to support Node.js HTTP handlers, which means express can also be used within Next.js API routes.
Here is some code to utilize express.static on a Next.js API route.
// pages/api/images/[name].js
// Tell Next.js to pass in Node.js HTTP
export const config = {
api: { externalResolver: true }
}
import express from 'express';
const handler = express();
const serveFiles = express.static('./path/to/files');
handler.use(['/api/images', '/images'], serveFiles);
// ^ ^
// Multiple endpoints are passed. The first one is used when visiting /api/images.
// The second one is used when visiting /images using the middleware rewrite I mention below.
// express is just a function that takes (http.IncomingMessage, http.ServerResponse),
// which Next.js supports when externalResolver is enabled.
export default handler;
However to get around visiting this endpoint via /api/images/filename, you can use Next.js's new middleware to rewrite the request!
// pages/images/_middleware.js
import { NextResponse } from 'next/server';
export function middleware(req) {
// Rewrite /images/... to /api/images/...
return NextResponse.rewrite('/api' + req.nextUrl.pathname);
}
With both these in use, visiting /images/photo.png will internally rewrite to /api/images/photo.png and in turn be handled by express.static, allowing you to serve files outside an API route and without using a custom server!
This code can surely be simplified and get rid of the need of initializing a express.js app just to handle a request, but its incredibly simple to integrate express.js into next.js without using a custom server!
I posted this question and my own answer here on StackOverflow because I was unable to find a good tutorial on how to do this. Nearly every google search says to use a custom server or to just put your files in the public folder, which is not what I was looking for. Hopefully, others who are looking for the same thing may find it here.
The public folder can only serve those files that were included at build time.
But we can do some workaround that can serve files that were not included at build time.
Solution starts here
We can create an api endpoint. For example /api/images-endpoint/[...slug].js
import fs from "fs";
import path from "path";
export default function handler(req, res) {
const imagePath = req.query.slug.join("/");
const filePath = path.resolve(".", `images-directory/${imagePath}`);
const imageBuffer = fs.readFileSync(filePath);
res.setHeader("Content-Type", "image/jpg");
return res.send(imageBuffer);
}
By this, our endpoint will read the image from the image directory and send it as a response.
Benifit/Note: This solution works for images that were added after Next project is build i-e npm run build or next build
Drawback: Using this, We can not build optimized images in Next JS Image component i-e next/image

Returned data being rendered instead of page structure in Nuxt

I'm trying to return data as JSON from the express server for a route. The data returns fine but when i open the NUXT page on the browser than the JSON data gets outputted instead of the page HTML.
Note the express route is the same as the page route. I know the routes are conflicting with each other. Do i need to have the server and front-end on different ports? Is there anything wrong i'm doing here?
Thanks
To avoid conflicts such as that you should use a prefix like /api/ or /api/v1/ something like that
In nuxt.config.js you need to define your server middleware
serverMiddleware: ["~/api/index.js"]
That file is your server. At the bottom you need to export it like this:
module.exports = {
path: "/api",
handler: app
}
Note here: app is your express app if you use express.js.
This here: const app = express();
If everything worked your root of your API should be available under host:port/api/
you cant do this if the routes for backend and frontend exactly same. this is route rules that they have to be unique and its not backend or frontend issue for e.x. you can have two routes with same url in express(api), nuxt too.
if we say the application for example is post office, the route are path to a house address (controller or action) so we can have two path to get the a house but its confusion have a same path(url or route) and different houses.
simple solutions:
as you said make the api and front separate with different ports or different domains or even have a prefix for your express routes
in express handle all of them, means return view or page with data needed instead of json data

prevent handling of missing files by other use/get in express

I'd like to keep the contents of the folder /public for use of local files by a local app. However, when a file is missing, instead of getting the 404, the user is getting the index.html page.
What I would like to do is have user get the 404 when accessing any resource that does not exist for anything under /public/*, but have the react app handle everything else from index.html
this is my setup:
app.use('/public', express.static(path.resolve(myPath, 'public')));
app.get('*', (req, res) => {
res.sendFile(path.resolve(myPath, 'index.html'));
}
should the get include a regex to not include public matches, or is there a way to handle this with use?
The problem is that you've explicitly told express to look into your public folder for ANY file that the user requests with this line:
app.use('/public', express.static(path.resolve(myPath, 'public')));
Express, by default, serves NO files. It makes nothing public by default. So, the first thing to do is to remove the line of code that makes that entire directory public. And, then if you want to serve some specific things from that folder, you need to either make very specific routes to only the files you want to be public or you need to move the "public" files out to a directory where everything can be public and then point express.static() at that directory.
Then, you've added another line to respond to any request possible with one file:
app.get('*', (req, res) => {
res.sendFile(path.resolve(myPath, 'index.html'));
}
This is just not really how you should be using express at all. Here are some steps to think through:
Figure out which static files you want Express to serve automatically (without a specific route being made for each file).
Then, organize those files so they are in their own sub-directory on your server hard drive.
Then, you can point express.static() at that directory without any fear of it serving files you don't want to be automatically public.
Think about organizing things into a hierarchy (not required, but sometimes simpler to manage) so that css files might be one place, client-side js another place, etc... such as /css/css files here and /js/js files here when laying out your hierarchy. Then, you can control the serving of each type of file separately if wanted and it may make maintenance easier (since separate people often maintain CSS files and JS files).
Then, design specific routes you want handled for other types of files.
Then, add an error handler route which determines what should be returned to the browser when no other route handler was found. How to do that is described here. Also, note that Express has a different default error handler based on whether the NODE_ENV environment variable is set to production or not.
Don't use * routes that handle everything with the same content, including things you don't want to provide a specific page for. You don't want search engines to index things that you don't have original content for and you don't want users to bookmark unintended URLs just because you happen to be using * in a route handler.
I was able to resolve it by adding
app.get('/public/*', (req, res) => {
res.status(404).send(req.path + ' not found');
})

Avoiding middleware for static content in express app

I'm developing a small web app using express framework and I'm having a tough time with static content configuration.
In my app I have several custom middleware functions and I don't want the static files (css, js, images,...) to be processed by them. I tried to put static configuration as the first middleware but it doesn't fix anything.
My code is something like this:
var app = express();
app.configure(function() {
app.set('port', 3000);
app.set('views', __dirname + '/views');
app.use(express.static(path.join(__dirname, 'public')));
app.use(myCustomMiddleware());
});
I log the requests that reach my custom middleware and I can see requests to css and js (files inside public folder) getting there.
My goal is to return as soon as possible when receiving a static file request.
Any tips on this?
Thanks.
I'm closing this question as the problem is my own code.
Trying to isolate the components I found that I was modifying incoming request URL unintentionally.

Using Swagger for Express alongside normal routes

I'm interested in using something like swagger-node-express to build the API for the project I'm working on. However, some parts of the app are non-api based (serving HTML instead). Has anyone got any ideas as to how I would use Swagger alongside normal routes?
You'll just need to make sure that your routes don't conflict with the Swagger routes, and your other routes will processed as usual. One easy way would be to make Swagger live under a subpath. Take a look at the docs on that score:
https://github.com/wordnik/swagger-node-express
var app = express();
var subpath = express();
app.use(express.bodyParser());
app.use("/v1", subpath);
swagger.setAppHandler(subpath);
Otherwise, you could just keep an eye out to make sure that none of the other URLs you are using in your application conflict with the Swagger URLs, and you should be able to define your routes and handlers normally. E.g. you can use Swagger to serve up docs under http://localhost:8002/api-docs.json/pet but have http://localhost:8002/foo/ do something else just by adding the route in the normal way:
app.get(/foo/, function(req, res, next) {
res.writeHead(200, {'Content-Type': 'text/plain'});
res.end('Hello Foo\n');
})
Using the subpath configuration is probably the cleanest approach but you could also just keep track of the routes yourself and make sure that the Swagger routes aren't conflicting with the routes in the rest of your app.