When I search for packages on NPM, I would like to see package sizes (in KB or MB, etc). NPM doesn’t seem to show this information.
How can I determine how much bloat an NPM package will add to my project?
What you probably want to measure is the impact a package has if you were to add it to your app bundle. Most of the other answers will estimate the size of the source files only, which maybe inaccurate due to inline comments, long var names etc.
There is a small utility I made that'll tell you the min + gzipped size of the package after it gets into you bundle -
https://bundlephobia.com
Take a look at this cost-of-modules project. It's an npm package that will list the size of a package and number of children.
Installation:
npm install -g cost-of-modules
Usage:
Run cost-of-modules in the directory you are working in.
I created Package Phobia early this year with the hope to get the package size information into npmjs.com and also track package bloat over time.
https://packagephobia.com
This is designed to measure disk space after you run npm install for server-side dependencies like express or dev dependencies like jest.
You can read more about this tool and other similar tools in the readme here: https://github.com/styfle/packagephobia
Update 2020
The "Unpacked Size" (basically Publish Size) is available on the npmjs.com website along with "Total Files". However, this is not recursive meaning that npm install will likely be much bigger because a single package likely depends on many packages (thus Package Phobia is still relevant).
There is also a pending RFC for a feature which prints this information from the CLI.
In case you are using webpack as your module bundler have a look at:
webpack-bundle-analyzer
webpack-bundle-size-analyzer
I definitely recommend the first option. It shows size in interactive treemap. This helps you to find the size of package in your bundled file.
The other answers in this post show you size of the project, but you might not be using all parts of the project, for example with tree shaking. Other approaches then might not show you accurate size.
I've created a tool, npm download size, which inspects tarball size for a given npm package, including all tarballs in the dependency tree. This gives you an idea of the cost (install time, disk space, runtime resources, security audit, ...) of adding the dependency up front.
In image above, Tarball size is tar.gz of package, and Total size is size of all tarballs. The tool is pretty basic, but it does what it says.
A cli tool is also available. You can install it like this:
npm i -g download-size
And use it like this:
$ download-size request
request#2.83.0: 1.08 MiB
The source code is available on Github: api, cli tool and web client.
Try to use package-size.
npx package-size vue,vue-router,vuex react,react-dom,react-router,redux
https://github.com/egoist/package-size
howfat is one more tool which can show total package size:
npx howfat jasmine
If you use Visual Studio Code, you could use an extension called Import Cost.
This extension will display inline in the editor the size of the imported package. The extension utilizes webpack with babili-webpack-plugin in order to detect the imported size.
You could check out npm-module-stats. It is an npm module that gets the size of an npm module and its dependencies without installing or downloading the module.
Usage:
var stats = require("npm-module-stats");
stats.getStats("glob").then((stack) => {
let dependencies = Object.keys(stack);
let totalSize = dependencies.reduce((result, key, index) => {
return result + stack[key].size;
}, 0);
console.log('Total Size in Bytes ', totalSize);
console.log('Total Dependencies ', dependencies.length-1);
}).catch((err) => {
console.error(err);
});
It might seem a little verbose but it solves the problem you described appropriately.
Before publishing the npm package, You can check the size of the package using the following command.
npm publish --dry-run
I have attached the result of my npm package.
A "quick & dirty" way is to use curl and wzrd.in to quickly download the minified package and then grep the file size:
curl -i https://wzrd.in/standalone/axios#latest | grep Content-Length
The download is minified but not gzipped, but you get a good idea of the relative size of packages when you compare two or more of them.
I prefer https://github.com/aholachek/bundle-wizard all the way since it was released.
It works on deployed sites: npx bundle-wizard reddit.com
It works on your local project:
For multi-page apps/sites adjust the last line with the path you want to check.
npm run build
npx serve -s build
npx bundle-wizard localhost:5000/
The interactive view is really helpful in discovering what's where.
In order to check the impact of different packages on your bundle. You can check out source-map-explorer.
Install:
npm install -g source-map-explorer
Usage:
source-map-explorer bundle.min.js
source-map-explorer bundle.min.js bundle.min.js.map
source-map-explorer bundle.min.js*
source-map-explorer *.js
This will open up a visualization of how space is used in your minified bundle.
I created bundlejs for this exact reason. Bundlejs was designed to be a fast and always accessible online bundler that can treeshake, analyze the bundle, minify, gzip & brotli the bundle, give the compressed size, and also return the fully bundled code, you can check it out at https://bundlejs.com.
The source code is available on GitHub at https://github.com/okikio/bundlejs
However, if you're goal is to just check what the npm install size for packages, I highly recommend https://packagephobia.com. I didn't create packagephobia but it's a high quality tool that covers the other niche when it comes to checking js package sizes
Related
I am getting rid of lodash, so some stuff I am redoing in es6 and other like this one I would like to keep but just this one function.
I see there is this npm https://www.npmjs.com/package/lodash.minby but im not sure if i install this it will only install this, or the whole library, can anyone please confirm?
Thanks.
From the documentation:
The lodash method _.minBy exported as a Node.js module.
So, yes, it just installs this specific function from lodash
Yes, if you install lodash.minby, at the time of this writing, you will get version 4.6.0 which contains a package.json file, a README.md file, a LICENSE file, and an index.js file. It has no other dependencies, so that is all you will get. The index.js file is over 2000 lines long, but that's what it takes apparently to implement the functionality.
In contrast, at the time of this writing, npm install lodash will give you version 4.17.21 which contains over 44000 lines of JavaScript spread out across many files. So that's about 22 times as large.
For large web apps npm install resp. yarn install does take a lot of time, mostly in a step called Linking Dependencies. What is happening here? Is it fetching the dependencies of the dependencies? Or something completely different? Which files are created during this step?
When you call yarn install, the following things happen in order:
Resolution: Yarn starts resolving dependencies by making requests to the registry and recursively looking up each dependency.
Downloading/Fetching: Next, Yarn looks in a global cache directory to see if the package needed has already been downloaded. If it hasn't, Yarn fetches the tarball for the package and places it in the global cache so it can work offline and won't need to download dependencies more than once. Dependencies can also be placed in source control as tarballs for full offline installs.
Linking: Finally, Yarn links everything together by copying all the files needed from the global cache into the local node_modules directory after identifying what's already there and what's not there.
yarn install does take a lot of time, mostly in a step called Linking Dependencies
You should notice that Step 3: Linking is taking more time than Step 1: Resolution and Step 2: Fetching where the actual download happens. During by this step we already have things that we need ready and downloaded, then why is it taking long, did we miss anything?
Yes, COPY to local project into node_modules folder...! The reason for this is that this copy is not equivalent to copying one large 4.7GB ISO file. Instead it's multiple super small files (Don't take it light when I say multiple, it can be 15k+ files :P ), hence take a lot of time to copy. (Also, it is important to note that when you download the packages, you download one large tar file per package, whose contents should then be extracted into the cache which also takes time)
It is slower due to
Anti-virus: Your antivirus is sitting in the middle and doing a quick inspect (in addition to our yarn checking if it already exists) on every single file yarn is trying to copy cutting its speed by so much. If you are on Windows, try adding your project's parent folder as exception to Windows Defender.
Storage medium's transfer rate: SSDs can improve this speed hugely (Sorry, SSHDs and FireCudas will not help either, this is gonna be one time).
But is this efficient? Can I have it taken from the global node_modules (after creating one)?
Nope for both questions. Because of the way node works each package finds its dependencies only relative to its own location. Also because each project may want to use different versions of the same package to ensure its working properly and not broken by package updates.
Ideally, the project folder should be lean. An efficient way of doing this would be to have a global node_modules folder. Any and all requested packages are downloaded if not already present AND used from this location. Actually Ruby does it this way. Here's my global Ruby's equivalent of node_modules folder. Notice the presence of different versions of the same package for use in different projects.
But keep in mind that it would reduce project portability. It's a trade-off that any manager (be it rubygems or node modules) has to make. I can just copy the node project folder (which in fact may take hours because you will be copying the (local) node_modules folder as well, but I can expect it to work if I have just that project folder, as opposed to copying a ruby project would only some seconds to few minutes, as there is no local packages (or gems as they call them) folder, but running the project on different system would require those packages to be present on the global gems folder.
The documentation for yarn install can be found here.
You can use the command
yarn install --verbose
Show additional logs while installing dependencies
The output will show what the yarn/npm install is doing.
It's good for debugging in case the process is failing or taking a long time.
The linking phase works in essentially 3 big steps:
Find every file that need to be in node_modules
Check this list versus what is already there and find what need to be copied around
from cache to node_modules
Do the copy
Maybe this issue on Github will help you out.
https://github.com/yarnpkg/yarn/issues/1496
I'm having interesting issue and I can't point my finger on the cause. Here is the story.
My goal is to manage Lambda functions with Terraform, using CircleCI as orchestrator. Process looks like this:
GitHub repository -> CircleCI build (npm install && zip) -> aws s3 sync ZIP to S3 -> Terraform picks up ZIP from S3 -> if SHA is different, deploy to Lambda.
It all works pretty well with one little problem. If I run the whole process twice, with no change in GitHub repository, Terraform always reports new SHA hash:
source_code_hash: "mx56e7pMbBdftYHes+pETvjFZInvKsP92vtZ5URqyJY=" => "dhe66opfifDLEr7wI+Ie9UE6ioUiG8hTajuPhctA+W0="
It seems the issue lies somewhere around npm install process, which perhaps do always something slightly different, thus result is also different. I don't think, that ZIPing folder should cause troubles, it's simple zip -r file.zip *.
Any explanation of this behaviour, please?
npm install with versions 4 or less is non-deterministic so there is no guarantee that your npm install in different builds will be the same.
npm solved it with version 5's package-lock.json. Try it again using the latest version of npm.
Or, you can try yarn or pnpm. Both are alternative package managers that aim to use deterministic installs.
I am new, and when i first created my app based on the documentation using npm create-react-app i found it there were a lot of package included inside folder node_module when i code and i only use react and react DOM and etc from the basic.
node_modules
acorn
timer
ansi
and many more
I wonder if anyone can help my how to understand each use inside the node_module or where can i find the documentation for each use?
or how can i just reduce to what i want to use only to decrease the app size?
The answers are 2:
because you're using an automated scaffolding tool, which essentially does everything for you, and, you have just to code, it is supposed to locally deploy all the packages it needs to work (for example webpack is needed to bundle your code, babel to transpile it, ...
under node_modules you will find all the packages of the whole app. That's means you will find both your dependencies and the dependencies of your dependencies (this rule has some exceptions and you can find them in the npm documentation.
example:
// your code depends on A
var dependency = require('A');
// but then, inside your A dependency you can also find something similar to:
var b = require('B');
how can i just reduce to what i want to use only to decrease the app size?
You basically can't do it. They are all needed.
Most of the libraries that we pull from npm have dependencies. You may use only react and react-dom but there are react-scripts that require lots of stuff. I don't think that you have to worry about the size of the node_modules. That's not what you are suppose to ship in production.
If you want to see what are these all modules about you may open their folder and fine README.md file.
I am using Browserify and Grunt to bundle our RendrJS webapp. I would like to analyze the bundle to see which modules it contains to see if any code is superfluous. This can happen since in Rendr some js code might only run on the server.
How do I get a list of the contents of the bundle. I have tried:
browserifyOptions: { 'list': true }
but it doesn't seem to have any effect. We are using Browserify 2.
Saw your question, dug up a tiny old script and just now pushed browserify-inspect-bundle to npm, grab it here:
https://www.npmjs.com/package/browserify-inspect-bundle
Usage is:
npm install -g browserify-inspect-bundle
browserify-inspect-bundle /path/to/your/bundle.js
This should get you something useful.