Install subset of dependencies with npm - npm

Is there a way to specify subsets of dependencies in npm, with an alias or "feature tag"? That is, if someone knows that they will only be using some limited subset of the features of my package, they can specify those features and, on npm install, only download the dependencies relevant to those features?
My package has a very large number of dependencies and takes nearly half an hour to install, but most users only need a subset of their functionality. I'm thinking of something like how dependencies can be divided into devDependencies and dependencies, but with n groups instead of just those two. For example:
npm install --feature feature1 --feature feature2
From reading the docs, I think the answer here is "no", but what would be your suggestion for this case? Split the package into smaller plugin packages and have users install the plugins that they want? I don't want something that is too complicated for users to configure.

The short answer is no, npm was not designed for this mostly because dependency trees are incredibly large, and this use case could really just complicate things for most users. That being said, I wrote a package to do it.
My package is install-subset, and can be installed globally with npm install -g install-subset
https://www.npmjs.com/package/install-subset
Essentially you build inclusion lists and exclusion lists for named subsets in your package.json like this:
"subsets": {
"build": {
"include": [
"babel-cli",
"dotenv"
]
},
"test": {
"exclude": [
"eslint",
"lint-rules",
"prettier"
]
}
}
Then call it with, for example, install-subset test
This will temporarily rewrite your package.json to not install those packages excluded, then restore it, which depending on the packages can save a lot of time and bandwidth.
Also works with yarn, is open source and issues/PRs are welcome.

Another third-party package that you can use to do this is group-dependencies. It allows you to define [GROUP_NAME]Dependencies arrays (note: not objects) in your package file, then install just that subset using deps install [GROUP_NAME].
Here's an example from their README:
{
...
"devDependencies": {
"intercept-stdout": "^0.1.2",
"jest": "^20.0.4",
"strip-color": "^0.1.0"
},
// our new group representing testing dependencies
"testDependencies": [
"jest"
]
...
}
Now you can install only the dependencies for this new group:
# This will install jest#^20.0.4:
$(npm bin)/deps install test

Related

Dependency resolution in NPM

Consider the following:
A project has a dependency for package X #1.2.3
A project may need to install package Y.
Package Y has a dependency for package X #2.3.4
The version 2 of package X is not backwards compatible with version 1.
Can this cause issues of any kind?
If you have npm installed them in the default way, your project is safe from dependency conflicts.
Just to be sure, look for those dependencies in the package-lock.json file and you should immediately notice that certain package dependencies are managed differently from project dependencies. In particular, this is a simplified entry created when you install X#1.2.3.
"node_modules/X": {
"version": "1.2.3",
...
"dependencies": {
...
}
}
Then you install Y, that depends on X#2.3.4.
"node_modules/Y": {
"version": "x.y.z",
...
"dependencies": {
"X": "2.3.4"
},
}
However, since X#2.3.4 has not been installed yet, a new entry is automatically created:
"node_modules/Y/node_modules/X": {
"version": "2.3.4",
...
"dependencies": {
...
}
},
As you can see, npm detects the potential dependency conflict: instead of registering the package in the global node_modules folder, npm installs it inside the node_modules folder of the package installed. You could have more than 2 nested level of node_modules if there are conflictual dependencies inside node_modules/Y/node_modules too.
I showed you an example of package-lock.json because understanding dependency management is straightforward looking at it, even if big. Nonetheless, in case you do not use it (I hope you do), npm reasons in the same way and you are never going to experience a dependency hell. The only exception is when you manually change semantic versioning expression allowing a major variation, but you must know what you are doing.

Why does `npm install` add / remove caret (^) to / from version numbers?

I have a project that I work on with two different laptops. Sometimes I add extra packages to my project, so I have to use npm install <package-name> (duh). When I do that, I git push up the new package.json and package-lock.json files, and when I switch computers I have to git pull those changes, then run npm install again to get that package onto the other computer.
I recently noticed and started caring that one laptop kept adding carets (^) to the beginning of every package version number. For example:
One computer set package version #s to look like this:
"regexpu-core": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/regexpu-core/-/regexpu-core-1.0.0.tgz",
"integrity": "sha1-hqdj9Y7k18L2sQLkdkBQ3n7ZDGs=",
"requires": {
"regenerate": "1.4.0",
"regjsgen": "0.2.0",
"regjsparser": "0.1.5"
}
},
The other set package version #s to look like this:
"regexpu-core": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/regexpu-core/-/regexpu-core-1.0.0.tgz",
"integrity": "sha1-hqdj9Y7k18L2sQLkdkBQ3n7ZDGs=",
"requires": {
"regenerate": "^1.2.1",
"regjsgen": "^0.2.0",
"regjsparser": "^0.1.4"
}
},
I understand that carets (^) mean the version is not 100% precise, but I'm trying to figure out WHY my different laptops create different formats for package versions! I checked this SO question which has some great explanations for the differences between ~ and ^, but I didn't find anything explaining why npm would sometimes add and sometimes remove carets (^) altogether. I also looked at this npm issue on Github which recommended looking at npm config settings, but both of my laptops have the same settings:
npm config get save = true (both computers)
npm config get save-prefix = ^ (both computers)
npm config get save-exact = false (both computers)
One laptop was running npm version 5.6.0, but I just updated it to 6.5.0. The other computer was running version 6.4.1, but I also updated it to 6.5.0. I tried running npm install in my project on both computers, but still I find that one computer always removes ^ and the other always adds ^.
Please let me know if there's something I'm missing. Thanks for any help!
Edit: According to the discussion in issue #20434 this occurs by design using npm >=6.0.0.
Why does this happen?
#rarkins elaborately explains the reasoning for why this happens (and it’s advantages) in this comment. For convenience his comment is quoted below (verbatim):
Let's say that you use pinned versions of dependencies 'aaa', 'bbb' and 'ccc'. Let's say they each depend on 'zzz' like so:
aaa depends on zzz#^1.0.0
bbb depends on zzz#^1.1.0
ccc depends on zzz#^1.0.1
i.e. all three of them depend on a range of zzz, and not an exact version.
And let's say that the latest version of zzz is 1.5.0.
Both before and after this change, it's pretty obvious that the resolved version of zzz should be 1.5.0, so the only difference is how the package-lock.json is structured and documents this sub-dependency.
Before, the lock file would show that all three of them depend on zzz#1.5.0, and the resolved version of z is 1.5.0.
Now, it documents the actual "original" dependency versions (e.g. ^1.0.0, ^1.1.0, etc) for each dependency, but still shows the resolved version of z as 1.5.0.
Then consider what happens when zzz#1.5.1 is released:
Before, the lock file would need to update from z#1.5.0 to z#1.5.1 in all four places.
Now, the lock file only needs to update the resolved version of z to 1.5.1 while the dependencies can keep the ^1.0.0, ^1.1.0, and ^1.0.1 because they haven't changed.
As I mentioned previously in the thread, you still get the exact same node_modules in both cases. The advantages of the new approach are:
You get to see what the dependencies actually require (e.g. a range, and not an exact version). before, you could not tell if aaa actually required exactly zzz#1.5.0 or that it was instead zzz#^1.0.0.
Instead of four lines changing in the lock file, you get only one. It's less churn, and it's more clear what's happened.
As an aside, yarn uses a similar concept with yarn.lock. e.g. here's an example where #sindresorhus/is is pinned, but it's sub-dependency symbol-observable is not:
"#sindresorhus/is#0.10.0":
version "0.10.0"
resolved "https://registry.yarnpkg.com/#sindresorhus/is/-/is-0.10.0.tgz#f42dd6a9d12cd79fa6f53b27cf5bea3a30d2cafa"
dependencies:
symbol-observable "^1.2.0"
Original Answer:
After you git pull the revised package.json and package-lock.json onto computer two try deleting the node_modules directory before installing the packages again.
For example:
Firstly cd to your project directory on computer 2.
Delete the existing node_modules directory by running: rm -rf node_modules.
Then run: npm install
Or you can chain the two aforementioned commands using the && operator:
rm -rf node_modules && npm install

Lerna does not support dependencies at the top level?

I am in the process of switching my monorepo (back) from yarn (with workspaces) to lerna/npm, because yarn is too slow and unstable. However, I made an surprising discovery. With the following trivial package.json:
{
"devDependencies": { "lerna": "^2.11.0" },
"dependencies": { "typescript": "^2.9.1" }
}
and an empty lerna.json (in other words, no packages at all), then when I run
$ lerna bootstrap
it does not install anything at all in any top-level node_modules directory. And if for some reason I have a node_modules directory with no .bin subdirectory, then lerna bootstrap fails to create or populate the .bin subdirectory.
Is lerna not designed to actually specify top-level packages which are to be installed (along with their binaries in .bin)? I do notice that if I try lerna add on a lerna monorepo with no packages, it complains that "lerna WARN No packages found in scope where tslint can be added."
I could not find anything related to this in the documentation. With yarn/workspaces, I was using the ability to install global (top-level) versions of things like TypeScript for use in my build scripts while maintaining control over the version installed.
From the Lerna docs:
You can add the root as a managed location (in the packages array of lerna.json) - if that's something you need. This would cause lerna to link root's dependencies to your packages' directories, run postinstall script along with the others, etc.

Is there a way to alphabetize package.json without installing a package?

I've been working on lots of old npm packages that have their dependencies all out of order. They're shrinkwrapped packages, so updating dependencies is a bit of work (testing and verifying that the dependency changes didn't break anything), but I'm manually moving some dependencies from the devDependencies key to the dependencies key, and I don't want to do anything except alphabetize them before I commit. Rather than doing it manually, is there an easy way to programmatically alphabetize them with npm?
The sort-package-json package sorts not only dependencies and devDependencies, but other keys as well. I know the original questions didn't ask about the other keys, but I think it's cool to have all keys sorted.
You can simply run:
npx sort-package-json
Example from the package page:
$ cd my-project
$ cat package.json
{
"dependencies": {
"sort-package-json": "1.0.0",
"sort-object-keys": "1.0.0"
},
"version": "1.0.0",
"name": "my-awesome-project"
}
$ npx sort-package-json
package.json is sorted!
$ cat package.json
{
"name": "my-awesome-project",
"version": "1.0.0",
"dependencies": {
"sort-object-keys": "1.0.0",
"sort-package-json": "1.0.0"
}
}
This does not remove the trailing newline like the npm-sort package mentioned by Wolfgang.
Multiple files
$ sort-package-json "my-package/package.json" "other-package/package.json"
$ sort-package-json "package.json" "packages/*/package.json"
Just run npm remove --save anything or npm remove --save-dev whatever and npm will sort that section, without actually touching any of the content. Of course you should make sure that the package name you pass it (which can be anything, spam your keyboard) isn't in your package.json.
In addition to martias answer, you can just run:
npx sort-package-json
This won't install it permanently. You need npm >5.2.
I have found the npm-sort package, which seems to work quite well, with the minor niggle that it removes the trailing newline from the package.json file.
If you're using WebStorm, just select the lines you want to sort and click Edit > Sort Lines.
But for VScode there is no internal solution, so use this plugin https://marketplace.visualstudio.com/items?itemName=ue.alphabetical-sorter
You might also want to take a look at fixpack, a CLI to update your package.json following their (slightly) opinionated order. You can however add a .fixpackrc to define your own rules, the defaults are:
name first
description second
version third
author fourth
all other keys in alphabetical order
dependencies and devDependencies sorted alphabetically
newline at the end of the file
you can simply remove or uninstall a non-existing package from your dependencies
npm remove kjkjhkjhkjhkj -f --save
or shorter
npm r -S
other solutions are good but have some drawbacks:
1- sorts other keys which may be unwanted behavior
2- installs external packages such as sort-package-json, even by using npx sort-package-json
in VS-code there's a good packagesorter ofr the whole json file..
there's also something out there called "sortier" which sorts more, and is awesome.

Need more elaboration regarding, --save-dev

I see --save-dev mentioned in Gulp tutorials and from what I see, it adds npm functionality to a project's dependency.
But what does that mean exactly? Is that significant when the project gets moved from one machine to another?
Thank you for any clarification of --save-dev importance with Gulp.
In a npm package there 2 types of dependencies: the production ones and the development ones.
{
"dependencies": {
// .. a list of production dependencies
// i.e. angular or express
},
"devDependencies": {
// .. a list of dependencies strictly needed only in development mode
// i.e. gulp or grunt
}
}
You need the former to make the application run in production. The latter are used when in development mode, so everything around the build system, minification, etc...
Gulp, as a building system, is more a devDependency by nature, than a production dependency. This is why you often find in Gulp/Gulp plugins tutorials things are:
$ npm install --save-dev gulp
That --save-dev flag will put the installed dependency you're asking in the devDependencies bucket while using --save sets the dependency in the dependencies (production) one.