Consider the following:
A project has a dependency for package X #1.2.3
A project may need to install package Y.
Package Y has a dependency for package X #2.3.4
The version 2 of package X is not backwards compatible with version 1.
Can this cause issues of any kind?
If you have npm installed them in the default way, your project is safe from dependency conflicts.
Just to be sure, look for those dependencies in the package-lock.json file and you should immediately notice that certain package dependencies are managed differently from project dependencies. In particular, this is a simplified entry created when you install X#1.2.3.
"node_modules/X": {
"version": "1.2.3",
...
"dependencies": {
...
}
}
Then you install Y, that depends on X#2.3.4.
"node_modules/Y": {
"version": "x.y.z",
...
"dependencies": {
"X": "2.3.4"
},
}
However, since X#2.3.4 has not been installed yet, a new entry is automatically created:
"node_modules/Y/node_modules/X": {
"version": "2.3.4",
...
"dependencies": {
...
}
},
As you can see, npm detects the potential dependency conflict: instead of registering the package in the global node_modules folder, npm installs it inside the node_modules folder of the package installed. You could have more than 2 nested level of node_modules if there are conflictual dependencies inside node_modules/Y/node_modules too.
I showed you an example of package-lock.json because understanding dependency management is straightforward looking at it, even if big. Nonetheless, in case you do not use it (I hope you do), npm reasons in the same way and you are never going to experience a dependency hell. The only exception is when you manually change semantic versioning expression allowing a major variation, but you must know what you are doing.
Related
Trying to install a dev dependency but one of its dependencies is lodash: 4.17.20. When Snyk does a scan of my dependencies, it marks this dependency as a high security vulnerability.
How can we have this dev dependency try to resolve a different version of lodash for a dev dependency and pass the Snyk test?
Was thinking that in the yarn.lock file, it somehow needs to resolve a higher version of lodash for this dev dependency, so I've referred to https://classic.yarnpkg.com/en/docs/selective-version-resolutions/
Doing something in my package.json like
"resolutions": {
"**/lodash": "^4.17.20"
}
Or
"resolutions": {
"<that dev dependency>/lodash": "^4.17.20"
}
Seems like it hasn't quite worked, and the Yarn.lock hasn't updated the lodash dependency for that dev dependency. Wanted to see if this was possible without updating the yarn.lock manually as I could see it being re-overwritten in the future. This is done in a Lerna monorepo.
Update from the Snyk team, they do not have monorepo support with Lerna as of 04/05/21
I'm kind of lost here!
I'm using a module which has another module nested in its' node_modules.
I.E.
my_project
node_modules
widely_used_module
parent_dependency
node_modules
widely_used_module
I have some fixes in my "own" widely_used_module (it could be just a minor version from the original distributor, but to be completely honest, in this case its' my fork on Github containing some critical fixes).
When I manually remove node_modules/parent_dependency/node_modules, parent_dependency starts to reference to my "widely used module" instead of its' own. But this of course gets overriden once I hit npm install again.
Can I somehow prevent a package to install its' own modules, or can I force a package to reference the root node_modules and ignore its' own?
Is that even the right approach to fixing such issues? I don't want to fork parent_dependency as well...
Thank you
Answering my own question;
Yarn has a built-in solution for this exact issue.
This could be achievable with NPM as well but yarn made it so easy to fix that I moved the project dependencies to be handled by yarn.
Full solution:
Installing yarn
Ran yarn in project's root path
Removed package.lock.json
Added resolutions to my package.json. In my case:
{
"dependencies": {
"...": "...",
"parent_dependency": "^x.y.z"
},
"devDependencies": {
"...": "..."
},
"resolutions": {
"parent_dependency/widely_used_module": "git+https://git#github.com/myuser/widely_used_module.git"
}
}
Ran yarn install.
Result: No more widely_used_module folder under parent_dependency.
I am in the process of switching my monorepo (back) from yarn (with workspaces) to lerna/npm, because yarn is too slow and unstable. However, I made an surprising discovery. With the following trivial package.json:
{
"devDependencies": { "lerna": "^2.11.0" },
"dependencies": { "typescript": "^2.9.1" }
}
and an empty lerna.json (in other words, no packages at all), then when I run
$ lerna bootstrap
it does not install anything at all in any top-level node_modules directory. And if for some reason I have a node_modules directory with no .bin subdirectory, then lerna bootstrap fails to create or populate the .bin subdirectory.
Is lerna not designed to actually specify top-level packages which are to be installed (along with their binaries in .bin)? I do notice that if I try lerna add on a lerna monorepo with no packages, it complains that "lerna WARN No packages found in scope where tslint can be added."
I could not find anything related to this in the documentation. With yarn/workspaces, I was using the ability to install global (top-level) versions of things like TypeScript for use in my build scripts while maintaining control over the version installed.
From the Lerna docs:
You can add the root as a managed location (in the packages array of lerna.json) - if that's something you need. This would cause lerna to link root's dependencies to your packages' directories, run postinstall script along with the others, etc.
Is there a way to specify subsets of dependencies in npm, with an alias or "feature tag"? That is, if someone knows that they will only be using some limited subset of the features of my package, they can specify those features and, on npm install, only download the dependencies relevant to those features?
My package has a very large number of dependencies and takes nearly half an hour to install, but most users only need a subset of their functionality. I'm thinking of something like how dependencies can be divided into devDependencies and dependencies, but with n groups instead of just those two. For example:
npm install --feature feature1 --feature feature2
From reading the docs, I think the answer here is "no", but what would be your suggestion for this case? Split the package into smaller plugin packages and have users install the plugins that they want? I don't want something that is too complicated for users to configure.
The short answer is no, npm was not designed for this mostly because dependency trees are incredibly large, and this use case could really just complicate things for most users. That being said, I wrote a package to do it.
My package is install-subset, and can be installed globally with npm install -g install-subset
https://www.npmjs.com/package/install-subset
Essentially you build inclusion lists and exclusion lists for named subsets in your package.json like this:
"subsets": {
"build": {
"include": [
"babel-cli",
"dotenv"
]
},
"test": {
"exclude": [
"eslint",
"lint-rules",
"prettier"
]
}
}
Then call it with, for example, install-subset test
This will temporarily rewrite your package.json to not install those packages excluded, then restore it, which depending on the packages can save a lot of time and bandwidth.
Also works with yarn, is open source and issues/PRs are welcome.
Another third-party package that you can use to do this is group-dependencies. It allows you to define [GROUP_NAME]Dependencies arrays (note: not objects) in your package file, then install just that subset using deps install [GROUP_NAME].
Here's an example from their README:
{
...
"devDependencies": {
"intercept-stdout": "^0.1.2",
"jest": "^20.0.4",
"strip-color": "^0.1.0"
},
// our new group representing testing dependencies
"testDependencies": [
"jest"
]
...
}
Now you can install only the dependencies for this new group:
# This will install jest#^20.0.4:
$(npm bin)/deps install test
I see --save-dev mentioned in Gulp tutorials and from what I see, it adds npm functionality to a project's dependency.
But what does that mean exactly? Is that significant when the project gets moved from one machine to another?
Thank you for any clarification of --save-dev importance with Gulp.
In a npm package there 2 types of dependencies: the production ones and the development ones.
{
"dependencies": {
// .. a list of production dependencies
// i.e. angular or express
},
"devDependencies": {
// .. a list of dependencies strictly needed only in development mode
// i.e. gulp or grunt
}
}
You need the former to make the application run in production. The latter are used when in development mode, so everything around the build system, minification, etc...
Gulp, as a building system, is more a devDependency by nature, than a production dependency. This is why you often find in Gulp/Gulp plugins tutorials things are:
$ npm install --save-dev gulp
That --save-dev flag will put the installed dependency you're asking in the devDependencies bucket while using --save sets the dependency in the dependencies (production) one.