I've got a NPM project which have some dependencies declared within its package.json.
In the meantime this project has grown quite old so I wondering how to update the packages to a newer version. But after installing the packages in the latest version, the output of the npm audit will still show a lot of critical entries. I've found out that the critical part are not the packages mentioned within my package.json but sub-dependencies. For example by installing package X, my project will inevitably also take risks, since package X itself is dependent on the outdated package Y.
How should I handle this?
Related
I'm responsible for maintaining a bunch of npm packages. Specifically, I am a maintainer of Apollo Server. Our latest major version combines over a dozen apollo-server-* packages into a single #apollo/server package.
We are going to use npm deprecate to mark the old packages as deprecated to help people find the new one. But I'm running into a bit of a pickle.
If my testing is correct, npm deprecate PACKAGENAME is actually equivalent to "individually mark all current versions of PACKAGENAME as deprecated". But if we then go ahead and publish another version of the package later, the new version appears to not be deprecated.
While we want people to upgrade to the new package, we still may publish some versions of the old package, for security fixes and the like. And unfortunately that will often mean publishing over a dozen separate packages.
So if I'm not confused, this will mean we have to re-run npm deprecate after any publish, or else the package will effectively end up non-deprecated?
So my next thought was to do the deprecation in a shell script, to either be run in CI post-publish or manually by a developer. But unless my testing was incorrect, it does not look like you can use NPM_TOKEN=xxx npm deprecate at all (whether it's an automation token or a publish token): I get an error that the package does not exist. So the script will have to be run manually... and will require me to enter dozens of OTPs.
So my question is: if I have a project that consists of dozens of packages that I want to keep deprecated even if I publish patches in the future, do I really need to maintain a shell script that runs npm deprecate dozens of times and requires me to manually enter dozens of OTPs? Or is there an easier way?
We're using NodeJS for some projects and are faced with an issue that must have a simple solution (seeing as nobody else seems to have the problem).
In the packages.json there are a bunch of dependencies mentioned with a minimum version, each of which may have overlapping dependencies of their own. The default way a dependency is added is using the ^ operator which seems to mean 'compatible with' or 'same major version, but minor versions may differ'.
The way I understand npm to work is on npm install to take the highest minor version available that matches. Unfortunately 'compatible with' is not quite as enforced as you'd hope.
The situation this puts us in is that for instance on a developer machine version 1.1.0 is installed, but between development and publishing a new version 1.2.0, that has a bug, is introduced. On our build machine a fresh build is made which ends up using 1.2.0 and we've introduced a bug that wasn't there in development.
We tried changing the ^ operator to = for instance, but this gives us trouble when dependencies have subdependencies that aren't compatible with the requested version.
All in all I'm a bit confused, but this thing keeps biting us anytime something changes since the development machines don't do anything on npm install if the package is already there, but the build machine always gets fresh copies.
I know from NuGet that it always takes the lowest version that matches all combined requirements. Since this is always the same for a given set of dependencies, I much prefer this approach. Is there a way to make npm work like this too?
To answer my own question:
npm has introduced a new command npm ci which does something similar to npm install but enforces that the specific versions are used that were also used when a package was initially added by using the package-lock file.
See https://docs.npmjs.com/cli/ci for more information
While using npm I've not once encountered a problem when some package somewhere deep in the dependency tree receives minor or even patch update, but actually introduces a breaking change. Finding the culprit package is not always easy. And the problem most of the time hits CI the hardest as it often performs clean npm install. So all that remains is waiting a day or two till package authors notice and fix the error.
Using exact versions in package.json doesn't help either as referenced packages have dependencies of their own and they are not always specified with exact versions.
Such breaking changes are a fact of life I suppose, as no one is immune to mistakes, and shear number of packages directly and indirectly used in any bigger than trivial project is huge.
So, how to prevent such inevitable breaking changes from disrupting development process?
The only thing I've been able to imagine is a hypothetical feature of npm that would only allow installing packages no younger than now().addDays(-2).
A package-lock.json (npm5) or a npm-shrinkwrap.json (npm 2-4) if created and committed to source control will lock in all of your dependencies and their dependencies so that the exact same versions will be installed. package-lock.json is now generated automatically, and npm-shrinkwrap.json can be created with npm shrinkwrap command. Once these files are created, they will be automatically updated on subsequent npm install --save ... commands.
After updating my NPM to the latest version (from 3.X to 5.2.0) and running npm install on an existing project, I get an auto-created package-lock.json file.
I can tell package-lock.json gives me an exact dependency tree as opposed to package.json.
From that info alone, it seems like package.json is redundant and not needed anymore.
Are both of them necessary for NPM to work?
Is it safe or possible to use only the package-lock.json file?
The docs on package-lock.json (doc1, doc2) doesn't mention anything about that.
Edit:
After some more thinking about it, I came to the conclusion that if someone wants to use your project with an older version of NPM (before 5.x) it would still install all of the dependencies, but with less accurate versions (patch versions)
Do you need both package-lock.json and package.json? No.
Do you need the package.json? Yes.
Can you have a project with only the package-lock.json? No.
The package.json is used for more than dependencies - like defining project properties, description, author & license information, scripts, etc. The package-lock.json is solely used to lock dependencies to a specific version number.
package-lock.json: records the exact version of each installed package which allows you to re-install them. Future installs will be able to build an identical dependency tree.
package.json: records the minimum version you app needs. If you update the versions of a particular package, the change is not going to be reflected here.
If your question is if lock file should be committed to your source control - it should. It will be ignored under certain circumstance.
I found it bloating pull requests and commit history, so if you see it change, do a separate commit for it.
Lets say I am working on a library that will be consumed by other developers. MyPackage has a dependency on moment. The developer that consumes my package also has a dependency on moment. So moment will exist as a "dependency" in both library package.json and application package.json (and thus get packaged twice). Is there a way to package it just once? If the consumer has it, use theirs, else use mine?
It's already happening by default on fresh installs if dependency ranges match.
npm v>=3 does gang the dependencies, depending on the installation order and depth, see here.
Also, if you kept working on the same folder for a while, there might be some cruft, which could be wiped using npm dedupe, see here.
In theory, moment should not be duplicated if both your library and developer's library are consuming the same version ranges of it. At least if npm dedupe is called or node_modules are wiped and npm i-nstalled.