I would like to hard-pin my NPM dependencies. "Hard-pinning" would mean that an automated process would check my dependency list for certain packages with certain versions and if a package has been locally upgraded, an custom error message should be shown (ideally, this should be integrated in a pre-push Git hook).
The reasons for wanting this behavior could be:
external dependencies (e.g. other teams integrating with your project, requiring certain versions)
broken or unwanted behavior because of certain issues (e.g. "wait until #124 is fixed")
known non-obvious migration effort for major upgrades
upgrade incompatibilities (e.g. newest version requires, but does not enforce, a newer peer dependency).
Normal pinning does not cut it in this case: it's trivial to update pinned packages anyway, comments do not work with package.json without extra effort and sometimes the reasons are too important not to be displayed explicitly.
How can I achieve such "hard-pinning"?
Related
Recently new versions of packages came out, which required new versions of another library (which in turn broke my production builds). However, they both only increased their patch version number (one was #types/prettier and the other one cheerio).
My projects use these libs only indirectly, so even though I set all my dependencies to a fixed version number, these changes affected my builds and cause a lot of grief.
If I could tell NPM not to consider newer versions of a package and always use only the specified one, this kind of problem would probably not appear. Is it possible to force NPM not to install newer versions of dependencies, even if they allow ranges?
If I install stuff using brew or brew cask (on macOS) or npm or pip or composer or similar package/code/library managers, I have always wondered:
Where do they get their content from, who or what entity is managing or facilitating or hosting all those software or packages or modules or libraries?
Are there any checks, safety filters, audits, validation, or other mechanisms that prevent malware in the packages or libaries that are being distributed though these package managers?
Composer
List of packages is stored at https://packagist.org/, but there are only metadata. Packages are downloaded directly from related repositories (usually GitHub or GitLab), Packagist does not store or analyze its content. So while it may look a bit scary, the security model is based on trust to the vendor or direct code review. There is no magic solution which will pretend that it protects you from malware - you need to think what you're doing and what dependencies you're including into your project (or at least use some malware scanner on your own).
Our company has a few web applications which in turn depend on a very long chain of internally created and hosted npm packages (we use JFrog Artifactory) each with their own dependencies (and so on). Whenever a bug is fixed or a feature is implemented in a low-level package the current process requires a developer to check in their changes, wait for the CICD build to complete and tests to run, update the parent package, and rinse / repeat all the way up the chain (which can be a very long process).
This can't possibly be a unique situation yet it impacts our productivity greatly and encourages monolithic package development to limit number of packages to update instead of proper code separation.
I can only think of two solutions:
1) Update the web application to use the transitive dependency directly in package.json. This however breaks "encapsulation" because how a direct dependency manages its job shouldn't be known to the web application. If the direct dependency were to use some other transitive dependency later the web application shouldn't be left referencing a now irrelevant package.
2) Modify the web application's package-lock.json to point at the new version of the transitive dependency. This however seems to only work temporarily as merge conflicts or new installs of direct dependencies tend to revert these changes.
I realize that the answer might be to optimize the build / publish process to be less painful and manual but I was hoping others might have encountered a different solution.
FYI - All dependencies are installed with '~' as a version prefix by default.
The correct way to do this (Be in mind that this feature was introduced recently in npm 8.x) is using the overrides section. This new feature introduced allows replacing a package in your dependency tree with another version, or another package entirely.
I consider to use Hexo (the static blog generator) based on npm. I wonder one thing, what if although one npm package (dependency) will not be longer available? Each package has its own author and it can cancel support or completely remove it from npm's repository at any time.
So what do I do if missing one of the npm package affect on running Hexo and consequently I'll not be able to generate my blog in the future?
Although this can happen (and happened at least once), it is not a serious problem ussually. While you will be waiting until somebody will fix the missing dependency (it take place quickly on popular packages like Hexo) you can use older working version. And if you want to be 100% sure, you can commit node_modules together with your web sources (see discussion here).
Today I found that an npm package version, Babel 6.0.15, that my application relies on had been removed from npm.
This caused compilation failure on a new pc, and I had to go manually find the closest available version for it, and all the cascading version changes it affected on related packages.
What is the best of way of dealing with npm packages, now that I know they can go missing at any time?
Do you check your node_modules folder into source control?
Is there a rule on npm about what versions (major, minor, etc) may be removed by the creator, and which are more 'long term support' and must be retained?
How do you get npm locally to inform you when 'npm update' fails on a new pc, rather than silently failing?
After thinking about this for a while I wrote a blog post summarising what I think is best practice. Reproduced below:
Summary
Specify exact versions of all npm modules, e.g. “alt”: “0.17.8”
Commit your node_modules folder into source control
Don’t use DefinitelyTyped or any other external library Typescript definition tools
Why?
Some of these principles might be controversial, so here’s my reasoning:
Specify exact versions of all npm modules
Semver (semantic versioning) says that breaking changes should occur only if the major version changes. So you should be able to say just “alt”: “0.17”
But I’ve found in practice that even patch changes (bugfixes) can break your application – because libraries that rely on these library often expect some tiny behaviour in a particular version not to change. So in order for all your particular versions of particular libraries to work, they need to rely on exact versions of other libraries.
Commit your node_modules folder into source control
I first assumed that all versions of famous npm libraries would remain there indefinitely. But I then discovered that creators often remove old versions of their software from npm – which then breaks the cascading chain of exact version number dependencies you’ve configured for your app.
Yes, committing all your npm libraries will take up space in your repository, but they’re text files after all, not .DLLs, so they’ll get compressed really small. And the alternative is one day not being able to compile your app at all on a new computer because a library has been completely removed from npm.
Don’t use DefinitelyTyped or any other external library Typescript definition tools
It’s wonderful to be given compile errors for external tools you use. But I’ve found it’s not worth the effort because:
there’s no way to match the definition file version number and npm library version number, so you get definitions that are out of sync with the library you are using
they often have bugs
the type bugs you catch at compile time are probably going to occur in your own app, not in how you call external libraries
Instead of using .d.ts files for external libraries, just say:
declare module 'lodash'
{
let x: any;
export = x;
}
Or use the –allowJS flag in Typescript 1.8 onwards.