Dependency resolution approach - comparing NPM to Homebrew? - npm

I recently got confused and almost installed a tool via brew install when in fact it was an npm package and all I needed to do was npm install -g.
So these tools are strangely similar yet obviously different.
What's the difference in crystal clarity?

NPM exists to resolve dependencies for application code, on a per app basis, allowing an app to be self-contained and portable. This means that (in its default mode of operation) it will install the same stuff many times, uniquely, repeatedly, and separately, for every app on your system that needs the same package, inside of that apps own directory and isolated from everything else.
Homebrew is not like this. The reason is it serves the system itself, not individual apps, so is more comparable to just the npm -g part of npm.
There is one extra bit to understand about homebrew, though - some system packages have specific dependencies and could even have conflicts. This means that for the global installs that homebrew provides, it also has to solve some nesting and conflict issues. It's a kind of magic?

Related

Group project uses both NPM + Yarn. How to transition to use only one?

As title indicates, I'm working on a project where different members have used different tools (NPM and Yarn) for handling packages and modules etc.
We aim to transition to use ONLY Yarn (not our decision). Would anyone be able to share resources detailing how to accomplish such a thing? Or help quickly walk me through the steps?
I tried googling for answers but every single result is yet another article explaining why you should ditch NPM/Yarn and move your project to Yarn/NPM, without explaining the steps one would need to take to move from using both to just one mid-project. Thanks!
It looks like Yarn has a page talking about how to migrate to it from NPM:
https://yarnpkg.com/lang/en/docs/migrating-from-npm/
In most cases, running yarn or yarn add for the first time will just work. In some cases, the information in a package.json file is not explicit enough to eliminate dependencies, and the deterministic way that Yarn chooses dependencies will run into dependency conflicts. This is especially likely to happen in larger projects where sometimes npm install does not work and developers are frequently removing node_modules and rebuilding from scratch. If this happens, try using npm to make the versions of dependencies more explicit, before converting to Yarn.
As of Yarn 1.7.0, you can import your package-lock.json state, generated by npm to Yarn, by using yarn import.
They use many of the same files and structures. The important thing is to check-in the yarn.lock file and make sure everyone is installing using Yarn instead of NPM.
If you have a build server, you could probably use it to enforce those dependencies, but it would be more work.

node-sass vulnerability and npm audit

I have projects to maintain that use node-sass npm module.
Since node 10.x, there is a tool (called npm audit) that is run every time we do a npm install. This seems to be a good tool for preventing vulnerability issues.
My problem is that the node-sass module has vulnerabilities. I saw that the maintainers of the project do not want to fix the issues with bad reasons.
https://github.com/sass/node-sass/issues/2262
People maintaining popular module like node-sass should correct as soon as possible the vulnerability issues, but unfortunately they don't.
I am not an expert in Security, so I prefer to rely on what indicates npm and not use anymore dependencies that print messages that let you think your software is crap.
But I like so much SASS for coding CSS that I would like to give it a chance to keep it.
Any idea for removing these vulnerability messages while keeping the project safe and not reducing developer experience ?
This security issue is mainly irrelevant to node-sass since it never sees any exposure to your live code.
node-sass runs on hosts normally used for development and usually are not visible in a public network.
You normally will use node-sass to pre-compile SCSS into CSS and vulnerabilities will not impact the resulting CSS code.
These warnings are relevant if you run the node.js server as a backend which is normally not the case. (or never the case)
One option is to use dart-sass. It has no vulnerability issues.
https://sass-lang.com/dart-sass
https://github.com/webpack-contrib/sass-loader

How to make npm use the lowest version that matches all requirements

We're using NodeJS for some projects and are faced with an issue that must have a simple solution (seeing as nobody else seems to have the problem).
In the packages.json there are a bunch of dependencies mentioned with a minimum version, each of which may have overlapping dependencies of their own. The default way a dependency is added is using the ^ operator which seems to mean 'compatible with' or 'same major version, but minor versions may differ'.
The way I understand npm to work is on npm install to take the highest minor version available that matches. Unfortunately 'compatible with' is not quite as enforced as you'd hope.
The situation this puts us in is that for instance on a developer machine version 1.1.0 is installed, but between development and publishing a new version 1.2.0, that has a bug, is introduced. On our build machine a fresh build is made which ends up using 1.2.0 and we've introduced a bug that wasn't there in development.
We tried changing the ^ operator to = for instance, but this gives us trouble when dependencies have subdependencies that aren't compatible with the requested version.
All in all I'm a bit confused, but this thing keeps biting us anytime something changes since the development machines don't do anything on npm install if the package is already there, but the build machine always gets fresh copies.
I know from NuGet that it always takes the lowest version that matches all combined requirements. Since this is always the same for a given set of dependencies, I much prefer this approach. Is there a way to make npm work like this too?
To answer my own question:
npm has introduced a new command npm ci which does something similar to npm install but enforces that the specific versions are used that were also used when a package was initially added by using the package-lock file.
See https://docs.npmjs.com/cli/ci for more information

npm package.json dependencies - for a library component

Lets say I am working on a library that will be consumed by other developers. MyPackage has a dependency on moment. The developer that consumes my package also has a dependency on moment. So moment will exist as a "dependency" in both library package.json and application package.json (and thus get packaged twice). Is there a way to package it just once? If the consumer has it, use theirs, else use mine?
It's already happening by default on fresh installs if dependency ranges match.
npm v>=3 does gang the dependencies, depending on the installation order and depth, see here.
Also, if you kept working on the same folder for a while, there might be some cruft, which could be wiped using npm dedupe, see here.
In theory, moment should not be duplicated if both your library and developer's library are consuming the same version ranges of it. At least if npm dedupe is called or node_modules are wiped and npm i-nstalled.

How do you deal with people removing npm package versions?

Today I found that an npm package version, Babel 6.0.15, that my application relies on had been removed from npm.
This caused compilation failure on a new pc, and I had to go manually find the closest available version for it, and all the cascading version changes it affected on related packages.
What is the best of way of dealing with npm packages, now that I know they can go missing at any time?
Do you check your node_modules folder into source control?
Is there a rule on npm about what versions (major, minor, etc) may be removed by the creator, and which are more 'long term support' and must be retained?
How do you get npm locally to inform you when 'npm update' fails on a new pc, rather than silently failing?
After thinking about this for a while I wrote a blog post summarising what I think is best practice. Reproduced below:
Summary
Specify exact versions of all npm modules, e.g. “alt”: “0.17.8”
Commit your node_modules folder into source control
Don’t use DefinitelyTyped or any other external library Typescript definition tools
Why?
Some of these principles might be controversial, so here’s my reasoning:
Specify exact versions of all npm modules
Semver (semantic versioning) says that breaking changes should occur only if the major version changes. So you should be able to say just “alt”: “0.17”
But I’ve found in practice that even patch changes (bugfixes) can break your application – because libraries that rely on these library often expect some tiny behaviour in a particular version not to change. So in order for all your particular versions of particular libraries to work, they need to rely on exact versions of other libraries.
Commit your node_modules folder into source control
I first assumed that all versions of famous npm libraries would remain there indefinitely. But I then discovered that creators often remove old versions of their software from npm – which then breaks the cascading chain of exact version number dependencies you’ve configured for your app.
Yes, committing all your npm libraries will take up space in your repository, but they’re text files after all, not .DLLs, so they’ll get compressed really small. And the alternative is one day not being able to compile your app at all on a new computer because a library has been completely removed from npm.
Don’t use DefinitelyTyped or any other external library Typescript definition tools
It’s wonderful to be given compile errors for external tools you use. But I’ve found it’s not worth the effort because:
there’s no way to match the definition file version number and npm library version number, so you get definitions that are out of sync with the library you are using
they often have bugs
the type bugs you catch at compile time are probably going to occur in your own app, not in how you call external libraries
Instead of using .d.ts files for external libraries, just say:
declare module 'lodash'
{
let x: any;
export = x;
}
Or use the –allowJS flag in Typescript 1.8 onwards.