I'm responsible for maintaining a bunch of npm packages. Specifically, I am a maintainer of Apollo Server. Our latest major version combines over a dozen apollo-server-* packages into a single #apollo/server package.
We are going to use npm deprecate to mark the old packages as deprecated to help people find the new one. But I'm running into a bit of a pickle.
If my testing is correct, npm deprecate PACKAGENAME is actually equivalent to "individually mark all current versions of PACKAGENAME as deprecated". But if we then go ahead and publish another version of the package later, the new version appears to not be deprecated.
While we want people to upgrade to the new package, we still may publish some versions of the old package, for security fixes and the like. And unfortunately that will often mean publishing over a dozen separate packages.
So if I'm not confused, this will mean we have to re-run npm deprecate after any publish, or else the package will effectively end up non-deprecated?
So my next thought was to do the deprecation in a shell script, to either be run in CI post-publish or manually by a developer. But unless my testing was incorrect, it does not look like you can use NPM_TOKEN=xxx npm deprecate at all (whether it's an automation token or a publish token): I get an error that the package does not exist. So the script will have to be run manually... and will require me to enter dozens of OTPs.
So my question is: if I have a project that consists of dozens of packages that I want to keep deprecated even if I publish patches in the future, do I really need to maintain a shell script that runs npm deprecate dozens of times and requires me to manually enter dozens of OTPs? Or is there an easier way?
Related
I want to split up the code base of several of my project into isolated package like projects. Those should be easily usable by npm but they do not seem significant enough to be published to the global npm registry.
So, my question is if there is a middle way to handling them like local provided packages and installing them with their path and publishing them in the global repository.
Concerns:
cluttering the npm registry with packages which don't seem to be significant enough to take up the name
the need to document and to create tests for each package seems to be too much and I would not sleep well publishing packages which are not well documented and tested
I take up a name which might be more appropriate to be used by a more sophisticated package and maintainers
I still want other to be able to easily try / use this package, to see if it fits their needs
Alternatives:
A) creating a private npm repository (with CouchDB?)
+ is pretty much identical to the npm repository and would be easy to use
+ the versioning is identical just pure semver lookup
- every user needs to set up this repository if they want to use this package or need it as a child dependency in their (public npm) package (even though this is unlikely)
- Need to invest time into setting it up and maintaining it
B) Using my username npm namespace
+ would solve pretty much every problem
- namespaces seem to be meant for projects and its sub packages which wouldn't be the case for my packages since their only connection is the creator
- it seems arrogant to prefix your packages with your name, like you are tagging it with a big sign THIS WAS DONE BY ME
C) Using GitHub with a special detached branch which contains the (tagged) releases
+ you could use it like the global npm repository since the npm resolving strategy allows the repository url with a semver range in place of the version
- special case which is bound to break
- GitHub is not meant to provide npm packages, about no developer expects a git url instead of the versionrange, tools and firewalls might have problem with this
- workflow is really not meant this way neither for git nor for npm
D) using a local package and install package by its path
+ easy to setup and use
- no version management
- build steps must be done manually beforehand
- can not publish packages depending on those packages
- all dependencies have to be installed locally
E) making those packages more useful, implementing edge cases, writing documentation and testing the whole package
+ would resolve about all problems
- ALOT of extra work, primarily thinking about edge cases and giving the developer a good api
- sometimes you can't really get the name for you package (it collides with other) which results in weird
- it is your responsibility, you have to maintain it, be responsible (test it well, edge cases)
- cluttering of the npm repository
So those are all the alternatives which came to mind when I tried to find a solution. Please leave a comment / answer if you have another idea or maybe you can remove / reduce the importance of those contra points.
Maybe you could include your own experience, so I get a better view for the whole problem.
Currently I would just try to make the package more helpful to the greater majority but this does not work in all cases.
Thank you all for your time!
Installing from git is pretty standard feature in package managers. npm doesn't have Github-support, it's generic support for any git repo. Unless you can find some discussion about deprecating it from npm, I'd not worry about it. It's used internally in many companies for private packages.
Of course, there is still some trade offs: build artifacts and maybe a bit more clumsy workflow. Things like npm outdated doesn't understand git semver. For build artifacts, I have seen many projects to commit them to master branch to support direct git-install. If you look around older open source projects for example, that's the case quite often.
We went for a private repository with verdaccio running in a docker container, which is very similar to version A. It took some setup, but for our developers all it took was a single npm command to add the private repo "in front of" npm for all packages of the namespace we created. Granted, our packages are project specific, but in a private repository that does not really matter either way, does it?
We considered the local package option at first, but the drawbacks were just too big for us, even if it's very easy to setup.
I'm not sure this helps, but this is at least the setup we decided upon when we had the same issue a few months ago.
We're using NodeJS for some projects and are faced with an issue that must have a simple solution (seeing as nobody else seems to have the problem).
In the packages.json there are a bunch of dependencies mentioned with a minimum version, each of which may have overlapping dependencies of their own. The default way a dependency is added is using the ^ operator which seems to mean 'compatible with' or 'same major version, but minor versions may differ'.
The way I understand npm to work is on npm install to take the highest minor version available that matches. Unfortunately 'compatible with' is not quite as enforced as you'd hope.
The situation this puts us in is that for instance on a developer machine version 1.1.0 is installed, but between development and publishing a new version 1.2.0, that has a bug, is introduced. On our build machine a fresh build is made which ends up using 1.2.0 and we've introduced a bug that wasn't there in development.
We tried changing the ^ operator to = for instance, but this gives us trouble when dependencies have subdependencies that aren't compatible with the requested version.
All in all I'm a bit confused, but this thing keeps biting us anytime something changes since the development machines don't do anything on npm install if the package is already there, but the build machine always gets fresh copies.
I know from NuGet that it always takes the lowest version that matches all combined requirements. Since this is always the same for a given set of dependencies, I much prefer this approach. Is there a way to make npm work like this too?
To answer my own question:
npm has introduced a new command npm ci which does something similar to npm install but enforces that the specific versions are used that were also used when a package was initially added by using the package-lock file.
See https://docs.npmjs.com/cli/ci for more information
While using npm I've not once encountered a problem when some package somewhere deep in the dependency tree receives minor or even patch update, but actually introduces a breaking change. Finding the culprit package is not always easy. And the problem most of the time hits CI the hardest as it often performs clean npm install. So all that remains is waiting a day or two till package authors notice and fix the error.
Using exact versions in package.json doesn't help either as referenced packages have dependencies of their own and they are not always specified with exact versions.
Such breaking changes are a fact of life I suppose, as no one is immune to mistakes, and shear number of packages directly and indirectly used in any bigger than trivial project is huge.
So, how to prevent such inevitable breaking changes from disrupting development process?
The only thing I've been able to imagine is a hypothetical feature of npm that would only allow installing packages no younger than now().addDays(-2).
A package-lock.json (npm5) or a npm-shrinkwrap.json (npm 2-4) if created and committed to source control will lock in all of your dependencies and their dependencies so that the exact same versions will be installed. package-lock.json is now generated automatically, and npm-shrinkwrap.json can be created with npm shrinkwrap command. Once these files are created, they will be automatically updated on subsequent npm install --save ... commands.
Today I found that an npm package version, Babel 6.0.15, that my application relies on had been removed from npm.
This caused compilation failure on a new pc, and I had to go manually find the closest available version for it, and all the cascading version changes it affected on related packages.
What is the best of way of dealing with npm packages, now that I know they can go missing at any time?
Do you check your node_modules folder into source control?
Is there a rule on npm about what versions (major, minor, etc) may be removed by the creator, and which are more 'long term support' and must be retained?
How do you get npm locally to inform you when 'npm update' fails on a new pc, rather than silently failing?
After thinking about this for a while I wrote a blog post summarising what I think is best practice. Reproduced below:
Summary
Specify exact versions of all npm modules, e.g. “alt”: “0.17.8”
Commit your node_modules folder into source control
Don’t use DefinitelyTyped or any other external library Typescript definition tools
Why?
Some of these principles might be controversial, so here’s my reasoning:
Specify exact versions of all npm modules
Semver (semantic versioning) says that breaking changes should occur only if the major version changes. So you should be able to say just “alt”: “0.17”
But I’ve found in practice that even patch changes (bugfixes) can break your application – because libraries that rely on these library often expect some tiny behaviour in a particular version not to change. So in order for all your particular versions of particular libraries to work, they need to rely on exact versions of other libraries.
Commit your node_modules folder into source control
I first assumed that all versions of famous npm libraries would remain there indefinitely. But I then discovered that creators often remove old versions of their software from npm – which then breaks the cascading chain of exact version number dependencies you’ve configured for your app.
Yes, committing all your npm libraries will take up space in your repository, but they’re text files after all, not .DLLs, so they’ll get compressed really small. And the alternative is one day not being able to compile your app at all on a new computer because a library has been completely removed from npm.
Don’t use DefinitelyTyped or any other external library Typescript definition tools
It’s wonderful to be given compile errors for external tools you use. But I’ve found it’s not worth the effort because:
there’s no way to match the definition file version number and npm library version number, so you get definitions that are out of sync with the library you are using
they often have bugs
the type bugs you catch at compile time are probably going to occur in your own app, not in how you call external libraries
Instead of using .d.ts files for external libraries, just say:
declare module 'lodash'
{
let x: any;
export = x;
}
Or use the –allowJS flag in Typescript 1.8 onwards.
Can someone explain to me the difference between NPM, Bower and Composer.
They are all package managers - correct?
But when should each one be used?
Also, each one appears to have a json file that accompanies it, does this store all the packages you require so they can be installed by cmd line? Why do you need this file?
[update, four years later]
bower is deprecated, and should not be used anymore for new projects. To a large extent, it has been subsumed into node dependency management (from their website: "While Bower is maintained, we recommend using Yarn and Webpack or Parcel for front-end projects").
yarn came out of the wood as a better npm (fixing several of npm flaws), and this is really what you should use now, as it is the new de-facto standard if you are doing front-end or node development. It does consume the same package.json as npm, and is almost entirely compatible with it.
I wouldn't use composer at this point (because I wouldn't use php), although it seems to still be alive and popular
[original answer]
npm is nodejs package manager. It therefore targets nodejs environments, which usually means server-side nodejs projects or command-line projects (bower itself is a npm package). If you are going to do anything with nodejs, then you are going to use npm.
bower is a package manager that aims at (front-end) web projects. You need npm and nodejs to install bower and to execute it, though bower packages are not meant specifically for nodejs, but rather for the "browser" environment.
composer is a dependency manager that targets php projects. If you are doing something with symfony (or plain old php), this is likely the way to go
Summing it up:
doing node? you do npm
doing php? try composer
front-end javascript? try bower
And yes, the "json" files describe basic package information and dependencies. And yes, they are needed.
Now, what about the READMEs? :-)
https://github.com/bower/bower
https://www.npmjs.org/doc/cli/npm.html
https://getcomposer.org/doc/00-intro.md