I know pnpm and yarn reuse modules that we already installed, what, in not updated tutorials that i see, we see that this is something that pnpm and yarn came to fix in npm, which downloaded the modules from the internet every time we install it. This still a thing? Does modern npm save cache or something to speed up installation?
Yes, npm has a cache of package tarballs. It does not download the packages from the internet all the time. In fact, you can verify that by running npm install --offline.
The reason npm is slower than pnpm is because of other reasons:
pnpm uses a content-addressable store. Each file inside the node_modules directory is a hard link to the content-addressable store. This makes pnpm faster and more disk space-efficient.
also, pnpm is running the installation stages separately for every installed package. npm cannot do all these operations concurrently as of the current latest versions (v6 and v7).
There might be other reasons pnpm is faster but these 2 must be the most important ones. npm's cache is not one of the reasons.
Related
Context
I have a .NET project and we use some npm packages for the UI. I use a pre-built check if the folder node_modules does not exist, run the command npm install.
When committing an update to package.json, npm install will not trigger for other people who are updating the repository. Because node_modules exists on their machine which leads to errors like Can't resolve....
Question(s)
Is the folder check obsolete? Is npm install smart enough to download only the necessary things and not all the dependencies? Or do i need a hash check on package.json?
There's no harm in running npm install multiple times as it will simply not do any operation in case there's nothing to do.
You don't need to check for the node_modules folder. npm will download and update any dependencies that are missing.
It's also important to run npm install since other machines may be running different systems and the dependencies may need to be compiled differently.
You can hash check package.json and/or package-lock.json for caching purposes, but it's not really necessary.
I can see create-react-app has added installation with npx. So it made me curious to check which one is better npm, npx or yarn. Which one is better and which is better to use and why?
I don't see why this got negative votes, not everyone comes with inbuilt knowledge on this stuff right ? and this is the place to ask 😅
npm: installation of packages (libraries), i.e. pieces of functionality to help you build your own applications.
npx: npx is a tool to execute packages without installing the packages.
yarn: also installation of packages. yarn is a replacement for npm that sits on top of the same packages repository.
npx isn't the same as the other two, it is a feature of npm to run packages without installing. As for which one is better between npm and yarn, there isn't a clear "winner" (general rule to apply in life too). I personally prefer yarn since in my experience it was faster and less verbose, another positive was it had a lockfile but now npm has one too (and I hear new versions are faster as well).
tl;dr: Either is fine really.
You can compare the feature of npm and yarn. yarn is faster than npm because it is doing parallel installation and npm is doing serial installation of modules. Previous version of npm does not have lockfile now both npm and yarn have lock file. Both are build on the top of same repository.
npx is totally different from npm and yarn. It is a tool to execute packages without installing it.
So I will suggest yarn if you want to decrease the build time of the application.
I have a project with a package.json file and an install bash script that, among other steps, runs npm install.
I'm thinking of updating the script so that it runs yarn install if yarn is available (to take advantage of yarn's caching, lockfile, etc), and falls back to npm install otherwise. As far as I can tell, all the packages seem to install and work ok either way.
Are yarn and npm interchangeable enough for this to be a viable approach, though? Or are there potential issues that this could lead to? Are we meant to just pick one, or is yarn interchangeable with npm in practice?
(nb. I've read this closely related question, but I'm asking this as a separate question because it's about explicitly supporting both yarn and npm install processes in a project)
Yarn and npm (version >=3.0.0) should be relatively compatible, especially moving from npm to Yarn, because compatibility is one of the stated goals of Yarn. As stated in Migrating from npm:
Yarn can consume the same package.json format as npm, and can install any package from the npm registry.
So, in theory, any package.json that is valid for npm should also work equally well for Yarn. Note that I say that npm v2 is probably less compatible - this is because npm migrated from a nested node_modules structure to a flat layout (which is what Yarn uses). That said, Yarn and npm v3 should produce very similar layouts, because, as stated in the issue I linked:
To a first approximation we should try to be very compatible with the node_modules layout for people who need that compatibility, because it'll be the most likely way to avoid long-tail compatibility problems.
However, you will not be able to take advantage of the Yarn.lock generated by Yarn, because (as the name suggests) it's only supported by Yarn, and npm shrinkwrap is not compatible.
Also, as noted by #RyanZim, older versions of Yarn don't support pre- and post-install hooks, but versions later than v0.16.1 do. If you rely on these hooks, you will need to specify to users that versions greater than v0.16.1 are required.
In summary, as long as you encounter no bugs and only use features that are shared by both package managers, you should have no issues whatsoever.
My project references mocha, phantomjs, etc, which takes a lot of time to download during npm install. This is not a problem in my local machine because I only download them once and can use them forever unless I decide to manually upgrade them.
However, in my CI machine, my jenkins server need to download them every time that I did a git commit and git push to do the testing and deploy.
So can I just speed up that process by set the npm not to download these slow packages from the remote server? Rather, install them from local cache or not to install them if I installed them globally?
Anyone knows how to configure that?
I found some packages that might be helpful
npm-install-changed will run npm install only if the contents of package.json's devDependencies and dependencies were changed, note that it assumes that node_modules persists across different builds which might not be helpful if your CI server always start from scratch
npm-install-cache runs npm install and then copies your current node_modules folder (to somewhere in \tmp), if you call the script again it will verify any changes to package.json (instead of changes done on devDependencies or dependencies), if it didn't change then it will copy the node_modules folder stored in \tmp, the only limitation I see is that it's not cross platform and that the cache folder is \tmp which is erased on reboot (or maybe even when a is process finished!)
The second package might not work as it is but it seems like a good place to start :)
You can specify all of the packages you want to use locally in devDependencies in package.json, and then running npm install -d will install those instead of the main dependencies.
When I deploy my app to AWS, it's copied into a new directory, so NPM will install all the same packages, during each deploy, which can take a lot of time. Most of these packages haven't changed between builds (if at all), so having it do a full npm-install seems like a waste.
My app server runs a bunch of different Node apps, so installing globally isn't an option. Instead I'd like to have the app store it's node packages in a location that isn't wiped out during deployment, but have the option to update packages as necessary during npm install.
Does NPM have a concept of an app-specific module directory that isn't located in a subfolder of an app? That way I can delete the app folder, and not have to reinstall the same packages over and over again.
I could achieve this by using symlinks, or migrating the current node_module directory.
If you lock down your dependencies versions, NPM is likely to cache the packages. So the installation wouldn't take much longer.
If you prefer not to do this, you can install dependencies globally and link them with the npm link command (which is basically creating a symlink yourself!). Then, it'll be up to you update the globally installed packages regularly.