How can I have npm tell me if all dependencies are already installed without checking the network?
My goal is to first test locally if anything needs to be installed, and then only if something is missing, I'll run a normal npm install to install it. I'm trying to avoid the initial check across the network though if everything is already there.
This is also given a package.json file with fixed versions, since obviously allowing auto upgraded packages would always require a remote repository check.
Update:
I've tested npm list which doesn't seem to access the network, and it prints out "UNMET DEPENDENCY" if something is in package.json but not installed. Is that the best way to accomplish this?
I'll probably end up with something like:
npm list | grep -c 'UNMET DEPENDENCY'
I'm not aware of anything that will explicitly tell you whether dependencies were installed from a remote repo or not. However I think that the shrinkpack package will help you achieve your aim.
Shrinkpack will cache your npm modules locally and only contact a remote repository when existing modules change or new modules are added.
I've used this in the past to reduce the number of network requests required for an npm install.
Related
I'm more familiar with the Python ecosystem at this point and have a question about how I can do something with npm that I'm used to doing with pip.
Let's say I have a wheel for a particular Python package, as well as a wheel file for each of the Python package's dependencies. And let's say I have all these wheel files in a folder called /path/to/wheel/files. To install this package and all of its dependencies, I could run something like pip install /path/to/wheel/files/*.whl --no-deps, where --no-deps keeps me from having to install the various dependencies in the proper order.
Does npm have an equivalent to this? I'm using npm-offline-packager to create a tarball that contains a Node package (as its own tarball) and all of its dependencies (as their own tarballs). I know I can tell npm install to install a particular tarball. However, when I do this, it tries pulling in the required dependencies from the online NPM registry instead of pulling in the dependencies from the tarballs I already have.
Ideally, I'd like npm install to use the tarballs to add the main package to my project's package.json while adding the package's dependencies to my project's package-lock.json. And of course, I'd also like the main package and all its dependencies to be installed to my project's node_modules directory as well.
TL;DR Does npm have something equivalent to pip install /path/to/wheel/files/*.whl --no-deps?
I'm responding to my own question here, but note that my answer is only applicable to my particular use case and may not be applicable in general.
For my use case, I have access to two computers: one that has access to the internet and one that doesn't. For the machine that doesn't have access to the internet, I was attempting to use Verdaccio as a way of creating a self-hosted NPM registry. However, publishing packages to Verdaccio wasn't working because it kept trying to pull in the package's dependencies from the public NPM repository. The solution was to remove all references to "npmjs" in Verdaccio's config file (which, for me, Verdaccio created at ~/.config/verdaccio/config.yaml).
So, in case anyone needs to do development on a machine that doesn't have access to the internet, the process for setting up Verdaccio looks something like this:
On the machine that has access to the internet, create an NPM project using npm init (I called my project "verdaccio_runner"). The reason I did this is because, without already having an NPM registry on the machine that doesn't have access to the internet, it was hard doing a global install of Verdaccio.
Run npm install verdaccio to install Verdaccio to the NPM project that was created in the previous step.
Transfer this project over to the machine that doesn't have access to the internet.
Once it's transferred over, run Verdaccio from the project like this: npx verdaccio.
Quit out of Verdaccio.
Remove all references to "npmjs" from the config file that Verdaccio created (again, mine was at ~/.config/verdaccio/config.yaml).
Run Verdaccio again to pull in those changes.
Tell NPM where your private registry is: npm config set registry http://localhost:4873/.
Add yourself as a user by running npm adduser and by then filling out the information you're prompted for.
And the process for publishing packages to Verdaccio on a machine that doesn't have access to the internet looks like this:
For the package you want to install, on the machine that has access to the internet, run npo fetch <package name> --no-cache (assuming you've already done a global install of npm-offline-packager on the machine that has internet access).
Bring the tarball that npo created for you over to the machine that doesn't have internet access.
Untar the tarball.
From the directory that's created, run for file in ./*.tgz; do npm publish $file; done.
The published packages can now be npm installed to projects on the machine that doesn't have internet access.
Note: in order for Verdaccio to be accessible to other machines on the private network, I also had to add the following to Verdaccio's config file:
listen:
0.0.0.0:4873
I am on different servers and run npm install
One server is ok. Another failed showing below error:
no matching version found for es-abstract#1.14.0
I tried npm ci. Same deal.
Then I did npm install es-abstract-1.14.0.tgz
But the size node_modules/ are different. I am using du -shc --apparent-size node_modules/ to ignore the sparse files within the folder. Why?
Could I accept that this is the network issue of the failed server？ And just keep doing this? I mean, is the installation still going to be ok in this way?
At the time that this answer was written, there are 33 release versions for es-abstract on github, but only 32 release versions are listed on its npm registry. The missing version in the registry is 1.14.0.
Perhaps on one of your servers, you had this package cached (maybe it was previously listed on the NPM registry?, or maybe from downloading it from github?), and on the other server you did not have this package cached.
I had this same error message when trying to npm install a project from github.
In my situation, the es-abstract package was not explicitly listed in the package.json file, but it was a dependency of another package. Therefore I explicitly added it with the next highest version listed on the registry, and it worked.
e.g.
"dependencies": {
"es-abstract": "1.14.1",
...
Just a guess, but maybe 1.14.0 used to be listed on the registry, but now it's not?
I have found a react-native module I would like to use. It contains an error which prohibits the installation.
I have forked the repository and corrected the mistake.
Now I want to install my module in order to use it, however during execution of the post installation scripts I get an error - file not found.
I have tried to find the reason for it, but the reason is quite simple, the module is not in the node_modules directory, and when npm tries to "enter" there to run the scripts, it can't find them.
I have tried to check where this is installed, but I can't.
I use npm install <user>/<repo> to install my module.
My project references mocha, phantomjs, etc, which takes a lot of time to download during npm install. This is not a problem in my local machine because I only download them once and can use them forever unless I decide to manually upgrade them.
However, in my CI machine, my jenkins server need to download them every time that I did a git commit and git push to do the testing and deploy.
So can I just speed up that process by set the npm not to download these slow packages from the remote server? Rather, install them from local cache or not to install them if I installed them globally?
Anyone knows how to configure that?
I found some packages that might be helpful
npm-install-changed will run npm install only if the contents of package.json's devDependencies and dependencies were changed, note that it assumes that node_modules persists across different builds which might not be helpful if your CI server always start from scratch
npm-install-cache runs npm install and then copies your current node_modules folder (to somewhere in \tmp), if you call the script again it will verify any changes to package.json (instead of changes done on devDependencies or dependencies), if it didn't change then it will copy the node_modules folder stored in \tmp, the only limitation I see is that it's not cross platform and that the cache folder is \tmp which is erased on reboot (or maybe even when a is process finished!)
The second package might not work as it is but it seems like a good place to start :)
You can specify all of the packages you want to use locally in devDependencies in package.json, and then running npm install -d will install those instead of the main dependencies.
This might be a stupid question but I believe I should know this since I am just starting out in the web development field rather than just assuming. I normally see this
Install via npm or bower (recommended) or manually download the package
or something of that sorts. My Assumption is that the node_module and bower_component updates the packages automatically, however I am not sure.
Sometimes I install with npm or bower, or sometimes I just mannually download the package to which I have seen no difference. Could someone please tell me why it is important to install via npm or bower so I can know for sure what is going on.
Package managers allow you to keep third party code separate from your code and have consistent versions of that code. With npm or bower you can set out exactly what dependencies you project has, and what versions through a single file, without having to bloat your codebase with the dependencies themselves.
This means that anyone who wants to set up the project can just download the core code and run npm install or the equivalent command, and install all the dependencies at the latest supported version.