How to avoid adding package to node_modules if it placed locally - npm

I have repository with app and modules. Modules includes in app by package.json like:
"application-module": "file:../modules/application-module"
After yarn install that dependency added to node_modules.
I want to make module-base application. app folder reproduce root module. Other modules like admin-panel-module, account-module should be in modules folder. So, app may have node_modules inside, but also modules folder for modules. Those modules will add by git subtree from another repos. So, this way I can develop independently
Is any way to avoid adding and use local directories?

Multiple node_modules and package.json
In any node/npm project, you can have multiple package.json across your directory tree, like:
app/
package.json
node_modules
src...
account_module/
package.json
node_modules
src...
admin_module/
package.json
node_modules
src..
When you invoke yarn (or npm install ofc) on any of the children modules, the dependencies listed in their local package.json will be installed on the local node_modules folder.
So basically you can solve your issue ensuring that every children has their own package.json with their dependencies.
Still, you can place common dependencies in the root app folder. If all your projects for instance use lodash, you can place the lodash dependency in the add's package_json. After performing yarn in the app folder, the lodash package will be installed in the app's node_modules.
After that, if you:
require('lodash');
In any of the children, they will search for lodash in the app's node_modules folder if they don't find lodash in their own node_modules.
If you don't have a root node_modules, you can still declare a package.json local to any of the submodules, and they'll have their own node_modules.
So maybe you may want to avoid common dependencies at all, or maybe you want to store common dependencies in the app folder. Npm has you covered either ways.
However, if you don't want to handle common dependencies, yet are concerned about having to store a lot of duplicated packages in local machines, you may want to checkout pnpm, which is a wrapper over npm which allows to save a lot of space in local development machines.

I'm having (almost) the same issue with yarn.
I have create a package (typescript) called "test". In this package, after built, there are mainly 3 directories : dist (after built), node_modules and src.
Then I have created another package "test2" and in this package, have added "test" as a dependency "yarn add c:/.../.../test".
The package is the well installed in the node_module of test2. BUT, in "test2/node_modules/test", I can find the node_modules of test ( "test2/node_modules/test/node_modules". Why ? It increases the size of the package a lot.
In both tsconfig.json (test and test2) node_modules is excluded...
thks

Related

NPM global install still creating a node_modules directory

In Ubuntu, I have a directory with my code /code which has my packages.json and such. My NPM prefix is /usr via npm prefix -g. I am trying to install packages globally so as to not create a node_modules directory in /code, but really as long as node_modules is not in /code, that will also work.
While in /code, I run npm install -g. I see a .staging directory created under my global prefix (eg, /usr/lib/node_modules/.staging) but then NPM takes another step and begins creating a /code/node_modules directory.
If I instead just run npm install -g <package> it correctly ends up in /usr/lib/node_modules/<package>. But I do not want to have to run npm install -g <package> for every package I need.
Why is NPM doing that and how can I make it stop doing that? At the very least, I would like the node_modules directory to NOT be in my /code directory (this is due to some environment constraints I can't change).
Globally installed packages are (almost always) not going to be used by your code. That's not how node or npm (conventionally) works. There isn't a shared global node_modules that programs all look in. (OK, there kinda is, but you don't want to use it if you can avoid it. Read on.) Having a local node_modules helps you avoid dependency hell. Global modules are for CLI tools that end up in your PATH and things like that.
You don't explain why you don't want a local node_modules for your project, so I will mention that if it is simply because it somehow bothers you, then please please please just get over it. You will be vastly happier. Type npm install and move on.
That said, for legacy reasons, your code in /code will look in /node_modules if there is no /code/node_modules. This is true for CommonJS modules that use require(). If you are using ESM modules via import, you may need to read the docs.
So, if you are using CommonJS/require(), you could have a directory structure like /project-name/code. You could have your index.js or whatever other code in the code subdirectory, while your package.json could be in project-name. Run npm install from /project-name and it will create a node_modules directory there rather than in the code subdirectory, and your Node.js files in code will find those modules.
If you don't like that, then for other options, you can review the relevant docs. At the time of this writing, those options include a NODE_PATH environment variable, $HOME/.node_modules, $HOME/.node_libraries, and $PREFIX/lib/node. However it all comes with this caveat that I implore you to abide by if you can:
It is strongly encouraged to place dependencies in the local node_modules folder. These will be loaded faster, and more reliably.

Install other package.json dependencies

Simple question : Is it possible, in a package.json, to reference another package.json, and install its dependencies ?
Thank you.
Yes, this is possible, and this is automatically done by npm install.
If you have pkg-a that depends on pkg-b, including pkg-a in your dependencies will install both pkg-a and pkg-b when running npm install. That is because dependencies are actually references to the package.json of other packages. NPM, upon running install, builds a dependency tree of all the packages that are indirectly required by your current project, and installs all of them in the node_modules directory, and keeps track of them all in package-lock.json.
Good question! but this is not possible as you cannot internally reference one json document from another (json is just a document format, it lacks any ability to process logic, import files etc), npm is configured to run using a single package.json file so your best best would be to put all your dependencies in a single package.json file or split your project into two directories with two separate package.json files, two npm installs etc, if for some reason you require your dependencies to be separate. You could then run your two node projects separately and connect via http if you wish.
The only way that you could come close to doing this would be to write an npm start script in the package.json that cds to another directory with a package.json and runs npm install, this would however only install the dependencies in the second directory node-modules/ folder

How to prevent nested node_modules inside node_modules

I've created my own npm package, let's call it XYZ, it has #material-ui dependency in it's package.json file.
When I install it in project A I have nested node_modules inside of XYZ folder(so it's A\node_modules\XYZ\node_modules\#material-ui), but when I install it in project B I don't have nested node_modules folder. Both project A and B has #material-ui in their package.json files with same versions.
How to force my XYZ package to use #material-ui from A\node_modules?
There are upside of having less nested folders and downside having more folders in node_modules folder directly and version control problems.
Use correct npm version
Correct yarn and npm (ie: npm v3) should not have such structure issue. It should always flatten the whole structure where possible and only have nested node_modules if the versions are incompatible with the one at top.
Check versions
So if you have it working properly on one project and not on another, its probably due to version. Check out if the #material-ui is same version on both. Maybe two different packages are conflicting with each other at some point.
Check how you are installing them
From your question, it says it's same version. However, you did not mention how you installed your package on both project. If you install with yarn link or npm link it should install dependencies properly as expected.
Check if you are using different packages
If you check the package, recently material-ui has been deprecated, and the notice says to upgrade to #material-ui/core instead. It might be some packages inside that folder is not same. Either way, it's like this whenever there is some dependency conflict. Check inside the #material-ui folder.
Flatten them manually (dangerous)
There are several packages to forcefully resolve this issue. They will go thru the nested node_modules folders and flatten them into single folder.
flatten-packages
Install with, npm install -g flatten-packages.
Run executable flatten-packages to rearrange all packages in node_modules folder in the project directory.
Flatten will delete older version of a package. You should take care of version breaking changes related errors.
You can use npm dedupe command to accomplish this.
You can put the command in postinstall script in package.json, and every time NPM installs package, the npm dedupe command will flatten all the duplicated packages in same version for you.
For more information, see https://docs.npmjs.com/cli/dedupe
npm postinstall script
I had the same issue in a React Native app with my NPM package.
The problem was that in project A the version of React Native used was (0.59.5) below the version used in my package (0.59.8).
Installing the package in a brand new project (B), of course was using the latest version of React Native in that moment, that was the same of my package (0.59.8).
I have another addition to the accepted answer:
Clear Local node_modules folder Cache
rm -rf node_modules
Handle with care: Sometimes migrating projects to new npm modules can cause weird cache issues inside a node_modules folder, especially those that have been around for a while, or happened to have newer versions of packages installed in sub-dependencies that differed from the installed version in root.
Once you remove direct dependencies via the package.json dependencies, the packages will be removed from the <root>/node_modules. This can cause a bug where the new modules are still nested under your dependency instead of being moved to root as expected.
So by wiping out your local node_modules, you can do a clean reinstall and let the flattening to its work.

dependencies and devDependencies when using webpack

Does it make sense to put any modules into package.json dependencies when I use webpack?
When I want to develope a package, I use git clone <url> then npm install, then npm installs all dependencies and devDependencies from package.json file and it makes sense.
When I'm end-user and I just want to install some package into my node_modules to use it in my project, I run npm install package-name, then npm installs package-name with only its dependencies, and it makes sense too.
But does it make sense to put any modules into dependencies when I use webpack? The webpack will bundle all dependencies into eg. bundle.js, so, for me, there is no need to install dependencies then (while they're included into bundle.js file).
Let's assume that I put all neccessary modules into devDependencies (keep the dependencies object empty) for my project: my-project, bundle it with webpack and publish:
the developer-user will use git clone <url to my_project>, then run npm install, then npm will install devDependencies from package.json (and ommit empty dependencies object), then it is ready to develope.
the end-user will use npm install my-project, then npm will install my-project, do not install devDependencies (because this is for production) and do not install dependencies (because dependencies object in package.json remain empty). Putting anything into dependencies would double the dependencies: both the dependencies would be installed, and the same dependencies would be accessible in the bundle.js file.
Am I right?
You are correct that there may be no dependencies once it's been transpiled with webpack. However, some packages are multi-purpose and may be used in multiple ways, so in some circumstances dependencies may still be required.
If you look at the package.json specification, there are two possible entry points, 'main' and 'browser'. There is also the proposed 'module' entry point. It is currently under discussion about how to handle these in webpack and users seem to want webpack to prioritize them as module > browser > main, however browser is currently used by webpack first.
The idea behind prioritizing them in the order module > browser > main is presumably that browsers could use the pre-transpiled stuff directly in "browser", whereas another project calling require() or include() on your package would use non-transpiled code from the "module" entry. The "module" entry code could contain modern JavaScript with new features and the project/package requiring it could then transpile it to their own specifications, using "browserslist" for example.
I found this question because I was wondering the same thing...

npm node_modules not being properly nested?

I'm getting unexpected behavior when using npm. For example when installing express with the command:
npm install express
I would expect that a folder named, "express" would be created in the "node_modules" directory and that it's dependencies would be installed within a "node_modules" sub-directory within this folder.
What I am seeing is that the "express" folder is being created but all of it's dependencies are being added to the root "node_modules" directory (same level as express) in my project folder and not nested within the "express" folder.
Why is this happening? (using npm v3.3.5)
It's a design change for npm#3, it deduplicates by default. See:
Flat, flat, flat!
Your dependencies will now be installed maximally flat. Insofar as is
possible, all of your dependencies, and their dependencies, and THEIR
dependencies will be installed in your project's node_modules folder with no
nesting. You'll only see modules nested underneath one another when two (or
more) modules have conflicting dependencies.
https://github.com/npm/npm/blob/ff47bc138e877835f1c0f419fea5f5672110678a/CHANGELOG.md#flat-flat-flat
https://github.com/npm/npm/issues/6912