I made a project using GULP and I downloaded several modules with the NPM cmd. For a new project, should I copy/paste my old project with the files "gulpfile.js", "package.json" and the folder "node_module" or i have to redownload all the modules ? I tried to copy/paste and it worked perfectly, but maybe there is an another way to do it (like download each module with one command line) ?
Thank you !
Copying your old package.json file into your new project is perfectly reasonable when you also use a copy of your existing gulpfile.
Now, instead of copying your node_module folder just run npm install. It does exactly the same thing as copying and pasting but with the added bonus of checking package versions, deprecated dependencies, and so on...
Quick tip: If you ever run into problems as I have deleting/removing packages given that theirs paths might be longer than what the OS allows use the tool rimraf. Install it globally and then you can call rimraf on anything. CAUTION! this tool will erase everything no questions asked
Related
I'm trying to set up an npx script to create a template project.
In package.json I have:
"bin": {
"init": "bin/init"
}
In the init script I'm using tag='v'$(npm pkg get version | tr -d '"') to get the version of the package. I then use git clone --depth 1 --branch $tag https://github.com/matriarx/typescript.git to clone that specific repository for that specific tag.
When I do yarn link and try use it locally, from within that specific project, it works because it's able to correctly pick up the package.json version. So the above only works if it's run inside an existing project. However that's not what I want to do.
I want to enable someone to run it even if they have nothing locally, by simply doing npx #matriarx/typescript init and it should create the new project by cloning it. More than that I want them to be able to clone any specific version by using npx #matriarx/typescript#0.0.1 init in order to clone a specific version.
However it seems that anything I try is only able to get the version from a local package.json that already exists.
I could just clone the current existing repository without specifying any tag, but that would defeat the point of having releases, then it would just clone any current code completely disregarding the release. So it has to clone the tagged release.
How can I get the remote package version stored on npm from the bin script without having anything locally before hand?
Alternatively is there a better way to do what I'm trying to do?
EDIT: I ended up just hardcoding the version in the script, which works but it sucks because it's tedious to have to update it every time I bump the version. Though for now I still don't know a better way to do it.
After some more time messing around I figured out there is a standard way of doing it, at least since npm 7.
If you have a project like example then you can create a completely separate project called create-example with a bin script.
When you use npm init example, npm will automatically search for a package prefixed with "create-" and execute its main bin script. So when running npm init example it will search for that create-example package and execute the bin script, which will install the example package.
This is how most of the bigger packages like react and next do it.
This approach comes with some disadvantages that I really don't like, for example it will show the incorrect dependencies on npm and it will cause you to have to maintain multiple projects and semvers on different projects. However it will also allow you to create a clean separation between the actual project and the installation of that project.
For some projects it might also make a lot more sense. For example if you have a project that doesn't have a package.json at all and you want to create a setup for it, it wouldn't make sense to create an npm package inside that project just for that. Instead you can create a separate "create-project" package just to set it up and keep npm out of the actual project. In other words it gives you a neat way to create bin scripts for a completely separate project that doesn't have anything to do with npm.
You could also just have created a normal shell script and execute it using curl but I guess npm just gives you another way to do it.
You still have to hardcode the version in that "create-project" package, I still have not seen a way to automatically determine the version from a remote package. The only way I've managed to do that is to completely download the package, get the version, then delete it, but that would be terrible for people with a slower internet connection or limited data.
Up until yesterday, whenever I installed a package (e.g. nodemon), they would be added to the node_modules dir. This morning, when I installed a bigger project, the powershell and cmd files were saved outside of that directory, in addition to a file without an extension. It looked like this:
dependencies
Why is example, example.cmd and example.ps1 saved outside of node_modules?
I would like to answer my own question. I had accidentally messed around with the npm prefix. If this happens to you, go to the cmd prompt and type npm set prefix C:/bin and you should be alright :)
I am using Vue 2 (doesn't really matter which version exactly).
I want to test some things that happen behind the hood in Vue. So I decided to add console.log('test123') in Vue's files in node_modules. It turns out that the console log never fires. I put that in all files of Vue in node_modules, even in all files of dist's folder of Vue.
How can I achieve this ? If I fork the repo, then I'd have to upload new versions each time on my repo and then run npm install. I know that will work but wanted to achieve this without forking.
Any ideas what I am missing ?
there are many ways .. but i feel more comfortable using this method :
you can download any npm package in a seperated folder next to your project...
open the folder of the package then run this in the terminal:
npm link
then open the project folder and run
npm link ../package-path # link the dir of your dependency
References
npm-link
How to test an npm package locally
I've created a web app template that I use frequently for many different projects.
I would like to create an NPM package for it so that it's easier to install for new projects, separate the template from the project files, separate the template dependencies from the project dependencies, and allow easier updating of the template across all projects.
The issue I have is that I need some files/folders to be installed in the root directory (i.e. where package.json is saved). Most can go in the node_modules folder however I have some files that must be placed in the root directory.
For example, the template uses Next.js with a custom _app.js file. This must be in the root directory in a folder named pages. I also have various config files that must be in the root directory.
Can this be done with NPM, or does everything need to be installed in the node_modules folder? I'm having trouble finding anything on SO or Google that answers this, so if you happen to know a guide online on how to do this or can outline things I should search for it would be much appreciated.
With pure npm, everything has to go to the node_modules folder, so you can't solve your issue this way.
Maybe going with a templating tool such as grunt init or yeoman could be a solution here, although – unfortunately – you'll then lose some of the benefits of being able to install a package via npm.
Another option might be to use GitHub template repositories, which have just been introduced recently.
Last but not least one option might also be to just have the files' contents in the npm package, but create the pages/_app.js manually, but inside of it simply require the file contents from an npm module, and that's it. This at least helps to have the content portable, but of course it still asks you to setup the file and folder structure on your own.
Sorry that I don't have a better answer, but I hope it helps anyway.
PS: One "solution" might also be to use the postinstall step in an npm module's package.json file to create folder structure, copy files to where they should be and so on, but at least to me this feels more like a clumsy workaround than like a real solution.
For large web apps npm install resp. yarn install does take a lot of time, mostly in a step called Linking Dependencies. What is happening here? Is it fetching the dependencies of the dependencies? Or something completely different? Which files are created during this step?
When you call yarn install, the following things happen in order:
Resolution: Yarn starts resolving dependencies by making requests to the registry and recursively looking up each dependency.
Downloading/Fetching: Next, Yarn looks in a global cache directory to see if the package needed has already been downloaded. If it hasn't, Yarn fetches the tarball for the package and places it in the global cache so it can work offline and won't need to download dependencies more than once. Dependencies can also be placed in source control as tarballs for full offline installs.
Linking: Finally, Yarn links everything together by copying all the files needed from the global cache into the local node_modules directory after identifying what's already there and what's not there.
yarn install does take a lot of time, mostly in a step called Linking Dependencies
You should notice that Step 3: Linking is taking more time than Step 1: Resolution and Step 2: Fetching where the actual download happens. During by this step we already have things that we need ready and downloaded, then why is it taking long, did we miss anything?
Yes, COPY to local project into node_modules folder...! The reason for this is that this copy is not equivalent to copying one large 4.7GB ISO file. Instead it's multiple super small files (Don't take it light when I say multiple, it can be 15k+ files :P ), hence take a lot of time to copy. (Also, it is important to note that when you download the packages, you download one large tar file per package, whose contents should then be extracted into the cache which also takes time)
It is slower due to
Anti-virus: Your antivirus is sitting in the middle and doing a quick inspect (in addition to our yarn checking if it already exists) on every single file yarn is trying to copy cutting its speed by so much. If you are on Windows, try adding your project's parent folder as exception to Windows Defender.
Storage medium's transfer rate: SSDs can improve this speed hugely (Sorry, SSHDs and FireCudas will not help either, this is gonna be one time).
But is this efficient? Can I have it taken from the global node_modules (after creating one)?
Nope for both questions. Because of the way node works each package finds its dependencies only relative to its own location. Also because each project may want to use different versions of the same package to ensure its working properly and not broken by package updates.
Ideally, the project folder should be lean. An efficient way of doing this would be to have a global node_modules folder. Any and all requested packages are downloaded if not already present AND used from this location. Actually Ruby does it this way. Here's my global Ruby's equivalent of node_modules folder. Notice the presence of different versions of the same package for use in different projects.
But keep in mind that it would reduce project portability. It's a trade-off that any manager (be it rubygems or node modules) has to make. I can just copy the node project folder (which in fact may take hours because you will be copying the (local) node_modules folder as well, but I can expect it to work if I have just that project folder, as opposed to copying a ruby project would only some seconds to few minutes, as there is no local packages (or gems as they call them) folder, but running the project on different system would require those packages to be present on the global gems folder.
The documentation for yarn install can be found here.
You can use the command
yarn install --verbose
Show additional logs while installing dependencies
The output will show what the yarn/npm install is doing.
It's good for debugging in case the process is failing or taking a long time.
The linking phase works in essentially 3 big steps:
Find every file that need to be in node_modules
Check this list versus what is already there and find what need to be copied around
from cache to node_modules
Do the copy
Maybe this issue on Github will help you out.
https://github.com/yarnpkg/yarn/issues/1496