NPM donwloads a lot of files needed for the webpack/libraries. From what i understand, webpack generates a one single bundle file, that contains all code for script working. After that, when i finish building my app, do i need to keep all those jquery/react files and webpack itself? Or should i just delete them?
It's common practice to make a project portable/shareable by following these steps;
Create a package.json and ensure to capture all dependencies,devDependencies and/or peerDependencies.
Add/commit this package.json and package-lock.json files to your version control
Create a .gitignore file and add node_modules to it (in essence, this cuts out that baggage)
For production purpose (e.g. to be shared with client finished product), build the project (which often results into a small files, often within /build or dist). And then you can always push that build file to AWS or Heroku or the clients' servers.
What does the above help you achieve?
You can easily start the project using any machine, as long as you run npm install which reads from your package.json.
Related
I am using VS 2019 for Core 3.1 development and I installed Font Awesome whith Yarn:
yarn add #fortawesome/fontawesome-free
However, whenI try to reference it in my HEAD section like this:
<script defer src="~/lib/#fortawesome/fontawesome-free/js/all.js"></script>
I get the following error:
the 'fortawesome' doesnt exist in actual context
Package managers like yarn, npm, etc. just add the packages to your project. They generally aren't ready to deploy directly at that point, but rather require a build pipeline to create the actual JS/CSS resources. The #fortawesome/fontawesome repo is an exception in that 1) it's not actually a package and 2) the files are already "built". Even then, though, they still won't be in the right location.
I'm not overly familiar with yarn, but npm, for example, puts everything in a node_modules directory. That directory is not served by default, and should not be served, because it contains "raw" stuff. You'd need a separate build pipeline (using npm scripts, webpack, gulp, etc.) to build/copy assets into a directory that is served (i.e. wwwroot). That is likely the piece you are missing.
For this, since there's no build actually required, you just need to have the assets copied to something like wwwroot/lib and then you'll be able to reference it via the script tag on the page. That can be done with any tool like npm, webpack, gulp, grunt, etc. so you'll just have to research and pick the one that fits your needs best.
I've created a web app template that I use frequently for many different projects.
I would like to create an NPM package for it so that it's easier to install for new projects, separate the template from the project files, separate the template dependencies from the project dependencies, and allow easier updating of the template across all projects.
The issue I have is that I need some files/folders to be installed in the root directory (i.e. where package.json is saved). Most can go in the node_modules folder however I have some files that must be placed in the root directory.
For example, the template uses Next.js with a custom _app.js file. This must be in the root directory in a folder named pages. I also have various config files that must be in the root directory.
Can this be done with NPM, or does everything need to be installed in the node_modules folder? I'm having trouble finding anything on SO or Google that answers this, so if you happen to know a guide online on how to do this or can outline things I should search for it would be much appreciated.
With pure npm, everything has to go to the node_modules folder, so you can't solve your issue this way.
Maybe going with a templating tool such as grunt init or yeoman could be a solution here, although – unfortunately – you'll then lose some of the benefits of being able to install a package via npm.
Another option might be to use GitHub template repositories, which have just been introduced recently.
Last but not least one option might also be to just have the files' contents in the npm package, but create the pages/_app.js manually, but inside of it simply require the file contents from an npm module, and that's it. This at least helps to have the content portable, but of course it still asks you to setup the file and folder structure on your own.
Sorry that I don't have a better answer, but I hope it helps anyway.
PS: One "solution" might also be to use the postinstall step in an npm module's package.json file to create folder structure, copy files to where they should be and so on, but at least to me this feels more like a clumsy workaround than like a real solution.
For large web apps npm install resp. yarn install does take a lot of time, mostly in a step called Linking Dependencies. What is happening here? Is it fetching the dependencies of the dependencies? Or something completely different? Which files are created during this step?
When you call yarn install, the following things happen in order:
Resolution: Yarn starts resolving dependencies by making requests to the registry and recursively looking up each dependency.
Downloading/Fetching: Next, Yarn looks in a global cache directory to see if the package needed has already been downloaded. If it hasn't, Yarn fetches the tarball for the package and places it in the global cache so it can work offline and won't need to download dependencies more than once. Dependencies can also be placed in source control as tarballs for full offline installs.
Linking: Finally, Yarn links everything together by copying all the files needed from the global cache into the local node_modules directory after identifying what's already there and what's not there.
yarn install does take a lot of time, mostly in a step called Linking Dependencies
You should notice that Step 3: Linking is taking more time than Step 1: Resolution and Step 2: Fetching where the actual download happens. During by this step we already have things that we need ready and downloaded, then why is it taking long, did we miss anything?
Yes, COPY to local project into node_modules folder...! The reason for this is that this copy is not equivalent to copying one large 4.7GB ISO file. Instead it's multiple super small files (Don't take it light when I say multiple, it can be 15k+ files :P ), hence take a lot of time to copy. (Also, it is important to note that when you download the packages, you download one large tar file per package, whose contents should then be extracted into the cache which also takes time)
It is slower due to
Anti-virus: Your antivirus is sitting in the middle and doing a quick inspect (in addition to our yarn checking if it already exists) on every single file yarn is trying to copy cutting its speed by so much. If you are on Windows, try adding your project's parent folder as exception to Windows Defender.
Storage medium's transfer rate: SSDs can improve this speed hugely (Sorry, SSHDs and FireCudas will not help either, this is gonna be one time).
But is this efficient? Can I have it taken from the global node_modules (after creating one)?
Nope for both questions. Because of the way node works each package finds its dependencies only relative to its own location. Also because each project may want to use different versions of the same package to ensure its working properly and not broken by package updates.
Ideally, the project folder should be lean. An efficient way of doing this would be to have a global node_modules folder. Any and all requested packages are downloaded if not already present AND used from this location. Actually Ruby does it this way. Here's my global Ruby's equivalent of node_modules folder. Notice the presence of different versions of the same package for use in different projects.
But keep in mind that it would reduce project portability. It's a trade-off that any manager (be it rubygems or node modules) has to make. I can just copy the node project folder (which in fact may take hours because you will be copying the (local) node_modules folder as well, but I can expect it to work if I have just that project folder, as opposed to copying a ruby project would only some seconds to few minutes, as there is no local packages (or gems as they call them) folder, but running the project on different system would require those packages to be present on the global gems folder.
The documentation for yarn install can be found here.
You can use the command
yarn install --verbose
Show additional logs while installing dependencies
The output will show what the yarn/npm install is doing.
It's good for debugging in case the process is failing or taking a long time.
The linking phase works in essentially 3 big steps:
Find every file that need to be in node_modules
Check this list versus what is already there and find what need to be copied around
from cache to node_modules
Do the copy
Maybe this issue on Github will help you out.
https://github.com/yarnpkg/yarn/issues/1496
When I build my Electron app for production I still get a node_modules folder with the dependencies. The folder is constituted by:
The dependencies which are installed via package.json I already noticed that I can just delete them from the folder (since their code is inside webpack bundle.js )
ffprobe-static, which actually occupies the largest amount with 40Mb
The nodejs modules such as ajv,deferential, debug, decamelize, etc (158 folders total, while I don't even know most of them, let alone use them directly)
Regarding 2: Is it mandatory to have the binary for ffprobe-static? Can I use ffprobe-static with the ffmpeg.dll given alongside Electron binary?
Regarding 3: Why do I need these and how can I get rid of them? Also, Electron binary already comes with an 18.9Mb node.dll file. Again, can't I use this instead of having again the node_modules?
I am trying to figure out how to integrate Semantic UI with my gulp-based frontend toolchain.
The npm artifact semantic-ui includes an interactive installer that will write a semantic.json file to the root of my project and install the less files, gulp tasks and some configuration into my project. All of these files will be put in subdirectories of a single base directory specified in semantic.json.
I do not want any dependency implementation files or any generated files in the git repository for my project because this will pollute revision history and lead to unneccessary merge conflicts. I would very much prefer to provide semantic.json only and .gitignore the semantic base directory. On npm install, the Semantic installer should install everything to the base directory specified in semantic.json. When building, I want the artifacts generated into a separate dist directory that does not reside under the semantic base directory.
However, if I do this, the installer will fail with a message stating that it cannot find the directories to update and drop me into the interactive installer instead. This is not what I want, because it means my build is no longer non-interactive (which will cause the CI build to fail).
How can I integrate Semantic UI into my build without having to commit Semantic and its generated artifacts into my git repository?
This is what we did in our similar scenario. The following are true:
Everything Semantic UI generates is in .gitignore. Therefore, the only Semantic UI files we have in our repo are:
semantic.json
semantic/src folder (this is where our theme modifications actually are)
semantic/tasks folder (probably doesn't need to be on git, but since it's needed for building, everything is simpler if we keep it in our repo)
We never need to (re)run the Semantic UI installer, everything is integrated in our own gulpfile.js.
Semantic UI outputs in our assets folder which is not in the same folder as its sources.
Semantic UI is updated automatically using npm as per the rules in my package.json.
Here are the steps needed to achieve this:
Install Semantic UI. By this I assume that you either used npm or cloned it from git (using npm is highly recommended), either way, you have semantic.json in your project's main folder and a semantic folder with gulpfile.js, src and tasks.
Make sure Semantic UI can be built. Navigate to semantic/ and run gulp build. This should create a dist folder in your semantic/ directory. Delete it and also delete Semantic UI's gulpfile.js since you'll no longer need it.
Edit semantic.json. You need to edit two lines:
Change "packaged": "dist/", to the path where you'd like to output semantic.css and semantic.js relative to Semantic UI's folder. In our case, it was "packaged": "../../assets/semantic/",
Change "themes": "dist/themes/" in the same way, since the themes/ folder contains fonts and images Semantic UI uses so it needs to be in the same folder as semantic.css. In our case, it was "themes": "../../assets/semantic/dist/themes/".
Edit your gulpfile.js so that it uses Semantic UI's build task. Add var semanticBuild = require('./semantic/tasks/build'); (if semantic/ is in the same folder as your gulpfile.js) and then simply register a task that depends on it, for example gulp.task('semantic', semanticBuild);.
Optionally, create a clean task. We used del for that.
gulp.task('clean:semantic', function(cb) {
del(['assets/semantic'], cb);
});