Using the search I already found some great answers to similar questions, but still I am not sure if I understood it correctly.
From these answers I learned that dependencies are required to run the application while devDependencies are only required while developing (like unit tests).
But how about this: My application depends on jQuery, but during a build step (with the help of my devDependencies), everything is bundled into one file. In this case, should I list jQuery as dependency or as devDependency?
To make my point more clear take a module like this:
define(['jquery'], function($) {
// use jQuery in this module
})
Later on, this module will be compiled into somehing like application.build.js which then contains this module and the jQuery dependency.
Since the end result is the same, there doesn't seem to be a definite rule here, but I found a couple of discussions on the matter:
If you're building an application
https://github.com/webpack/webpack/issues/520
A browser app built by [insert build tool/bundler] has no runtime node dependencies, and thus all frontend dependencies should be listed as devDependencies. The dependencies vs devDependencies naming convention stems historically from node being a server side package manager (...) It is as far as I can tell harmless to list frontend dependencies under dependencies, but it is wrong.
(...) as a general recommendation for everyone, move everything into devDependencies until it is actually needed under dependencies.
If you're building a library:
https://github.com/inuitcss/inuitcss/issues/225
In many frontend projects, all code served to the browser is compiled, there are no runtime dependencies. This would mean that there are no dependencies, only devDependencies – all dependencies are included in the build done during development.
One could also argue that dependencies are required for development as well, so it would be okay to list everything unter dependencies.
I think the fact that we have the optional distinction indicates a reasonable way to use them. It makes sense (to me) that the dependencies designation would represent the 'minimum viable' code to use and as an indicator of what's nonessential for something to work.
As I see it, anything that goes on to become part of the production code is a dependency.
Epilogue
Personally, I agree more with the last quote. It makes sense that dependencies tells us what the application code needs to run, and devDependencies what the developer needs to build/deploy/whatever the application/library.
One caveat though is that if someone npm installs your library to use the bundle as a module in their own application, they will download a lot of dependencies they don't actually need.
Related
I've run into a situation where I need to access a transitive dependency in a stable way. In other words, I'd like to declare a dependency whose version is "whatever library Foo is already using".
Specifically, I'm setting up an Eleventy site and want to use markdown-it-anchor with it. Both libraries involve slugifying text, for which markdown-it-anchor allows you to specify a custom function. To keep everything consistent, I want to tell markdown-it-anchor to use the same slugifying function as Eleventy. Eleventy doesn't export its slugifying function, but it's just using #sindresorhus/slugify, so I can import that directly.
The problem came in when I added a direct dependency on #sindresorhus/slugify — I added a dependency with a splat version, on the assumption that npm would simply resolve it to the version of #sindresorhus/slugify which was already present in node_modules/. Instead, it resolved to the latest version. I tried playing around with editing package.json and even package-lock.json manually, but npm is very firm about installing Eleventy's version of #sindresorhus/slugify where I can't reach it and not installing the same version for my own use unless I duplicate the version specifier in my package.json.
What I want is to be able to freely update Eleventy in the future and have reasonable confidence that markdown-it-anchor will continue to be passed the correct version of #sindresorhus/slugify without having to manually verify each time that Eleventy hasn't bumped their dependency. Is there any way to accomplish that?
Well, I kind of got this to work. I say "kind of" because I'm depending on the splat version ("*") of my transitive dependency, which is pretty fragile. Getting it to work at all was pretty ugly, too, so there are multiple ways in which this isn't a "proper" solution.
Opened up package-lock.json and looked at the transitive dependency's dependencies (and transitive dependencies). Fortunately, for #sindresorhus/slugify, this isn't too bad.
Rearranged node_modules/ to move the transitive dependency to the top level (where my package can find it), and all of it's dependencies to where it can find them, without introducing new version conflicts. Again, in my case, this wasn't too bad.
Edited both package-lock.json and node_modules/.package-lock.json to reflect the moved packages' new locations.
Ran npm ci both to verify that I hadn't made any terrible mistakes and to make sure package-lock.json and node_modules/.package-lock.json were formatted the way it liked. (The only change it made was to reorder the packages to keep their directories sorted.)
Manually added a dependency on the (now formerly) transitive dependency with a splat version.
Ran npm install and verified that it didn't actually install or rearrange anything.
After all that, #sindresorhus/slugify works as expected when used directly from my site's build. There are a couple of serious caveats, though:
I'm not sure what npm's behavior will be if/when Eleventy updates its dependency on #sindresorhus/slugify. It may well simply update the latter where it's already located, in which everything will be fine. Otherwise, it probably won't.
I'm also not sure what npm's behavior will be if/when #sindresorhus/slugify gets added as a dependency anywhere else. It may well leave the existing version where it is and install new, conflicting versions under the …/node_modules/ folder of whatever packages require them, in which case everything will be fine. Otherwise, it probably won't.
In other words, I discovered a way to put a fair amount of effort into creating a situation which seems to work, but may not actually do what I originally wanted. 😅
I'm coming from a background much more familiar with composer. I'm getting gulp (etc) going for the build processes and learning node and how to use npm as I go.
It's very odd (again, coming from a composer background) that a composer.lock-like manifest is not included by default. Having said that, I've been reading documentation on [shrinkwrap], [npm-lockdown], and [npm-seal]. ...and the more documentation I read, the more confused I become as to which I should be choosing (everyone thinks their way is the best way). One of the issues I notice is that npm-seal hasn't changed in 4 years and npm-lockdown in 8 months -- this all leads me to wonder if this because it's not needed with the newest version of npm...
What are the benefits / drawbacks of each?
In what cases would I use one over another in Project A, but use a different one in Project B?
How will each impact our development workflow?
PS: Brownie points if you include the most basic implementation example for each. ;)
npm shrinkwrap is the most standard way how to lock your dependencies. And yes, npm install does not create it by default which is a pity and it is something that npm creators definitely should change.
npm-lockdown is trying to do the same things as npm shrinkwrap, there are two minor points in which npm-lockdown is better: it handles optional dependencies better and it validates checksums of the package:
https://www.npmjs.com/package/lockdown#related-tools
Both these features seem not so relevant for me; I'm quite happy with npm shrinkwrap: For example, npmjs guarantees that once you upload certain package at certain version, it stays immutable - so checking sha checksums is not so hot (I've never encountered an error caused by this).
seal is meant to be used together with npm shrinkwrap. It adds the 'checksum checking' aspect. It looks abandoned and quite raw.
Good question - I'm going to skip everything but shrinkwrap because it is the de-facto way to do this, per NPM's docs.
Long story short the npm-shrinkwrap.json file is akin to your lock files you are used to in every other package manager, though NPM allows different versions of the same package to play nice together by isolation - literally scoping and copying different entire versions to node_modules at different levels of the tree. If two projects that are parent-child to each other use the exact same version, NPM will copy the version to only the parent and the child will traverse up the tree to find the package.
Best practice is simply to update package.json for your direct dependencies, run npm install, verify that things are working while developing, then run npm shrinkwrap when you are just about to commit and push. NOTE: make sure to rm npm-shrinkwrap.json before running npm install during active development - if your direct dependencies have changed, you want package.json to be used, and not the lock! Also include node_modules in your .gitignore or equivalent in your source control system. Then, when you are deploying and getting to run the project, run npm install like normal. If npm finds an npm-shrinkwrap.json file, it will use that to recursively pull all locked modules, and it will ignore package.json in both your project and all dependent projects.
You might find shrinkpack useful – it checks in the tarballs which npm install downloads and bundles them into your repository, before finally rewriting npm-shrinkwrap.json to point at that local bundle instead.
This way, your project is totally locked down, completely available offline, and much quicker to install.
I am new to browserify and its usage is not completely clear to me although the benefits seem to be compelling.
I have a couple of questions and was hoping someone could clarify.
I've seen blog posts about using browserify-shim in the package.json to include third party libraries like jquery and bootstrap. I've also seen posts where tools like gulp are used generate the bundle file. I can't seem to find a good answer on why browserify-shim is required if tools like gulp are able to automate the process. Can someone please shed some light? Why is browserify-shim even required? I feel the gulp solution is a little cleaner although a little more involved. It won't pollute package.json with browserify specific stuff that is a build thing and therefore goes together with gulp (just my personal opinion)
How does one deal with third party libraries that are not in npm and also not public? For example, we purchase a script from a third party. That script is not any common js, but is a regular js file with some dependencies (example, on jquery and underscore).
Browserify lets you take the world of Node and bundle it up for delivery to a browser. It understands Node modules, which define dependencies via CommonJS require statements.
But what if you have some JS code or library that is not defined as a Node module and does not support CommonJS? Enter browserify-shim. It provides a shim wrapper around any script, like your private third party script, so that it can be defined as and used as a Node module that Browserify understands.
The use of browserify-shim is completely orthogonal to how you execute Browserify. There are basically two options there: A) Use Browserify's command line API or B) Use Browserify's JS API.
Using a build tool, like Gulp, implies the second option, since you'd use Browserify's JS API in your Gulp build script (i.e. gulpfile.js). A lot of people prefer the use of Gulp because it has a good ecosystem of plugins that allow you to do a lot more than just call Browserify (e.g. compile CoffeeScript, compile SASS, run JSHint, etc).
So, to answer your specific questions:
Browserify-shim is only required if you have JS code that is not written as a Node/CommonJS module that you need to bundle via Browserify. To do so, you will need to tell browserify-shim which files to shim as modules (and what dependencies they have) in package.json. Note that this is totally unrelated to Gulp; so you will need it whether you use Gulp or not.
What you describe is the perfect use-case of browserify-shim. Put your third party script(s) in your project, configure the files to be modules in package.json per b-shim's documentation, require them in your code, and execute Browserify to bundle them up with your code. You could also bundle them up separately, put them in their own project, etc - however you want to structure it.
A couple things to note:
You can shim just about any JS library this way.
Shimming a JS library to be a Node module changes global scope to be private scope. Hopefully everything in the library is namespaced so that all of its functionality can be exported as a single module, but if it's not, you might have to modify the shimmed code to explicitly attach things to window (which is easy but not recommended) or split the code up into separate files/modules.
Both Browserify and Gulp use streams in their JS API, but Browserify uses native Node streams while Gulp uses Vinyl streams. Since they don't play well together, you'll probably have to use vinyl-source-stream to adapt Gulp to Browserify (e.g. for renaming files in a Browserify pipeline), or use vinyl-transform to adapt Browserify to Gulp (e.g. wrap a Browserify stream for use in a Gulp pipeline).
Hi i have a quite simple question about maven dependency. I have 2 modules of one project. So first module is dependent on second. Second module has dependencies on some libs. Second module can be itself installed as standalone app. So when i build the project the first war will contain packaged second module as well as all libs that second module depends on.
I need that when i package first module the second module should be included without it's dependencies. How it is possible to manage?
Thx
I need that when i package first module the second module should be included without it's dependencies. How it is possible to manage?
Somehow, this is an hint that the dependencies of the 2nd module are provided when it gets packaged inside the war. IOW, declaring the dependencies of the 2nd module with a provided scope would do it, they wouldn't get pulled transitively.
Depending on how you create the standalone distribution of the 2nd module, you might need to combine dependencies scope with profiles, or not. Can't say since you didn't say anything about this :)
References
Dependency Scope
Suppose, I have an opensource project that depends on some library, that must be patched in order to fix some issues. How do I do that? My ideas are:
Have that library sources set up as a module, keep them in my vcs. Pros: simple. Cons: some third party sources in my repo, might slow down build process, hard to find a patched place (though can be fixed in README)
Have a module, like in 1, but keep patched source files only, compile them with orignal library jar in classpath and somehow replace *.class files in library jar on build. Pros: builds faster, easy to find patched places. Cons: hard to configure, that jar hackery is non-obvious (library jar in repository and in my project assembly would be different)
Keep patched *.class files in main/resources, and replace on packaging like in 2). Pros: almost none. Cons: binaries in vcs, hard to recompile a patched class as patch compilation is not automated.
One nice solution is to create a distinct project with patched library sources, and deploy it on local/enterprise repository with -patched qualifier. But that would not fit for an opensourced project that is meant to be easily buildable by anyone who checks out its sources. Or should I just say "and also, before you build my project, please check out that stuff and run mvn install".
One nice solution is to create a distinct project with patched library sources, and deploy it on local/enterprise repository with -patched qualifier. But that would not fit for an opensourced project that is meant to be easily buildable by anyone who checks out its sources. Or should I just say "and also, before you build my project, please check out that stuff and run mvn install".
This is what I would do (and actually what I do) for both a corporate and an opensource project. Get the sources, put them under version control in a distinct project, patch them, rebuild the patched library (and include this information in the version, something like X.Y.Z-patched), deploy it to a repository (you could use SVN for this, a la Google Code1), declare the repository in your POM and update the dependency to point on your patched version.
With this approach, you can say to your users: check out my code and run mvn install and they will just get the patched version without any extra action. This is IMHO the cleanest way (not error prone, no class path order mess, no increase of the build time, etc).
1 Lots of people are deploying their code to their hosted subversion repository (how-to in this post).
One nice solution is to create a distinct project with patched library sources, and deploy it on local/enterprise repository with -patched qualifier. But that would not fit for an opensourced project that is meant to be easily buildable by anyone who checks out its sources. Or should I just say "and also, before you build my project, please check out that stuff and run mvn install".
I'd agree with this and Pascal's answer. Some additional notes:
you may use dependency:unpack on the original artifact and then combine that with your compiled classes if you don't want to rebuild the whole dependant project
in either case, your pom.xml will need to correctly represent the dependencies of that library
you can still integrate this as part of your project's build to avoid the 'deploy to a repository' step
make sure you honour the constraints of the project's license when doing all this!