Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 1 year ago.
Improve this question
We are using Maven for a large build process (> 100 modules). We have been storing our external dependencies in source control, and using that to update a local repo.
However, we are ready to graduate to a local repo that can cache central so that we don't have to proactively download all 3rd parties (but we can still have a local repo to pull from). In addition we want to publish our internal build artifacts from a nightly build so that developers don't have to build the world.
We are considering Nexus and Artifactory. What are the reasons for preferring one over the other? Are there others we should be considering?
I'm sure that if you only talk about storing binaries from "mvn deploy" both will do fine.
We use Artifactory very extensively with all upgrades along the way. Lots of projects, numerous snapshots deployed and external repos proxied. Not a single problem. I find it hard to explain how other people experience issues with its DB, indexing or anything else. Nothing like that ever happened to us. Also, Artifactory allows to store data on a disk and only use a DB for storing metadata, it is quite flexible (see more here).
What makes those applications very different is their approach towards integration with other build tools and technologies. Nexus and Sonatype are pretty much locked on Maven and m2eclipse. They ignore anything else and only recently started to work on their own proprietary Hudson integration (see their Maven 3 webinar).
EDIT: This is not true anymore as of 2017 Nexus gives a much larger support for other build tools End of Edit
Artifactory provides an awesome Hudson, TeamCity and Bamboo integration, and Gradle / Ivy support. So while Nexus gives you nothing once you step out of Sonatype "comfort zone" (Maven, m2eclipse), Artifactory embraces and collaborates with all major build tools.
In fact, being able to deploy build artifacts from Hudson, when job has finished, and not by "mvn deploy" is a huge difference: Artifactory Hudson plugin makes an atomic-like deploy of all artifacts at once, only when a build job finished successfully. "mvn deploy" runs after each module and can deploy a partial set of artifacts if a build job fails in the middle. Deploying from Maven on module completion and not from a build server on job completion is really a bad thing to do.
As you see, Artifactory thinks "outside the box" while Nexus thinks "inside the box" and only cares about Maven and Maven artifacts.
Something else that makes Artifactory more accessible is their cloud-based Artifactory Online solution. For about $80 a month you have your own Artifactory instance, no need to dedicate any server for it.
Artifactory has a simple and straightforward REST API, don't know how it works for Nexus.
Edit Nexus has also a REST API that you can use easily as well.
To summarize, for basic storage of Maven artifacts I think both are fine. But while Nexus stops there being strictly a "Maven repository manager", Artifactory goes on and on, being a general "Binaries storage" for binaries of any kind, from any build tool and CI server.
I don't know about Artifactory but here are my reasons for using Nexus:
Dead simple install (and since 1.2, dead simple upgrade, too)
Very good web UI
Easy to maintain, almost no administrative overhead
Provides you with RSS feeds of recently installed, broken artifacts and errors
It can group several repositories so you can mirror several sources but need only one or two entries in your settings.xml
Deploying from Maven works out of the box (no need for WebDAV hacks, etc).
it's free
You can redirect access paths (i.e. some broken pom.xml requires "a.b.c" from "xxx"). Instead of patching the POM, you can fix the bug in Nexus and redirect the request to the place where the artifact really is.
Artifactory supports both file-system and database storage backends. Storage is checksum based and identical binaries are stored only once, no matter how many times they appear in the repo, which makes Artifactory more efficient storage-wise. Move and copy are also very cheap because of this architecture (in Nexus there's no REST for move/copy - you have to move stuff on the file system, then run corrective actions on the repo to let it know content has changed).
Another important differentiator is Artifactory has unique integration with Hudson and TeamCity for capturing information about deployed artifacts, resolved dependencies and environment data associated with build runs, which provides full build traceability.
Artifactory stores the artifacts in a database, which means that if something goes wrong, all your artifacts are gone. Nexus uses a flat file for your precious artifacts so you don't have to worry about them all getting lost.
If you need the "Pro" features of either (e.g. Staging repos, artifact promotion, NuGet), , then you need to consider the different pricing models, which are displayed on their websites.
http://www.jfrog.com/home/v_pricing
http://www.sonatype.com/nexus/purchase
In summary:
Artifactory Pro
you pay per server
you can pay more for increased service hours
Nexus Pro
you pay per seat, i.e. how many developers downloading artifacts
support service is Mon-Fri 0800-2000 ET only, no matter what you pay
No matter how many users you have, Nexus Pro offers a support service that's broadly equivalent to Artifactory's $7,450/year "Silver Value Pack".
$7,450/year will buy you approximately 67 Nexus Pro seats (1-50 # $108, the rest # $120).
On price and support alone then, Nexus Pro makes sense until you get to 67 users, at which point Artifactory becomes the cheaper option.
If you're doing all the support in-house; however, that magic point is about 23 users (Artifactory's most basic support offering is $2,750/year).
I made some research recenly about Artifactory 2 and Nexus 1.3. I'll list here the main differences I found:
Artifactory stores metadata and optionally files in DB, Nexus writes directly to file system. There are pros. and cons. for each approach. DB supports transactions, while in FS stored files can be accessed directly.
Artifactory has higher system requirements especially for disk space.
The most complete comparison: http://binary-repositories-comparison.github.io/
You should use Artifactory
Its latest version was a real jump
You can backup incrementally your repositories , which means you can have all your artifacts saved and maintain
Its has a easy to use web ui
and is really easy to set up
i enjoyed it a lot
check out its new version 2.0
From a learners point of view I note some specific differences between the two.
Sonatype .war deployment is not supported on Jboss application server at the time, although it does run under Tomcat.
Sonatype does not offer me an Amazon Machine Image (AMI), at present, that I could quickly stand up and test.
An Artifactory AMI is provided by Bitnami and takes a only a few minutes to stand up and a few more minutes to configure, maybe several tens of minutes dependant upon what you're trying to achieve.
Artifactory offer a SaaS version of Artifactory in the cloud so you can focus on getting things done rather than infrastructure.
I've no experience with Nexus but I've found Artifactory very intuitive and easy to configure, at least initially.
Added - I do note that the Artifactory User Guide, which may be OK for a seasoned pro, is a bit light on for some in depth explanations. For instance, starting out, one unzips and then addes a Repository, say RedHat's Jboss EAP Enterprise Repo. All goes fine but then when I tried to view the artifacts that were imported Artifactory reports zero artifacts? No errors or warnings so I'm now looking for an explanation. Is this normal or not normal? A simple explanation in the doco can quickly point one in the right direction. Being a good contributor I'm adding these comments to the project for the benefit of other starters.
All politics/religion aside, licensing makes a difference for some organizations.
Nexus is GPL now AGPLv3 and now Eclipse Public License (EPL).
Artifactory is Apache licensed LGPLv3 licensed as of version 2.1 of the product.
You may also want to consider Archiva, just for comparison's sake. It's Apache 2.0 licensed.
I see that Nexus usage is growing, while Artifcatory usage is generaly staying flat.
Picture is taken from here http://blog.sonatype.com/2014/11/42000-nexus-repository-managers-and-growing/
There is also matrix-comparison http://docs.codehaus.org/display/MAVENUSER/Maven+Repository+Manager+Feature+Matrix
Both Artifactory and Nexus have more or less similar feature set but Artifactory's LDAP support makes it more attractive over Nexus. Though Nexus also have LDAP support but in paid version :-(
Hmmm...my experience with artifactory is awful...but I'm a relative newbie so take it with a grain of salt. My overall complaint is that jar files recently uploaded to Artifactory do not seem to get indexed right away - as in for hours - and there does not seem to be a good way to force it. I've tried various things that appeared as if they should have worked, but didn't. I have been working with m2eclipse, adding dependencies to a project that i'm converting from ant. When I try to add a jar that I have just added to artifactory, I expect it to show up as a choice in the selector but it does not.
a coworker told me that they had installed nexus and so far they like it...but I can't vouch for it yet. I'm about to install that on a Linux box as soon as IT can find me one.
Related
In our team we have a number of APIs specified using the Open API Specification (formerly Swagger). We use Maven and OpenAPI Generator to generate code, build and publish the artifact to our local nexus. We build our code on TeamCity. The artifact is given the version that is specified in the pom.xml file of Maven.
During development we only use snapshot versions, that is versions which can be overwritten and will be cleaned up. This is opposite to release versions, that cannot be overwritten and needs administrative privileges to clean up. The reason for this is, that a developer usually changes a little bit at the time, which is much more convenient with snapshot versions. This also makes cleaning up outdated unreleased artifacts much easier.
Our problem is, that from time to time a developer makes API changes but forgets to set a new version. This works fine locally, but when the code is build on TeamCity the changed API overwrites the artifact of an older version. A developer not working on this branch will then experience a compile error, because the code does not match the API artifact being used.
What does others do? Is there a best practice? Preferably with standard tools. We have tried many things and nothing works well. At the same time this issue is so basic that someone must have a good solution - or at least experience enough to point to the least bad solution.
I'm looking for a "repository" to store derived information (build artifacts).
We have a repository (currently Mercurial) to store our source code. When something is pushed to the source repository the code goes through a continuous integration server and we do an incremental build and as a result some dlls will be changed. This should be added to some "repository" so that everybody can use that version without needing to do the build again.
I'm looking for the following features:
It should be easy to update the source code and get the corresponding binaries (we could probably make a script for that)
You should easily get all binaries at once (not only those that changed during the last incremental build.
Binaries that weren't changed should only be stored once in the repository.
When updating the source code and the binaries only the changed binaries should be transferred (and not all binaries). This is similar to what happens for source code.
When updating to some version, only that version should be stored locally, not the complete history.
We should be able to remove certain versions from the binary "repository" after a while. However if the dlls are still necessary for subsequent incremental builds, these dlls should of course not be completely removed from the "repository"
What would fit these requirements?
I agree with Manfred, what you are looking for is a binary repository manager. Besides the Nexus repository manager you should consider Artifactory.
As for the feature list you asked about:
As you have mentioned the CI server should be responsible for identifying a change in the version control and starting a build process which creates the binaries. The CI server/build tool should also deploy the generated binaries to the repository manager, in case the build was successful. Artifactory offers a build integration feature which takes care of deploying the binaries together with the build metadata.
Using the build integration feature of Artifactory, you can get a list of all the binaries generated by a specific build and download them as an archive. Artifactory provides a REST API for those actions.
There are different approaches for storing the artifacts in a repository manager. Some tools stores a multiple copies of the same binary. Other, for example Artifactory, use a checksum based storage which keeps only one copy per binary (based on its checksum). This pays of if you keep multiple copies of the same binary in different repositories, especially if you are dealing with large binaries (war files, docker images, ISOs etc.). Another benefit are cheap copies/moves between repositories which is a common practice for promotion workflows.
The Artifactory build integration uses checksum based deployment which deploys only binaries which does not exist in Artifactory. For binaries which do exist and have not changed, it only created a new reference to the existing binary saving the need to send the actual bytes.
Artifactory provides multiple option of cleaning up binaries, including built in cleanup policies and the option to develop your own custom logic using user plugins and the Artifactory query language (AQL)
In addition, I highly recommend to take a look at the binary repository comparison matrix.
Disclaimer: I am working for JFrog the company behind Artifactory
You are basically asking for a repository manager like the Nexus Repository Manager as you have correctly identified with the tags.
In terms of specific requirement from your questions here are a couple of ideas.
binary components are typically identified via some coordinates that most of the time includes some sort of name and version. A release and build process changes those and deploys them to the repository. This allows you to match source code with binaries. You can also embed information like git refs in the produced binaries.
accessing the binaries is typically done via HTTP, so its easy. You then just have to determine what it means to get "all binaries".
not duplicating binaries that are essentially the same can be supported by the underlying file system or the build tool. I have seen both processes to work. Often it is however not worth the effort since storage is cheap.
there are various ways to automatically clean up repositories including scheduled tasks that do it regularly. Worst case you have to implement your own logic in an extension
Disclaimer: I work as community advocate and trainer for the Nexus Repository Manager with Sonatype.
Recently I started to incorporate good practices in my development workflow, so I split the development server and the production one. I also incorporated a versioning system using Subversion (Tortoise SVN).
Now I have the problem of synchronize the production server (Apache shared hosting) with the files of the last development version in my local machine.
Before I didn't have this problem because I worked directly with the server files through Filezilla. But now I don't know how to transfer the files in an efficient way and what are the good practices in this aspect.
I read something about Ant and Phing but I'm not sure if this appropiate to me or is unnecessary complexity.
Rsync is a cross-platform tool designed to help in situations like this; I've used it for similar purposes on multiple occasions. This DevShed tutorial may be of some help.
I don't think you want to "authomatize" it, rather establish control over your deployment and integration process. I generally like SVN but it has some bugs and one problem I have with it is that it doesn't support baselining -- instead you need to make a physical branch of your repository if you want to have a stable version to promote to higher environments while continuing to advance the trunk.
Anyway, you should look at continuous integration and Jenkins. This is a rather wide topic to which not a specific answer can be given. There are many ins, outs, what-have-yous. Depends on your application platform, components, do you have database changes, are you dealing with external web services or 3rd party APIs etc.
Maybe out there are more structured solutions but with Tortoise SVN you can export only the files changed between versions in a folder tree structure. And then, upload as always in Filezilla.
Take a look to:
http://verysimple.com/2007/09/06/using-tortoisesvn-to-export-only-newmodified-files/
Using TortoiseSVN, right-click on your working folder and select
“Show Log” from the TortoiseSVN menu.
Click the revision that was last published
Ctrl+Click the HEAD revision (or whatever revision you want to
release) so that both the old and the new revisions are
highlighted.
Right-click on either of the highlighted revisions and select
“Compare revisions.” This will open a dialog window that lists all
new/modified files.
Select all files from this list (Ctrl+a) then right-click on the
highlighted files and select “Export selection to…”
Side note:
You have to open more details about your workflow and configuration - applicable solutions depends from it. I see 4 main nodes in game: Workplace, Repo Server, DEV, PROD, some nodes may be united (1+2, 2+3), may have different set of tools (do you have SSH, Rsync, NFS, Subversion clients on DEV|PROD). All details matter
In any case - Subversion repositories have such thing, as hooks, in your case post-commit hook (executed on Repository Server side after each commit) may be used
If this hook (any code, which can be executed in unattended mode) you can define and implement any rules for performing deploy to any target under any conditions. You must only know
Which transport will be used for transferring files
What is your webspaces on servers (Working Copies of just clean unversioned files - both solution have pro and contra sets) - it will define, which deployment-policy ("export" or "update") you have to implement in hook
Some links to scripts, which export files, affected by revision (or range of revisions) into unversioned tree
We have a largish standalone (i.e. not Java EE) commercial Java project (10,000+ classes, four or five SVN repositories, ten or twenty third-party libraries) that's in the process of switching over to Maven. Unfortunately only one engineer (in a team of a dozen or so distributed across three countries) has any prior Maven experience, so we're kind of figuring it out as we go.
In the old Ant way of doing things, we'd:
check out source code from three or four repositories
compile it all into a single monolithic JAR
release that (as part of a ZIP file with library JARs, an installer, various config files, etc.)
check the JAR into SVN so we had a record of what the customers had actually got.
Now, we've got a Maven repository full of artifacts, and a build process that depends on Maven having access to that repository. So if we need to replicate what we actually shipped to a customer, we need to do a build against a Maven repository that has all the proper versions of everything. This is doable, I guess, if in (some version of) the (SVN-controlled) POM files we set all the dependencies to released versions?
But it gives our release engineer the creepy-crawlies, because there doesn't seem to be any way:
to make sure that somebody doesn't clobber the copy of foo-api-1.2.3.jar on the WebDAV server by mistake (the WebDAV server has access control, but that wouldn't stop a buggy build script)
to detect it if they did
to recover afterwards
His idea is, for release builds, to use a local file system as the repository rather than the WebDAV server, and put that local repository under SVN control.
Our one Maven-experienced engineer doesn't like that -- I guess because he doesn't like putting binaries under version control? -- and suggests that maybe the professional version of the Nexus server can solve the clobbering or clobber-tracking/recovery problem.
Personally, I'm not happy (sorry, Sonatype readers) with shelling out money for a non-free build system when we haven't even seen any benefit from the free version yet, and there's no guarantee it will actually solve the problem.
So our choices seem to be:
WebDAV server
Pros: only one server, also accessible by devs, ...?
Cons: easy clobbering, no clobber-tracking/recovery
Local file system
Pros: can be placed under revision control
Cons: only works with the distribution script
Frankly, both of these seem like hacks to me, and I have to wonder if there isn't a better way to do this.
So: Is there a right thing to do here?
I'm not sure to get everything but I would:
Use the maven-release-plugin (which automates the release process i.e. execute all the steps documented in release:prepare).
Use WebDAV with anonymous read-only and authenticated write policy (so only release engineer can actually deploy released artifacts to the corporate repo).
There is a no need to put generated artifacts under version control (if you have the poms under version control). I don't see the benefits of using the local file system instead of WebDAV (this is not providing more security, you can secure WebDAV as well). I don't see what the commercial version of Nexus would solve here.
Nexus has a setting which prevents you from clobbering an already released artefact in a release repository.
For a team of about a dozen, the free version of Nexus should be enough.
I'm wondering how Software Development Team distribute their Standard IDE(s)?
E.g. developing with Eclipse, custom Code formatter, svn Resository, Copyright Header..
At the moment my Team has a standard zip File which is then distributed withhin the developers.
Problem:
If one file, a Plugin or the IDE itself changes, e.g. new Coding Guidlines, Upgrade Eclipse 3.5.1 the whole distribution has to be done again. Every developer needs to unzip the bundel again. Imagine your working with different Workspaces (Jetty, different Tomcamt Versions, WTP) due to Project History That doesn't scale
I know that there are some related Articels
A new version of Eclipse just came out. Is there anything I can do to avoid having to manually hunt down my plugins again?
Manage Your Eclipse Install With A Local Git Repository
And some comercial Programs.
Eclipse also has a new Update-Installer Approach
But I don't see the Killer App. How do your team solve this? Is there a best practice?
I guess best would be a Program letting you choose your current Project and then downloads the configured IDE from the Server and leting you know if Project Config Files are Updated
For eclipse look at Buckminster it targets exactly your target I suppose, didn't use it personally through.
At my previous company they wrote a custom update agent that pulled from a centrally configured server which was updated by the team leaders. It worked well, until people wanted to install their own plugins.
Basically, a developer wanted a plugin, fought in futility to get it included in the default (managed) repo, installed it himself, then updates broke on his machine when the team lead had a sudden stroke of common sense and included it.
They never did come up with a 'good' way to manage it. But, at least they didn't put us all on terminal servers with thin clients.