Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 6 years ago.
Improve this question
ZooKeeper is a highly available coordination service for data centers. It originated in the Hadoop project. One can implement locking, fail over, leader election, group membership and other coordination issues on top of it.
Are there any alternatives to ZooKeeper? (free software of course)
I've looked extensively at Zookeeper/Curator, Eureka, etcd, and consul. Zookeeper/Curator and Eureka are in many ways the most polished and easiest to integrate if you are in the Java world. Etcd is pretty cool and very flexible, but It is really just a HA key store so you would have to write a lot of code to turn it into an opinionated service discovery system.
Consul is (to me) the best of both worlds. It is an opinionated service discovery system written on top of serf, using raft for cluster consensus and gossip for communication. It exposes the discovery / registration endpoints with a well documented REST api, and also allows you to discover services with DNS SRV records, and register services with configuration (i.e. so you can register a database or application you can't integrate a client with, or if you just want to keep your service discovery decoupled from your app)
I've written a blog post about consul where you can learn more and walk through my "try it out" demo
I've also discussed service discovery with etcd & docker if you want to see more about what that custom code might look like.
One last thing! etcd & consul are written in go, so maintaining them is much easier then java solutions like zookeeper. All you need is the consul / etcd binary. no dependencies, no linked libraries, no jvm.
There's a very promising alternative to ZooKeeper called etcd (github.com/coreos/etcd), written by CoreOS team. Unlike Doozerd, etcd is being actively developed.
Just discovered Accord (C) and OpenReplica/ConCoord (Python) which may be interesting solutions
[EDIT] The Hashicorp crew, of Vagrant and Packer fame, are cooking "a decentralized solution for service discovery and orchestration" called Serf.
[EDIT2] Hashicorp strikes again ! They just released Consul, built on top of Serf. The pitch: "a solution for service discovery and configuration, completely distributed, highly available, scalable to thousands of nodes and services across multiple datacenters".
Yes, there is also Doozerd (https://github.com/ha/doozerd). Take a good look at it, it's a nice, single binary distributed coordination service developed by Heroku. With bindings/libraries for java/python/ruby/node. Very easy to get started with and play around.
Take a look at Serf. There is a comparison vs Zookeeper here.
OpenReplica from my research group is a highly available FOSS coordination service for data centers. It can be used for implementing locking, fail over, leader election, group membership and other coordination services. It differs from ZooKeeper in two critical ways:
It uses an object-oriented API. This makes it much easier to write coordination services. Synchronization code for OpenReplica looks exactly like its textbook counterpart; there is no need to master a file and upcall-based API like in ZooKeeper and Chubby.
It enables dynamic membership updates to the replica set. There is no need for static configuration files. The system is integrated into DNS (authoritative, slave for OpenReplica, or Amazon Route 53).
We actively support the system, do not hesitate to let us know if you have further questions.
There's a project called Noah on github that looks interesting, it says that it's "loosely based on Apache ZooKeeper" https://github.com/lusis/Noah with REST support being a key feature (ZK has this as a contrib/option rather than built in).
There are different tools that optimize for different engineering trade-offs.
ZooKeeper Scales marginally for reads; writes with many observers can be slow. It is proven and has a sizable community.
Accord Seems interesting for write-intensive uses, however typical use-cases already have domain-specific solutions (i.e., logging, telemetry).
The others are somewhat interesting but generally unproven. Don't get this wrong if intended for production usage.
I'd found this comparison of Zookeeper, etcd and Doozer:
http://devo.ps/blog/zookeeper-vs-doozer-vs-etcd/
Serf (serfdom.io) is also a nice solution as it is simple! But you must consider, SERF is just a cluster-manager which enables you to send custom events to all cluster nodes. Thats nice, but you have to write your own shell scripts (aka events).
See this example: "https://www.digitalocean.com/community/articles/how-to-set-up-a-serf-cluster-on-several-ubuntu-vps"
The advantage is, you're getting a very simple cluster-manager and you're able to combine this with your favorite configuration, deployment or continuous integration tool.
It seems Corosync is also like ZooKeeper.
I know this post is quite old, but someone who is looking at all possible alternatives I would also like to suggest JGroups library which is mature enough to be used in production environment. I have used it successfully in one of my projects mainly for distributed coordination and to share messages between cluster. It also support AWS support in addition to its flexible architecture where you can customize its stack to get what you need. I suggest you to have a look at it
Related
I'm managing a rather widely distributed software application in a semi industrial environment. The software at its heart is based on SOA and employs OPC-UA to make communications between important processes (on local or LAN-based machines) possible. These processes are either a server (e.g. an outer network management server, hardware managers server etc.) or a client (customer panel) or both (servers talking to each other).
OPC-UA has the following problems:
Configuring and maintaining the configurations is a hard job (just Config file settings takes lots of time)
Security measures are too much detailed for my needs (certificate management and sudden invalidation of certifications on customer systems)
Modeling and networking overheads in the library make it hard to work with in my communications (high data rates usually ends in server and client disconnecting)
Unspecified and weird errors like UA Discovery Server stopping to work or respond, etc. which I have reported to OPC GitHub forum many times.
Troubleshooting in internal parts of OPC UA is nearly impossible.
Overall, for me its performance and stability are not reliable enough. I am willing to sacrifice features for better performance and reliability. I've even considered to write sockets from bottom-up for my inter-process (IPC) needs. This way I could at least trace errors to their core. Since I do not need its most advertised feature (aka PLC support) I'm desperate to find a good alternative for it. My main requirements are:
OPC-UA like Data Modeling support that enables me to provide a clean interface to customers and other teams (something like IDL).
Publish/Subscribe, Remote Commands, Update Notifications and Node Based Behavior.
Tough Security is not my concern as my network is closed.
High performance for data rates up to 1Gbps (this could mean UDP support).
I am entirely working in .NET framework. So C# support of OPC-UA is a great help for me.
I've looked at DDS (lacks commands and Update Notifications) and WCF (lacks cross platform support) and many more.
This link also notes about MQTT: Alternative to OPC-UA
What about Google's gRPC + protobufs?
https://grpc.io/
Is it practical or wise to use Lagom in production without ConductR? The commercial licensing is putting me off. This framework looks like it could be pretty arduous to deploy and custom tooling for that can take a lot of effort to get right.
(disclaimer: I'm a Lightbend employee, currently core member of the Lagom team)
Edit (Nov 2018): Please refer to https://www.lagomframework.com/documentation/current/java/ProductionOverview.html#Running-Lagom-in-production for up to date information on this topic.
(original answer, Aug 2017)
A lot has changed in the Lightbend stack since this question was added over a year ago. For example: ConductR is now free to use in production for up to three nodes.
Also, the team behind ConductR is also working on providing tools to deploy a Lagom application on Kubernetes. The efforts on that front are very advanced and some of our sample apps can be deployed in Kubernetes already.
Your question is rather open-ended and so let's start with "it depends".
If
you're comfortable managing your scaling within the configuration of your Akka cluster,
your usage doesn't violate the open-source licensing terms of lagom, play and Akka, AND
you don't have sufficient cash flow to justify leveraging Lightbend's production suite,
then you arguably can deploy with a minimum of effort and custom tooling.
If those conditions don't hold, your options are to go elsewhere (e.g., Spring Cloud) or retain Lightbend. You may find going elsewhere has it's own cognitive load and/or commercial expense.
Hope that helps even 7 months later.
The other answers are higher level, but I can essentially say "yes." I'm currently deploying a Lagom service to be hosted on a Kubernetes cluster, and I'm not using ConductR or any of the commerically licensed components.
You will need to dig a little bit into some Play internals to start the service properly, and if you want to hook in with some other service locator you may need to implement one yourself, but it certainly isn't impossible and I think it's less effort on net.
Being honest it really worth to pay, because you got much more, good reporting dashboards, automatic cluster formation and what really cool is split brain resolver....
But sometimes agree when you are working on a project that don't have a lot of money, you can do some small tricks and get it work and may be then do all what really lucks or buy enterprise subscription.
So Lagom can be very easy being used without service discovery at all if you will delegate all to Kubernetes DNS or without Kubernetes just put load balancer before each service and use its address.
How this can works
Each lagom service that you have should be external service
Production run configuration should be mixed with
ConfigurationServiceLocatorComponents
Service that should communicate with another one should be started with extra parameter that tell lagom that this service is external one and can be communicated directly(all can be in JAVA_OPTS as for play application)
-Dlagom.services.your_service_name=http://k8s_service_name.default.svc.cluster.local:9000
Apologies if what I am going to ask is too generic and please feel free to mark it as community wiki or even close it but after getting a good answer please.. Here goes nothing.
I had a heated debate over the term application infrastructure with a consultant from a BIG company and hence I am here to see what the experts think as I myself need a good understanding as well. I am a puny software developer and was trying to have a chat about software architecture and stuff and then we kinda got stuck up at the term application infrastructure.
As per my rival's understanding, this is solely used re the back-end hardware components and whereas I thought he is mixing the contexts and from the context of software eng it should refer to stuff that supports the application such as logging, ORM, or even framework (.net) etc.
To what extent am I wrong and if I am please shed some light on the definition...it's really gonna help a lot.
Thanks
Here are a few links with a little description about the term Application Infrastructure extracted from them:
Application Infrastructure -- f5.com
Application infrastructure, comprised of application servers, web servers, and often database servers, is a core component in most network architectures. This part of the network infrastructure delivers high performance application services to the LAN as well as to employees, partners and customers on the WAN.
Some of the key functionality of application infrastructure includes transaction management, clustering, reliable application-to-application messaging, system management, advanced application development tools, proprietary access, and interoperability with legacy technologies.
Application Infrastructure -- networkmagazineindia.com
A (common/standard) platform is required to make different applications in an enterprise work across geographies or multiple locations, and to manage a large number of users and transactions taking place within an enterprise. This platform which ensures that different applications work with each other, is known as an application infrastructure.
Application Infrastructure -- bitpipe.com
A high performance, reliable, and secure integrated technology infrastructure for managing multiple hosted applications by Application Service Providers.
What he seem to be describing is the platform for software (I'll leave it open if that is the same thing as the software platform. I do include software service in the Application Infrastructure definition and also include connectivity/networking along with hardware and software as the three foundation corner stones of Infrastructure. Infrastructure is the general purpose commodity components as opposed to the domain specific components which will typically always be Software; (A possible exception would be an organisation uses custom hardware). AIH; IBM pitch WebSphere as Appliction Infrastructure and Microsoft Consider their Windows Server to the Application Infrasture both agree with you and I.
From what I can see it speaks to HW and SW (middleware and API management).
Gertner definition : Application infrastructure is software platforms for the delivery of business applications, including development and runtime enablers.
Interesting info from Gartner:
http://wso2.com/resources/analyst-reports/comparing-vendors-of-comprehensive-application-infrastructure-suites/
I am starting to plan a new platform which needs to integrate various services from various externals platforms. Essentially I'm tying together a bunch of internal, homegrown services and several outside services we license from 3rd parties.
Generally speaking the external services are all web services but they are a mishmash of REST, SOAP and XML-RPC.
Some of our internal services have REST API's but there are many things that aren't so easy: XMPP, Hessian, custom socket protocols, Java RPC, uWSGI, and the list goes on.
From my research it seems like an ESB like Apache ServiceMix might be a good fit for my needs. However it looks REALLY complex. I'm not launching rockets but I do need transactional messaging (mostly for eCommerce and entitlement stuff). I feel like the message queue ServiceMix uses under the hood (ActiveMQ) might be enough on its own.
Can anyone explain what ServiceMix provides above and beyond ActiveMQ? I know there is a lot but it is hard for an ESB n00b like me to really grasp the tangible difference when I'm waste-deep in buzzwords.
Thanks!
ServiceMix is an OSGi based container that allows you to deploy and run applications in a controlled runtime environment (like a J2EE container but less heavy weight and without programming to e.g. J2EE contracts).
Thanks to OSGi you can partition your applications into parts and update/evolve these parts independently from each other. You can upgrade parts of your application without having to take down the entire application. There is far better life cycle management in OSGi then you get with standalone Java processes.
If you think of creating an application that will evolve over time, then OSGi is something you should consider. And ServiceMix provides you a runtime OSGi container to deploy your applications to. I highly recommend the book "OSGi in Action" from Manning.
For tying together different external services that might even use different transport protocols I recommend Apache Camel, which btw also deploys nicely into ServiceMix.
Btw, existing applications can be deployed into an OSGi container with fairly little effort (without requiring code changes).
Torsten Mielke
FuseSource
Web: www.fusesource.com
Blog: http://tmielke.blogspot.com
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 2 years ago.
Improve this question
are you aware of the best companies which provide apache solr AS A SERVICE? where i can simply upload (or edit via some web control panel) my index and config files for SOLR and simply start using it
i do not want to be breaking my head with any sort of server administration on tomcat
just update my index and config files... tell solr where to look for data to index (via data import handlers) and thats it just start using it
any sort of load balancing / mirrors would be like icing on the cake
price does not matter as its for mission critical apps
please do not suggest me to boot up my own servers on amazon or rackspace or xyz and then deploy solr on them and manage all the administration - because thats what i want to avoid in the first place completely
thanks in advance
I wanted to update this post from 2011 now that it's 2017! Today for folks looking for pure self service and simple Solr search we continue to recommend www.websolr.com. If you are looking for a managed complete Solr instances + some nice search analytics capability, then we've used SearchStax very successfully.
there is also this one:
http://www.opensolr.com/
Opensolr proposes 3 types of Solr instances :
CMS instances (WordPress, Drupal, Joomla, eZ Publish, Typo3)
API instance (RESTful Web service) with fully customizable configuration files
Web sites Crawler instances
There is a free account (i think it can be used actually only for test the platform), and the price for the other kind of account is cheap (4$/8$ a month for medium accounts)
(i didn't try it yet, but it seems promising)
If you use Drupal, there is also Midwestern Mac's Hosted Apache Solr service, which works with Drupal 6 and 7, and all the different Solr integration modules. (Disclaimer: I'm the owner of Midwestern Mac—let me know what you'd like to see and I'll try to make it happen!).
The company IndexDepot (www.indexdepot.com) offers a hosted Solr service. It's easy to use, because you log into an web interface to edit your configuration files. Special configurations fitting your requirements are negotiable, e. g. dedicated master/slave Solr servers.
You can try contacting Lucidworks with that question.
I heard they were working with Boomi on PaaS/Saas for their Lucidworks 1.4.
Although Boomi don't explicitly say that they support Solr, this webinar may suggest they were working with Lucidworks to include their Solr-based search engine in their portfolio.
Even if Lucidworks people don't provide SaaS, they're surely the right address to ask who does.
Good luck in your search and please get back to us with the information you manage to find...
EDIT 04/2012:
If I had to make that choice today, I'd seriously consider CloudBees (which has WebSolr plugin). It's a complete ALM & CI cloud framework for JVM-based languages, with loads of partner plugins (Jenkins, NewRelic, Sonar, MongoHQ, Cloudant, ...), many of them with free base options.
The most significant difference, when comparing with other SaaS/PaaS services, is that you can set up development environment and even deploy your app (on one node, of course) without even leaving your credit card details.
Just to expand horizons.
please do not suggest me to boot up my own servers on amazon or rackspace or xyz and then deploy solr
You can go with ready to use:
http://aws.amazon.com/cloudsearch.
Cloudsearch provides simple API.
And more and more hosted Elastic Search solutions appears recently. I think it's because of cloud ES nature (easy to maintain search cloud).
http://indexisto.com
http://qbox.io
But of course this is the matter of how sticky are you with SOLR.
These guys also so hosted Solr.
https://www.hosted-solr.com/?locale=en
They also have a custom extension that integrates Solr into TYPO3 CMS.
http://www.typo3-solr.com/en/home/
Take a look at our Fully Managed Solr Cloud Hosting, where we enable customers like you to not worry about managing and maintaining Solr infrastructure, but instead focus on building your application.
We offer shared clusters that caters to price conscious customers and dedicated nodes and dedicated clusters with white glove service for companies who want to completely offload search infrastructure and management.
We are a technology partner with AWS and depending upon the critical needs, can customize a hosted solr solution per our customers needs.
please do not suggest me to boot up my own servers on amazon or rackspace or xyz
and then deploy solr
You can try the below solr service provider, looks cheaper as well
http://indiasolr.com/