I'm looking into Rhino Service Bus for a project instead of using commercial NServiceBus (which I know from my workplace).
In NServiceBus there's the concept of a distributor, which load balances tasks between server automatically to allow scale up. I searched and found a similar question about RSB, but with no example on how to actually implement it. RSB is a bit short in documentation (they have example projects on GitHub, but non with such example).
Does anyone know how to implement load balancing in RSB? or have any link to an article about it?
Thanks
Related
Advanced Attacks Detection in a Platform-as-a-Service(PaaS) Environment
In the first part of this project, i'm supposed to monitor incoming packets
in a web service, accept only HTTP & HTTPS (TCP)packets for later analysis and, drop the rest.
I was thinking doing this in JAVA, because i think it's a very flexible and
complete language and, it's present in every PaaS Environment! So, my idea is
to build a simple web page in JSP/JSF with a bean to attend this first step
of the project.
This is where i need some guidance! Because i've started considering
libpcap JAVA wrappers like jNetPcap, Jpcap and Pcap4J. But none of them is able to drop packets!
Forgetting JAVA, i also have red about other libraries like: libnet, libdnet and libcrafter.
libnet can not handle the task!
libdnet has network firewall rule manipulation capabilities, but it's a very old library and, i'm not sure it can handle integration with iptables!
libcrafter is the best! Because it's an actual updated project and, it allows the use of iptables rules in the code.
And, of course, working directly with netfilter would be the ideal scenario!
But working with libcrafter or netfilter, to follow my simple idea of a web service with a JAVA bean, i would have to write my own java wrapper by JNI! Which i assume NOT to be a simple task!
Now, what is raising many doubts in my mind, is the fact that this has to be
done in a PaaS environment! None of them (PaaS providers) seem to have the
same restrictions. There are some more flexible like AWS and Microsoft Azure that let you choose and manage a VM with the OS distro you want. Others like OpenShift, BlueMix or Cloud Foundry, in a project, only give you the option of defining the programming language, application server and, that's it! So, one might not have permissions to install libraries and control network & transport layers to manage the packets! Since the hole OS administration is handled by the provider.
Considering only the main purpose of this project, which is managing the packet flow pointed to a domain located in a PaaS environment, without the help of other servers like tcp proxies, i am desperately in need of someone pointing me a direction to start from! Because with that, i can dig as deep as needed to get a solution. Please HELP!
Thank you very much for your time and consideration.
Is it practical or wise to use Lagom in production without ConductR? The commercial licensing is putting me off. This framework looks like it could be pretty arduous to deploy and custom tooling for that can take a lot of effort to get right.
(disclaimer: I'm a Lightbend employee, currently core member of the Lagom team)
Edit (Nov 2018): Please refer to https://www.lagomframework.com/documentation/current/java/ProductionOverview.html#Running-Lagom-in-production for up to date information on this topic.
(original answer, Aug 2017)
A lot has changed in the Lightbend stack since this question was added over a year ago. For example: ConductR is now free to use in production for up to three nodes.
Also, the team behind ConductR is also working on providing tools to deploy a Lagom application on Kubernetes. The efforts on that front are very advanced and some of our sample apps can be deployed in Kubernetes already.
Your question is rather open-ended and so let's start with "it depends".
If
you're comfortable managing your scaling within the configuration of your Akka cluster,
your usage doesn't violate the open-source licensing terms of lagom, play and Akka, AND
you don't have sufficient cash flow to justify leveraging Lightbend's production suite,
then you arguably can deploy with a minimum of effort and custom tooling.
If those conditions don't hold, your options are to go elsewhere (e.g., Spring Cloud) or retain Lightbend. You may find going elsewhere has it's own cognitive load and/or commercial expense.
Hope that helps even 7 months later.
The other answers are higher level, but I can essentially say "yes." I'm currently deploying a Lagom service to be hosted on a Kubernetes cluster, and I'm not using ConductR or any of the commerically licensed components.
You will need to dig a little bit into some Play internals to start the service properly, and if you want to hook in with some other service locator you may need to implement one yourself, but it certainly isn't impossible and I think it's less effort on net.
Being honest it really worth to pay, because you got much more, good reporting dashboards, automatic cluster formation and what really cool is split brain resolver....
But sometimes agree when you are working on a project that don't have a lot of money, you can do some small tricks and get it work and may be then do all what really lucks or buy enterprise subscription.
So Lagom can be very easy being used without service discovery at all if you will delegate all to Kubernetes DNS or without Kubernetes just put load balancer before each service and use its address.
How this can works
Each lagom service that you have should be external service
Production run configuration should be mixed with
ConfigurationServiceLocatorComponents
Service that should communicate with another one should be started with extra parameter that tell lagom that this service is external one and can be communicated directly(all can be in JAVA_OPTS as for play application)
-Dlagom.services.your_service_name=http://k8s_service_name.default.svc.cluster.local:9000
My only experience with SOA comes from working with NServiceBus. It's really well designed and has some nifty features that I really like:
Processing of messages is transactional (and taken care of for you), so it's "once and only once" (as opposed to "at least once")
Built-in saga support
Automatic retry support
Pub/sub support (but I assume all service/message busses support this)
All that stuff leaves me to concentrate exclusively on the app's logic, but I've only used it in .NET, and Windows is almost certainly a non-starter for the project I'm working on. So my question is...
What service bus technologies out there exist in the *nix ecosystem (including experiences running NSB on Mono) and how do they measure up to NServiceBus running on Windows?
Bonus: In addition to the features above, what languages can send/receive/process messages from the bus? Having the flexibility to write one service in language A and another in language B would be very advantageous for this project.
SonicESB / MQ
I can speak from experience with SonicESB on Linux. It's basically SonicMQ - a JMS compliant MOM - with a process mediation layer on top; that's the service bus.
Pros
Itinerary-based routing; messages contain their own workflow for what services to visit
Intra-container messaging; increases performance by routing messages within the ESB container, avoiding the broker entirely.
Web service endpoints; ESB service may be invoked over HTTP endpoints, bridging the gap between non-Java/JMS clients and the bus.
Good development tools in eclipse for developing and debugging esb services and processes in Java.
Administration tools aren't bad either.
Cons
Transactions don't span service boundaries (may have changed since version 7.5)
No message retry mechanism; we rolled our own using a utility service and message parameter manipulation (may have changed in 7.5)
Pricey; licensed by core, so expect to pay for it.
Client Support
In terms of what technologies can send and receive to/from SonicESB, anything Java JMS, that's for sure. Sonic also offers a library in .net/C#/VB for Windows technology, so you can send and receive messages from the bus. This was a huge plus for us since our development teams use RIA's built in C#.
You may want to check out MuleESB, an open source service bus.
Hope it helps,
It has been quite a while since I've seriously used NServiceBus (in fact, back then, it wasn't even called that - it was just replacement library Udi wrote, to an awful comm library we had to use on a joint project :) ) so I don't know about all NServiceBus features.
I would say, based on what I do know, that Apache Kafka (originally developed by LinkedIn) would be a good starting point. It has relatively strong ordering guarantees, supports publish/subscribe to pools of consumers and is supportive of workflow behaviours e.g. Samza which provides a streaming framework on top of it
Also regarding the client interfaces. You can find the current list here
I am looking for a tool capable of generating multiple Xmpp connections to load-test a XMPP server with a secure connection, especially starttls.
For a xmpp plain text authentication I had used jab_simul(followed this tutorial) and tsung both with success.
But I was unable to use the tolls above for the starttls,I peeked into the code of both tools and tried different configurations of the tools.
Another option I am pondering is using a xmpp library like eXmpp and make a specific load-testing tool myself with, instead of altering jab_simul (C software with comments in language i do not understand) or altering tsung(all purpose load-testing tool, so lots of place where you can go wrong).
short-story - I am looking for a tool or advice to stress-testing/load-testing a xmpp server.
We are facing exactly the same challenge right now. After deep consideration we found out that only especially build software can deliver the load we want to test. (Remember, you can configure ejabberd to something very specific :-)
For that we developed a small library called xmpp_talker https://github.com/burinov/xmpp_talker (Apache Licence) which is a kind of xmpp client made as a gen_server. I find it is a very nice starting point to build any kind of load simulation software. There is also echo_worker example included. So, you have good base to start. At the moment xmpp_talker is suited for exmpp 0.9.7. As far as I know in a few days will be out version 1.0.0. (or 0.9.9?) There are many bug fixes (trust me you don't want to know about them). On monday I will release xmpp_talker for exmpp 0.9.8 with proper service interruption handling.
In case you deside to go the same way xmpp_talker could be useful for you.
Added: Here is also great article that is realted to the topic: https://support.process-one.net/doc/display/EXMPP/Scalable+XMPP+bots+with+erlang+and+exmpp
There's also the recently started XMPP benchmarking project called xmppench which aims to be a high-performance benchmarking tool simulating some reasonable use cases of XMPP servers. It's written in C++, based on Swiften and boost.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 6 years ago.
Improve this question
ZooKeeper is a highly available coordination service for data centers. It originated in the Hadoop project. One can implement locking, fail over, leader election, group membership and other coordination issues on top of it.
Are there any alternatives to ZooKeeper? (free software of course)
I've looked extensively at Zookeeper/Curator, Eureka, etcd, and consul. Zookeeper/Curator and Eureka are in many ways the most polished and easiest to integrate if you are in the Java world. Etcd is pretty cool and very flexible, but It is really just a HA key store so you would have to write a lot of code to turn it into an opinionated service discovery system.
Consul is (to me) the best of both worlds. It is an opinionated service discovery system written on top of serf, using raft for cluster consensus and gossip for communication. It exposes the discovery / registration endpoints with a well documented REST api, and also allows you to discover services with DNS SRV records, and register services with configuration (i.e. so you can register a database or application you can't integrate a client with, or if you just want to keep your service discovery decoupled from your app)
I've written a blog post about consul where you can learn more and walk through my "try it out" demo
I've also discussed service discovery with etcd & docker if you want to see more about what that custom code might look like.
One last thing! etcd & consul are written in go, so maintaining them is much easier then java solutions like zookeeper. All you need is the consul / etcd binary. no dependencies, no linked libraries, no jvm.
There's a very promising alternative to ZooKeeper called etcd (github.com/coreos/etcd), written by CoreOS team. Unlike Doozerd, etcd is being actively developed.
Just discovered Accord (C) and OpenReplica/ConCoord (Python) which may be interesting solutions
[EDIT] The Hashicorp crew, of Vagrant and Packer fame, are cooking "a decentralized solution for service discovery and orchestration" called Serf.
[EDIT2] Hashicorp strikes again ! They just released Consul, built on top of Serf. The pitch: "a solution for service discovery and configuration, completely distributed, highly available, scalable to thousands of nodes and services across multiple datacenters".
Yes, there is also Doozerd (https://github.com/ha/doozerd). Take a good look at it, it's a nice, single binary distributed coordination service developed by Heroku. With bindings/libraries for java/python/ruby/node. Very easy to get started with and play around.
Take a look at Serf. There is a comparison vs Zookeeper here.
OpenReplica from my research group is a highly available FOSS coordination service for data centers. It can be used for implementing locking, fail over, leader election, group membership and other coordination services. It differs from ZooKeeper in two critical ways:
It uses an object-oriented API. This makes it much easier to write coordination services. Synchronization code for OpenReplica looks exactly like its textbook counterpart; there is no need to master a file and upcall-based API like in ZooKeeper and Chubby.
It enables dynamic membership updates to the replica set. There is no need for static configuration files. The system is integrated into DNS (authoritative, slave for OpenReplica, or Amazon Route 53).
We actively support the system, do not hesitate to let us know if you have further questions.
There's a project called Noah on github that looks interesting, it says that it's "loosely based on Apache ZooKeeper" https://github.com/lusis/Noah with REST support being a key feature (ZK has this as a contrib/option rather than built in).
There are different tools that optimize for different engineering trade-offs.
ZooKeeper Scales marginally for reads; writes with many observers can be slow. It is proven and has a sizable community.
Accord Seems interesting for write-intensive uses, however typical use-cases already have domain-specific solutions (i.e., logging, telemetry).
The others are somewhat interesting but generally unproven. Don't get this wrong if intended for production usage.
I'd found this comparison of Zookeeper, etcd and Doozer:
http://devo.ps/blog/zookeeper-vs-doozer-vs-etcd/
Serf (serfdom.io) is also a nice solution as it is simple! But you must consider, SERF is just a cluster-manager which enables you to send custom events to all cluster nodes. Thats nice, but you have to write your own shell scripts (aka events).
See this example: "https://www.digitalocean.com/community/articles/how-to-set-up-a-serf-cluster-on-several-ubuntu-vps"
The advantage is, you're getting a very simple cluster-manager and you're able to combine this with your favorite configuration, deployment or continuous integration tool.
It seems Corosync is also like ZooKeeper.
I know this post is quite old, but someone who is looking at all possible alternatives I would also like to suggest JGroups library which is mature enough to be used in production environment. I have used it successfully in one of my projects mainly for distributed coordination and to share messages between cluster. It also support AWS support in addition to its flexible architecture where you can customize its stack to get what you need. I suggest you to have a look at it