Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 8 years ago.
Improve this question
I have been working on Active MQ for quite some time and familiar with the Active MQ architecture.
Recently I have been hearing a lot about Kafka as a messaging system.
What advantages does it have over Active MQ and other messaging system? Is it just another Big data buzz word?
Also is kafka suitable for zero loss messaging system?
This is too broad to discuss but in my opinion the most important factor about Kafka over ActiveMQ is the throughput. From the wiki page
Kafka provides an extremely high throughput distributed publish/subscribe messaging system. Additionally, it supports relatively long term persistence of messages to support a wide variety of consumers, partitioning of the message stream across servers and consumers, and functionality for loading data into Apache Hadoop for offline, batch processing.
Also is kafka suitable for zero loss messaging system?
In very brief kafka Guarantees these following :
1) Messages sent by a producer to a particular topic partition will be appended in the order they are sent. 2) For a topic with replication factor N, it will tolerate up to N-1 server failures without losing any messages committed to the log.
Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 2 years ago.
Improve this question
I want to choose a message broker between the ZeroMQ and RabbitMQ for micro-services communications.
Question is which one is faster for data transfer? ZeroMQ(ZMTP protocol) Or RabbitMQ(AMQP protocol)?
Q : " ... which one is faster for data transfer? "
My bet is ZeroMQ being fastest, using inproc:// Transport Class for PAIR/PAIR Scalable Formal Communication Pattern archetype. The same if moving data over cluster-wide using tipc:// Transport Class and having also a few vmci://-connections plus serving epgm://-multicast services at once. Not speaking about MIL-STD-grade norm:// Transport Class applications...
Q : " Differences between ZMTP and AMQP protocols ?"
Quite many. The first one, the ZeroMQ being a universality & many-protocol & many-archetype equipped spin-off of AMQP ( refactored if not reinvented from the core-architecture all the way bottom-up ) having since v2.1+ endless list of language-bindings & wrappers, as evolution continues forwards.
AMQP being a lingua franca for Financial behemots' systems, that are not so keen on re-architecting the massively deployed status-quo in their systems.
AMQP is an agreed, evolving standard, not a Product per-se.
RabbitMQ is a Product, conform with some parts of the AMQP standard, while RabbitMQ may use wire-level protocol and other AMQP-standard parts for declared compatibility reasons, it is not an AMQP per-se.
Did I mention that ZeroMQ is a Broker-Less almost Zero-Latency framework?
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
I'm researching with a friend about the idea of using an event queue/stream system, such as kafka or rabbitmq, as a way to store the adverts in a queue instead of a traditional database.
The required system would need to provide a stream of events holding various fields, be filterable and searchable. Allow the stream to hold the events indefinitely or for a certain amount of time (for example to let the adverts expire). We are just not sure whether to go for a message queue/event stream, or whether the traditional database is the way to go.
Has anybody experience with this, would you recommend to investigate one system over another?
Kafka would support the usecase, as you can treat it not as messaging queue, but as a transaction log:
you can re-read the same message multiple times with different consumers
the messages would be persisted until they expire (configurable on server) or until they are compacted
(data propagation) there are tools such as mirror maker (or own streaming applications) to replicate data among data centres (or some part of them, e.g. if you decide to put some attributes in one topic, and other in another)
I do not know if generic messaging solution (like RabbitMQ) would suit you, as messages consumed would disappear, so you would need to re-publish them to keep them (and in case of multiple consumers, you'd need to use RMQ logic like fanout exchanges that messages get multicasted to multiple queues, each per consumer).
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 8 years ago.
Improve this question
As I understood AMQP 0.9.1, the main benefit was that you could send and receive messages and configure your exchanges / bindings / queues in a broker-independent way, thus you were able to switch your broker implementation without too much headache.
Now, AMQP 1.0 only defines a wire-level protocol, so you actually have to know your broker specifics in order to implement most messaging patterns.
Then why would I favour a message broker that is AMQP compliant over one that is not?
If the broker implements AMQP 1.0, I'm still locked in with broker specific client code. With AMQP 0.9.1, I am theoretically broker independent but would most likely end up with RabbitMQ, since they seem to be the only ones to sincerely maintain the full support for AMQP 0.9.1.
Unfortunately, your concerns are very valid.
Pieter Hintjens has great post on that, claiming that "AMQP/1.0 will go down in history as a prime documented example of how to fail a standardization effort and hurt a lot of people in the process."
To your question: "why would I favour a message broker that is AMQP compliant over one that is not?" Definitely not because you get the option to change your message broker provider, as it is now not required. There are only two situations, I would think of favouring AMQP message broker:
1) You are already used to that and have that broker easily available
2) AMQP still brings the advantage of having clients available in multiple programming languages. But STOMP would serve here too.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 2 years ago.
Improve this question
I am trying to figure out the best MQ option for my requirements. I need to have the ability to transfer both text and binary messages within and across geographically diverse data centers with high reliability. Fast is nice but scaling is an option as well. Support is nice to have as with RabbitMQ.
Here are some assumptions:
Use federation or shoveling messages to push identical messages across data centers.
Use AMQP to transfer binary messages and since we are a .Net/Python shop.
I want to make sure my assumptions are valid and need help with which MQ to pick. I have used ActiveMQ+MySQL in the past but I like the option of Mnesia for messaging with persistance. Also, is it alright to use AMQP 0.9 instead of 1.0. Looks like RabbitMQ support 1.0 via a plugin.
Appreciate any alternate suggestions I can get.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 7 years ago.
Improve this question
We have to implement a queuing system for our ROR Application.We have evaluated the following options:
Amazon SQS: High Availability but relatively slow performance.Requires constant poling.
CloudAmqp: Looks promising but doubtful about the support.
RabbitMq setup on EC2: Needs user bandwidth to manage the setup,may result in downtime if some issue arises in the setup.
Right now there won't be any dedicated team/person to manage the setup full-time so implementing our own RabbitMq setup on Ec2 may result in downtime in case something goes wrong.
I want to know considering the situation which is our best option?
I use SQS and I am happy with it; I don't worry about the support aspect, because I also don't have time to deal with setting up my own server and supporting myself when I can pay AWS pennies to do it for me.
If you don't want to poll constantly, considering pairing up your SQS queue with an SNS topic and it can do push notifications to your application instead. Don't know the nature of your application, but its something to look into. http://aws.amazon.com/sns/
ALso keep in mind the slow performance of SQS (relative to Rabbit) is not apples to apples. SQS is redundant and distributed, a single instance of RabbitMQ on a single box is not; can your application deal with the queue not being available for a period of time?
At CloudAMQP all our servers are redundant, each cluster has at least two instances in different availability zones. For support we have email support 24/7, as we have staff in different timezones. We do have phone support for our largest plans too.