I was reading RabbitMQ specification on the AMQP 0-9-1 implementation and followed along the examples from the tutorial page which were helpful on explaining Exchanges and Queues.
The new latest and major release of the AMQP spec v. 1.0 has been out for two months and according to this wikipedia article the definition of Exchange/fanout/direct/topic bindings have been removed. Here is the full spec for AMQP v1.0.
I've been reading the full v1.0 spec but it's very technical and has no general explanation of how it can be used in a scenario for example a Producer and consumer application or how it's different from 0-10 and how to correlate the old Binding and Exchange notion to the new one.
How does all this Exchange functionality fit in the new protocol. If there are example usages with code example of some lib implementation that would be helpful.
AMQP 1.0 in its current state does not specify broker behavior (i.e. how brokers handle and distribute messaages), only how to 2 nodes in AMQP network communicate between them self. There is draft document named "Broker behavior" that is intended to specify this area, but it is in very early stage since most of the 1.0 activity up to now has been concentrated around core protocol specification. So you could say that in some regards, 1.0 specifies much less than old 0.x versions
So until "Broker behavior" part of the spec reaches at least some maturity, it would be hard to come up with 1.0 example that would be equivalent to publishing to exchange and subscribing to it, since different amqp 1.0 entities could implement it quite differently. Probably, many amqp vendors will implement similar behavior to their 0.9 products. For instance RabbitMQ experimental 1.0 implementation provides 1.0 access to their existing 0.9 broker. But that is just my guessing. Here are my some other thoughts about current state of AMQP 1.0
Related
Could you, please, suggest - if I need maximum intersystem/language compatibility, while the main server will be implemented on .NET, could I use NServiceBus or MassTransit, or is it better to use pure RabbitMQ instead?
NServiceBus or MassTransit would give pretty nice abstracrion level, but the easiness of communication between different solutions and environments are critical. I'm thinking more of using pure RabbitMQ now, but would be really gratefull for pointing put some pros and cons if I'm taking it into account wrong.
If you have multiple applications with different languages internally that must send and receive messages I would not recommend NServiceBus or MassTransit. They require certain message headers that they themselves add. You would never be able to leverage all the functionality offered by these messaging frameworks. However, if internally you are all .NET and you have multiple applications that will use the messaging infrastructure then NServiceBus and MassTransit will add a lot of value.
Regarding interoperability with third parties. A strength and weakness of both NServiceBus and MassTransit is that you must send and receive messages as strongly typed classes or interfaces. These get serialised to JSON/XML/BSON etc and deserialised back to types again.
Because of this they require message headers that indicate the type of the message for deserialisation purposes. Without the message type headers they won't work.
Working with types is much easier but it can cause interoperability issues. When integrating with third parties that send XML or JSON messages with no types requires you to create a translation tier between your services and the third party.
You could translate to your own type that maps onto the XML/JSON or translate to a simple type that contains a string property for the XML/JSON. Either way, your translation tier would publish the messages internally using MassTransit/NServiceBus and so the messages would contain all the necessary headers to fully exploit all the functionality they offer.
For sending messages to the third party, the translation tier would translate the messages to the XML/JSON expected by the third party.
How much of your messaging system is involved with 3rd party/inter-system integrations? If the answer is very little then NServiceBus and MassTransit would be great options as they offer lots of great functionality.
If the answer is a lot, then they might still be good options. Having a translation tier will shield your internal services from being exposed to the requirements and changing schemas of your third parties. It will give you greater control and flexibility at a cost of extra moving parts.
Ultimately, the translation tier is not as complex as implementing the patterns offered out-of-the-box by NServiceBus and MassTransit. So I would seriously consider them as a viable option.
Some links regarding inter-operability
https://docs.particular.net/nservicebus/messaging/third-party-integration
http://masstransit-project.com/MassTransit/advanced/interoperability.html
We use rabbitmq in production and really happy with it. we are planning to use rabbitmq fanout feature, but would like to understand the internals.
Here are my questions:
1. When a message is fan'ed out to multiple queues, is the message copied to different queues rather than just the reference?
2. if the entire message is duplicated on all the fan'ed out queues, Is there a way to make it efficient?
Thank you,
-Bala
To store AMQP message content, RabbitMQ uses "binaries" which are a particular type in Erlang.
The Erlang VM only copies those binaries if they are small enough (the limit is 64 bytes if I remember correctly), otherwise, it uses reference counting. There is a chapter in the Erlang documentation about how binaries are implemented. Paragraphe 4.1 should be of interest to you.
We've been using StreamInsight for a while to process various events. In the search of a good ESB platform, we also wonder if we could use StreamInsight for the same purpose.
A couple of the key differences between StreamInsight and something like NServiceBus are
Max message/event size (StreamInsight is obviously limited in this area)
Pub/Sub (takes some work to make StreamInsight work)
Easily turn something into a WCF endpoint (NServiceBus is good for this)
But limitations aside, couldn't StreamInsight serve as a poor man's ESB? Or I'm just trying to fix a square peg in a round circle?
I have no experience with StreamInsight but took a look at the documentation briefly to see what it is all about. Based only on this look, it seems to have some powerful event based features, the ability to source events from certain systems using adapters and output adapters for the target system. It seems to be stream based with a query syntax perhaps with a more analytical capability than integration. Aspects of it remind me of EAI tools but my opinion is that process events is a nice feature of ESB it could be a bit of a square peg in a round hole. The various tools do overlap however.
There are many definitions of an ESB on Stackoverflow and elsewhere. I like the simple definition that Hohpe uses on the EAIPatterns website:
http://www.eaipatterns.com/MessageBus.html
The Bus products, including NServiceBus, do over some simiar features, but IMO they tend to offer a more dynamic and potentially fine grained way to integrate. In Nservicebus messages are usually C# POCO classes. Some can be large and it does support a databus. The bus messages are move from endpoints on the bus transactional through a transport. PubSub is built in and intentional. Rather than using adapters, NServiceBus uses a C# API that allows the code to integrate to the bus.
I hope this helps.
Regards,
Joe.
On the MSDN we can read :
The WS2007HttpBinding class adds a system-provided binding similar to WSHttpBinding but uses the Organization for the Advancement of Structured Information Standards (OASIS) standard versions of the ReliableSession, Security, and TransactionFlow protocols. No changes to the object model or default settings are required when using this binding.
But I don't find any documentation which can explain me WHY I would like to move wsHttpBinding to ws2007HttpBinding, it seems to me that the standard are the same.
Can someone can give me a good explanation ?
The bindings support different protocols. This page on MSDN actually has a nice matrix that explains what protocols are supported by which binding in WCF. So if you need interop with services/clients that implement OASIS protocols, use the ws2007httpbinding binding, otherwise, there's no reason to not use the wshttpbinding.
If you want to get into details of the different protocols, check out their websites: WC3 and OASIS. I'm sure there's tons of resources that highlight the differences in those protocols.
Different large enterprise and governments needs to use web services and have different requirements. Thus, different standards make sense.
From OReilly book "Programming WCF services" (p.28-29) they say that Ws2007HttpBinding derives from the WsHttpBinding. It adds support for emerging standard and updates for the transaction, security and reliability standards.
Using the lastest standard sounds like a good practice, but just keep in mind that WS2007HttpBinding is only supported by clients that are running at least .NET runtime versions 3.5 SP1 or 3.0 SP1.
ws2007HttpBinding defined by OASIS is a newer version than wsHttpBinding.
It added ReliableSession, Security, and TransactionFlow protocols on top of Transactions, Reliable messaging, and WS-Addressing protocols of wsHttpBinding.
It could be easier to start with simple binding, and if required in the future, you can always expose the existing services in a newer binding coexisting with the older version.
I'm considering using AMQP (using qpid) to enable a mixture of Python and Java services communicate with each other. Basic text messaging seems simple enough but, as with every other messaging technology I've investigated, that's where it seems to stop. Except for building instant messaging applications, I would have thought sending strings wasn't a particularly useful thing to do yet example after example demonstrates sending unformatted text around.
My instinct then is to use XML (de-)serialization or something similar (JSON, YAML, Protocol Buffers etc.) which has good library support in both languages. Is this a best practice and, if so, which (de-)serialization protocol would people recommend? Or am I missing the point somewhere and should be quite content sending small bits of text?
Owen, may I offer a few words about RabbitMQ.
AMQP is a binary protocol and you can certainly do much more than send strings around! Which Python client do you plan to use? We recommend Barry Pederson's client for most uses: http://barryp.org/software/py-amqplib/ You are most welcome to come to the RabbitMQ list and ask any questions you like about anything in relation to your post and the comments :-)
As James points out, JSON is goodness. RabbitMQ supports JSON-RPC over HTTP connecting to an AMQP back end. People also use RabbitMQ with Orbited for comet type apps.
In addition we are fans of, and support XMPP, and STOMP too which James invented. STOMP is handy for a certain class of messaging apps and RabbitMQ supports it for both direct and topic based routing. We've found it a fine way to interop with ActiveMQ, preferring it to JMS in that scenario.
I hope you find the right server for your use cases, and recommend you try out different combinations, for best results.
Cheers,
alexis
For what it's worth, I've been having a good experience using AMQP + Protocol Buffers.
If the sender is serializing the messages, you will probably need to include an id in the header so that you know how to de-serialize on the receiving side. However, this isn't too much trouble to accomplish.
XML or JSON are probably the easiest. Protocol buffers is cool but I'd treat it as an optimisation to think of later on if you really need to (as its a bit harder to use being essentially a binary wire format).
BTW you might want to look at Stomp rather than AMQP; its got way more client libraries and supported message brokers. e.g. Apache ActiveMQ which is way more popular than qpid or rabbitmq - or indeed any JMS provider would work just fine.