I am looking in for the best way to implement the RabbitMQ consumer by using .Net Client which should be run as windows service.
I referred the RabbitMQ documentation and found the way to consume messages by using .Net client (https://www.rabbitmq.com/tutorials/tutorial-one-dotnet.html).
My current scenario is like, RabbitMQ is installed in AWS VM machine. I have to install dotnet client consumer service resides in On-premise network which should consume messages.
Which one is the best way, to always listen the Queue (AMQP protocol) or HTTP API which should get messages on demand (https://pulse.mozilla.org/api/).
Please advise.
Thanks,
Vinoth
I believe the answer is "neither." You should have your message queue as a back-end service behind the firewall, and expose your application functionality through a set of carefully-specified web services. The web services, which are exposed through the firewall but can communicate to services behind the firewall, would produce messages that would be transmitted to the server. Any services needing to produce or consume messages would need to do so via the web services, which would perform safety/security checking prior to forwarding the request on to the AMQP server.
If you need to expose AMQP directly to clients (i.e. that is the purpose of your app), then the recommendation is to do so via STOMP. I think a valid use case for exposing AMQP directly over the internet would be a rare thing to come across. The security implications of doing so would be immense.
Related
We are working towards an architecture like one below but we will have micro services on cloud and some on premises which will talk to each other using queue(s) and bus(es),
Now I am confused with where we should host MassTransit and RabbitMq, also should it be a ASP.NET Core project on its own ? if yes what I will be doing in it ? starting a bus ? creating queues ? I am not able to move forward with this
The simple MassTransit examples are just that, the absolute simplest examples of interacting with queues.
RabbitMQ is your message broker. It is hosted separately.
MassTransit is a development framework that makes it much easier to interact with RabbitMQ (or Azure Service Bus) by abstracting away the implementation-specific "plumbing."
You write any number of .NET services that either publish messages to a queue, or subscribe to queues.
We recently worked on something similar, the way we did it is:
RabbitMQ was hosted separately, and buses/queues creation and management were done from the services that use messaging.
For each service that receives messages you use Maastransit to create a queue because service will be receiving messages using this queue.
You will be using publish/subscribe way of messaging so as mentioned above, inside each service, create a queue with logical name and connect to RabbitMQ server address.
Services that represent senders will publish messages of a custom type you create, and services that represent receivers will subscribe to this type of messages by having a consumer for this type registered inside the bus created.
Hope it helps.
We usually use message passing to send messages to decoupled services. This makes service discovery a non-issue, because (with AMQP in RabbitMQ for instance) you can use the broker's routing capability to dispatch messages to the right queues that feed the correct services. Load balancing is also handled by the message broker.
Enter kubernetes.
The use case that is usually laid out when talking about service replication and re-spawning failing services, is when your clients use some active protocol like http to contact a service, even if this service handles requests asynchronously. In this context, it is a natural fit to have replication controllers, that manage a group of services and a single entry point to load balance between them.
I like kubernetes' intuitive concepts, like rolling deployments, but how to you control this beasts that don't have an http interface ?
UPDATE:
I am not trying to set up a cluster of message brokers. I am looking at message consumers as services. Service clients don't connect directly to the services, they send messages to the message broker. The message broker acts as a load balancer in a way, and dispatches the messages to the subscribed queue consumers. These consumers implement the service.
My question gravitates around the fact that most usage patterns in demos handle services that are called via http, and kubernetes does a good job here to create a service proxy for these services, and a replication controller. Is it possible to create replication controllers for my kind of service, which does not have a http interface per se, and have all the benefits of rolling updates, and minimum instances?
I'm not sure I entirely understand the question. Are you asking how to use RabbitMQ with Kubernetes? Or how to set up a RabbitMQ cluster: https://www.rabbitmq.com/clustering.html? Or how rolling updates interact with RabbitMQ? Or something else?
I think you should be able to create one service and one replication controller per server, and then use the service DNS names in the cluster configuration file. This is the current approach used to run Zookeeper, also. We have a long-standing TODO to make this less verbose (https://github.com/GoogleCloudPlatform/kubernetes/issues/260), but the current approach should be straightforward. You do lose the ability to use a single kubectl rolling-update command to update the cluster, but it's also straightforward to update the instances individually.
I am using ActiveMQ embedded in Glassfish with both the default 61616 communication port and a port with MQTT enabled.
Is there a way to publish to both of these MQs in one call if ActiveMQ is configured a certain way?
If not, is the only way to connect to the MQTT server from the J2EE server through a 3rd party MQTT client?
If so, is there a MQTT lib that can take advantage of J2EE container's connection pools?
All protocols in ActiveMQ will share the same topics and queues.
You can subscribe and publish as you wish from java/JMS and the data will be accessible on the same topic using MQTT.
Of course, there will be some issues if you use JMS-only features, such as ObjectMessage and whatnot, but that is pretty obvious. Stick to text messages on topics and you should be fine.
I'm trying to figure out what is the best solution to work with rabbitmq cluster via wcf.
Current setup:
2 IIS web servers (act as message produces and post messages to queue via amqp wcf client).
2 servers with rabbitmq broker (clustered with mirrored queue, rabbit1 and rabbit2)
Windows service ( worker) with hosted amqp wcf service that listens to incoming messages.
Web role posts messages to rabbit1 node and worker listens to rabbit1 node too. If rabbit1 node fails system(both web and worker) should switch to rabbit2. And that's the question, how to implement this in more elegant way rather than handling connection failures in application code.
First and the only approach I see now is to use wcf4 routing backup endpoints feature. This way solves problem on client side(web role) only but doesn't solve problem on wcf service side(worker role).
One way is to create a wrapper around your service host, used for storing a list of connection strings (which can come from config).
Add a handler to the service faulted event, where you can close and reopen the host with a different connection string.
I have a WPF app that uses WCF (duplex netMsmqBinding) to talk to a self-hosted service app in our domain.
I'm now trying to move this WPF app to the big wide world out there, to talk to the WCF service over the internet (well, make them talk to each other).
I've installed MSMQ and created the services on both, and checked queue and firewall permissions. Based on the reading I've found, what I needed to do then is:
1) Enable MSMQ's HTTP addon.
2) specify the binding's useActiveDirectory=false and queueTransferProtocol="Srmp" to ensure that it uses DIRECT to find the private queue.
However I did that and the WPF app receives the error
An error occurred while opening the queue:Unrecognized error
-1072824215 (0xc00e0069). The message cannot be sent or received from the queue. Ensure that MSMQ is installed and running. Also ensure that
the queue is available to open with the required access mode and
authorization.
Which translates to MQ_ERROR_REMOTE_MACHINE_NOT_AVAILABLE.
I can browse to the remote server ok, and the rest services work fine, so I must be missing something with HTTP or the fact that it's self hosting. Any ideas?
MSMQ over HTTP is a push technology (client -> web server).
You cannot open a remote queue and pull messages from it.