I have 2 java application and they are communicating using Rabbit MQ. Here communicating meaning sending of files between 2 applications. Now I have a requirement to verify the content of the file. I am able to do that. But my question is shall I put the file validation in the message queue module or outside the message queue module? Which will be a better approach?
As per my understanding
the message queue is only used for sending, receiving and temporarily storing the data and validation should not be used inside the rabbit message queue module.
Related
I have 2 applications, called appA and appB. They respectively have a aQueue and bQueue, that both application have an ReceiveEndpoint. Both application use the same host on RabbitMQ.
appA is sending the command CreateEntityCommand to appB, into the bQueue, with bus.Send method.
In appB, I have a consumer that, consume CreateEntityCommand.
** so far so good **
Question #1 :
If my appB consumer successfully create the entity, i'm publishing a EntityCreatedEvent. My EntityCreatedEvent consumer in appA got it right, but the event is also added to bQueue_skipped, why?
Question #2 :
Now, if my appB consumer has an exception, my appA has to be notified. A Fault is generated, in the bQueue. I would like my appA to consume the Fault, but the Fault is automatically on bQueue. If I add an ReceiveEndpoint in appA to listed bQueue, I got a lot of dead_letter (skipped queue).
As a rule of thumb, if your messages get to the dead-letter (skipped) queue, it means that there is a binding between the message type exchange and the queue exchange, but your endpoint has no consumer for a given message type.
It usually happens, when you used to have a consumer and then removed it. MassTransit won't remove the binding for you, but it also won't know how to process messages that keep coming.
You can delete the obsolete binding by going to the RMQ management UI doing the following:
Open the endpoint queue
Click on bindings, there is only one there, pointing to the endpoint exchange
Follow the link to open the endpoint exchange and see the bindings to message type exchanges
There, you can remove those bindings that you no longer need
If you have no messages in the queue, you can also just remove it and MassTransit will create everything for you, from scratch.
I try to send message to multiple queues according this documentation ActiveMQ Wildcards. The idea is to send message to PRICE.> queue and receive them in queues PRICE.STOCK.NASDAQ.ORCL and PRICE.STOCK.NYSE.IBM (queues are created). But instead of forwarding messages to PRICE.STOCK.NASDAQ.ORCL and PRICE.STOCK.NYSE.IBM activemq create new queue PRICE.> that become this message.
I tried to send message with activemq admin tool (send mask) and spring boot application. Behavior is the same - message is placed in new created queue PRICE.>.Activemq was not additionally configured, I'm using configuration provided with activemq 5.15.7.
The feature is only supported for subscribers, you need to send to a specific named destination when publishing a message. You can use the Virtual Destinations feature of ActiveMQ to define a target destination that forwards to some defined set.
Right now we have a queuing system(activemq) which is storing the messages. And we have written a separate java application that will read the queue and then trigger a exe to do some processing. But we want to do away with this extra application that is linking our activemq and exe. So i want to know whether any queuing system houses a code which will help me run the exe without any extra code written by me.
Any inputs regarding which queuing system can get me this done will be greatly appreciated.
This isn't really how Message Brokers work. You could however embed a broker inside your own application or create a broker plugin to do something. In the end though the best way to do this is to create your own client that can implement your business logic and let the Message Broker do what it was designed to do, route message traffic.
If you want more of a 'push' solution rather than a producer-consumer solution (which sounds to me like you are) you could look into the use of WebSockets. That would be another way of dealing with messages.
As others have said it doesn't look like using a message broker is the solution you want if you don't want to have some additional middleware to provide asynchronous communication.
So you just need something to launch an EXE on message arrival?
Message Queuing Triggers
Just some additional information for you... Triggered applications are natively supported in IBM Websphere MQ via a Trigger Monitor application that runs as a service (in windows implementations) or a daemon (in UNixish implementations).
When a message arrives in a queue, the MQ software will generate another message ("Trigger" message and send it to the "Trigger" queue, which is being monitored by the Trigger Manager app. The app then starts the required application.
So your implementation of an "app to start an app" is not odd ball or strange at all.. IBM do it in their implementation. I see nothing wrong with your implementation and if you can integrate it tighter to activeMQ then you are on a winner.
What about IBM MQ's triggering feature ?
WebSphere MQ provides a feature that enables an application or channel to be started automatically when there are messages available to retrieve from a queue. A message is put to a queue defined as triggered. If a series of conditions are met, the queue manager sends a trigger message to an initiation queue.
https://www.ibm.com/support/knowledgecenter/en/SSFKSJ_9.0.0/com.ibm.mq.dev.doc/q026940_.htm
I've got a webservice that accepts messages that can be sent to a RabbitMQ cluster using whatever queue they define. This is so front-end devs can send messages via javascript.
I want to make the webservice more robust so that when we have network trouble, the webservice can still accept messages and then handle them when the network is back up. After some initial reading, it seems that the Shovel plugin should handle this nicely.
What I was thinking was to install a local instance of RabbitMQ on the webservice box with shovel turned on. I can then send all messages through the local RabbitMQ instance and have it push all messages to the cluster and deal with the network problems.
My problem is after looking at the documentation it seems that I have to configure every queue I want to forward to in the shovel config file. If that's the case I'm not sure this will work since we allow clients to define a queue through the webservice on the fly.
I would like to have the webservice take the messages, hand them off to the local rmq instance and have it pass the messages off to the cluster using the same queues/exachanges/etc.
Has anyone tried this or can explain how the shovel plugin works?
Have you considered sending messages to an exchange instead of a queue. Send all messages to one exchange possibly a topic exchange if you need that kind of flexibility. Then have the consumer handle the different messages or different queues from the exchange. Sending to one exchange would make configuring the shovel considerably easier.
I have a scenario where a client sends an http request to download a file. The file needs to be dynamically generated and typically takes 5-15 seconds. Therefore I am looking into a solution that splits this operation in 3 http requests.
First request triggers the generation of the file.
The client polls the server every 5 seconds to check if file is ready to download
When the response to the poll request is positive, the client starts downloading the file
To implement this I am looking into Message Queue solutions like RabbitMQ. They seem to provide a reliable framework to run long running tasks asynchronously. However after reading the tutorials on RabbitMQ, I am not sure how will I receive the result of the operation.
Here is what I've in mind:
A front end server receives requests from clients and it posts messages to RabbitMQ as required. This front end server will have 3 endpoints
/generate
/poll
/download
When client invokes /generate with a GET parameter say request_uid=AAA, the front end server will post a message to RabbitMQ with the request_uid in the payload. Any free worker will subsequently receive this message and start generating the file corresponding to AAA.
Client will keep polling /poll with request_uid=AAA to check if task was complete.
When task is complete client will call /download with request_uid=AAA expecting to download the file.
The question is how will the /poll and /download handlers of the front end server will come to know about the status of the file generation job? How can RabbitMQ communicate the result of the task back to the producer. Or do I have to implement such mechanism outside RabbitMQ? (Consumer putting its results in a file /var/completed/AAA)
The easiest way to get started with AMQP, is to use a topic exchange, and to create queues which carry control messages. For instance you could have a file.ready queue and send messages with the file pathname when it is ready to pickup, and a file.error queue to report when you were unable to create a file for some reason. Then the client could use a file.generate queue to send the GET information to the server.
You hit the nail on the head with your last line:
(Consumer putting its results in a
file /var/completed/AAA)
Your server has to coordinate multiple jobs and the results of their work. Therefore you will need some form of "master repository" which contains an authoritative record of what has been finished already. Copying completed files into a special directory is a reasonable and simple way of doing exactly that.
It doesn't necessarily need RabbitMQ or any messaging solution, either. Your server can farm out jobs to those workers any way it wishes: by spawning processes, using a thread pool, or indeed by producing AMQP events which end up in a broker and get sucked down by "worker" queue consumers. It's up to your application and what is most appropriate for it.