I wanted to know if RabbitMQ has any built capabilities to call an external exe once its message queue get populated. I understand that we can implement task queues/worker queues in rabbitmq but it has to be done by writing an external application(say in java like they have mentioned in tuttorials http://www.rabbitmq.com/tutorials/tutorial-two-java.html) . Please help me out with this
Adding to my previous question :
I have decided to write an application that will run an exe . But i dont want the application that i write to poll my queue. Instead i want my rabbitmq to trigger my application whenever there is a new message by sending a job to process. Can i do this? how can i add jobs to the queues?
You are probably going to have to write your own consumer. The question is what is sending the messages in the first place and what is the format o the message and do you need that data.
Python is probably the best choice for this task.
Related
I am looking to replace an in-house key-value store and dispatch system and I keep hearing that RabbitMQ may be a solution.
I understand that sends and receives messages using queues, and that these events are triggered by producers creating messages, and consumers receiving them.
But what happens if a consumer is created after a message was sent? Can the consumer ask the queue what its last message was? If not, do I need to include some sort of database to store these messages? Or am I looking for some other technology?
A use case is that I want a GUI to get/set parameters that are used by other apps on a local network. On initialization, the GUI needs to know what the last values were.
In an attempt to answer my own question, it may be that RabbitMQ is not what I am looking for. I may want to instead use Kafka which stores its latest key:value pair in a table. Or I may want to use Redis. What do you think?
Thank you for your assistance.
I think I found a satisfactory answer to my question. I'm looking to create a request-reply model, which RabbitMQ is quite capable of handling. Upon opening the GUI, it sends a request to some other process for some variable, stored either in memory or in a database. That process responds with the requested data. Easy enough.
Recently, I have implemented RabbitMQ for a couple of use cases. Sending mails is one of them (which is quite common in practice)
My Problem Statement:
A web service(say service A) needs to publish 1000 messages in the queue (which will be picked by some mail sending engine). But unfortunately, after publishing 500 messages to the queue, my app crashes.
Now, if I hit the same service again then the 500 messages that were already pushed in the first go will be pushed again. Though the mails duplication isn't a big deal for now, but is definitely not desired. How to deal with this one. Any thoughts ?
Solutions that I came up with:
Using the batch feature - but it is not supported by AsyncRabbitTemplate so I'm restrained from using that.
Using the database. But that's definitely cumbersome. I won't use this one as well.
If you can identify the duplicates, you can use the Idempotent Receiver enterprise integration pattern on the consumer side.
Spring Integration has an implementation.
However, it's not clear why you are using the async template since that is for send and receive operations. This application sounds like it only needs to send the requests, not wait for a reply.
It's also not clear how batching can help since the crash could occur on the consumer side after it has processed half of the batch.
In either case, you need to track where you got to before the crash.
I'm using dask.distributed.Client to connect to a remote Dask scheduler running that manages a bunch of workers. I am submitting my job using client.submit and keeping track of the returned Future:
client = Client("some-example-host:8786")
future = client.submit(job, job_args)
I want to be able to know if/when the job has been sent to and accepted by the scheduler. This is so that I can add some retry logic in cases when the scheduler goes down.
Is there an easy way to get confirmation that the scheduler has received and accepted the job?
Some additional points:
I realise that distributed.client.Future has a status property, but I'm hesitant to use it as it was not documented in the API.
I have tried using dask.callbacks.Callback but with no success. Any assistance with using callbacks with distributed.Client would be appreciated.
EDIT: I could also have the job post back a notification when it starts, but I would like to leave this approach as a last resort if the Client does not support this.
I'm working on an application where-in I have a listener on a rabbit mq queue. Depending on the kind of message, the listener goes ahead and performs a task. My problem is I need a way to spawn a new listener if a single listener isn't able to cope up with the queue. As far as I can tell, I can use the rabbitmq json api to find the len of the queue and take actions based on that. So, I write a script that checks using curl the queue length and spawns a new listener process. Am I on the right path here? Is there a better way to achieve this? I'm looking for a solution that kinda scales with load to a certain limit atleast.
Checking the RabbitMQ API to see the length of the queue is one way, and it would definitely work.
You should try to predict when the load is spiking so that you slowly can increase the number of consumers if needed, so that you don't see a sudden spike of instances spawning. Having many instances spawning simultaneously could cause unnecessary load on your system.
Here is what I am trying to accomplish by using NServiceBus.
I have a publisher and subscriber. The publisher publishes a message from its queue to the subscriber. Then, the subscriber takes the message and writes to a file. The file will be an input for a third-party GUI application (It fires when the file is created and ready to be accessed). (It has to be a file since the GUI application does not have the MSMQ functionality).
I think I can write the message into a file in the Handle() of the subscriber, but I am not sure how I can achieve this. Since, the subscriber will be fire as soon as the message arrives.
Any help would be appreciate it.
Thanks.
Yes, you can absolutely do this. It's just a matter of doing File.WriteAllLines("myfile.txt", "some values here") using the standard File class from the .NET API.
In your case you will most likely want to pull different values from the message being received, but the specific format and structure of the file being written will depend heavily on the needs of the input/receiving application.