I'm having a play with Swift Vapor and want to build a web API using microservices.
What's the best way for sending messages between microservices built with Vapor and a message queue such as RabbitMQ?
The option right now I think is Redis Pub/Sub which there are a Vapor package for.
We have done a little work on a Kafka package but it never got finished. Besides this we can see what the new Server Side Swift working group comes up with. They have Kafka on their list, and I have personally suggested RabbitMQ.
I just got aware of this package, looks like it could do what you want :) https://github.com/johnbona/disque
Related
I have a FastAPI service, a light weight pass through service. On receiving a POST request, the
service publishes a message to a RabbitMQ queue.
Since the pika library is not thread safe, I will have to create a new connection for every request .
But the RabbitMQ documentation suggests otherwise - the recommendation is to use long lived connection(s) for an application, and channels for different threads/requests.
My application does not expect heavy loads, so is it alright to use the pika library and create a connection for every request ? Or, is rabbitpy a better library for my use case?
Thank you.
Perhaps not the answer you're looking for but I have just had exactly the same problem and think I have solved it using the pika AsyncioConnection hooked into the asyncio event loop used by FastAPI.
The code is callback hell but you can hide it in a module.
My solution
I'm kind of new to these protocols, and just started exploring Message brokers like Apache Apollo and RabbitMQ.
So my broker receives MQTT messages from a publisher. And I would like to convert it into AMQP (preferably) or STOMP protocol to send to a web server. But I've so far been unable to do so.
I looked into RabbitMQ, and tried enabling the MQTT plugin, but when I do load it, I'm unable to start the server.
I was wondering if anyone can guide me here? Is there an API that can help me? And I'm very confused about RabbitMQ. I've been able to load other plugins easily,like stomp, management utilities etc.
I'm 100% sure it is doable. I am doing it right now with robomq.io broker. One cause could be sometimes bugs in your client library restrict you doing so.
Another thing you should be aware of is that internally, RabbitMQ MQTT adapter is mapped into amq.topic exchange by default, so on your STOMP peer, you should subscribe or send to /topic/yourTopic; on your AMQP peer, bind your queue to amq.topic exchange or publish to that exchange.
Follow this example code and documentation to build your client.
If you can't figure out your server, just get a free trial from robomq.io. It saves you time and money.
The development tool I am using is robomq.io broker, producers in Python (AMQP library: pika, MQTT library: paho, STOMP library: stompest), consumer in Node.js (library: amqplib).
Hope it helps!
Well, I'm not sure if this question should be taken down. But if it has to be I leave it to the discretion of the moderators and the stackOverflow community in general.
btw, I use Ubuntu 14.04.
About the RabbitMQ broker
So Mosquitto was running un the background occupying the port 1883 normally used for MQTT. I could have changed the port for RabbitMQ, but decided against it and tried to kill the Mosquitto process. But for some reason, I could NOT.
For now, my quick fix was removing Mosquitto completely and this freed the port, enabling RabbitMQ to use it.
About the protocols
I've used Paho and the RabbitMQ libraries provided to code out simple programs that can publish and receive messages in AMQP/MQTT via the RabbitMQ broker.
(My Googling needs to be better!)
Still haven't converted one to the other. But that shouldn't be too big a step to achieve.
Still would be nice to know if there's an API or something that can help me achieve the conversion in a very simple manner. Of course, if there's not, I'll figure it out ASAP
Any suggestions/comments are heartily welcome. I'm brand new to all this and could really use advice from all you seasoned pros :)
Right now we have a queuing system(activemq) which is storing the messages. And we have written a separate java application that will read the queue and then trigger a exe to do some processing. But we want to do away with this extra application that is linking our activemq and exe. So i want to know whether any queuing system houses a code which will help me run the exe without any extra code written by me.
Any inputs regarding which queuing system can get me this done will be greatly appreciated.
This isn't really how Message Brokers work. You could however embed a broker inside your own application or create a broker plugin to do something. In the end though the best way to do this is to create your own client that can implement your business logic and let the Message Broker do what it was designed to do, route message traffic.
If you want more of a 'push' solution rather than a producer-consumer solution (which sounds to me like you are) you could look into the use of WebSockets. That would be another way of dealing with messages.
As others have said it doesn't look like using a message broker is the solution you want if you don't want to have some additional middleware to provide asynchronous communication.
So you just need something to launch an EXE on message arrival?
Message Queuing Triggers
Just some additional information for you... Triggered applications are natively supported in IBM Websphere MQ via a Trigger Monitor application that runs as a service (in windows implementations) or a daemon (in UNixish implementations).
When a message arrives in a queue, the MQ software will generate another message ("Trigger" message and send it to the "Trigger" queue, which is being monitored by the Trigger Manager app. The app then starts the required application.
So your implementation of an "app to start an app" is not odd ball or strange at all.. IBM do it in their implementation. I see nothing wrong with your implementation and if you can integrate it tighter to activeMQ then you are on a winner.
What about IBM MQ's triggering feature ?
WebSphere MQ provides a feature that enables an application or channel to be started automatically when there are messages available to retrieve from a queue. A message is put to a queue defined as triggered. If a series of conditions are met, the queue manager sends a trigger message to an initiation queue.
https://www.ibm.com/support/knowledgecenter/en/SSFKSJ_9.0.0/com.ibm.mq.dev.doc/q026940_.htm
How to use RabbitMQ message broker in Java application? I have not found any document or link to understand how to do it.
There's plenty or RabbitMQ documentation.
In particular, you may be interested in the Java client API guide and the javadocs.
I have tried and tested the JMX API and it is pretty simple to use and provides a vast number of statistics required for monitoring ActiveMQ.
But the problem is, i dont want to monitor my ActiveMQ remotely and also i dont want to use another API.To be more precise, i want to use the JMS API itself to get statistics related to various destinations and the broker itself.
Advisory messages seem to be an alternative but they provide limited Amount of Administrative Messages to monitor.
Any input is highly appreciated...
There is no built-in support for this. But you can implement a JMS topic which publishes the monitoring data every few seconds. Make the connection non-persistent so that it doesn't pile up when there are no listeners or when they loose connection.
Now you can write a client that connects to this topic and it will receive updates.
AMQ-2379 resulted in a broker plugin for grabbing statistics from destinations by sending a simple JMS message. Check out the docs that show how to use it here:
http://activemq.apache.org/statisticsplugin.html
The statistics plugin is available in the 5.3 release.
You can checkout this http://issues.apache.org/activemq/browse/AMQ-2379, it will be avaiable in upcoming 5.3.0 release
There's a blog post queued up to go on http://issues.apache.org/activemq/browse/AMQ-2379 - will post it in a couple of days or so