I have a FastAPI service, a light weight pass through service. On receiving a POST request, the
service publishes a message to a RabbitMQ queue.
Since the pika library is not thread safe, I will have to create a new connection for every request .
But the RabbitMQ documentation suggests otherwise - the recommendation is to use long lived connection(s) for an application, and channels for different threads/requests.
My application does not expect heavy loads, so is it alright to use the pika library and create a connection for every request ? Or, is rabbitpy a better library for my use case?
Thank you.
Perhaps not the answer you're looking for but I have just had exactly the same problem and think I have solved it using the pika AsyncioConnection hooked into the asyncio event loop used by FastAPI.
The code is callback hell but you can hide it in a module.
My solution
Related
I'm having a play with Swift Vapor and want to build a web API using microservices.
What's the best way for sending messages between microservices built with Vapor and a message queue such as RabbitMQ?
The option right now I think is Redis Pub/Sub which there are a Vapor package for.
We have done a little work on a Kafka package but it never got finished. Besides this we can see what the new Server Side Swift working group comes up with. They have Kafka on their list, and I have personally suggested RabbitMQ.
I just got aware of this package, looks like it could do what you want :) https://github.com/johnbona/disque
I am experimenting the new version of NServicsBus. I find following step by step sample on particular site.
https://docs.particular.net/samples/step-by-step/
Can any one tell me how to configure MSMQ for Transport. Here is my scenario.
Client create message
Client message should be stored in MSMQ
Server Application running on same machine which subscribe the message.
Server handler get message from MSMQ and process it further. i.e Store in DB or send to other web service.
Retry to process message if it does not worked first time
after 3 retries send message to error queue
How do i configure this sample to use MSMQ for my scenario.
Helpful information to include
Product name:NServiceBus.Core
Version: 6.3.4
Stacktrace:
Description:
Did you know that we have released a LearningTransport and LearningPersistence just for purposes like these? Have a look at it here.
Having said that, the transport swapping should be rather seamless so even if you have setup a small PoC using this transport/persistence, you can change it to MSMQ or other production-ready transports/persistence when you go live.
Again, as stated in the documentation page and as the name suggests, this is not for use in production.
I would recommend you walk through this.
https://docs.particular.net/tutorials/intro-to-nservicebus/
Will answer your questions, and future ones you have.
Right now we have a queuing system(activemq) which is storing the messages. And we have written a separate java application that will read the queue and then trigger a exe to do some processing. But we want to do away with this extra application that is linking our activemq and exe. So i want to know whether any queuing system houses a code which will help me run the exe without any extra code written by me.
Any inputs regarding which queuing system can get me this done will be greatly appreciated.
This isn't really how Message Brokers work. You could however embed a broker inside your own application or create a broker plugin to do something. In the end though the best way to do this is to create your own client that can implement your business logic and let the Message Broker do what it was designed to do, route message traffic.
If you want more of a 'push' solution rather than a producer-consumer solution (which sounds to me like you are) you could look into the use of WebSockets. That would be another way of dealing with messages.
As others have said it doesn't look like using a message broker is the solution you want if you don't want to have some additional middleware to provide asynchronous communication.
So you just need something to launch an EXE on message arrival?
Message Queuing Triggers
Just some additional information for you... Triggered applications are natively supported in IBM Websphere MQ via a Trigger Monitor application that runs as a service (in windows implementations) or a daemon (in UNixish implementations).
When a message arrives in a queue, the MQ software will generate another message ("Trigger" message and send it to the "Trigger" queue, which is being monitored by the Trigger Manager app. The app then starts the required application.
So your implementation of an "app to start an app" is not odd ball or strange at all.. IBM do it in their implementation. I see nothing wrong with your implementation and if you can integrate it tighter to activeMQ then you are on a winner.
What about IBM MQ's triggering feature ?
WebSphere MQ provides a feature that enables an application or channel to be started automatically when there are messages available to retrieve from a queue. A message is put to a queue defined as triggered. If a series of conditions are met, the queue manager sends a trigger message to an initiation queue.
https://www.ibm.com/support/knowledgecenter/en/SSFKSJ_9.0.0/com.ibm.mq.dev.doc/q026940_.htm
What is the best way to combine a single instance WCF service that uses ActiveMQ and runs within IIS/AppFabric?
Our Services need to support both HTTP transports and ActiveMQ (listening and sending messages). We've elected not to use MSMQ, and will use Spring.Net.NMS. The fundamental issue I have now is that ActiveMQ needs to connect to the queue(s) at startup and remain connected, but WAS is getting in the way with it's message-activation feature. If the service is not activated until a message arrives (HTTP/MSMQ, etc) then there is no trigger to have the connection to AMQ occur.
I know I can disable the recycling behavior, and I know I can do self-hosting with a Windows Service. But I want to take advantage of the monitoring and other features in AppFabric. I've already been down the route with IServiceBehavior and will use that for other nice things. But that interface is not called until a (non-AMQ) message arrives. So it won't work for this. What I was hoping for was something along the line of how ServletContextListeners work in Java, where you get both the start up and shutdown events. But it seems no such thing exists in WAS... it is driven only by messages arriving.
I've scoured every inch of web info for 3 days and the only thing I came across was to use a static class construction (C#) trick as the trigger. That's a hack, but i can live with it. It still leaves the issue of cleanly shutting down, which I can figure out later.
Anyone have a solid solution to this?
The direct WCF support for ActiveMQ that Ladislav mentions is still being supported. There just hasn't been an official release for the module in a while. However, you can still get the latest version of it from the 1.5.x branch or trunk and compile it yourself.
1.5.x branch for use with Apache.NMS 1.5.0:
https://svn.apache.org/repos/asf/activemq/activemq-dotnet/Apache.NMS.WCF/branches/1.5.x/
Check out instructions:
http://activemq.apache.org/nms/source.html
There was direct WCF support for ActiveMQ but I guess it is not developed anymore. Your problem actually is the IIS / WAS (provides hosting for non-http protocols) hosting architecture. Services in WAS are always activated when message arrives - there is no global startup. The reason for this is that WAS hosting expects that there is separate process (windows service) running the listener all the time and this process has adapter which calls WAS and uses message level activation. I guess you don't have such process for ActiveMQ and because of that you will have trouble to use ActiveMQ endpoint hosted in WAS. Developing such listener can be challenging task (example for UDP).
Creating custom listener can be probably avoided by using IIS 7.5 / AppFabric auto start feature. There is also not very well documented way to run the code when the application starts.
I have a multi-gui one-service architecture (i implement both, service and client) (using .net 3.5, c#, windows application/service).
whats the best practice to update the guis? i know duplex communication, but if the client doesn't send any requests for some time, the channel is closed! increasing the timeout is not the best solution...?!
Regards
Marcus
The GUIs are sending commands to the service per TCP Duplex. Each time, one GUI is changing some things, the other guis have to be informed about the changes, happend to the service.
first solution: the guis can poll each x seconds the service to ask: are there updates?
but i don't really like this solution. each time a change happend, the service send an "gui you have to update" via callback to each gui. but mayby all the channels are not open anymore...
hope this helps to understand (sorry for my very bad english)
If you want architecture where client registers to the service and service pushes data changes to clients immediately when changes happen than duplex binding is the solution. Otherwise don't use duplex binding.
If you don't like the idea of increasing connection timeout you can always implement your own keep alive mechanism. Client will continue sending some keep alive messages to the service in predefined interval so connection will be still opened.