Im experimenting with Apache NiFi and the ConsumeAMQP processor. However it seems not to be possible to create and bind a queue to an RabbitMQ exchange. Am I missing something or is this just not implemented?
The ConsumeAMQP Processor seems to have only some basic settings in the property tab.
Related
This is a follow-up question of How to implement HTTP request/reply when the response comes from a rabbitMQ reply queue using Spring Integration DSL?.
We were able to build the Spring Integration application and the SCDF stream successfully locally. We could send a http request to the rabbitMQ request queue which was bound to the SCDF stream rabbit source. We could also receive the response back from the rabbitMQ response queue which was bound to the SCDF stream rabbit sink.
We have deployed the SCDF stream into PCF environment which had a binding of an internal rabbitMQ broker. Now we need to specify the spring rabbitMQ connection information in the Spring Integration application properties - currently it's using the default localhost#5762, which is no longer valid. Does anyone know how to get this rabbitMQ configuration properties? We already checked the SCDF stream rabbit source/sink log files but couldn't find the information. I know we probably need to check internally whoever set up the SCDF/rabbitMQ in PCF environment, but so far we haven't heard the answers from them.
Also, it appears we can have a different approach that binds both the SCDF stream and the integration application to a separate rabbitMQ instance (instead of using the existing one bundled with the SCDF configuration). Is it a recommended solution?
Thanks,
It is unclear whether you're using the SCDF tile or the SCDF OSS (via manfest.yml) on PCF.
Suppose you're using the OSS, AFA. In that case, you are providing the right RMQ service-instance configuration (that you pre-created) in the manifest.yml, then SCDF would automatically propagate that RMQ service instance and bind it to the apps it is deploying to your ORG/Space. You don't need to muck around with connection credentials manually.
On the other hand, if you are using the SCDF Tile, the SCDF service broker will auto-create the RMQ SI and automatically bind it to the apps it deploys.
In summary, there's no reason to manually pass the connection credentials or pack them as application properties inside your apps. You can automate all this provided you're configuring all this correctly.
EDIT2: My issue here was caused by an insufficient understanding of how transport connectors work in ActiveMQ. TL;DR is that ActiveMQ will implicitly "transform" or "relay" messages between your transport connector configurations defined in activemq.xml.
EDIT: Additional info, the STOMP messages received by the Angular application are used for debugging and demo purposes. Hence, simply converting the OpenWire message to a blob of readable text is sufficient.
I'm creating an Angular application (preferably website, avoiding native applications), which objective is to "tap in" by web sockets on an ActiveMQ server and subscribe to OpenWire messages. How do I let ActiveMQ transform OpenWire messages to STOMP messages and send these to any clients (i.e. my Angular application) connected to the ActiveMQ WebSocket connector?
In addtiion, it would be nice-to-have if I could transform STOMP to OpenWire as well.
It must be Angular
Avoiding the use of native applications on the client-side is preferable although not a deal-breaker.
Adding extra processing stress on the ActiveMQ server must be done with caution.
To the best of my knowledge, it is only possible to let Angular "talk directly" with the ActiveMQ server by STOMP messages send by web socket, if I am to avoid using native applications.
I already have an Angular application capable of STOMP communication by web sockets (e.g. something like https://github.com/stomp-js/ng2-stompjs-angular7).
I am missing information on how to configure the ActiveMQ server to transform OpenWire-->STOMP through its transport connectors.
In my understanding, what I am trying to do should be possible. It is noted by other users but not how. E.g. users hint that what I want is possible in ActiveMQ but not Apollo: ActiveMQ to Apollo transition, Openwire to Stomp protocol configuration.
I expect (preferably) the need to use something like an ActiveMQ transformer (e.g. adding transformer to the connector configuration: AMQP & Openwire - Activemq broker and 2 different consumers) or maybe writing an ActiveMQ plugin (http://activemq.apache.org/developing-plugins.html). On ActiveMQ's website, an existing transformer is mentioned (http://activemq.apache.org/stomp.html Message Transformations section):
Currently, ActiveMQ comes with a transformer that can transform XML/JSON text to Java objects
... but no mention of how to use this and I am unsure if I can benefit from this and if this means that there are no transformers for OpenWire-->STOMP or vice versa.
I expect I might have misunderstood some of the concepts, and a "you're going in a wrong direction, do this instead" can work out as a good answer for me. At the time of writing, I expect I will have to create an ActiveMQ plugin using their Message Transformer interface (http://activemq.apache.org/message-transformation.html) although their sub links are 404. I hope to achieve a more simple solution, e.g. an existing OpenWire-->STOMP transformer:
<transportConnector name="openwire" uri="{some-openwire-uri}?transport.transformer=stomp"/>
ActiveMQ will "transform" any Openwire message into a STOMP message and vice versa as needed based on client connections. I an Openwire based JMS client connects and places a message onto a queue and a STOMP based client comes along and subscribes to that queue the message will be converted into a STOMP message to send to that client.
Without knowing more about what issue you are having it is hard to provide more insight than that though. There are some cases where the transformation from Openwire to STOMP might not yield exactly the right thing for you such as a MapMessage or StreamMessage and definitely an ObjectMessage so some care needs to be taken about cross protocol messaging.
You do of course need to add a transport connector for each of the protocols you want to support, Openwire, STOMP, AMQP etc. The clients need something to connect to, then once they connect the broker manages the message transformations amongst subscriptions on Topics and Queues.
I am using RabbitMQ as a Stomp broker for Spring Websocket application. The client uses SockJS library to connect to the websocket interface.
Every queue created on the RabbitMQ by the Spring is durable while topics are non durable. Is there any way to make the queues non durable as well?
I do not think I can configure on the application side. I played a bit with RabbitMQ configuration but could not set it up either.
Example destination on RabbitMQ used for SUBSCRIBE and SEND:
services-user-_385b304f-7a8f-4cf4-a0f1-d6ceed6b8c92
It will be possible to specify properties for endpoints as of RabbitMQ 3.6.0 according to comment in RabbitMQ issues - https://github.com/rabbitmq/rabbitmq-stomp/issues/24#issuecomment-137896165:
as of 3.6.0, it will be possible to explicitly define properties for endpoints such as /topic/ and /queue using subscription headers: durable, auto-delete, and exclusive, respectively.
As a workaround you can try to create queues by your own using AMQP protocol and then refer to that queues from STOMP protocol.
I've got a webservice that accepts messages that can be sent to a RabbitMQ cluster using whatever queue they define. This is so front-end devs can send messages via javascript.
I want to make the webservice more robust so that when we have network trouble, the webservice can still accept messages and then handle them when the network is back up. After some initial reading, it seems that the Shovel plugin should handle this nicely.
What I was thinking was to install a local instance of RabbitMQ on the webservice box with shovel turned on. I can then send all messages through the local RabbitMQ instance and have it push all messages to the cluster and deal with the network problems.
My problem is after looking at the documentation it seems that I have to configure every queue I want to forward to in the shovel config file. If that's the case I'm not sure this will work since we allow clients to define a queue through the webservice on the fly.
I would like to have the webservice take the messages, hand them off to the local rmq instance and have it pass the messages off to the cluster using the same queues/exachanges/etc.
Has anyone tried this or can explain how the shovel plugin works?
Have you considered sending messages to an exchange instead of a queue. Send all messages to one exchange possibly a topic exchange if you need that kind of flexibility. Then have the consumer handle the different messages or different queues from the exchange. Sending to one exchange would make configuring the shovel considerably easier.
I have a legacy application running on Glassfish which I have just recently configured to use activemq rather than openMQ. My activemq broker is running in a separate process outside of glassfish. I was thinking it would be nice to configure a camel route that logs messages as they are sent to the queue. I want to do something like this
from("activemq:myqueue")
.to("activemq:myqueue")
.wireTap("direct:tap")
.to("log:myqueue");
I don't think that makes sense though. What I want to happen is for camel to log the message transparently to the consumer. I don't want to have to change code so that the producer sends to an "inbound" queue and the consumer receives from an "outbound" queue and camel hooks them up, since that would require changes to the legacy app. I don't think this is possible, but just wondering.
Yeah I was about to suggest looking for a broker solution as it would be the most optimized and performant. Obvious monitoring the message flow in the broker is a common requirement and thus ActiveMQ has features for that:
http://activemq.apache.org/mirrored-queues.html
I think I just found out how I can do what I want with mirrored Queues:
http://activemq.apache.org/mirrored-queues.html
This is a change to the broker, and not purely done in camel.