EdgeX uses Redis PubSub by default for its messaging bus (https://docs.edgexfoundry.org/2.3/microservices/application/Triggers/).
I have started the Redis server locally.
I have Core Data and/or Device Services running, which I believe is
also configured defaultly to use Redis Pub/Sub.
I have a Virtual Device Service that publishes data to the
edgex/events/# topic
(https://docs.edgexfoundry.org/2.3/microservices/device/virtual/Ch-VirtualDevice/).
Finally, I have configured my Application Service to subscribe to
the topic edgex/events/#, as shown in the example.
[Trigger.EdgexMessageBus]
Type = "redis" # message bus type (i.e "redis`, `mqtt` or `zero` for ZeroMQ)
[Trigger.EdgexMessageBus.SubscribeHost]
Host = "localhost"
Port = 6379
Protocol = "redis"
SubscribeTopics="edgex/events/#"
[Trigger.EdgexMessageBus.PublishHost]
Host = "localhost"
Port = 6379
Protocol = "redis"
PublishTopic="" # optional if publishing response back to the MessageBus
The Application Service is able to recieve all the messages sent to the topic.
However, when I go directly to the redis server (using redis-cli) and subscribe to SUBSCRIBE edgex/events/# or any other variant (edgex/events,edgex), nothing appears. Even checking PUBSUB CHANNELS shows that there are no active channels.
I am assuming that since EdgeX is using my localhost redis server (or any remote server, for that matter), that I'd be able to directly check with that redis server, subscribe to the topic that EdgeX is publishing to, and see the same messages.
Am I missing anything?
Thanks!
The EdgeX implementation is using PSUBSCRIBE with wildcards; the only command that will give you visibility is PUBSUB NUMPAT. You will need to identify the correct pattern for what you are trying to subscribe to AND have your subscriber running before anything is published as Redis PubSub is fire/forget.
Rather than going directly to Redis, I recommend using the EdgeX Application Services to subscribe and then either operate on the results directly or feed that to an external service.
Related
I'm trying to create a pub/sub design based on ASP.NET Core SignalR (Server), Redis, .NET Core Hosted Service (Client).
With Docker, I've created a cluster consisting of two SignalR servers running on Port 8080 & 8081 respectively and a Redis running on port 6379. Both the servers are listening for messages sent to a Redis channel (topic).
Next, I fire up two instances of Client, one subscribing for a specific SignalR group to the SignalR server running on port 8080 and another client doing the same thing for SignalR server running on port 8081.
When this system is running, I publish messages to Redis channel by another service, those messages are picked up by SignalR servers because they both are listening to the same channel. My problem starts from here, on every client I am receiving same message twice. Remember one client is connected to one server. Any suggestions?
N.B.: In Redis-cli I am getting each message only once. So no issues with Redis.
I wish to run an experiment in which the publisher loses connection with the broker and then enqueues messages in its own queue and then when it regains connectivity it sends all its queued messages to the broker. How can I I do this since if I call close connection, I can no longer send(raises an exception). A trick that I can think of is to use a network of two brokers and simulate the above by breaking the connection between the two brokers. Is there an API call that I can use to do the above?
This is very much like facebook messenger or whatsapp acting as a publisher and enqueuing our to-send messages if we are offline and sending them once we are connected.
There is plenty of solutions you could use to break the connection in order to test, here is a non-comprehensive list :
Make a script that can set/unset a firewall rule on your environement blocking the connection port
If you are working with VMs, you can suspend/resume the one running Activemq, you can even automate it with tools like vagrant (vagrant suspend, then vagrant up)
Tweak the connection manualy accessing the activemq jmx
Develop an activemq plugin able to trash connections on demand (or maybe there is one ?)
Now in order to have the behavior you wish to obtain there is two options :
1) Make sure your connection is failover so it can be reestablished, and store your message on disk before sending them with your producer.
2)Produce to a local broker embbeded in your app, and connect this one to the remote broker.
I try to build a system with multiple servers messages exchange.
I have server called Master and another server called Slave.
Master sends messages to Slave and Slave sends messages To Master asynchronously .
I have rabbitmq server on both servers and use federation plugin on both of them to get messages.
So publishers and consumers on both servers communicate only with local rabbitmq server. And all messages exchanges between servers are done only with rabbitmq .
It works fine. When both servers are online.
My requirement is that when there is no network connection between servers then messages should be accomulated until a connection is back.
And it doesn't work with federation plugin . If federation connection is not active then messages are not stored on local rabbitmq.
What should i do to have a model where messages can wait for connection to be delivered to other rabbitmq server?
Do i need to provide more info on my current model?
There is simpler description
RabbitMQ1 has exchange MASTER. RabbitMQ2 created federation with a link to RabbitMQ1 and assigned permissions to the exchange MASTER
Publisher writes to RabbitMQ1 to exchange MASTER with routing key 'myqueue'
Consumer listens RabbitMQ2 on exchange MASTER and queue 'myqueue'.
If there is connection then all works fine
if no connection then messages posted to RabbitMQ1 are not delivered to RabbitMQ2 when connection is back.
How to solve this?
I found the solution for this. Federation is not good plugin for such solution
I used shovel . It does exactly what i need
I am using ActiveMQ embedded in Glassfish with both the default 61616 communication port and a port with MQTT enabled.
Is there a way to publish to both of these MQs in one call if ActiveMQ is configured a certain way?
If not, is the only way to connect to the MQTT server from the J2EE server through a 3rd party MQTT client?
If so, is there a MQTT lib that can take advantage of J2EE container's connection pools?
All protocols in ActiveMQ will share the same topics and queues.
You can subscribe and publish as you wish from java/JMS and the data will be accessible on the same topic using MQTT.
Of course, there will be some issues if you use JMS-only features, such as ObjectMessage and whatnot, but that is pretty obvious. Stick to text messages on topics and you should be fine.
I went thought NServiceBus documentation including the durable messaging one. What I understand is that when the server is offline the messages continue to go into the server's input queue which get picked up when server comes back online.
But what if the server is completely down and the input queue is not accessible?
I'm using Bus.Send from the client.
It depends on what transport you're using.
In the case of a brokered message queue, like Azure Service Bus, as long as that service is available, the fact the machine that will eventually retrieve the messages is offline is irrelevant, as that machine is simply asking the external queuing service for messages. The same goes for a transport like SQL Server.
In the case of a transport like MSMQ, which is a store a forward style queue, the messages will remain in a local outgoing queue until the remote machine becomes available.
Can you double check that you are looking in the correct spot? If you aren't getting an error out of NServiceBus when you Send, then MSMQ is installed. If it can't be reached or the service is stopped you should get errors.
The Outbound queues are in a different place as illustrated here:
http://blogs.msdn.com/cfs-filesystemfile.ashx/__key/communityserver-components-postattachments/00-09-06-31-16/outgoingempty.JPG
As RMD indicated, this is an advantage of the store and forward MSMQ transport.. the local outbound queue should just stack these up until the remote server is available.
Thx.
Joe