I have few questions, I could not find the answers from Redis tutorial
1) how can I view/check values of a Redis PubSub channel? Monitor command is there to debug Redis, but I want to check what previously has pushed to channel.
2) What is the exact differece between channel and a queue?
3) how can I monitor Redis cluster in a free web based application?
1) You cannot view/check values that were published on a channel in the past. You can think of pubsub as fire and forget. Redis publishes a messages on a channel to clients which have subscribed to it, but does not persist the message for future reference. You can only monitor the messages published in realtime
2) Channel is a reference used by Redis to know which clients have subscribed to received messages published on that channel.
Queue is a data structure which stores values, these values can be accessed in future in FIFO order. So if you are using a queue for messaging, the messages will remain in the queue until you explicitly delete them
3) IMO there isn't any great free monitoring tool for Redis out there. See some available options here
As a side, regarding questions 1) and 2) : in case you are looking for reliable messaging, check out Redis Streams.
Related
I am working on a node JS app that connect to redis server and subscribe to a channel to get the messages.
There is a bit of confusion, should we really enable "key space notifications" on redis config to get the events in client
The same scenario I have tried using rdis cli, with which i see "key space notifications" are not enabled at the same time I have subscribed to a channel with a pattern, so when ever I publish a message from the other client, I am able to capture that event in subscribed client.
Is the "key space notifications" mandatory , but the POC says other way.
Does any one know what should be the right approach here, subscribing to channel is suffice to get messages, and its nothing to do with "key-space-notifications" ??
From Redis Keyspace Notifications
Keyspace notifications allow clients to subscribe to Pub/Sub channels in order to receive events affecting the Redis data set in some way.
Examples of events that can be received are:
All the commands affecting a given key.
All the keys receiving an LPUSH operation.
All the keys expiring in the database 0.
Events are delivered using the normal Pub/Sub layer of Redis, so clients implementing Pub/Sub are able to use this feature without modifications.
So, if you need just pub/sub, there is no need of extra configuration regarding Keyspace Notifications
Is the message order of pubsub messages in a redis cluster in any way guaranteed?
We are using a Redis cluster (v3.2.8) with 5 master nodes, each with one slave connected & we noticed that we sometimes get pubsub messages in wrong order when publishing to one specific master for one specific channel and being subscribed to slave nodes for that channel.
I could not find any statements related to pubsub message order in cluster on redis.io nor on the redis-github repo.
First of all, if you are using PUBLISH, then it is blocking and returns only after messages have been delivered, so yes the order is guaranteed.
There are 2 problematic cases that I see: Pipelining and Client disconnection.
Pipelining
From the documentation
While the client sends commands using pipelining, the server will be forced to queue the replies, using memory.
So, if a queue is used, the order should be guaranteed.
Client disconnection
I can't find it in the documentation, but if the client is not connected or subscribed when the message is published, then it wont receive anything. So in this case, there is no guarantee.
If you need to persist messages, you should use an a list instead.
I'm learning on how to get data from redis using seneca js but seneca provides multiple plugins to connect to redis. and available plugins are the ones mentioned in the title. which should I use just to fetch a couple of keys from redis? and what is the difference between the two?
seneca-redis-pubsub-transport and seneca-redis-queue-transport are both used for transporting messages between services using redis.
seneca-redis-pubsub-transport is a broadcast transport. All subscribed services will receive all messages. seneca-redis-queue-transport on the other hand is a queue transport. Messages are sent to only one of possibly multiple subscribed services.
If you only want to get/set some values that take a look at seneca-redis-store. This plugin allows you to get and set values using redis.
Provided that both the client subscribed and the server publishing the message retain the connection, is Redis guaranteed to always deliver the published message to the subscribed client eventually, even under situations where the client and/or server are massively stressed? Or should I plan for the possibility that Redis might ocasionally drop messages as things get "hot"?
Redis does absolutely not provide any guaranteed delivery for the publish-and-subscribe traffic. This mechanism is only based on sockets and event loops, there is no queue involved (even in memory). If a subscriber is not listening while a publication occurs, the event will be lost for this subscriber.
It is possible to implement some guaranteed delivery mechanisms on top of Redis, but not with the publish-and-subscribe API. The list data type in Redis can be used as a queue, and as the the foundation of more advanced queuing systems, but it does not provide multicast capabilities (so no publish-and-subscribe).
AFAIK, there is no obvious way to easily implement publish-and-subscribe and guaranteed delivery at the same time with Redis.
Redis does not provide guaranteed delivery using its Pub/Sub mechanism. Moreover, if a subscriber is not actively listening on a channel, it will not receive messages that would have been published.
I previously wrote a detailed article that describes how one can use Redis lists in combination with BLPOP to implement reliable multicast pub/sub delivery:
http://blog.radiant3.ca/2013/01/03/reliable-delivery-message-queues-with-redis/
For the record, here's the high-level strategy:
When each consumer starts up and gets ready to consume messages, it registers by adding itself to a Set representing all consumers registered on a queue.
When a producers publishes a message on a queue, it:
Saves the content of the message in a Redis key
Iterates over the set of consumers registered on the queue, and pushes the message ID in a List for each of the registered consumers
Each consumer continuously looks out for a new entry in its consumer-specific list and when one comes in, removes the entry (using a BLPOP operation), handles the message and moves on to the next message.
I have also made a Java implementation of these principles available open-source:
https://github.com/davidmarquis/redisq
These principles have been used to process about 1,000 messages per second from a single Redis instance and two instances of the consumer application, each instance consuming messages with 5 threads.
How long will messages published to a Redis channel stay there ? Also, is there is a way to configure the max. lifetime of a message per channel ? Is there a way to control the channel size or does the channel continues to store messages as long as the Redis server has free memory ?
Redis pub/sub doesn't persist published messages. What you are looking for seems more like a message queue which can be implemented using a combination of pub/sub and lists. For more information see pattern sections in RPOPLPUSH command.