I'm working on messaging project, where I need to have a large number of Redis PubSub channel subscriptions as shown in the graphic below . I'm on NestJS and use 'ioredis' library. I have few questions regarding this design,
Assuming there is only one subscriber client, do all channel subscriptions get multiplexed through a single Redis connection?
Is there a limit to how many channel subscriptions a client can have assuming Redis cluster is able to scale.
Thanks.
Related
I am new to Redis. I have a use case that a Redis stream shall be consumed by every member of a group of clients. However, currently the concept of consumer group in Redis stream means a message shall be consumed only once by any of the client in the group.
Does Redis stream has built in support for this particular use case?
Thank you
---Patrick
I have few questions, I could not find the answers from Redis tutorial
1) how can I view/check values of a Redis PubSub channel? Monitor command is there to debug Redis, but I want to check what previously has pushed to channel.
2) What is the exact differece between channel and a queue?
3) how can I monitor Redis cluster in a free web based application?
1) You cannot view/check values that were published on a channel in the past. You can think of pubsub as fire and forget. Redis publishes a messages on a channel to clients which have subscribed to it, but does not persist the message for future reference. You can only monitor the messages published in realtime
2) Channel is a reference used by Redis to know which clients have subscribed to received messages published on that channel.
Queue is a data structure which stores values, these values can be accessed in future in FIFO order. So if you are using a queue for messaging, the messages will remain in the queue until you explicitly delete them
3) IMO there isn't any great free monitoring tool for Redis out there. See some available options here
As a side, regarding questions 1) and 2) : in case you are looking for reliable messaging, check out Redis Streams.
I need to design a system that allows
Users to subscribe to any topic
No defined topic limit
Control over sending to one device, or all
Recovery when offline clients, (or APNS) that drops a notification. Provide a way to catch up via REST
Discard all updates older than age T.
I studied many different solutions, such as Notification Hubs, Service Bus, Event Hub... and now discovered Kafka and not sure if that's a good fit.
Draft architecture
Use an Event Hub to listen for mobile deviceID registrations, and userIDs that requests for topic subscriptions .. Pass that to Reddis, below
If registering a phone/subscribing to a topic, save the deviceID userID to the topic key.
If sending a message to a topic, query Reddis for the topic key, and send that result to a FIFO queue for processing.
Pipe the output of the previous query into the built in Reddis Pub/Sub features to alert worker roles that there is work pending.
While the workers send notices to Apple and Firebase, archive out the sent notices to some in-memory store below.
Archive server maintains a history of sent events, so that out-of-sync devices can get the most up to date information LIFO-queue style.
Question
What are your thoughts on using this approach to solve the above needs?
What other things should I learn, research, or experiment (measure)?
I'm learning on how to get data from redis using seneca js but seneca provides multiple plugins to connect to redis. and available plugins are the ones mentioned in the title. which should I use just to fetch a couple of keys from redis? and what is the difference between the two?
seneca-redis-pubsub-transport and seneca-redis-queue-transport are both used for transporting messages between services using redis.
seneca-redis-pubsub-transport is a broadcast transport. All subscribed services will receive all messages. seneca-redis-queue-transport on the other hand is a queue transport. Messages are sent to only one of possibly multiple subscribed services.
If you only want to get/set some values that take a look at seneca-redis-store. This plugin allows you to get and set values using redis.
How long will messages published to a Redis channel stay there ? Also, is there is a way to configure the max. lifetime of a message per channel ? Is there a way to control the channel size or does the channel continues to store messages as long as the Redis server has free memory ?
Redis pub/sub doesn't persist published messages. What you are looking for seems more like a message queue which can be implemented using a combination of pub/sub and lists. For more information see pattern sections in RPOPLPUSH command.