Could anyone tell me whats the maximum number of concurrent channels Redis pub-sub can support?. Is there any cap to the number of subscribers and publishers
Redis uses a dict, same struct as for keys, to store channel subscriptions, both per client and for all clients (keeps a per-subscription hash with a list of clients subscribed), so it is up to 2^32 channel subscriptions in total.
It uses a list to store pattern subscriptions per client, so it is theoretically limited only by the node memory available.
However, in general, you can have infinite channels. Think of a channel as a label when a message is published. Messages are never stored. When the message is published, Redis will look for the clients subscribed to that channel, and test for every pattern subscription. The channel really exists only while the message is being published.
As there are pattern subscriptions, there are unlimited 'logical' channels.
Just in events notifications we have 2^32 * databases * key event types possible 'logical' channels.
Regarding the number of subscribers and publishers, it is limited by the maxclients setting, 10,000 by default. There is no limitation for subscribers and publishers, but the maximum clients (connections) limit applies.
As indicated by #Roman, there are buffer limitations, but this refers mostly to throughput (message processing).
Pub/Sub clients have a default hard limit of 32 megabytes and a soft limit of 8 megabytes per 60 seconds.
Is that what you have been looking for?
documentation
Related
I am trying to create chat rooms app and use redis pub/sub broadcasting messages across subscribers. Each meeting room is 1 channel. The problem is I can have infinite number of channels over time.
Is there any limit for max number of channels I can have in redis?
Is there any way to auto expire/delete channels which have not received any messages for publishing?
Is there any limit for max number of channels I can have in redis?
Redis doesn't set any limit on the number of channels. Channels are saved into a hash, whose size is of type long. So in theory, the limit is 2**63 on a 64 bits machine. However, since your memory is limited, you cannot reach the limit. In a word, practically, there's no limit.
Is there any way to auto expire/delete channels which have not received any messages for publishing?
There's no way to do that. Redis deletes a channel, only when all clients of the channel have unsubscribed it, i.e. when a channel has no subscribers, Redis remove the channel automatically. If there's at least one client subscribing it, Redis keep the channel even if there's no message published to it.
We are using redis message bus and handling messages using a channel. But if our application is deployed in multiple instances then the request and response is passed to all the instances. To avoid this scenario which of the below approach is better?
Create a channel for each instance of the application
Create a channel for each user
Any suggestions will be highly appreciated
The limiting factor here is the number of subscribers to the same channel. Number of channels can be large as such. So you can choose the granularity accordingly. Read more here:
https://groups.google.com/forum/#!topic/redis-db/R09u__3Jzfk
All the complexity on the end is on the PUBLISH command, that performs
an amount of work that is proportional to:
a) The number of clients receiving the message.
b) The number of clients subscribed to a pattern, even if they'll not
match the message.
This means that if you have N clients subscribed to 100000 different
channels, everything will be super fast.
If you have instead 10000 clients subscribed to the same channel,
PUBLISH commands against this channel will be slow, and take maybe a
few milliseconds (not sure about the actual time taken). Since we have
to send the same message to everybody.
Similar question asked before : How does Redis PubSub subscribe mechanism works?
I have a requirement where I want to publish the data to more than 100,000 channels and the subscribers will subscribe to whatever channels they want to.
So I want to know is there a hard limit to the number of channels that can be available on Redis?
If I want to benchmark the performance of Redis with all there channels and the subscribers to the channels is there any available tool that I can use?
So I want to know is there a hard limit to the number of channels that can be available on Redis?
In practice, there's NO limit. You can have as many channels as possible, until the memory is full.
I want to create a Publish-Subscribe infrastructure in which every subscriber will listen to multiple (say 100k) channels.
I think to use Redis PubSub for that purpose but I'm not sure if subscribing to thousands of channels is the best practice here.
To answer this I want to know how subscribing mechanism in Redis works in the background.
Another option is to create a channel per subscriber and put some component in between, that will get all messages and publish it to relevant channels.
Any other Idea?
Salvatore/creator of Redis has answered this here: https://groups.google.com/forum/#!topic/redis-db/R09u__3Jzfk
All the complexity on the end is on the PUBLISH command, that performs
an amount of work that is proportional to:
a) The number of clients receiving the message.
b) The number of clients subscribed to a pattern, even if they'll not
match the message.
This means that if you have N clients subscribed to 100000 different
channels, everything will be super fast.
If you have instead 10000 clients subscribed to the same channel,
PUBLISH commands against this channel will be slow, and take maybe a
few milliseconds (not sure about the actual time taken). Since we have
to send the same message to everybody.
A recent upgrade to RabbitMQ server sees a change in the prefetch count reported for a consumer in the admin panel.
What is the difference between the channel prefetch count and consumer prefetch count as seen below?
We are running a set-up where we have multiple threads consuming off of a single consumer/channel. We allow the thread count and prefetch count to be adjusted on the fly. What combination of parameters to the Model.BasicQos(prefetchLength, prefetchCount, global); call on the c# client should we be using?
Basically prefetch in both cases means (maximum allowed) number of unacknowledged messages, per channel or per consumer. Check out here under title Channel Prefetch Setting (QoS). Important to note in case of channel prefetch (I'll just quote from aforementioned link)
Once the number reaches the configured count, RabbitMQ will stop
delivering more messages on the channel unless at least one of the
outstanding ones is acknowledged.
so no more messages to any of the consumers on that channel!
Nice examples on what numbers mean what and what happens when both of the prefetch values are set can be found here, so really what you need for the values in your case is arbitrary but obviously depends on various factors like: frequency of publishing, size of messages, number of consumers etc.
In the screenshot it seems that your consumer prefetch is bigger than channel prefetch. Now I don't know what the web management ui is reporting nor how do rabbitmq or the c# library handle this, but I would say that based on what is written in the documentation that consumer prefetch count can only be less or equal to channel prefetch count (I mean of course when it's actually used in runtime, not at declaring).