I have a use case in which I want to enable notification only for a certain set of keys, so that when those keys expire I can get a notification from redis.
I have followed this answer to implement this.
I have set parameter notify-keyspace-events to "Ex"
To accomplish this I am adding keys that I want notification for in DB-0 and the other keys in DB-1. But I am recieveing notification for both the DBs. Is there any way to just get notification from a particular DB?
According to redis documentation :
"Redis can notify Pub/Sub clients about events happening in the key space.
This feature is documented at http://redis.io/topics/notifications
For instance if keyspace events notification is enabled, and a client
performs a DEL operation on key "foo" stored in the Database 0, two
messages will be published via Pub/Sub:
PUBLISH keyspace#0:foo del
PUBLISH keyevent#0:del foo
"
But I am receiving notification from both DB-0 and DB-1.
PS : I know I can filter keys in my application, but I store too many expiring keys in redis and sending notification for all the expiring will increase load on my redis server.
I think you subscribed to a pattern that matches all DBs' notification message, e.g. PSUBSCRIBE __key*__:*.
In fact, you can specify the db index in the subscribed pattern: PSUBSCRIBE __keyspace#0__:* and PSUBSCRIBE __keyevent#0__:*. This way, you'll only received notification of DB0.
Related
I am working on a node JS app that connect to redis server and subscribe to a channel to get the messages.
There is a bit of confusion, should we really enable "key space notifications" on redis config to get the events in client
The same scenario I have tried using rdis cli, with which i see "key space notifications" are not enabled at the same time I have subscribed to a channel with a pattern, so when ever I publish a message from the other client, I am able to capture that event in subscribed client.
Is the "key space notifications" mandatory , but the POC says other way.
Does any one know what should be the right approach here, subscribing to channel is suffice to get messages, and its nothing to do with "key-space-notifications" ??
From Redis Keyspace Notifications
Keyspace notifications allow clients to subscribe to Pub/Sub channels in order to receive events affecting the Redis data set in some way.
Examples of events that can be received are:
All the commands affecting a given key.
All the keys receiving an LPUSH operation.
All the keys expiring in the database 0.
Events are delivered using the normal Pub/Sub layer of Redis, so clients implementing Pub/Sub are able to use this feature without modifications.
So, if you need just pub/sub, there is no need of extra configuration regarding Keyspace Notifications
The keyspace notifications have been essential for a recent web api I've been
developing.
We have redis setup in azure. The api mostly works, we use notifications to figure out if data on memory cache needs to be updated.
Right now, we want to handle notifying the flush event to clear local memory cache if redis database is flushed. But we can not get the flushdb event by Keyspace notification.
And the keyspace events is enable as "AKE". "AKE" string means all the events.
PS: we can get notification with 'set' event like '__keyevent#2__:set'
The subscription code is like below.
subscriber.Subscribe(
"*",
(channel, value) =>
{
// Some codes here
});
Just as the other answer mentioned, there's no such notification.
After all, Keyspace Notification is a notification for events on a single key. Each notification is associated with a key. For keyspace event, the key name is part of the channel name. For keyevent event, the key name is the message.
PUBLISH __keyspace#0__:key_name comamnd
PUBLISH __keyevent#0__:command key_name
Each command that sending a notification must have a key as argument. e.g. del key, set key val. However, the flushdb command has no key as argument. The command doesn't affect a single key. Instead, it removes all keys in the database. So there's no such notification for it. Otherwise, what do you expect from the channel? All keys that have been removed? It's not a good idea.
However, you can simulate an event for flushdb
set a special key, e.g. flushdb-event: set flushdb-event 0
subscribe on the corresponding channel: subscribe __keyspace#0__:flushdb-event
set the special key before you call flushdb: set flushdb-event 1
In this way, you can get the simulated flushdb notification.
According to Redis Documentation, there is no notification for Flushdb.
I think you have a couple of options.
You could call the INFO command periodically and check for a change in the number of flushdb or flushall calls. Here is the output from INFO that I am referring to...
Commandstats
cmdstat_flushall:calls=2,usec=112,usec_per_call=56.00
cmdstat_flushdb:calls=1,usec=110,usec_per_call=52.00
You could run the MONITOR command and parse the output. Note that this is not really a good option - it has heavy performance impact on the server side and would require a lot of processing on the client side.
We have a requirement that we need to get a notification on changes to a Redis data structure. Based on my research I found out that I can use Redis key space notifications for doing the same. However, Redis key space notifications send the events to Redis pub/sub channel which is fire and forget i.e once the clients lose the connections all the events till the connection is up again are lost.
Redis streams solve this problem. Also I want to use consumer group feature of Redis streams. So is there any way that Redis key space notifications can be pushed to Redis streams instead of Redis pub/sub channel?
The only way this can be done afaik with the current - Redis v5.0.3 - is to use the Modules API to develop a module that registers for keyspace notifications, processes them and adds the relevant messages into the Stream.
With RedisGears it's pretty simple to register a listener that will automatically write each event to a Stream.
e.g. the following register.py script will write for each HSET or HMSET call on person:* key prefix an event to mystream Stream.
register.py:
GearsBuilder() \
.foreach(lambda x: execute('XADD', "mystream", '*', *sum([[k,v] for k,v in x.items()],[]))) \
.register(prefix="person:*", eventTypes=['HSET', 'HMSET'])
To run it all you need to do is call:
$ gears-cli run register.py
We have some Redis keys with a given TTL that we would like to subscribe to and take action upon once the TTL expires (a la job scheduler).
This works well in a single-host environment, and when you subscribe in ServiceStack, using its Redis client, to '__keyspace#0__:expired', that service will pick it and take action. That's fantastic...
... until you have a high-availability topology set up, with more than one API instance in that cluster. Then every single host appears to be picking up on that message and potentially doing things with it.
I know keyspace notifications don't work exactly the same as traditional pub/sub or messaging-layer events, but is there a way to perform some kind of acknowledgement on these kinds of events, so that, at the end of the day, only one host will carry on with the task?
Otherwise, is there a way to delay a message publishing?
Thanks!
As describe in https://redis.io/topics/notifications
very node of a Redis cluster generates events about its own subset of the keyspace as described above. However, unlike regular Pub/Sub communication in a cluster, events' notifications are not broadcasted to all nodes. Put differently, keyspace events are node-specific. This means that to receive all keyspace events of a cluster, clients need to subscribe to each of the nodes.
So client should create separate connection to each node to get redis keyspace notification.
My understanding of your question: You need an event based unicast notification whenever a key is expired.
This solution will be helpful to you if above assumption is correct. It's kind of crude solution but works!
Solution:
You need to put(may be using a service/thread) the expired keys in the Redis List/queue. Then blocking B*POP operation from the client instances on this list/queue will give you what you want!
How does it work?
Let's assume, a single background thread will continuously push the expired keys into a redis list/queue. The cluster of API instances will be calling blocking pop on this list/queue.
Since, blocking pop operation on each item of redis list will be consumed by only one client, only one API instance will the get the notification of expired key!!!
Ref:
List pop operation: https://redis.io/commands/lpop
Similar problem with pub/sub: Competing Consumer on Redis Pub/Sub supported?
I'm using Redis as distributed cache. I have different applications which listen only particular keys. For example:
App1 listen App1.*
App2 listen App2.* and so on.
And my applications using following pattern to receive notifications:
App1: "key*:APP1."
App2: "key*:APP2."
I need to send notifications only about set, del, expired, evicted events that is why I have tried to use notify-keyspace-events "AK". If works fine for me but in this case of "AK" configuration redis starts to send extra notifications like "expire" which I don't need.
So according to documentation http://redis.io/topics/notifications I have tried to implement custom property:
notify-keyspace-events "Ksxe" to send only set, expired and evicted notifications. But in fact in this case I receive only expired notifications..
Questions:
1. Why I doesn't receive set and evicted events?? Why I receive only expired events?
2. Is there any way to make redis send only del notifications??
I also have tried "Ks" but redis doesn't send notifications about set events
A Alias for g$lshzxe, so that the "AKE" string means all the events.
"Kg$lshzxe" doesn't works correctly too..
I think you are misunderstanding the "s" flag. It has nothing to do with the set command. It tells Redis to only send commands that alter keys of the type "redis set" such as sadd or a key of the Redis set type expiring.
Thus, in your example "Ksxe" you instruct Redis to send you a notification anytime a key containing a Redis set is evicted or "expired". The "Ks" options instruct Redis to only send you notifications on keys of the type "set" being altered, not when a string is created using set command. To translate that to english, you told Redis "tell me when a key of type 'set' is expired or evicted".
If you want to know when a key of the type string is created or altered using the set command, has an expiration added to it, setting an expiration immediately deletes the key, or is evicted, you need to instead use "K$xeg". The "g" is important because it catches use of the expire command itself, and the '$' indicates the string type.
Also note that the "g" flag will result in "expire" events, but an expiration event will be labelled as "expired", enabling you to tell the difference. If you don't care about the creation of a TTL on a key, you can leave off the "g".