I want to flush out all keys older than 3 months. These keys were not set with an expire date.
Or if that is not possible, can I then delete maybe the oldest 1000 keys?
Using the object idletime you can delete all keys that have not been used since three months. It is not exactly what you ask. If you created a key 6 months ago, but the key is accessed everyday, then idletime is updated and this script will not delete it. I hope the script can help:
#!/usr/bin/env python
import redis
r = redis.StrictRedis(host='localhost', port=6379, db=0)
for key in r.scan_iter("*"):
idle = r.object("idletime", key)
# idle time is in seconds. This is 90days
if idle > 7776000:
r.delete(key)
Are you NOW using an expire? If so, you could loop through all keys if no TTL is set then add one.
Python example:
for key in redis.keys('*'):
if redis.ttl(key) == -1:
redis.expire(key, 60 * 60 * 24 * 7)
# This would clear them out in a week
EDIT
As #kouton pointed out use scan over keys in production, see a discussion on that at: SCAN vs KEYS performance in Redis
A bit late, but check out the OBJECT command. There you will find object idle time (with 10 second resolution). It's used for debugging purposes but still, could be a nice workaround for your need.
References: http://redis.io/commands/object
Sorry, that's not possible, as stated in the comments above. In Redis it's crucial to create your own indexes to support your access patterns.
Tip: What you should do is to create a sorted set (ZADD) with all new or modified keys, and set the score to a timestamp. This way you can with ease fetch the keys within a time period using ZRANGEBYSCORE.
If you want to expire your existing keys, get all keys (expensive) and set a TTL for each using the EXPIRE command.
Related
I know there are several ways to set a specific ttl for a key, but is there a way to add some extra time for a key which has a counting down ttl?
There's no built-in way to extend TTL. You need to get the current TTL, and then add some more TTL to it.
Wrap these two steps into a Lua script:
-- extend 300 seconds
eval 'local ttl = redis.call("TTL", "key") + 300; redis.call("EXPIRE", "key", ttl)' 0
Good question
there is no such command
I think it is a bad idea to have a command like that, you have to be careful when you use it.
Probably end up adding more time to the ttl than we expect. If you set it like 5 mins, the actual expire time will be close to 5 mins even if setting it multiple times in that request. But if you add multiple 5 mins to it, then we can`t be sure of the actual expire time
in Redis you can configure the creation of snapshots, e.g. "save 60 10" would save the database after 60 seconds if at least 10 keys were changed.
If the SAME key was changed 10 times, would a snapshot be saved? Or does this refer to 10 unique/different keys that have to be changed?
Thank you!
The documented config doesn't say anything about "if at least 10 keys were changed". It says the snapshot will happen if "the given number of write operations against the DB occurred". Simple commands like SET and DEL count as one write operation. More complicated commands like HMSET and ZINTERSTORE might count as more than one write operation depending on the number of values they affect. Nothing takes into account the number of unique keys that were written to since the last snapshot.
Using Redis as cache service to cache some non-important data, and there is a case that need to update the value without reset or override the expire time, is there any good way to resolve this problem?
I've searched and found follows 2 solutions
Using setrange command, since the value is a little more complex, so it is not good in this situation.
Get the ttl time then set it as the expiration time when update the value. it's seems a little more redundant.
Any good idea to resolve this problem?
You don't need to do any of those two things. You just need to use the KEEPTTL flag when you set your value.
Like this:
> set my_key this_is_my_value EX 60
This will set a value for a key with 60 seconds expiration.
Then, when you change the value and don't want to change the expiration of your key, just do:
> set my_key this_is_my_new_value KEEPTTL
More information on REDIS docs.
Another idea to so resolve this could be using INCRBY.
For this you have to do some steps.
Get the existing value. For example, 10.
Determine the update value.For example, 17 .
INCRBY the value of their difference 17-10. That means, 7
This won't change the ttl
I have case where i need to save values to redis SET structure under given key so i am using command from my code in the loop
SADD key value
EXPIRE KEY 100
However, i would like to set expiration time only on first save of the set key.
Is it possible to set expire time only at the moment of first set key creation ?
it is also should be noted that i can use EXISTS key call to redis to check if key exist and depending on that set expiration time or not - but this operation is not atomic.
To answer the question - no, there isn't such a command.
As you noted, this could be worked around with EXISTS. To address the atomicity (and save on network) requirement you can use a Lua script (see EVAL).
Redis 7.0 has a new option "NX -- Set expiry only when the key has no expiry". So, you could solve the problem with EXPIRE KEY 100 NX.
However, that it doesn't matter if you call EXPIRE KEY 100 1 or, let's say, 5 times. The key will expire after 100 seconds, thus checking EXISTS key is also an option.
I got a quick simple question,
Assume that if server receives 10 messages from user within 10 minutes, server sends a push email.
At first I thought it very simple using redis,
incr("foo"), expire("foo",60*10)
and in Java, handle the occurrence count like below
if(jedis.get("foo")>=10){sendEmail();jedis.del("foo");}
but imagine if user send one message at first minute and send 8 messages at 10th minute.
and the key expires, and user again send 3 messages in the next minute.
redis key will be created again with value 3 which will not trigger sendEmail() even though user send 11 messages in 2 minutes actually.
we're gonna use Redis and we don't want to put receive time values to redis.
is there any solution ?
So, there's 2 ways of solving this-- one to optimize on space and the other to optimize on speed (though really the speed difference should be marginal).
Optimizing for Space:
Keep up to 9 different counters; foo1 ... foo9. Basically, we'll keep one counter for each of the possible up to 9 different messages before we email the user, and let each one expire as it hits the 10 minute mark. This will work like a circular queue. Now do this (in Python for simplicity, assuming we have a connection to Redis called r):
new_created = False
for i in xrange(1,10):
var_name = 'foo%d' % i
if not (new_created or r.exists(var_name)):
r.set(var_name, 0)
r.expire(var_name, 600)
new_created = True
if not r.exists(var_name): continue
r.incr(var_name, 1)
if r.get(var_name) >= 10:
send_email(user)
r.del(var_name)
If you go with this approach, put the above logic in a Lua script instead of the example Python, and it should be quite fast. Since you'll at most be storing 9 counters per user, it'll also be quite space efficient.
Optimizing for speed:
Keep one Redis Sortet Set per user. Every time a user sends a message, add to his sorted set with a key equal to the timestamp and an arbitrary value. Then just do a ZCOUNT(now, now - 10 minutes) and send an email if that's greater than 10. Then ZREMRANGEBYSCORE(now - 10 minutes, inf). I know you said you didn't want to keep timestamps in Redis, but IMO this is a better solution, and you're going to have to hold some variant on timestamps somewhere no matter what.
Personally I'd go with the latter approach because the space differences are probably not that big, and the code can be done quickly in pure Redis, but up to you.