This question already has an answer here:
Redis Database TTL
(1 answer)
Closed 6 years ago.
Does Redis have some default expire values? How does it get rid of old unused values? Does it do it at all? Or, does it just collect data until I delete it manually or until it runs out of memory? Can't find anything about it in docs and FAQ.
You can manually set time out for your keys. after the time out is reached, redis will automatically delete the key and its value.
see: https://redis.io/commands/expire
Related
in Redis you can configure the creation of snapshots, e.g. "save 60 10" would save the database after 60 seconds if at least 10 keys were changed.
If the SAME key was changed 10 times, would a snapshot be saved? Or does this refer to 10 unique/different keys that have to be changed?
Thank you!
The documented config doesn't say anything about "if at least 10 keys were changed". It says the snapshot will happen if "the given number of write operations against the DB occurred". Simple commands like SET and DEL count as one write operation. More complicated commands like HMSET and ZINTERSTORE might count as more than one write operation depending on the number of values they affect. Nothing takes into account the number of unique keys that were written to since the last snapshot.
I'm currently studying Redis, and have the following case:
So what I have is a sorted set by google place id, for which all posts are sorted from recent to older.
The first page that is requested fetches posts < current timestamp.
When a cursor is sent to the backend, this cursor is a simple timestamp that indicates from where to fetch the next posts from the ZSET.
The query to retrieve posts by place id would be:
ZREVRANGEBYSCORE <gplaceId> <cur_timestamp> -INF WITHSCORES LIMIT <offset:timestamp as from where to fetch> <count:number of posts>
My question is what is the recommended way to clean up members of the ZSET.
Since I want to use Redis as a cache, I would prefer to limit the number of posts per place for example up until 50. When places get new posts when already 50 posts have been added to the set, I want to drop the last post from the set.
Of course I realise that I could make a manual check on every insert and perform operations as such, but I wonder if Redis has a recommended way of performing this cleanup. Alternatively I could create a scheduler for this, but I would prefer not having to do this.
Unfortunately Redis sorted set do not come with an out of the box feature for this. If sorted sets allowed a max size attribute with a configurable eviction strategy - you could have avoided some extra work.
See this related question:
How to specify Redis Sorted Set a fixed size?
In absence of such a feature, the two approaches you mentioned are right.
You can replace inserts with a transaction : insert, check size, delete if > 50
A thread that checks size of the set and trims it periodically
Using Redis as cache service to cache some non-important data, and there is a case that need to update the value without reset or override the expire time, is there any good way to resolve this problem?
I've searched and found follows 2 solutions
Using setrange command, since the value is a little more complex, so it is not good in this situation.
Get the ttl time then set it as the expiration time when update the value. it's seems a little more redundant.
Any good idea to resolve this problem?
You don't need to do any of those two things. You just need to use the KEEPTTL flag when you set your value.
Like this:
> set my_key this_is_my_value EX 60
This will set a value for a key with 60 seconds expiration.
Then, when you change the value and don't want to change the expiration of your key, just do:
> set my_key this_is_my_new_value KEEPTTL
More information on REDIS docs.
Another idea to so resolve this could be using INCRBY.
For this you have to do some steps.
Get the existing value. For example, 10.
Determine the update value.For example, 17 .
INCRBY the value of their difference 17-10. That means, 7
This won't change the ttl
I want to flush out all keys older than 3 months. These keys were not set with an expire date.
Or if that is not possible, can I then delete maybe the oldest 1000 keys?
Using the object idletime you can delete all keys that have not been used since three months. It is not exactly what you ask. If you created a key 6 months ago, but the key is accessed everyday, then idletime is updated and this script will not delete it. I hope the script can help:
#!/usr/bin/env python
import redis
r = redis.StrictRedis(host='localhost', port=6379, db=0)
for key in r.scan_iter("*"):
idle = r.object("idletime", key)
# idle time is in seconds. This is 90days
if idle > 7776000:
r.delete(key)
Are you NOW using an expire? If so, you could loop through all keys if no TTL is set then add one.
Python example:
for key in redis.keys('*'):
if redis.ttl(key) == -1:
redis.expire(key, 60 * 60 * 24 * 7)
# This would clear them out in a week
EDIT
As #kouton pointed out use scan over keys in production, see a discussion on that at: SCAN vs KEYS performance in Redis
A bit late, but check out the OBJECT command. There you will find object idle time (with 10 second resolution). It's used for debugging purposes but still, could be a nice workaround for your need.
References: http://redis.io/commands/object
Sorry, that's not possible, as stated in the comments above. In Redis it's crucial to create your own indexes to support your access patterns.
Tip: What you should do is to create a sorted set (ZADD) with all new or modified keys, and set the score to a timestamp. This way you can with ease fetch the keys within a time period using ZRANGEBYSCORE.
If you want to expire your existing keys, get all keys (expensive) and set a TTL for each using the EXPIRE command.
Im currently having a redis data set with key representing ids and values as a json . I need to add a new entity in the json for every userid(keys). Is there any existing opensource tool? what is the way i should proceed to update for 1M keys of data.
There are a few possibilities:
Here's some pseudo code for doing this with Redis 2.6 Lua scripting.
for userid in users:
EVAL 'local obj = cjson.decode(redis.call("GET", KEY[1])); obj.subobj.newjsonkey = ARGV[1]; redis.call("SET", KEY[1], cjson.encode(obj));' 1 userid "new value!"
Short of that, you may need to stop the service and do this with GETs and SETs since you probably don't have a locking mechanism in place. If you can enforce a lock, see http://redis.io/commands/setnx
There are a few tools for updating an rdb. https://github.com/sripathikrishnan/redis-rdb-tools https://github.com/nrk/redis-rdb
Note, this answer was adapted to my answer to: Working with nested objects in Redis?