In terms of functionality, i can work around this limitation, but want to understand whether there an architectural reason why it is not supported out of the box?
PS: (this is not a question).
I have work around e.g.: call EXPIRE individually for each key which I am setting with mset. I don't know if that is any better than calling set individually for each key (in a pipeline if transaction does not matter), as set allows to specify TTL.
For anyone wondering, i did some benchmarks and calling set individually for each key is always faster. By approximately 15% for 2 keys and approximately 7% for 200000 keys. The benchmark was made with Go though and Redis calls were parallelized.
Related
Is there an advantage to set a default value for an entry that will be heavily queried in Redis or will querying for the unset key take the same time?
Given the keys are stored in a distributed hash, it will have to check that the key is not in the bucket before returning on a miss, which may be a bit slower than finding and stopping at a hit. Is the bucket sorted of linear? Does anything else make it slower either way?
Redis is setup in a cluster and has many million entries in this case.
I'm assuming you're just talking about strings & hashes here here (so the only operations you care about are set/get, maybe hget/hset) - From Redis' perspective, a cache hit and cache miss have the same time complexity, if anything, a cache miss will be faster because redis will not have to transfer any data back over the socket to your app.
I will be storing key-value pairs in Redis but the number of keys will be just 4. Since there will be multiple processes updating the values parallelly, I plan to use Redis transactions using WATCH, MULTI and EXEC commands.
My algorithm is something like this:
GET key
WATCH key
MULTI
SET key new_val
EXEC
My main concern is that, since WATCH uses optimistic locking, when I will have multiple processes (much more than the number of keys, which are only 4) trying to update values, the transaction failure rate will be very high.
Is this correct? Is there a way to prevent this?
It is unclear why you'd need a transaction here unless you're doing something in your application with the reply for GET key. I'll therefore assume that you are using the value for something meaningful, otherwise, you can drop the transaction semantics and just call SET key new_val.
Optimistic locking is mainly intended to be used in cases where there's low contention for the resources. Since the use case that you're describing is clearly the opposite, it would probably result in high failure rates. This isn't saying that Redis and your application will not work, but it does mean there a potential for a lot of wasted effort.
I would advise that you consider switching if possible to using Redis' server-side Lua scripts. These are blocking, atomic, and let you read and meaningfully manipulate the data in Redis programmatically. See the EVAL command for details.
I have a Redis cluster that is still active and not sure if it is still been used. Logged in and tried command KEYS * and found around 30k+ keys.
How to identify if it is still been used if not can process to terminate?
First of all, you shouldn't use KEYS command in production, it can jam your Redis. Use SCAN instead.
If you mean you would like to know if some queries are still being made against Redis, you can use the Monitor command. Be careful though, it reduces your throughput by about 50%, so only use when necessary.
I want to make the key live longer if some condition met.
What I think is if I can query all the keys close to expire (like 10 minutes to expire), then I can do the query-check-activate intervally.
I seached but not found any clue, if you know how to achieve it, please tell me.
Any help will be highly appreciated!
Querying for keys based on their expiration time is not supported by Redis at the moment.
You can work around this by not using Redis' built-in expiration and managing it yourself, e.g. by using a Sorted Set to track expiration times and implementing passive and active expiration in your application.
Alternatively, you can use SCAN to crawl the keyspace, fetching TTLs and performing your evaluations.
I'm using MSETNX (http://redis.io/commands/msetnx) as a locking system, whereby all keys are locked only if no locks already exist.
If a machine holding a lock dies, that lock will be stuck locked - this is a problem.
My ideal answer would be that all keys expire in 15 seconds by default, so even if a machine dies it's held locks will auto-reset in a short time. This way I don't have to call expire on every key I set.
Is this possible in any way?
To build a reliable lock that is high available please check this document: http://redis.io/topics/distlock
The algorithm is still in beta but was stress-tested in a few sessions and is likely to be far more reliable than a single-instance approach anyway.
There are reference implementations for a few languages (linked in the doc).
Redis doesn't have a built-in way to do MSETNX and expire all keys together atomically. Nor can you set a default expiry tube for keys.
You could consider instead:
1. Using a WATCH/MULTI/EXEC block that wraps multiple 'SET key value EX 15 NX', or
2. Doing this using a Lua server-side script.