Is there a command in redis where I can set a default value for a key if it does not exist?
For example if get hello returns (nil) I would like to default it to world. But if the key hello already exists, I would like to return this value.
You can do it with a Lua script:
local value = redis.call("GET", KEYS[1])
if (not value) then
redis.call("SET", KEYS[1], ARGV[1])
return ARGV[1]
end
return value
Save this as script.lua and call it like this:
$ redis-cli eval "$(cat script.lua") 1 myKey defaultValue
You could also use SETNX to put the default value and then do a normal GET.
A simple Get should be the fastest thing to do
// practically Get would suffer a miss only the first time
value = Get key
if value not nil
return value
// Get did not find key => setNX
saved = SetNX key default
if saved
return default
// SetNX discovered key value was already set by someone else => Get it
return Get key
While it is possible as you can see in the other answers, be aware that depending on your use case, it may be faster to make 2 requests instead. For 2 reasons. The first is that it forces you to "compute" the output which is what you are trying to avoid with a cache in the first place. And second you will send a script AND the computed data with each requests which slows down traffic, where if your data stays the same for a while, you will only send a simple get request.
This is why a get-or-set is better if implemented in your app or at the driver level. I don't know the language you use, but here is a pseudo code version:
function get_or_set(key, callback_function)
value = REDIS.get(key)
if value.nil?
value = callback_function(REDIS) # pass REDIS to the anonymous function just in case
value_str = pack(value) # could be JSON or MsgPack
REDIS.set(key, value_str)
else
unpack(value)
end
end
This simplified because it only takes care of string (packing to make sure they are strings) and no expire is implemented, but you get the idea.
While it sounds overkill, it is how you get better performance. Otherwise it definitely would be a Redis feature already.
Related
I'm building a cache implementation in Java using Redisson. I want to use it to cache a numerical value. So I'm using getAtomicLong() like so:
RAtomicLong userNumber = redissonClient.getAtomicLong("my-key");
long value = userNumber.get();
However, the docs aren't very descriptive about what happens here, and so I have a few questions:
Assume that "my-key" does not yet exist in the cache. What does getAtomicLong() return?
If "my-key" does not exist, what does userNumber.get() return?
I wrote a silly Java example program and did some experimenting, so let me answer both the question you actually asked and then question I think you were trying to ask.
Assume that "my-key" does not yet exist in the cache. What does getAtomicLong() return?
An instance of RAtomicLong- because you might do something with the key/value in the future (like incrementAndGet or something).
If "my-key" does not exist, what does userNumber.get() return?
Zero. Yes, not null, yes not an exception, just 0. This was reasonably surprising in my test program.
The real interesting part about the Reddison API is that it leans hard on the atomic stuff - great if you could be updating a value from multiple threads - but seems to not document the simpliest use case: I want to read or write from Redis and it's not a number / not atomic / my threads or data is structured in such a way that they won't clobber each other.
That seems to be what Reddison's RBucket stuff is for.
That will return a null if the object is not in Redis.
RBucket<String> back = client.getBucket("foo");
String value = back.get();
if (value == null) {
System.out.println("NOPE, NULL");
} else {
System.out.println(value);
}
I really wish that would have been documented better - looking back I see "bucket" as a container for stuff, but for a while I assumed it meant some advanced Redis pattern and not "a holder for a value of a generic type".
(If you really do want the fancy stuff, there's an excellent baeldung artigle on it ).
I'm using Redis Key Space Notification to get my application notified when a specified key gets expired.
But when the key gets expired, Redis deletes the key, i need to remove this behavior because my application can use this expired information in another moment.
Is there a way to remove this behavior?
As #sazzad and #nitrin0 said, there's no way to change that.
As another option to get a similar result, I'd suggest you use a sorted set to track these "psuedo-expirations", and when they "expire", a background process does whatever else you need the key for: move it, transform it, reset the expiration, etc.
Use the command zadd to both create a new sorted set and to add members to it. The key for the set can be anything, but I'd use the members as the keys from the data that expires so you can easily work with both the real data, and the member in the sorted set.
ZADD name-of-sorted-set NX timestamp-when-data-expires key-of-real-data
Let's break this down:
name-of-sorted-set is what you'd use in the other Z* commands to work with this specific sorted set.
NX means "Only add new elements. Don't update already existing elements.". The other option is XX which is "Only update elements that already exist. Don't add new elements." For this, the only options are NX or nothing.
timestamp-when-data-expires is the score for this member, and we'll use it as the exact timestamp when the data would otherwise "expire", so you'll have to do some calculations in your application to provide the timestamp instead of just the seconds until it expires.
key-of-real-data is the exact key used for the real data this represents. Using the exact key here will help easily combine the two when you're working with this sorted set to find which members have "expired", since the members are the keys you'd use to move, delete, transform, the data.
Next I'd have a background process run zrangebyscore to see if there are any members whose scores (timestamps) are within some range:
ZRANGEBYSCORE name-of-sorted-set min-timestamp max-timestamp WITHSCORES LIMIT 0 10
Let's break this down too:
name-of-sorted-set is the key for the set we chose in ZADD above
min-timestamp is the lower end of the range to find members that have "expired"
max-timestamp is the higher end of the range
WITHSCORES tells Redis to return the name of the members AND their scores
LIMIT allows us to set an offset (the 0) and a count of items to return (the 10). This is just an example, but for very large data sets you'll likely have to make use of both the offset and count limits.
ZRANGEBYSCORE will return something like this if using redis-cli:
1) "first-member"
2) "1631648102"
3) "second-member"
4) "1631649154"
5) "third-member"
6) "1631650374"
7) "fourth-member"
8) "1631659171"
9) "fifth-member"
10) "1631659244"
Some Redis clients will change that, so you'll have to test it in your application. In redis-cli the member-score pair is returned over two lines.
Now that you have the members (keys of the actual data) that have "expired" you can do whatever it is you need to do with them, then probably either remove them from the set entirely, or remove them and replace them. Since in this example we created the sorted set with the NX example, we can't update existing records, only insert new ones.
When I execute a transaction (MULTI/EXEC) via SE.Redis, does it hit the server multiple times? For example,
ITransaction tran = Database.CreateTransaction();
tran.AddCondition(Condition.HashExists(cacheKey, oldKey));
HashEntry hashEntry = GetHashEntry(newKeyValuePair);
Task fieldDeleteTask = tran.HashDeleteAsync(cacheKey, oldKey);
Task hashSetTask = tran.HashSetAsync(cacheKey, new[] { hashEntry });
if (await tran.ExecuteAsync())
{
await fieldDeleteTask;
await hashSetTask;
}
Here I am executing two tasks in the transaction. Does this mean I hit the server 4 times? 1 for MULTI, 1 for delete, 1 for set, 1 for exec? Or is SE.Redis smart enough to buffer the tasks in local memory and send everything in one shot when we call ExecuteAsync?
It has to send multiple commands, but it doesn't pay latency costs per command; specifically, when you call Execute[Async] (and not before) it issues a pipeline (all together, not waiting for replies) of:
WATCH cacheKey // observes any competing changes to cacheKey
HEXIST cacheKey oldKey // see if the existing field exists
MULTI // starts the transacted commands
HDEL cacheKey oldKey // delete the existing field
HSET cachKey newField newValue // assign the new field
then it pays latency costs to get the result from the HEXIST, because only when that is known can it decide whether to proceed with the transaction (issuing EXEC and checking the result - which can be negative if the WATCH detects a conflict), or whether to throw everything away (DISCARD).
So; either way 6 commands are going to be issued, but in terms of latency: you're paying for 2 round trips due to the need for a decision point before the final EXEC/DISCARD. In many cases, though, this can itself be further masked by the reality that the result of HEXIST could already be on the way back to you before we've even got as far as checking, especially if you have any non-trivial bandwidth, for example a large newValue.
However! As a general rule: anything you can do with redis MULTI/EXEC: can be done faster, more reliably, and with fewer bugs, by using a Lua script instead. It looks like what we're actually trying to do here is:
for the hash cacheKey, if (and only if) the field oldField exists: remove oldField and set newField to newValue
We can do this very simply in Lua, because Lua scripts are executed at the server from start to finish without interruption from competing connections. This means that we don't need to worry about things like atomicity i.e. other connections changing data that we're making decisions with. So:
var success = (bool)await db.ScriptEvaluateAsync(#"
if redis.call('hdel', KEYS[1], ARGV[1]) == 1 then
redis.call('hset', KEYS[1], ARGV[2], ARGV[3])
return true
else
return false
end
", new RedisKey[] { cacheKey }, new RedisValue[] { oldField, newField, newValue });
The verbatim string literal here is our Lua script, noting that we don't need to do a separate HEXISTS/HDEL any more - we can make our decision based on the result of the HDEL. Behind the scenes, the library performs SCRIPT LOAD operations as needed, so: if you are doing this lots of times, it doesn't need to send the script itself over the network more than once.
From the perspective of the client: you are now only paying a single latency fee, and we're not sending the same things repeatedly (the original code sent cacheKey four times, and oldKey twice).
(a note on the choice of KEYS vs ARGV: the distinction between keys and values is important for routing purposes, in particular on sharded environments such as redis-cluster; sharding is done based on the key, and the only key here is cacheKey; the field identifiers in hashes do not impact sharding, so for the purpose of routing they are values, not keys - and as such, you should convey them via ARGV, not KEYS; this won't impact you on redis-server, but on redis-cluster this difference is very important, as if you get it wrong: the server will most-likely reject your script, thinking that you are attempting a cross-slot operation; multi-key commands on redis-cluster are only supported when all the keys are on the same slot, usually achieved via "hash tags")
Let suppose I want to pop 3 elements from the set, how I ensure that only pop if 3 elements present in a set otherwise return any error or other msg
How to use "spop" command with "count" argument.
What you want is to call SCARD myKey to test the number of members, and based on the result call SPOP.
SPOP with COUNT will return up to COUNT members, meaning if your set only has one or two, they'll be SPOPed and returned.
You probably want to do this with one atomic operation. So you have to use Lua Scrips:
EVAL "if redis.call('SCARD', KEYS[1]) >= tonumber(ARGV[1]) then return redis.call('SPOP', KEYS[1], ARGV[1]) else return redis.error_reply(KEYS[1]..' does NOT have at least '..ARGV[1]..' members') end" 1 myKey myNumber
Let's take a look at the script:
if redis.call('SCARD', KEYS[1]) >= tonumber(ARGV[1]) then
return redis.call('SPOP', KEYS[1], ARGV[1])
else
return redis.error_reply(KEYS[1]..' does NOT have at least '..ARGV[1]..' members')
end
KEYS[1] refers to the key parameter, the set you're interested in. It is important to pass keys through parameters for your script to be supported in a Redis Cluster.
ARGV[1] is an additional argument to pass your number of desired members, in your question, it is 3.
The script is run atomically server-side within Redis, and it is compiled only once as Redis caches it internally.
You can use SCRIPT LOAD to load the script and then reuse it with EVALSHA, to also improve networking performance.
Is there good way to support pop members from the Redis Sorted Set just like the api LPOP of the List ?
What I figured out for poping message from the Redis Sorted Set is using ZRANGE +ZREM , however it is not thread security and need the distributed lock when multi threads accessing them at the same time from the different host.
Please kind suggesting if there is better way to pop the members from the Sorted Set?
In Redis 5.0 or above, you can use [B]ZPOP{MIN|MAX} key [count] for this scenario.
The MIN version takes the item(s) with the lowest scores; MAX takes the item(s) with the highest scores. count defaults to 1, and the B prefix blocks until the data is available.
ZPOPMIN
ZPOPMAX
BZPOPMIN
BZPOPMAX
You can write a Lua script to do the job: wrap these two commands in a single Lua script. Redis ensures that the Lua script runs in an atomic way.
local key = KEYS[1]
local result = redis.call('ZRANGE', key, 0, 0)
local member = result[1]
if member then
redis.call('ZREM', key, member)
return member
else
return nil
end