I'm working with redis and am incrementing a key by 1, however I want to keep the associated TTL with that key when I increment it, unless it doesn't have a TTL, in which case I want to set it to 1.
I've read that there's a KEEPTTL property on the SET command, and that operations such as SET that change a value will set the TTL to either persist if one is not defined in the SET command, or to whatever it is set too. And hence I assumed that INCR does the same.
Related
I'm using Redis Key Space Notification to get my application notified when a specified key gets expired.
But when the key gets expired, Redis deletes the key, i need to remove this behavior because my application can use this expired information in another moment.
Is there a way to remove this behavior?
As #sazzad and #nitrin0 said, there's no way to change that.
As another option to get a similar result, I'd suggest you use a sorted set to track these "psuedo-expirations", and when they "expire", a background process does whatever else you need the key for: move it, transform it, reset the expiration, etc.
Use the command zadd to both create a new sorted set and to add members to it. The key for the set can be anything, but I'd use the members as the keys from the data that expires so you can easily work with both the real data, and the member in the sorted set.
ZADD name-of-sorted-set NX timestamp-when-data-expires key-of-real-data
Let's break this down:
name-of-sorted-set is what you'd use in the other Z* commands to work with this specific sorted set.
NX means "Only add new elements. Don't update already existing elements.". The other option is XX which is "Only update elements that already exist. Don't add new elements." For this, the only options are NX or nothing.
timestamp-when-data-expires is the score for this member, and we'll use it as the exact timestamp when the data would otherwise "expire", so you'll have to do some calculations in your application to provide the timestamp instead of just the seconds until it expires.
key-of-real-data is the exact key used for the real data this represents. Using the exact key here will help easily combine the two when you're working with this sorted set to find which members have "expired", since the members are the keys you'd use to move, delete, transform, the data.
Next I'd have a background process run zrangebyscore to see if there are any members whose scores (timestamps) are within some range:
ZRANGEBYSCORE name-of-sorted-set min-timestamp max-timestamp WITHSCORES LIMIT 0 10
Let's break this down too:
name-of-sorted-set is the key for the set we chose in ZADD above
min-timestamp is the lower end of the range to find members that have "expired"
max-timestamp is the higher end of the range
WITHSCORES tells Redis to return the name of the members AND their scores
LIMIT allows us to set an offset (the 0) and a count of items to return (the 10). This is just an example, but for very large data sets you'll likely have to make use of both the offset and count limits.
ZRANGEBYSCORE will return something like this if using redis-cli:
1) "first-member"
2) "1631648102"
3) "second-member"
4) "1631649154"
5) "third-member"
6) "1631650374"
7) "fourth-member"
8) "1631659171"
9) "fifth-member"
10) "1631659244"
Some Redis clients will change that, so you'll have to test it in your application. In redis-cli the member-score pair is returned over two lines.
Now that you have the members (keys of the actual data) that have "expired" you can do whatever it is you need to do with them, then probably either remove them from the set entirely, or remove them and replace them. Since in this example we created the sorted set with the NX example, we can't update existing records, only insert new ones.
I have a data buffer that I would want to set/store into a redis DB to be reused/get by downstream modules. I have a pointer to the data buffer and I would want to emulate some kind of memcpy directly from the buffer pointer into the redis key value and I know the exact length of the data I want to copy.
I can do this in 2 stages
1) Fwrite the buffer into a file say buffer.bin for the size data length
2) Simulate 'redis-cli -x set buffer1
I confirmed I can get the file contents back with
redis-cli -x get buffer1 > /home/buffer-copy.bin
But I would want to avoid additional file operation which I see as completely redundant/costly operation if I can save from my memory pointer directly into the redis key-value. Can you please share your thoughts on how I can do this?
Edit: trying to use "C" hiredis interfaces to access redis
Is there good way to support pop members from the Redis Sorted Set just like the api LPOP of the List ?
What I figured out for poping message from the Redis Sorted Set is using ZRANGE +ZREM , however it is not thread security and need the distributed lock when multi threads accessing them at the same time from the different host.
Please kind suggesting if there is better way to pop the members from the Sorted Set?
In Redis 5.0 or above, you can use [B]ZPOP{MIN|MAX} key [count] for this scenario.
The MIN version takes the item(s) with the lowest scores; MAX takes the item(s) with the highest scores. count defaults to 1, and the B prefix blocks until the data is available.
ZPOPMIN
ZPOPMAX
BZPOPMIN
BZPOPMAX
You can write a Lua script to do the job: wrap these two commands in a single Lua script. Redis ensures that the Lua script runs in an atomic way.
local key = KEYS[1]
local result = redis.call('ZRANGE', key, 0, 0)
local member = result[1]
if member then
redis.call('ZREM', key, member)
return member
else
return nil
end
Is there a command in redis where I can set a default value for a key if it does not exist?
For example if get hello returns (nil) I would like to default it to world. But if the key hello already exists, I would like to return this value.
You can do it with a Lua script:
local value = redis.call("GET", KEYS[1])
if (not value) then
redis.call("SET", KEYS[1], ARGV[1])
return ARGV[1]
end
return value
Save this as script.lua and call it like this:
$ redis-cli eval "$(cat script.lua") 1 myKey defaultValue
You could also use SETNX to put the default value and then do a normal GET.
A simple Get should be the fastest thing to do
// practically Get would suffer a miss only the first time
value = Get key
if value not nil
return value
// Get did not find key => setNX
saved = SetNX key default
if saved
return default
// SetNX discovered key value was already set by someone else => Get it
return Get key
While it is possible as you can see in the other answers, be aware that depending on your use case, it may be faster to make 2 requests instead. For 2 reasons. The first is that it forces you to "compute" the output which is what you are trying to avoid with a cache in the first place. And second you will send a script AND the computed data with each requests which slows down traffic, where if your data stays the same for a while, you will only send a simple get request.
This is why a get-or-set is better if implemented in your app or at the driver level. I don't know the language you use, but here is a pseudo code version:
function get_or_set(key, callback_function)
value = REDIS.get(key)
if value.nil?
value = callback_function(REDIS) # pass REDIS to the anonymous function just in case
value_str = pack(value) # could be JSON or MsgPack
REDIS.set(key, value_str)
else
unpack(value)
end
end
This simplified because it only takes care of string (packing to make sure they are strings) and no expire is implemented, but you get the idea.
While it sounds overkill, it is how you get better performance. Otherwise it definitely would be a Redis feature already.
Simple enough, I am using Redis INCR to ensure atomic increments of a counter, the counter has an indeterminate start value less than Int64.MaxValue.
Does Redis reset the value when it reaches Int64.MaxValue or throw an error?
I read the documentation but it does not say what happens, and I really do want to maintain the atomic nature at rollover
It will throw an error. I did a small experiment for your use case
127.0.0.1:6379> set value 9223372036854775807 (2 power 63 -1)
OK
127.0.0.1:6379> incr value
(error) ERR increment or decrement would overflow
127.0.0.1:6379>
Redis can hold upto 2 power 63. and throws error if it exceeds that limit. It might be either "out of range" error or "overflow" error
Upon error you can catch that exception and reset the value in your application logic.