Let's say I have two Redis keys
1) Test1 -> expiry/ttl after 2 days
2) Test2 -> expiry/ttl after 1 day
3) Test3 -> expiry after 1month
Is it possible to just query keys by TTL or expiry time?
I could have read the keys & compare the ttl but that would be very expensive operation.
Related
For example, assume all operation is fast within 1 sec.
I accept solution with multiple step
but it should be atomic
and perofrmance is key for me
like it will update 1000 times in a sec
TTL mykey <- 10
INCRTTL mykey 5 // I need something like this
TTL mykey <- 15
TTL mykey <- 10
INCRTTL mykey 5
INCRTTL mykey 5
INCRTTL mykey 5
TTL mykey <- 25
I create Redis (v7) Sorted Sets with ZADD. Each set has multiple members. The score I use is a timestamp.
With ZRANGE I'm able to select members with scores between min and max. I want to select - based on one timestamp - one member after the timestamp and one before the timestamp.
Example (score, member) for one key:
(10, A)
(15, B)
(20, C)
I.e. SELECT 17 should give me (15, B) and (20, C).
Looks like Redis has no command to do this. What would be the most efficient way to do that?
You need two commands:
# get the one after the timestamp
zrangebyscore k (17 +inf limit 0 1
# get the one before the timestamp
zrevrangebyscore k (17 -inf limit 0 1
In order to be more efficient and atomic, wrap these two commands into a Lua script.
I want to use only 1 key value pair in Redis database. and the value will be decreased by 1 in every 60 seconds. Is it possible?
An interesting question :) Yes, you can do that with a trick.
As we known Redis TTL can be decreased automatically over time. So you can use TTL as the value, and the TTL will decrease by 1 every second.
Say, you want to set a value of N, in order to achieve your goal, you can set a key-value pair with expiration TTL = 60 * N:
SET key N EX TTL
When you want to get the value, just get its TTL, and do some math:
ttl = TTL key
if (ttl > 0)
value = ttl / 60 + 1
else
// no longer exist
I have a table in sql with 3 columns: BIGINT StartNumber, BIGINT EndNumber, BIGINT LocationId, and I need to be able to do something like this
Select LocationId where StartNumber < #number and EndNumber > #number.
for example:
StartNumber EndNumber LocationId
1 5 1
6 9 1
10 16 2
and when I have #number = 7 I should get LocationId = 1
How can I do this in redis?
I was thinking to move this table to redis, use sorted set and ZRANGEBYSCORE but it did't work for me:
1) When I am using ZADD key score member [score] [member], I am unable to add 2 elements with the same member and different score even with nx parameter:
zadd myset nx 1 "17" 2 "17" - it will add one element and then update its score instead of adding two elements.
2) when I am adding this: zadd set1 2 "a" 4 "b" 6 "c" 10 "d" and then trying to do zrangebyscore set1 3 3 (want to get member whose score include 3) I em getting empty result
P.s. All commands are executed on the example pages of redis website.
So as I understood the task, you don't have overlaps and each interval maps to only one location (?) and intervals don't have gaps. Based on this you can use only one sorted list with lower (or upper) bound values:
ZADD StartNumber 1 "1:5:1" 6 "6:9:1" 10 "10:16:2"
Then you can use:
ZREVRANGEBYSCORE StartNumber 7 -inf LIMIT 0 1
And it will be O(log(N)).
Put differently, your question is "how can I map N ranges of numbers to a location". One way of doing this is using two Sorted Sets, one for the StartNumber and the other one for EndNumber. Since members have to be unique, we'll also need to ensure that by using the Start/End values as part of the member. For example, with your example data, this could be done like so:
ZADD StartNumber 1 "1:5:1" 6 "6:9:1" 10 "10:16:2"
ZADD EndNumber 5 "1:5:1" 9 "6:9:1" 16 "10:16:2"
To find the location for #number=7, do ZRANGEBYSCORE StartNumber -inf 7 and ZRANGEBYSCORE EndNumber 7 +inf and intersect the results. All that remains is to split the intesect's result(s) on the colon (:) and use the 3rd element as the location.
Note: if your app ensures that there are no overlapping ranges and that there can be only one location per "number", you can get the same results with only one set.
(this is the first time that I'm giving two answers to the same question - maybe I'll get a badge or sumthin' ;))
The double Sorted Set approach is a generalization and, as such, aims to solve a bigger set of problems than what the OP needs (as put in the comments to the first answer). That approach is also not effective as the query is O(logn)+O(N) so when N is large (e.g. 5M) that's probably not a good idea.
However, to satisfy the requirements and given that the ranges do not overlap, one could actually use only a single Sorted Set and a simpler query. The set's members should be added by concatenating the EndNumber and LocationId and the their scores should be set to their respective StartNumber, so for the sake of the example:
ZADD ranges 1 "5:1" 6 "9:1" 10 "16:2"
Given #number, obtain the relevant LocationId with the following Redis Lua code (O(logn)):
-- rangelookup.lua
-- http://stackoverflow.com/questions/32185898/redis-get-member-where-score-is-between-min-and-max/32186675
-- A **non inclusive** range search on a Sorted Set with the following data:
-- score = <StartNumber>
-- member = <EndNumber>:<LocationId>
--
-- KEYS[1] - Sorted Set key name
-- ARGV[1] - the number to search
--
-- reply - the relevant id, nil if range doesn't exist
--
-- usage example: redis-cli --eval rangelookup.lua ranges , 7
local number = tonumber(ARGV[1])
local data = redis.call('ZREVRANGEBYSCORE', KEYS[1], number, '-inf', 'WITHSCORES', 'LIMIT', 0, 1)
local reply = nil
if data ~= nil and number > tonumber(data[2]) then
local to, id = data[1]:match( '(.*):(.*)' )
if tonumber(to) > number then
reply = id
end
end
return reply
Sample output:
$ redis-cli --eval rangelookup.lua ranges , 7
"1"
$ redis-cli --eval rangelookup.lua ranges , 9
(nil)
$ redis-cli --eval rangelookup.lua ranges , 99
(nil)
this is probably something idiotic..
doing this in redis console
zincrby model 1 20140101
zincrby model 1 20141010
zincrby model 1 20141010
why does this work
zrangebyscore model 00000000 99999999 withscores
1) "20140101"
2) "1"
3) "20141010"
4) "2"
but this doesn't
zrangebyscore model 20140000 20149999 withscores
#> (empty list or set)
ZRANGEBYSCORE is for score ranges lookups and you're using your members instead (in the 3rd snippet). Since 1, 2 << 20140000, 20149999 you're getting nothing back.
EDIT after some comments back n forth
Generally, you need to make a decision about the space/time tradeoff, i.e. more RAM and less CPU or vice versa, and that actually depends on your performance & data size requirements. Usually I'd try to use a sorted set for each model/event that being tracked per aggregation level needed. Key expiration is useful, but sometimes manually removing members from sorted sets is also needed.
IIUC, you only need per-model daily counters so following your initial design, my "schema" would probably be:
Sorted set key name pattern: <model>:daily
|
+- Member value: <day timestamp at 12AM UTC>
+- Member score: <count>
Use ZINCRBY to increment today's hits:
ZINCRBY <model>:daily 1 <today's timestamp at 12AM UTC>
Get a date's hits:
ZSCORE <model>:daily <date timestamp at 12AM UTC>
Notes:
You can't easily do date ranges with this approach as your scores keep the counts. You'll basically need to do multiple ZSCOREs (O(log(N))), looping through each date in the range.
You can keep additional rolling or static aggregates to speed up commonly accessed ranges.
You'll have to manually "expire" the older set members for housekeeping.
An alternative approach that allows ranges is to have the following in place:
Sorted set key name pattern: <model>:daily
|
+- Member value: <day timestamp at 12AM UTC>:<count>
+- Member score: 0
Here, you can use ZRANGEBYLEX to get a range of dates, but since the timestamp and count are concatenated you'll have to do a little processing client-side or with Lua to get the count (ZSCORE will always return 0) or to increment it (you can't use ZINCRBY anymore).