Redis get member where score is between min and max - redis

I have a table in sql with 3 columns: BIGINT StartNumber, BIGINT EndNumber, BIGINT LocationId, and I need to be able to do something like this
Select LocationId where StartNumber < #number and EndNumber > #number.
for example:
StartNumber EndNumber LocationId
1 5 1
6 9 1
10 16 2
and when I have #number = 7 I should get LocationId = 1
How can I do this in redis?
I was thinking to move this table to redis, use sorted set and ZRANGEBYSCORE but it did't work for me:
1) When I am using ZADD key score member [score] [member], I am unable to add 2 elements with the same member and different score even with nx parameter:
zadd myset nx 1 "17" 2 "17" - it will add one element and then update its score instead of adding two elements.
2) when I am adding this: zadd set1 2 "a" 4 "b" 6 "c" 10 "d" and then trying to do zrangebyscore set1 3 3 (want to get member whose score include 3) I em getting empty result
P.s. All commands are executed on the example pages of redis website.

So as I understood the task, you don't have overlaps and each interval maps to only one location (?) and intervals don't have gaps. Based on this you can use only one sorted list with lower (or upper) bound values:
ZADD StartNumber 1 "1:5:1" 6 "6:9:1" 10 "10:16:2"
Then you can use:
ZREVRANGEBYSCORE StartNumber 7 -inf LIMIT 0 1
And it will be O(log(N)).

Put differently, your question is "how can I map N ranges of numbers to a location". One way of doing this is using two Sorted Sets, one for the StartNumber and the other one for EndNumber. Since members have to be unique, we'll also need to ensure that by using the Start/End values as part of the member. For example, with your example data, this could be done like so:
ZADD StartNumber 1 "1:5:1" 6 "6:9:1" 10 "10:16:2"
ZADD EndNumber 5 "1:5:1" 9 "6:9:1" 16 "10:16:2"
To find the location for #number=7, do ZRANGEBYSCORE StartNumber -inf 7 and ZRANGEBYSCORE EndNumber 7 +inf and intersect the results. All that remains is to split the intesect's result(s) on the colon (:) and use the 3rd element as the location.
Note: if your app ensures that there are no overlapping ranges and that there can be only one location per "number", you can get the same results with only one set.

(this is the first time that I'm giving two answers to the same question - maybe I'll get a badge or sumthin' ;))
The double Sorted Set approach is a generalization and, as such, aims to solve a bigger set of problems than what the OP needs (as put in the comments to the first answer). That approach is also not effective as the query is O(logn)+O(N) so when N is large (e.g. 5M) that's probably not a good idea.
However, to satisfy the requirements and given that the ranges do not overlap, one could actually use only a single Sorted Set and a simpler query. The set's members should be added by concatenating the EndNumber and LocationId and the their scores should be set to their respective StartNumber, so for the sake of the example:
ZADD ranges 1 "5:1" 6 "9:1" 10 "16:2"
Given #number, obtain the relevant LocationId with the following Redis Lua code (O(logn)):
-- rangelookup.lua
-- http://stackoverflow.com/questions/32185898/redis-get-member-where-score-is-between-min-and-max/32186675
-- A **non inclusive** range search on a Sorted Set with the following data:
-- score = <StartNumber>
-- member = <EndNumber>:<LocationId>
--
-- KEYS[1] - Sorted Set key name
-- ARGV[1] - the number to search
--
-- reply - the relevant id, nil if range doesn't exist
--
-- usage example: redis-cli --eval rangelookup.lua ranges , 7
local number = tonumber(ARGV[1])
local data = redis.call('ZREVRANGEBYSCORE', KEYS[1], number, '-inf', 'WITHSCORES', 'LIMIT', 0, 1)
local reply = nil
if data ~= nil and number > tonumber(data[2]) then
local to, id = data[1]:match( '(.*):(.*)' )
if tonumber(to) > number then
reply = id
end
end
return reply
Sample output:
$ redis-cli --eval rangelookup.lua ranges , 7
"1"
$ redis-cli --eval rangelookup.lua ranges , 9
(nil)
$ redis-cli --eval rangelookup.lua ranges , 99
(nil)

Related

Redis sorted set leader board ranking on same score

I'm using Redis sorted set to implement the leaderboard of my game, where I show the user ranking in descending order. I'm stuck in a case where two or more users have the same score. So in this case, I want the higher ranking of the user who gets the score first. For example, I'm adding the following entries in Redis.
127.0.0.1:6379> zadd testing-key 5 a
(integer) 1
127.0.0.1:6379> zadd testing-key 4 b
(integer) 1
127.0.0.1:6379> zadd testing-key 5 c
(integer) 1
and when I'm querying for the rank in reverse order, I'm getting this
127.0.0.1:6379> zrevrange testing-key 0 10
1) "c"
2) "a"
3) "b"
but in my case, the ranking should be like
1) "a"
2) "c"
3) "b"
So is there any provision in Redis to give higher precedence to the entity which entered first in the set with the same score?
I found one solution to this problem. In my case, the score is an integer so I converted it into decimal and added Long.MAX_VALUE - System.nanoTime() after decimal. So the final score code will be like
double finalScore = score.(Long.MAX_VALUE - System.nanoTime());
So the final score of the player who scored first would be higher than the second one. Please let me know if you have any better solution.
If your leaderboard's scores are "small" enough, you may get away with using a combination of the score and the timestamp (e.g. 123.111455234, where 123 is the score). However, since the Sorted Set score is a double floating point, you may lose precision.
Alternatively, keep two Sorted Sets - one with each player's leaderboard score and the other with each player's score timestamp, and use both to determine the order.
Or, use a single sorted set for the leader board, encode the timestamp as part of the member and rely on lexicographical ordering.

Checking if IP falls within a range with Redis

I am interested in using Redis to check if a IP address (converted into integer) falls within a range of IPs. It is very likely that the ranges will overlap.
I have found this question/answer, although I am not able to fully understand the logic behind it.
Thank you for your help!
EDIT - Since I got a downvote (a comment to explain why would be nice), I've removed some clutter from my answer.
#DidierSpezia answer in your linked question is a good answer, but it becomes hard to maintain if you are adding/removing ranges.
However it is not trivial (and expensive) to build and maintain it.
I have an answer that is easier to maintain, but it could get slow and memory expensive to compute with many ranges as it requires cloning a set of all ranges.
You need to save all ranges twice, in two sets. The score of each range will be its border values.
Going with the sets in #DidierSpezia example:
A 2-8
B 4-6
C 2-9
D 7-10
Your two sets will be:
ZADD ranges:low 2 "2-8" 4 "4-6" 2 "2-9" 7 "7-10"
ZADD ranges:high 8 "2-8" 6 "4-6" 9 "2-9" 10 "7-10"
To query to which ranges a value belongs, you need to trim the ranges that the lower border is higher than the queried value, and trim the ranges that the higher border is lower.
The most efficient way I can think of is cloning one of the sets, trimming one of it sides by the rules gave above, changing the scores of the ranges to reflect the other border and then trim the second side.
Here's how to find the ranges 5 belongs to:
ZUNIONSTORE tmp 1 ranges:low
ZREMRANGEBYSCORE tmp (5 +inf
ZINTERSTORE tmp 2 tmp ranges:high WEIGHTS 0 1
ZREMRANGEBYSCORE tmp -inf (5
ZRANGE tmp 0 -1
In this discussion, Dvir Volk and #antirez suggested to use a sorted set in which each entry represent a range, and has the following form:
Member = "min-max" range
Score = max value
For example:
ZADD z 10 "0-10"
ZADD z 20 "10-20"
ZADD z 100 "50-100"
And in order to check if a value falls within a range, you can use ZRANGEBYSCORE and parse the member returned.
For example, to check value 5:
ZRANGEBYSCORE z 5 +inf LIMIT 0 1
this will return the "0-10" member, and you only need to parse the string and validate if your value is in between.
To check value 25:
ZRANGEBYSCORE z 25 +inf LIMIT 0 1
will return "50-100", but the value is not between that range.

How to get DIFF on sorted set

How do I get most weighted elements from a sorted set, but excluding those found in another set(or list or hash).
>zadd all 1 one
>zadd all 2 two
>zadd all 3 three
>sadd disabled 2
>sdiff all disabled
(error) WRONGTYPE Operation against a key holding the wrong kind of value
Is my only option is to get elements from the sorted set one-by-one and compare to the list of "disabled" items? Wouldn't that be very slow because of so many transactions to a server?
What is the approach here?
Note: I assume you've meant sadd disabled two
As you've found out, SDIFF does not operate on sorted sets - that is because defining the difference between sorted sets isn't trivial.
What you could do is first create a temporary set with ZUNIONSTORE and set the intersect's scores to 0. Then do a range excluding the 0, e.g.:
127.0.0.1:6379> ZADD all 1 one 2 two 3 three
(integer) 3
127.0.0.1:6379> SADD disabled two
(integer) 1
127.0.0.1:6379> ZUNIONSTORE tmp 2 all disabled WEIGHTS 1 0 AGGREGATE MIN
(integer) 3
127.0.0.1:6379> ZREVRANGEBYSCORE tmp +inf 1 WITHSCORES
1) "three"
2) "3"
3) "one"
4) "1"

Redis sorted set combine

Redis is awesome !
If I have 2 sorted sets like - "set1 = key1:100, key2:200, key3:300"
and another like "set2 = key1:1, key2:2, key3:3"
then is it possible to get like this - set3 = key1:100_1, key2:200_2, key3:300_3
I do not want to add values of same key or take min/max of same key.
Is it possible to get these values together ?
I want to get both of these values in together for that same key.
A score in a sorted set has to be a numerical value, so it is not possible to return a simple concatenation of the scores in intersection/union operations.
However, if you can bound the value of the scores for each set to a given range, it is possible to use the SUM operation to encode two values in a single value with simple arithmetic.
Example:
# For s1, we assume the score is between 0 and 999
> zadd s1 100 key1 200 key2 300 key3
(integer) 3
# For s2, we assume the score is between 0 and 99
> zadd s2 1 key1 2 key2 3 key3
(integer) 3
# Get the intersection, dividing the s2 score by 100
> zinterstore s3 2 s1 s2 WEIGHTS 1 0.01
(integer) 3
# Get the result with scores
> zrange s3 0 -1 withscores
1) "key1"
2) "100.01000000000001"
3) "key2"
4) "200.02000000000001"
5) "key3"
6) "300.02999999999997"
In the resulting scores, the integer part contains the score coming from s1, the fractional part contains the score coming from s2.
Please note the score is a double floating point number. It has some consequences:
you may have to round it properly if you manipulate only integers (i.e. 0.0299999... is actually 0.03)
encoding two numerical values in one means the precision is divided by 2 (16 digits to 8 digits only). It may or may not suit your needs.

Redis: How to intersect a "normal" set with a sorted set?

Assume I have a set (or sorted set or list if that would be better) A of 100 to 1000 strings.
Then I have a sorted set B of many more strings, say one million.
Now C should be the intersection of A and B (of the strings of course).
I want to have every tuple (X, SCORE_OF_X_IN_B) where X is in C.
Any Idea?
I got two ideas:
Interstore
store A a sorted set with every score being 0
interstore to D
get every item of D
delete D
Simple loop in client
loop over A in my client programm
get zscore for every string
While 1. has way too much overhead on the redis side (Has to write for example. The redis page states quite a high time complexity, too http://redis.io/commands/zinterstore), 2. would have |A| database connections and won't be a good choice.
Maybe I could write a redis/lua script which will work like zscore but with an arbitrary number of strings, but I'm not sure if my hoster allows scripts...
So I just wanted to ask SO, if there is an elegant and fast solution available without scripting!
There is a simple solution to your problem: ZINTERSTORE will work with a SET and a ZSET. Try:
redis> sadd foo a
(integer) 1
redis> zadd bar 1 a
(integer) 1
redis> zadd bar 2 b
(integer) 1
redis> zinterstore baz 2 foo bar AGGREGATE MAX
(integer) 1
redis> zrange baz 0 -1 withscores
1) "a"
2) "1"
Edit: I added AGGREGATE MAX above, since redis will give each member of the (non-sorted) set foo a default score of 1, and SUM that with whatever score it has in the (sorted) set bar.